Kent Carter:
Hey everybody, I'm here again with Guy Jones and we're gonna be talking more about game music. And Guy Jones is one of our, one of ALIBIs game composers. Thank you so much for being here, Guy.
Guy Jones:
Yeah. No, thank you. I'm really looking forward to having a chat with you, and we've got a whole bunch of topics lined up over the course of these conversations. And actually, I think it would be a really good thing for you to explain who you are and what you do as well for everybody listening, just in case they don't know.
Kent Carter:
Oh, sure. I'm a composer with ALIBI as well. I'm also a vice president of Strategic Initiatives. And so basically creating music that's adaptive, which we're gonna talk about today, and that's suitable for games. That's the thing that I do and that's what you and I have been working on.
Guy Jones:
Yeah. Vice-president and so professional.
It's, it, It's so wonderful to be able to Chuck that into conversation. Isn't it? (laughs).
Kent Carter:
I feel way more important already.
Guy Jones:
Yeah. (laughing) Just the weight in the, your weight in the conversation has gone up tenfold.
Kent Carter:
Exactly (laughs).
Guy Jones:
Yeah, yeah.
Adaptive Music 101
Kent Carter:
We wanted to just talk about adaptive music for game developers. Let's just start at the very beginning and just talk about, what is adaptive music and adaptive audio for video games?
Guy Jones:
Yeah. It's a great question. And I think you set it up perfectly, as you said, what is my version of adaptive music, and how do I understand it? Because it does vary hugely. And the reason it varies massively from game developer to game developer is because there are so many types of software that they can use to get music into their video games. And it can all work in unison together, but there's very different techniques that go about it. To break it down in a very simple way, there's horizontal techniques and vertical techniques. And it's quite good to group the different ways of putting music into the game, in that sense. But it really starts from the beginning and just in case people are listening to this and they have no idea about how music and video games differs from that of say, in a film or a trailer or something like that.
Guy Jones:
It's really important to pick up on that. Video game music is a non-linear, film music is linear. So, in film, it has a start point and an end point, and you usually ramp it up price up until the end, and then it has an outro and an end. And in that sense, you're kind of writing a song in a normal form. With non-linear music that is used in video games, you have to tell a story within a minute or two minutes of a loop that is gonna keep on looping round. And I think a really good way and something that I try and keep in my mind when I'm right in a loop is, if you imagine someone sits down to play a game and they're playing around and stuff and the loop starts, but then they go away to make a cup of tea.
“ In film, music has a start point and an end point, and you usually ramp it up until the end, and then it has an outro and an end. And in that sense, you're kind of writing a song in a normal form. With non-linear music that is used in video games, you have to tell a story within a minute or two minutes of a loop that is going to keep on looping.” Guy Jones
Guy Jones:
And that loop is still going while they make a cup of tea, but they're disengaged from the game. When they come back and sit down, they have to be invested in the world the second they sit back down. So, the loop has to have a real impact on the world that is trying to sell. Not only that, it has to be ready to shift gears whenever that character or whenever that player sort of changes to a level of intensity. So, say they walk through a door and a boss is there. All of a sudden, the music has to have an impact and it has to change. And that's why it's non-linear because those moments can happen at any point, and the player decides when it happens. So the techniques you're talking about and adaptive audio is that's the basic form of it is just sort of getting those loops right and making sure that they work in that sense.
Guy Jones:
And then there are different techniques that can be used to adapt the music to the game play. That's called adaptive music. One more, from what I understand, a more modern version of this is using a vertical technique where you have different layers of a loop that all get triggered whenever something happens. And if anybody's familiar with Elias or FMOD, it allows you to do that really easily. So, you'll have what's called a base layer, and that will be maybe a drum kit. Like just sort of talking in a band sense.
And that's to say you've got your drums that are going, and they're chilled and they're chugging along, and you're drifting around the game and doing your own thing. Then all of a sudden, an enemy pops out, and you might have an electric guitar kick in. That's all triggered within the software that can be used within Unity. The way I know how to do it is Unity showing a path to FMOD, but that path can be created in any form of middleware that people want to use.
Vertical Music Techniques
Kent Carter:
Yeah. So, I wanna dig in a little bit right there. When we were talking about vertical music techniques, like how would you visually describe that to somebody else?
Guy Jones:
Vertical is exactly what it is, which is stacks of stems. Where it is basically just a base layer exactly as I said, and those stems that are underneath, it can all be triggered based on what's happening in the gameplay. That's why it truly is adaptive music.
Kent Carter:
I see. So maybe for those who don't know, you know, describe actually what a stem is.
Guy Jones:
Okay. Yeah. So, a stem would be, part of a song. So, say that, I think the simplest way to describe a stem would be, if you imagine a band like the foo fighters or something or Nirvana. Let's go in the Nirvana 'cause they're a three piece. So, the stems for a band like Nirvana would be Dave Grohl's drums. The, who's the bass player in Nirvana? I can't even remember what his name is now.
Guy Jones:
And then Kurt Cobain's guitar and Kurt Cobain's vocals. So, in that band, you would probably have about four stems. And it's, it gets a bit more complicated than that, but in essence, stems are just the track that you hear in full, broken down into multiple different files. And this is why having loopable stems is such an asset to have as a game developer, because not only do you have a full track, you also have the amount of stems that are given to you and you can have different variations within that track very easily.
Guy Jones:
And it doesn't even matter if you want to trigger those stems based on a boss a pair in or anything like that. You can just add variation to the loop over the course of 10 minutes by letting those different stems fade in and out, because they're always running together. Even when say four stems are muted of the five, they're always running in unison and they'll always loop back together, and you can just have them triggering together when they want to come in and out. And you can even fade in stems in and out as well underneath what would be a base layer.
Kent Carter:
One, one good example of this vertical technique is in a game called City Skylines. And basically you can zoom in and out. And as you zoom out, the music gets more, more sparse. And likewise, when you zoom in, the different looping stems are faded in to make it feel like you're getting closer and closer to civilization. It's a really cool effect.
Guy Jones:
Yeah, that's fantastic. I think it's part of the unity tutorial game where you can actually create, it teaches you how to create four blocks that all has different music or different stems. And if you're in the middle, it all plays. And as you creep out to one corner, like that becomes more prominent and the others fade out. As you creep down one square, it favors those two cubes on that side and so on. It's really quite clever and a good way to practice. So, I think if any game developers have Unity, there might be something in there that can show you how to really easily program this stuff. If you've got the files to handle, that is.
Kent Carter:
Very cool. Yeah. And then that's obviously been a project we've been working on on ALIBI as well.
Guy Jones:
Yeah.
Kent Carter:
Maybe you could talk about that for a little bit.
Horizontal Music Techniques
Guy Jones:
So, I was quite fortunate in that I've had the chance to work with both vertical styles of game editing and horizontal. And horizontal can be tricky, 'cause there you're basically creating chunks of audio that all have to work in unison together. So, there's almost like sharp cuts into certain aspects. So, you have like an intro that cuts into a whole loop and then an outro that will be sharply cutting to that as well. And there are techniques that you can use to fade and smooth those transitions over, but that's the basic audio editing level. It is going from one audio clip, jump into the next one and you have to make sure that they all fit perfectly. And I did a massive project where an enormous amount of music had to all work in that sense. And you had to make sure that, that outro to the loop, or that transition out of the loop could happen pretty much anywhere in the track.
Guy Jones:
And that was a real challenge. But based on that challenge, we, we developed a technique with ALIBI where we could create loops with stacked stems that not only allowed us to have those horizontal techniques in place, but also allow us to create adaptive music through vertical techniques, because those stems could all work in unison and loop, and you could fade in and out. So, we managed to get the best of both worlds through experiences with other game developers, which has been incredible. And I think if we hadn't had those experiences, we would have been really missing a trick by just making the music available to certain techniques. Whereas actually to get the loops that we've created and we've edited, have been edited so they can be used in any kind of audio editing, which I think we've done well. I think we've done really well with it.
Tempo Mapping for Smooth Game Transitions
Kent Carter:
Yeah, I would agree. And talk about how we're using tempos to really help make those transitions smooth. Maybe talk about that and also about just tempo mapping and how a game developer can use that technique to help with the horizontal technique that you're talking about right now.
Guy Jones:
Hmm. Yeah. I mean the tempo can be a really important, aspect of both techniques anyway, both vertical and horizontal. It's always good to have a gauge of what the tempo is doing, because it will help you work out how music is clipping to video and how it's going to transition and stuff. So the tempo in horizontal techniques is absolutely crucial. It's, it's, and it's why any music that I write for any games, they do need to be horizontal and they're going to be edited in a horizontal way. The BPM is always at the top of every file, and there's always a way to integrate into the software that you're using to create the game. Sometimes it can be easier to have something like Elias or FMOD or Wwise, because it's a very big feature because BPM goes a long way to making sure that music runs really smoothly.
Guy Jones:
Particularly if it's music that has a lot of sharp transitions. So, a transition is when you get the initial hit of a snare drum or the very second a guitar is strung. Just like those initial clips, which are the very start of the bar of, of the beat. And the, when that first click of the BPM comes in, it's really crucial to making sure that that's really clean. And to the editor's dismay we actually go through every single bar and we make sure everything is smooth no matter what bar the tracks come out of. And the only way to do that is by matching it with BPM and tempo. And I think some people call it compass markers as well, or is that maybe the time signature, but we do include that as well.
“ When that first click of the BPM comes in, it's crucial to make sure that it's really clean. And to the editor's dismay, we actually go through every single bar and we make sure everything is smooth no matter what bar the tracks come out of. And the only way to do that is by matching it with BPM and tempo.” Guy Jones
Kent Carter:
Yeah, very cool. I mean, from an audio perspective, let's say for example, you're playing a game and then say, you're doing a racing game and you cross the finish line, and then boom, the music transitions to an ending.
Guy Jones:
Yeah.
Kent Carter:
And it's in time with the music track as well.
Guy Jones:
Yeah.
Kent Carter:
And that happens right on a downbeat, so it actually feels very musical like the composer inside the the box was right in the music right along there with you, with her gameplay.
Guy Jones:
Absolutely. And that, and that's the beauty, and that, that's why our editors have done so well, and why I get help doing it when I'm writing for any video game. Because those transitions and those outros are so key to hit in those moments. Like, when you go over the finish line, it's no good if that sort of boom outro, happens two seconds after you've gone over the finish line. Because that sense of relief from the game player has already happened, and then you're getting smacked in the face with a kind of celebration boom or something going off.
Guy Jones:
But if you cross that finish line, you just feel that boom, and you feel something change or happened with the music. That, that's music working well with video games. That's music being integrated into the game software as well. And it's very easy to spot when it's not. Very easy. And any average person who knows nothing about music, maybe wouldn't pick up on the fact that the music isn't quite sinking up, but it would definitely have an effect on their experience playing the game, without a doubt. And they wouldn't know what was affecting their experience, but it would be the music and the side effects.
Leveraging Stingers & Drones for Video Game Sound Design
Kent Carter:
Cool. Yeah. And just, just having dynamic music and games like that is really great. I wanna talk about one other technique and I know you've done this recently with a game that you've been working on, and that's using drones and stingers.
Guy Jones:
Yes. Now, this is, this is a tricky one. This is quite a tricky one. So we used a form of vertical technique with this and the reason we've done this is because it's an exploration game. It's actually called a Metroidvania game. So if anybody's played a Metroid, you have this character and you're working your way around a map and slowly unlocking it. It's a weird middle ground between an open world game and a side scrolling game like Sonic. You're not going from start to finish, you're exploring the map and slowly unlocking it and it becomes this much bigger thing as the game goes on. So, we had these, these sections of the game where people are probably exploring the same area and coming back to that area. I mean, they could be in there for an hour.
So, I decided the best thing that we could do, and there are lots of very intense bosses throughout the game as well and it's a very quick shift. So, the drone and how we did the drone is I made the drones about six or eight minutes long, and they've, they're loopable. So, they'll, they'll keep on looping, but because there are so many textures moving in and out of it and everything, we, we decided that was the best way. And we didn't want to do vertical too many like adaptive layers in it, because we'd soon get a bit confused with where we were going with it. And especially because we wanted to trigger boss tracks as well, the easiest way we could do it is have the drone as a base layer throughout the whole area.
And then when the boss kicks in, the boss track is actually a series of stems that goes right on top of the drone and doesn't end, the drone becomes part of the soundtrack of the boss. But you don't really realize it is, because it's just the boss kicking in. And what that means is, you go into the area, the boss sees you, the new music kicks in on top of the drone that becomes that subtle layer of the track. And then when you finish and you kill the boss, the boss music pulls out and there's that gradual drop into the drone. So, it's like that and it works really well. And we're really pleased with how that method has worked for this particular game. So it's, it is a form of vertical layer in and adaptive in that sense, but it's just a little bit dragged out.
Masking Transitions with Sound Design
Kent Carter:
Cool. So, it's almost acting like you think of your traditional stinger when something happens, but it's actually part of the vertical technique that you've been talking about.
Guy Jones:
Exactly. Yeah, exactly. And the, the stingers are a really good thing to pick up on, because when you have soundtracks like this, and I suppose at any game you can use sound effects to mask what's happening within the soundtrack. And there's a really good article actually from, a composer who wrote the Ori and the Black Forest soundtrack. And it's a beautiful soundtrack. And he talked about the struggle of, a bit of music that was adaptive and they were talking about the adaptive music and layering that you spoke about with the other game, where there's a scene in the game or a level in the game where you start at the bottom of this tree, and water starts rising and you have to jump up and up and up.
Guy Jones:
And he goes on for quite a while. Like it's quite a big section of the game. And as it's rising, the main loop is going, but they're introducing new elements to that truck. All the way up to the top. So, they had to work out how they could bring in the section when he's right at the top, when it's just about to open up. They wanted something new at that point when you're just about to see the sky, and then you pop out, and then there's this gigantic, well I won't spoil it in case anyone's gonna play actually, 'cause you're not supposed to know (laughs). But this big thing that comes up then, and all of it is done by using sound design that masks the transitions.
Guy Jones:
And they, they do it fantastically and it's incredibly cinematic and incredibly adaptive. And that'swhat we've adopted with hiker as well. And any big moments, any cut scenes that are moving between the loop and a cut scene, we're using sound design to mask over, and stingers to mask over those little messy bits that might sound a bit odd without it. But at the trigger of say a boss dying is triggering both the stinger and the drop of the boss music. So, it works perfectly in sync with each other.
Kent Carter:
That's very cool.
Guy Jones:
Yeah.
Kent Carter:
Well, Hey, Guy, thank you so much for taking the time to answer these questions and we'll be meeting again soon.
Guy Jones:
Yeah. No, thank you very much. I enjoyed it. Cheers, Kent.
Kent Carter:
Cheers.
