The original idea for the game was to make it Night Vale-themed, so I started the music with a Disparition vibe in mind. The game didn’t turn out that way in the end, but that’s okay, since the music didn’t either! It’s suitably dark and has a driving beat to it, so I think fits the game pretty well.
My move to San Francisco is just a few weeks away, so I’ve sold most of my studio gear, including my audio interface and keyboard. That left me using my on-board sound card to run JACK and Ardour, but that turned out just fine — with no hardware synths to record from, not having a proper audio interface didn’t slow me down.
As some of you guessed, the toy in the mystery box in my last post was indeed a Teenage Engineering OP-1. It filled in as my MIDI controller here, and while it’s no substitute for a full-sized, velocity-sensitive keyboard, it did a surprisingly good job.
Software-wise, I used Rui’s samplv1 for the kick and snare drums, which worked brilliantly. I created separate tracks for the kick and snare, and added samplv1 to each, loading appropriate samples and then tweaking samplv1’s filters and envelopes to get the sound I was after. In the past I’ve used Hydrogen and created custom drum kits when I needed to make these sorts of tweaks, but having the same features (and more!) in a plugin within Ardour is definitely more convenient.
The other plugins probably aren’t surprising — Pianoteq for the pianos, Loomer Aspect for everything else — and of course, it was sequenced in Ardour 3. Ardour was a bit crashy for me in this session; I don’t know if it was because of my hasty JACK setup, or some issues in Ardour’s current Git code, but I’ll see if I can narrow it down.
This weekend was Ludum Dare 26, and as usual when Switchbreak enters such things, I took the opportunity to tag along. The theme was “minimalism”, but his game, called MinimaLand, deliberately eschews that theme; it tasks the player with bringing life and detail in to a very minimalist world.
In MinimaLand, the player brings life to an abstract, minimalist world
I wasn’t sure at first how to fit music to the game, but it soon became clear: if I was going anti-minimalist, I wanted to use an orchestra. Ever since I heard about the Sonatina Symphonic Orchestra, a CC-licenced orchestral sample set, I’ve wanted to try recording something with it; what better time to try it than with a deadline looming!
Given that I had just a few hours for this, I kept the music itself very simple — just three chords and a short melody. The music itself is almost irrelevant in this case, though, since it’s really just a means of delivering those orchestral sounds to the player. Initially, the melody and harmony are on strings, with rhythmic staccato stabs on brass, then the whole thing repeats, with the stabs moving to strings and the melody/harmony to woodwinds and horns.
It’s funny that, even when I’m dealing with sampled instruments instead of my own synth sounds, I still think in terms of sound and feel first, and chords and melodies second. I guess that’s just how I roll!
Working with LinuxSampler
That’s a lot of LinuxSampler channels!
Not unexpectedly, I sequenced everything in Ardour 3 and hosted the SSO instruments, which are in SFZ format, in LinuxSampler, using a LinuxSampler build from a recent SVN checkout. I didn’t use anything else on this one, not even any plugins, since all it really needed was some reverb and the SSO samples already have plenty of it.
Recent versions of LinuxSampler’s LV2 plugin expose 32 channels of audio output; I guess the idea behind this is to allow you to run multiple instruments to dedicated outputs from within a single plugin instance, but I’m not sure why anyone would actually want to do that. I think my workflow, with each instrument in its own plugin instance on its own track, makes a lot more sense, so I patched the plugin to return it to a simple stereo output per instance.
I’ve been keen to try SSO mostly to see just how usable it is, and in this case, I think it worked pretty well. With just 500MB of samples, it’s never going to sound as good as a commercial library (where individual instruments can take several GB), but some of the samples, such as the string ensembles, sound quite nice at first listen.
The biggest problem is with the availability of different articulations for each instrument. You do get staccato samples for most instruments, and pizzicato for the strings, but beyond that you just get “sustain” sounds, which are great for held notes (as long as the sample’s long enough), but far less suitable for faster legato parts. You can hear this in the horn part in the second half of the track, where some short notes take so long to start playing that they barely become audible before they end.
Many of the solo instruments are quite weak, too — you can hear audible jumps between certain notes in several of them, where the instrument jumps from one discrete sample to the next, while others have odd tuning problems.
There’s also a tonne of reverb on every instrument. SSO’s instrument samples come from a variety of sources, so each instrument has its own reverb characteristics; in an attempt to even out the differences and make them all sound at least vaguely like they’re in the same room, the library’s creator added quite a bit of extra reverb to each instrument. It’s a necessary evil, and it works, but it has a smearing effect that only exacerbates those problems with short notes.
So, SSO was well suited to this track — most notes were either staccato/pizzicato or were held for quite some time, I didn’t need to use any solo instruments, and the wall of reverb helped make it sound suitably pompous. If your needs are different, though, then you’ll have a hard time getting good results from it.
Having said that, it is far-and-away the best free option, and it’s also quite easy to get working under Linux, which can’t be said for many commercial libraries. Despite my mostly good experience with it, though, I’m keen to investigate just what commercial alternatives are available that will work under Linux.
I’ve spent the last week working on a game for the Seven Day Roguelike Challenge with Switchbreak and Amanda Lange, and by virtue of the fact that it’s a seven-day project, it’s now finished! It’s called Cosplay Mystery Dungeon, and you can play it here (if you have Flash installed).
A week isn’t long, but I’m really impressed with the finished game — Amanda did great work on the art and the game design, and Switchbreak put a tonne of work in to the code. Here’s how my part of it all came together.
Getting in with the right crowd
It all started innocently enough, with a tweet:
@second_truth Hey, if you don’t find any tools to your liking I’d be willing to code your design – is 7DRL cool with collaboration?
Once Switchbreak was involved, it wasn’t long before I jumped on board, too. Amanda came up with the concept and had written up a lot of notes about the design, so I had a good idea of what sounds would be needed, and what they should sound like, from the get-go.
Early in the week, I worked on the basic player and weapon sounds, making a few versions of most of the weapon sounds to avoid any annoying reptition. The magic effects came later; Amanda’s design included various spells, in the form of collectable comic books, but with the deadline looming I wasn’t sure which of those would make it in to the game. As it turned out, Switchbreak managed to implement them all in the closing hours, so my last hour-or-so was a race to create the matching sounds.
The sounds were a mix of synthesis (using both my Blofeld and Loomer Aspect) and recorded sounds. Some used both synthesis and recording, in fact, such as the lightsaber — after reading about how the sound was created originally, I created a suitable humming sound, played it through one of my monitors, and then swung my mic back and forth past the speaker, recording the results.
Music in a hurry
I hadn’t planned to write music for the game, but it felt so odd without it that, with less than a day left, I decided to take a stab at writing something. I’ve written short pieces within hours for past Switchbreak games, but they’ve been much smaller than this. A run-through of Cosplay Mystery Dungeon can take an hour or more, not to mention the time spent on unsuccessful attempts, so the music needed enough length and variety to carry it over a longer playtime.
I started with the bass line and fast-paced drums, and I knew from the start that I wanted to add a later section with slower, glitchy drums, so those went in early, too. Soon after I nailed down the chord progressions and structure, and started filling in the melody, pad, and stabby brass lines.
This is what an all-MIDI, all-softsynth Ardour session looks like
As with my RPM Challenge work, I worked quickly by sticking with MIDI (no bouncing to audio), using softsynths instead of hardware, and mixing on-the-fly, with minimal (or none, in this case) EQ and compression. Synths do let you cheat a bit — if you find that a part is too bright or too dull, or needs more or less sustain, you can just edit the synth patch (tweaking filters and envelopes, for example) instead of using EQ and compression to fix those things after the fact. You can’t solve every mix issue that way, but I find that I can get a perfectly decent, listenable mix that way more quickly than I could otherwise.
All up, I think the music took about 5-6 hours to record, with another half-hour or so after that creating musical cues for the endgame screen and the game over screen, using the same instruments. That left me with just enough time to finish the magic sound effects before the deadline.
Loomer Aspect and Sequent earned their keep, alongside open-source plugins like Invada Tube and the TAL and Calf delays
Loomer Aspect really paid for itself on this one. I used TAL NoiseMaker on the chorus lead sound (it’s a modified preset), and the Salamander Drumkit with the LinuxSampler plugin for the drums, but every other sound came from Aspect, mostly using patches that I created on-the-fly. For such a capable synth, it’s surprisingly easy to program — everything’s laid out in front of you, and it’s fairly easy to follow. It lacks the Blofeld’s distortion options, but using distortion plugins in Ardour (TAP TubeWarmth and Invada Tube Distortion) helped address that.
I also had an excuse to use Loomer Sequent — it provided the glitch effects on the drums in the slower section. The presets were all a bit too random to be usable on such sparse parts, so I edited the effects sequence in Sequent to match the parts, adding just a bit of randomness to its loop-slicing parameters.
This was the first track I’d recorded since the official release of Ardour 3, too. It worked really well — it was stable, reliable, and predictable throughout, a definite improvement on the betas. If you haven’t tried it yet, now’s definitely the time!
Over the last week I’ve put a bunch of time in to my new game project, a Switchbreak game called Civilian. I’ve been working on music for it, but this blog post isn’t about music — it’s about the crazy stunts you can pull in modern interpreted languages.
Dynamic music in Flash?
Most Flash games use a looping MP3 for background music — it takes just a couple of lines of code to implement, and while the looping isn’t perfectly seamless (there are brief pauses at the start and end, added by the MP3 encoder) it’s usually close enough. For Civilian, though, I wasn’t satisfied with a simple looped track. It’s a game about the passage of time and about player progression, and I wanted the music to reflect those things.
What I really wanted was a dynamic music system, something that would let me alter the music’s sequence or instrumentation on-the-fly in response to the player’s actions. There was no way that was going to work with simple, non-seamless looping MP3s, though — I needed to start working with audio data on a much lower level.
Writing a low-level mixer in AS3
Thankfully, the Flash 10 APIs do give you low-level audio functionality. You can get raw audio data out of an MP3 file, and then send that to audio buffers for playback; I’d already done just that in fact, to implement a seamless MP3 looper, and that gave me a crazy idea: if I could get audio data from one MP3 and play it back, could I also get data from two or more MP3s, mix them, and play them back all at once?
Once I’d confirmed with a simple proof-of-concept that the answer was an emphatic “yes”, I set about adding more tracks, and then implementing features like panning and volume conrol. By this point, the amount of CPU power required to run this mixing was significant — about 40% of one core on my 1.7Ghz i5 Macbook Air — but Flash had no trouble keeping up while running some simple gameplay at 60FPS.
A screenshot from my test app, with five channels of audio running
From mixer to sequencer
A few days later I had more than just a mixer: I had a simple pattern-based sequencer. Instead of looping MP3s from start to finish, it splits the MP3 tracks in to bars, and then plays those bars in accordance with a sequence stored in an array in the AS3 code.
This actually fits quite well with how I tend to write my music. I can arrange the track basically how I want it in Ardour, then record each unique section of each track to audio, and string those sections together to produce a single MP3 track for each instrument. Then, I can create a sequence within the AS3 code that reassembles those sections in to my original arrangement.
Each bar can have its own settings, too, somewhat like the effects on each note in a tracker. So far, these just let me set the panning or volume for each track, or set up a volume slew (ie: a fade in or fade out) to run over the course of the bar.
Making the music dynamic was just a matter of replacing the static sequence array with code that generates the sequence on-the-fly. I have pattern templates for each track which I combine to create the sequence one bar a a time, adding or removing tracks or replacing one part with another (perhaps with a nice fade in/fade out) based on what’s happening within the game world.
Pushing interpreted languages
As if all the above wasn’t enough, I decided to add an optional audio filter on the output. For certain scenes in the game I want to be able to make the music sound like it’s coming from a radio, so I added a simple bandpass filter, based on a Biquad filter implementation from Dr. Dobbs. If the filter is having any impact on my sequencer’s CPU usage, it’s far too small to notice.
Eventually, I gave up trying to think of efficient ways of doing things, and just started doing them in the simplest way possible. I’ve since done some optimisation work, to help retain a steady frame rate on slower systems (using my old Latitude E6400, clocked down to 800Mhz, as my test machine), but those optimisations are totally unnecessary on more typical systems.
Ten years ago, I wrote audio mixing code for the GBA, and it looked something like this
The last time I wrote audio mixing code, it was for the ARM7 CPU inside the Gameboy Advance. On that system, compiled C code wasn’t fast enough, so I had to re-write the critical loops in hand-optimised ARM assembler code to get the necessary performance. To see an interpreted language do the same things so easily is still somewhat mind-boggling, but it’s a testament to the advances made in modern interpreters, and to just how fast modern PCs are.
Things have been quiet here of late, but I’ve actually been quite busy! I’ve just finished the sound design for Candy Grapple, the latest game from my good friend Switchbreak. It’s based on one of his Ludum Dare games, Waterfall Rescue, but it’s been fleshed out in to a full game, with much more complete gameplay, many more levels, and a spooky Halloween theme. It’s out now for Android, and there’s an iOS version on the way, too.
Switchbreak asked me to make some suitably spooky-cheesy music for it, and I happily agreed; once I started working on that, I realised he’d also need sound effects, so I offered to create those, too. Read on for details!
The bulk of my time went in to the in-game background music. Halloween music was new territory for me, but my mind went straight to The Simpsons Halloween specials, and the harpsichord and theremin closing credits. I thought about other “spooky” instruments and came up with the organ, and while it’s not spooky as such, the tuba seemed suitably ridiculous for the kooky carnival sound I was after.
I didn’t want to over-use the theremin, so I stuck with organ for the melody for the most part, and saved the theremin for the bridge, where the harpsichord and tuba drop away in favour of some organ triplets and piano bass notes.
A standard drum kit didn’t seem like a good fit (with that bouncy tuba part, it was in danger of becoming a polka), so I stuck with more random, wacky bits of percussion, like castanets and a vibraslap. I did use some cymbal rolls and crashes in the bridge, though.
Now, for the instruments: I used Pianoteq for the harpsichord and piano, as you’d probably expect; the percussion sounds were from the Sonatina Symphonic Orchestra, played using the LinuxSampler plugin; and the theremin was a simple patch on the Blofeld.
Pianoteq doesn’t just simulate pianos — it also handles other melodic percussion, like harpsichords
The tuba and organ, surprisingly, come from the Fluid GM soundfont. I’m not usually a fan of instruments from GM sets, and I did try a few alternatives, but the Fluid sounds were very well-behaved and sat well in the mix, so I didn’t let myself get hung up on where they came from.
Faking the theremin was fairly straightforward — it’s just a single sine-wave oscillator, but with some portamento to slur the pitch changes and an LFO routed to the oscillator pitch to add vibrato, both of which make that sine wave sound suitably theremin-ish.
I used TAL NoiseMaker at first, but switched to the Blofeld so I could use the modwheel to alter the amount of vibrato (the Blofeld’s modulation matrix makes this sort of thing easy); in hindsight, it would’ve been just as easy to stick with NoiseMaker and alter the vibrato by automating the LFO depth.
The mix came together fairly quickly. There’s a bunch of reverb (I had trouble getting the IR plugin working, so I used TAP Reverberator instead), a little EQ on the tuba and organ to brighten them a bit, and some compression on the piano to add sustain, but that’s about it as far as effects go. The only tricky part was making sure the transition in to the bridge wasn’t too abrupt, but all that really required was some careful balancing of levels.
It was, of course, all recorded and mixed in Ardour 3 — it has an annoying MIDI monitoring bug right now, but I’m hoping that’ll be fixed soon.
I wanted to add some music to the title screen, too, so I come up with a little organ fanfare-ish thing and recorded it in to Ardour. The organ is the setBfree plugin, a Hammond B3 emulation based on an old app called Beatrix.
Beatrix had taken on near-legendary status in Linux audio circles, partly due to its great sound, and partly due to being near-impossible to run. It lacked JACK support and had various other issues, and its strict licencing forbid forking it or distributing patched versions.
Somehow, though, the setBfree devs managed to negotiate a suitable licence, and have added JACK support, LV2 plugin support, and a basic GUI. The GUI is a separate app that talks to the synth engine (whether it’s the JACK app or the LV2 plugin) via MIDI; it lets you configure the organ stops manually, or load presets.
setBfree’s GUI is a stand-alone app that talks to the synth via MIDI
The thunder sound was my own recording — I have a habit of setting up my Zoom H1 and letting it record during thunderstorms, and that’s finally come in handy!
Sound effects are hard; I’ve had a little experience with this, working on another game for Switchbreak which is still in development, but it’s all still fairly new to me. I used synths for some — Pianoteq came in handy once again here, for its tubular and church bells — but the rest were recorded sounds, mostly of me using things to hit other things. For the flapping bat wings, for instance, I slapped rubber gloves together, and idea I saw on this list of sound effects techniques.
I’m pretty happy with the fact that there are two vocal samples in there, too — the ghost and the witch are both me. The witch’s cackle just took some pitch shifting and a bunch of reverb.
Video editing in progress, using Kdenlive
As the game neared completion we realised it’d need a trailer, so I volunteered to make one, using Kdenlive. I used ffmpeg to record video from the Flash version of the game, then brought that in to Kdenlive, where I composited it on top of the phone image and background. It was a fairly straightforward edit, but I had some fun with it — I hadn’t played with wipes before now, for instance, so I took the opportunity to ham it up and throw some in.
My good friend Switchbreak produced another game recently — Waterfall Rescue, a Flash game with a great single-button control mechanic that’s a lot of fun to play. He wrote the game in 48 hours for Ludum Dare 20, which is a solo competition, but he spent some time after that working on the graphics, so I whipped up a quick theme tune for it, too.
You really should check out the game, but if you just want to check out my music, here it is: I’m feeling lazy, so I’m including this as an embedded player from my Bandcamp page instead of uploading it again and using my usual HTML5 player.
Here’s another quick piece done quickly for a purpose: my friend Switchbreak spent the weekend developing a short Flash game for the So Many Rooms game jam, where each developer had 36 hours to produce a game that challenges the player to get from a starting door to an ending door, using whatever obstacles or gameplay mechanics they like. Switchbreak’s game is full of bouncing balls, so when he asked me to produce a quick tune for him, I made sure that it was appropriately bouncy.
This was whipped up on Sunday night mostly in Qtractor, with Hydrogen for the drums, and my Blofeld for all the other sounds. I’d normally record everything in to Ardour and mix it there, but I stayed in Qtractor for this one, and it did a fine job; I had no trouble replicating my usual trick of running the drums on to separate tracks so that I can apply individual effects to each, for instance. The result is a bit trite, but it’s fun, it loops pretty smoothly, and I think it suits the game well.