livin’ in america!

Posted on by
Reply

Today I finished a pack of 100 coffee filters; that wouldn’t normally be much of a milestone, but this is the first pack of filters I’ve finished since moving to the US. After three months in San Francisco we’re feeling quite settled now, but through the stresses of moving, sorting out paperwork, and learning to live in a new city and country, making my morning coffee has been a reassuringly routine part of each day.

Australia and the US are similar in a lot of ways, of course, but we’ve still had to do a surprising amount of adjustment — much more than I expected. Of those adjustments, though, I think apartment living has probably been the easiest. We worried a little about moving in to a high-rise apartment building, especially given that we had to sign a lease before seeing our apartment, but it’s turned out very well; the building is great, and our apartment is quiet and comfortable, and seems to have plenty of room despite being smaller than our old house. Our two cats moved with us, and they seem to love it too!

Cozmo looks out the window at her new city

Cozmo looks out the window at her new city

Living without a car has also been surprisingly easy. Between Amazon and Google Shopping Express there’s not a lot we can’t get delivered, and San Francisco’s public transport has been pretty good, even if most of it is by bus. We’re also right near where the cable cars meet, and while the Powell-Hyde and Powell-Mason lines are usually too busy to catch, we can (and do!) often catch the California cable car, doesn’t get old. For the times we’ve needed a car around town, we’ve used UberX, which is super-convenient and (generally) cheaper than a taxi.

On the other hand, shopping has at times been a baffling ordeal; the common refrain is our house is “everything is slightly different and I don’t like it!”. It seems trivial, but having to find a new favourite brand of everything all at once, it really is a hassle. Even basic produce is a problem — good lamb has been hard to find, and this weekend we struggled to find a pork roast that still had its skin on (curse you, health nuts!). We’ve also had zero luck finding good Greek food, and little more success finding a good bahn mi like we’d have back in Melbourne.

Guatamalan hot dog with avocado and grilled pineapple? Don't mind if I do!

Guatamalan hot dog with avocado and grilled pineapple? Don’t mind if I do!

It’s not all bad, though! Good Mexican food (and food from countless other Latin American countries, too) is cheap and plentiful, and it’s impossible to walk more than 5 metres without tripping over great bread, ice cream, or craft beers. Whole Foods is a great place to shop even if it does indulge in a lot of psuedoscientific crap, and Trader Joe’s is strangely awesome (especially the cookie butter!). This really is a great town to eat and drink in, both at home and out-and-about — we’ve just had to accept that food is different here, and learn how to make the most of that while we’re here.

Now that we’re settling in, I have time to get back to my hobbies (beyond exploring our new city, of course!); I haven’t done anything musical, but I’m starting Japanese classes again shortly. I’ve also built a new HTPC with a decent CPU and video card so it can do double-duty as a gaming system, running Steam in Big Picture mode and playing Wii games under Dolphin.

Well, that’s one box of coffee filters down, but I’m sure there’ll be many more to come!

ludum dare 29: underground city defender

Posted on by
Reply

This weekend was Ludum Dare again, and again Switchbreak asked me to write some music for his entry. It’s called Underground City Defender, and it’s a HTML5/Javascript game, so you can play it in your browser here!

The original idea for the game was to make it Night Vale-themed, so I started the music with a Disparition vibe in mind. The game didn’t turn out that way in the end, but that’s okay, since the music didn’t either! It’s suitably dark and has a driving beat to it, so I think fits the game pretty well.

My move to San Francisco is just a few weeks away, so I’ve sold most of my studio gear, including my audio interface and keyboard. That left me using my on-board sound card to run JACK and Ardour, but that turned out just fine — with no hardware synths to record from, not having a proper audio interface didn’t slow me down.

As some of you guessed, the toy in the mystery box in my last post was indeed a Teenage Engineering OP-1. It filled in as my MIDI controller here, and while it’s no substitute for a full-sized, velocity-sensitive keyboard, it did a surprisingly good job.

Software-wise, I used Rui’s samplv1 for the kick and snare drums, which worked brilliantly. I created separate tracks for the kick and snare, and added samplv1 to each, loading appropriate samples and then tweaking samplv1’s filters and envelopes to get the sound I was after. In the past I’ve used Hydrogen and created custom drum kits when I needed to make these sorts of tweaks, but having the same features (and more!) in a plugin within Ardour is definitely more convenient.

The other plugins probably aren’t surprising — Pianoteq for the pianos, Loomer Aspect for everything else — and of course, it was sequenced in Ardour 3. Ardour was a bit crashy for me in this session; I don’t know if it was because of my hasty JACK setup, or some issues in Ardour’s current Git code, but I’ll see if I can narrow it down.

studio slimdown

Posted on by
5

Last weekend, almost exactly five years after I bought my Blofeld synth, I sold it. With plans to move to the US well underway, I’ve been thinking about the things I use often enough to warrant dragging them along with me, and the Blofeld just didn’t make the cut. At first, the Blofeld was the heart of my studio — in fact, if I hadn’t bought the Blofeld, I may well have given up on trying to make music under Linux — but lately, it’s spent a lot more time powered off than powered up.

Why? Well, the music I’m interested in making has changed somewhat — it’s become more sample driven and less about purely synthetic sounds — but the biggest reason is that the tools available on Linux have improved immensely in the last five years.

Bye bye Blofeld -- I guess I'll have to change my Bandcamp bio photo now

Bye bye Blofeld — I guess I’ll have to change my Bandcamp bio photo now

Back in 2009, sequencers like Qtractor and Rosegarden had no plugin automation support, and even if they had, there were few synths available as plugins that were worth using. Standalone JACK synths were more widespread, and those could at least be automated (in a fashion) via MIDI CCs, but they were often complicated and had limited CC support. With the Blofeld, I could create high-quality sounds using an intuitive interface, and then control every aspect of those sounds via MIDI.

Today, we have full plugin automation in both Ardour 3 and Qtractor, and we also have many more plugin synths to play with. LV2 has come in to its own for native Linux developers, and native VST support has become more widespread, paving the way for ports of open-source and commercial Windows VSTs. My 2012 RPM Challenge album, far side of the mün has the TAL NoiseMaker VST all over it; if you’re recording today, you also have Sorcer, Fabla, Rogue, the greatly-improved amsynth, Rui’s synthv1/samplv1/drumkv1 triumvirate, and more, alongside commercial plugins like Discovery, Aspect, and the not-quite-so-synthy-but-still-great Pianoteq.

I bought the Blofeld specifically to use it with a DAW, but I think that became its undoing. Hardware synths are great when you can fire them up and start making sounds straight away, but the Blofeld is a desktop module, so before I could play anything I had to open a DAW (or QJackCtl, at the very least) and do some MIDI and audio routing. In the end, it was easier to use a plugin synth than to set up the Blofeld.

Mystery box of mystery!

You can probably guess what’s in the box, but if not, all will be revealed soon

So, what else might not make the cut? I only use my CS2X as a keyboard, so I’ll sell that and buy a new controller keyboard after moving, and now that VST plugins are widely supported, I can replace my Behringer VM-1 analog delay with a copy of Loomer Resound. I might also downsize my audio interface — I don’t need all the inputs on my Saffire PRO40, and now that Linux supports a bunch of USB 2.0 audio devices, there are several smaller options that’ll work without needing Firewire.

I’m not getting rid of all of my hardware, though; I’ll definitely keep my KORG nanoKONTROL, which is still a great, small MIDI controller. In fact, I also have two new toys that I’ll be writing about very soon. Both are about as different from one another as you could get, but they do share one thing — they’re both standalone devices that let you make music without going anywhere near a computer.

console retirement plan

Posted on by
Reply

With new consoles on the market, and plans to move to San Francisco later this year, I figured it was time to cut down on some of the console clutter I’ve collected over the years. For now, I still have our 360 and PS3, but I’ve traded in a bunch of 360 games, given the Wii to my parents, and sold our long-disused PS2 and all our old PS2 and GameCube games. I’ve also sold off a bunch of DS and GBA games, and my DS Lite and GBA SP consoles themselves won’t be far behind.

Thankfully, if I ever get nostalgic, those systems I’ve retired — the PS2, GameCube, Wii, DS, and GBA — can all be emulated. In my last post I talked about using Dolphin to emulate the GameCube and Wii; for the PS2, there’s PCSX2, which like Dolphin can render games at high resolutions to improve quality; for the DS, there’s DeSmuME (and an excellent Android port of it, called DraStic); and for the GBA, there’s VBA-M.

Animal Crossing Wild World running (in Japanese, coincidentally) under DraStic, an excellent DS emulator for Android

Animal Crossing Wild World running (in Japanese, coincidentally) under DraStic, an excellent DS emulator for Android


For the Wii, GameCube, and PS2, I was even able to back up my save games, using GCMM and Savegame Manager GX on the Wii for the GameCube and Wii saves, respectively, and uLaunchElf on the PS2, to dump the saves to a USB stick. In both cases, I had to pull some tricks to run third-party code — on the Wii, installing the Homebrew Channel is fairly straightforward, but on the PS2 I had to use my Swap Magic discs.

To next-gen, or to PC?

New console launches are usually a cause for excitement, but their release just reminds me of what we’re losing with the end of the previous generation. Consoles have come and gone in the past, of course, but the move to digital distribution has made separating the consoles from their content much more difficult this time around. If I sold my 360 or PS3 today, I couldn’t resell all their digital content; I still own that content, and if I bought a new 360 in a few years I could reinstall it, but what about 5 years from now, or 10?

Even if it’s not perfect, the PC does offer a better alternative. Some old games are unplayable today, but if a game has a dedicated community, chances are that someone’s worked out how to run it on modern PCs. Steam’s DRM has so far proven mostly benign, and services like GOG and the Humble Store offer DRM-free downloads, too.

With dozens of great games on Steam, using Linux for PC gaming has never been more viable

With dozens of great games on Steam, using Linux for PC gaming has never been more viable

In fact, I seem to have made the jump to PC already, almost by accident. I’ve played a couple of big-name games this year, like Bioshock Infinite and Tomb Raider, but I’ve spent much more time playing smaller, more interesting games, like Kerbal Space Program, Gone Home, The Stanley Parable, Brothers: A Tale of Two Sons, Kentucky Route Zero, Papers, Please, and most recently, the Double Fine Adventure game, Broken Age, almost all of which are PC exclusives.

PC power

PC gaming has a lot of baggage — the image of the hardcore gamer that spends as much time upgrading and tweaking their Windows PC as they spend actually playing games on it — but these smaller games often defy that image. Most run on Linux and Mac OS X, and most also run on fairly modest PCs; in fact, I’ve spent more time gaming on my now 2011 Macbook Air in the last year than I have on any other system.

I’m sure I’ll want to play another big-budget graphical powerhouse eventually, and I’m not yet sure what I’ll do about that. By that time, a gaming PC with the power of the next-gen consoles might only cost as much as those consoles cost now. I like the idea of gaming laptops, but they’re expensive and clunky; only the Razer Blade delivers suitable power in a sleek, elegant form-factor, but at US$2000 it’s even more expensive than other laptops.

i’ve been playing: alan wake, super mario sunshine (and dolphin)

Posted on by
1

The draw of the familiar can be a funny thing; I don’t often replay games, but sometimes, after enough time has passed, all it can take is a brief mention somewhere to bring back a wave of fond memories, and the desire to relive them. Over the last few weeks, I’ve been doing exactly that, replaying two games that, beyond my nostalgia for them, couldn’t be more dissimilar: Alan Wake and Super Mario Sunshine.

With Alan Wake, all it took to get me playing again was a series of bargain sales of the PC version. My first time through was on the 360, with a borrowed copy, but it was always a game I wanted to revisit, both to play through the two DLC episodes, and to play it after having read Rob Zacny’s fascinating analysis of the game’s events.

I totally get people that don’t like it; the gameplay, while neat in parts, gets a bit repetitive, and the story walks a fine line between cleverness and being entirely up its own ass (a games writer writing a game about a writer who writes about writing a book?). That story really works for me, though — it starts with a bang, and the episodic pacing keeps the story moving brilliantly, with little goals and revelations that gradually reveal the key details about the game’s setting and events.

Super Mario Sunshine was always a strange Mario game, but that made it no less fun

Super Mario Sunshine was always a strange Mario game, but that made it no less fun

In comparison, the story in Super Mario Sunshiney couldn’t be more tedious, but once you get past the dreadful unskippable cut-scenes in its first few hours, it’s a joy to play. The tropical-themed Sunshine wasn’t what people expected after the groundbreaking Super Mario 64, and after being followed on the Wii by the incredibly inventive Super Mario Galaxy, I’d just about forgotten the great fun I had with it until reading Eurogamer’s recent Sunshine retrospective.

That article made me realise what it was that made Sunshine so different — that it takes place in a world that’s not designed precisely around Mario’s capabilities. Instead, it challenges the player to use their skills and Mario’s water-powered hover-pack to explore a more organic world. That’s never more apparent than in Delfino Plaza, the game’s hub world, which you can easily spend hours exploring in search of those elusive Shines.

Swimming with Dolphins

Oddly enough, I’ve been playing through Sunshine on PC, too, using Dolphin, an open-source, cross-platform GameCube and Wii emulator that improves upon the real hardware by rendering games at high resolutions. While rendering GameCube and Wii games at 1080p with anti-aliasing isn’t enough to make them comparable to more modern consoles, it really does help, especially with the clean, stylised graphics featured in most of Nintendo’s first-party titles.

At 1080p, and with the widescreen hack in effect, The Wind Waker could almost pass for a current-generation game

At 1080p, and with the widescreen hack in effect, The Wind Waker could almost pass for a current-generation game

Sunshine at 1080p really does look quite nice, but it’s nothing compared to The Legend of Zelda: The Wind Waker — I don’t think Nintendo will have to change much with its HD remake of the game for Wii U, since with a couple of million extra pixels it already looks remarkably close to a current-generation game. The thing that ages it most is the lack of a widescreen mode, but Dolphin actually has a hack that adds widescreen support to many games, Wind Waker included.

The catch with Dolphin is that it requires a reasonably powerful PC. It doesn’t need the latest video card — my 8800GT is perfectly fine — but it does need a decent CPU, and it’s there that my Athlon II X2, overclocked to 3.58Ghz, is barely enough for many games. Wii games seem to suffer the most, perhaps unsurprisingly; Zelda: Skyward Sword is just a bit too slow to be playable on my PC, and from all reports, Super Mario Galaxy and its sequel don’t run smoothly on anything less than the fastest PCs available.

With a real Wii Remote connected via Bluetooth, even MotionPlus-heavy games like Zelda: Skyward Sword are playable in Dolphin

With a real Wii Remote connected via Bluetooth, even MotionPlus-heavy games like Zelda: Skyward Sword are playable in Dolphin

For Sunshine, I’ve been using a 360 controller along with a $10 Xbox wireless receiver from DealExtreme, and the xboxdrv user-space Xbox controller driver; with its analog triggers and the very GameCube-esque controller layout in general, it’s a great fit. For Wii games, Dolphin can emulate a Wiimote of sorts, but it’s much easier to use a real Wiimote via Bluetooth.

ludum dare 26: anti-minimalist music and sampled orchestras

Posted on by
3

This weekend was Ludum Dare 26, and as usual when Switchbreak enters such things, I took the opportunity to tag along. The theme was “minimalism”, but his game, called MinimaLand, deliberately eschews that theme; it tasks the player with bringing life and detail in to a very minimalist world.

MinimaLand screenshot

In MinimaLand, the player brings life to an abstract, minimalist world

I wasn’t sure at first how to fit music to the game, but it soon became clear: if I was going anti-minimalist, I wanted to use an orchestra. Ever since I heard about the Sonatina Symphonic Orchestra, a CC-licenced orchestral sample set, I’ve wanted to try recording something with it; what better time to try it than with a deadline looming!

Given that I had just a few hours for this, I kept the music itself very simple — just three chords and a short melody. The music itself is almost irrelevant in this case, though, since it’s really just a means of delivering those orchestral sounds to the player. Initially, the melody and harmony are on strings, with rhythmic staccato stabs on brass, then the whole thing repeats, with the stabs moving to strings and the melody/harmony to woodwinds and horns.

It’s funny that, even when I’m dealing with sampled instruments instead of my own synth sounds, I still think in terms of sound and feel first, and chords and melodies second. I guess that’s just how I roll!

Working with LinuxSampler

That's a lot of LinuxSampler channels!

That’s a lot of LinuxSampler channels!

Not unexpectedly, I sequenced everything in Ardour 3 and hosted the SSO instruments, which are in SFZ format, in LinuxSampler, using a LinuxSampler build from a recent SVN checkout. I didn’t use anything else on this one, not even any plugins, since all it really needed was some reverb and the SSO samples already have plenty of it.

Recent versions of LinuxSampler’s LV2 plugin expose 32 channels of audio output; I guess the idea behind this is to allow you to run multiple instruments to dedicated outputs from within a single plugin instance, but I’m not sure why anyone would actually want to do that. I think my workflow, with each instrument in its own plugin instance on its own track, makes a lot more sense, so I patched the plugin to return it to a simple stereo output per instance.

Sonatina quality?

I’ve been keen to try SSO mostly to see just how usable it is, and in this case, I think it worked pretty well. With just 500MB of samples, it’s never going to sound as good as a commercial library (where individual instruments can take several GB), but some of the samples, such as the string ensembles, sound quite nice at first listen.

The biggest problem is with the availability of different articulations for each instrument. You do get staccato samples for most instruments, and pizzicato for the strings, but beyond that you just get “sustain” sounds, which are great for held notes (as long as the sample’s long enough), but far less suitable for faster legato parts. You can hear this in the horn part in the second half of the track, where some short notes take so long to start playing that they barely become audible before they end.

Many of the solo instruments are quite weak, too — you can hear audible jumps between certain notes in several of them, where the instrument jumps from one discrete sample to the next, while others have odd tuning problems.

There’s also a tonne of reverb on every instrument. SSO’s instrument samples come from a variety of sources, so each instrument has its own reverb characteristics; in an attempt to even out the differences and make them all sound at least vaguely like they’re in the same room, the library’s creator added quite a bit of extra reverb to each instrument. It’s a necessary evil, and it works, but it has a smearing effect that only exacerbates those problems with short notes.

So, SSO was well suited to this track — most notes were either staccato/pizzicato or were held for quite some time, I didn’t need to use any solo instruments, and the wall of reverb helped make it sound suitably pompous. If your needs are different, though, then you’ll have a hard time getting good results from it.

Having said that, it is far-and-away the best free option, and it’s also quite easy to get working under Linux, which can’t be said for many commercial libraries. Despite my mostly good experience with it, though, I’m keen to investigate just what commercial alternatives are available that will work under Linux.

cosplay mystery dungeon: sound design for a seven-day roguelike

Posted on by
1

I’ve spent the last week working on a game for the Seven Day Roguelike Challenge with Switchbreak and Amanda Lange, and by virtue of the fact that it’s a seven-day project, it’s now finished! It’s called Cosplay Mystery Dungeon, and you can play it here (if you have Flash installed).

screenshot

A week isn’t long, but I’m really impressed with the finished game — Amanda did great work on the art and the game design, and Switchbreak put a tonne of work in to the code. Here’s how my part of it all came together.

Getting in with the right crowd

It all started innocently enough, with a tweet:

Once Switchbreak was involved, it wasn’t long before I jumped on board, too. Amanda came up with the concept and had written up a lot of notes about the design, so I had a good idea of what sounds would be needed, and what they should sound like, from the get-go.

Early in the week, I worked on the basic player and weapon sounds, making a few versions of most of the weapon sounds to avoid any annoying reptition. The magic effects came later; Amanda’s design included various spells, in the form of collectable comic books, but with the deadline looming I wasn’t sure which of those would make it in to the game. As it turned out, Switchbreak managed to implement them all in the closing hours, so my last hour-or-so was a race to create the matching sounds.

The sounds were a mix of synthesis (using both my Blofeld and Loomer Aspect) and recorded sounds. Some used both synthesis and recording, in fact, such as the lightsaber — after reading about how the sound was created originally, I created a suitable humming sound, played it through one of my monitors, and then swung my mic back and forth past the speaker, recording the results.

Music in a hurry

I hadn’t planned to write music for the game, but it felt so odd without it that, with less than a day left, I decided to take a stab at writing something. I’ve written short pieces within hours for past Switchbreak games, but they’ve been much smaller than this. A run-through of Cosplay Mystery Dungeon can take an hour or more, not to mention the time spent on unsuccessful attempts, so the music needed enough length and variety to carry it over a longer playtime.

I started with the bass line and fast-paced drums, and I knew from the start that I wanted to add a later section with slower, glitchy drums, so those went in early, too. Soon after I nailed down the chord progressions and structure, and started filling in the melody, pad, and stabby brass lines.

This is what an all-MIDI, all-softsynth Ardour session looks like

This is what an all-MIDI, all-softsynth Ardour session looks like

As with my RPM Challenge work, I worked quickly by sticking with MIDI (no bouncing to audio), using softsynths instead of hardware, and mixing on-the-fly, with minimal (or none, in this case) EQ and compression. Synths do let you cheat a bit — if you find that a part is too bright or too dull, or needs more or less sustain, you can just edit the synth patch (tweaking filters and envelopes, for example) instead of using EQ and compression to fix those things after the fact. You can’t solve every mix issue that way, but I find that I can get a perfectly decent, listenable mix that way more quickly than I could otherwise.

All up, I think the music took about 5-6 hours to record, with another half-hour or so after that creating musical cues for the endgame screen and the game over screen, using the same instruments. That left me with just enough time to finish the magic sound effects before the deadline.

Loomer Aspect and Sequent earned their keep, alongside open-source plugins like Invada Tube and the TAL and Calf delays

Loomer Aspect and Sequent earned their keep, alongside open-source plugins like Invada Tube and the TAL and Calf delays

Loomer Aspect really paid for itself on this one. I used TAL NoiseMaker on the chorus lead sound (it’s a modified preset), and the Salamander Drumkit with the LinuxSampler plugin for the drums, but every other sound came from Aspect, mostly using patches that I created on-the-fly. For such a capable synth, it’s surprisingly easy to program — everything’s laid out in front of you, and it’s fairly easy to follow. It lacks the Blofeld’s distortion options, but using distortion plugins in Ardour (TAP TubeWarmth and Invada Tube Distortion) helped address that.

I also had an excuse to use Loomer Sequent — it provided the glitch effects on the drums in the slower section. The presets were all a bit too random to be usable on such sparse parts, so I edited the effects sequence in Sequent to match the parts, adding just a bit of randomness to its loop-slicing parameters.

This was the first track I’d recorded since the official release of Ardour 3, too. It worked really well — it was stable, reliable, and predictable throughout, a definite improvement on the betas. If you haven’t tried it yet, now’s definitely the time!

dr strangedrive redux: should you swap on an SSD?

Posted on by
1

In my recent post about SSDs I did make one major omission, which a friend pointed out on Twitter afterward:

Indeed, I don’t run a swap partition in my desktop PC — RAM is cheap, so I have 12GB of it, and if you’re debating the cost of an SSD, you can probably afford 8-12GB of RAM, too. Let’s play devil’s advocate, though, and say that you can’t upgrade your RAM for whatever reason. Conventional wisdom says that swapping on an SSD is a sure-fire way to send it to an early grave, but is that really the case?

Individual flash cells do have a finite limit on the number of times they can be erased, so it makes sense that if one part of your SSD (say, your swap partition) sees a lot more writes than other areas that it would wear out more quickly. That doesn’t actually happen on a modern SSD, though — they use wear leveling to spread writes as evenly as possible across all available flash. Even if you overwrite a single disk block repeatedly, the SSD’s controller will keep moving that block to different flash cells, transparently remapping things to hide the details from the OS.

Swapping on an SSD, then, should cause no more stress than any other write activity, so it should be perfectly safe, as long as those extra writes don’t push the SSD beyond what it can handle. This calls for another test!

The test

I forced my PC to use swap in a civilised manner, without resorting to pulling out sticks of RAM

I forced my PC to use swap in a civilised manner, without resorting to pulling out sticks of RAM

As in my last post, I observed my write traffic across a typical work day, but with one difference: I removed 8GB of RAM (by rebooting and adding “mem=5G” to my kernel command line, which left me with just over 4GB of RAM once various bits of hardware address space had been accounted for) and replaced it with a swap partition.

The write activity was much more spiky — there are several times when substantial amounts of data are written to swap — and it’s higher on average, too, but it’s clear from the graph that there’s still nothing to worry about. Across the day, about 2.7GB of data was written to swap, and the total data written was 13GB, well below the 5-year lifespan threshold of 40GB/day that I established in my last post.

made with ChartBoot

In fact, if you’re stuck with a PC with limited RAM, I’d heartily recommend swapping on an SSD! It’s so fast that you never really notice that you’re swapping, especially without the sound of a busy hard drive to remind you. In fact, I barely noticed that two-thirds of my RAM was missing.

Swap tuning

With some tuning, you may in fact find yourself using less swap on an SSD than you would on a hard drive. If you’ve been using Linux for a while, you’re probably learned (perhaps after making a semi-panicked “what’s using all my RAM?” post on a Linux forum) that Linux will use all of your free RAM as disk cache to improve performance. However, Linux goes further than that: it’ll sometimes push application data from RAM to swap just to grow its disk cache.

If this seems odd, consider a scenario where you have some apps running in the background that you’re not using at the moment. Doesn’t it make sense to page out those apps and free some RAM for disk caching to improve the performance of the apps you are using? On a hard drive, it certainly does, but random reads on an SSD are so fast that the benefits of that extra disk cache probably aren’t worth the cost of swapping.

You can control how likely the kernel is to use swap by altering the appropriately-named “swappiness” parameter. The default value is 60, and reducing this makes the kernel less likely to swap; on an SSD, you can probably drop this all the way to 0. To do that, add this to your “/etc/sysctl.conf” file, and then either reboot or run “sudo sysctl -p” to put it in to effect:

vm.swappiness = 0

Another parameter, “vm.vfs_cache_pressure”, is often mentioned in SSD tuning guides, too — this controls caching of directory and inode objects, with values lower than the default of 100 making the kernel more likely to keep those cached in RAM. The effect this has isn’t entirely clear to me, but if you want to experiment, add this to your “/etc/sysctl.conf” file:

vm.vfs_cache_pressure = 50

Of course, these values are just guides — if you find other values that work better for you, go with those instead. Don’t be afraid to leave these at their default values, either; Linux has a multitude of tunable paramaters, and just because you can tune something doesn’t mean you should, especially if you’re unsure what effect different values might have.

A note on drive life estimates

After two weeks, I'm yet to go through a single full erase cycle on my drive. That's reassuring!

After two weeks, I’m yet to go through a single full erase cycle on my drive. That’s reassuring!

It’s worth mentioning, too, that this 72TB estimate of the M4’s lifetime seems to be somewhat conservative. Its flash cells can handle about 3000 erase cycles before failing, so if you overwrote all 256GB of flash 3000 times, you’d get not 72TB of writes, but 750TB. The factor-of-ten disparity between these two figures is due to a phenomenon called write amplification, where the shuffling of data performed by wear leveling and garbage collection causes some data to be written to the underlying flash more than once.

The controllers inside SSDs strive to keep write amplification as close to 1 as possible (that is, no amplification), and some even use compression to push it below 1 in some cases. How successful they are depends on the several factors: the nature of the workload, how much spare flash the controller has to work with (this is where TRIM really helps), and just how good the controller’s algorithms are. A write amplification factor of 10 is really quite extreme, so I’d expect my M4 to last far beyond 72TB of writes (assuming the controller doesn’t fail first).

The 3000 erase cycles is just a conservative estimate, too — that’s when flash cells are likely to start dying, but they won’t all die at once, and most SSDs include some amount of spare flash that they can substitute for failed cells. In one endurance test, a user managed 768TB of writes to a 64GB Crucial M4; at that smaller size, that works out to more than 12000 erase cycles.

dr strangedrive or: how I learned to stop worrying and love SSDs

Posted on by
3

I’ve had bad luck with hard drives lately — in the last month or so I’ve lost two of the drives from my desktop PC. Luckily, I’d set up RAID-1 for my Linux install just beforehand, so I didn’t lose anything important (just my Windows drive, hah), but with just one drive left, I needed some kind of replacement.

I could’ve bought another hard drive, but damnit, spinning disks are from the past, and we’re living in the future! Instead, I bought myself a shiny new SSD.

Wolf in mini-sheep’s clothing

To be specific, I got a 256GB Crucial M4 — it’s not the latest and greatest SSD, but it’s been on the market long enough to prove its reliability. It looks so unassuming in its tiny, silent 2.5″ case, but it’s crazy-fast, with read speeds of 450MB/s, write speeds of about 260MB/s (not as fast as some newer drives, but perfectly respectable), and insanely-fast seek times that can make it dozens or even hundreds of times faster than a hard drive in real-world applications.

More than anything else, an SSD makes your PC feel strangely snappy. Boot times and application launch times both benefit hugely — Firefox now takes less than a second to spring to life, even if I’ve only just booted my PC, and staring LibreOffice takes maybe half a second.

Even when attached to a 3.5" bay extender, SSDs look tiny compared to 3.5" hard drives

Even when attached to a 3.5″ bay extender, SSDs look tiny compared to 3.5″ hard drives

To get some numbers, I tested something that’s always been slow on my studio PC: loading large instruments in to LinuxSampler. LS streams most of the sample data on-the-fly, but it still needs to cache the start of each sample in to RAM, and that requires a bunch of seeking. Here you can see the load times for Sampletekk’s 7CG Jr, a 3GB GigaSampler file, and the Salamander Grand Piano, a 1.9GB SFZ, from both my SSD and my old 1TB Seagate Barracuda 7200.12 hard drive — the SSD is about 4-to-6 times faster:

made with ChartBoot

Is flash’s limited lifetime really worth worrying about?

So, SSDs have fantastic performance, and they’re now (relatively) affordable, but I did have one concern: the fact that flash memory cells can only be erased a certain number of times before they wear out. Modern SSDs use techniques like wear-leveling and over-provisioning to minimise writes to each flash cell (this Ars Technica article is a great read if you want to know more), but it’s hard not to think that every byte you write to the drive is hastening its demise.

I worried even more after I ran “iotop” to look at per-process disk usage, and saw that Firefox was writing a lot of data. It writes several things to disk on a regular basis — cached web content, knowing malware/phishing URLs, and crash recovery data — and that can add up to several MB per minute, or several GB per day.

To see if this really was a problem or not, I used iostat to capture per-minute disk usage stats across a typical day. I did all my usual things — I left Firefox, Chrome, Thunderbird, and Steam running the whole time, I spent my work hours working, and then I toyed with some music stuff in the evening. The results are graphed below:

made with ChartBoot

There’s one hefty spike in the evening, when I copied 3.6GB of guitar samples from my hard drive to my SSD (maybe this wasn’t an entirely typical day!), but for the most part, I was writing about 5-15MB per minute to the SSD. The total for the day was 15GB.

That sounds like a lot, but it’s nothing my SSD can’t handle. It’s rated for 72TB of writes over its lifetime, and while that’s an approximate figure, it’s a useful baseline. Over a five-year lifespan, that works out to 40GB of writes a day, or 27.8MB per minute — that’s the red line on the graph above, which was well above my my actual usage for almost the entire day.

When you see a graph like this, it flips your perceptions. If I’m happy to accept a five-year lifespan for my SSD, then every minute I’m not writing 27.8MB to it is flash lifetime that’s going to waste! Smaller SSDs tend to have shorter lifetimes, as do cheaper SSDs, but with typical desktop usage, I don’t think there’s any reason to worry about the life of your SSD, especially if you’re not using your PC 10-12 hours a day or running it 24/7 like I often do.

SSD tuning

There are dozens of SSD tuning guides out there, but most of them spend a lot of time whipping you in to a “don’t write all the things!” frenzy, so instead of linking to one of those, I’ll just reiterate two things that you should do to get the most from your SSD.

The first is to enable TRIM support. This lets the OS tell the SSD when disk blocks are no longer needed (because the files they contained were deleted, for instance); that gives the SSD more spare space to use, which helps reduce drive wear and increases write performance. To enable TRIM, add “discard” to the mount options on each filesystem on your SSD, like so:

/dev/mapper/ssd-ubuntu_root  /  ext4  discard,errors=remount-ro  0  1

IF you’re using LVM, like I am, then you’ll also have to edit the “/etc/lvm/lvm.conf” file, and add the line “issue_discards = 1” to the “devices” section, to make sure that LVM passes the TRIM commands through to the SSD.

The second is to select an appropriate IO scheduler. IO schedulers are bits of code within the Linux kernel that arrange read and write operations in to an appropriate order before they’re sent to the disk. The default scheduler, “CFQ”, is designed to keep for desktop loads on regular hard drives, but its efforts are wasted on SSDs, where seek times are so much lower.

For SSDs, you’re better off with the “deadline” scheduler, which is designed for high throughput on servers, where disks tend to be faster, or you can even use the “noop” scheduler, which does no reordering at all. To set the scheduler on boot, add this to your “/etc/rc.local” file (most Linux distros have one of these):

echo deadline >/sys/block/sda/queue/scheduler

To be honest, the choice of IO scheduler probably won’t make much difference — it just improves performance a little (it won’t have any impact on lifespan), but your SSD is going to be so fast regardless that I doubt you’d ever notice. It’s an easy fix, though, so it’s worth the 10 seconds it’ll take to perform.

So go forth, buy an SSD, make a couple of minor tweaks, and then don’t be afraid to enjoy it!

creating a dynamic soundtrack for switchbreak’s “civilian”

Posted on by
1

Over the last week I’ve put a bunch of time in to my new game project, a Switchbreak game called Civilian. I’ve been working on music for it, but this blog post isn’t about music — it’s about the crazy stunts you can pull in modern interpreted languages.

Dynamic music in Flash?

Most Flash games use a looping MP3 for background music — it takes just a couple of lines of code to implement, and while the looping isn’t perfectly seamless (there are brief pauses at the start and end, added by the MP3 encoder) it’s usually close enough. For Civilian, though, I wasn’t satisfied with a simple looped track. It’s a game about the passage of time and about player progression, and I wanted the music to reflect those things.

What I really wanted was a dynamic music system, something that would let me alter the music’s sequence or instrumentation on-the-fly in response to the player’s actions. There was no way that was going to work with simple, non-seamless looping MP3s, though — I needed to start working with audio data on a much lower level.

Writing a low-level mixer in AS3

Thankfully, the Flash 10 APIs do give you low-level audio functionality. You can get raw audio data out of an MP3 file, and then send that to audio buffers for playback; I’d already done just that in fact, to implement a seamless MP3 looper, and that gave me a crazy idea: if I could get audio data from one MP3 and play it back, could I also get data from two or more MP3s, mix them, and play them back all at once?

Once I’d confirmed with a simple proof-of-concept that the answer was an emphatic “yes”, I set about adding more tracks, and then implementing features like panning and volume conrol. By this point, the amount of CPU power required to run this mixing was significant — about 40% of one core on my 1.7Ghz i5 Macbook Air — but Flash had no trouble keeping up while running some simple gameplay at 60FPS.

A screenshot from my test app, with five channels of audio running

A screenshot from my test app, with five channels of audio running

From mixer to sequencer

A few days later I had more than just a mixer: I had a simple pattern-based sequencer. Instead of looping MP3s from start to finish, it splits the MP3 tracks in to bars, and then plays those bars in accordance with a sequence stored in an array in the AS3 code.

This actually fits quite well with how I tend to write my music. I can arrange the track basically how I want it in Ardour, then record each unique section of each track to audio, and string those sections together to produce a single MP3 track for each instrument. Then, I can create a sequence within the AS3 code that reassembles those sections in to my original arrangement.

Each bar can have its own settings, too, somewhat like the effects on each note in a tracker. So far, these just let me set the panning or volume for each track, or set up a volume slew (ie: a fade in or fade out) to run over the course of the bar.

Making the music dynamic was just a matter of replacing the static sequence array with code that generates the sequence on-the-fly. I have pattern templates for each track which I combine to create the sequence one bar a a time, adding or removing tracks or replacing one part with another (perhaps with a nice fade in/fade out) based on what’s happening within the game world.

Pushing interpreted languages

As if all the above wasn’t enough, I decided to add an optional audio filter on the output. For certain scenes in the game I want to be able to make the music sound like it’s coming from a radio, so I added a simple bandpass filter, based on a Biquad filter implementation from Dr. Dobbs. If the filter is having any impact on my sequencer’s CPU usage, it’s far too small to notice.

Eventually, I gave up trying to think of efficient ways of doing things, and just started doing them in the simplest way possible. I’ve since done some optimisation work, to help retain a steady frame rate on slower systems (using my old Latitude E6400, clocked down to 800Mhz, as my test machine), but those optimisations are totally unnecessary on more typical systems.

Ten years ago, I wrote audio mixing code for the GBA, and it looked something like this

Ten years ago, I wrote audio mixing code for the GBA, and it looked something like this

The last time I wrote audio mixing code, it was for the ARM7 CPU inside the Gameboy Advance. On that system, compiled C code wasn’t fast enough, so I had to re-write the critical loops in hand-optimised ARM assembler code to get the necessary performance. To see an interpreted language do the same things so easily is still somewhat mind-boggling, but it’s a testament to the advances made in modern interpreters, and to just how fast modern PCs are.

It’s somewhat fitting that this was the week that the GNOME developers announced that JavaScript would become the preferred language for GNOME app development. That announcement caused a surprising amount of backlash, but I think it makes perfect sense: not only is JavaScript a capable and incredibly flexible language with a huge developer community. but it performs incredibly well, too. In fact, I doubt that any other interpreted language has ever had as much developer time invested in improving its performance.

The writing’s on the wall for Flash, of course, but HTML5 and JavaScript are improving rapidly, and frameworks are being written that should make it just as easy to write games for them as it is to write for Flash today. When that happens, it should be a simple matter to port my dynamic music system to JavaScript, and I’ll be very excited to see that happen.