Christian Montoya

The Story Behind Emoji Love Story

This entry is cross posted from Medium

In November of 2013, while on my way home from work, I had an idea for a chord progression that I thought would work well for a disco revival track. By that night, after a few hours of working in Ableton Live with a handful of virtual synthesizers and drum machines, the first draft of “Love Stories” was born.

The initial direction of the project had a more unusual synth palette than I originally intended, and subsequent revisions took a distinct Italo Disco direction. I grew up with the music ofLime and Gary Low so I was perfectly OK with this. I worked on Love Stories on and off for months before I had a possible final draft in April of 2014 that I was ready to share with the world. A younger version of myself would have probably published it to Soundcloud immediately, but after 5 years of working on and releasing music I was starting to put more effort into planning each release, so I uploaded Love Stories as a private track on Soundcloud and sent the link to a handful of trusted musician friends for feedback.

The response was mixed; my friends liked it but they weren’t blown away. One of my friends, Daniel Davis (who performs under the moniker an0va) suggested that the track needed a melodic lead line to really carry it, and offered to take a stab at adding this. We had collaborated previously on a project for Namco Bandai’s web series Wonder Momo and were already accustomed to working together remotely in the cloud. I published Love Stories as a private project on Blend and invited Daniel to collaborate; within minutes he had the project open on his computer and began improvising over the track with his guitar.

Daniel continued working on his version for just over a month. When he finally shared his revision with me I was impressed. The track now had a catchy melody that sat well between the Italo Disco foundation of the song and the complex, upbeat melodic nature of Videogame music that has always been a huge shared inspiration for us. I ended up taking the track a step further by using sounds from the classic SEGA videogame Sonic the Hedgehog for the melody. The presets for these Sonic the Hedgehog sounds actually came from a sample pack published on Gumroad by Hunter Bridges, a musician I met on Blend.

I will definitely be using this in future projects

At this point I was ready to publish Love Stories, but Daniel had reservations. He felt that the arrangement needed better transitions and extended breaks in the melody so that it wouldn’t feel like a keyboard player was playing nonstop throughout the track. He took the track and began making additions to the arrangement, and we ended up going back and forth through multiple revisions after that. The segments from 2:07–2:39 and 4:14–5:17 came out of this process, as well as the violin layers and the addition of two more instruments to change up portions of the main melody. Clocking in at 5:54 & with 25 layers in total, this became one of the most complex music projects I have ever worked on, and probably the most exciting one yet.

Obligatory Ableton Live screenshot

So this brings our story to October of 2014, and armed with a complete track that both Daniel and I were happy with, I found myself at the next set of challenges: naming the track, making the cover art, and planning how I would release it. These steps can be just as hard as making the track itself.

The Name

Richard Alexander Caraballo, a friend of mine who performs under the moniker minusbaby, has often said that Emoji is the pidgin language of our generation. It’s an apt description, and one I’ve thought about many times while watching this little library of iconography proliferate across the world of computing (and the world in real life, too). Call it perfect timing or just a coincidence, but while wrapping up Love Stories I was thinking about how emoji are analagous to this little song I had been working on for almost a whole year. Like emoji, an instrumental track is devoid of words, and it attempts to communicate an idea through a familiar conveyance (one, a recognizable arrangement of pixels resembling a real life object, the other, a recognizable melodic rhythm resembling a common style of dance music). And like my song, Emoji are both reminiscent of something we’ve all seen before (italo disco, chat smileys) and part of a continuously evolving medium of communication (electronic dance music, the emoji standard?—?why is there still no duck emoji, by the way?). And so Love Stories became a Love Story dedicated to Emoji, this new but very familiar communication feature that invaded so many of my digital messages over the past year.

The Cover Art

I wanted to incorporate emoji into the cover art, obviously, but also achieve that slightly cheesy and yet cool style of Internet art that could be associated with a track like this. I was inspired by the cover art for various songs from Internet-savvy musicians that I had been listening to over the past few years (like the one below).

Aurora by Anamanaguchi (Artwork by David Mauro)

So armed with my limited and dangerous knowledge of photoshop, I managed to mashup the three things I could think of that would fit this track well: the ‘rotating hearts’ emoji, a photo of a beach, and a photo from a NASA satellite. I also added the ‘100' emoji because it is often used to represent 100%, which is what I usually type into to the description of my music projects when publishing a finished track on Blend.

The final cover art

The Release

At this point I had been working on this song and associated materials for a full year. I also had a remix provided my friend and frequent collaborator Alexander Tregaskis (who performs under the moniker radionarcotix) to round out a 2-track EP. I felt like too much work had gone into this so far to just upload it to the Internet as it was. I wanted to do something new and clever to promote it?—?in the past I’ve done things like DJ mixes and music videos, but this time I wanted something different. I ended up deciding to make a game because I have experience in that and I thought it would fit the theme well, but moreso because I thought it would be cool to use emoji for the art. So I came up with the idea of doing a matching game using emoji for the cards and set out to find if there were any tutorials or examples of this.

Fortunately I came across MemoryGame.js by Mark Rolich. I was able to fork it and use EmojiOne to render emoji in the browser without having to worry about device support. For the audio I wrote code from scratch to build a player with emoji for the controls and the ability to play continously through a playlist with pause & next track support. The whole thing was relatively easy and I was even able to use the HTML5 Audio API to detect support for audio file formats and choose the best format for the user’s browser (between AAC, OGG and MP3).

The web game

Unfortunately I came across one major caveat: there’s no emoji for a stop button! There’s play & fast forward, but for stop I had to use :white_square_button: instead. Not ideal!

Play / Forward

Pause / Forward

The next phase involved porting the game to mobile with the help ofCordova, the open-source foundation for PhoneGap. This is, of course, where things get interesting. I won’t get into all the gory details of what it took to get working builds of the app for Android and iPhone—instead, I’ll share a list of gotchas I ran into along the way:

  • The game relies on CSS3 animations for flipping the cards. Support for this in mobile browsers is, unfortunately, not very good. On iOS, flipping a card can cause the whole grid to flicker, while on some Android devices, the emoji will disappear as soon as the flipping animation completes (I call this Hard Mode). I’ve come to the conclusion that while CSS3 animations work pretty well on modern browsers on the desktop, they are not sufficiently supported in mobile browsers and it was a mistake to rely on them for this project.
  • On iOS and Android, native emoji are displayed. This works fine except for two issues: 1. positioning is unpredictable 2. apparently you cannot resize emoji in iOS Safari or webviews. The native emoji are displayed very small by default (the size you usually see in your text messages) and the only way that I could resize them was by using -webkit-transform.The first game on the app store made entirely of emoji doesn’t seem to have this problem because it’s using native display code.
  • The simple HTML5 Audio code that I wrote to handle playing music did not translate well in Android and iOS. iOS was the worst culprit—iOS Safari does not allow autoplaying audio at all and the audio doesn’t even work if user saves the app to their homescreen from the web version. I ended up completely rewriting the audio code to use Cordova’s Media plugin instead. This worked pretty well, with the exception that I could not get the player to autoplay the next track when the current track completed (I could have fixed this if I spent more time, but I pushed the apps to the app stores before I even found the problem).
  • Even with Cordova’s Media plugin handling audio support, there was a lot left to be desired. The Media plugin has a lot of outstanding fixes that have been sitting in the pull request queue for months, the most important one (in my opinion) being to play audio in iOS in a background thread, which is really the only way that apps should implement background music—this is a pull request that’s been sitting in limbo since February 2014. So basically, my app doesn’t implement audio well at all, and it would take quite a bit of work (and going off the official plugin path) in order to fix the implementation.
  • I developed the iOS app by creating an Xcode project in Cordova using the “prepare” command and then opening that project in Xcode to make sure that all the settings were correct. After all my testing I made the careless mistake of setting an XIB as the Main Interface to be loaded on app launch—apparently, this is a bad idea when using Cordova because there is no XIB to load. The result of this was that my first build submitted to Apple crashed on every load no matter what, and I was only able to fix the issue after frantically searching on Google to find if anyone else had made the same mistake. I’m not even sure where the blame lies for this bug, but I can’t help thinking that the Cordova CLI should warn you about this every single time you build your app.

Not even once

  • The amount of work involved in making icons and splash screens for all the different device resolutions across iOS and Android is absolutely untenable. It was the most unpleasant part of the whole process. Next time I am definitely going to find a way to generate these files quickly, but I’m disappointed to see how bad this step has gotten.

Please, stop

  • I would have developed a Windows app just for the sake of doing it, but I was on vacation while I was building the apps and I didn’t have access to my Windows machine. Even if I could have done it, though, I wouldn’t have had a Windows Phone device to test on. I have an iPhone and an iPad, and my wife and most of my relatives have Android phones. That was my test suite for this project. It’s amazing to me that even as a hobbyist app developer, you need a box full of devices just to be able to test your mobile apps—it seems to me like one of the biggest barriers to entry for anyone wanting to do mobile development. It doesn’t help that compatibility from one mobile OS version to the next can be so abysmal, forcing testers to have multiple devices running different OS versions just to hunt down version-specific issues. I have nothing more to say on this topic other than it shouldn’t have to be this way.
  • If I had more time I would have looked into storing the player’s high score and allowing sharing of the results via Twitter and Facebook, but I still need to learn how to do these things efficiently with Cordova.

There’s probably more that I could say about the app development phase, but in the interest of not rambling, I’m going to recommend that anyone with questions about the app(s) is welcome to talk to me via Twitter?—?I’ll answer any questions I can.

(Just) Putting It Out There

I wanted to publish Emoji Love Story in as many places as possible. So far the list includes:

The response has been good so far. I’m doing this without a marketing budget or PR support so I’m just relying on word of mouth via my social channels, and to be honest, after spending so much time on this project I don’t even care how people respond—the project is cool on its own. Still, the whole thing was a valuable learning experience, from collaborating on the track to building a full game to go with it, and I enjoy the opportunity to share that knowledge with others. I always enjoy seeing music presented in fresh ways, and if others are inspired to do projects like this in the future, that would be a very positive outcome.

So, to you, thanks for reading this (very long) write-up, and thanks for listening to—and playing—Emoji Love Story.

Thank you for reading • Published on December 11th, 2014 • Please take a moment to share this with your friends