Highlights of the first Web Audio Conference

I recently had the pleasure of attending and presenting at the first Web Audio Conference, co-sponsored by IRCAM and Mozilla. I wanted to share some of the great things that I learned about there.

Meyada, Hugh Rawlinson

Meyada is a library for computing various properties of an audio signal, including various measure of perceived loudness. These measures are great for creating visualizations that users perceive as correlated with sounds. Hugh used them to make live visualizations inside his talk slides, very slick.

Extending Csound to the Web

Csound is a well-established programming language for generating and manipulating sound. Because it is written in standard C and has very few platform dependencies, it is an ideal candidate for cross-compilation with Emscripten and asm.js. The authors noted that the resulting code was not as fast as the PNaCL version they created, but that it was plenty fast enough to render many sounds in near real-time, and currently has a better cross-platform story.

musicn.js, Chris Lowis

Csound traces its lineage back to the MUSIC-N languages originated by Max Mathews at Bell Labs in 1957. Chris’s talk showed both how much has changed since then and how the fundamental ideas from MUSIC-N endure.

Lissajous, Kyle Stetz

Kyle Stetz described a very opinionated system he built for controlling live music by writing code in the Google Chrome Console window. He put his ideas on the line by doing a live demo in front of us, which was quite well-received. Although the system may not generalize to all styles of music, I think we all admired his clarity of purpose and great presentation.


HyperAudio is a clean, simple UI for searching, browsing, and excerpting videos with spoken word audio, especially speeches and debates. The experience does rely on the hard work of creating  transcripts of the audio with very accurate timings; as automation makes that process easier, tools like HyperAudio will change the way ordinary people relate to video.

EarSketch, Jason Freeman

Jason Freeman and his colleagues at Georgia Tech brough a unique perspective to WAC. Instead of concentrating on how to use web technology to make music, they are interested in how to use web-based music tools to teach fundamental computer science concepts. When they use these techniques with high school students, they can show a significant increase in the number of traditionally under-represented groups in computer science.

The Tomb of the Grammarian Lysias, Ben Houge

The evening of the second day of the conference featured six different “Web Gigs”, which were participatory musical experiences designed around many mobile phones in a concert space with a centralized coordinator and sound system. In my opinion, the most successful of these was Ben Houge’s The Tomb of the Grammarian Lysias. The piece consists of solo vocalist singing an ancient poem in the original greek accompanied by snippets of vocal sounds. The accompaniment is played through the mobile phone speakers of everyone in the room, under the composer’s control. The carefully designed sounds emanating from dozens of speakers throughout the room gave a wonderfully diffuse, ethereal quality to the piece, while his clear singing voice provided a visceral, earthy contrast to all the technology involved.

Posted in Uncategorized | Leave a comment

Using Google Maps in a responsive design

[When I’m not in the office, I’m often out birding and photographing birds. To keep track of my life list and bird photos I wrote a Ruby on Rails site hosted at birdwalker.com; you can see all the source on github. This post is about some the evolution of that code]

I love maps and I love visualizing data, so when I started to accumulate location-based birding observations, I knew I wanted to visualize my data on a map. I created a Rails partial template called _google.html.erb that let me insert maps into any of my other templates. All was well.

Since the first version of that template I’ve struggled with two issues that I expect other web developers will encounter.

1. Making Google Maps responsive

When I first brought my Google Maps into a Twitter Bootstrap design, I found the Bootstrap CSS would resize the container for the map, as I hoped. Unfortunately, Bootstrap could not magically adjust the scaling or center of the map. As the map got smaller, it would show only the upper left corner of the original map area.

To fix this problem, I wrote a function to retrieve the width of the container, pass that info the Google Maps API, and trigger a resize event.

function resizeBootstrapMap() {
    var mapParentWidth = $('#mapContainer').width();
    $('#map').height(3 * mapParentWidth / 4);
    google.maps.event.trigger($('#map'), 'resize');

I also added an event listener to invoke this function whenever the window changed size:

// resize the map whenever the window resizes

2. How much interactivity should I provide?

When I finally got my maps to size up and down responsively, I was very pleased. I used Bootstrap’s grid layout, so that the multiple columns on big screens would collapse down to one column when viewed on phones, with the map taking up almost the whole width of the screen (you can use Firefox’s responsive design view to see this in action).

When you embed a Google Map, you get a lot of interactivity by default — panning, zooming, the ability to click on map pins and other objects. On laptops and desktops, I like the result. I can embed a map showing you an overview of all the places I’ve birded in my home county, where you can also let you zoom in and find the names and descriptions of each. But this interactivity can be a problem on phones. It became difficult to scroll through and beyond one of these maps. I’d be dragging my finger on the phone, and instead of scrolling the page, I’d be panning the map.

I then tested two alternatives. First, I tried using the Google Static Maps API, which has no interactivity. However, it doesn’t work with Bootstrap’s responsive layout for all the same reasons I discussed in my post about D3.js. Second, I tried using the dynamic Maps but disabling most event handlers. This also works, but eliminates the ability to browse the map and see the metadata I’ve associated with each map pin.

So, for now, I’ve restored the panning controls and draggable maps and just try to scroll carefully. Thoughts? I’d love to hear from you how you solved this problem.


Posted in HTML5, Ruby on Rails | 4 Comments

Video of my YUIConf talk about open web app development

yuiconf-wfwalkerThe YUIConf folks have just posted their recording of a talk I gave there last year. I had planned to talk generally about the Firefox Marketplace, how it differs from other app stores, and how those differences create an overall apps ecosystem unlike other app ecosystems. All true, and all important.

I ended up giving a completely different talk. I decided I really wanted to show to use the Firefox developer tools to test, deploy, and debug an open web app running live on a Firefox OS phone. I had a lot of fun demoing live, including use of my patent pending cardboard and WebRTC ELMO replacement.

Take a look, I hope you enjoy it!


Posted in HTML5, Mozilla, Trip Reports | Leave a comment

Replacing Google Image Charts with D3.js

[When I’m not in the office, I’m often out birding and photographing birds. To keep track of my life list and bird photos I wrote a Ruby on Rails site hosted at birdwalker.com; you can see all the source on github. This post is about some recent improvements to that code]

When I first discovered Google Image Chart API, I had written little JS and was excited about how easily I could create decent-looking bar charts without including big libraries or writing a lot of code. I added a method in my Rails ApplicationHelper class to generate a URL for the Google Image Chart API by encoding my parameters and data values into the expected format:

def counts_by_month_image_tag(totals, width=370, height=150) 
  monthly_max = 10 * (totals[1..12].max / 10.0).ceil

  stuff = {
    :chco => 555555,
    :chxt => "y",       
    :chxr => "0,0," + monthly_max.to_s,
    :cht => "bvs",
    :chd => "t:" + totals[1..12].join(","),
    :chds => "0," + monthly_max.to_s,
    :chs => width.to_s + "x" + height.to_s,
    :chl => Date::ABBR_MONTHNAMES[1..12].join("|")

  chartString = ("http://chart.googleapis.com/chart" + "?" + 
    stuff.collect { |x| x[0].to_s + "=" + x[1].to_s }.join("&")).html_safe

  ("<img src=\"" + chartString + "\" alt=\"Totals By Month\" width=\"100%\"/>").html_safe

I started using this helper from my various page templates and all was well. Time went by.

Eventually, I became disenchanted with this solution. I can develop and test the rest of the site locally on my laptop, but the external image URL’s like those aren’t reachable offline. The Google chart images are statically sized, so they’re ill-suited to responsive design frameworks like Twitter Bootstrap. And, rendering charts as images doesn’t allow for any interactivity. [note: Google addressed these deficiencies in a subsequent version of their Chart API, which does most of the work client-side, and adds a lot of scope for interactivity].

Meanwhile, I became more comfortable with JS and began to create more of birdwalker.com using it. After attending a conference talk about D3, I decided to copy a sample bar chart and adapt it for my purposes. I put the finished code in a Rails partial called _species_by_month.html.erb

D3.js code works by chaining many function calls together. Each call has a very specific purpose, and the calls can generally go in any order. By chaining them together, you get the overall behavior you want. For example, to create the bars in my bar graph I do this:

var bars = svg.selectAll("rect").remove().data(sightingData).enter()
  .attr("x", function(d, i) { return x(d.month); })
  .attr("y", function(d, i) { return y(d.count);})
  .attr("height", function(d, i) { return height - y(d.count); })
  .attr("width", function(d, i) { return x.rangeBand(); })
  .attr("class", "bargraph-bar"

The calls to attr() specify the location and size of an SVG rectangle and assign it a CSS style name. By calling data() you can iterate over an array of data; by calling enter() you can create new SVG nodes for each item in the array. In this case, we iterate over twelve numeric values representing birding activity during each month of the year and create twelve rectangles.

These new graphs resize nicely as elements within my Bootstrap layout. And I was able to create a tooltip when mousing over each bar like this:

svg.selectAll("rect").on("mouseover.tooltip", function(d){
    d3.select("text#" + d.month).remove();
    .attr("x", x(d.month) + 10)
    .attr("y", y(d.count) - 10)
    .attr("id", d.month);

And remove the tooltip on mouse exit like this:

svg.selectAll("rect").on("mouseout.tooltip", function(d){
    d3.select("text#" + d.month).remove();

The tooltips are styled with CSS just like the other graph elements.

So far, my experience with D3 has been really great. I now have graphs that work well in a responsive layout, don’t require external image links, and have some interactivity. Suggestions and code reviews welcome!


Posted in HTML5, Ruby on Rails | Leave a comment

DevCon5 — on learning from games, Monty Sharma, Mass Unity

Monty Sharma gave a great talk this morning at DevCon5 about the current trends in games and what the rest of us can learn from them. He centers on three big ideas:

  • Free-to-Play. Find a way to let dedicated users pay more to get more stuff, but offer free functionality to everybody who asks
  • Compulsion Loops. Find ways to get people hooked into coming back to your software in order to satisfy some compulsion
  • Engaged Communities. Give up some control to bring your users into a community with one another.

I was really interested in his retelling of the EVE Online community responding to something called GoonSwarm. I can’t quite follow the internal politics of the game, but I can see that thousands of users organized themselves into highly effective ad hoc groups in order to battle one another within the game. Are your users that passionate and engaged?

Posted in Trip Reports | Leave a comment

jsEverywhere: Apathy is the Enemy of Awesome by Nancy Lyons

Nancy Lyons of geekgirlsguide gives a real end-of-day pep talk about how failed communication and collaboration can kill projects. I’ll put in a few of her zingers and how I understand them:

  • Learn how to talk about what you do to people who have no idea what you’re talking about.

Her evangelical zeal is very refreshing; I can imagine her listening to all these tech talks and thinking, “none of this is going to save you if your team can’t talk to each other.”

  • Don’t drop truth bombs

She urges us to imagine what non-technical clients who haven’t thought about iterations and known bugs will hear when you talk about bugs.

  • Don’t define scope in a consulting proposal

Instead, define the requirements collaboratively alongside the client. You have no idea what you’re talking about when you start!

Posted in Uncategorized | Leave a comment

jsEverywhere: PhoneGap CLI and PhoneGap Build Steve Gill, Adobe

Steve Gill is demoing “Cordova Client”, in which you have command-line tools to build, deploy, and manage Cordova-based applications on Blackberry, Android, and iOS. This is still in beta. It’s at github.com/filmaj/cordova-client. It requires Node.js, should be available in npm.

Posted in Uncategorized | Leave a comment