Move to the Music

This past weekend I had the awesome experience of seeing some HCI in the wild. I was at the Mingei Museum for one of their Early Evening events featuring Bostich and Fussible from Nortec Collective, and they were creating music using an unusual table. The table is a glowing blue surface with a collection of acrylic blocks with funky symbols placed on it. The artists create and change the music simply by moving and spinning the blocks in relation to each other. I was thrilled to recognize the table as the reacTable, a research project from the Music Technology Group at Universitat Pompeu Fabra, in Barcelona, that I’d read about in my Ubiquitous Computing class.

The Tangible Interface


reacTable in action

The table is a unique interface to a software based synthesizer for electronic music. Before the reacTable, the DJ using this synthesize had to use a mouse to control the music, while viewing the waveforms on screen. As you can imagine, creating complex music with the mouse interface is not very natural or intuitive. The Reactable allows more than one DJ to manipulate multiple tracks easily and naturally on a shared table.

The DJs associate tracks with a certain set of blocks that each have unique patterns on the base of the block. When those blocks are placed on the surface, a video camera beneath the surface can “see” the unique pattern, recognize the track and start playing it. The waveform is displayed on the glowing table to provide visual feedback to the musician. By moving it closer or further from the center or other blocks, the music can be transformed – sped up, slowed down, filtered, amplified. You can see a video of how this looks below.

Even if you have never DJ-ed before, you might be able to see how manipulating these blocks is a much more natural interface than using a mouse to move various sliders. The table itself is a joy to look at, adapting into the dark environment of a club without the harsh glare of a laptop screen. As a result, the reacTable has been quite successful; the first major event being Björk’s world tour in 2007, which premiered at the Coachella Music Festival that year. In 2010, Reactable released a award-winning mobile versions (without the blocks) for the iPhone and iPad and more recently, one for Android.

Musical Instruments as Interface

Musical instruments pose a unique problem for HCI. Think about it… We wouldn’t call a violin poorly designed, given the beautiful music it can create and its enduring nature. But at the same time, a user study with novices trying out a violin for the first time is bound to prove disastrous.
Will they find it easy and enjoyable to use? Probably not.
Will they achieve the goal of creating beautiful music? Almost certainly not.
So how do we qualitatively design and test for the unique experience of creating music? I find this to be a unique challenge in HCI.

We all agree that musical instruments are powerful and valuable interfaces, as evidenced by the variety we have preserved for so many generations. However, in the brand new world of digital everything, I’m interested to see if we are able to create similarly enduring interfaces that will stand the test of time as well as the violin has.



The best thing about Zebranet is the fact that it’s actually a network solution designed for zebras! Why do zebras need a network, you ask? Well, the network is actually for scientific researchers to study the migration patterns of wild zebras as well as their daily, social behavior.

Zebras by Chris Willis

Personally, I also love two specific aspects about this work, the first being that the researchers had to design for and work with real zebras in the wild and the second, that they got to travel to Kenya to do it.

The Design Challenge

The ZebraNet proejct from Princeton University had two major parts, designing a collar that zebras could wear that would collect information about their movement patterns and designing a peer-to-peer network protocol that would allow a large fraction of the data to return back to the researchers even if many of the zebras are out of the range of the receivers.

In addition, the receivers (or base stations) are not standing devices like cellular base stations. Instead, the researchers drive vehicles around the savanna to collect the data, hoping to get in range of a few zebras.

How it works

ZebraNet Project

ZebraNet Project

So how does this work? Essentially, the collars collect data on the movements of the zebras using GPS. They the forward this data to other zebra collars that have historically been successful at transferring data to the base station. Perhaps these zebras are the bravest of the bunch and venture furthest from the pack, zebras with a mind of their own. This protocol allows the data from the more conservative zebras to reach the researchers despite being out of range of the base station.

Another key factor is the requirement of a very long lifetime of the collars. Zebras can’t be counted on to charge the collars every night, so the collars have to be able to store all the required data and work for several months or more without intervention. To solve this issue, the collars recharge themselves using a solar array and then use that energy very efficiently. For more details, you can check out the NSF page, or this excellent article on the BBC.

Survival in the Wild

When I first read this paper, it was right on the heels of the big San Diego blackout, and I realized how useful it would be for cellphones to function in this way when power systems are interrupted. While many users lose all form of communication, some may have backup power and internet at their workplaces, or even a cellphone connection. Emergency text messages could relay through nearby users until they found a way to an actual base station, internet connection, or even the intended recipient. The message could include GPS tracking information if needed as well.

In emergency situations, this could be very useful at getting a message out to family saying you are okay or alternatively, that you need help. With the use of newer technologies like Bluetooth Low Energy, this could be done at very low power consumption, allowing your phone to work in emergency mode for a long time.

What do you think? Shouldn’t our phones be better equipped to help us during emergencies? What other applications could be helpful in emergency situations?



When SDGE and Google Powermeter provided a web interface for energy consumption back in 2010, I found myself hopelessly irritated and helpless with the lack of usable information. I could see that my power consumption had spiked a week earlier at 6pm but I had no idea what caused it. The tool provided more information than I had access to before, but just enough to be more frustrating than useful.

One unique way in which our environment can interact with us is by providing information that we are not able to access through our normal senses or abilities, an example of which is power consumption. With the increasing cost and consequences of high energy usage, we all want to reduce our footprint but how? What we need is better information to make good decisions.

Google Powermeter. Image from Mapawatt.

Google Powermeter. Image from Mapawatt.

What’s On?

Enter ElectriSense – a solution for detecting and classifying electrical devices in the home. Basically, ElectriSense is a component you would plug into one power outlet and be able to detect the switching on and off of multiple different devices throughout your home. It uses the unique pattern of noise that SMPS or Switch Mode power supplies (the brick on most of your power adapters) generate on the home’s power wiring.

Demo Interface from ElectriSense

Demo Interface from ElectriSense

This research comes from Duke University’s UbiComp Lab, from Sidhant Gupta, Matthew Reynolds and Shwetak Patel, and the implications are fascinating. A product with the ElectriSense technology and wifi could communicate with a display in your home, or even your computer or cellphone, and provide information on the current level of power consumption. It could collect usage information and allow you to look back at periods of high consumption to figure out what appliances were running at the time. My exasperating Powermeter problems would be solved! One of the most compelling features of this technology is that it requires no installation and can simply be plugged in to one power outlet – just one!

Of course, the one issue that I can see would be privacy. Many homes have one outlet outside the house and a malicious person could plug this product in that outlet to obtain not only a list of electronics within the home, but also information on when the house is likely unoccupied. However, I suppose every new technology comes with new privacy issues and since this one requires access to the house, it is limited in its scope.

If your interest has been peaked, you can read an interview with one of the publishers of this paper, Shwetak Patel, who has gone on to win the MacArthur Genius Award. He’s full of great ideas for the future of the home. In addition, Belkin has acquired his company in 2010 so watch out for some innovative ElectriSense products to hit the shelves.