Hacking the Microsoft Kinect for the Real World

Everyone’s heard of the Microsoft Kinect, the gaming technology which cause a huge stir when it launched in 2010. In fact, in 2011 the Kinect broke the Guinness World record for being the “fastest selling consumer electronics device”, selling 8 million units in its first 60 days.

The Kinect features color and depth cameras that can detect the user’s body and limb positions, without the need for a physical controller. The Kinect also contains an array of microphones which enable voice control in addition to gesture control. In other words, the Kinect can “see” and “hear” you, without your ever having to touch an interface; it’s a Natural User Interface or NUI.

Naturally, this means sports and dance games are very popular in the Kinect line-up, but there’s a whole other line-up that fewer people know about – innovative real world applications that take advantage of the hardware of the Kinect. It’s not that this technology has not been around, but until now it has not been as cheap, robust and accessible as it is now.

Hacker Bounty!

After Adafruit, an open source electronics advocate, offered a bounty for them, open-source drivers for the Kinect showed up mere days after the product release. Since then, enterprising hackers and startups have been coming up with their own non-gaming applications for the Kinect.

Tedesys, in Cantebria, Spain is developing an interface that helps surgeons during long procedures. Normally, to access necessary medical information during the procedure, the surgeon must leave the sterile environment to use a computer and then scrub back in to the OR. Tedesys’ interface allows doctors to navigate the medical information they need using gesture and voice control, without contaminating the sterile environment.

At the Royal Berkshire Benefit Hospital in the UK, doctors are using the Kinect to make rehabilitation therapy for stroke victims less frustrating. There are no complex controls to worry about and simple games make the rehabilitation exercises enjoyable. The system  improves their strength, control and mobility, as well as tracking their improvement over time.

At the Lakeside Center for Autism in Issaquah, Wash., staff are using the Kinect to help children with Autism work on skill-building, social interactions and motor planning. You can read more about these uses at Kinect Effect.

Microsoft’s Reaction

Until recently, it appeared Microsoft had decided mostly to look the other way as people used their hardware with the non-official hacked drivers. But earlier this year, in a surprising about face, Microsoft decided to jump back into the game by releasing a new version of their technology, called Kinect for Windows, specifically designed for PC applications, with a free SDK and drivers. They also decided to motivate and assist startups using their technology by providing 10 promising finalists with $20,000 to innovate with the Kinect.

Some new startups that were accelerated by this program are :

• übi interactive can turn any surface into a touchscreen.
• Ikkos Training aims to use theories of neuroplasticity to train athletes and improve their performance.
• GestSure allows surgeons to navigate medical data in the OR, as described earlier.
• Styku creates a virtual “smart” fitting room for retailers.

Frankly, I find these latest uses less exciting than I hoped, but perhaps now that Microsoft has made the Kinect a commercially viable interface, startups will be encouraged to bring their ideas to life. I really do believe that this technology is game-changing; Instead of all our interactions with technology being funneled through physical interfaces like keyboards and mice, we can use our full range of motion in intuitive and ergonomic ways. Once the idea of NUIs really starts to permeate the social consciousness, we will see many innovative uses of this technology.