More Sci-Fi UI


by Tim Moran

I recently wrote a little piece about a gestural-computing endeavor called Toscanini, described by one of its creators as a "bridge between movement and digital instruments." Something like Microsoft's Kinect, but for making performance art rather than playing Xbox, or the use of a gesticular interactive UI in the movie, "Minority Report."

So tonight I'm doing my usual read of the web, and I stumble on news of a hack that makes the user appear invisible on Kinect's cameras. One Takayuki Fukatsu has used OpenFrameworks -- an open source C++ toolkit -- to develop the hack. Apparently, because Kinect can distinguish between a human and the background, it's possible to overlay the background on the image of the user, which renders that human image somewhat invisible. There's video at the link above; it's not Harry Potter "invisibility cloak" invisible, but it's still pretty cool.

Then I recalled that I had seen a video a week or so ago about another Kinect hack. A quick YouTube search turned up the one I was thinking of: Using Kinect to Capture Video in 3D. The description explains that, "by combining the color and the depth image captured by the Microsoft Kinect, one can project the color image back out into space and create a 'holographic' representation of the persons or objects that were captured." While it's a rather interesting idea, the actual result, shown in the video, is not all that exciting.

You must be asking by now, how does all this connect Kinect to Toscanini? Well, as I scanned the results page of "Kinect hacks" videos, this caught my eye: "Kinect hack lets you control a web browser and Windows 7 using only The Force." Basically, this bunch of guys got together to make JavaScript interact with Kinect to navigate Web pages.

In this case, the video is actually much better than the description is. Shown is someone's hand opening and closing and moving up and down in front of a screen on which are various Web pages. The gestures allow the navigation of open tabs; selection of pages and scrolling through them; and panning and zooming.

One commenter says, simply: "Minority report!" We're getting there.