Paper Reading #11: Combining multiple depth cameras and projectors for interactions on, above and between surfaces

Comments:

TBD

Reference Information:

Title: Combining multiple depth cameras and projectors for interactions on, above and between surfaces

Authors: Andrew D. Wilson, Hrvoje Benko, Microsoft Research, Redmond, WA, USA

Presentation Venue: UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology

Summary:

LightSpace is a small room where various activities related to interaction with computers can be studied. The room is equipped with a number of cameras and 3D projectors that can display geometry on any surface that is visible to both the cameras and the projectors. The internal area is tracked in 3D space in real time. It is also possible to utilize depth information to simulate interaction with virtual objects that are not attached to any virtual surface. Another feature is the ability to transfer an interactive display by simultaneously touching an active and inactive display surface.

This prototype system was demonstrated before an audience of more than 800 people. The researchers were able to gain a lot of feedback about the people's perceptions about it. One thing they have discovered is that with the size of the current prototype, it is best suited for a maximum of 6 people interacting with it. This is one of the improvements they hope to achieve with future iterations. Another improvement they intend to make is to integrate the system with a real-time physics engine to give a more accurate sense to certain features, such as picking up objects.

Discussion:

If I had to express the way this paper made me feel in one word, that word would be "thisthingistotallyfreakingawesome!" Being something of a science fiction geek and enjoying the occasional trip into the world of Star Trek (read: not a hard core Trekker, but like watching the series...), this system goes a great deal of the way to being able to create the HoloDeck. Ever since the Next Generation series came out, I have wanted to be able to interact in something resembling the HoloDeck, and this seems like almost the next best thing.

With further advances and system modifications, I can see how this would make a great training tool for any  industry, including the military. I see the possibility of creating an interactive television setup where you not only can choose to passively watch the "main" story created by the designers, but pick branching paths like a visual "Choose Your Own Adventure" book. It would also be impressive to be surrounded by the action. Not only could you hear everything in surround sound, but you can be immersed in the display, as a part of it. Actors could practice their art by taking one of the characters roles and interacting with the other characters being projected into the scene.

Now we just need to figure out the holography part....