As I was designing the operating system for the V1 Microsoft Surface I was fascinated with the interplay of physical and digital. On the side, I teamed up with someone in the optics team who provided some clear objects that could be tracked on the Surface. I was was enamored with the idea of extending the display into 3D space. First just a simple cube. Then tubes.
Then I had a bit of an epiphany when I realized I could re-project the image from the Surface on any surface using a fiber optic grid. I worked with our industrial designer Curt Aumiller and the Microsoft model shop to create a toy whose face can become a display when placed on the Surface. I teamed up with Luis E. Cabrera to develop an application to demo the little monster.
We presented a paper titled Augmented Reality, Surface Style at SIGCHI 2010. I had been talking with Patrick Baudisch about the process, and he took it to the next level, creating his Lumino Blocks that could be tracked even while stacked. Later Disney referenced my paper and applied it to their Printed Optics to create their own monsters with Carnegie Mellon.
Much later at Artefact, I tried to resurrect the idea at hackathon week. I created a creation interface for it and partnered with an industrial designer again to create the little guy. We presented it at a panel at CES that year.
Here's my daughter trying it out. Oh memories...