In 2017 Intuitive Surgical hired Artefact to help design the interfaces for their da Vinci line of minimally invasive surgical robotics. Our initial engagement was to refine the touchscreen panels that controlled their existing devices. Our second engagement was to design the surgeon interface for a new device. This new version only required one incision rather than four and enabled new types of surgery that previously were impossible.
One problem they were having was that surgeons had a hard time understanding how the robot was oriented, and where instruments were positioned off-screen. After lying under the device I had the suggestion that perhaps they should show the robot like a rearview mirror. They said they hadn't tried it. I suggested that they could quickly prototype this by recording the view from the robot's camera with a mirror placed beneath it. We traveled home and the next day they sent us a video they shot in a mirror. I took it and mocked up this video.
The team loved the concept and we were off to a good start. I also suggested that we use an old concept by Patrick Baudisch called Halo that used circles as off-screen indicators. When I mocked up that concept, using a custom-made stereoscopic Google Cardboard prototype, I realized that changing the size of the circles independently of changing their distance in 3D space didn't have the intended effect. This was the first time I really understood that 2D and stereoscopic displays played by different rules entirely.
Through a back and forth with the product team, our team at Artefact came to this as the final design. I drove the initial interaction design, then worked with a motion designer to finish all the UI. I created the 3D scene in Cinema 4D and edited the thing together. Here's the full walkthrough of the product.
Much later the Da Vinci Single Port system came out and I have yet to see how much of our original work is still in there.
The rearview mirror made the cut for the shipping product. Tabish Mustufa reminded me that they ended up protecting the concept in the Systems and methods for displaying an instrument navigator in a teleoperational system patent.
Rapid Prototype in Shapes XR
While I had some extra time between jobs I was feeling inspired to create a template project of the surgical interface in Shapes XR. I wanted to make a scene where I could look into the surgeon's display and look inside the patient. Key concepts here are the mirrored robot view and the out-of-camera instrument indicator. It took 30 minutes to create and probably an hour to edit. Compared to the tools of 2017, Shapes XR makes this type of exploration probably 50x faster.