We’re at one of those inflection points in the history of technology. Things that have been bouncing around academia and corporate R&D labs for decades have finally become inexpensive enough to bring experiences to consumers that were previously inaccessible. What’s more, a combination of these technologies has the potential to upgrade an experience all of us have every day. It’s something that hasn’t changed since the advent of the Macintosh and the debut of Windows: the mundane business of getting things done on our computers.
To kick off this trip down the rabbit hole, refer to this gem from Oliver Kreylos, a researcher at UC Davis who has been working on immersive virtual environments for more than a decade. What you’ll see in the following video is a combination of three different technologies, all originally intended for gaming, but here neatly repurposed for “real” work: A set of Oculus Rift goggles to project a 3D environment directly into Kreylos’s eyes and track the orientation of his head, two Razer Hydra joysticks to track the movement of his hands and give him some buttons to press, and a Kinect motion tracking device for a purpose I don’t want to give away in advance. (But skip to 3:10 in the video if you must.) Trust me when I say that you’ll need to watch the first 60 seconds of that video, at least, for the rest of this piece to make any sense.
One of the magical things about Kreylos’s deadpan delivery is the utter nonchalance with which he shows off things like menus that appear wherever he needs them to in the three dimensional space surrounding his head. He’s showing off technology that is mind-blowing in its departure from what most of us are used to thinking about as viable ways to interact with a computer, but to him it’s hum-drum because he’s been “building holodecks for a living” since before the first iPhone woke us all up to the possibility that windows and keyboards might be just one of many ways to productively interact with a computer.
But what about actually making something with this 3D interface? In his second video, Kreylos shows off a test he’s used often in the past in order to determine just how usable a 3D interface is: building a molecule known as a Buckminsterfullerene (a ball made of carbon atoms). Kreylos says that this task can take as long as 45 minutes with a keyboard and a mouse, but a good 3D environment allows a person to do it in under two minutes.
In order to make these videos, Kreylos used about $500 worth of electronics, not including the price of his PC. That’s well within the comfort range of consumers who are already buying tablets and spending hundreds a month on their wireless plans, and once systems like this are integrated into the PCs we use every day, that price is sure to fall until we start to think of the peripherals required to interface with virtual environments as we do keyboards and mice. The fact that electronics that inexpensive yield results that were previously only available in systems that cost 10 or 100 times as much is the reason we’re at an inflection point in the adoption of this technology.
Once they’re in wide adoption, it’s likely that 3D interfaces of the future won’t look exactly like this one. Indeed, we are certain to see a diversity of approaches. It would be possible, for example, to replace the Razer Hydra joysticks (and their successors) used in these videos with Playstation Move controllers, which are a quarter of the cost. Or engineers could create a system that requires nothing but our hands, by incorporating the Leap Motion controller, which some PC makers are already integrating into their notebooks. The resolution on the second-generation Kinect from Microsoft may also be high enough to allow control with nothing but our hands.
So-called “augmented reality” isn’t yet ready for prime time, but eventually displays no more obtrusive than eyeglasses will be projecting these 3D interfaces into the world around us, eliminating the need for bulky, opaque headsets. Or we’ll just project them into meatspace, as Kreylos demonstrates here, with a twist on that old playground favorite, the sandbox.
In the meantime, as Kreylos demonstrates in the first video in this piece, we can always hack our way around the shortcomings of headsets that block our vision by using a 3D video camera like Kinect to selectively project reality into our otherwise visually sealed virtual environment. One thing that’s on the horizon but isn’t likely to be accessible to consumers anytime soon are various means of providing people with physical feedback as they interact with otherwise ethereal virtual environments.
If you want to read more about Kreylos’ work, it’s worth checking out his ongoing series of blog posts on the Oculus Rift and the challenges of virtual reality in general. He’s a gifted communicator.