Microsoft announced the latest edition of its HoloLens mixed reality headset at MWC Barcelona on Sunday, promising vast improvements over the previous device released four years ago. The HoloLens team is led by Alex Kipman, a Microsoft veteran who has been with the company for more than 17 years and, before joining HoloLens, worked to develop the Kinect for Xbox.
The biggest change for the second iteration of the headset is the wider field of view. The HoloLens team has also refined the gesture control and greatly increased the device’s eye-tracking capabilities. Digital text, floating in augmented reality, now automatically scrolls based on how fast the user is reading. A surprising challenge, Kipman says, was figuring out the right latency for eye tracking, which is the amount of time between the HoloLens seeing where your eyes are looking and it acting on this information. The HoloLens is actually faster than the human brain to process what you’re looking at, which the team realized only after testing the new functionality.
The HoloLens 2 is pitched towards business customers, and Microsoft is partnering with engineering, health care, and manufacturing companies for the initial rollout of the device.
Quartz sat down with Kipman at MWC Barcelona today (Feb. 25) to get his insights on how the device was designed and how it might be used in everyday situations.
I’m curious how you use HoloLens in your day-to-day life. Do you use it as a productivity tool? Do you use it for design?
I use it all the time because I created, right? I’m not your normal person out there. But we do use it for two scenarios at work, as if we didn’t work on it. One is actually to create HoloLens.
All of our 3D models of HoloLens are obviously done in 3D like anybody else’s. When we do reviews of these things, like I’m not going to go pull a flat monitor to go look at a 3D [model].
The other one is communication. To talk to people from around the globe, HoloLen is a very comfortable device to do that.
I haven’t had enough time yet with this one and I may just, you know, start using [it] as my productivity device, as in like, just why have a monitor? I had a friend, one of our engineers has been here, that took it to his hotel. And it’s like, man, laptop in my hotel versus using a HoloLens over Bluetooth keyboard and mouse. And he was enjoying it. So this [HoloLens] is immersive and comfortable enough that I think you’ll start seeing me and people on the team during that more.
How did the design change between the first and second version of the device?
In the previous version of HoloLens, a lot of the constraints we had more around how do I fit with the single product over the population of human head sizes, across genders, across ethnicities, across the ages, over glasses, without glasses to really be comfortable to the humans.
So we added essentially the floating ring [the headband of the previous device] as a concept with the hardware around it because we wanted you to have the freedom to move it in six degrees of freedom. Good idea, but it turns out that it was complex and a lot of people took a long time to get to the right spot.
The motion here we wanted to hit was, it has to be as easy as putting on a hat. I want it to be instinctual. I don’t want it to be a learned behavior. So when we set out to build HoloLens 2 the theme was it has to be on like a hat.
At the end of the day, comfort, like field of view, like immersion, is many variables. Weight is one component, but it’s not the most important one. The most important one is where’s the center of gravity of the device. So we’re like, look, we need to shift this back. HoloLens 1 is 11 millimeters from your eye, and HoloLens 2 is 70 millimeters. So it goes from the weight being [in front of your face] to being behind your ear, which makes the device kind of float.
Was there anything that surprised you when researching human-machine interaction for this version of the device?
I’ve been doing this now since 2008, so 11 years into it, anytime you try to [understand] people, places or things, it’s generally a game of trying to erase latency. You’re running compute, which takes time. You’re throwing photons out, reading those photons, getting the data, parsing it. It just takes time, and the human wants to just push a button and have it pushed.
With HoloLens 2, this is the first time I’ve had to introduce latency [more time between an eye movement and an action from the HoloLens]. And that was with eye tracking. In the beginning, none of us could get it right, and we’re like, how could this be that we’ve written this wrong? Like it was unusable.
A surprise. The reason being because I [the HoloLens], for the first time, have better data than a human. I knew where your eyes are gazing before you process it. Because now your eyes are just the sensors that also have latency before your brain can perceive it. So as you’re comfortably starting to already gaze down and your brain’s processing while you’re reading, I’m like, I know I need to scroll because I’m ahead of you.
So 11 years of trying to erase latency, with eye tracking we had to introduce latency to catch up. And once you get it, it’s magical.
This interview has been condensed and edited for clarity.