It seems ushering in “a complete shift in visual computing” looks a lot simpler up close.
New research from secretive augmented-reality startup Magic Leap shows the company has been working on an AI-based approach to estimating the size and shape of rooms, called RoomNet. The paper, written by the company’s director of deep learning and three engineers and released on the open publishing repository ArXiv, says the task is important for indoor navigation, scene reconstruction, and augmented reality.
Magic Leap’s work specifically focuses on finding the layout of a room from regular images, ones that any camera would provide. The paper identifies 10 different ways a camera can view a room, identifies “keypoints” or indicators of the edges between walls, ceilings, and floors, and then describes a deep neural network to identify these keypoints in new rooms.
Augmented reality, software that places digital items alongside real ones using a smartphone camera and screen or special glasses, depends on being able to correctly perceive the device’s surroundings. Companies such as Apple, Microsoft, HTC, and Google have all invested time and energy into similar projects for their respective AR and VR pursuits, each developing its own proprietary techniques for software that understands the physical world. Magic Leap has published work concerning this idea before, particularly on the ability to track points within a 3D space. Putting the two together, you could have the ability to place a digital object in a 3D space and then keep it in the confines of a room—so the whale in the Magic Leap demo video (a visualization of what the company hopes to one day achieve, not a real demo) could realistically look like it was bursting through the floor.
Unrelated to this research, Magic Leap was awarded a patent yesterday for “a compact optical see-through head-mounted display capable of combining, a see-through image path with a virtual image path,” where virtual objects can actually obstruct real ones.
Rony Abovitz, CEO and cofounder of the company, invited developers to work with the company’s headset in May, and has hinted at a 2017 unveiling.