Skip to navigationSkip to content
Sponsor Content By Accenture

From remote surgery to X-ray vision, doctors are showing other industries how to use AR/VR

By Accenture

The average consumer has only recently woken up to the mainstream applications of virtual and augmented reality. Doctors, on the other hand, have embraced both technologies—collectively known as “XR” or “extended reality”—for almost two decades: The first successful remote surgery occurred in 2001, when a New York surgeon used remote robotic controls and 360-degree screens to perform cholecystectomy on a woman in France.  

The theoretical and actual applications of XR for healthcare professionals have only expanded since. In 2016, Doctor Shafi Ahmed live-streamed a surgery in 360 degrees with AR glasses, providing 13,000 students and clinicians a direct POV learning opportunity via an app, which can be viewed in stereoscope by slotting your phone into a $15 cardboard headset.

XR collapses distance — between information, people, experiences

Despite the many technical challenges that currently limit XR’s widespread adoption, the technology’s long-term potential is still clear in many sectors, particularly healthcare. Thanks to the technology’s applications in surgery, diagnostic imaging, patient care, and medical education, virtual and augmented reality could represent a $5.1 billion opportunity for the healthcare industry by 2025. That’s second only to the gaming industry.

The exciting part? The use cases go beyond healthcare

Just as surgeons can better understand their surgical plans and patients with XR, engineers, architects, and designers can more thoroughly examine their envisioned product designs. XR is improving digital collaboration and enhancing knowledge access and retention across fields.

XR’s various applications in healthcare are a helpful signal for other industries looking to integrate VR and AR platforms into workflows. And by examining each, we can see a critical subsection of use cases for all industries considering XR.

Bionic sight — from X-ray to sonic vision

XR gives doctors X-ray vision—well, sort of. By donning an AR headset, a surgeon can see CT and MRI scans of patients’ internal anatomy projected directly onto their bodies. Scopsis has designed an AR projection system that’s helped spinal surgeons track the position of patients’ vertebrae in over 10,000 operations. The system can also label body parts beforehand, creating virtual guidelines, and even see live virtual information like vitals, keeping hands free and reducing risk of contamination. And since surgeons can see patient anatomy without creating any openings, lots of procedures can be done externally without creating additional incisions that would increase risk of complications and prolong recovery.

How about infrared or sonic vision? AccuVein employs infrared lasers to detect and project vein structure onto skin, making it three times more likely that physicians will find a vein on the first stick.  Likewise, several AR companies have developed smart glasses that can see ultrasound.

There’s also AR available for first aid and emergency response. AED4EU provides digital locations of heart-restarting defibrillators or AEDs through your phone’s camera. This geolocation of digital information has extensive potential in other sectors—for example, for projecting price information in stores or engineering specs in factories.

Streamable surgeons

Globally, five billion people lack access to safe surgical care. In Sierra Leone, for instance, there’s only one surgeon per 600,000 people, leaving many in the country with a tough set of choices: wait, travel, or fail to have an operation at all. Augmented reality, however, offers a new possibility, explains Nadine Hachach-Haram, founder of Proximie, which builds AR tech to train surgeons. “As long as you have a smartphone… a competent colleague on the ground, and an AR [headset], an expert surgeon can remotely insert herself into any operation, and can interact with a local doctor, guiding and mentoring step-by-step,” explains Hachach-Haram. Proximie’s AR platform coordinates collaboration among physicians across cities and continents.

Empathic immersion

Beyond its ability to let users remotely experience foreign countries and sporting events, virtual reality technology is also unlocking new ways to create empathetic connections between doctors and patients, particularly those dealing with pain and recovery. In one study, VR halved use of sedatives like fentanyl, reducing surgery costs by around 25%. Massachusetts-based startup Brain Power has turned to Google Glass to help children with autism, teaching them social skills, language, and positive behavior. Other similar applications have included using VR to help people with Parkinson’s and multiple sclerosis regain the use of their legs and the  treatment of people diagnosed with post-traumatic stress disorder.

VR is being used as a tool for building empathy. Embodied Labs created “We Are Alfred,” a VR experience that shows young medical students what ageing means by immersing them in the perspective of a 74-year-old man with audio visual impairments. The developers’ goal is to solve the disconnection between young doctors and elderly patients due to their age difference. Fostering empathy between caretakers and their charges is much easier when physicians can see things from the patients’ perspectives.

Virtualized education

According to Steve Dann, who co-founded Medical Realities alongside Dr. Ahmed, comprehension and information retention rate when using VR in education runs as high as 80% (against a standard 20%). Dr. Ahmed lectures 300 students at Royal London and St Bartholomew’s hospitals, but can only bring six students into any given operation. The first live-streamed surgery he led reached 55,000 people in 142 countries. In partnership with London universities, Medical Realities is constructing virtual courses from 360 video feeds, including from cameras placed inside bodies.

The greater efficacy and reach of virtualized education is why many medical schools are investing in dedicated XR centers and partnerships. Recently, Case Western began piloting a virtual lab, which allows students to visualize a human body in virtual space, expanding and exploring anatomical layers without dissection or expensive chemically laden cadavers. AR can also be combined with haptic feedback, allowing students to “feel” virtual anatomy.

In 2017, medical drama The Good Doctor faced backlash after an episode depicted surgeons rehearsing an operation in VR. The scene, while deemed implausible by many fans, was, in fact, accurate. In May 2017, pediatric surgeons at the University of Minnesota consulted VR simulations before separating conjoined siblings. The surgeons explored the infants’ walnut-sized hearts by expanding them to the size of a living room. They quickly discovered connective tissue between the hearts, completely changing their original surgical plan and leading to a successful surgery.

XR simulation is also being used for retail employees rehearsing for Black Friday crowds, quarterbacks practicing blitzes, and soldiers simulating combat. XR eliminates distance not just between student and teacher, but also concept and practice.

To view more trends shaping the future of business, read Accenture’s Tech Vision 2018 report.

This article was written on behalf of Accenture by Quartz Creative and not by the Quartz editorial staff.

Subscribe to the Daily Brief, our morning email with news and insights you need to understand our changing world.