Surgical Augmented Reality: Bringing the Vision Pro to the OR
Not long ago, 5–7 years ago, augmented reality (AR) was a cool trick but not much more. It was most often relegated to trade show one-off experiences, a few ambitious mobile games that fell somewhat flat, and several high-profile product demos that were cool but not practical for mass adoption. As a 10 year AR veteran and founder of augmented reality development company DAS Labs, I’ve seen significant evolution within the industry since that time, including advancements in MR for healthcare.
The Background of AR, MR, and VR
By now many people have heard of the terms AR, MR (mixed reality), and the most recognizable VR (virtual reality), but might not be familiar with the meaning of each abbreviation or the technology’s most valuable applications. For this article, I’ll use AR and MR interchangeably, defined as a program that adds virtual content and a computer interface to a user’s field of view in a useful or novel way. Gaming and entertainment tend to get attention from the masses, but we’re seeing AR’s potential flourish in enterprise environments — like operating rooms and labs.
About eXeX
eXeX is revolutionizing the operating room. Surgical teams often face inefficiency in the OR setup due to human factors like increased procedural complexity and high surgical team turnover. eXeX set out to solve this problem with a sophisticated augmented reality and artificial intelligence platform. The solution is novel because it focuses on enhancing the entire surgical team’s performance and flow state — not just the surgeons. The platform provides instructions for efficient OR tool and table setup, visually assists the team through customized intra-operational sequential lists of tool and procedure prompts, and offers data to assist with post operation reports.
Apple’s Vision Pro in Surgical Procedures
In the first week of February, Apple released the long-awaited first iteration of their Spatial Computing (ehem, AR/MR) headset. While everyone was posting on social media Vision Pro headset unboxing videos, eXeX announced a historical first surgery done on the recently released Vision Pro.
An Interview with the Founders of AR Surgical Innovator eXeX
I had the pleasure of speaking with the founders of eXeX to shed light on surgical innovations. eXeX was founded by a group of passionate futurists that have been leaders in the fields of neurosurgery, healthcare, and immersive technology. CEO Dr. Robert Masson has built and managed an internationally recognized spine surgery institute, two surgical facilities, developed several spinal implant systems, innovated procedures and technology, and has been a leader in his field for more than 30 years.
1.) What problem/problems is your AR/AI suite of products solving?
A: Over the last several decades, surgical procedures have grown more complex, but surgical organization has remained unchanged, leading to increasing medical errors, waste, and cost. eXeX is solving Surgical Disorganization.
Evidence of Surgical Disorganization include:
• Medical Errors estimated to cost over $20 Billion each year.
• Up to 54% of Nurses and Physicians suffer from burnout.
• Nearly $1,000 in unused surgical supplies are wasted each procedure.
The implications of Surgical Disorganization include poor surgical team performance, low moral and eventually high turnover rates which further exacerbates team performance. Facilities lose money and are inefficient with their use of supplies and time. If left unmitigated, the implications of Surgical Disorganization eventually risk patient safety, recovery, and overall experience.
2.) How has eXeX improved surgical outcomes for procedures in which it has been used?
A: In a series of clinical tests, our system showed a 39% improvement on accuracy for surgical inventory setup, a 50% more improvement on net efficiency for operating room setup, and a 53% improvement on net efficiency for completing surgical procedural steps.
eXeX officially went live in multiple operating rooms, across multiple continents in 2023. Our system was used in over 1,000 surgical procedures in 2023.
A list of participating operating rooms using our system can be found here: https://exex.ai/explore/
3.) What technologies are you currently using?
A: We develop products that utilize Artificial Intelligence (AI) and Mixed Reality (MR).
We use AI to optimize the surgical setup and workflow, and we use MR to deliver these optimized guides to the technicians within the sterile field of the operating room. While our system is hardware agnostic, On the Mixed Reality side we use the Hololens 2 and now are using Apple’s Vision Pro headsets.
Our AI System is trained on a closed set of data that we gather from our surgeons and surgical procedures. This is part of our commitment to build a “Safe” AI system. We do not gather, hold, or use any patient information or patient data. Our AI System only works with surgical data, such as the tools, equipment, and their layout and orientation within a specific OR. Each procedure that uses AI is human checked and approved before being deployed. This is also part of our commitment to build a “Safe” AI System.
Our AI System learns and grows from each completed procedure, and gets better with time.
4.) How does eXeX enable surgical teams to perform in a flow state?
A: Our value proposition to a facility and surgical team is an increase in efficiency, flow state, and organization.
Viewer X is an iOS application — allowing the Nurses, OR Managers, and Scrub Technicians a powerful and simple way to organize and set up a surgical procedure visually using Apple iPads. With AI optimized Surgical Steps, Room Orientation, and Tool Setup, the application delivers everything the team needs to accurately and efficiently prepare and set up the procedure.
Experience X is a Mixed Reality application — allowing the Scrub Technicians a touch-free heads up display, offering interactive capabilities from within the sterile field of the operating room. The system offers Live Consumable Tracking, Tool Placement, Room Orientation, and also both Pre-operative and Intra-operative Step by Step Guides for accurately and efficiently setting up and completing the procedure.
We look forward to seeing how AR further advances the healthcare industry over the coming years, and DAS Labs is thrilled to be at the heart of AR development.
