Saving lives with VR

It’s news no parent wants to hear.

Three days after their daughter Annika was born, Allison and Adam Seed learned their baby girl had multiple heart defects, including a rare combination of defects called “tetralogy of fallot with pulmonary atresia,” which affects blood flow and oxygenation. Three days later, Annika had the first of several risky open heart surgeries at Boston Children’s Hospital.

Fortunately for the Seeds, Annika’s physicians were some of the best in the business.  To address her life-threatening condition, Annika would undergo six surgeries by the time she turned 2 years old. By the age of 6, she’d had three open-heart surgeries and five cardiac catheterizations, a trial-and-error process that allowed the surgeons to learn what would be best for Annika as they observed her progress.

While their daughter’s devastating diagnosis left her parents with what they describe as the “complete feeling of powerlessness,” the aggressive treatment Annika received over those two years at Boston Children’s Hospital has given them hope that Annika will be one of the lucky ones to live a full life that most of us take for granted.

The Seeds were fortunate to live near Boston, where their doctor and his team at Boston Children’s Hospital had access to the best technology, which now includes access to a secret weapon: The Living Heart Project. For patients like Annika, the cardiac care team now uses realistic digital heart simulations, a virtual twin of their heart, customized to their specific physiology, to plan and practice surgeries that will yield precise and predictable results.

Founded in 2014, the Living Heart Project developed the first, fully functioning 3D virtual heart that could be customized for a patient’s specific physiology, allowing surgeons to test and validate their treatment plans before they operated, creating a unified team and greatly increasing their chances of success.

The simulations also give families more confidence and ownership in decision-making. The sense of powerlessness that the Seed’s felt is no longer necessary as they can see in vivid 3D the internal structures and vessels in their child’s body, understanding what isn’t working and what the surgeons plan to do to improve her condition. With the help of the Living Heart, both the medical care team and the parents are able to understand the personalized surgical intervention, increasing everyone’s confidence in the chosen course of action. 

“It was mind blowing for us to see the possibilities, not only to have more ownership as part of the care team as parents, but to see this technology at work in the hospitals,” Allison Seed said. “It filled us with so much hope for our daughter.”

Today, Annika Seed is a healthy teenager. Her mother now works for the American Heart Association, promoting the role of technology in pediatric heart health. And the breakthrough technology used to create the Living Heart is now being applied to modeling even more organs … and, eventually, to the entire human body.

Using virtual twins, from a surgeon’s perspective

For Dr. David Hoganson, a pediatric cardiac surgeon at Boston Children’s Hospital and Harvard Medical School, utilizing virtual twins helps him plan and perform anatomic repair in the most complex cases, like Annika Seed’s. The 3D models and simulations reduce variability and improve outcomes in pediatric cardiac surgery.

“The highest risk patients require meticulous planning that goes beyond experience,” Hoganson said. “Better planning equates to higher margins at every level, which then translates to better [clinical] outcomes. … [With this technology, we can] make things that we can control as perfect as possible.” 

Hoganson and his team at Boston Children’s Hospital were early adopters and partners in this leading-edge technology. With each success story, confidence and evidence that the technology works grows. As their supporting data and peer-reviewed papers are published, the Living Heart becomes accepted and adopted more widely – eventually, potentially, becoming the standard of care.

How virtual twins came to medicine

For years, virtual twins have been applied to various industries: aerospace, manufacturing, consumer packaged goods and retail. But how did they come to be applied to humans in the life sciences and healthcare industries?

As it did for the Seeds, the story started with news no parent wants to hear.

Steve Levine’s daughter was born with a congenital heart disease. Her left and right ventricles were reversed, disrupting the heart’s natural functions and increasing the likelihood the weaker ventricle will fail as she ages. She also had a one-in-a-million “congenitally corrected” heart, meaning her heart adapted to the changes in the womb. These changes posed a mystery to physicians as they hoped to help her live as long as possible.

The first failure was her electrical system, requiring a pacemaker at the age of 2, much earlier than doctors expected. Typically, such life-threatening complications arise around 30, suggesting her case and her future was unclear.  Driven as any parent would be to do whatever he could, years before she would reach that milestone, Levine dedicated himself – and his skills – to helping save not only his daughter’s life, but the lives of all patients with unusual heart structures.

Levine, an engineer by training, had spent years helping design extremely detailed 3D crash simulations for the automotive and aerospace industries. He thought: Why can’t we use similar technology in the medical field? If this technology could be used to build a virtual model of the human heart, could it provide insights into medical decision making – the same way automotive and aerospace companies learn from vehicle simulations?

As the chief strategy officer for a division at Dassault Systèmes, Levine was inspired by his company’s goal to not only help develop safe and efficient cars and planes, but to help build a more sustainable world, including the people in it. With that in mind, he launched the Living Heart Project in 2014, when his daughter was in her mid-20s. The project’s ambitious vision? To “advance the development of safe and effective cardiovascular products and treatments by uniting engineering, scientific and biomedical expertise to translate cutting-edge science into improved patient care.” It meant bringing experts from around the world to work together and not only achieve what had never been done before, but to make it available to doctors everywhere.

To build the “living” heart, work started with MRI and CT imaging scans of donor hearts, which were reconstructed and enhanced with new muscle tissue models and detailed fiber orientations in scientifically accurate 3D. The heart would need a detailed electrical system, fully functioning valves, fluid chambers and the other details of this complex organ that nature had perfected over millions of years of bioengineering. Finally, a circulatory system model for blood flow was created to place the heart within a realistic human body at rest, under stress etc. Together, these elements combined to create a scientifically accurate computer model of a functioning heart. Because it works from the same principles as a real heart, the resulting 3D simulation can help all stakeholders – from patients and families to cardiologists, radiologists and surgeons – communicate more effectively, while helping researchers develop new treatments and surgeons plan, test and practice complex medical procedures. 

The project was a success. “We rely on simulation to transform the world around us,” Levine said. “Now is the time to transform the world within us. We can now create virtual twins of the human body to visualize, test, understand and predict what cannot be seen – from the way drugs affect a disease to complex surgical outcomes – before the patient is treated. The virtual twin can be built to represent most of us, and then be personalized for each.” 

What’s next for virtual twins in complex medical procedures?

Now in its 8th year, the Living Heart project has grown from changing lives to changing the industry. Today, the project has support from the US Food and Drug Administration (FDA) and more than 175 member organizations in 24 countries. Hospitals use the models to plan and predict surgical outcomes, companies use to design new therapies or replace testing on animals and the FDA is evaluating it as a substitute for people in clinical trials. Its success has led to sister projects: the Living Brain and Living Lung, and Liver as well as complete circulatory and musculo-skeletal system models for improved orthopedic treatments.

Now, cases of using virtual reality, artificial intelligence, 3D imaging and virtual simulations to dramatically improve medicine are occurring around the world. Recently, surgical teams in Brazil and the United Kingdom worked together using VR technology to safely and successfully separate conjoined twins connected at the brain. At the H. Hartmann Institute for Radiosurgery and Radiotherapy in France, the VORTHEx project lets patients experience their treatment beforehand, in virtual reality; through a twin of radiotherapy sessions, patients mentally prepare for what they will experience. And research organizations and healthcare providers around the world are beginning to develop, test and use technologies for virtual twins of not just the heart and brain, but lungs, skin and, eventually, the entire human body.

“The heart project proved that if we work together, we can systematically understand and reproduce how the body works. The next step is to diversify and build all the key organs and body systems, like the brain, the liver, kidneys and the lungs,” Levine said. “The work on these virtual twins is not as advanced as the heart, but that will happen much more efficiently using the template that was created by the Living Heart.” 

Combining virtual twins with other technologies, like 3D printing, creates even more opportunities for complex training and patient-specific solutions. French-American startup Biomodex, for example, creates biorealistic haptic simulators for physicians to use in rehearsing complex medical procedures such as brain aneurysm surgeries. The technology is also used to advance surgeon training, leading to safer procedures and improved outcomes.   

“Using the models for pre-procedural rehearsals allows us to know in advance if an approach is going to work and, if not, helps us to determine which medical device would be more effective,” said Dr. Anthony Le Bras, an interventional neuroradiologist at CHU Rennes who has used Biomodex simulators. “This not only boosts physician confidence, but also reduces operating times and decreases the risk of complications during the real procedure.”

The use of virtual twins in complex medical procedures is changing – and saving – lives today. These sophisticated technologies are poised to further transform the healthcare ecosystem by incorporating virtual modeling and simulation into regulatory frameworks, medical device development, delivery of personalized healthcare and medical education for patients and caregivers. 

As the healthcare industry embraces the use of virtual twins, innovators around the world will continue to transform the way the medical profession gains knowledge, shares experience, treats patients and safely tackles the unsolved mysteries in medicine.  

And Levine’s daughter? She is now 33, an MD/PhD practicing pediatric neuroscientist, waiting for the Living Brain to help her solve the toughest challenges of neurological disorders in children … and her heart is bigger and stronger than ever.

Learn more about DassaultSystèmes’ solutions for the Life Sciences and Healthcare industry.