From VR to the Metaverse: How Cincinnati Children’s Hospital Is Embracing 3D Technology in Pediatric Cardiology

We use 3D technology in a variety of ways. Through our Digital Experience Technologies Program, we have 3D modeling, 3D printing, AR, and VR, and we use them for clinical use. These technologies help us to plan cases for surgeons so that they can see what they're going to do before they go into the operating room. We also use them for educational purposes, both for other healthcare providers and the patients directly

One of the main issues of a patient on the transplant list is being on the waitlist for a long time. Being on a waitlist comes with a 30% increased risk of mortality, so we want to be able to match them as quickly as possible to a donor.  

Historically, donors and recipients have been matched based on weight. There is usually a correlation between weight and organ size, but not always. That means the patient may be missing out on potential donor matches because we're not using the actual size of the organ. Also, because organ sizes aren't readily available before an operation, there's a concern that recipients might be getting too big of an organ, or something of that nature.  

We wanted to prove that size mismatch wasn't as much of a problem as it was perceived to be, by matching a patient’s actual organ size to a donor’s actual organ size. What we do is take a CT scan or an MRI and, with Materialise Mimic Innovation Suite software, we're able to create a model of the heart and get the total cardiac volume for each patient who's listed for a transplant. We have also worked with Materialise to create a total cardiac volume algorithm, by using our scan data and creating an AI model for it. 

We can also take a separate set of potentially normal patient information, normal CT scans, and get almost a population-based donor size. 

As we have the AI model for this algorithm, we are able to apply that model when matching a donor with a recipient. So, matching a donor, something that would usually take 20 to 30 minutes, can now take just 10 seconds. When a patient gets an offer for an organ from a donor, decisions need to be made very quickly, so that the organ can be accepted and transplanted as soon as possible.  

In a situation where every minute matters, this algorithm means that we avoid delays in the process. 

3D printing, to me, is just an output of the most important part of what we create, which is the digital twin of a patient's anatomy. With this ability to create an anatomic digital twin, you can then 3D print it, put it into augmented reality, put it into virtual reality, gamify it, animate it; there are so many different options available once you have that very important digital copy. 

What we’re doing now is developing a VR platform for surgeons, together with the game engine company Unity.  

Until now, most of the VR platforms that people use are based purely on visualization. What we're really trying to do is manipulate the model to plan out a surgery directly on the model itself. Then we have the surgeon, who's not used to working in a 3D space on a laptop or a desktop, work in a 3D space to plan out what the exact operation is going to be. They can then transfer that knowledge over to the OR to plan more complex cases.  

We’ve done this for hundreds of cases now, and it has been extremely helpful for patients with complex congenital heart disease. 

We use 3D-printed models to help patients learn more about their anatomy. We have setups where we either print models for patients and families, and give them to the kids.  

In addition, we have setups where we do virtual reality. And then the kids will come into the VR space, often with their surgeon, and then be able to look at the operation together. Parents are obviously involved in that process. This is one of the newer ventures that we've recently made possible. 

I have also created a character called Hank, which stands for Heart, Activity, Nutrition, and Kindness. This character is one we use to teach kids about the heart. We created a children's book and animations with Hank, and now that I've learned 3D printing skills with Materialise, we even produce 3D-printed Hank figurines.   

I think the biggest challenge right now is adoption. It’s like we've seen with other technologies — it takes time for people to become comfortable with them. I think there's always a big hype around this stuff, but once you validate the benefits you can find the value. And that's another thing where we've worked with Materialise for a long time. Overall, I think that showing how digital twin anatomical modeling is hugely beneficial during surgical planning is what we need to emphasize. 

A lot of people are talking about the Metaverse. The Metaverse needs a lot of work and motivated people to really get it to that serious use case, but we’ve done a lot of work on our side.  

We're partnering with Unity to build a metaverse setup, and I think it's going to involve a lot of different technology partners, including Materialise.  

For us, the purpose of creating a metaverse would be for planning operations. This especially includes any sort of complex operation that may involve collaboration between different surgeons. For example, our surgeon in Cincinnati may want to plan together with a surgeon in Japan. In the Metaverse, they can get into that same space and really talk about the challenges of the procedure, and even start to plan out what the operation would be.  

And then from there, I think another benefit is having translation capability, so people can speak in their native language, and it can be translated in the Metaverse space. There are so many potential things to do with 3D technology — we just have to really work on refining it and defining what we're going to do.