Brain Surgery Utilizes 3D Imaging

A new technological advancement in brain surgery comes to Mt. Sinai Hospital in New York City.  Reporter Maddie Orton takes us into the operating room to see how one surgeon utilizes 3D imaging and augmented reality.  A warning, this story contains graphic images of the surgery process. Funding for this segment was provided by Levin Capital Strategies.


A new technological advancement in brain surgery comes to Mount Sinai Hospital in New York City.

Reporter Maddie Orton takes us into the operating room to see how one surgeon uses 3-D imaging and augmented reality.

A warning -- This story contains graphic images of the surgery process.

Lisa Galioto of Long Island, New York, was experiencing pain in her neck when she got an MRI in March 2018.

The test revealed an unrelated and shocking discovery.

Galioto had five benign tumors in her brain, one of which was so large it had reached the size of an orange.

Making matters worse, the tumor had arteries wrapped around it.

Removing a brain tumor is a challenging procedure, but Galioto's doctor, Joshua Bederson, at the Mount Sinai Hospital in New York City, has high-tech tools to help navigate these tricky surgeries.

He's one of the first brain surgeons in the country to use 3-D images of the brain integrated with augmented and virtual reality.

Augmented and virtual reality help us manage a situation like this in many ways, from the initial patient consultation all the way through the planning of the case and generally navigating towards safe quarters of surgery.

Here is how it works.

Two-D MRI and CT scans, referred to as DICOM images, are digitally fused together to create a 3-D image of the patient's brain.

Software is then used to paint outlines of the tumors, arteries and veins different colors on that 3-D image.

This creates a comprehensive map of targets and no-fly zones for the procedure ahead.

Holly Oemke researches new technology in the surgery process.

She helps Dr. Bederson implement and optimize these tools.

We have to tell the computer what pieces of the anatomy we're interested in.

So, we're using a tool called Smart Brush to actually paint on the DICOM images the tumor as a whole.

All those flow voids in there...


...those just have to go.

We've outlined the sinus, which is a vein, a major vein in the brain, and then the arteries that are either feeding the tumor or are close to the tumor that have potential to cause damage if we were to disrupt them in some way.

Inferior sagittal sinus.

For Lisa Galioto's surgery, this color-coded 3-D map of her brain helps Dr. Bederson and his team better develop their plan.

By preoperatively planning this case, we're able to visualize ahead of time what the tumor looks like before even getting into the operating room.

The patient, Galioto, is also able to visualize what the tumors look like and how Dr. Bederson plans to address them, a big benefit for laypeople about to undergo a major surgery.

It's one thing to be told by a surgeon that 'You have a tumor in right frontal lobe, and we have to make an incision over the top of your head.'

It's another thing to see your own tumor, to see it in three dimensions, to see it in relation to your own facial features and to understand exactly why we would be making an incision here and where we would be doing the opening.

Although this is still an invasive surgery, it seemed a little less invasive to me that he kind of knows where he's going.

Galioto gets rolled into the operating room.

It's in here that the technology perhaps comes most in handy.

The 3-D color-coded lines that were created to outline Galioto's tumors, veins and arteries are available to Dr. Bederson in the operating room through augmented reality.

He can view his patient's anatomy with the naked eye and then look through a microscope that provides an overlay of the digitally drawn outlines so he can see where the arteries, veins and tumors lie before he even reaches them.

This is referred to as a heads-up display.

The name originates from pilots using the technology to view information while looking ahead in flying rather than looking down to check their instruments.

Dr. Bederson says the same concept applies here.

Prior to heads-up-display capability, a surgeon's methods would've been very different.

You would've used a map.

You would have looked at the MRI scan, CT scan, gone back to the patient, looked at the patient, internalizing what you've seen on the MRI scan and trying to project that onto the patient in an accurate way.

More recently, Dr. Bederson relied on a GPS-like navigation probe.

The tip of what looks like a pen touches the patient's anatomy.

It's synced with a map of the brain to tell the surgeon where he is in real time.

This is what most surgeons currently use.

Dr. Bederson also incorporates this navigation probe into his process, but he doesn't have to rely on it solely anymore.

That's a big advance and very, very helpful, but it still requires that you stop what you're doing, look at the map and then go back to what you were doing.

In the analogy of flying a plane, you have to stop flying the plane to look at your information and then start flying the plane again.

That's no longer an issue.

Dr. Bederson says heads-up-display technology for surgery has made his work faster and safer.

To use the heads-up-display projection of the virtual-reality reconstruction onto the scalp while we're planning the skin incision so that we can position the opening precisely and give us the maximum exposure for the minimum opening.

Likewise, after the skin incision, we'll be able to position our craniotomy, which is the bone opening, right over the tumor by projecting the tumor onto the surface of the skull and sort of seeing through the skull to the tumor to more precisely outline our opening.

The heads-up display helps the rest of the team in the OR see what Dr. Bederson sees, as well, because the microscope he uses also functions as an exoscope, providing a magnified video feed to a screen complete with the overlay of color-coded outlines, viewed in 2-D or in 3-D with special glasses.

As the operation continues, the team can see the clear, solid outlines of the patient's tumors, veins and arteries despite blood flowing from the tumor, and they can anticipate where these structures will be as they move further into the brain thanks to dotted outlines that provide depth perception.

Six hours later, the surgery is a success.

Can you stand on one foot?


Then the other foot.


Today is my follow-up visit with Dr. Bederson.

I had brain surgery 2 weeks ago, and I'm feeling awesome.

And you never had any seizures, so I think that driving is okay for you now.


So that's good.

Get my independence back.

I think the technology made a big difference.

It helped us identify the important vascular structures that had been displaced and enveloped by the tumor and were at considerable risk when we reached the deep parts of the tumor removal.

So, knowing exactly where those were, being able to navigate to them and then to stay away from them once we were nearby was key in preventing a stroke.

Dr. Bederson is already excited about new technological features under development with several of the hospital's partners.

What if we could also provide haptic feedback to my surgical instruments?

We could provide auditory information.

We could assign different tissues different sounds, for example.

And with each technological advance, the hope is for more and more outcomes like Lisa Galioto's.

She's as normal as can be.

That's as good as you... Can't make someone better than normal, I don't think.