SciTech Now Episode 303

In this episode of SciTech Now, augmented reality’s sudden fame and applications; NASA is creating a smart glasses system that can be used by ground operations technicians on Earth and by astronauts in space; mapping the topography of the brain; and scientists in North Carolina are using the Carolina Chickadee as an example of how songbirds can survive climate change.



Coming up... Technology blurring the lines between virtual and physical reality.

This could be video-game characters running around in here with us.

It could be information about your computer, so I could look at it, and I could see how to actually take the computer apart.

Mapping the mysteries of the brain.

Its architecture -- so, what is going on at the microscopic level.

Its function -- how different parts of the brain are connected.

And then, how individual areas of the brain are organized.

The effect of climate change on bird nests.

This is a smaller nest.

So, you can see that they typically will build a layer of moss, and then they'll build a cup on top.

It's all ahead.

Funding for this program is made possible by...


I'm Hari Sreenivasan.

Welcome to 'SciTech Now,' our weekly program bringing you the latest breakthroughs in science, technology, and innovation.

Let's get started.

After its launch, Pokémon GO swept the nation in popularity, as millions of people broke their everyday routine to go outside in search of these mythical, digital Pokémon creatures.

The game also put the spotlight on augmented reality, technology that adds virtual elements into our physical realm.

Joining us is Mark Skwarek, director of New York University's Mobil Augmented Reality Lab.

First of all, I didn't know there was such a thing until this story came up, but this is cool.

You guys have been thinking about augmented reality for a long time.

This is just the most popular incarnation of it that people are now paying attention.

We should clarify for our audience.

Augmented reality and virtual reality -- two different things.


So, virtual reality -- I'm basically in a simulation.

I'm in a virtual simulation.

So, the real world is closed off.

I'm completely in a virtual simulation, whereas augmented reality is a mixture of the virtual and the real coming together.

So, I have overlays on top of the real world.

This could be video-game characters running around in here with us.

It could be information about your computer, so I could look at it, and I could see how to actually take the computer apart.

And it could be data visualization or something like that.

So, I could be seeing the stock prices of whatever the computer has.

I remember there was a time when I could turn my phone around and see different restaurant reviews that were gonna pop up on the other side in kind of it's in that direction that I need to walk, right?

One of the things that sometimes with maps is you sit there and you stand there, and you're like, 'Well, I wonder if this is the direction that it thinks I'm going, or is it in the opposite direction,' right?

Now Pokémon has kind of made a splash, and it's growing around the world slowly.

But what was the big breakthrough?

Why did people connect to this game, besides the fact that maybe there was a generation of people that grew up with Pokémon cards?

That's a good question.

I've been doing this stuff for years, and we've been actually working on games which were somewhat similar to Pokémon GO.

Technically, this was possible to do years ago.

But what's happened, and this is kind of the consensus of a lot of the developers.

It's the branding.

Perfect timing -- you have Pokémon, which has a huge fan base.

You have basically everybody knows how to play the game, and now we kind of play in reality.

So, it's kind of a mixture of the technology with the perfect kind of branding at the same time.

Underlying technology is something that's not Nintendo.

And it's not the people who own Pokémon specifically that own all this.

Explain the back end of that.

So, this would be Niantic.

So, it would be Niantic that would be doing app-development game studio.

They worked on a game previously called 'Ingress.'

You hear a lot of people basically saying that they basically reskinned Ingress.

Taking the other game, swapped out the old models, and moved the new models in.

Kind of going back to your last question, one of the things that also kind of pushed off that Ingress did was, it's the game play.

Even though the technology was there, this is a new form of media.

And coming up with these unique experiences for this new technology -- it's sort of virgin territory.

So, they worked the game play out quite well.

So, they've done an excellent job.

This is maybe one of the first big games you see as a designer, as a gamer.

What's the next big thing?

What are you thinking about?

I think that the approach that Niantic's taken with this game Pokémon GO is really smart.

They basically kind of put a seat belt on it, like safety features.

You hear a lot of people kind of debate over whether or not Pokémon GO is really augmented reality or not.

We can kind of talk about that more.

But basically the object loads in front of me.

Even if I'm looking at the map, it should be loading over here, by the 7-Eleven or something.

I'm looking over here, and it's loading in front of me, but it should be over there.

So, kind of safety features so I wouldn't have to swing around.

My attention is focused on this little device, and my field of view, everything's kind of coming out of focus around me.

And if I start swinging around, I start running in a certain direction.

You've seen some of the crazy videos online of Pokémon GO players just sort of charging off after something that they -- a virtual object that's not really there.

What we're dealing with right now I would say is augmented reality if it's used under the correct situation.

If I'm not running around or moving around and looking at things, and it's lining with the real world, then it would be an augmented experience.

But where it has to go from here would be having experiences which are really starting to mix the physical and virtual.

We have a little bit of that, if it's used correctly, but the virtual content should be intelligent to the real world.

It should know that the table is here.

It should know that the door is there.

And then it could be racked into realtime feeds, as well.

So, tell me, what are the applications for this besides gaming?

What could you conceive of augmented reality helping a wider slice of the population?

Oh, lots of stuff.

So, first would be probably task-based assistance.

This would be very useful.

Augmented reality, I think, has the potential to democratize technology or democratize knowledge, essentially.

I could look at anything, and I could become incredibly intelligent about it very, very quickly.

The comparison would be to 'The Matrix.'

So, you have Keanu Reeves looks at the helicopter, or he wants to learn kung fu, he plugs his neck into the jack, and all of a sudden he knows to fly the helicopter, like, within seconds.

I could look at a complicated object, and I could understand how it works very well.

The comparison could be IKEA products or something like that.

Everybody's had to assemble some sort of product.

Got all these little bolts all over the floor.

With this new technology, I could basically scatter this stuff across the floor, and it would illuminate the bolt that I have to use the next step, and it would draw a little line to the hole that it has to go to, and I could -- no-brainer.

It's telling me how many times I have to turn.

If I turn too much, it strips the threads, and it could say, 'You're getting close.

Slow down.'

And you could really become very intelligent about things you have no idea about.

The person who's not inclined to change the light bulb or to fix the broken part, even with the new technology probably won't.

They're probably still gonna kind of have that distance, but applied to third-world circumstances, you could have something akin to an industrial revolution.

Knowledge could disseminate freely across the world, and you could see basically the class of people arise, their wealth.

You're taking instead of just watching a YouTube, that layer could be right on top of, if it was about to, say, fix your car or change the bicycle wheel.

It could be talking to me as I'm doing it, telling me how many times to screw the wheel, how many times to do this, that.

Other things -- entertainment, which we're seeing with Pokémon GO.

You're seeing the very tip of the iceberg.

This is, again, like, low-res augmented reality.

This wouldn't be what people would consider to be this really immersive experience.

They're like, 'Wow!'

But I'm not quite sure the entire general public's ready for it quite yet.

So, just get used to the idea that you have content or information located at specific geographic locations and then how to kind of access that safely.

People will start to kind of become able to access it more easily.

Other things could be navigation.

Like, we're sort of saying, as well, I could -- you were bringing up basically the Yelp iteration.

One of the ones that was famous was the subway stops.

It was one of the first apps that came out for smartphones.

I can look around.

I can see all the subway stops around me.

I can see the bus stops.

I can see my distance to the bus stop, and then the newer iterations would actually show you how long until the next bus would show up.

All right.

Mark Skwarek of NYU.

This future is augmented, and it's possible.

Thank you for joining us.

Thank you very much.

So, 'IDEAS' stands for the Integrated Display and Environmental Awareness System, and it's a project that NASA is working on where we're trying to develop a smart-glasses system that can be used by our ground-operations technicians here on Earth, but also in the future by our astronauts up in space.

So, the technology basically it's a computer, and it's displaying the computer's information on glasses.

The hardware that you see here consists of the heads-up display, which basically is just used to be able to show the information from the CPU up to the user's line of sight.

Followed by that, we have the helmet, or the hard hat, which contains the processing unit, as well as the different camera systems -- infrared and regular camera.

People working in factories, people working on an oil rig, people working in the middle of a national forest -- this type of technology can be helpful for them.

They could get instant feedback directly from the expert engineer out in a remote location.

It's very exciting to work on this project because the technology is on the forefront of what's going on out in industry.

And it's even doubly exciting knowing that it's gonna help NASA achieve its goal of sending humans far beyond our planet.

I look forward to the day when we have humans walking on Mars, and they'll be using our technology.

So, it's really exciting to be able to be a part of this.

For more than a hundred years, neuroscientists have argued over a single, seemingly simple question -- how many different areas make up the brain?

A team of scientists at Washington University in St. Louis, Missouri, have charted what may be the most accurate map of the brain to date.

Reporter Andrea Vasquez talks to Matthew Glasser and David Van Essen, neuroscientists at Washington University's School of Medicine, via Google Hangout.

And, Matthew Glasser and David Van Essen, thank you for joining us.

Thank you.

Our pleasure.

When we're talking about this map of the brain, even when we're talking about road maps, there are different types.

There are some that map topography or roads or state boundaries.

What exactly are you mapping here?

So, we're mapping basically four properties of how the brain is organized.

Its architecture -- so, what is going on at the microscopic level.

Its function -- how different parts of the brain are connected.

And then, how individual areas of the brain are organized.

So, those four key things are what we were mapping to make this new parcellation or map of the brain.

And if I can add just a little bit, our focus is on the human cerebral cortex.

And what we're mapping are the equivalent by analogy of the political subdivisions of the Earth's surface.

We map those on top of maps of the convolutions, or folds, of the cerebral cortex, which are analogous to the mountains and valleys of the Earth's surface.

And this is coming out of a longer-standing project, the Human Connectome Project.

Can you tell us more about that?

That's correct.

The Human Connectome Project started in 2010, and it's wrapping up in its current studies of the young adult Human Connectome.

We studied 1,200 healthy, young adults, and in projects that are just starting as the young adults Human Connectome Project wraps up, we'll be studying the development and structure and function and connectivity in children and in older adults to get a better picture of the entire human life-span.

There are also a number of projects that are focused on comparing connectomes in normals to connectomes in various either neurological or psychiatric diseases.

David, do you remember how many?

There are 13 projects funded already by the National Institutes of Health and several more that are likely to be funded in the coming year or so.

And a connectome -- what exactly does that entail?

A connectome -- I like to say it's a comprehensive map of connections in the brain.

But then I immediately put 'comprehensive' in quotes because it's only to the resolution or ability to resolve the basic units of the human brain.

And for the Human Connectome Project, our resolving power are little, tiny-volume units that are a tenth of an inch or so on a side and literally contain hundreds of thousands of neurons and millions and million of synapses between neurons.

So, we'd like to get even better, but there are technical limits of what one can achieve with modern technology.

I'd say there are two kinds of connectomes that we're interested in measuring here.

One is what we call a functional connectome, and that's basically showing how different brain areas, how their activity correlates with each other.

So, if you put somebody in the scanner, an MRI scanner, and you just have them rest, and let their mind wander and think whatever thoughts they want to think, while you're doing that, we measure the MRI signal, and you look for where different brain areas are showing similar fluctuations in that MRI signal.

That's one kind of connectome.

And then the other kind is structural connectome.

And there we're trying to map the actual physical connections between brain areas, albeit in a very indirect way.

Up until now, there have been other diagrams and maps of the brain, but how have those fallen short, clearly, of what you are now accomplishing and working to keep building up?

There have been maps, or what we call parcellations, of the human cerebral cortex for more than a century.

The classical maps from a century ago are kind of quaint and charming but well out of date.

They're analogous to 16th-century maps of the Earth's surface.

In more recent years, there have been major advances in getting better maps, but previous mapping efforts are focused on one type of information, or what we call one imaging modality at a time.

And, as a consequence, those earlier maps got some regions, some areas more or less correct but missed out on or misidentified others.

So, as Matt said a moment ago, by virtue of looking at four independent types of information at one time, we feel confident that we've gotten more accurate and more comprehensive maps, albeit it's not the end the story.

There's still a lot more to learn.

And how much of a given person's brain would align with some of these maps that are being developed, and how much is shaped by unique experiences and genetic factors?

That's a great question, and the answer is we don't know yet.

What we can say is in the great majority of the hundreds of individuals we examined using the computer algorithm Matt mentioned a moment ago, we can identify nearly all of the 180 distinct cortical areas in each hemisphere in nearly all individuals.

So, they are present and to a high degree accounted for, but they aren't identical.

They differ in size by more than a factor of two from one individual to another.

And then a very interesting subset, about 10% of our population that we've studied, at least some areas are what we call atypical in their arrangement.

They've switched their neighborhood relationship with nearby areas, and so they have the same basic constellation of areas, but it's like California and Nevada had switched their places on the map.

Matt Glasser and David Van Essen, thank you very much for joining us.

Thanks for having us.

Our pleasure.

For most animals, odors are the central sensory modality used to communicate with the world.

How is it that we can recognize a vast repertoire of odors in the world?

Through molecular chicanery, we were able to show that we do this by virtue of having in cells neurons in our nose dedicated to the recognition of odors, and these neurons as a population make in most species very large numbers of what are called 'receptors,' each capable of binding to different combinations of odors.

And it's not until you get to evolutionarily highly evolved creatures -- primates, monkeys, man -- that smell begins to take second place among the senses to vision.

Scientists in North Carolina's Great Smoky Mountains National Park are studying Carolina chickadees build their nests in different climates.

They hope to use this research to understand how songbirds will adapt to climate change in the future.

Let's take a look.

[ Birds chirping ]

The Great Smoky Mountains and even a larger region of the southern Appalachian Mountains are a treasure trove of biodiversity.

Forests and streams, as well as the mountains and valleys, are filled with a seemingly endless variety of life, both large and small.

And all of these things fit together and help tell the story of the Smokies and what we're trying to protect.

[ Insects chirping ]

20,000 different species counted so far in Great Smoky Mountains National Park alone, but this is the story of just one tiny creature filling one tiny spot in that web of life.

Meet the Carolina chickadee.

Most all songbirds in North America, most of our breeding birds are in decline.

And so, if we know more about them, it might enable us to be able to better protect and conserve them.

Scientists know some things about this little bird.

It weighs about .4 ounces.

That's about the same as a half-slice of bread.

The Carolina chickadee is about 5 inches tall.

The wingspan is about 8 inches across.

Carolina chickadees, they do a single brood in a season.

So, that means they have one shot at it every year.

And if they fail, then they have to wait till the following year.

Like most songbirds, the female Carolina chickadee builds the nest.

Once the nest is ready, the female lays one egg per day Once there is a full clutch, the female begins incubating the eggs.

And that is a really critical time for both themselves and the nestlings.

So, they need to keep the temperature of the eggs above a certain threshold in order for them to develop properly.

And it turns out nest building is critically important to the survival of Carolina chickadees.

The better the female builds the nest, the better she'll be able to take care of her little ones because she will be healthier.

So, being on the eggs and keeping them warm means they're not out foraging and taking care of themselves.

So, they have to balance their activity to take care of the eggs and keep them at an optimal temperature for development and taking care of themselves so that they can survive to reproduce again in the following year.

And it seemed like a good opportunity for us to ask what is really critical about female behavior that's going to lead them to have a lot of reproductive success.

But to study nesting behavior, researchers need to observe nests.

Nest boxes were placed at varying elevations.

That meant different temperatures, and that translated into different times for nesting.

Temperature sensors were placed outside and inside the boxes, as well as inside the nests for comparison.

These are just a quick example of some of the variations that we found.

This is a smaller nest.

So, you can see that they typically will build a layer of moss, and then they'll build the cup on top.

You can see they'll use a little grass or fur that they find.

And you can see how this one of course is so much taller and deeper and a little more insulated than this one.

Researchers found the female Carolina chickadee didn't build one standard type of nest.

The style depended on where the nest box was located.

So, the females want to try to reach about 100 degrees Fahrenheit in the nest, temperature.

If the eggs are not kept at a proper temperature, then the embryos won't develop correctly, so they won't hatch.

And as far as the nestlings, when they're first born, they're unable to regulate their body temperature for three to four days.

So, they really depend on warmth from their mother and from the nest to regulate their temperature.

Different types of insulation and different types of nest construction were used to adapt the nest to the climate.

That allowed the female to stay stronger and take better care of her nestlings.

This is an investment that they're making that might help them when they're incubating.

So, it might alleviate some of the pressure of being on the nest so often to keep the temperature of the eggs up, because if there's a lot of insulation in the nest, then the temperature of the eggs might fall more slowly when she leaves the nest.

So, she can be off for a little bit longer.

That means the female can forage longer and be stronger and healthier.

The question -- what happens as the climate changes?

There's probably a lot of factors that are involved with decline of songbirds in North America, but the fact that these birds have one shot at a reproductive in a season is probably gonna be a big factor for chickadees.

And that wraps it up for this time.

For more on science, technology, and innovation, visit our website, check us out on Facebook and Instagram, and join the conversation on Twitter.

You can also subscribe to our YouTube channel.

Until next time, I'm Hari Sreenivasan.

Thanks for watching.

Funding for this program is made possible by...