Pokemon Go and the future of augmented reality

After its launch, Pokemon Go swept the nation in popularity. Mark Skwarek, director of New York University’s Mobile Augmented Reality Lab joins Hari Sreenivasan to discuss how the game put the spotlight on augmented reality, technology that superimposes virtual elements on top of thephysical world.


After its launch, Pokémon GO swept the nation in popularity, as millions of people broke their everyday routine to go outside in search of these mythical, digital Pokémon creatures.

The game also put the spotlight on augmented reality, technology that adds virtual elements into our physical realm.

Joining us is Mark Skwarek, director of New York University's Mobil Augmented Reality Lab.

First of all, I didn't know there was such a thing until this story came up, but this is cool.

You guys have been thinking about augmented reality for a long time.

This is just the most popular incarnation of it that people are now paying attention.

We should clarify for our audience.

Augmented reality and virtual reality -- two different things.


So, virtual reality -- I'm basically in a simulation.

I'm in a virtual simulation.

So, the real world is closed off.

I'm completely in a virtual simulation, whereas augmented reality is a mixture of the virtual and the real coming together.

So, I have overlays on top of the real world.

This could be video-game characters running around in here with us.

It could be information about your computer, so I could look at it, and I could see how to actually take the computer apart.

And it could be data visualization or something like that.

So, I could be seeing the stock prices of whatever the computer has.

I remember there was a time when I could turn my phone around and see different restaurant reviews that were gonna pop up on the other side in kind of it's in that direction that I need to walk, right?

One of the things that sometimes with maps is you sit there and you stand there, and you're like, 'Well, I wonder if this is the direction that it thinks I'm going, or is it in the opposite direction,' right?

Now Pokémon has kind of made a splash, and it's growing around the world slowly.

But what was the big breakthrough?

Why did people connect to this game, besides the fact that maybe there was a generation of people that grew up with Pokémon cards?

That's a good question.

I've been doing this stuff for years, and we've been actually working on games which were somewhat similar to Pokémon GO.

Technically, this was possible to do years ago.

But what's happened, and this is kind of the consensus of a lot of the developers.

It's the branding.

Perfect timing -- you have Pokémon, which has a huge fan base.

You have basically everybody knows how to play the game, and now we kind of play in reality.

So, it's kind of a mixture of the technology with the perfect kind of branding at the same time.

Underlying technology is something that's not Nintendo.

And it's not the people who own Pokémon specifically that own all this.

Explain the back end of that.

So, this would be Niantic.

So, it would be Niantic that would be doing app-development game studio.

They worked on a game previously called 'Ingress.'

You hear a lot of people basically saying that they basically reskinned Ingress.

Taking the other game, swapped out the old models, and moved the new models in.

Kind of going back to your last question, one of the things that also kind of pushed off that Ingress did was, it's the game play.

Even though the technology was there, this is a new form of media.

And coming up with these unique experiences for this new technology -- it's sort of virgin territory.

So, they worked the game play out quite well.

So, they've done an excellent job.

This is maybe one of the first big games you see as a designer, as a gamer.

What's the next big thing?

What are you thinking about?

I think that the approach that Niantic's taken with this game Pokémon GO is really smart.

They basically kind of put a seat belt on it, like safety features.

You hear a lot of people kind of debate over whether or not Pokémon GO is really augmented reality or not.

We can kind of talk about that more.

But basically the object loads in front of me.

Even if I'm looking at the map, it should be loading over here, by the 7-Eleven or something.

I'm looking over here, and it's loading in front of me, but it should be over there.

So, kind of safety features so I wouldn't have to swing around.

My attention is focused on this little device, and my field of view, everything's kind of coming out of focus around me.

And if I start swinging around, I start running in a certain direction.

You've seen some of the crazy videos online of Pokémon GO players just sort of charging off after something that they -- a virtual object that's not really there.

What we're dealing with right now I would say is augmented reality if it's used under the correct situation.

If I'm not running around or moving around and looking at things, and it's lining with the real world, then it would be an augmented experience.

But where it has to go from here would be having experiences which are really starting to mix the physical and virtual.

We have a little bit of that, if it's used correctly, but the virtual content should be intelligent to the real world.

It should know that the table is here.

It should know that the door is there.

And then it could be racked into realtime feeds, as well.

So, tell me, what are the applications for this besides gaming?

What could you conceive of augmented reality helping a wider slice of the population?

Oh, lots of stuff.

So, first would be probably task-based assistance.

This would be very useful.

Augmented reality, I think, has the potential to democratize technology or democratize knowledge, essentially.

I could look at anything, and I could become incredibly intelligent about it very, very quickly.

The comparison would be to 'The Matrix.'

So, you have Keanu Reeves looks at the helicopter, or he wants to learn kung fu, he plugs his neck into the jack, and all of a sudden he knows to fly the helicopter, like, within seconds.

I could look at a complicated object, and I could understand how it works very well.

The comparison could be IKEA products or something like that.

Everybody's had to assemble some sort of product.

Got all these little bolts all over the floor.

With this new technology, I could basically scatter this stuff across the floor, and it would illuminate the bolt that I have to use the next step, and it would draw a little line to the hole that it has to go to, and I could -- no-brainer.

It's telling me how many times I have to turn.

If I turn too much, it strips the threads, and it could say, 'You're getting close.

Slow down.'

And you could really become very intelligent about things you have no idea about.

The person who's not inclined to change the light bulb or to fix the broken part, even with the new technology probably won't.

They're probably still gonna kind of have that distance, but applied to third-world circumstances, you could have something akin to an industrial revolution.

Knowledge could disseminate freely across the world, and you could see basically the class of people arise, their wealth.

You're taking instead of just watching a YouTube, that layer could be right on top of, if it was about to, say, fix your car or change the bicycle wheel.

It could be talking to me as I'm doing it, telling me how many times to screw the wheel, how many times to do this, that.

Other things -- entertainment, which we're seeing with Pokémon GO.

You're seeing the very tip of the iceberg.

This is, again, like, low-res augmented reality.

This wouldn't be what people would consider to be this really immersive experience.

They're like, 'Wow!'

But I'm not quite sure the entire general public's ready for it quite yet.

So, just get used to the idea that you have content or information located at specific geographic locations and then how to kind of access that safely.

People will start to kind of become able to access it more easily.

Other things could be navigation.

Like, we're sort of saying, as well, I could -- you were bringing up basically the Yelp iteration.

One of the ones that was famous was the subway stops.

It was one of the first apps that came out for smartphones.

I can look around.

I can see all the subway stops around me.

I can see the bus stops.

I can see my distance to the bus stop, and then the newer iterations would actually show you how long until the next bus would show up.

All right.

Mark Skwarek of NYU.

This future is augmented, and it's possible.

Thank you for joining us.

Thank you very much.