The science behind the yellow first-down line

Sports Media Technology Company, SMT, based in Durham, North Carolina, is a pioneer in real-time virtual sports graphics, blending technology, computer science, and art. The company has created a revolution in televised sporting events.

TRANSCRIPT

There's a lot of science and technology that goes into creating the yellow first-down line marker, one of the most iconic images in televised football games.

Sports Media Technology Company, SMT, based in Durham, North Carolina, is a pioneer in real-time virtual sports graphics, blending technology, computer science, and art.

SMT has created a revolution in televised sporting events.

Here's the story.

An artist dreams of having tens of thousands of people view their work, their creations.

And then I'm making sure that my color is good, I'm making sure that the line is in the correct spot, and then I insert it, all before the director is getting ready to take the game camera.

More than 18 million people see Iam Prokes' handiwork every week.

And you want to make sure that, when you put it in, it has to be in the exact right position, because then everyone at home is going to see that it's not right, right?

So a little bit of pressure, right?

I was going to say -- no pressure there.

Yeah.

The remote operations technician blends technology, computer science, and art to draw the first-down line on 'Sunday Night Football,' the NFL's biggest game of the week.

That's right, the iconic yellow line.

It's all up to him.

You have to do a lot of little things before the next play.

So you got to make sure that you have your ball position and everything is ready to go before they come back to the game camera.

Otherwise, the line is going to come in late.

The first-down line may be the best-known work of Durham-based SMT, but these days, the company adds visual aids for most televised sports.

Think strike zones in baseball, pointers for NASCAR drivers, the flight paths of golf balls.

For us, it really is about informing the viewer.

It has to start with information and something that gives the viewer a piece of information that they wouldn't have otherwise and putting it in a place where they can see it very easily.

It's a kind of television revolution.

While technology now allows real-time data to be displayed, viewers must be comfortable with graphics constantly on the screen and now being virtually inserted into a scene.

For us, just very logically, 'Hey, we've now put something on the screen that people are used to seeing, but how about now we put it into the scene itself?'

So, just how does the first-down line happen?

It's all computer algorithms.

We have to build up a mathematical model.

This is a very complex lens.

It has 19 different elements in it, of glass.

And so we need to be able to build a mathematical model.

We get readings off of the lens that tell us a number of how it's zoomed and a number of how it's focused.

We get readings back from the panhead that tell us how the lens is titled and how it's panned.

But we need to take those numbers and turn that into, 'Hey, I'm looking at this area from the left 45 to the right 40, and I'm zoomed to this level, so I see this much of the field.'

And that information can be used to build a mathematical model of how the camera sees the field.

We take, then, a model of where we want to put things on the field, that I want to put a line of scrimmage here, at the right 46.

Then I want to put the first-down line at the right 36.

And, together with those two mathematical models, we can figure out where to draw our elements and to keep them on the field.

So, 60 times a second, we get new information from the lens and from the panhead saying, 'Here's how we're zoomed.

Here's how we're pointed.'

60 times a second, we say, 'Here's where we should draw these things.'

And because so much data is reported so frequently, that first-down line moves with the camera.

But there's something else that goes into making the first-down line appear as if it is on the field and part of the game.

The other piece here is that we need to figure out how to make it appear behind the player like this, and the way we do that is by analyzing the colors of the image.

So, ahead of time, our operator can go through and pick areas of the field -- You can see there are different greens here on the field.

You pick different areas of the field and you identify a set of colors that you're willing to draw on top of, which we're doing here -- we're drawing on top of this green in the field -- and colors you're not willing to draw on top of.

So, we're not willing to draw on top of whites in the uniform, black in the helmet, the yellows in the pants.

The algorithm is sophisticated.

It identifies various hues, saturations, and shades of color -- in this case, the green of the field.

It takes technicians about 3 hours to set up the cameras with the technology.

There are usually six cameras used in an NFL game.

But for all the technology, it's the technician that must enter the information into the system in about 10 seconds.

So, essentially, everything is in right now.

Iam let me give it a try.

So the ball -- they just moved it down to the 14-yard line, right?

Right.

So what you want to do is -- you want to take the mouse.

And I would immediately pull everything out, make sure everything is locked up together so it's automatically 10 yards, and then move it down -- What is that?

The 30-yard line?

30.

So I move it down this way, down the field, to the 30.

And so then you can go ahead and fade in your line of scrimmage.

And then you go down here and make sure that it's first and 10.

So you'll click on that third arrow, make it first.

And then you fade in your graphic.

Yep.

Bam!

Sweet!

There it is.

Of course, three plays were ready by the time I completed one graphic.

You could call the first-down line high-pressure art.

So, you do this and you do variations of that every --

Every single play.

Every single play.

It is pretty much nonstop.