NFL Network

The NFL Network migrated its production facility to ST 2110 IP, with Imagine’s SNP delivering low-latency multiviewing.

Introduction
Proof of concept
Low-Latency Processing
Virtual Re-entry
Distributed Monitoring

*The article below originally appeared on digitalmediaworld.com.

Charley Haggarty is Broadcast Infrastructure Architect ST 2110 at the NFL Network. He joined the broadcast industry about 10 years ago, working first for Pac-12 Networks in San Francisco, where he and his team pioneered the idea of at-home production over IP networks for live sports. “We built a WAN between the 12 Pac-12 universities, so we only sent cameras and microphones – and talent! – to the site. We produced 850 live events this way from our offices in San Francisco,” he said.

Then in 2016, he met Bruce Goldfeder, VP of Broadcast Engineering for the NFL, who had set the goal to migrate the entire network’s production facility to SMPTE ST 2110 IP. Charley joined the NFL Network in 2017, and rapidly became an expert in just about anything related to ST 2110.

Proof-of-Concept ST 2110 Lab

“Alan Wollenstein, Director Engineering Systems for the NFL, and I built a proof-of-concept lab to evaluate products, as well as test interoperability,” said Charley. “I am fortunate in having hands-on experience with nearly every piece of broadcasting gear that supports ST 2110. After nearly a year of testing anything we could get our hands on, Imagine’s Magellan SDNO Control System was selected for the router control and the Selenio Network Processor (SNP) for video processing and multiviewing.”

Multiviewing was one of the major challenges for the NFL Network’s IP migration project. In live sports production, there is a need to monitor signals from a variety of sources with immediacy and accuracy. Charley and Alan looked at all seven of the multiviewers on the market that were capable of ST 2110 at that time. In particular, they looked for a robust, agile, low-latency system that would fit the way they worked and scale as they grew.

Charley said, “Our goal was a system that used ST 2110 but felt like SDI. We had to deliver template building and layout modification. Our multiviewer had to include excellent alarm functionality, and integrate seamlessly with our existing tally system.

“The SNP, with its Layout Designer software, met all of those requirements. More than that, it opened our eyes to the possibilities of a software-configured device. Using Layout Designer, the flexible SNP frame could be a frame sync one day and a multiviewer the next, with the click of a mouse.”

“I am fortunate in having hands-on experience with nearly every piece of broadcasting gear that supports ST 2110. After nearly a year of testing anything we could get our hands on, Imagine’s Magellan SDNO Control System was selected for the router control and the Selenio Network Processor (SNP) for video processing and multiviewing.”

Charley Haggarty NFL Network, Broadcast Infrastructure Architect ST 2110

A closeup of a hand over a bowl of chips.

Consistent Low-Latency Processing

Low latency was a critical factor. Sports is fast-moving and operators need to be able to talk to people on-site at events, look at the feeds and see instantly what is happening. “It is critical for our key team members to see incoming and outgoing data as close to real time as possible. A major benefit of the SNP has been the consistent low-latency processing of ST 2110 workflows when it’s used as a multivewer – so much so that our operators can match what we are sending from production to the net return,” said Charley.

“It is extremely important for us to have minimal latency through the MV system to accurately communicate on-site, where we switch interchangeably between a studio signal and a return signal from off-site, e.g., a remote stadium. When we take the feeds from off-site and ingest them through our facility, our on-site technical directors can communicate to the crew members to make changes, while monitoring these changes on the MV wall in our control rooms.

“We needed to reduce or eliminate as much delay as possible from inherent systems like the MV, so we did not end up with a facility with inconsistent delays in the monitoring tools. We found the SNP-MV to be the lowest latency, with the most consistent platform to meet our needs and expectations for this project.”

Virtual Re-entry

As well as viewing the signals, the network relies on the SNP to process alarms, including tally status, virtual re-entry status, audio meters for two ST 2110-30 streams (for 16 channels), and video alarms like freeze, black and signal loss. They are now looking forward to the next software release when they will be able to add monitoring of IP stream status, colorimetry and ancillary data such as VITC and caption decoding.

To explain the virtual re-entry process, Charley said, “Our SDNO routing database is comprised of SMPTE 2110 multicast essences. By default, when making a route, it routes in a concise group. When we need to combine these essences in new ways to achieve workflow goals, then we use ‘virtual re-entries’ that have 2110 data inputs that can comprise multiple original groups, which then become routable in a single route using that virtual re-entry.”

Visually, here is an example of how it may appear in a workflow:

A look at the alarm view available in SNP.

In this case, they have a virtual re-entry (VR) group named ‘A PGM’. A PGM is taking the 1080p video from the source ‘PCA MPGM’ (Sony switcher card output), and taking audio ch 1-16 (in two multicast groups) from ‘ACA PGM 1’ – this is a Dante-to-2110 converter and represents an output of their SSL audio console. Their ST 2110 routing typology consists of two identical, redundant fabrics. All senders and receivers are simultaneously interacting with both networks, creating a fully fault tolerant environment.

“To monitor multiple source essences from diverse physical fabrics, we rely on the SNP to alert when a receiver has a problem, such as not getting the expected audio, video or metadata packets, or if one of the redundant paths is having an issue,” said Charley.

A look inside the NFL Los Angeles Facility

Distributed Monitoring Infrastructure

The NFL Los Angeles facility is format agnostic, capable of receiving and creating HD and Ultra HD productions from any of its sources or stages, simultaneously. The camera CCUs output 1080p and 4K continuously, so the ability of the SNP-MV to accept any format and scale appropriately is another advantage. In fact, the NFL Hollywood Park production control rooms have the ability to change from HD to UHD from one event to the next.

Charley believes the NFL’s decision to use SNP-MV multiviewers has been fully justified. “They allow us to have a distributed monitoring infrastructure,” he remarked. “They make it simple to set up whatever layouts the operators require and to switch between them from event to event. They deliver high, format-agnostic image quality and when we do not need a particular multiviewer channel, the SNP path can be reallocated to a completely different task.”