Film and TV Measurements of Success Are Changing in Major Ways

Hard numbers still matter, but the more ephemeral metric of impact is now what defines a project’s success. In the future, that will be even more true.

IndieWire turns 25 this year. To mark the occasion, we’re running a series of essays about the future of everything we cover.

What is the number-one movie of 2021? We may never know. It could be Marvel’s “Shang-Chi and the Legend of the Ten Rings”; at $224 million, its domestic gross is the year’s biggest to date. You could also make an argument for “Free Guy” at $121 million, which spent its theatrical afterlife in sustained on-demand dominance. Then there’s the big-budget action-comedy “Red Notice” — Netflix’s “biggest opening ever,” tweeted star Dwayne “The Rock” Johnson. Per Netflix, the film streamed for a total of 277.9 million hours in its first week.

Not long ago — say 2019, circa “Avengers: Endgame” — it was easy to determine the number-one movie: It was the one that grossed the most in its North American theatrical release. Back then, movies competed only with other films in theaters. TV success was similarly easy to measure: the most successful TV show was the one with the biggest rating on a given night.

Ah, the good ol’ days. The pandemic installed the attention economy, in which all media competes in an increasingly fractured landscape. Now imagine viewing that fragmentation through frosted glass. Those who relied on publicly reported ratings or grosses — that is to say, everyone — must now contend with viewership reporting that’s opaque by design and likely to stay that way.

“We don’t know how these films perform on a streaming platform unless they come out and say, ‘This is our number-one film of the year,’” said entertainment attorney Jeff Finkelstein, partner at Del Shaw Moonves Tanaka Finkelstein & Lezcano. “It’s purely voluntary on their part and usually just to serve the purpose of promoting a film that does well and calling attention to that platform.”

Disney claimed one-third of the U.S. box office in 2019, making it the undisputed entertainment king. In 2021, it holds about a quarter of the box office to date, but it’s a new game. At this writing, Disney stock has dropped nearly 17 percent since its November 10 fourth-quarter earnings report revealed Disney+ subscriptions were far below expectations.

Three of the studio’s top releases offer a collective case study: “Shang-Chi” is the top release in domestic theaters — but it’s only the eighth worldwide, with most dates played. “Black Widow,” the number-three domestic release, opened to a same-day Disney+ $29.99 premium VOD offering (sparking a lawsuit from star Scarlett Johansson). “Free Guy,” a theatrical exclusive, is number seven — but it delivered a better home-viewing audience than any other film this year according to results gleaned through tea-leaf readings of VOD charts.

Things are similarly messy at WarnerMedia. “Dune” premiered day-and-date on HBO Max and in theaters, where it grossed $94 million domestic ($354 million worldwide) on a $165 million budget. That’s not enough to break even, according to classic box-office math, but Warners already announced a sequel.

Meahwhile, “Succession” saw just 564,000 people watch the Season 3 premiere on linear HBO on October. According to the streamer’s selectively reported numbers, over 830,000 others watched the premiere on digital platforms, including HBO Max. Its saving grace: Within those “Succession” numbers is a diehard fanbase that makes memes and dresses up as characters for Halloween. The critically acclaimed series received a fourth-season renewal nine days after its October 17 premiere.

Data as privilege

Charts rank movie performance on some VOD platforms, but never with actual revenues. Very occasionally, a studio might provide a rough estimate. Without this data, it’s impossible to gauge profit — although we do know studios keep a much higher percentage of VOD revenue than they do from theaters.

For now, most grosses will continue to be reported — but Warners no longer reports weekdays, only full weeks. For Netflix, Amazon, and Apple, keeping silent on box-office data is all but policy. Netflix’s “Red Notice” spent an unreported week or two in 750 theaters, but the streamer’s share of ticket sales is worth far less to the company than Johnson’s Twitter account.

For years, the industry was so reliant on metrics like box office and Nielsen ratings that it was easy to take them for granted. Companies have to share that information, don’t they? Turns out they don’t — and never wanted to in the first place. Viewership for broadcast television became transparent because companies needed to share that data to court advertisers. Box office provides bragging rights on the back of privileged business information. Streamers have no interest in following suit.

“The reason Nielsen became so popular is the data is valuable,” said Doug Clinton, managing partner of VC tech fund Loup Ventures. “Advertisers wanted to know their reach and what their budget should be, but when users are paying for the content and ads are less important in that structure, I wouldn’t be surprised if it goes away.”

Box-office reporting became prominent in the 1980s, when studios began releasing their films simultaneously across the country rather than market-by market. That created the ideal of a film opening as the number-one movie in America. In 1997, the massive success of “Titanic” turned box-office into headline news and invaluable marketing. Today, films live and die on opening-weekend results and films with the most potential to open big — i.e., franchises — are the ones to make.

Today, even the most detailed box-office results are incomplete. Even if, by some miracle, VOD, streamers, theatrical numbers, and linear TV offered relevant performance insights, how do you compare or aggregate the results of so many disparate platforms? Studios like keeping the data genies in their respective boxes, but that separatism means they risk losing the informational upper hand with litigious stars or powerful guilds.

NBCUniversal made a stab at resolving this issue with the October 14 launch of its Measurement Innovation Forum, its ongoing effort to create an industry-wide measurement system that can work across film, TV, and streaming to accurately reflect consumer behavior and its impact on advertisers.

To figure out what that system could be, NBCUniversal put out a call for proposals from data and analytics firms and received more than 100 responses. “We thought we knew the measurement landscape, but once we put out a call for RFP, we did not expect the level of response we received,” the company’s EVP of measurement and impact, Kelly Abcarian, told the companies in a memo. Usual suspects like Nielsen and Comscore are in play, but so are newer companies like Oracle’s Moat and iSpot.TV, as well as Adobe and Vizio.

Among the areas that NBCUniversal wants to track: audience measurement and verification, brand measurement, sales impact, multi-touch attribution, and business outcome guarantee. As the company notes at the top of the chart outlining those areas, “industry in state of flux, subject to change.”

NBCUniversal’s breaks down the companies vying to define measurement


Buying into the attention economy

For now, the current gold standard for success may be engagement. “The one thing we can promise international creators is the possibility of having a ‘Squid Game’ experience, where the star of your show in Korea can go from 400,000 social media followers to 15 million in five days,” Ted Sarandos told investors on the Netflix third-quarter call.

“What matters is creating an emotional connection with whatever product you’re selling,” said Alejandro Rojas, director of applied analytics at Parrot Analytics. “It goes beyond viewership, which is important, but you’d track other activities, such as people going to Wikipedia to read about a show, watching videos on YouTube, or commenting on social media. All of that is important because it gives you a more holistic perspective of how audiences may be interacting with a show that go beyond just viewership. The more you do this, the more the show is capturing your attention. In an attention economy, that’s the most valuable resource you can get.”

OK, but how do you assign a value to those 14 million new “Squid Game” followers, or a surge in “Tiger King” searches and Reddit boards, or those “Succession” costumes? There’s no clean metric, but that may be the point. When all is anarchy, you can’t game a system. “People can recognize a hit show,” said Clinton. “It’s all over our social feed. We don’t need to see numbers on ‘Game Of Thrones’ to know how successful it is. We don’t lose the cultural feeling or knowledge that something is a hit.”

For a cautionary tale, look no further than Quibi: Blind to the attention economy, it ran straight into a wall. When the made-for-mobile streaming service launched in April 2020, its executives billed it as a kind of premium YouTube, where viewers could indulge their short attention spans. But since Quibi didn’t allow viewers to screenshot episodes, share them on social, or create GIFs, it effectively muzzled its own word-of-mouth. The company allowed screenshots in July 2020, but the service shut down four months later.

What’s good for shareholders isn’t always good for talent

Rather than report data, streaming conglomerates favor occasional, vague, and upbeat press releases. Shareholders don’t care; stock prices rise and fall on the metrics of subscriber growth and retention. That puts people making content at a disadvantage.

“We’re in an increasingly volatile time relative to talent compensation because the dynamics of the industry, both TV and film, are shifting in such a way that we’re losing any transparency as to what the financial upside is for the media companies,” Finkelstein said.

Talent compensation negotiations traditionally utilize TV ratings and box office grosses; guild agreements base their fees, residuals, and profit participation on publicly reported viewership. Streaming upends all of that, as the world saw when Johansson sued Disney after the studio chose to premiere “Black Widow” simultaneously in theaters and on Disney+.

Her attorneys argued that streaming viewership ate into the box-office returns, which impacted her pay. Disney made a unilateral decision in the interest of shareholders, not the artists that created “Black Widow,” Johansson argued in the now-settled suit.

Black Widow/Natasha Romanoff (Scarlett Johansson) in Marvel Studios' BLACK WIDOW, in theaters and on Disney+ with Premier Access. Photo by Jay Maidment. ©Marvel Studios 2021. All Rights Reserved.

Scarlett Johansson in “Black Widow”


Disney pioneered fixed bonuses for streaming, but grand-slam backend payouts are gone. “The possibility to make well into the eight figures, if not even the low-nine-figures — that’s gone, completely gone, with the notion of fixed bonus points,” Finkelstein said.

For now, negotiations for streaming projects are often based on an awkward formula: historical comps for what a given movie might have made in theaters. It’s clear that practice is unsustainable. Finkelstein believes talent will need to push back.

“This may take another guild negotiation, or two, or three,” he said. “I do think at some point the streamers are going to reveal the viewership for specific shows. As they become an increasingly important, and even the dominant platform for the production and distribution of movies and television, the guilds are going to be forced to focus on them much more greatly than they have in the past.”

Ultimately, the solution could be a simple one: If you can’t beat them, join ’em. “Talent has historically had a vested interested in the success of the film, of the success of the TV show by having a piece of the backend,”…