New Rules of Codec Development
It took about 10 years to launch both H.264 and HEVC, so MPEG's much more ambitious launch of VVC, EVC, and LCEVC in 2020 and 2021 makes you wonder about its chances for success. Not only does it have to effectively differentiate each codec against the other, it also has to create compelling business cases against the Alliance for Open Media's (AOMedia) AV1 and existing MPEG codecs like HEVC and even H.264, which is still going strong. Recent developments have resulted in what I am calling the new rules of codec deployment, and they will complicate the success of any standards-based codecs.
The first rule is that fewer companies will deploy a royalty-bearing, standards-based codec without knowing the royalty cost. In the past, many companies started adopting H.264 and HEVC before the business terms were known because their success seemed assured and earlier standards had rational royalty policies. All that changed with HEVC, which saw three patent pools launched within 1–4 years after the standard finalized and after many companies had already committed to the technology, assuming that it would be reasonably priced. It wasn't; known annual royalty caps skyrocketed from around $10 million per year for H.264 to more than $60 million, and that's without knowing the cap from one of the pools that won't publicly disclose it.
In the words of David Ronca, director of video coding at Facebook, "When EVC and/or VVC have clear royalty terms, they may be interesting codecs. In the meantime, don't be surprised if there are few companies willing to jump on the 'deploy our codec now, and trust us for fair and clear royalty terms' train. We did that in 2014. Didn't go so well."
Along these lines, the Media Coding Industry Forum (MC-IF) was created, in part, to help accelerate and coordinate the formation of a cohesive and reasonable royalty policy. MC-IF has started the patent-fostering process to identify a single patent pool administrator for forming a single pool and setting and releasing royalty terms. But it hadn't announced the administrator as of Dec. 8, 2020, and after the pool is formed, it will take a few months to fix and announce the royalty. So, it's reasonable to expect royalty terms sometime after mid-2021.
EVC reached First Draft International Standard (FDIS) in June 2020, and MPEG is hoping for license terms to launch within 2 years of FDIS. If it takes this long, and potential implementors wait for royalty terms before committing to the technology, it could be sometime in 2022 before substantial integration work begins. [Editor's note: The preceding paragraph has been modified to reflect the current situation. Thanks to Stefan Schreiber for the clarification.]
The only technology that's ahead of the curve is LCEVC, which announced the availability of licensing terms to implementers in December 2020. Interestingly, V-Nova, which claims to own virtually all of the IP in LCEVC, plans to charge a small capped royalty on streaming publishers that use the codec, not on the browsers, TVs, and other players that enable playback.
The second rule is that standard-based codecs can forget playback without browsers owned by Google and Mozilla and perhaps Microsoft. Though Google never followed through on its 2011 promise to remove H.264 playback from Chrome, Google has refused to integrate HEVC playback in any version of Chrome, even on devices where Google could leverage existing HEVC decoding on the system without paying a royalty. Ditto for Mozilla.
While premium content services enjoy significant playback on living room devices that don't require a browser, the overwhelming majority of smaller publishers see their videos consumed on browsers that are on computers and mobile devices. If the codec isn't supported in the browser, these publishers can't use it.
Of course, browsers aren't the only segment where AOMedia members dominate; you have content publishers (Netflix, Amazon, YouTube), mobile and desktop operating systems (Apple, Google, Microsoft), chipmakers (Intel, AMD, NVIDIA), OTT devices (Google, Amazon, Apple), and phone/tablet vendors (Samsung, Google, Apple). All of these companies are in a position to block or slow the integration of any new codec that doesn't meet AOMedia's goals, which, of course, is any codec other than AV1 and its successors.
There are some traditional markets like contribution and news collection where AOMedia doesn't have a significant role and where standards-based codecs will continue to flourish. But these markets are shrinking, while streaming-related markets continue to expand. If I owned IP in any standards-based codec, I would concentrate on what I could control, which is quickly announcing a royalty policy that promotes usage rather than detracts from it.
Related Articles
If you're charged with configuring your encoding ladder, there are a few datapoints you should have nailed. The most important is the VMAF score of the top rung of your encoding ladder.
09 Jul 2021
Take a 5-minute survey about codecs, encoding workflows, and premium video features, and enter for a chance to win a $500 Amazon gift card.
24 May 2021
Companies and Suppliers Mentioned