The State of VVC Adoption and Implementation
Codec adoption is like that slow-moving development being built across town. It seems to take forever to get the roads, storm drains, and sewage systems in and the lot grading completed. The developer creates a model home or two to spur interest, but no one buys until at least the first few homes are completed. Adventurous first movers take the leap, spawning additional lookie-loos who turn into buyers and generate more moving vans and trampled yards. Months or years later, all of the homes are completed and occupied; trees, shrubs, and flower gardens are growing; and the development becomes a mature neighborhood.
I recently participated in a panel on VVC at Streaming Media East. Where is VVC in the codec adoption process? Well, the roads, storm drains, and sewage systems are all in, and there’s even a model home or two. If you read the tea leaves, you know that VVC has location, location, location, so it’s bound to be a success. But, unfortunately, most of the homes are years from being ready for occupancy.
OK, that’s it for the metaphor, I promise. In this article, I’ll introduce you to the panel’s participants, briefly summarize key details from their talks, and share a download link to their presentations. Stay with me through this lengthy article, and you’ll better understand where VVC is and when it will come into prominence. In the end, I’ll share some thoughts on what this all means for the AV1 codec.
For you TL;DR types, I’ll say that if you’re a video publisher who doesn’t rank high in the VVC patent list, it’s likely that you won’t consider VVC until 2026 or later. If you’re looking to augment your H.264 encodes with a more efficient codec before then, your most likely choices are HEVC and AV1, with LCEVC also an option. However, if you’re a product or service provider in the streaming media ecosystem, it’s long past the time to start considering where and when VVC is going to fit in for you and your target customers.
Let’s meet the panelists and get an overview of their presentations.
VVC Performance and Standardization Efforts
The first speaker was Alan Stein, VP of technology at InterDigital, a research and development organization focusing on wireless and video technologies. (Download his presentation here.) Stein started his discussion by outlining VVC’s architecture and new coding tools, which have contributed to a quality improvement of 40% over HEVC, with only a 1.6x decode complexity versus HEVC. As you’ll see, this translates to several early VVC deployments and efficient software-only playback on mobile devices.
Then Stein moved to the deployment status, partially shown in Figure 1. The deployments in the figure largely relate to application standards that promote and accelerate VVC adoption in future devices. For example, inclusion in the DVB toolbox means that the codec can be deployed in regions or countries that adhere to the DVB standard, which will naturally accelerate VVC’s deployment. Inclusion in the ATSC standard means that at some point, after the ATSC 3.0 standard becomes mandatory, TVs sold in the U.S. and Canada will be increasingly likely to support VVC, again accelerating its deployment.
Figure 1. VVC Adoption in Application Standards (from Alan Stein)
Other standards simplify implementing VVC for potential publishers and ecosystem providers. For example, the CTA WAVE Content Specification is used to develop compatibility tests for encoding and player developers and computer, mobile, and living room device manufacturers. This helps ensure that when an encoder produces a VVC stream, it plays without a hitch on all available players, simplifying the deployment for the content publisher.
All of this is the plumbing that I referred to; as VVC is incorporated into more standards, it means more compatible devices in the future and faster deployments. The simpler it is to deploy VVC from a testing and compatibility perspective, the faster and more profitable the rollout. These are great harbingers, but only that.
The VVC Royalty Structure
The next speaker was Robert J.L. Moore, a patent attorney from Volpe Koenig, who discussed the two VVC-related patent pools, the major patent owners that are not in a pool, the royalty structure, what’s subject to a royalty, and what’s not (see Figure 2). Moore said that, as with HEVC, the vast bulk of the royalties will be paid by device manufacturers. As it relates to content, the Via LA pool, formed by the merger of MPEG LA and Via Licensing, does not charge for content. While the Access Advance pool does charge for VVC-encoded content, it’s only on content shipped on physical media, like a disk or thumb drive, which is the same schema Access Advance uses for HEVC. If you’re streaming VVC-encoded video, there’s no charge, even for pay-per-view or subscription.
Figure 2. The two VVC pools and their respective royalty structure (from Robert J.L. Moore)
Moore also delineated the various pricing tiers for the Access Advance pool and pointed out that Access Advance pool licensees that also license VVC will pay only 25% more for devices that use both codecs. In addition, Moore pointed out that large VVC IP owners like Qualcomm aren’t in either of the two pools and license their IP bilaterally with device manufacturers. This schema, in which the largest IP owners license directly, while smaller IP owners join pools, is common for most technology standards.
Moore concluded by talking about the merger between MPEG LA and Via Licensing, noting that the ownership of the combined Via LA was very similar to that of Access Advance. As reported, he said there is some speculation that Via LA may dissolve its VVC pool to enable the market to move toward a single VVC patent pool owned by Access Advance.
VVC Playback Status
Next, I spoke about VVC playback support in the three primary markets: computer, mobile, and the living room (bit.ly/VVC_Playback). Within the context of my neighborhood example, I should have spoken last, as playback support represents the finished homes that publishers move into. Without playback support, all of the plumbing in the world won’t convince a publisher to deploy a new codec.
Here, the news is less optimistic, although not surprisingly so. Specifically, the VVC spec was finalized in July 2020; about 3 years later, there have been no announced computer or mobile CPUs, GPUs, or SoCs with VVC playback or encoding. Given the development cycle for chips and devices built around them, this means that it’s likely we won’t see our first computer or mobile device with VVC until mid- to late 2024 and won’t see an addressable market for hardware playback until 2–3 years beyond that.
For perspective, AV1 was announced in April 2018, and in January 2023, just fewer than 5 years later, a ScientiaMobile study reported that mobile hardware support for AV1 was still under 3%. Unfortunately, it takes a long time for hardware support to reach critical mass, and, if anything, VVC seems to be running behind AV1.
Despite the lack of hardware support, several software companies, including Kwai, ByteDance, Tencent, and MX Player, seem to be delivering VVC to mobile devices for CPU-only playback. Figure 3 shows the impressive playback statistics published by Kwai and ByteDance in January 2022. Note that both companies own significant patents contributed to VVC, so they are incentivized to accelerate its deployment.
Figure 3. Impressive playback statistics on mobile devices from two VVC stakeholders, Kwai and ByteDance
In addition, note that MX Player and MX TakaTak from India were reportedly delivering VVC streams to up to 20% of their installed base. Neither company is listed as a significant IP owner, so the motivation to deploy VVC might be just for the bandwidth savings. Whatever the motivation, these reports do confirm the statements from InterDigital’s Stein about relatively efficient VVC playback.
Fortunately, the integrated circuit picture was much brighter in the living room, where there are multiple existing hardware decoders from MediaTek, Realtek, and VeriSilicon, and, in fact, several TVs shipping in 2023 with VVC decode (see Tommy Flanagan’s excellent article). As happened with HEVC, we may see VVC widely deployed to the living room before being targeted for computer and mobile playback.
The other point about the living room is that it’s much easier to deploy a new codec when your service installs the set-top box that controls playback. So, it’s possible that we’ll see some greenfield IPTV VVC deployments long before we see it used for general-purpose deployments to mobile devices and computers. These may look a lot like some of Ateme’s VVC trials that are described later in this article.
Encoding and Packaging VVC for VOD
This leads us to our next speaker, Igor Oreper, chief architect at Bitmovin, who covered encoding and packaging for VOD. (His presentation is at bit.ly/VVC_File_Package.) Oreper started by identifying some of the VVC encoders that had been recently announced and/or were available. He mentioned two open source versions, one each from Fraunhofer and the JVET standard VVC development team, and commercial encoders from MainConcept, ByteDance, Tencent, Alibaba, Ateme, and Bitmovin.
Oreper related that Bitmovin based its cloud implementation on the Fraunhofer encoder, which I reviewed in 2021. He shared the chart shown in Figure 4, comparing the quality of VVC (in red), AV1 (in blue), and HEVC (dotted line in gray). The HM and VTM encoders are the reference codecs produced by the standards development teams. They implement all available tools at their highest quality settings to represent the maximum quality available for that codec.
Figure 4. Available VOD encoding options for VVC (from Igor Oreper)
As you can see by the legends in Figure 4, lower means better quality, and left means faster. The chart positions both versions of VVC as slightly faster and of meaningfully higher quality than AV1 and slower than x265 but with much higher quality. The VVenC encoder comes impressively close to the maximum quality represented by the VTM reference encoder, with a much faster encoding time. To achieve these results, Oreper reported that VVC’s compute needs were 8x more than AVC, 4x more than HEVC, and 2x more than AV1, which is an indication of where pricing will be for cloud services and how costly VVC will be to produce in house.
He concluded by noting that packaging VVC for distribution will be very similar to AVC and HEVC, with both the ISO-BMFF (aka MP4) and MPEG-2 TS formats available along with the AAC and MPEG-H audio codecs. DASH and HLS adaptive bitrate options should also be similar, using the same formats for captions and subtitles and the same options for encryption and DRM. Taking into account the standards and testing status mentioned by InterDigital’s Stein, as well as these packaging considerations, deploying VVC once the target audience exists should be a breeze.
Live VVC Encoding
The next speaker was Jan Outters, director of technology and standards for Ateme, which started broadcasting live events with VVC back in 2020. He began his talk by discussing what’s involved in implementing a new codec. As you would expect, when you have a sophisticated encoding tool like Ateme’s TITAN encoder, much of the work previously performed to pre-process, analyze, and apply rate control carries over to the new codec. Interestingly, Oreper had the same sentiment, commenting that when Bitmovin dropped the Fraunhofer VVenC encoder code into its cloud encoder, “It just worked.” This indicates that for software-based encoders, live or VOD, implementing VVC should be straightforward.
As is often the case, VVC’s live results didn’t quite match those produced for VOD encoding, with one slide showing that VVC produced an “Overall practical gain of 20% compared to HEVC for 8K.” Outters then reviewed the various VVC trials that Ateme has helped produce, including the first-ever VOD broadcast trial in June 2020; the first-ever live, low-latency trial in November 2020; and the more recent French Open trial in June 2022, which involved Qualcomm, France Television, LG, Orange, Enensys, and Viaccess-Orca.
More recently, Ateme produced the 2022 World Cup trial with Brazil’s TV 3.0. In this broadcast, Ateme combined 4K HDR VVC encoded video with MPEG-H audio into a 15Mbps stream distributed in MPEG DASH format and decoded on LG television sets.
In one of his later slides, Outters showed the headwinds that might accelerate European VVC deployments and the tailwinds that might slow them (see Figure 5). On a positive note, he mentioned strong interest from DVB members and a “leap” in efficiency delivered by VVC. Slowing deployments is the dominant installed base of HEVC and AVC decoders along with continued increases in HEVC efficiency. He also noted several relatively recent or planned deployments that involved HEVC or even AVC. As previously mentioned, although the implementation of VVC seems inevitable, that doesn’t mean we’ll see much progress in the short term.
Figure 5. Headwinds and tailwinds impacting VVC deployment in Europe (from Jan Outters)
VVC and HDR
The final talk of the day was by Patrick Griffis, VP of technology, standards, and industry engagement at Dolby, who discussed VVC and HDR. (See Patrick's presentation here.) Griffis started by reviewing the origins of HDR, with an in-depth approach that included how our eyes perceive luminance and color. The final takeaway was the data shown in Figure 6, which ranks the benefits of the available quality “improvements” by the actual impact on the human visual system and cost in pixels. As you can see, increasing the dynamic range has the most impact and is tied for the lowest bitrate cost, with increased color slightly behind a faster frame rate in impact, but far more efficient from a data rate perspective. In short, the two elements of HDR, color volume and dynamic range, deliver the most bang for the buck in terms of noticeable improvements, which is why HDR is such a priority with VVC.
Figure 6. Ranking quality improvement approaches by data rate cost and visual impact (from Patrick Griffis)
A later slide states, “VVC was designed from the start with both SDR and HDR in mind. ‘HDR is now a first-class citizen.’” In essence, this means that HDR will be a feature of VVC players from the beginning.
Key Takeaways
Here are the key takeaways from all of the presentations:
- VVC should deliver more than 40% greater efficiency than HEVC and 10%–15% greater efficiency than AV1 for VOD deployments. These numbers will be lower for live deployments.
- The plumbing has been well laid for VVC deployments, which should be relatively smooth from an implementation perspective.
- The market for mobile and computer playback with hardware probably won’t be large enough for most publishers to pay attention to until 2026 or later.
- Software VVC playback is relatively efficient and may be an option for publishers targeting either computer or mobile devices. Historically, however, only codec stakeholders like Meta have pursued this option; most independent publishers wait for hardware support.
- The living room is a good year or two ahead of mobile/computers and VVC launches with extensive HDR support. It looks like VVC will succeed sooner in the living room than for general-purpose deployment.
What About AV1?
It’s tempting to view the codec world as a zero-sum game—if VVC wins, AV1 loses. But that seldom is the case. Alliance for Open Media (AOMedia) members own the four major operating systems (Windows, Mac, iOS, and Android) and all of the major browsers (Chrome, Edge, Safari, Firefox). They are massive in computer and mobile chips (Intel, AMD, NVIDIA), mobile devices (Apple, Google, Amazon, Samsung), the living room (Samsung, Apple, Amazon, Google, Roku, Microsoft, LG), and content (YouTube, Netflix, Amazon, Meta). Notwithstanding Apple’s head-scratching lack of support for AV1 and the significant pool of VVC-related patents, AV1 isn’t going away soon.
VVC has a 10%–15% efficiency advantage and inclusion in many TV standards, but AV2 is coming. Even in the living room, where VVC stakeholders seem to outmuscle AOMedia members, particularly in China, and VVC’s HDR advantages really shine, Figure 7 shows why consumer product manufacturers selling into the living room must support VP9 and will likely end up supporting AV1. Call it the YouTube effect.
Figure 7. Why Apple supported VP9 and will ultimately support AV1
The table in the figure shows how YouTube encoded the Top Gun: Maverick trailer, which has accumulated more than 30 million views. As you can see in Figure 7, if you upload a 4K video, YouTube caps the H.264 encodes at 1080p and encodes at higher resolutions in either VP9 (all UHD videos) or VP9 and AV1 (the most popular UHD videos). So, if you manufacture smart TVs, gaming platforms, set-top boxes, or other OTT devices, you must support VP9 or AV1 so your buyers can see Tom Cruise’s famous smile at full resolution. This was the leverage that finally pressed Apple to support VP9.
It’s unlikely that YouTube will abandon VP9 encodes anytime soon, but at some point—perhaps after its second-generation ASIC-based Argos encoder with AV1 comes online—YouTube may start to cut back on VP9 encodes, perhaps encoding up to 2K in VP9 and only AV1 beyond that. So, it’s likely that AV1 support will continue to expand.
Besides, AV1 has a healthy head start over VVC in the living room, although the AV1-related HDR is still unproven. Overall, looking ahead, it’s hard to predict anything but a two-codec world: one standards-based, the other from AOMedia.
Related Articles
When it comes to making roadmap decisions about adopting emerging codecs for media & entertainment content delivery between now and 2030, what do the numbers say, and how has the forecast changed since the pandemic shook up the streaming market? In this in-depth exclusive interview with Streaming Learning Center's Jan Ozer, Rethink Technology Research Senior Analyst Alex Davies discusses the findings of Rethink's new M&E Codecs Market Forecast.
17 Jul 2023
Anytime you start working with a new codec, there are some fundamental tests that you should run to achieve optimal performance/quality optimization. In this article, I'll take you through those tests while encoding VVC using a version of FFmpeg that includes the Fraunhofer VVC codec.
21 Mar 2023
With the recent launch of version 1.0, SVT-AV1 appears to have caught up with libaom in quality, with very definite performance advantages. Its two-pass rate control is tested and proven. If you're creating an AV1 encoding workflow today that emphasises encoding speed and quality, SVT-AV1 should definitely be on your short list.
08 Sep 2022
Companies and Suppliers Mentioned