How to Make the Move to HEVC
While few companies of any kind are actually making money from HEVC (H.265) today, the successor to H.264 will become increasingly important during the next 2–3 years, perhaps even earlier in some markets for some producers. So understanding the current status of the technology and how to encode and potentially deploy HEVC in the near term is very relevant for most streaming media producers. Accordingly, in this article, I’ll review the state of HEVC and take a high-level look at the first generation of HEVC encoders.
Let’s start with where we are and what we know.
Current Technology Status
As an overview, HEVC offers two key advantages over the reigning codec king, H.264: quality equivalent to H.264 at half the data rate and the ability to deliver 4K video, or Ultra HD video (UHD). The first advantage can be used with existing HD and lower-resolution content, whether to save bandwidth costs or to deliver higher-resolution video over bandwidth-restricted connections. As such, it potentially impacts nearly every streaming video producer and all companies in the distribution food chain.
The second advantage, Ultra HD video, is totally dependent upon a number of factors that most streaming producers can’t control, from the availability of UHD content to how quickly consumers start buying 4K TV sets. The ability to distribute UHD video will beneficially impact only a small percentage of streaming producers in the foreseeable future. For this reason, I’ll focus this article on HEVC’s initial advantage: essentially H.264 quality at half the bitrate. So the first topic to address is whether HEVC actually delivers on this promise.
HEVC Quality
That turns out to be a pretty interesting question without a simple answer. Before addressing this question, you should understand a couple of key points. First, few, if any, vendors have incorporated all of the encoding algorithms and techniques available in the HEVC spec into their existing HEVC codec. Second, even once the specification is fully implemented, codec quality tends to improve over time, as the engineers and programmers optimize their codec-related algorithms and techniques.
So, while H.264 has probably reached most of its potential quality-wise, HEVC is just getting started. Therefore, the HEVC quality we see today doesn’t represent HEVC’s full potential, and certainly not the quality it should achieve when you actually implement it down the road.
That said, where are we today? This depends. I looked at three test cases -- 640x360x29.97@700Kbps (H.264)/350Kbps (HEVC), 1280x720x29.97@400Kbps/800Kbps, and 1920x1080x29.97@2Mbps/4Mbps. For each of the technologies, I evaluated the quality of full-motion playback and multiple still-frame comparisons.
In the 640x360 comparison, HEVC did not deliver equal quality to H.264 at half the data rate. Specifically, H.264 retained more detail and showed very few motion-related artifacts, while HEVC exhibited significant mosquitoes, or hovering artifacts, around large regions of moving content, such as a banjo player or dancer. In the 720p comparisons, HEVC frequently, but not always, showed better detail retention than H.264 at half the data rate. It also exhibited more motion artifacts, again in the form of mosquitoes.
The most favorable test case was the last, at 1080p. Here, the 2Mbps HEVC file won most of the still image comparisons while the mosquitoes evident in the first two comparisons were no longer obvious, though this test involved a different test clip than the first two. In some panning shots, however, the 2Mbps HEVC clip showed some tearing, which was offset by judder in the 4Mbps H.264 clip (Figure 1).
The obvious lesson is that HEVC will deliver more bandwidth savings at larger resolutions, and you should expect this dynamic to continue. Looking forward, during the next 12 months, as vendors fully implement the HEVC feature set and continue to improve their code, I would guess that 720p will be the tipping point at which HEVC will deliver equivalent quality at half the bitrate of H.264. At smaller resolutions, you should expect less savings, with increasing savings as resolutions increase.
When you make your own HEVC/H.264 comparisons, remember to assess both still-frame and real-time playback quality. Judging from the admittedly small sample of test clips that I reviewed, low bitrate HEVC streams showed more motion-related artifacts than H.264. Perhaps this relates to my test clip, which involve lots of stage performances by musicians and dancers. In particular, the mosquitoes surrounding these moving performances were the most noticeable artifact exhibited in the HEVC encoded clips. If you often encode similar content, make sure your test comparisons contain this type of footage.
Where Will It Play?
Quality is great, of course, but if the video won’t play at full-frame rates, no one will care. Let’s consider computers and notebooks first, and then segue into mobile. Table 1 shows the CPU required to play back H.264 and HEVC video at 720p and 1080p resolutions on some of the lower-power computers in my office. Note that I played all files using the VLC Player.
As you can see, playing back the 720p HEVC videos was well within the reach of all tested computers. Since Intel’s Core 2 Duo started shipping right around the time I acquired the Precision 390 in 2006, this means that there’s a huge installed base of computers capable of 720p HEVC playback. On the other hand, it looks like a four-core computer will be necessary to comfortably play back 1080p video, which eliminates much of that huge installed base. That said, judging from the tests and surveys that I’ve performed, very few producers outside of YouTube stream 1080p video to computers, so the inability to play back 1080p may not be a significant negative for general-purpose streaming.
Issues are a bit more contentious in the mobile space. In a frequently cited report titled “HEVC Decoding in Consumer Devices,” senior analyst Michelle Abraham from the Multimedia Research Group estimated that the number of consumer devices shipped in 2011 and 2012 that were capable of HEVC playback with a software upgrade totaled around 1.4 billion, with more than a billion more expected to be sold in 2013.
While Frost & Sullivan analyst Avni Rambhia doesn’t necessarily disagree with those numbers, she identified a big “yeah, but” in her Streaming Media West 2013 presentation, “Cutting Through the Hype of HEVC (H.265).” Specifically, not counting PCs, Rambhia estimates that more than 1 billion HEVC-capable devices will ship in 2014 (Figure 2). However, since fewer than 5 million of those will include HEVC playback hardware, Rambhia feels that the power consumed during HEVC playback makes HEVC untenable.
In her presentation handout, Rambhia states, “Mobile use of HEVC hinges upon silicon. If you are not truly mobile, then HEVC is not a game changer for HD content delivery. The 1 billion capable device statistic is true, but essentially irrelevant to HEVC uptake in the short term.” Later in the presentation, Rambhia predicts, “HEVC and AVC will run as parallel, complementary technologies until late 2014. HEVC will gain momentum starting in 2015, particularly with telepresence, mobile OTT, and UltraHD. AVC will endure as a mainstream format at least until 2018, and remain a significant format well beyond 2020.”
Today, many producers prepare separate streams for mobile and desktop delivery, so it’s not unreasonable to anticipate that some producers will encode with HEVC to harvest bandwidth savings in streams delivered to computers while continuing to use H.264 to distribute to mobile. However, If you’re hoping that HEVC will enable the delivery of HD video to mobile devices, you may have to wait until hardware-accelerated playback, or much higher capacity batteries, are the norm.
What’s the Status of Playback?
Another obstacle to using HEVC for general-purpose streaming is the lack of a pervasive player. For example, H.264 was not widely used for streaming until Adobe incorporated H.264 playback into the Flash Player. Though Rovi boldly shipped DivX 10 with HEVC encode/decode in September 2013, with VideoLan adding HEVC to the VLC Player 2 months later, even combined, these players don’t offer the penetration necessary for publishers to deploy HEVC for general-purpose streaming.
Battery life issues aside, HEVC won’t be relevant in the mobile space until playback is incorporated into iOS and Android. Sure, in both the computer and mobile space, you could build a custom player that supplies HEVC decode, but that alternative is unavailable to all but the largest producers. Basically, general-purpose HEVC deployments are on hold until Adobe, Apple, and Google add HEVC playback to their respective players and operating systems.
What About Royalties?
In January, 2014, MPEG LA, the licensing group that administers the largest H.264 patent group, and has formed a similar group for HEVC, announced its proposed royalty policy for HEVC. Briefly, the proposal is publisher friendly, with no proposed royalty on HEVC-encoded video, even that distributed via pay-per-view or subscription. In contrast, after certain minimums, there is a content-related royalty on H.264.
On the other hand, the terms or more expensive for encoder/decoder vendors. Both H.264 and HEVC excepted the first 100,000 units, and both charge $0.20 per unit after the first 100,000. However, with H.264, the royalty dropped to $0.10 after 5 million units and is capped at $6.5 million through 2115. The HEVC royalty never drops, and is capped at $25 million.
The MPEG LA announcement made clear that not all companies with HEVC-related intellectual property joined the group, so there is a possibility that these terms may change, or that other IP owners may attempt to enforce their own royalties. Still, these terms are likely close to final for the MPEG LA group, and you’d have to assume that any proposed third-party royalties would be in the same ballpark or less. MPEG LA hopes to announce final terms in early 2014.
Related Articles
Which codec delivers better image quality? Which is more compatible? And what about Daala, the spoiler codec currently being developed from scratch?
09 May 2014
Elemental says HEVC gains now approach 50 percent for high-definition content, but the codec's rollout and growth will take a decade.
27 Jan 2014
The news is good for content owners and not so good for encoding/decoding vendors, but gives everyone a feel for how the issue will play out.
17 Jan 2014
Streaming Ultra HD video to the home will require serious compression. Look for the HEVC codec to make the next jump in video resolution possible.
30 Dec 2013
Companies and Suppliers Mentioned