HLS Now Supports Fragmented MP4, Making it Compatible With DASH
Apple today announced the inclusion of byte-range addressing for fragmented MP4 files, or fMP4, allowing content to be played in HLS without the need to multiplex it into the traditional MPEG-2 Transport Stream. Compatible with newer versions of its operating systems (macOS, tvOS, and iOS 6.0 or higher) this new fMP4 feature makes HLS compatible with the industry standard MPEG-DASH for browser-based HTML 5 playback.
After a session on fMP4 and HLS at today's Worldwide Developers Conference (WWDC), Apple posted the draft specifications. Industry reaction has been positive.
DASH and HLS: A Look Back
Just under a year ago, the Bitmovin team announced the release of Bitdash 3.0, which enabled Apple HLS delivery inside Bitmovin’s HTML5 and Flash video player.
With Bitdash 3.0 came Apple HTTP Live Streaming (HLS) native support, which meant HLS was no longer tied to Apple devices but could be used on any web browser.
Still, for all that integration between Flash and HLS in a Flash-powered player, there was one weak spot in any integration of MP4-based playback coupled with HLS playback: the Achilles' heel known as MPEG-2 Transport Stream (M2TS).
M2TS—a transport stream technology harking back to an era where ATM meant asynchronous transfer mode, not automated teller machine—used a packetization technique that carved a video and one to two audio channels into small segments that could be transported over intermittent delivery methods such as ATM or satellite uplinks.
Packets had a program ID (PID) and enough information to allow re-assembly of the video and audio channels at the far end, even if packets arrived out of order.
That technology served broadcasters well when they had only one resolution and a few alternate audio channels (think SAP), but has been poorly suited to scaling to the myriad of resolutions, bitrates, and multi-lingual requirements of a global internet television audience for OTT.
The main reason was multiplexing, the act of pairing a single resolution with a discrete audio stream. Add to this the complexity of Adaptive Bitrate (ABR) with its 5 to 7 resolutions and 10 to 20 audio tracks for various languages, and the complexity creating that many permutations of video per second easily expands logarithmically to a big hairy mess.
To address this issue, using standard MP4 video files and AAC or MP3 audio files, the MPEG committees set about creating an ABR solution that used what the industry calls "late binding," where a manifest file (an intra-video playlist of sorts) eliminates the need to pre-segment content.
Using byte-range addressing, these MP4 and audio files would remain intact, and the segmenting would be virtualized using a pointer that delivers up a small fragment of one file at a particular bitrate or resolution, followed by a small fragment of another one of the MP4 and audio files if network bandwidth between the end viewer and the hosted content (most often on a CDN) increased or decreased enough to warrant sending the next segment at an equivalent bitrate. This ABR solution from MPEG is known as MPEG-DASH.
Late binding works well for these standalone files, and the term fragmented MP4 has been used for years now (starting around the time I wrote a joint whitepaper for Adobe and Microsoft on the benefits of fragmented MP4).
In that white paper, I mentioned that M2TS, as used in HLS, didn't scale well for an internet-sized audience (this was before the term OTT, or over-the-top delivery, became popular). Regardless, there's always been a performance hit when the short segments of a video file needed to be multiplexed into an MPEG-2 Transport Stream.
Today's News: Why Fragmented MP4 Support Matters
Solving for this is why today's announcement that Apple will update HLS to include the use of fragmented MP4 files is such a big deal.
Not only will it allow late binding and eliminate the need for multiplexing, it also has the potential to marry the industry's standards-based DASH players with the de facto HLS delivery standard.
Now before we all do the happy dance, here's a word of caution from the team at Bitmovin:
"Apple still forces content providers to make use of HLS streaming on iOS for assets longer than 10 minutes," writes Reinhard Grandl, solutions architect for Bitmovin, in a blog post.
Still, he notes, fragmented MP4 inclusion can be a huge advance from an encoding perspective:
"Up to now, it has been necessary to encode content in different formats to maximize browser coverage and reach most of today’s end-user devices. This increases the storage footprint of your content by 2x and reduces also CDN efficiency as content cannot be effectively reused across devices."
A 2x increase in storage footprint is putting it nicely as most content owners face a 5x increase due to the possible permutations, so fragmented MP4 will save a significant amount in storage costs.
In addition to storage savings, there's also transport cost savings, with fewer files needing to transit the CDNs internal network to be stored at closer caching points.
In addition, the workflow savings are equally large, as the just-in-time approach offered by late binding means that files can be output from a non-linear editing system (NLE) as MP4 without further encoding or transcoding required.
This frees the NLE up to do more paid editing work, which means additional indirect cost savings.
In other words, HLS using fragmented MP4 is a big deal, not only technically, but also for your bottom line.
Related Articles
The upcoming third edition of DASH will address several missing features, says a Comcast principal architect, and will drive down live video latency.
01 Jun 2016
At Streaming Media East, Iraj Sodagar, president and chairman of the DASH Industry Forum, explained what's coming with version 3.0.
27 May 2016
The industry is turning away from plug-ins and embracing HTML5 everywhere. Here's how the vendor-independent streaming standard is gaining momentum.
29 Mar 2016
Companies and Suppliers Mentioned