Video: How Do We Define Quality of Experience for Streaming Video?
Learn more about real-world video metrics at Streaming Media's next event.
Read the complete transcript of this clip:
Ribal Najjar: What are quality of experience metrics? It's the degree of delight or annoyance of a user while watching. In this use case, while watching a video stream, it also touches on their perceived improvement or degradation, so yet another layer is pretty hard to measure and try to get a sense of what it is. At the end of the day, it's the customers’ or the viewers’ satisfaction with the media experience.
It's obviously pretty hard to try and fetch a metric that gets to measure that. Thankfully, there are a bunch of working groups across the industry that are doing a lot of work trying to pull some information from the player and analyze and study what the customer's experience is.
So for instance, rebuffering ratio, which I single out here--there's a whole bunch of other metrics. Rebuffering ratio is the measure of the percent of the time that you're sitting watching you video and you're seeing the little wheel turning around and the video loading. So, if you watched a 99-minute movie and you spent a minute looking at the wheel, the spinning wheel that's about 1% rebuffering ratio, to give you an essence. So industry standard is that 1%, that's bad, more than that, that's really bad. So some work has been done to try and measure what’s the right threshold for taking this metric and trying to simulate what the customers experience is.
Obviously, the customer’s experience is going to vary from a person to another and from scenario to another. If you're watching on your mobile, you might have a different set of experience if you're watching on your 4K 70" TV you have some high expectations. So things vary. But let's assume and all agree that you don't want to look at more than a minute of this spinning wheel while you're watching a 99-minute movie.
So, that’s a starting point right, and hat will change and probably at some point in the future we won't be okay with two seconds or five seconds or maybe some of us aren’t already.
Reuffering ratio is a metric that we base the machine learning algorithm on. So I single it out here.
Other metrics are things like how long did the video take to start. If you're sitting there waiting for 10 seconds that might not be acceptable to you, and so on.
So these are some of the definitions. So there's some challenge in trying to bend those out and produce them, trying to quantify them to measure the customer experience. But that's a separate engineering task that has to be studied, analyzed or maybe it's beyond engineering as well.
Related Articles
Limelight's Jason Hofmann, Citrix' Josh Gray, and REELY's Cullen Gallagher discuss best practices for training AI systems at Streaming Media East 2018.
12 Nov 2018
Google's Matthieu Lorrain cautions of the risks of doing AI for its own sake in this clip from Streaming Media West 2018.
08 Nov 2018
RealEyes Director of Technology Jun Heider discusses the importance of internal self-assessment and which use-case elements to consider when choosing a platform for video AI in this clip from Streaming Media East 2018.
01 Nov 2018
RealEyes Media Director of Technology Jun Heider identifies the key players in the AI platform space in this clip from Streaming Media East 2018.
29 Oct 2018
RealEyes Director of Technology Jun Heider outlines the first steps in choosing an AI platform in this clip from his presentation at Streaming Media East 2018.
25 Oct 2018
Microsoft Principal Product Manager Rafah Hosn makes the case for reinforcement learning as a machine learning paradigm for content personalization in this clip from Streaming Media East 2018.
22 Oct 2018
Microsoft Principal Product Manager Rafah Hosn discusses the benefits and limitations of a content personalization strategy based on supervised machine learning in this clip from Streaming Media East 2018.
18 Oct 2018
Microsoft Principal Product Manager Rafah Hosn explains how Microsoft's machine learning-driven decision services helps brands target viewers and increase engagement in this clip from Streaming Media East 2018.
15 Oct 2018
Comcast Technical Solutions Architect Ribal Najjar discusses how operationalizing commonalities between QoE and QoS metrics to deliver a "super-powerful" dataset in this clip from Streaming Media East 2018.
11 Oct 2018
IRIS.TV CEO & Co-Founder breaks down discusses IRIS.TV's approach to helping traditional media companies capture and leverage audience data and machine learning in this clip from Streaming Media East 2018.
04 Oct 2018
Gannett Senior Director Kara Chiles discusses how USA Today leveraged IRIS.TV and data to localize and personalize their Winter Olympics 2018 coverage in this clip from Streaming Media East 2018.
01 Oct 2018
ZoneTV's Tom Sauer describes how machine learning can be used to overhaul the TV world and deliver more individualized experiences in this clip from Streaming Media East 2018.
27 Sep 2018
REELY CEO Cullen Gallagher makes the business-growth case for content owners developing an AI strategy in this clip from Streaming Media East 2018.
24 Sep 2018
IBM Watson Media's David Clevinger discusses how media entities are currently using video AI in this clip from Streaming Media East 2018.
20 Sep 2018
Citrix Principal Architect Josh Gray explains how video enables higher-acuity metrics analysis in this clip from Streaming Media East 2018.
17 Sep 2018
Limelight VP of Architecture Jason Hofmann discusses how AI impacts content delivery optimization in this clip from Streaming Media East 2018.
13 Sep 2018
Citrix' Josh Gray provides tips on AI model development and Reality Software's Nadine Krefetz and IBM's David Clevinger speculate on the possibilities of metadata-as-a-service in this clip from Streaming Media East 2018.
10 Sep 2018
A working group overseen by the CTA is creating recommendations for measuring performance quality, and some of the biggest names in the industry are participating.
07 Aug 2018
Google's Leonidas Kantothanassis explores the vast range of applications for machine learning in the media workflow and supply change in this clip from his Content Delivery Summit keynote.
19 Feb 2018