Your browser is not supported. Please update it.

13 February 2020

Set tops register inexcusable 50% more errors than smartphones

Video streaming, both live and on-demand, is burdened by an unfortunate trend – video quality up, user tolerance down. There is no quick fix but, as a result of streaming advancements together with viewer impatience, QoE video analytics has evolved from a nice-to-have, into a must-have.

At the core of this correlation, is the fact that not only are smartphones and smart TV sets sending video streaming figures off the charts, but are doing so while embarrassing PCs and set tops – which together register 50% more errors per play.

Barcelona-based QoE video analytics vendor Nice People at Work (NPAW) has published its global streaming and broadcasting report for H2 2019, in what is effectively a major marketing campaign for operators to attach a video analytics suite to their footprint of legacy devices, or risk continued subscriber churn on an unprecedented scale. Preferably for NPAW, this would be its very own YouBora platform.

While legacy set tops and PCs (which are replaced far less frequently than smartphones) registering high playback errors isn’t alarming, another standout finding showed VoD content incurring a buffer ratio of almost double compared to live content. Surely, there’s some sort of mistake?

Apparently not, according to NPAW’s VP of Product Solutions, Marc Maycas. “For live content, companies often include lower bitrates in their manifest files to ensure uninterrupted streaming. It makes sense because, due to the ‘here and now effect’ of live streaming, users tend to care more about uninterrupted viewing than image quality. That is why VoD appears to have a higher buffer ratio overall. Some users even force players to start live streams in the lowest possible rendition, which also reduces the join time,” stated Maycas.

However, it is important to consider the breakdown of errors. While VoD content was battered by higher buffer ratios than live streaming during H2 2019, data shows that users are five-times more likely to receive an error during live playback compared to VoD. The causes of this are more obvious – prominently from large traffic spikes caused by a popular live sports event.

But such is the importance of live streaming to viewers, that they will persist through errors – registering a higher overall average effective playtime for live streams (11.5 minutes) compared with VoD content (9.8 minutes).

NPAW defines four types of errors – in-stream errors, in-stream errors crash, start-up errors, and start-up errors crash – each pertaining to a different fault within a video delivery system that can lead to a crash.

Above all else, NPAW sees the exit before video starts rate (EBVS) as the fundamental metric in identifying user frustration and the key to maintaining a high QoE – in which buffer ratio and join time are most closely related. The graph below shows a visual of VoD completion rate versus buffer ratio, where the lowest buffer ratio of 0.2% results in an average completion rate of 68%, which still seems too low. Surely a service provider would want 90% completion rate of VoD content minimum when providing a near-perfect QoE, although it’s interesting that there is apparently no such thing as 0% buffer ratio in NPAW’s data.

Smartphones and TVs each accounted for 23% of viewing time during the second half of 2019, while set tops registered 16%, followed by PCs with 15%, tablets with 7%, and finally consoles with just 1% of view time.

So, more importantly, what can NPAW do about it? During a recent conversation with the second-placed QoE analytics vendor (behind Conviva), Faultline learned about the vendor’s recent successes in securing telco set top deployments, which aligns with the data in its report regarding high set top streaming errors. NPAW has been most successful of late in the area of unifying OTT and IPTV analytics, where telcos have been the principal growth driver.

Meanwhile, the advertising analytics arm of its YouBora suite is seeing unprecedented growth. Other vendors have echoed similar observations. NPAW’s H2 2019 report didn’t include any data from the advertising analytics arm, which hopefully will feature in future versions of the report. What’s clear is that telcos and ads are today’s two main drivers for the QoE analytics industry.