Pay TV players will often drum up the consistent reliability of cable and satellite video services as the one field where IP-delivered video will forever be in traditional pay TV’s shadow. New data argues otherwise, however, showing that smartphones produce fewer video playback errors than set tops – devices which are purpose-built for the task.
Handset makers and mobile streaming vendors shouldn’t celebrate just yet though, as a couple of factors played into the hands of flattering smartphone results published by Spanish video analytics outfit Nice People at Work this week.
Taking a leaf from the Conviva rival book of analytics, NPAW has published a report detailing a bunch of juicy findings collected by its Youbora platform during the first half of 2019. The H1 2019 Global Streaming & Broadcasting Report shows that set tops created the most errors per play with a score of 0.181, compared to smartphones which came out ranked top with only 0.068 errors per play, as displayed in the chart below.
However, the data may be skewed in favor of smartphones on the basis of the volume of live content going through set tops, compared to mostly VoD content on smartphones. To that end, NPAW data shows that live content produces on average 0.136 errors per play, while VoD content incurs just 0.054 errors per play, meaning viewers are more than twice as likely to see errors such as buffering during live content.
Not a huge surprise, but then NPAW confuses the scenario somewhat with a separate measurement of “interruptions” which it defines as different to errors, where live content scored better than VoD.
NPAW defines these interruptions by the number of buffer events that occur, irrespective of the length of buffer time, while the earlier “errors” are more serious playback issues such as crashes which indicate faults within the video delivery system itself. Findings show that VoD content saw 0.52 buffer interruptions per play session, while live content saw an average of 0.32 interruptions per play. Conclusively then, live content is more susceptible to serious errors, while buffer interruptions are a common caveat for VoD content, which to an extent helps explain why set tops ranked so poorly.
In terms of frequency of these less serious interruptions by device, tablets performed strongest with 0.4 interruptions per play, while consoles scored highest with 1.9 interruptions per play, as shown in the second graph here. The data shows that typically more expensive devices like PCs and consoles are being out-performed in video streaming QoE by tablets and smartphones. Interestingly though, set tops and TVs – the dedicated video devices – are looking distinctly average. NPAW notes that the great contrast in play quality demonstrates that the infrastructure for live broadcasting is improving in real-time environments, although smartphones and tablets are usually replaced with newer models more frequently than hardware like set tops.
Just to the throw the cat among the pigeons, despite smartphones showing low levels of errors and interruptions per play, they have one of the lowest video completion rates at just 26%. If three quarters of content is never completed in the first place, then naturally there will be fewer errors and interruptions – so another factor in the supposedly strong performance of smartphones.
Tucked away towards the end of the paper is an almost contradictory statement from NPAW. “Many streaming services are offering cross-platform synchronization to allow users to move from their PC or TV to their mobile, however, it seems as though the infrastructure to offer a mirrored quality of experience isn’t there. On the other hand, despite the unpopularity of consoles (2%), their average effective playtime (23.6 minutes) and average completion rate (55.6%) was the highest of any device. TV also performs very well when it comes to keeping eyes glued to screens with an average effective playtime of 21.3 minutes and completion rate of 42.5%.”
An abundance of metrics within the video delivery ecosystem makes analytics a particularly tough game and reporting it in a simplistic manner is even trickier, yet NPAW has unfortunately crossed a few wires here and lacks the finality of an opinionated conclusion which is almost always absent from vendor-authored reports.