SSIMWave – remember that name. The Canadian start-up, a specialist in perceptual video quality measurement, first drifted onto our radar in summer 2018 lauding the world’s most widely-used QoE algorithm, but still to be told SSIMWave has virtually every tier 1 North American operator either currently evaluating or already trialing its SSIMPlus technology left us atypically lost for words.
The main marketing clout behind the technology is claiming to be the only software to break through the 90% correlation accuracy mark between compute objective and human subjective content. But such is the complexity of algorithms developed in its SSIMPlus Structural Similarity technology (we’re talking neuroscience level intricacy here) that it’s perhaps simplest to imagine the software as encompassing the accuracy of hundreds of thousands of human viewers – data which is then used to apply scores to video content. As explained to us this week by SSIMWave’s VP Marketing Saj Jamal, the scoring system is as follows: Excellent (81-100), Good (61-80), Fair (41-60), Poor (21-40), and Bad (1-20) – whereby a score of 80+ is generally considered the equivalent of an HD TV broadcast.
SSIMWave is essentially as close as the industry is right now to putting the physiology of the human eye into technology which can ultimately be used to improve QoE. In fact, the company isn’t really a start-up after all, with the core algorithms some three decades in the making, emerging first from research conducted at the University of Texas and then through the University of Waterloo, pioneered by Professor Zhou Wong. It was released into the open source community in 2004 and eventually SSIMWave was formed in 2013. However, while the open source version remains widely used, we must hit home that SSIMPlus today is a paid product, which is why there are a number of contentious conclusions from trials – most famously from Netflix.
This related to Netflix testing the original 2004 algorithm at scale but concluding that it didn’t quite capture human visual perception accurately enough, resulting in Netflix collaborating with academic circles to improve its own open source technology called VMAF (Video Multi-method Assessment Fusion).
“SSIMPlus is a paid product but it’s winning, so we must be doing something right. The main benefit of SSIMPlus over VMAF is that VMAF can’t do live and there is still huge money to be made in live streaming, so we don’t see Netflix and VMAF as a competitor,” said Jamal. Well, as long as Netflix remains in the SVoD game then SSIMWave can’t view Netflix as a customer either, so we’ll place it somewhere around the middle of the spectrum considering SSIMPlus does both live and VoD.
The essential idea of VMAF is that individual quality metrics, such as visual information, detail loss and motion (temporal difference between adjacent frames), only give partial measures of perceptual quality, but when fused together can yield a more accurate “master metric”. Netflix has applied conventional machine learning to assign weights to different metrics and tune the model until it matches human perception as closely as possible.
Importantly, Jamal made the point that SSIMWave is not in the QoE game in the traditional sense our readers will be most accustomed to, but what he described as “true QoE”, in doing so dismissing QoE vendors in fields like buffering or stalling duration as “pseudo QoE”. Our natural response in defense of QoE frontrunners like Conviva and Nice People at Work is that they would not take kindly to such potentially damaging pseudo remarks. Although, admittedly, it’s not quite that simple.
“They don’t like it that’s for sure. But some of these QoE companies have approached us and admitted to not having the best QoE software, so we are working with three or more at a time, never partnering with just one,” added Jamal.
In fact, SSIMWave can go much deeper than QoE metrics like bitrate, frame rate and network conditions; it can score factors based on specific device brands and models, be it size of the screen, distance from the screen (based on sound), brightness, number of pixels, and it can even acknowledge that viewing video on a smartphone likely means the handset is tilted at approximately a 30-degree angle – factoring this into its decision making. Device manufacturers are now asking SSIMWave if this QoE viewing data is important to carriers. The answer, of course, is yes – very.
Vendors such as Interra Systems and Eurofins Digital Testing continue to use the open source technology from 2004, which really is testament to the technology standing the test of time.
For now, SSIMPlus can only talk publicly about Telefonica, where it remains in the trial stage with the Spanish operator for now – assessing just what SSIMPlus is capable of. Jamal expects something to happen “quite soon”, although as far as specific territories or services go, SSIMPlus doesn’t quite have those details nailed down yet.
SSIMWave is also a Dolby partner licensed for monitoring Dolby Digital Plus for Live and VoD workflows – collaborating on monitoring the preservation of Dolby Vision and Atmos content across distribution.
Research conducted recently by Akamai posed the question of what “good” actually looks like in terms of streamed video? The CDN giant worked with Eurofins to examine various influencing factors to perceived video quality, including the encode profile, video device or player, content viewed, and typical network conditions. When introducing SSIMPlus software to complex encode footage, played through a range of devices at different bitrates, researchers were surprised to find results showing significant variation in the bitrate required to deliver a HD image (SSIMPlus score of 80+) to different devices at the same network conditions.
It gives the example of two smartphones achieving optimal perceptual quality scores with bitrates of less than 3 Mbps but having significant perceptual quality variation between them. Smart TVs, meanwhile, required encode profiles over 6 Mbps to reach a similar SSIMPlus score.
The significance and implications of all this for OTT video distributors, according to the researchers, is “seeking to optimize video content for a range of different devices and players, where cost of creating and storing multiple encode profiles for each content asset is significant.”
That said, being future proof is another of SSIMPlus’ pivotal marketing exploits. “The physiology of humans has to fundamentally change in order for the technology not to work,” is how Jamal described it – a point which will be impossible to argue with should even a handful of these tier 1 operator trials come to fruition. Still though, SSIMWave admitted a major D2C deployment would be a much bigger story – perhaps Disney+ or WarnerMedia?