Implying a fruitful first quarter, Synamedia has committed to increasing R&D around content-adaptive encoding – technology which the company sees as key to carving a path into emerging markets like ATSC 3.0, 4K devices and onto 8K over-the-air delivery.
To us, it looks as if Synamedia’s refined dedication to the content-adaptive encoding cause has been driven by – and perhaps even funded by – AOMedia itself.
Content-adaptive encoding is a term we most associate with specialists in adaptive bitrate techniques like Bitmovin and Beamr, while Synamedia has been hovering around the concept of content-adaptive encoding for a while, without explicitly committing to the term until this week. We said as much at IBC 2019, when Synamedia’s Smart Rate Control software was spruced with intelligent encoding algorithms designed for minimizing bandwidth requirements from expensive CDNs.
So, now we know that Smart Rate Control is, as we suspected, based on content-adaptive encoding, Synamedia’s compression team will begin supporting newer codecs in research, including AOMedia’s AV1 (which Synamedia signed up for only days ago) and the MPEG Versatile Video Coding (VVC) codec.
That said, AOMedia is pushing more high-level content-adaptive encoding as one of the primary enhancements to AV2, as well as building on features already supported in AV1, such as motion warping, which allows for changing shapes and perspectives of objects such as animals or human figures as they move from frame to frame.
Further to this point is Synamedia’s mention of using neural networks in its content-adaptive encoding exploits. Neural networks are among the AI techniques referenced in AV2 literature, designed to analyze video more deeply at the object level and increase computational power.
These neural networks break objects into constituent parts – down to tiny, intricate levels of detail if the resolution is sufficient. Another technique coined “neuro fuzzy segmentation” aims to build geometrical relationships between objects which can be encoded for further efficiency, knowing for example where a nose is relative to eyes.
This involves firstly clustering an image into groups of similar pixels in perhaps 4×4 blocks and then assembling them into larger areas. Then detection of objects takes place by analyzing these clusters, identifying faces by picking up on their characteristic values for chrominance or color and limited range of luminance or light intensity. Finally, refinement is applied to help identify objects in more unclear areas of a frame, such as the edges where only part of an image such as a face is showing. This principle can be extended across time so that relationships between objects within the content can be exploited for motion prediction across a number of frames.
Also known as content-aware encoding, this class of techniques improves coding efficiency by exploiting properties of the content to reduce bitrate. For example, content with little motion and static images is encoded using few bits, while content with high-motion scenes is encoded using the necessary bits for quality reproduction.
If we look at Beamr’s CABR for example, which was introduced a decade ago in 2009 and has come on leaps and bounds since, this technology optimizes on a frame by frame basis using a closed loop quality measure to ensure that the original quality is always preserved. For live 8K tests, Beamr told Faultline last year that it didn’t even turn on its CABR system, as this consumes additional CPU resources. But Beamr expects to be able to utilize CABR in the near future as it further optimizes Beamr 5 for 8K live encoding.
Even without CABR, Beamr insists there is room for refinement of the encoding architecture at 8K. The higher the resolution, the greater the relative scope for video compression, especially when there is a large amount of spatial or temporal redundancy. Beamr has claimed bandwidth savings ranging from 20% to 40% with a combination of HEVC and content-adaptive encoding, compared with HEVC alone.
Synamedia, meanwhile, said it has achieved bitrate savings of 10% in early content-adaptive encoding test results. Faultline told Synamedia this seemed low compared to other bitrate saving results and we are awaiting an explanation.
Ateme is another encoding specialist worth keeping a close eye on concerning content-adaptive encoding. The French firm recently won a grant of an unspecified sum for its work on applying artificial intelligence to video delivery. Ateme’s research included bitrate reduction through core encoding improvements, higher video quality using content-adaptive encoding, and optimizing media supply chains and cloud storage via elastic encoding orchestration. As part of the i-Nov Innovation Contest, the IA4SEC project is funded by the French government, starting in January this year.
While new hire announcements are not generally newsworthy on their own, making three new appointments at the same time signals a significant shake-up of the Synamedia executive team – and boy has Synamedia made a statement. As well as poaching Netflix’s Manager of Video and Image Encoding as Synamedia’s new Director of Codec Development, the company has brought in two executives with a combined experience of about 12 years working with Ericsson, Envivio and Mediaroom – which of course comprise the technical foundations that make rival MediaKind tick. It will be an interesting year where this rivalry is concerned.