Close
Close

Published

Quality vs speed – AV1 3,000 times slower than HEVC, claims report

Rivals of the AV1 video codec finally have something to cheer about, after a study found that AV1 was between 2,500 and 3,000 times slower than its competitors. HEVC supporters won’t be shouting the news from the rooftops just yet, however, as the trial states the AV1 encoder was not optimized and we suspect running an outdated version – meaning more processing cycles are required to paint an accurate picture of AV1’s potential.

Moscow State University recently conducted an updated codec comparison introducing high-quality encoders including AV1 and VP9, unsurprisingly finding that AV1 produced superior quality over its predecessor VP9 and main competitor HEVC. Yet research concluded that the AV1 encoder produced the lowest speed of all 12 encoders tested, followed by x265 Placebo presets.

The question therefore for AV1 adopters is one of quality versus speed. Clearly AV1 is not the final, well-polished item and some additional development work is required, but sending less data is more important than less processing cycles in the encode. In addition, there is no evidence provided (in the free teaser report) on which version of AV1 has been used in conducting the research, so there is a good chance the latest version would not be as slow as these results suggest.

AV1 has been put head to head with open source and reference codecs by Moscow State University, including some niche software from China, which do not provide an accurate representation of performance in the real world, up against high-efficiency and high-quality codecs used by major players in the streaming industry.

Nevertheless, the arrival of AV1 into the mainstream is approaching fast and delays have been rife. Pressure is mounting for the Alliance for Open Media’s (AOMedia) royalty-free standard, although it may take a while for all devices to support AV1 – perhaps years. The codec is built on Google’s VP9’s codec with assistance from coding tools including VP10, Cisco’s Thor and Mozilla/Xiph.org’s Daala.

Other evidence has shown AV1 churning out performance improvements of between 20% to 30% over HEVC, although this has been disputed as not based on a sufficient number of video instances or devices.

Moscow State University’s table below shows that AV1 produced the highest quality output, at 55% of the average bitrate, while x264, an AVC encoder developed by VideoLAN, produced similar quality at 67% and 69% of the average bitrate. Streamingmedia cites that in an independent third-party test, using settings supplied by the vendors, VP9 produced higher quality than HEVC on 31 HD video streams.

Previous tests between HEVC and VP9 have produced very similar performance results in terms of bandwidth savings, and Mozilla reported back in November that AV1 slashed file sizes by between 25% and 35% compared to HEVC and VP9.

The exact definition of quality being measured in pre and post filtering by Moscow State University is complex and may vary from study to study. It outlines that quality is measured using its own measurement tool, analyzing 16 different metrics including Brightness Flicking, Drop Frame, Noise Estimation and Scene Change.

Moscow State University applied a technique it developed in 2016 for test dataset sequences selection, designed to create datasets containing representative real-life sequences for encoders. It analyzed over 512,000 videos hosted on Vimeo looking for 4K and Full HD videos with high bitrates, selecting 50 Mbps as the lower bitrate boundary. Considering the global average internet speed is 7.2 Mbps, with the US in 10th position at 18.7 Mbps, selecting 50 Mbps as the lower boundary is unrealistic.

The comparison was performed on an Intel Core i7-6700K processor at 4 Ghz with 8GB of RAM. For this platform it considered three key use cases with different speed requirements: Fast/High Density (1080p at 60fps), Universal/Broadcast VQ (1080p at 25fps), and Ripping/Pristine VQ (1080p at 1fps). The full report from Moscow State University, which is dated 2017 but includes updated 2018 results from AV1 tests, can be purchased on compression.ru.

Close