This week we came across another specialist which claims that it has a system for cutting 50% of video bandwidth using simple “bit rate reduction” algorithms.
At the past two IBCs we met with Beamr, an Israeli system, which relies on trying to emulate perceptual filters. It takes ingested video and has perfected the identification of tiny changes it can make to pixels, typically to make two pixels close together the same, which will result in a file which compressed down further. Each change has to pass the test of whether or not a human could notice the difference.
After these adjustments are made, the frame is sent to the encoder and encoded and savings of between 35% and 50% were experienced in the final delivery bit rate.
Faroudja Enterprises is trying something very similar but does this in a slightly different way. It’s essential patent is called a method for multidimensional video processing, and works in three dimensions, left to right, a line at a time, and from frame to frame – a bit like most codecs.
Essentially the pre-processing tries to reduce the amount of data that is encoded in an initial signal and sends separate image data in parallel, also going through the encoder and being placed into the stream and later that that data is added back in. This means that a specialized player has to be built which can place this data back where it belongs in a post processing phase.
As far as we can work out it works by taking away half the lines in a screen, uses some algorithms to estimate what must be there, and transmits only the differences between the estimate and what was actually there.
The patent talks about separating horizontal high and low frequencies from the video signal using low-pass filtering, and outputting an information-reduced signal. Then the filtered component generates a second supporting signal, which tells the remote devices whether to add frequencies to the output and to what extent.
It also deletes every other line in the vertical direction and a similar filtered signal is sent, which can add them back, and then again every other frame is also treated in the same way.
It looks to us like the codec only has to encode half the frames and only about 25% of the data in each frame, for the weaker of the two signals, but the penalty is this separate parallel data stream with all the detail.
Effectively this is taking a non-standard, dynamic encoding process that takes about as much effort to produce as another H.264 encode pass, and adding it as a pre-processing stage. After that the data that is extracted, essentially the “fine detail” of the image, are sent separately, both encoded.
“The gains from this are additive to the codec and we are working towards demonstrating this with an HEVC codec. So far we have compressed a 4K 30 fps file using an H.264 encoder down to about 8 Mbps,” said Bill Herz, Chief Operating Officer and VP of Business Development, for Faroudja. As a comparison Netflix offers a similar format at 15 Mbps.
The company said that it had just been granted three more patents around this process which improve the system’s efficiency (called F1) without reducing perceived quality. It works with any codec the company said and was first introduced a year ago.
This system could be used in a number of ways, and that includes preserving devices which are already out there. Yes it could eventually preserve bandwidth, by being included in chips which are being readied for handsets and tablets by modern chip vendors. But equally, if you can download an app with the post processing element on it, and place the pre-processing system into the cloud just ahead of the encoder stage, then this could be retrofitted to existed systems and save half of video bandwidth.
Founder and President Yves Faroudja, told us, “The decompression can be placed into a plug in for a media player and it will work on an Android device that is 3 or 4 years old and there is no significant increase in the battery usage for the process.”
The two separate streams can also be used to simultaneously deliver a 4K program in both 1080p and 4K resolutions, with no extra bandwidth cost and each device could choose which stream to use.
Herz added, “This is not an end user, consumer play. This is for large content developers like HBO and Netflix. It can also be used for reducing bit rates in teleconferencing.
“We are running the pre-processing on a Xeon server with an Nvidia GPU and then it was able to keep up with an H.264 real time encoder. Once it is ported to Intel I7 servers we will be able to move to HEVC encoding.”
In effect we think this technique is very similar to that at Beamr, but instead of trying to emulate a group of expert viewers and working out what they can notice, Faroudja has tried to create two layers of detail in two different data streams and organized them in such a way that they compress more naturally. But it admits that it tests the system in much the same way – a group of viewers who try to see if there are any artefacts which draw their eye to them.
When he addressed the SMPTE Faroundja said, “that there are a number of techniques which do not relate to compression that can improve video,” and he listed, noise reduction, non‐linear edge enhancement, line multiplying, scaling, de‐interlacing, reverse telecines, super‐resolution, chrominance manipulation and chroma enhancement. And he added, “It seems that the eye naturally gravitates towards the worst of these effects.”
His philosophy seems to be by all means change the video but don’t draw attention to the changes and make it easier to compress.