It has always been a long game for the two big standards bodies in the Ultra HD (UHD) arena but now they are raising the tempo as they seek finally to exorcize the ghost of 3D that has been hovering over the field. Both are umbrella bodies aiming to direct development of standards by existing groups such as MPEG, ETSI, SMPTE and the ITU rather than create new ones themselves. The UHD Alliance is concerned with each end of the delivery cycle, video capture and display, while the Ultra HD Forum addresses all the infrastructure in between.
Both were founded in 2015 at a time when memories of the 3D debacle two years earlier were still fresh, when it had become clear that was an abortive effort by the TV industry to generate a market in high end displays and premium services that were literally a pain in the head for consumers.
There was a suspicion that UHD was little more than the next cynical effort to generate similar spurious value. The cause was not helped by early dithering and blundering on the part of the industry focusing too much just on higher resolutions billed as 4K or 2160 x 3840 pixels, which gave little discernable improvement in experience for most content on TV displays smaller than 55 inches.
The definition of UHD was quickly changed to being either 4K resolution, or full HD resolution at 1080p plus some form of HDR (High Dynamic Range). In fact focus groups almost invariably found that consumers reported a significantly better experience from the latter. In other words increasing the contrast ratio delivers a much bigger boost in perceived picture quality for most content than resolutions above 1080p.
The Ultra HD Forum has also admitted it went astray in breaking up emerging UHD standards into phase A and phase B around 2016 because this sowed confusion, especially among smaller operators that had yet to make any move. They were unsure whether to make do with phase A for the foreseeable future and wait until phase B had been thoroughly tested and debugged before deploying it. Phase A was just the 4K, or full HD plus HDR and the option of enhanced WCG (Wide Color Gamut) at the BT 2020 standard defined by the ITU. Phase B would add in object-based NGA (Next Generation Audio), and High Frame Rate at 100 fps or beyond.
Another difference is that phase A just supports static metadata while phase B introduces dynamic metadata as standardized by SMPTE ST2094, designed to future proof HDR so that it will work with legacy TV sets to improve the pictures rather than make them worse. The metadata does this by tone mapping, adjusting on the fly to the changing minimum, maximum and average luminance and gamut requirements for each scene. It ensures displays can still show content as well as possible even when the mastering displays are at or near the 2020 color gamut limits with peak brightness at the maximum 10,000 nits, while the TV sets may be capable of only 1000 nits.
Some of the phase B elements are not yet supported commercially and it will be a few years before they roll out into services. HDR has crystallized into two options, PQ and HLG with dynamic metadata now supported by some services for the former, so that element of phase B is on the way. The Ultra HD forum has been trying to convince us there are no HDR wars anymore and that broadcasters or service providers will be able to choose between the two since they are becoming compatible. A set top can receive content formatted in HLG and display it on a TV set that only supports PLQ. Dynamic metadata is then a layer on top capable of working with either.
NGA is in principle ready to roll after ETSI ratified the DVB’s choice of MPEG-H 3D and Dolby AC-4 as the options for UHD services and products in April 2017 but with full commercial deployments yet to come. HFR on the other hand is being held back by the bandwidth challenge given that it doubles the amount of data over and above phase A UHD at 50 fps. This will also require TV sets capable of changing pixel colors and processing data fast enough, which most cannot at present. LG however now supports HFR on some of its top end OLED TVs at least.
HFR will raise the bandwidth stakes even further and for this reason the Ultra HD Forum is promoting an additional technology as part of its armory, Content Aware Encoding (CAE). We have discussed this before at length, but largely just in the context of on-demand content with reference in particular to Netflix’ gargantuan effort of re-encoding its entire catalog, which is now complete. The Ultra HD Forum is more interested in applying CAE to live content because that is where the bandwidth crunch will be felt most, certainly for OTT services.
By definition the effectiveness of CAE depends on the content type, with far greater scope for additional compression beyond that achieved by a conventional codec such as HEVC or H.264 given slow moving action or lack of contrast. This is the case for talking heads or animations, while fast moving sports and some action scenes in nature programs with high contrast have less scope. But that is the whole point, because even within given sequences such as sports scenes some frames or small groups of frames can be compressed much more heavily than others without reducing overall quality. That was one reason why Netflix decided to re-encode its catalog rather than rely on ABR (Adaptive Bit Rate) key ladders as it had done in the past where each resolution such as HD or Standard Definition is matched to a given bit rate, irrespective of the content type.
This so called static ABR forcing streams into a one-size-fits-all key ladder to encode a variety of content types for different devices resulted in inconsistent quality while failing to make most efficient use of the bandwidth. Netflix realized there was both scope for further bit rate reduction and improvement in quality for fast moving scenes. Now the Ultra HD Forum has come to the same view for live streaming, inspired by developments from three of its members, Harmonic, Brightcove and Beamr. Harmonic is the largest with its EyeQ content-aware encoding software, while Boston-based Brightcove has its Context Aware Encoding technology as part of its online video platform. Beamr specializes primarily in CAE with its content-adaptive bitrate (CABR) technology.
When applied on a frame by frame basis there is even scope for considerable additional compression of high action content by exploiting the fact that it is not all fast moving. Beamr published results of a test in a white paper conducted on ABR streams at 720p resolutions already fully compressed with the H.264 codec for transmission at 2 Mbps in the absence of additional CAE. There were three categories, animation which has greatest scope for additional compression, low action content such as talking heads and high action content. For animations Beamr’s CAE cut the bit rate almost in half while maintaining the same level of perceptual quality according to its formula, down to 1.065 Mbps. For low action content the bit rate came down to 1.275 Mbps while for high action content the reduction was still 23% down to 1.536 Mbps.
Of course applying CAE on the fly to live content requires more computation power and inevitably it cannot then match the compression levels possible for on demand content when there is the luxury of time to make repeated passes across the sequence. For live streams the bit rate savings achievable will depend on the computational resources available and so we would expect performance to increase with time and costs to come down. We have not yet seen any detailed data on this, so will be following up in future issues. The Ultra HD Forum is convinced already that it will make a huge impact which is why it has added it to its list of core technologies. It cannot quite call it another “pillar of UHD” because CAE can be applied equally to any content irrespective of the quality or resolution. But it will be essential during the early days of UHD deployment.
There is also a sub text here, which explains why the Ultra HD Forum, or at least some of its members, are so enthusiastic about CAE. It gives them a stick to beat the upstart compression technology vendors such as V. Nova. With the additional compression achieved by CAE they reckon they can see off the challenge posed by alternatives that already in some form employ content awareness within their primary product.