8K has been so long coming that many have doubted it would ever really arrive, given that at face value it offers little perceptible improvement over 4K with HDR for most content and almost all viewers. However, it has become clear that content shot at 8K downscaled to 4K or 1080p full HD looks better than when created at lower resolutions. This is because it is easier to determine the optimum color and luminance of a pixel when there were originally eight occupying that space, giving advanced algorithms more scope to generate the best picture possible.
Furthermore, there are use cases where 8K is advantageous, such as allowing users to zoom in on pictures to blow up some detail or image. The number of pixels is then substantially reduced so you need a larger number in the first place to sustain even an acceptable resolution. 8K is also proving its worth for still image capture from video sequences, where again the quality traditionally has been nowhere near good enough for any professional purpose. But at 8K, image stills taken from the video are good enough for production in printed magazines. There is also growing interest in use of 8K production for archive material to incorporate some future proofing, providing a source of high enough quality for all conceivable applications at present.
On the transmission front, though it is very early days just 9 months after NHK started broadcasting in 8K for the first time in December 2018 – over 20 years after it began its research program leading towards the introduction of its Hi Vision system around 2010. This will go down as a significant milestone in broadcasting history, coinciding with maturation of various technologies that will make 8K distribution possible, even if not viable yet for mainstream services.
At IBC 2019, Beamr was among vendors peddling 8K-capable technology, in this case its HEVC software encoder, insisting that this too represented some milestone. It did in the sense that it marked the coming together of components in the end to end path, in this case the combination of chip set design and software enabling the parallelism needed to encode live 8K video quickly enough to avoid increasing latency too much.
The Beamr demonstration is worth considering because it raises some interesting points, including the distinction between software and hardware in descriptions of products. The firm’s Beamr 5 encoder was not designed specifically for 8K but for efficient implementation of the HEVC codec for any high-quality video, including 4K and different frame rates, as well as 1080p full HD. For initial tests and deployments, it has been optimized for AMD’s EPYC processors, although there is no reason it could not be for other chipsets. It does though require a high degree of parallelism in the chip to achieve the performance levels required and as Beamr has as good as admitted in its technical literature, the software has been written to match key features of the EPYC architecture.
The starting point of the architecture is that AMD has 64 cores for each CPU that can be plugged into a socket of the hardware device, compared say with 2 or 4 cores for a typical desktop PC, which means that the resulting hardware has very high encoding density. Each core embedded in a single chip is effectively a separate processor or “virtual CPU” capable of executing independent logic threads in parallel if these can be created for an application or process, which is the case for video encoding. Then each of these cores has been designed for as rapid throughput as possible, with large integral memory and short clock times, which means more content can be encoded in a given interval and in practice all the different variants of resolution, the so-called adaptive bitrate (ABR) ladders for streaming, can be created on a single machine.
The Beamr 5 then slots into this architecture by use of micro level parallelism whereby the task is broken down into as small units of each frame as possible, while taking account of the video structure to determine which components can be processed independently. It begins with a two-level motion estimation process where incoming frames are analyzed by the encoder, which determines the scene complexity and calculates initial motion vectors, while estimating with a high degree of precision the number of bits needed to encode that frame. This estimate then guides a second stage where the encoder directs encoding activities to the most meaningful or significant aspects of each frame where there is greatest scope for compression, refining the previous estimate further.
By partitioning the encoding process this way, more unproductive calculations are avoided, and the process is accelerated as a result. The system wastes less power than earlier Beamr codecs and also than other HEVC codecs according to the company, although we cannot verify that. At any rate, the system avoids consuming CPU cycles by repeatedly writing and fetching data to and from memory. Instead, these micro tasks can execute across the whole frame in a well-balanced manner, such that all cores are kept uniformly busy and none are left waiting for the next task.
It is then a data-driven architecture designed to spread out across the whole system architecture, where the data controls the flow of the program, rather than the logic of the task. The alternative is the more widely adopted event driven programming, where the task controls the logical flow of execution rather than the associated data. Data driven approaches work well for tasks that can be split into many components that are temporarily independent, because they can then run in parallel across multiple cores or programming threads. However, for less parallel tasks, data driven approaches would run too slow because the components would be held up waiting for the result of some other computation, even if all cores in a CPU were being used.
So, while in this case Beamr correctly refers to its HEVC encoder as a software product because it is not embedded in dedicated hardware, in its present form this is almost splitting hairs given its dependence on the AMD EPYC. However, Beamr’s VP Marketing Mark Donnigan argues that the software could equally well be deployed on other chips even if performance would depend on the architecture. “There was no special build for AMD and when Intel has a processor with the same number of cores, we are very confident that our results will be the same,” he said.
Certainly Beamr 5’s performance is good, having been evaluated earlier encoding a 4Kp60 clip at the same quality with up to 50% higher throughput compared with a standard H.265 codec of unspecified make, while utilizing only 25% of the CPU resources. The improved throughput meant that it could operate at a higher frame rate for the same quality per frame, say 60 fps instead of 40 fps.
For the 8K demonstration Beamr claimed a world first by encoding video at 8K resolution and 79 fps with the 10-bit color (10-bit 4:2:2) required for HDR in a single EPYC 7742 processor, plugged into just one hardware socket. This was achieved at 60 Mbps, but Beamr’s VP Marketing Mark Donnigan insisted there was a lot more potential to bring this down, not least because the firm did not turn on its Content Adaptive Bitrate System (CABS), which reduces the bit rate when there is not a lot of movement or detail.
Even without CABS, Donnigan told us there was room for refinement of the encoding architecture at 8K, noting here that the higher the resolution, the greater the relative scope for video compression, especially when there is a large amount of spatial or temporal redundancy. At the extreme, a frame where all pixels are the same color can be compressed to the same bit rate at 8K or standard definition. Therefore, this capacity to compress redundant parts almost as much at 8K resolution as say full HD means that, given codecs such as HEVC, the bit rate differential is not as great as might be expected.
Donnigan also pointed out correctly that there is more scope for bit rate reduction with VoD, because there is time to analyze the content in greater depth for more efficient encoding. “But we are focused on live since this is where the demand is for 8K,” said Donnigan. “But soon enough we’ll be demonstrating bitrates that are shockingly low.”
This will be achieved with the help of Beamr’s CABR, which was introduced a decade ago in 2009 and refined since, optimizing on a frame by frame basis using a closed loop quality measure to ensure that the original quality is always preserved. “For the live 8K test we did not turn on our CABR system, as this consumes additional CPU resources, but we expect to be able to utilize CABR in the near future as we further optimize Beamr 5 for 8K live encoding,” said Donnigan. “It’s an extraordinary challenge, as each uncompressed 32 megapixel frame is around 100 MB, so we have 6 GB of raw YUV 420p 10-bit pixels (represented as 16-bit words) per second to encode.”
The principle of content adaptive encoding was exploited in one demonstration by the Ultra HD Forum at IBC 2019 showing footage of an equestrian event with just one horse moving in the background, shot in 8K and yet encoded at 14 Mbps, with no discernible loss of quality. This seems hard to believe when Netflix for example specifies 22.5 Mbps just for 4K, but simply reflects the low level of movement and therefore great scope for compression along the time axis, that is between frames, in this case. The point was to highlight scope for efficient statistical encoding of multiple channels at Variable Bit Rate (VBR) through use of CAE (Content Aware Encoding) along the same lines as Beamr’s CABR.
The Ultra HD Forum staged a masterclass at IBC where some other interesting points emerged from a panel including speakers from five European operators and broadcasters, the BBC, KPN, Mediapro, France Televisions and BT. All agreed with a point we have made more than once, that 4K resolution has much lower “wow factor” than HDR, so that all are now focusing more on the latter.
At the same time, though these five also agreed that HDR was still at an experimental stage and that the hardest part of the value chain to sort out was the end point, whether this is a set top or software in a connected device such as a smart TV. They concurred though that establishment of hybrid workflows incorporating HDR and SDR was no longer a choke point and could be overcome.
Among areas lacking consensus is NextGen Audio, which some operators such as BT are enthusiastic about, having already started delivering Dolby Atmos, while others still question its value. The point is that Next Gen is of little interest to the majority of viewers but of great interest to some higher net worth consumers, especially HiFi enthusiasts, the question being whether it will always be a niche market.
Another area of disagreement was over the bandwidth challenge, with not surprisingly some broadcasters concerned that even 4K still needed over 30 Mbps, although it looks like it will succeed with rather less. But the Forum itself pointed out that CAE has scope to cut bit rate significantly further, harping back to its 14 Mbps 8K demo.
One more point raised concerned emerging standards and how the Ultra HD Forum intends to set profiles recommending particular combinations of the different components. It argues that greater resolutions call not just for HDR but also higher frame rates to deliver the full “wow factor”. For example, while say 60 fps might be recommended for 4K, 128 fps would be preferred to accompany 8K.