Nvidia catches ambiguous AI upscaling fever with third-gen Shield TV

It seems a lifetime since video aficionados became as animated about a set top as industry news outlets have about the new Nvidia Shield TV, despite the third generation Android TV streaming device leaning heavily on a taboo term – 4K upscaling.

Upscaling HD video to 4K is par for the course in most modern 4K TV sets, yet it isn’t uncommon to hear 4K upscaling referred to as fake 4K, However, we feel the practice of upscaling content to 4K is becoming more accepted in the industry with benefits to various parties involved in the chain.

Nvidia claims its own method of upscaling uses artificial intelligence, although the company has been naturally vague about the AI element here, describing only “computational prowess” trained on a deep neural network, allowing Shield TV users to upscale 720p and 1080p video streams in 4K resolution. Nvidia did not respond to Faultline’s queries about whether this process uses proprietary algorithms on the new Tegra X1+ processor or whether Nvidia is sourcing these AI upscaling algorithms from a third-party vendor.

This may well use AI capabilities that Nvidia has previously applied to its GPU architectures for gamers, specifically one called Deep Learning Super Sampling which can render input images into high-resolution outputs, with the model trained with rendered frames and intermediate buffers.

In the meantime, while upscaling is common practice particularly among the South East Asian TV manufacturers, the term AI upscaling is something we first heard closer to home, from UK compression vendor V-Nova when it introduced machine learning algorithms to its Perseus codec in mid-2018. These algorithms work using a technique called neural network convolution, which are trained to identify edges and curves by looking at millions of hours of video then sharing this across its various video layers.

This all combines to reduce residual data requirements for reconstruction by identifying where boundaries between the 12 or 14 layers should be for the lowest bandwidth usage. In simple terms this is AI upscaling plus fewer residuals to equal near-perfect reconstruction.

Huawei for example describes “real” 4K UHD as a resolution of 3840 x 2160 at a frame rate of 50fps, rather than being mastered in 2K or 1080p then upscaled to 4K.

However, we recall a conversation with Akamai when the CDN heavyweight slammed the practice, telling Faultline that upscaling content to 4K on the device itself ultimately hinders its ability to sell more bandwidth. Surprisingly then, rival Limelight Networks argues the opposite, in the past praising upscaling for freeing up bandwidth.

Upscaling HD to 4K is seen as perfectly acceptable for elements of the entertainment ecosystem like the UI itself, but for the resolution buffs among us, upscaling has been frowned upon and the fanatics will argue that an upscaled 4K picture isn’t even close to resembling the quality of a pure 4K delivered image.

The self-confessed prowess of the new Tegra X1+ processor, which delivers up to 25% more performance than its predecessor, has also allowed Nvidia to cater for Dolby Vision and Dolby Atmos.

Nvidia has been highly regarded as one of the best – if not the best – manufacturer of high-end Android TV devices since the first Nvidia Shield TV launched back in 2015, specifically targeted at the gaming community – and the California-based company has really turned up the heat with the latest eye-catching tubular model.

Android TV devices at retail, such as the Nvidia Shield TV and Amazon Fire TV Stick, represent a chaotic market with many variants, not all of which access the Google Play Store and some of which are illegal, like certain Kodi devices with piracy add-ons.

But Nvidia has had to react to changing tides elsewhere. The company has been under mounting pressure from Intel in the gaming sector, although we have noted in the past how the likes of Nvidia cannot afford to bring out chips dedicated to AI given the scales they are seeking, and so generally silicon firms focus on chips that are competitive across a wide range of use cases including gaming.

The standard third-gen Nvidia Shield TV costs $149.99, while the Pro version is priced at $199.99, which gets you additional memory and storage, plus two USB ports for running a Plex Media Server and hooking up high-capacity hard drives. Both come with a Google Assistant-powered voice remote with a dedicated Netflix button.