It was inevitable that the media would focus on the AI aspects of Qualcomm’s just-launched Snapdragon 855 mobile chip and, of course, on headline performance improvements over predecessors, but the greatest interest for us lay in its vision processing features. These promise to give smartphones the kind of video manipulation and enhancement until recently confined to high end cameras costing up to $100,000, well beyond all but the most affluent or dedicated consumers.
Qualcomm should not get too much credit, since advanced image and video processing features are also being implemented by other chip makers. These include Nvidia, Huawei and also Apple whose latest A13 chip is being made exclusively by Taiwan Semiconductor Manufacturing Company (TSMC). There is a trend towards incorporation of advanced features under the banner of AI or machine learning into more general-purpose chips, just as parallelism became intrinsic to GPUs and before that math functions in general purpose CPUs. In fact, the likes of Nvidia cannot afford to bring out chips dedicated to AI given the scales they are seeking and so focus on chips that are competitive across a wide range of use cases including gaming.
One headline is that the Snapdragon 855, which in fact is also being fabricated by TSMC, but not related to the Apple A13, uses the latest 7nm FinFET (Fin Field-effect transistor) process, against 10nm for the preceding Snapdragon 845. This is responsible for perhaps a third of the power reduction achieved over its predecessor the SD845, which Qualcomm claims is as much as 75% which would be a big deal for smartphones where battery life is such a choke point.
We were amused by several reviews complaining that the 5G modem in the chip is only optional, which is surely a benefit since it keeps costs down for handsets sold in regions where 5G or something resembling it will not be available for years. More to the point is that the chip has an integrated 4G modem, called X24, which is faster than its predecessor and able to connect at up 2 Gbps, which is more than anything billed as 5G will need in the foreseeable future.
But the most interesting part of the chip is the new Spectra 380 camera ISP (image signal processor) combined with the 690 DSP (digital signal processor), which together provide the foundation for the new video processing features. These include the ability to shoot 4K video at 60fps in HDR with support for 10-bit color and HDR10+. This compares with basic HDR10 and lower frame rates for the 845 and here we can confirm that the results are pleasing to the eye.
Naturally, even more powerful video processing features are supported by higher end chips, such as Nvidia’s latest GPU, the Tesla T4 chip unveiled in September 2018. This is more dedicated to AI than before and 12 times faster than Nvidia’s preceding P4 chip at the half precision FP16 floating point arithmetic relevant for many AI calculations. With this chip, Nvidia is really targeting consumer applications for the first time, with technologies originally optimized for high performance industrial and commercial computing. A notable image and video processing feature on the chip is support for ray tracing, which has great potential for 3D graphics, VR and AR, as well as image rendering and enhancement.
Optical ray tracing is a technique for simulating 3D images by following the path of an imaginary ray of light across the field until it encounters an object such as a ball. At that point, the system will calculate what would happen to the ray of light according to factors including the angle of incidence and the nature of the material to determine the degree of reflection, absorption, refraction and in rare cases fluorescence where light is remitted at a different wavelength, usually longer.
The benefit lies in being able to simulate visual artefacts such as shadows and reflections much more realistically than any other known algorithmic method, while the fact each ray can be analyzed independently lends itself to parallel processing. The latter is vital, as although optical ray tracing is computationally extremely intensive, there is scope for making it run efficiently given the right hardware and especially chip design. That is why Nvidia has incorporated dedicated ray tracing processors in the T4, as part of its new Turing architecture, which succeeds its preceding Pascal architecture.
The overall thrust is that advances in video processing have become a major driver for chips down to the consumer level in smartphones, tablets, PCs and gaming consoles.