Qualcomm goes heavy on AI and extended reality with Snapdragon 845

Qualcomm has unveiled its latest flagship system-on-chip (SoC), the Snapdragon 845, focusing on enhanced performance for ‘extended reality’ (XR) and for on-device artificial intelligence (AI), as well as a new Secure Processing Unit (SPU) for “vault-like” security features.

The SoC is being linked to the upcoming Samsung Galaxy S9, and Qualcomm will hope to reinforce its reputation of industry leading performance, as it battles to fend off Apple’s lawsuits and Broadcom’s hostile takeover attempts.

To support its claims to be creating a full AI platform, the chip supplier also announced a strategic alliance with Chinese search engine Baidu, to create an AI voice solution based on the new Snapdragon combined with Baidu’s DuerOS conversational AI system. The technologies will be optimized for one another, as the basis of a voice-activated smart assistant solution similar to Apple Siri, Amazon Alexa and Google Assistant.

“We are committed to the development of on-device AI, including audio,” said Keith Kressin, SVP for product management at Qualcomm. “The collaboration with Baidu will bring AI for voice-enabled solutions to next generation Snapdragon mobile platforms, allowing users to wake up smartphones and IoT devices using their voice in their natural language at any time and at extremely low power, while utilizing Baidu’s DuerOS voice services.”

The DuerOS is already incorporated in about 100 branded products targeted at the Chinese market, and numbers 130 supporters. It is included in devices from handsets and smart speakers to set-top boxes, refrigerators and air conditioners.

The Baidu deal is part of Qualcomm’s aggressive moves into on-device AI, but in other areas of emerging technology, it still needs to make more progress, especially in the Internet of Things. Cameras, gateways, drones and smart home devices are all promising new avenues for its processors, but Intel lurks at every corner, and a plethora of other ARM-based chips are lined up to bid on every RFP. Its expansion in the automotive market, via the proposed purchase of NXP, has also turned into a headache amid regulatory delays and the uncertainties created by the Broadcom bid.

Nonetheless, the spec sheet for the Snapdragon 845 sound impressive. Qualcomm claims 64x HDR capture for its camera, which now supports the full 2020 spec for UHD video. This level of video performance is being sold as ideal for augmented, mixed and virtual reality, both for phones and for powering head-mounted displays (HMDs). Qualcomm says the 845 is the first mobile platform to enable room-scale ‘six degrees of freedom’ (6DoF), for more immersive XR experiences.

On the AI processing side, this is the third generation of Qualcomm’s AI mobile platform, and it claims a threefold improvement in overall performance compared to the previous SoC, the 835. The first generation, which introduced the Neural Processing Engine (NPE), an evolution of Qualcomm’s Zeroth project, was the Snapdragon 820.

For smartphones, improved AI means better digital assistant performance, better image capture and better XR experiences, but Qualcomm doesn’t want to limit itself to handsets. For some time, it has been pushing the Snapdragon family as a platform for network edge devices, ranging from gateways to drones, which can be used to capture and process IoT data and devices, out in the field.

Edge processing will be required to reduce the volume of data that needs to be streamed back to a central cloud application – a volume that will generate a rather expensive data bill at the end of a month, potentially negating the entire business case. As such, having the ability to pre-sort or filter that information could prove extremely valuable for a developer, but the key is to have that performance inside a power package that is achievable on something like a drone or robot. This is where Qualcomm’s mobile experience should come to the fore – although Intel is definitely looking to do the same sort of thing.

To this end, the 845 SoC will support Google’s TensorFlow and Facebook’s Caffe machine learning frameworks (both open source), with the NPE software developers’ kit (SDK) also now supporting TensorFlow Lite and the new Open Neural Network Exchange (ONNX) specification, enabling Caffe2, Microsoft’s Cognitive Toolkit (CNTK), and MxNet. Also supported is Google’s Android Neural Network API.

The heart of the 845 is the Kryo 385 CPU core (based on the ARM Cortex-A75), an eight-core unit that houses four heavy-lifting 2.8 GHz cores and four smaller 1.6 GHz cores to save power consumption during less intensive processes. Qualcomm says the 2.8 GHz clock speed is 25% higher than its previous generation, and it has also increased the amount of L3 and system cache to help increase throughput. The CPU is built using a 10nm Low Power Plus (LPP) FinFET process.

As for the GPU, the 845 uses an Adreno 630, which claims a 30% improvement in graphics and video rendering. It also features ‘Adreno Foveation,’ a display output technology that claims to be able to support two 2K displays (2048 x 1080p) at 120 Hz, with low power requirements – which sounds ideal for VR headsets.

Foveation, as the name hints at, tinkers with the field of view, allocating less rendering power to tiles of the screen that are not in view of the user. The Quick Charge 4 support will prove invaluable here, for topping up those batteries, as they get depleted by the heavy workloads in VR apps – which aren’t going away, even with the addition of Foveation.

The integrated Hexagon 685 digital signal processor (DSP) is being pitched at AI and imaging tasks, using the Hexagon Vector DSP (HVX) and Qualcomm’s All-Ways Aware sensor hub to make sense of the analog world – as the DSP is responsible for taking inputs from the Analog-to-Digital Converter (ADC) and turning them into something that can be used by the system. For voice and sound applications, core parts of many AI projects, high performance DSPs are vital.

As for image capture, the Spectra 280 Image Signal Processor (ISP) features active depth sensing, noise reduction, and something called ImMotion (sic) Computational Photography – which helps to improve the clarity in an image’s fine detail when a user zooms in on it, partly by using on-device machine learning tricks to filter noise and smooth out the details. Qualcomm says the Spectra 280 can capture 16MP UHD (3840x2160p) images at 60fps, with slow motion capture of 720p at 480fps.

The active depth sensing is being pitched at machine vision systems, but Qualcomm hopes that the advanced camera systems it is providing will be used for things like biometric authentication (iris and facial scanning), as well as for straightforward image capture – for video that might then be subjected to further machine learning processing, in applications other than taking the perfect selfie or plate of food on a smartphone.

Connectivity is handled by a Snapdragon X20 LTE modem, supporting LTE Cat 17 (1.2Gbps), License Assisted Access (LAA), CBRS shared spectrum, and DSDV (Dual-SIM Dual VoLTE). For WiFi, the SoC houses an 802.11ad diversity module, and has 2×2 802.11ac as its main connection. The system also supports Carrier WiFi (802.11k/r/v), and the latest Bluetooth standard, Bluetooth 5.