Qualcomm’s push into the IoT space continues, ramping up from the recent announcement of the Snapdragon 820E series with the new Vision Intelligence Platform (VIP). The new designs are hoping to steal a lead in the machine-vision market, giving eyes to devices that are early adopters of the technology, and hopefully entrenching Qualcomm’s position in the nascent sector.
The QCS605 and QCS603 are the first two members of a new ‘purpose-built’ IoT family of processors. Using a 10nm FinFET design process, the chips have been designed for on-device camera-based machine-learning applications. The goal here is to give a device’s eyes the brain power to compute what it is seeing, within a power budget that suits IoT devices.
For devices hooked up to a mains power supply, power budget is not such a pressing concern. For mobile devices, battery life is one of the most important design considerations, and so Qualcomm is hoping to build a platform that provides energy-efficient processing that can entice developers – trading on its reputation in smartphones.
Drones are a particularly interesting area for Qualcomm, but the VIP package has been designed to serve consumer smart security cameras, wearable cameras, VR and 360-degree cameras, robotics, and smart digital displays. Two companies, KedaCom and Ricoh Theta, have been announced as the first customers for the VIP chips, for their security cameras and 360-degree cameras, respectively. Qualcomm adds that reference designs for these cameras will be available in the second half of 2018.
Qualcomm has also been fostering the development ecosystem, on the software side of things. It has roped in SenseTime for face, image, and object recognition, and Pilot.ai for action and object detection and classification, meaning that other developers should be able to build on these
The two new SoCs use Qualcomm’s latest image signal processor (ISP), and its AI Engine (which includes the Snapdragon Neural Processing Engine), as well as a computer vision SDK for building applications with. The ARM-based CPU, combined with a Qualcomm GPU and its ISP provide a heterogeneous compute environment, which should provide a decent level of adaptability, when it comes to new machine-learning workloads.
Qualcomm is looking to stay ahead of the competition here, before its rivals catch up in a race to the bottom. Qualcomm has long held a reputation as a master of the premium end of the market, but in high-value applications like computer vision, its R&D prowess should let it take an early lead in new applications – before cheaper rivals pour in.
The QCS605 is based around an 8-core Kryo 360 CPU, up to 2.5GHz, with an Adreno 615 GPU, and the Hexagon 685 Vector DSP. For connectivity, the SoC includes 2×2 802.11ac WiFi (peak speed of 867 Mbps apparently) and Bluetooth 5.1 (which should support Bluetooth Mesh). Some hardware-based security features have also be added, with root-of-trust (RoT) and trusted execution environments (TEE), on top of secure boot. These should all really be standard fare for IoT devices, these days.
For video processing, Qualcomm says VIP can handle 4K video at 60 fps, or 5.7K resolutions at 30 fps, or multiple video feeds at sub-4K resolutions. The aforementioned ISP that is handling this workload is the Spectra 270, a dual 14-bit processor that can manage ‘staggered HDR’ – a technique that helps avoid ghosting artifacts in video streams.
There is a heap of fancy audio technologies on board too, which are vital to capturing an accurate input for use in natural language processing. The voice services that are natively supported are Amazon’s Alexa Voice Services, Microsoft Cortana, Baidu’s DuerOS, and Google Assistant.
In terms of software and framework support, the VIP includes Caffe, Caffe2, TensorFlow, Open Neural Network Exchange (ONNE), the Android Neural Networks (ANN) API, and Qualcomm’s Hexagon Neural Network library. In the current configuration, Qualcomm says that the VIP system can delivery 2.1 TOPS of compute performance for DNN tasks at 1W power consumption, which it says is more than double that of some leading alternatives. Of course, such benchmarks are highly subjective, when it comes to choosing comparisons.
“Our goal is to make IoT devices significantly smarter as we help customers bring powerful on-device intelligence, camera processing and security. AI is already enabling cameras with object detection, tracking, classification and facial recognition, robots that avoid obstacles autonomously, and action cameras that learn and generate a video summary of your latest adventure, but this is really just the beginning,” said Joseph Bousaba, VP Product Management at Qualcomm Technologies. “The Qualcomm Vision Intelligence Platform is the culmination of years of advanced research and development that brings together breakthrough advancements in camera, on-device AI and heterogeneous computing. The platform is a premier launchpad for manufacturers and developers to create a new world of intelligent IoT devices.”