Qualcomm has launched a software developers kit (SDK) for its Neural Processing Engine (NPE), a system for running machine learning functions on its Snapdragon processors, and has signed Facebook up as an early adopter.
Machine learning and AI-based technologies are among the growth areas which Qualcomm has identified to diversify its business model amidst smartphone market slowdown, a string of legal attacks, and a prolonged approvals process for its proposed takeover of NXP.
While NXP will help it expand in automotive and IoT spaces, Qualcomm is also augmenting existing platforms, as the NPE SDK indicates. The SDK is compatible with Qualcomm’s Snapdragon 600 and 800 system-on-chip families for mid- and high-tier mobile devices. It allows developers to pick the preferred processor core for their application, from the Kyro CPUs, Adreno GPUs, or Hexagon DSPs. The SDK has been optimized for the more common deep learning frameworks, such as Caffe, Caffe2, and Tensorflow.
Available via Qualcomm’s Developer Network, the kit is pitched at developers in the mobile, automotive, healthcare, security and imaging industries – anywhere with a requirement for on-device processing, which the chip giant hopes to meet via Snapdragon. It is therefore building up the processors’ capabilities in ML-enabled computer vision, natural language processing, object tracking and avoidance, and gesture and text recognition.
As for early adopters, Facebook has apparently confirmed plans to integrate the NPE SDK into its smartphone application, to boost its augmented reality features, which are based on Caffe2 (open sourced by Facebook). The social media giant says this will provide a fivefold performance improvement by harnessing Adreno GPUs, instead of relying on a generic CPU implementation, providing a smoother AR experience.
The goal is to get more out of the mobile performance while keeping energy consumption low, essential in a handset or battery-powered IoT device. This is where Qualcomm’s CPU offload approach kicks in. On the Snapdragon 835 SoC, the chip vendor claims that the ML performance is eight times as energy efficient and four times as powerful on the Adreno GPU, compared to a generic CPU approach, while the Hexagon DSP boasts improvements of 25 times in energy efficiency and eight times in performance.
Based on those figures, the NPE SDK could free up CPU resources, helping the central processor to remain in lower power consumption states while the ML functions are in use, on the GPU or DSP.
In addition, Qualcomm is providing the tools that will help to create the trained models that will power the ML applications, and a ‘model conversion’ tool which will allow an existing model to be adapted to meet a new requirement, without having to train the specific model from scratch again. This should save a lot of development time, supporting both convolutional neural networks (CNNs) and customized layers.
These models are highly specialized systems for spotting patterns in data – whether that’s audio files, images, or text. When the models begin their training, they are very inaccurate, but as they handle more data, they get better. The ML element comes from having a machine validate the model’s findings, rather than a human, which drastically improves training times.
Eventually, just the model, and not the massive accompanying data set, can be exported to the end device – able to run on optimized hardware and software, instead on a massive cloud-based cluster. The training process, according to Qualcomm, begins with training the model, converting it to a Deep Learning Container (DLC) format, adding the NPE runtime code to the mobile application, and then loading the model onto the end device – with only that last piece taking place on the Snapdragon SoC.
“We developed the Snapdragon NPE SDK to help meet the increasing demand for running deep neural networks efficiently on mobile and other edge devices powered by Snapdragon,” said Gary Brotman, director of product management. “The Snapdragon NPE was created to give developers the tools to easily migrate intelligence from the cloud to edge devices, and affords them the flexibility to target the appropriate Snapdragon core with the power and performance profile for a given user experience.”