Qualcomm launches neural-net SDK, as Apple patent pressure mounts

Qualcomm has launched its Neural Processing Engine (NPE) Software Development Kit (SDK), a system for running machine-learning functions on its Snapdragon range of mobile processing platforms. However, the launch comes as the pressure on Qualcomm builds, as Apple disputes patent royalties it claims it does not owe to Qualcomm – causing quarterly net income to fall 40%.

Those Q3 figures may well get worse in Q4, as Apple appears to be having some success in rallying the industry to sue Qualcomm – with Foxconn notably taking up arms. Recent legal losses to BlackBerry ($940m) and the Korean Free Trade Commission (KFTC – $927m) have savaged its operating cashflow, which fell from $1.8bn in Q3 2016 to just $100m in the most recent quarter.

Qualcomm warns that Q4 earnings-per-share (EPS) will fall by 34% to 41%, with flat year-on-year revenues at best, or a 13% fall at worst. Unlike a technological problem, which could be solved by Qualcomm’s gargantuan R&D efforts, this patent and anti-trust dispute is a very different beast – subject to the often unpredictable legal process.

While Qualcomm’s proposed purchase of NXP drags into its ninth month, a core part of its diversification strategy that would entrench it in the automotive market, Qualcomm has also identified machine-learning and AI-based technologies as lucrative new avenues for it to pursue. The IoT, as part of this diversification, is important, but Qualcomm knows that augmenting an existing core product is a safe bet in these turbulent times.

The new NPE SDK is compatible with Qualcomm’s Snapdragon 600 and Snapdragon 800 range of SoCs – its mid and high tier mobile processor platforms. It allows the developer to pick the optimal processor core for their application; from Qualcomm’s Kyro CPUs, Adreno GPUs, or Hexagon DSPs. The SDK has been optimized for the more common deep learning frameworks, and Qualcomm points to Caffe, Caffe2, and Tensorflow as examples.

Available via Qualcomm’s Developer Network, the company is pitching it at developers in the mobile, automotive, healthcare, security, and imaging industries – anywhere there is a requirement for on-device processing. Qualcomm is advertising the Snapdragon line as the platform for ML-enabled computer vision, natural language processing, object tracking and avoidance, and gesture and text recognition.

As for early adopters, Facebook has apparently confirmed plans to integrate the NPE SDK into its Facebook smartphone application, to boost its Augmented Reality features, which are based on Caffe2 (open-sourced by Facebook). Facebook says this will provide a 5x performance improvement through the Adreno GPUs, instead of relying on a generic CPU implementation, providing a much smoother experience.

The goal here is to get more out of the mobile performance – especially when it comes to reduced energy requirements. A power-intensive system just isn’t feasible in a smartphone, or in most battery-powered IoT devices, as the trade-off of improved performance or capabilities is a moot point if the device runs out of juice and goes offline.

This is where Qualcomm’s claims of CPU-offload make a lot of sense. On the Snapdragon 835 SoC, the developer claims that the machine-learning performance is 8x as energy efficient and 4x as powerful on the Adreno GPU, and 25x energy efficient and 8x as powerful on the Hexagon DSP – at least when measuring the performance of the inference use case with the Inception v3 model.

For mobile, based on those figures, the NPE SDK looks like it could significantly free up CPU resources, helping the central processor to remain in lower power consumption states while the ML functions are in use. Instead of maxing out the CPU, the application can use the GPU or DSP to carry out its ML functions, hardware that can be essentially turned off once the task has completed.

In addition, Qualcomm is providing the tools that will help to create the trained models that will power the ML applications, and a ‘model conversion’ tool sounds especially interesting – which should allow an existing model to be adapted to meet a new requirement, without having to train the specific model from scratch again. This should save a lot of development time, if the tool works correctly – supporting both convolutional neural networks (CNNs) and customized layers.

Essentially, the models are highly specialized systems for spotting patterns in data – whether that’s audio files, images, or text. When the models begin their training, they are very inaccurate, but as they handle more data, they get better. The machine-learning part of this comes from having a machine validate the model’s findings, rather than a human (“yes, that is a cat in the picture”), which drastically improves training times.

Eventually, just the model, and not the massive accompanying data set, can be exported to the end device – able to run on optimized hardware and software, instead on a massive cloud-based cluster. The training process, according to Qualcomm, begins with training the model, converting it to a Deep Learning Container (DLC) format, adding the NPE runtime code to the mobile application, and then loading the model onto the end-device – with only that last piece taking place on the Snapdragon SoC.

“We developed the Snapdragon NPE SDK to help meet the increasing demand for running deep neural networks efficiently on mobile and other edge devices powered by Snapdragon,” said Gary Brotman, director, product management, Qualcomm Technologies. “The Snapdragon NPE was created to give developers the tools to easily migrate intelligence from the cloud to edge devices, and affords them the flexibility to target the appropriate Snapdragon core with the power and performance profile for a given user experience.”