Nvidia used the Computex tradeshow in Taiwan to show off its new Nvidia EGX platform, which aims to bring AI-powered computing to the network edge. But there’s been little fanfare in the press about the announcement, and the enthusiasm for both AI and edge developments seems to have worn thin.
There’s a definite sense that “AI” is no longer a sexy buzzword, and that the PR and marketing types have moved on. The hot new thing seems to be “5G” and “AI” has come to hang out with “IoT” and likely “blockchain.” We welcome “AI” to this slightly forlorn club, and can reconcile it by saying that it’s largely a good thing that people have moved on, as it’s easier to get on with the task at hand.
However, there is a major difference between our view of the collective AI market and the wider IoT market – the AI one is much smaller, and dominated by a collection of large players. At the silicon level, consolidation is going to quickly take hold, and at the cloud layer, there are only a dozen or so major customers.
The software ecosystem looks most diverse, but it is stymied by the prevalence of open source offerings, and again, much of this is going to be swallowed up by the cloud computing vendors, who want to supply much more than bare metal. The more innovative hardware-focused startups are going to find their niches, but the big volume deals are likely going to orbit the cloud computing platforms that are going to capture much of the market.
To this end, the edge is mostly just an extension of the cloud, rolled out to a place where there’s a business model for having local processing, rather than shipping the data off and waiting for a cloud-hosted application to come up with a response to the data input. Whether it’s prohibitive backhaul traffic costs or the need for an immediate decision that can’t tolerate the round-trip latency of the cloud, there’s definitely room to grow in the edge.
Nvidia is pitching EGX at companies that need to perform low-latency AI processing at the network edge. It cites continuous data streaming between 5G base stations as one use case, as well as warehouses, retail stores, and factories, saying that EGX was built ‘to meet the growing demand to perform instantaneous, high-throughput AI at the edge, where data is created, with guaranteed response times, while reducing the amount of data that must be sent to the cloud.’
As for the size of this market, Nvidia is using the somewhat nebulous term “machine sensors”, saying that there will be 150bn of these things and IoT sensors by 2025, which will collectively stream continuous data that will need to be sifted. The term stems from a Seagate-sponsored IDC whitepaper, and donning our tinfoil hats, it seems quite convenient that a storage vendor has found such a big number.
Nonetheless, EGX is an offering that ranges from just one Jetson Nano, which was unveiled back in March, which might be found in a device consuming a few watts of electricity, all the way up to a rack of Nvidia T4 servers. This spans a range of 0.5 TOPS all the way through to a claimed 10,000 TOPS, from an application like image recognition all the way to real-time speech recognition and similar AI tasks, says Nvidia.
The Nvidia silicon is one component, but its new Edge Stack is equally important. Deployed using Kubernetes, for that sweet container management flexibility, Nvidia has also partnered with Red Hat (IBM), to integrate this stack with OpenShift – one of the leading Kubernetes orchestration platforms. Cisco is also a launch partner, and Nvidia’s recent Mellanox purchase crops up, by way of the option to install Mellanox Smart NICs in your box.
As for the list of server providers, it currently reads: ATOS, Cisco, Dell EMC, Fujitsu, Hewlett Packard Enterprise, Inspur and Lenovo, Abaco, Acer, ADLINK, Advantech, ASRock Rack, ASUS, AverMedia, Cloudian, Connect Tech, Curtiss-Wright, GIGABYTE, Leetop, MiiVii, Musashi Seimitsu, QCT, Sugon, Supermicro, Tyan, WiBase and Wiwynn.
Nvidia points to video analytics applications as being ideal for retail and smart city applications, with options available from AnyVision, DeepVision, IronYun and Malong Technologies. Some unspecified healthcare-focused software offerings are also cited, from 12 Sigma, Infervision, Qunatib and Subtle Medical.
And so this does seem a little lackluster, which might explain why the announcement has flown under the radar. Nvidia does say that there are over 40 early adopters, which include BMW Group Logistics, which is using EGX and Nvidia’s Isaac robotics platform, and Foxconn, which is using EGX to power quality control inspection in its PC production line, with an apparent 40% increase in throughput.
Also namedropped are GE Healthcare, which deploys Nvidia T4 GPUs into its magnetic resonance (MR) systems and its Edison Intelligence platform, and Seagate crops up again, which like Foxconn, is using EGX to power quality inspection on its hard drive production lines.
“At Seagate we have deployed an intelligent edge GPU-based vision solution in our manufacturing plants to inspect the quality of our hard disk read-and-write heads. The NVIDIA EGX platform dramatically accelerates inference at the edge, allowing us to see subtle defects that human operators haven’t been able to see in the past. We expect to realize up to a 10 percent improvement in manufacturing throughput and up to 300 percent ROI from improved efficiency and better quality,” said Bruce King, senior principal data scientist, Seagate Technology.