Xilinx’s new FPGA-powered Smart Network Interface Card (NIC) announcement opened the door for an introduction with Mipsology, a start-up that provides a software abstraction layer that it claims massively simplifies programming for the hardware acceleration silicon. Specifically, Mipsology is focused on video and images, and such services should facilitate increased adoption of FPGAs in core video workloads.
The headline news was that Xilinx had unveiled its new Alveo SN1000 Smart NIC – the evolution of the NIC that connects servers together and to the outside world. It incorporates a 64-bit 16-core Cortex-A72 ARM CPU into the design, to run the additional application code that sets a Smart NIC apart from a regular NIC.
However, a new set of services and integrations were the bigger draw, with Xilinx showing off a new AI-assisted ‘smart world AI video analytics solution,’ which caught Faultline’s eye. Founded in 2015, headquartered in France, with around 40 staff, Mipsology is providing a software toolset that it claims allows for the easy migration of existing AI-assisted applications from GPUs and into the new Alveo FPGA-assisted platform.
This would then enable Xilinx to begin poaching customers that were historically using GPUs to power their AI workflows, as well as target new clients that want to use AI-based inference acceleration capabilities. For Mipsology, it hopes to get a slice of any such migration, with developers using its Zebra abstraction layer to get to grips with the finnicky FPGAs.
We spoke to Mipsology founder and CEO Ludovic Larzul, and Robert Lara, who heads up go to market strategy and business development. Lara explained that in the course of the last 18 months, Mipsology had observed that while the strength of FPGAs was evident in AI and video workloads, their downside was the high level of hardware expertise that is needed to reprogram them.
Lara added that it seemed there was a clear industry consensus that once you make FPGAs easier to work with, they would take off and pass GPUs. This is where Zebra comes in, of course. Lara said that Zebra was something of a secret weapon for Xilinx, but Mipsology remains hardware-agnostic – meaning that Zebra should work with FPGAs from Intel’s Altera, as well as smaller outlets like Achronix, Lattice, or Microchip. The list of supported vendors currently stands at 14.
The conversation turned to the likelihood of FPGAs displacing ASICs in video processing workloads. Larzul recalled the AWS purchase of Twitch, and its shift from CPU to GPU. However, due to the numerous formats that Twitch was having to support, from its myriad of viewing devices, Twitch found that GPUs were not as flexible as much of the industry would have had it believe. This then led to AWS’ implementation of FPGAs – not long after Microsoft went all-in, and still being one of the first, believes Larzul, and predominantly Xilinx.
Another benefit for FPGAs is that they can be updated for the latest codec implementations, while an ASIC would need to be ripped and replaced. Total Cost of Ownership (TCO) now seems to be the main concern for customers, rather than just an upfront cost, and this favors the FPGA sales teams. It is not uncommon to see ten-year deployment schedules with FPGAs, although this is more common outside the data center than in it, particularly in industrial and communication applications.
When asked whether there was a preferred silicon vendor, Larzul said that Xilinx had been ahead for the past few years, but that there are signs of change. Intel is gaining ground, he believes, but cannot say whether one vendor has a clear advantage in video transcoding. However, the only real challenger to Intel and Xilinx is Achronix, as the other FPGA vendors are aimed more at low-power designs not intended for the data center.
Larzul warned that despite the suitability of FPGAs for transcoding, you cannot assume that all the large cloud providers are going to embrace the technology uniformly. As an example, he pointed to Google’s pursuit of its Tensor Processor Unit (TPU) in AI workloads, which was born from a frustration with Nvidia’s dominant position as a GPU supplier and the high prices it was charging.
Google designed its own ASIC to alleviate that pressure, but the project has not been a complete success. It still buys a lot of Nvidia GPUs, and in video transcoding, designing and ASIC can take up to two years – at which point, the codec has likely changed again.
Larzul’s view is that there will always be a mix of silicon in transcoding. Static loads will shift to ASICs, with a lot of the dynamic tasks being allocated to FPGAs. He also warns that ASIC vendors making big claims in terms of TOPS (trillion operations per second) should be viewed with skepticism, as real-world testing can show throughput as low as 10% of what is claimed.
The growth of AMD in the enterprise and consumer markets has given Larzul hope that its ongoing acquisition of Xilinx would not stall FPGA momentum. The growing market for FPGAs is creating a lot of opportunities for companies like Mipsology.
Returning to the Xilinx announcement, in addition to Mipsology, Aupera was on board, using the new FPGA capabilities to push a smart city and smart retail offering, using the Xilinx video processing suite for machine vision functions. DeepAI was also tapped, announcing a claimed 10x performance per cost advantage in AI system training tasks, when using the new FPGAs, compared to using GPUs.
Machine-vision was one of the pillars of the IoT hype-cycle, as video was a way for data to be harvested from the real world and integrated into digital environments. You could monitor traffic, track patterns in shopper or worker movements, identify inefficiencies in industrial processes, and all this data could be put to use somewhere.
Of course, the IoT was abused by overzealous marketers, who then moved onto blockchain, AI, and now 5G, much to the sector’s detriment. The technology, however, has evolved, and now even our consumer connected doorbells are pulling off elements of machine-vision processes.
From the smart home perspective, we have long held that there will soon be an uptick in the amount of data being sent upstream from consumer homes – generated by a growing number of connected devices. If multiple live video feeds were involved, then this could pose a crippling burden to ISP networks. Machine-vision, or rather the ability to process this video and then only send the relevant insight upstream, is the best way to nip that in the bud.
Machine-vision has less direct relevancy on the content side of things, but there are insights to be gleaned from applying these technologies to what we watch, and discerning patterns from that process. Marketers are always looking for an edge, and being able to promote more suitable or applicable content to viewers is something that seems to be enabled by machine-vision.
However, the NICs themselves have little to do with machine-vision. The new interest in evolved NICs stems from FPGAs enabling new capabilities, allowing you to process data on the NIC itself before sending it onward to the rest of the components in the server. This shift is from passive functions to active ones.
Intel had something like a 60% market share in NICs, before it was essentially forced to purchase Altera for $16.7 billion in 2015, in order to protect that market share. It was becoming increasingly clear that an augmented NIC was an excellent return on investment, and Intel’s plain old NICs were not going to cut it anymore.
Companies like Napatech were designing their own Smart NICs, and we took a look at a shiny new Xilinx-based offering at MWC 2018. Napatech was very aware that the FPGA vendors themselves wanted to expand into this market. Well, with the new Alveo SN1000, Xilinx has made that jump.
That year, Intel had won a reprieve, as Microsoft Azure was including an Intel FPGA inside every new server it commissioned, however, Microsoft committed to buying Xilinx FPGAs too, later in the year. Amazon’s AWS had also moved to offering FPGAs as a service too. The following year, Napatech had diversified its supply, moving from Xilinx-only to purchasing Intel silicon too.