Varnish Software, the Sweden-based caching specialist, is claiming a world record breakthrough for content delivery in the categories of speed, power, and total cost of ownership (TCO). The results will intrigue anyone involved in large scale live streaming and those rooted in driving more sustainable video delivery pipelines.
A medley of modern technologies from Varnish, Intel, and Supermicro have achieved record throughput of 1.3 Tbps on a single edge server consuming approximately 1,120 Watts. That works out as an almost unbelievable throughput of 1.17 Gbps per Watt.
This Gbps-per-Watt measurement is head and shoulders above anything else seen on a single edge server before.
It beats what French multicast ABR expert Broadpeak achieved in December 2022, when its BkS450 video streaming software churned out a four-fold reduction in the power required to deliver a 1 Gbps video stream. Ultra-efficient hardware resources allowed the BkS450 to deliver video services at a theoretical maximum of 725 Gbps in HTTP on servers consuming 900 Watts – which works out at 0.8 Gbps per Watt, compared to Varnish’s new record of 1.17 Gbps.
If we invert to Watts-per-Gbps, then Varnish is at 0.94 Watts/Gbps, versus Broadpeak at 1.2 Watts/Gbps.
While the respective achievements from Varnish Software and Broadpeak are worth comparing, there are some important distinctions to make. In order to be truly comparable, the workloads and design semantics need to be the same – same hardware, no hardware accelerators, video object size, cache hit rate, stream size, etc. Of course, that is never going to happen, so we have to work with the tools provided.
In this case, Varnish has the upper hand in the squeeze for more throughput with less power, as its results include TLS (Transport Layer Security), which is easier to deliver than a HTTP workload, as delivered in Broadpeak’s tests.
This is because Varnish’s tests used 4th Gen Intel Xeon Scalable Processors (known in the industry by the codename Sapphire Rapids), powering Supermicro 2U CloudDC serverprocessors running Varnish Enterprise 6.0 software.
The latest generation of Intel processors help with TLS offload, while Broadpeak’s earlier tests were based on 3rd Gen Intel Xeon Scalable processors, which do not. However, Varnish points out that the 1.3 Tbps CDN live linear network throughput was achieved without the use of specialized TLS offload cards, further saving costs and power.
Broadpeak tells us that it is currently in the process of completing newer benchmarks, but cautions that it has proven difficult to procure Sapphire Rapids configurations from OEM vendors. We are informed that Supermicro was probably the first server slinger to have a viable CDN configuration powered by Sapphire Rapids silicon.
Either way, it is important to point out that the latest throughput and energy efficiencies achieved by Varnish Software are not exclusive to Supermicro servers. Speed, power, and TCO can all be optimized on a broad range of servers, the company promises, depending on customer requirements.
As it stands, these same efficiencies are not available on any old processor. However, it is worth pointing out that AMD’s 4th Gen Epyc CPUs – codenamed Genoa – launched recently, in November 2022. Like Intel’s Sapphire Rapids, it has been difficult for vendors to get their hands on the first samples of AMD’s Genoa chips, although we understand they have become officially available to order from this week.
AMD’s Genoa processors are expected to challenge Intel’s Sapphire Rapids, to further lower power consumption per Gbps, especially when moving to a single CPU rather than the dual CPU configurations used by many content delivery vendors like Broadpeak and Varnish Software.
This is notable because Netflix Open Connect servers are powered by AMD Epyc processors. Netflix’s Open Connect servers are in the mix with performance, with the most recent tests from around 4 months ago showing a Netflix CDN experiment (codenamed Mabel), set a record speed of 731 Gbps.
Netflix highlights that this was achieved with real production traffic, serving US East Coast customers, all from a single 2U machine.
For this experiment, Netflix’s hardware configuration took in 2x AMD Epyc Milan 7713 CPUs, running 4x Nvidia ConnectX-6 Dx NICs, on a Dell PowerEdge R7525, with MBX Systems doing the hardware testing and integration. Unfortunately, we could not find any public information on energy draw.
That brings us to the elephant in the room – pricing. This measurement is almost always purposely censored from these types of announcements, and it will surprise no one that this is always a case of you get what you pay for.
Intel’s 4th Gen Xeon Scalable Processors ship for $10.7k each. Varnish Software used two of these (2x Intel Xeon Platinum 8480+ processors, to be precise), meaning that Varnish has $21.4k worth of CPU in this configuration alone.
To compare, Broadpeak’s benchmark from 3 months ago used a more modest configuration with 2x Intel Xeon Gold 5318Y processors worth just under $1.5k each, so a configuration of less than $3k.
In comparison, AMD Epyc 7713 CPUs, as sourced by Netflix, are worth $7.4k each. So that’s a $14.8k CPU configuration for the Netflix CDN. It’s not often you see a vendor the size of Varnish Software outspend Netflix.
We touched base with Varnish Software’s CTO, Frank Miller, who told us, “At the end of the day, setting a common test reference for software/hardware/load would be helpful for industry comparisons as there is some variability.”
Food for thought here for Greening of Streaming working groups, the non-profit video industry group in which Varnish Software, Broadpeak, Intel and AMD are all members. Netflix and Supermicro are not.
“We will naturally continue to improve the performance and power efficiency across all ranges of systems, from large to small, and associated workloads,” added Miller.
Varnish has worked closely with Intel before on benchmarking, but the name Supermicro is a new addition to the Swedish firm’s bar-setting streaming research. This is probably because of what we highlighted earlier, that we believe Supermicro was first to offer an actually workable CDN configuration.
You will probably remember the name Supermicro best as the Silicon Valley server maker whose name was dragged through the mud following a 2021 scandal which Bloomberg blew the lid on. Then trading as Super Micro Computer, it was reported that the company’s servers had been compromised by tiny Chinese spy chips, but the exposé was discredited soon thereafter. This is just for background; those controversies are irrelevant to what we are exploring here.
Varnish describes Supermicro’s CloudDC as a “well rounded server that gives the best cost to optimized performance ratio.”
Supermicro’s CloudDC server line is an optimized platform targeting private and public clouds offered in 1U and 2U form factors in single or dual processor configurations. These servers are optimized for balance among processor, memory, storage, expansion, and networking to give the best efficiency.
The latest benchmarks build on the work of Varnish Software on power-efficient CDN systems in partnership with Intel Network Builders, by processing memory access locally. This collaboration spawned the latest NUMA-aware (non-uniform memory access) version of Varnish Enterprise 6.0, which is the technology behind the new world record.
Intel and Varnish have together also shown that moving from one to two sockets doubles performance. As you scale up, the Varnish caching software squeezes as much performance as possible from the hardware.
Open source HTTP acceleration software is Varnish’s bread and butter, with the Varnish Cache HTTP engine used by more than 3.4 million websites. But, since around 2015/2016, Varnish has been pushing into bigger markets as the business expanded from open source to enterprise services. Enterprise clients demand bigger tools, and so Varnish began working with more generic languages, particularly with machine learning languages, marking changes in the Varnish Configuration Language.
“We have entered a new era of CDN cache performance,” declared Miller, following Varnish Software’s unofficial Guinness World Record.
What is clear is that Varnish Software, Intel and Supermicro have stolen a march on CDN performance benchmarks early in 2023, and that wider availability of the latest CPU technology among OEMs and CDN software vendors could mean that Varnish’s celebrations are short-lived. Of course, wearing its Greening of Streaming members hat, Varnish will claim that even the advancements of its closest rivals are wins for a more sustainable industry as a single cohesive unit.