Of Facebook’s two major open hardware initiatives, the Telecom Infra Project (TIP) has gained most attention in the mobile industry because it is specific to telco networks. However, its older stablemate, the Open Compute Project (OCP), is also important to operators, as their cloud and network platforms converge, both in equipment and services terms. AT&T, the most aggressive of all the carriers about introducing open source norms to the closed telco supply chain, is putting an operator slant on the OCP’s work, by submitting its own specification for white box cell site gateway routers to the project.
Both OCP and TIP aim to establish open, unified specifications for commoditized white box hardware, which will support cost-effective deployment and massive scalability for cloud and network systems. Both were initiated by Facebook and seeded with some of the social giant’s own designs, but the aim has been to secure contributions from as many players as possible, to agree on a rich set of open specs to which any suppliers would need to conform.
AT&T is starting with a router, but may well go on to submit more of its white box efforts, if the first one goes down well with OCP participants. The telco has already said it will install the routers, running its own network operating system, dNOS, in 60,000 locations over the next few years, replacing all its current cell site routers (AT&T has 60,000 towers and 5,000 central offices).
Any supplier wanting to be part of that roll-out will have to conform to the specs, which will enable AT&T to source boxes from multiple suppliers, choosing the cheapest or most innovative, while ensuring interoperability.
If the design is taken up by other OCP members, the scale of the ecosystem, and the consequent price competition, could be very significant, taking cellular networks closer to the open, competitive base of suppliers in WiFi networks, in which vendors compete on quality, performance and cost, not with lock-ins to proprietary systems. The WiFi industry makes heavy use of reference designs, which reduce time to market for smaller vendors, and this is the approach of OCP and TIP too.
AT&T’s reference design can be used as a guideline by any hardware vendor, though it has to be based on a specific chip (the Broadcom Qumran-AX switch-chip). Submitting it to OCP should encourage more suppliers to rise to that challenge (and other chips might follow in future). Of course, 60,000 devices is not trivial, but for many suppliers, it will not be worthwhile to design a new product, to a whole new set of specs, for just one operator. The prospect of a wider base of customers should help to open up the supply chain.
The gateway router design is supposed to support current and future cellular backhaul systems, being future-proof to some extent, by embracing a wide range of speeds on the client side, including 5G baseband units operating at 10G/25G and backhaul speeds up to 100Gbps.
Chris Rice, SVP of network cloud and infrastructure at AT&T, said: “We now carry more than 222 petabytes of data on an average business day. The old hardware model simply can’t keep up, and we need to get faster and more efficient. We believe this white box approach helps us meet that demand while allowing us and others now to innovate faster than ever before.”
Michael Satterlee, VP of network infrastructure and services at AT&T Labs, added: “We’ve future-proofed it with high speed interfaces and density, timing features, QoS capabilities and a powerful processor. We’re confident other mobile service providers will look at this spec and conclude, like us, that this platform can meet their needs.”
The hardware design is disaggregated from AT&T’s dNOS software, so that it can support other operators’ own choice of switch operating system. However, many elements of dNOS will also be available to other vendors and MNOs, because it has been contributed to the Linux Foundation as the basis of its Danos project to develop an open network OS for white boxes and switches. Danos also brings together other existing projects, including Free Range Routing, aiming to create an “uber-operating system” for white boxes to accelerate commercialization.
dNOS is based on software which AT&T acquired with Vyatta in 2017. Robert Bays, assistant VP of Vyatta development at AT&T Labs, commented: “The Vyatta network OS stack is production-hardened and an excellent match for the demanding functional and reliability requirements of the cell site gateway router. Consistent with our previous announcements to create the Disaggregated network OS (DANOS) open source project, hosted by the Linux Foundation, we are now sorting out which components of the open cell site gateway router NOS we will be contributing to open source.”
When it published the white paper to launch dNOS as an open platform in March, AT&T promised “a new approach for router platform development and procurement”, saying it would “evolve its router platform sourcing process to give preference to dNOS vendors whose products (or committed product roadmap) are based on using this platform.” This is a big deal for vendors – AT&T claims to have “100,000 interconnected IP/MPLS routers” in its networks.
In a blog post, Chris Rice, SVP of AT&T Labs’ Domain 2.0 architecture and Design activity, enlarged on the need for a white box OS, writing on a blog post: “While ONAP is the orchestration software for the entire network, each individual machine also needs its own operating system. We want to get hardware and software makers, open source developers, telecom companies, standards bodies and others to others to start thinking about how we can all push this concept forward.”
AT&T says in its white papers that it has three main objectives for dNOS:
- Faster introduction of technologies, designs and new features enabled by a collaborative and open ecosystem.
- Flexibility in network design and service deployment with plug-and-play hardware and software components that can scale up and down cheaply and responsively.
- Cost reduction by using merchant silicon and standardized hardware and software technology, which will enable very large numbers of elements to be deployed to support high capacity, device density and ubiquitous coverage.
AT&T says that, to achieve all this, “it’s critical that both hardware and software include standardized interfaces that a community of developers can coalesce around. A single, standardized NOS is the most efficient and effective means to this end. A single NOS allows for qualification of a common, shared integration infrastructure and APIs to help developers rapidly launch new applications. It allows for ecosystem developers to focus on value adding applications rather than the basic building block components required in all network infrastructure.”
A move to transform the capex and opex costs of networks, and allow AT&T to build truly multivendor systems, is a complex one. Although AT&T has started revealing results of its tests of white box routers and switches this year, these have been going on for several years already, with the cell site router the first commercial implementation.
Earlier this year, when it announced the dNOS plan, Andre Fuetsch, CTO and president of AT&T Labs, said in a statement: “White box represents a radical realignment of the traditional service provider model. We’re no longer constrained by the capabilities of proprietary silicon and feature roadmaps of traditional vendors.”
That was an unequivocal throwing down of the gauntlet to those traditional suppliers, challenging them to match the cost base and innovation of open platforms or be sidelined in the operators’ next generation deployments.
Of course, MNOs have tried before to open up their supply chains and drive down cost, but the results have been limited. There is the risk, this time too, that most operators will play it safe and stick with traditional vendors and platforms. But as Patrick Lopez, VP of networks innovation for Telefónica, said in a conference earlier this year, the difference this time around is that “all the operators involved have skin in the game. They are investing not only time and money and network but they actually have teams of people that are developing code, and writing open source code … This is a strong signal to the market showing that beyond intent there is actual effort, progress and code being developed to show the way we would like networks to evolve.”
Fuetsch added: “We’re writing open hardware specifications for these machines, and developing the open source software that powers these boxes. This means faster hardware upgrades, since anyone can build to these specs. And software upgrades that move at internet speed. We’re doing this all while keeping costs low so we can focus on expanding our nationwide mobile 5G footprint for our customers as quickly as possible.”
By pushing white boxes and open software out beyond the core and transport networks and into the cell sites, AT&T is extending virtualization throughout its wireless RAN in preparation for a fully software-driven 5G network with a heavy focus on edge computing, integrated with connectivity. White box routers can reduce the cost of delivering wireless connectivity but also provide an affordable way to distribute 5G and cloud services to the edge.
AT&T has not yet announced its white box router suppliers. It has also tested white box switches and CPE, the former powered by dNOS and the P4 open programming language, which has replaced Openflow as the Open Networking Foundation’s default protocol for switching. Chip vendors including Broadcom, Barefoot, Cavium and Mellanox now make merchant switch-chips for white box platforms, and AT&T has tested systems with the first two of those suppliers. Quanta and Delta are among the first companies to make white box switches themselves.
A year ago, Fuetsch demonstrated the use of P4 on merchant silicon for a white box switch, saying in a conference keynote: “This is more than just about lowering cost and achieving higher performance. Frankly that’s table stakes. This is really about removing barriers, removing layers, removing all that internal proprietary API stack that we’ve lived with these legacy IT systems, now we can bypass all of that and go straight to ONAP” to achieve fine-grained per-packet visibility.
AT&T has worked with Barefoot Networks and SnapRoute on its switch. Barefoot provided its Tofino chip and surrounding software, while SnapRoute (in which AT&T is an investor) developed the network operating system, Flex Switch.
Fuetsch said the firm “took this very novel approach of building a hardware abstraction layer and running open source networking modules like BGP and OSPF on top”. This layer can then operate independently of the silicon, and AT&T said it has other white boxes, based on different chips, in the works, with the same network OS.
All this will, operators hope, add up to a transformed cost base and supply chain for 5G. As Fuetsch said: “Here’s the big message for the OEMs. It is really a call for them to open up their architectures, open up their software and their own hardware so they can participate. They are going to have to make a choice here – do you want to be at the table or on the plate?”