AT&T achieves major milestone in its white box cell site router program

AT&T has been a driving force behind the conversion of telco networks to white box platforms, and last week, at the ONF (Open Networking Foundation) Connect conference, it announced its latest milestone – large scale production testing of white box cells site gateway routers.

The progress was discussed at the event by Andre Fuetsch, chair of the ONF, president of AT&T Labs and CTO.

He said: “This is really exciting. This is a first for us. We believe in our community of operators it’s a first for the industry.” He believes the use of white boxes in this area of the network will be essential to keep the costs of backhauling increasingly dense 5G networks under control.

The white box routers run AT&T’s own dNOS network operating system and software framework, which the telco open sourced in March under the auspices of the Linux Foundation and its Disaggregated Network Operating System (DANOS) project.

In March, AT&T said it planned to deployi more than 60,000 open source, software-driven white boxes at its cell towers over the next few years; and in October, it released its white box router specifications to the Facebook-led Open Compute Project, which aims to drive commoditized hardware into cloud infrastructure platforms.

“With DANOs coming shortly you will have the puzzle pieces that everyone can take and employ whether you are a large operator or small operator,” Fuetsch said. “You can take this and make it happen. We think this is significant and a really critical component here.”

He added: “Next year we’ll begin to deploy more of these routers. We expect to have several thousand cell sites deployed with these boxes and this is what is powering our 5G network that we’re building right now. Our intent is to make this ubiquitous across our network over the next few years.”

White boxes are also going into other parts of the AT&T network, including on top of rack switches. “We’re now using those in the disturbed core that we’re now building for our network cloud, which will basically become 5G as it gets rolled out from the core,” Fuetsch said. “As you can see, all of these implementations are beginning to take root on all of our locations big and small.”

The various technologies which AT&T has developed and open sourced are starting to come together now in a common framework – one which other operators could emulate if they adopt the same pieces. For instance, AT&T is using ONAP (Open Network Automation Protocol) to orchestrate all its white boxes, and Orange and Bell Canada are among the other telcos which are piloting combinations of ONAP for management and orchestration (MANO) with specifications such as DANOS and Open RAN (another AT&T-initiated project).

“We’re seeing some significant returns from that from the large ecosystem that has evolved around [ONAP],” added Fuetsch. “We’re hoping do similar thing here with these projects we have at ONF.” This is not confined to the US either – AT&T has already deployed white boxes in Toronto, Singapore and Amsterdam, and plans to expand that to over 70 global locations, with the white box routers supporting Internet and other global services.

AT&T has not yet announced its white box router suppliers. Now that AT&T has submitted its white box router specs to the OCP, any supplier wanting to be part of its cells site gateway roll-out will have to conform to those specs. That will enable AT&T to source boxes from multiple suppliers, choosing the cheapest or most innovative, while ensuring interoperability.

If the design is taken up by other OCP members, the scale of the ecosystem, and the consequent price competition, could be very significant, taking cellular networks closer to the open, competitive base of suppliers in WiFi networks, in which vendors compete on quality, performance and cost, not with lock-ins to proprietary systems. The WiFi industry makes heavy use of reference designs, which reduce time to market for smaller vendors, and this is the approach of OCP and TIP too.

AT&T’s reference design can be used as a guideline by any hardware vendor, though it has to be based on a specific chip (the Broadcom Qumran-AX switch-chip). Submitting it to OCP should encourage more suppliers to rise to that challenge (and other chips might follow in future). Of course, 60,000 devices is not trivial, but for many suppliers, it will not be worthwhile to design a new product, to a whole new set of specs, for just one operator. The prospect of a wider base of customers should help to open up the supply chain.

The gateway router design is supposed to support current and future cellular backhaul systems, being future-proof to some extent, by embracing a wide range of speeds on the client side, including 5G baseband units operating at 10G/25G and backhaul speeds up to 100Gbps.

Chris Rice, SVP of network cloud and infrastructure at AT&T, said in October: “We now carry more than 222 petabytes of data on an average business day. The old hardware model simply can’t keep up, and we need to get faster and more efficient. We believe this white box approach helps us meet that demand while allowing us and others now to innovate faster than ever before.”

Michael Satterlee, VP of network infrastructure and services at AT&T Labs, added: “We’ve future-proofed it with high speed interfaces and density, timing features, QoS capabilities and a powerful processor. We’re confident other mobile service providers will look at this spec and conclude, like us, that this platform can meet their needs.”

The hardware design is disaggregated from AT&T’s dNOS software, so that it can support other operators’ own choice of switch operating system. However, many elements of dNOS will also be available to other vendors and MNOs, because it has been contributed to the Linux Foundation as the basis of its Danos project to develop an open network OS for white boxes and switches. Danos also brings together other existing projects, including Free Range Routing, aiming to create an “uber-operating system” for white boxes to accelerate commercialization.

dNOS is based on software which AT&T acquired with Vyatta in 2017. Robert Bays, assistant VP of Vyatta development at AT&T Labs, commented: “The Vyatta network OS stack is production-hardened and an excellent match for the demanding functional and reliability requirements of the cell site gateway router. Consistent with our previous announcements to create the Disaggregated network OS (DANOS) open source project, hosted by the Linux Foundation, we are now sorting out which components of the open cell site gateway router NOS we will be contributing to open source.”

When it published the white paper to launch dNOS as an open platform in March, AT&T promised “a new approach for router platform development and procurement”, saying it would “evolve its router platform sourcing process to give preference to dNOS vendors whose products (or committed product roadmap) are based on using this platform.” This is a big deal for vendors – AT&T claims to have “100,000 interconnected IP/MPLS routers” in its networks.

In a blog post, Chris Rice, SVP of AT&T Labs’ Domain 2.0 architecture and Design activity, enlarged on the need for a white box OS, writing  on a blog post: “While ONAP is the orchestration software for the entire network, each individual machine also needs its own operating system. We want to get hardware and software makers, open source developers, telecom companies, standards bodies and others to others to start thinking about how we can all push this concept forward.”

AT&T says in its white papers that it has three main objectives for dNOS:

  • Faster introduction of technologies, designs and new features enabled by a collaborative and open ecosystem.
  • Flexibility in network design and service deployment with plug-and-play hardware and software components that can scale up and down cheaply and responsively.
  • Cost reduction by using merchant silicon and standardized hardware and software technology, which will enable very large numbers of elements to be deployed to support high capacity, device density and ubiquitous coverage.

AT&T says that, to achieve all this, “it’s critical that both hardware and software include standardized interfaces that a community of developers can coalesce around. A single, standardized NOS is the most efficient and effective means to this end. A single NOS allows for qualification of a common, shared integration infrastructure and APIs to help developers rapidly launch new applications. It allows for ecosystem developers to focus on value adding applications rather than the basic building block components required in all network infrastructure.”

Fuetsch is bullish about the radical impact of white boxes on carrier economics. When announcing the OCP deal, he said in a statement: “White box represents a radical realignment of the traditional service provider model. We’re no longer constrained by the capabilities of proprietary silicon and feature roadmaps of traditional vendors.”

Next step – white boxes in the access network:

The next step will be to push the open hardware and virtualized network efforts into the wired and wireless access space. “That’s really where the big money and big opportunity is,” Fuetsch said, adding that AT&T is trialling the ONF’s SDN-Enabled Broadband Access (SEBA) reference design in Georgia and Texas. This is a lightweight version of another open source project, R-CORD (Residential Central Office Re-architected as a Datacenter). SEBA supports access technologies, such as PON, G.Fast and DOCSIS, at the edge of service providers’ networks. Deutsche Telekom and Turk Telecom are also planning SEBA deployments.

It has also tested white box switches and CPE, the former powered by dNOS and the P4 open programming language, which has replaced Openflow as the Open Networking Foundation’s default protocol for switching. Chip vendors including Broadcom, Barefoot, Cavium and Mellanox now make merchant switch-chips for white box platforms, and AT&T has tested systems with the first two of those suppliers. Quanta and Delta are among the first companies to make white box switches themselves.