Comcast cuts truck rolls with open source AI system

[This article originally appeared in our sister publication, Faultline Online Reporter, which is focused on the video industry.]

Comcast claims to have saved tens of millions of dollars of truck roll expenses, by using a machine learning program that can predict with 90% accuracy whether or not it will need to send a technician to a customer’s home to fix connectivity problems.

Every operator would love to reduce truck rolls. Estimates of the average costs vary, but tend to be somewhere between $50 and $100 per visit. Even if a company is using the most efficient vehicles possible, those costs are increasing as fuel and labor costs rise.

All service operators are well aware that they end up dispatching technicians more often than is really needed, because many problems could be solved easily if the operator’s staff could diagnose them remotely, or if they had a clear view of what was happening inside the customer’s home.

The Comcast AI solution was developed internally and is based in part on open source technology, including the open source H2O AI platform. The key question it can answer is whether the problem is indeed inside the customer’s home, or if it is actually located somewhere else in Comcast’s network. Adam Hertz, vice-president of engineering at Comcast’s Innovation Center in Silicon Valley, told an audience at the Mobile Future Forward event that the company developed the application by assembling and analyzing a wide range of datasets covering many different aspects of its operations, including data related to calls to its customer service center, as well as network operations data.

Reviewing the customer services call data revealed that many problems could be solved with very simple fixes, including changing a customer’s subscription details or preferences, resetting a modem, or even just replacing the batteries in a remote control. Surely then this is about de-skilling the helpdesk staff, because it would take mere weeks for them to know that stuff.

Other operators and service providers have used different methods to try to reduce truck rolls. In 2016 Swisscom announced it was using Network Function Virtualization (NFV) technology from Hewlett Packard Enterprise to provide virtual customer premise equipment in the cloud for its business customers, removing the need for any truck rolls related to on-premise equipment.

In 2016, Nokia had added machine learning capabilities to customer experience software that it claimed could remove 90% of “inappropriate” truck rolls. Within its Motive Service Management Platform and Care Analytics packages, a Call Anomaly Detection feature can detect problems within networks, mobile services, customer premises equipment, third party applications; and can spot other issues affecting IPTV, high speed internet and broadband services.

Attempts to address this issue may also have an impact on the fast-moving contest to establish dominance in the evolving market for multi-access point (AP) wireless networks. In May, Faultline reported on Comcast’s rollout of technology from Plume that can interrogate information related to a WiFi mesh in customers’ homes to determine how best to use WiFi APs and other devices within the wireless network. The solution can also feed information back to the help desk to enable remote problem-solving, making it possible, for example, to identify instances where a specific device is creating interference, then allowing the operator to amend or switch off that device.

Other technology providers are also keen to demonstrate how their wireless monitoring and management tools could help to reduce truck rolls: WiFi mesh technology provider XCellAir claims it can cut truck rolls by a third and customer care calls by 50% by offering a more effective mix of mesh, steering and other WiFi technologies, and by tracking and resolving network and site level issues.

In 2016, Bulent Celebi, chairman at wireless technology specialist AirTies, told Faultline he had talked to one operator that sent out 1 million truck rolls every year, of which 90% turned out to be ‘no fault found’ calls, mostly related to WiFi.

Faultline also reported AirTies Wireless launching its Remote View software, which helps engineers to identify, understand and fix problems in customers’ WiFi networks. It provides a ‘map’ of a subscriber’s home network, showing which device is linked to which access point (AP), and allowing it to examine signal strengths and speeds between devices – without sending out a truck.

It doesn’t seem very likely that anyone will develop entirely foolproof, or 100% reliable on-premise equipment at any point in the near future, so the best way to attack this problem seems to be to increase the operator’s ability to remotely diagnose and solve problems when customers contact them. Operators have been experimenting with different ways of using data analytics to do this for several years, and it is beginning to look as if those technologies, combined both with methods for diagnosing problems remotely, might soon be saving operators a great deal of time and money. So is there any real need for machine-learning here?

One in development for AT&T shows actual real-time data speeds around a mesh network in a remote home, against what they would be expected to be, and includes a diagnostic app at the helpdesk. Other approaches are to lay out the data paths for an App, so that each home can see for themselves what’s causing the problem – usually a heavyweight device over-using data or a phone which is attached to the wrong AP.