Your browser is not supported. Please update it.

22 January 2016

Google’s 13 would-be crashes show great improvement, long way to go

Google recently disclosed that its self-driving cars would have been involved in 13 accidents during the trial process – 10 of which were the fault of the cars’ navigation systems. With a great improvement in the figures, Google still has a lot of driving to do to gather enough data to persuade the public and regulators that autonomous vehicles are a safe investment.

The (small) irony here is that the crashes were all averted when the test pilot took manual control of the cars, using the steering wheel that almost solely exists to comply with a recently introduced Californian traffic regulation. Google’s current range comprises a two-seater vehicle that Google hopes will have no navigational controls installed when it makes it to market, and a retrofitted Lexus SUV.

Walking through the report, which spans the period between September 2014 and November 2015, Google reported that its test drivers had to take control 341 times – which it refers to as a disengagement.

A total of 272 of these were attributed to “failure of autonomous technology.” In these instances, the car detected a fault with its navigational systems and then handed control over to the test driver, using “a distinct audio and visual signal.”

Google says that the threshold for a disengagement is deliberately set quite low, and that it isn’t focusing on keeping this number small as a priority. Rather, it is trying to gather data that can be used to improve the system, but has ensured that small anomalies in sensor readings are enough to trigger the alarm. The last thing Google wants is the tabloid sensation of an autonomous car pileup.

Although such a feat would be impressive in the smaller of Google’s cars. With a top speed of around 30mph, the quick-transit vehicle is designed to ferry users around dense urban environments, where the vehicle isn’t expected to reach highway cruising speeds. Still, its small frame might raise safety concerns from potential travelers, if the vehicle gains a reputation for leaping into dangerous situations.

But the data doesn’t seem to show that there is a significant risk of this happening. During the period, the self-driving cars covered 424,331 miles – with most of the vehicles exceeding in a month the typical driving distance that a US adult manages in two to four years.

On that note, the average distance driven per disengagement has increased from 785 miles per event in Q4 2014, to 5,318 in Q4 2015. That’s a very substantial improvement, and one that should only increase as the technology advances.

At some point, autonomous vehicle advocates have to present a strong case for adoption to a skeptical public. The argument must be that the occasional error is worth the improved safety record of the cars – with the very strong additional benefit of improved energy efficiency incurred by computers being more efficient and attentive drivers than humans.

The additional 69 disengagements that were reported were definitely more serious in nature, however. Classified as an event where “safe operation of the vehicle requires control by the driver,” these would include errors on the fault of the car, as well as those from other drivers.

Google reviews the sensor data following these events, feeding it back into a simulator to find out what would have happened without intervention. Of these 69, Google says 13 would have resulted in some sort of contact with another road-user.

Now, two of those scenarios would have seen the car strike a traffic cone, which I think we can all agree is a fairly minor event at low speeds – although the speed of transit does appear to be absent from Google’s self-reporting. It’s one thing to run over the edge of a cone at 15mph, but quite another to lodge it in a wheel arch at 70mph.

So of the 13 total, Google takes the blame for 10, saying that the other three were attributable to other drivers on the road, but still required manual override on the part of the test driver.

With eight of the 13 potential crashes occurring in the last quarter of 2014 and 53,000 miles driven, Google showed significant signs of improvement. The remaining five disengagements took place over 11 months and 370,000 miles driven – which gives Google a figure of one crash per 74,000 miles driven.

Data from Virginia Tech says that a US driver logs one crash per 238,000 miles driven on average. Initially, this suggests that the autonomous cars have a long way to go, but you have to note that the collision speeds and accident severity will be much different with self-driving vehicles.

Google’s threshold in the simulator was simply contact, not an injury to an occupant. With these sorts of figures, you begin to be able to ask the worth of these autonomous systems – and should be able to build a strong advocacy argument to persuade both legislators and the public.

Remember, there hasn’t been a death attributed to a driverless car to date, and that’s from all the companies testing, not just Google. Google has reported 11 accidents to date, which were all minor, and all attributed to other drivers colliding with the car.

By comparison, there were 32,296 road deaths in the US in 2014, resulting from some 5,419,000 reported crashes, over a total of 2,967,000,000,000 miles travelled collectively by the total US fleet of some 253 million vehicles. The total crash figure is going to be much higher, as many minor incidents are not reported by those involved, for a number of reasons including insurance claims affecting premiums, and the inconvenience of reporting.

That’s an average of 88 deaths per day in 2014, from 14,846 crashes – which makes Google’s figures pale in comparison. If Google can improve its potential crash figures, it will have a much stronger leg to stand on when advocating wider self-driving vehicle testing and adoption.

Comparing Google’s results to other companies is a little tricky, and doesn’t look particularly fair. For instance, Bosch reported 625 disengagements to the Californian DMV, over a distance of 935 miles driven – which sounds woeful until you read that these were all planned tests of the technology.

Similarly, Tesla reports no disengagements at all, but doesn’t say how far it’s driven, VW reported 260 disengagements in 14,945 miles, with Delphi reporting 405 disengagements in 16,662 miles, but noting that 212 had been due to the car failing to make out road markings or traffic lights rather than imminent threat of collision.

With different manufacturers using different conditions and criteria to evaluate their systems, it’s rather hard to draw direct comparisons. But Google has been leading this industry since the beginning, and the automakers are now waking up to the scale of their missed opportunity.

To this end, Google-parent Alphabet is reportedly planning an Uber-rival using its driverless vehicle, and Google was expected to announce a manufacturing deal with Ford at CES – which failed to materialize at the event.