Alphabet's autonomous taxi company Waymo recently published a report showing its autonomous driver software outperforms human drivers on several benchmarks. The analysis covers over seven million miles of driving with no human behind the wheel, with Waymo cars having an 85% reduction in crashes involving an injury.
Waymo compared their crash data, which the National Highway Traffic Safety Administration (NHTSA) mandates that all automated driving systems (ADS) operators report, to a variety of human-driver benchmarks, including police reports and insurance claims data. These benchmarks were grouped into two main categories: police-reported crashes and any-injury-reported crashes. Because accident rates vary by location, Waymo restricted their comparison to human benchmarks for their two major operating areas: San Francisco, CA, and Phoenix, AZ. Overall, Waymo's driverless cars sustained a 6.8 times lower any-injury-reported rate and a 2.3 times lower police-reported rate, per million miles traveled. According to Waymo,
Our approach goes beyond safety metrics alone. Good driving behavior matters, too — driving respectfully around other road users in reliable, predictable ways, not causing unnecessary traffic or confusion. We’re working hard to continuously improve our driving behavior across the board. Through these studies, our goal is to provide the latest results on our safety performance to the general public, enhance transparency in the AV industry, and enable the community of researchers, regulators, and academics studying AV safety to advance the field.
Waymo published a separate research paper detailing their work on creating benchmarks for comparing ADS to human drivers, including their attempts to correct biases in the data, such as under-reporting; for example, drivers involved in a "fender-bender" mutually agreeing to forgo reporting the incident. They also recently released the results of a study done by the reinsurance company Swiss Re Group, which compared Waymo's ADS to a human baseline and found that Waymo "reduced the frequency of property damage claims by 76%" per million miles driven compared to human drivers.
Waymo currently operates their autonomous vehicles in three locations: Phoenix, San Francisco, and Los Angeles. Because Waymo only recently began operating in Los Angeles, they have accumulated only 46k miles, and while they have had no crashes there, the low mileage means that the benchmark comparisons lack statistical significance. Waymo's San Francisco data in isolation showed the best in comparison to humans: Waymo's absolute incidents per million miles was lower there than overall, and human drivers were worse than the national average.
AI journalist Timothy B. Lee posted his remarks about the study on X:
I'm worried that the cowboy behavior of certain other companies will give people the impression that AV technology in general is unsafe. When the reality is that the leading AV company, Waymo, has been steadily building a sterling safety record.
In a discussion about the report on Hacker News, one user noted that in many incidents, the Waymo vehicle was hit from behind, meaning that the crash was the other driver's fault. Another user replied:
Yes, but...there is something else to be said here. One of the things we have evolved to do, without necessarily appreciating it, is to intuit the behavior of other humans through the theory-of-mind. If [autonomous vehicles consistently] act "unexpectedly", this injects a lot more uncertainty into the system, especially when interacting with other humans.
The raw data for all autonomous vehicle crashes in the United States is available on the NHTSA website.