STRUGGLES WITH STATISTICS

Tesla’s first accident report claims it’s four times safer than the US average. Maybe.

The Tesla vehicle involved in an accident.
The Tesla vehicle involved in an accident.
Image: AP Photo/Richard Vogel
We may earn a commission from links on this page.

Tesla likes to say it’s critical to build the safest cars in the world. It’s not clear if its first safety report delivers on that promise yet.

The company released the report on its website Oct. 4, claiming its vehicles are involved in four times fewer accidents than the US average. It registered one accident or near miss (Tesla described as a “crash-like event”) per 1.92 million miles driven during the third quarter of 2018. For those driving with its Autopilot driver-assistance system engaged, Tesla said the number of incidents was one per 3.34 million miles.

That compares favorably, Tesla claims, with US National Highway Traffic Safety Administration (NHTSA) data, which estimates there to be an automobile crash every 492,000 miles driven in the US. A review of NHTSA’s published data (pdf) from 2016 (the most recent data available) shows a similar figure: one crash every 436,000 miles, a figure that includes “police-reported crashes” causing injuries, deaths and/or property damage.

Yet it’s hard to say if those two numbers in its latest safety report make Tesla vehicles quite as safe as they appear. While it’s laudable Tesla is now reporting figures—most other crash data on car companies is reported by by third-parties—its latest blog post is thin on details and context, and notable for the lack of actual fatality statistics.

Tesla’s has made its aggressive, often exaggerated, claims before when it comes to fatality figures. Since 2016, at least two Tesla drivers have been killed, one in Florida and another in California, while their Autopilot systems were engaged (paywall). A Tesla blog post in March noted that Tesla has only recorded one fatality per 320 million miles traveled in Tesla vehicles equipped with Autopilot hardware. “You are 3.7 times less likely to be involved in a fatal accident if you are driving a Tesla equipped with Autopilot hardware,” it stated. “Our vehicle autonomy reduces the probability of a death by 30%,” CEO Elon Musk claimed on the company’s a May 2 earnings call, according to Sentieo. ”The statistics are unequivocal that Autopilot improves safety.”

Both those statements are misleading, argues Steven Shladover, a research engineer at the University of California, Berkeley. Tesla’s claims were “pretty outrageous” since they do not appear to be based on a statistically valid analysis, Shladover told Quartz. Tesla has not released its data for independent analysis, and was not immediately available for comment.

Since Tesla drivers are typically a safer demographic (richer, older people) traveling on safer roads (highways instead of rural and suburban streets), comparison with NHTSA’s average data is also misleading. Tesla’s claim that cars “equipped with Autopilot hardware” does not show Autopilot reduces fatalities—other systems such as standard collision alert systems, Shladover argued, may account for any differences. And Tesla’s Autopilot system recently came in second behind Cadillac’s driver-assistance technology in a study conducted by Consumer Reports. Several safety researchers have said it is unclear if Tesla vehicles have driven enough miles with Autopilot engaged to draw statistically significant conclusions from the data. Since fatalities may occur every few hundred million miles on average on US roads, it’s unknown if Tesla has actually logged enough miles with Autopilot engaged to support statistically significant claims when it comes to fatalities.

Tesla’s cars already have some of the highest safety ratings in NHTSA crash tests, but Tesla seems to have a hard time giving a straight story when it comes to safety statistics.