What the autonomous vehicles industry should learn from the Boeing 737 Max crisis

Cutting corners can cost you.
Cutting corners can cost you.
Image: REUTERS/Lindsey Wasson
We may earn a commission from links on this page.

We’re about to enter a world in which autonomous vehicles (AVs) routinely make life-and-death decisions. Before hopping in, we should draw lessons on how to regulate them from recent missteps in another industry: airlines. With stronger oversight, it appears, the two fatal Boeing 737 Max crashes could have been prevented.

For decades, we’ve trusted the air-safety system. The US Federal Aviation Administration exhaustively studies crashes and proactively forces airlines and manufacturers to improve safety by adding features or redesigning aircraft. Since the 1970s, the fatal accident rate for global air travel has plunged by a factor of 16 (paywall) to one accident every 3 million flights of large commercial passenger planes.

Meanwhile AVs are nowhere near the human safety standard of one fatality for every 100 million vehicle miles traveled (although we are promised they will be safer). Several deaths have been linked to (paywall) fully or partially autonomous vehicles. Firms like Waymo (pdf) and Tesla have slowed ambitious timelines for rolling out their technology.

But so far, regulation of AVs in the US has mostly been reactive. (China, South Korea, and others have implemented rules for the industry.) The latest guidance (pdf) from the US government allows companies to self-certify their new AV technologies without safety tests

The 737 Max tragedies point to the risk this model poses for AVs. To win swift approval for its update to the 1960s-era 737, Boeing cut corners on safety. Crucial cockpit alerts were optional, and pilot training consisted of “an iPad lesson for an hour.” When safety concerns did arise, Boeing fought them. The FAA let this slide, approving a team of Boeing engineers to conduct inspections on its behalf, and never correcting a fatal flaw that caused an autonomous steering system to repeatedly force the 737 Max into a dive. Those decisions turned preventable technical mistakes into failures that killed nearly 350 people.

Last year, AV legislation didn’t pass the US Congress. “Some senators distrusted the technologies and the developers of those technologies and the regulators of those technologies,” writes Bryant Walker Smith, a University of South Carolina law professor, via email. Those suspicions were well placed, he argues. Although we often ask whether the public trusts AVs, Smith suggests we should instead be asking if we trust the companies and regulators deploying these vehicles in the first place.

This essay was originally published in the weekend edition of the Quartz Daily Brief newsletter. Sign up for it here.