In This Story
Surely, the billions of dollars invested in AI chatbots will increase productivity and put economic performance into hyperdrive, right?
Hold the phone, say researchers at the National Bureau of Economic Research (NBER), a think tank in Cambridge, Massachusetts. “Despite substantial investments [in chatbots], economic impacts remain minimal,” they write in a new report.
Economists Anders Humlum and Emilie Vestergaard estimate in the report, called “Large Language Models, Small Labor Market Effects,” that productivity gains from AI chatbots amounted to a mere 3% in time savings.
“AI chatbots have had no significant impact on earnings or recorded hours in any occupation, with confidence intervals ruling out effects larger than 1%,” they write. Considering the corporate hype promising revolutionary change in a post-ChatGPT world — both Shopify and Duolingo announced recently that managers would need to justify hiring humans instead of using AI — the NBER report lets a lot of air out of the balloon.
The researchers collected the majority of their data in Denmark, a country with high AI adoption and detailed record-keeping. They found that AI adoption had yet to lead to massive layoffs, but neither did it deliver considerable financial advantages to either employers or employees. Instead, most of the hype is based on corporate FOMO and a desire to keep up with rivals.
The report says that earlier studies focused mostly on areas where the time-saving advantages of AI chatbots were most obvious, like with customer support specialists, who are being replaced en masse. Humlum and Vestergaard looked beyond the obvious and studied 7,000 workplaces that included fields such as law, journalism, bookkeeping, financial advice, and teaching.
“Software, writing code, writing marketing tasks, writing job posts for HR professionals — these are the tasks the AI can speed up,” Humlum told Fortune, saying that earlier studies weren’t wrong, just incomplete. “In a broader occupational survey, where AI can still be helpful, we see much smaller savings.”
Employee time freed up by AI was used for other work tasks — including fixing mistakes created by AI in transcription, or making it difficult for students to use AI to cheat.
Earlier this year, 2024 Nobel prize winner Daron Acemoglu predicted that AI adoption will increase the U.S. GDP by only as much as 1.6 percent in the next decade, while productivity would only increase 0.05 percent.
“We’re still going to have journalists, we’re still going to have financial analysts, we’re still going to have HR employees,” he told MIT Technology Review. “It’s going to impact a bunch of office jobs that are about data summary, visual matching, pattern recognition, etc. And those are essentially about 5% of the economy.”
Acemoglu went on to suggest that “hype is making us invest badly in terms of the technology.” “We’re using it too much for automation and not enough for providing expertise and information to workers,” he said.