As the Alexas and Siris of the world proliferate, it’s only a matter of time before we all have personal AI assistants to tend to our every whim. But as AI gets smarter, our roles will reverse, and we will inevitably start working for AI instead of it working for us.
As the Alexas and Siris of the world proliferate, it’s only a matter of time before we all have personal AI assistants to tend to our every whim. But as AI gets smarter, our roles will reverse, and we will inevitably start working for AI instead of it working for us.
This may sound like a distant sci-fi dystopia, but it’s already happening to Uber $UBER and Lyft $LYFT drivers. An algorithm serves the role of manager, dictating how much they’ll be paid, mandating performance levels like acceptance and cancellation rates, and setting a schedule based on consumer demand and delivery services. Our tools once shaped us—but now we serve as their tools.
Join 500,000+ readers who start their day with Quartz.
By subscribing, you agree to our Terms of Service and Privacy Policy.
The truth is that nearly any cognitive task—whether determining a schedule, assigning tasks, providing feedback, or making a decision—can be broken down into a series of instructions. And that’s all an algorithm really is: a set of steps that should be followed to achieve a given outcome.
If nearly any task can be governed by an algorithm, that means nearly any management role can be performed by software. The white-collar jobs that were assumed to be safe from automation may in fact be even more vulnerable—and your boss’s job might be automated before yours is. Before rushing into this brave new world, managers, employees, and AI engineers should consider the benefits and failings of AI management.
Partly due to the fact that humans can’t keep up with their rapidly scaling workload, AI management is more popular than ever. For example, when Uber was expanding in 2014 and 2015, the company was adding up to 50,000 drivers every month. Even the best managers wouldn’t be able to schedule these drivers and make sure they’re performing at their best, so some system of virtual management had to be put in place.
Although having an AI boss sounds like something out of a futuristic dystopia, it’s much less HAL 9000 and more like Bill Lumbergh from Office Space: a middle manager that handles the more mundane, repetitive tasks involved in coordinating a large group of employees. After all, even the best, most conscientious human bosses are still fallible. They make poor decisions when tired; their biases (whether unconscious or not) affect how they treat employees; and their egos and personalities can determine how well they work with others. Algorithmic bosses would theoretically eliminate those problems: They could make decisions constantly and consistently, reduce bias, and effortlessly modify their management style to the needs of employees.
So why haven’t we seen more management roles be assisted or replaced entirely with AI?
Humans are hard to update. Changing a rule in an algorithm is relatively straightforward: Just push an update, and the entire system refreshes itself instantly. Unfortunately, human operators aren’t as flexible and can only handle so much change at once. (And despite their protests otherwise, humans often don’t like change in the first place.) Moving toward this style of management is a hurdle many don’t want to jump over.
Humans build in bias. As long as humans are writing the algorithms, there’s the possibility of unconscious bias being incorporated into the algorithmic boss itself. For instance, a software program designed to assess the likelihood of criminals to reoffend was biased against African Americans—even though race was not one of the questions accounted for in the algorithm.
Humans don’t always know what the answer should be. All the steps in an algorithm might be right, but they won’t help if they add up to the wrong answer. We don’t even understand what truly makes for effective management: Despite management fads coming and going, Gallup engagement scores have been stuck at around 32% for the past 15 years.
Humans dislike inequality. Automation and AI could lead to conflict between work cultures and greater inequality in society at large: Those who work on or with software, and those who are managed or replaced by it. In fact, a recent survey indicated that 76% of Americans are concerned that automation will exacerbate economic inequality.
Humans find loopholes in systems. Any incentive structure can be gamed, and if people don’t feel like they’re being treated fairly, they’ll quickly look for ways to even the score. When Uber drivers felt that the algorithm was making it difficult for them to achieve their bonuses, they formed online communities to share information and coordinated practices to trick the system, such as logging out of Uber in order to trigger surge pricing.
Despite these concerns, Silicon Valley runs the risk of pushing forward with the technology first and figuring out the ethics later. They’re creating modern-day Frankenstein’s monsters: experiments designed to highlight the best of humanity but have the potential to become twisted and terrifying through neglect.
It’s unlikely that any pushback will come from the Zuckerbergs and Kalanicks of the world—they’re the ones who stand to profit the most from the shift to greater automation. Instead, if recent mea culpas are any indication, core employees a layer or two down from the CEO are more likely to question the impact of technology like AI on society. At Facebook $META, for instance, the employees who built the ad platform are now lamenting how their work may have allowed Russians to spread fake news, while the engineer behind the “Like” button now questions its addictive nature.
For those employees currently engineering the future of automated managers, here’s a basic code of conduct:
