Will it also reshape how we understand the economy?
Harvard’s Sendhil Mullainathan is one of a small number of economists who has delved into the world of machine learning, the subfield of computer science concerned with using algorithms to learn from data. His research, along with the work of Stanford’s Susan Athey, suggests that while machine learning may not revolutionize economics, it will greatly expand its possibilities, and more economists should be using it.
When economists analyze data, Mullainathan explains in a recent paper (pdf) with Harvard PhD student Jann Piess, they are typically trying to understand the causal relationship between two variables. For example, an economist might sift through real estate data to figure out how much the size, location, or other factors affect how much people are willing to pay for a home.
Machine learning, in its current form, won’t tell you much about those relationships. Rather, what an algorithm can do well is take the relevant information and make a really good prediction about how much the house is worth, without providing an explanation why.
Yet even if machine learning doesn’t offer much direct insight, Mullainathan and Piess argue that there are four key ways economists can take advantage of the technology.
First, machine learning can be used to collect new sources of data. Economists have already been able to convert satellite data into estimates of economic growth, as well as to measure neighborhood income levels in Boston and New York using Google Street View (pdf). Machine learning also allows economists to analyze language as data: Algorithms can be used to classify whether the sentiment of the text in financial filings or newspaper articles are positive or negative.
Second, machine learning is useful for testing major economic theories. Economists have put the machines to work on efficient market hypothesis, the idea that share prices always incorporate all relevant information, making it impossible to predict future stock fluctuations. But they found that a firm’s past earnings indeed have predictive value for future stock prices, violating the theory.
Third, for some economic problems predictions are crucial, and thus machine learning may be a useful tool. For example, judges must decide whether to set bail for a defendant based on predictions of how likely a person is to show up for a future court appearance. Algorithms can assist judges to make better decisions on this. They can also be used to target antipoverty programs at the people most in need of them, say Mullainathan and Piess.
Finally, many older analytical techniques involve tasks in which the precision of machine-learning technology could be useful. For example, when economists run long-term surveys, people often drop out over time. Machine learning could be used to verify whether those dropouts were random or predictable, which affects how the survey results should be interpreted. Also, when an economist wants to assess the impact of a program which was not randomly selected, machine learning can be used to identify groups of people who are similar to those who received the program to compare them with, creating a controlled trial.
To date, machine learning is not employed by many economists and is not a feature of mainstream economics curriculums. But expect the next generation of economists, more comfortable with computer programming and big data, to fully harness machine learning’s possibilities.