People are sensitive. A little bit of encouragement can change the trajectory of our lives.
This appears to be particularly true when it comes to the results of the advanced placement (AP) test—the exams high school students take to get college credit at American universities.
A recently published study shows that students are more likely to major in a subject if they received a higher score on the AP test for that subject, even compared to students who performed almost identically in raw numbers, but whose results are rounded to a lower score. The researchers, from Stanford University, Harvard University and the College Board, find that the effect is primarily a response to positive feedback.
Ever year, millions of students take AP tests, choosing from over thirty different subjects—more than 2.6 million students took over 4.7 million AP Exams in 2016. Based on answers to multiple choice and free response questions, students receive whole integer scores from 1-5 (5 is the highest). Most schools in the US will give college credit for a score of three or higher, though others require at least a four.
While the scores the students receive are round numbers, their performance is scored more granularly by the College Board, which grades the tests. A student’s raw score could have been just barely high enough to get a 3, but also could have just narrowly missed a 4, and they would never know it. This means there are many students who get similar raw scores—like a 2.99 versus a 3.01—but receive entirely different signals about their ability. The researchers used the sharp cutoff between integer scores as a natural experiment to test people’s sensitivity to positive feedback.
Students were more sensitive than they assumed. The researchers found that obtaining a higher score meant that students were 5% more likely to major in that subject, and for certain subjects the effect reaches 30%. And it is not because they are more likely to get college credit. The impact is greatest for students who receive fives instead of fours—a difference that rarely makes any practical difference to college credit obtained.
The researchers happened upon the finding almost by accident. “When we were plotting the raw AP scores, we found these linear upward trends in the probability that a student would major in a subject depending on their raw score,” said Michael Hurwitz, one of the authors of the study. “But we saw notches in these trends that coincided exactly with the integer boundaries, which set off a lightning bolt in our brains.” The notches suggested something strange was going on for students who just barely received a higher score. The chart below shows the notch between four and five.
Students are most affected by high scores in subjects that aren’t graduation requirements at most colleges such as economics, biology or psychology. Getting high scores in these kinds of subjects is likely to lead a student to believe they will be good at them, thus they choose them as majors.
Hurwitz points out that the signaling from the AP tests only explains a small portion of why a student might choose a particular major. Aptitude, a good teacher in the subject, and career aspirations are more important. But a slightly better AP test score is often decisive. A little positive feedback can go a long way.
The story was updated with corrected statistics on the number of students who took AP tests in 2016.