TESTS FAIL

Why high-stakes entrance exams are a terrible idea

A particularly bad day of smog—or in this case, a bad sandstorm in Israel—is enough to dent performance on a college entrance exam, reduce college enrollment, and put a dent in wages 10 years after the test according to new NBER working paper.

Victor Lavy and his co-authors looked at score data from Israeli students who took the Bagrut, a series of high-stakes exams that play a significant role in university placement, and their education level and wage data 10 years later. They found random variation in air pollution can have a substantial negative impact. A 10-unit increase in a measure of air pollution (PM2.5) reduced composite scores by .023%, time spent at university by 0.15 years, and caused a 2% decline in wages 10 years later.

An additional point on the composite exam score is worth about $18 a month in monthly earnings 10 years later, and each point increase increases chances of enrolling in post-secondary education by 1.9%.

The effect of pollution was particularly strong for those with asthma (supporting the idea of a physiological mechanism), larger for boys than girls, twice as large for weaker students, and a fifth higher for those of low socioeconomic status.

For the researchers, air pollution was a truly random component, helping to separate out the effects of having a good test day and the effect of higher cognitive ability over time.

Pollution impacts cognitive performance, likely because aspirating fine particulate affects oxygen intake, which affects the brain. Extremely polluted days have a bigger effect. Students don’t pick their exam date, don’t know pollution in advance, can’t reschedule—and Israel has frequent sandstorms, creating lots of variation in the data.

Around the world, exams like the SAT determine a large portion of people’s lives. Unlike the SAT, many exams don’t allow retakes. This means students who can afford prep classes or have strong support systems are at a great advantage. If, as the NBER study shows, random variables on the day of the exam can influence the results, then it calls to question the point of such testing.

Pollution is a relatively small random factor, and there are countless others. Putting so much emphasis on one or a few exams is inevitably inefficient. Supporters of such testing say it creates a meritocracy, but theres an uncomfortable element of luck that’s separate from the already very real questions of privilege and inequality

“If completely random variation in scores can still matter 10 years after a student completes high school, this suggests that placing too much weight on high-stakes exams like the Bagrut may not be consistent with meritocratic principles,” the authors write.

Put more bluntly, the course of someone’s life shouldn’t be altered by bad weather on an exam day. The authors argue these tests can lead to inefficient allocation of talent if a bad score matches a high-ability person with the wrong career or educational institution.

The alternative is to reduce the random element, to diminish the importance of high-stakes tests in favor of the overall record. The high-stakes system, prevalent around the world, manages to create a tremendous amount of stress for no particular reason.

home our picks popular latest obsessions search