Quick. Which of these two products would you buy on Amazon: Thing A, which has an average rating of 2.7 stars and 258 reviews, or Thing B, which has an average rating of 2.7 stars and 14 reviews?
Did you pick Thing A? Most people pick Thing A. In a study recently published in the journal Psychological Science, subjects asked to choose between hypothetical products on Amazon’s website preferred more-reviewed items to less-reviewed ones in virtually all circumstances. This isn’t terribly surprising. Though a tactic handily disproven by research and observation, humans have a natural tendency to use popularity as an indicator of quality.
But the researchers also found that when it comes to Amazon’s ratings system, this bias toward popularity can actually lead people to choose what is statistically likely to be a worse-quality product.
After analyzing 15.6 million reviews of more than 350,000 products on Amazon.com, the team built a statistical model that suggested shoppers choosing between two products with similarly poor ratings should pick the one with fewer reviews. In purely rational terms, more reviews of a poor-to-mediocre product (think three stars or fewer) should only increase a shopper’s confidence that it is, in fact, a poor-to-mediocre product, suggesting people should steer clear of it.
But when they posed such choices to actual humans, people did just the opposite. People’s preference for an item only increased as the number of reviews went up—even for poorly-rated products.
“Our research suggests that, in some cases, people might take this information [about review numbers] and make systematically bad decisions with it,” said the study’s lead author, Derek Powell, a postdoctoral researcher in psychology at Stanford University.
In one experiment, researchers showed participants a pair of products with identical ratings—3.1 stars—but different review counts (29 in one case, 154 in the other). The researchers’ statistical model showed only a 40% chance that the more-reviewed product was the better one—it had, after all, received far more middling reviews than the less-reviewed alternative. Yet more than 90% of study subjects said they’d rather buy the more-reviewed option.
In fact, across 11 different conditions in which the more-reviewed product was statistically likely to be inferior, subjects opted for the more–reviewed product 72.3% of the time. When it comes to consumer choices, a bias toward popularity can be self-defeating.