Amazon’s algorithms are misleading customers and causing them to spend way more than they should

Amazon $AMZN’s algorithms are causing customers to pay significantly more than they should for popular products, while promoting those that benefit the corporation, according to an investigation by journalism non-profit ProPublica published today (Sept. 20).

The e-commerce company doesn’t disclose exactly how its algorithm selects products from certain merchants, or itself, to be recommended in its “buy box”—the area on a product’s page where customers can click to add an item to their baskets. To shed light on how it works, ProPublica wrote a program that made it seem like a non-Prime member located in Brooklyn, New York, was shopping on the site. The program scraped Amazon’s listings for 250 popular products, from electronics to household supplies, over a few weeks this summer. Here’s what it found:

• A customer who bought everything recommended by Amazon’s buy box would have paid 20% more than if he had bought the same products at their lowest price on the platform. That translates to an additional $1,400 for the basket of goods. Update: The customer must have bought the items individually, and must have been careful to stay below the $49 free shipping threshold, according to Amazon and ProPublica.
• The average difference between the recommended product and the cheapest listing was $7.88.
• Nearly all the sellers in the buy box who did not have the cheapest product listing were either Amazon itself or “fulfilled by Amazon” merchants. Just 6% were independent merchants.
• Amazon doesn’t count shipping costs for its own products, or those it fulfills for others, when customers rank a product by “price + shipping.” This means Amazon was ranked higher in the list 80% of the time. Update: The rankings are correct for Prime subscribers, and for those who get free shipping by buying $49 or more in eligible products, according to ProPublica.

In a statement to Quartz, Amazon said: “With Prime and Super Saver Shipping (which requires no membership and ships orders above $49 for free), the vast majority of our items ordered – 9 out of 10 – can ship for free. The sorting algorithms the [ProPublica] article refers to are designed for that 90% of items ordered, where shipping costs do not apply.”

Amazon says price isn’t the only criteria for winning the top spot in the buy box. “Price is one component of being customer-centric,” Amazon said in a statement to ProPublica, which has made the data it scraped available for download here.

The company’s publicly disclosed reasons for getting into the buy box are vague. “Sellers must meet performance-based requirements to be eligible to compete for [buy box] placement,” according to information for users on its site. It goes on to note: “For many sellers, [buy box] placement can lead to increased sales.”

The buy box mystery has serious repercussions for independent merchants. Getting preferred placement has driven sellers to outsource their product fulfillment to Amazon, in return for fees amounting to 10% to 20% of sales. Even being “fulfilled by Amazon” and offering a lower price doesn’t guarantee a position in the buy box, as ProPublica discovered for one merchant: “Amazon is not really fair in terms of competition, but we don’t have much choice. We have to be there,” the merchant, Gokhan Erkavun, told ProPublica.

As ProPublica notes, Amazon is far from the only platform accused of tilting its algorithms in its own favor and sometimes to the detriment of users. It’s precisely why the European commission is investigating Google $GOOGL over its comparison-shopping service. The commission’s charge sheet against Google says the search engine favors its Google Shopping service over others, among other advantages. That investigation is ongoing.

Update [Sep. 21]: This article has been updated to include a statement from Amazon.