Pandora tested listeners’ tolerance for ads by experimenting on 35 million users

Image: Retuers
We may earn a commission from links on this page.

Pummel old people with advertising, and they will pay for a subscription service. Do the same to the young, and they will find another option.

These are among the insights from a massive advertising experiment conducted by the music streaming service Pandora on its users. The findings from the experiment, recently made public, may be useful for the many advertising-dependent internet services to understand how revenue-generating interruptions change users’ behavior.

In June 2014, Pandora took nearly 35 million of their 80 million active users in the US, Australia, and New Zealand—the three countries Pandora was active in at the time—and randomly put them into groups to receive more or less advertising through April 2016, or just under two years. People who did not use Pandora on mobile or already paid a subscription not to hear advertisements were excluded from the experiment.

At the start of the experiment, the average Pandora listener received six ads per hour. After the trial commenced, the luckiest set of users received only three advertisements per hour, while the most unfortunate received 12. Most were served a number of ads somewhere in between.

The researchers found that the people who received more advertisements didn’t listen as much. One extra advertisement per hour led to about 2% less listening in the final month of the experiment.

More surprising was that each additional advertisement seemed to cause the same 2% drop. Receiving four advertisements instead of three caused a 2% drop, and so did getting eight instead of seven.

Breaking up the advertisements didn’t matter much, just the total time Pandora forced users to listen to them. Receiving six interruptions every hour with two ads had the same effect on listenership as six interruptions with one ad. Pandora released only relative gains and losses, not the exact numbers of listening minutes for each case.

The researchers attributed the ad-related decline in listening to three different mechanisms:

  1. Shorter sessions. 18% of the decline in total hours listened was due to users spending less time on the service on active days.
  2. Fewer active days. 41% of the decline was a result of active listeners using the service on fewer days. Perhaps knowing it would be a less pleasant experience made people less likely to fire it up.
  3. Leaving the service. The remaining 41% of lost listening time was due to people who stopped using Pandora altogether. If you are getting a lot of advertisements, Spotify, Apple Music, or good old terrestrial radio can become more appealing options.

For Pandora, serving more ads means generating more revenue. Another benefit of boosting ads, its research shows, is that upping the ads customers receive might convince more of them to pay for ad-free subscriptions. That’s what happened in Pandora’s case, but not equally among all of their listeners.

For listeners 55 and older, each additional advertisement per hour made them 0.21% more likely to become subscribers. This was more than double the effect that an additional ad had on 18-24 year olds. At the same time, people under 55 were far more likely to stop listening altogether because of the new advertisements.

The lesson for ad-supported internet services? Old people don’t want you to waste their time, while the young don’t want to waste their money.