Trump’s impeachment offered an opportunity to see exactly how the Trump campaign tests its fundraising ads on Facebook. This technique, called A/B testing, was pioneered by the Obama campaign, but used fruitfully by Trump’s to maximize the money raised from his fervent base of supporters.
Quartz analyzed 760 ads that began running hours after Trump was impeached Wednesday evening (Dec. 18) that included the same video, but four different variations of the accompanying text. At first the Trump campaign spent approximately equal amounts of money on each ad. But by Monday (Dec. 23), one variant had pulled far into the lead, with about 60% of the money put toward that version.
Presumably, the Trump campaign spent more money on that variant than the others because that ad was most successful at raising money.
Can you guess which variant of the text best harnessed Trump supporters’ anger over impeachment for donations?
- As I stepped off Air Force One for last night’s rally, the Dems voted to IMPEACH me. I need a list of donors in the next hour who stand with me. DONATE NOW for 3x matching!
- My team is sending me a list of everyone who does their part and donates before MIDNIGHT. I need you on my side in this Impeachment Fight! Please contribute ANY AMOUNT in the NEXT 24 HOURS and your gift will be instantly TRIPLE-MATCHED
- This is an attack on Democracy. An attack on freedom. An attack on everything we hold dear in this country. It’s US against THEM in this impeachment war. And we need to strike first.
- Nancy is a liar and a fraud. We just witnessed the most PARTISAN SCAM in American history and it resulted in the house officially IMPEACHING me. My team is sending me a list of everyone who does their part and donates before MIDNIGHT. I need you on my side in this Impeachment Fight!
The answer is No. 1. The Trump campaign spent at least $78,100 for that ad in about four days. The campaign spent at least $40,000 on the other variants combined. No. 3 appears to have been a particular failure, as the campaign spent closer to $3,000 on it.
Facebook reports ad spending in ranges, and the actual tends to fall toward the bottom of those ranges. The ranges reported for these ads, through midday Monday, were:
- $89,700-$116,175
- $10,700-$31,391
- $3,300-$24,981
- $46,700-73,488
The video that appears with all of the tested ads begins with Trump saying: “This isn’t an impeachment, this is a coup.”
Each combination of text and video also appeared in about 100 seemingly-identical variants; it’s almost certain that each variant was targeted to a different slice of the population, though Facebook doesn’t reveal information about the advertiser’s targeting choices. In addition to these ads, the Trump campaign continued running many that pre-dated the impeachment.
The success of version No. 1 can’t be chalked up just to the ad’s creative; Facebook also uses machine learning to assign ads a “relevance score,” which affects how often the ad is shown to its audience.
In A/B testing, advertisers run several variants of an ad and, after a short period of time, measure which ad led to the best results, like clicks, purchases or, in this case, contributions received. The technique was famously put to serious use in 2008 by Obama’s first presidential campaign, though Republicans pioneered it a few years earlier, according to “Prototype Politics,” a book about tech in political campaigns by Daniel Kreiss, a professor at the University of North Carolina. It has been used to used to determine which color of a button prompts the most donations, along with these sorts of slight variations in text.
The technique, also common in non-political advertising, was used to great effect in 2016 by Brad Parscale, then Trump’s digital director and now his campaign manager. Parscale said that the Trump campaign ran up to 100,000 variations a day.