In the never-ending war between art and commerce, commerce seems to be winning. The latest proof emerged earlier this month with Vanity Fair’s look at the workings of Yahoo’s heavily-trafficked homepage. Sarah Ellison reports:
The home-page editing staff has been cut way back from its peak, and the reliance on human intervention declines as the engineers, who have undergone their own staff reductions, insist that the algorithm is improving all the time and serving up what users actually want—not what the New York media types think they should read.
This isn’t an isolated incident. Analytics and algorithms have emerged as key weapons in capitalism’s brawl with journalism across the web. And the struggle has real consequences for all of us.
In the one corner is technocapitalism, which drives us toward computer-curated editorial processes that use “objective” quantitative measurements to shape the news. This approach holds that media companies ought to give readers what they want, and that what readers want is more Kim Kardashian. The goal of media companies that embrace this approach is to capture user attention, monetize it, and prove their value in the only way one can—by making staggering sums of money.
In the other corner is humanism, which tells us that there is no such a thing as an “objective” measurement of a story’s value. Humanism prizes human-curated editorial processes, and it values media not as a function of how much money it makes, but the civic movements it creates.
These tensions have existed since the first newspaper rolled off the printing press. But the internet altered the balance between capitalism and humanism. Analytics tempted publishers with promises of “better,” clearer ways of tracking reader behavior. But the truth is this that web metrics have largely failed to benefit either publishers or readers. Most of the time, they don’t actually work the way they’re supposed to.
A question of performance
News outlets have taken a lot of hits lately for the disproportionate amounts of coverage given to some world events and not others. “You’ll see people complaining that the media doesn’t give as much prominence to terrorism atrocities outside of Western Europe as it does to those that take place in cities like Paris or Brussels,” the Guardian’s Martin Belam writes at Medium. “The data shows it is much, much harder to get people to read those stories.”
The Guardian’s analytics showed that while several stories about the Lahore attacks in Pakistan were featured prominently on the site, the terrorist attack failed to garner much traction among readers. Lahore “was being out-performed by stories about a logic puzzle, the ruins of Palmyra, a woman having lots and lots of sex but being dissatisfied about it and John Kerry’s comments on the US Presidential race,” Belam writes.
The definition of a story’s performance is a nebulous one. It can mean how many people shared a story, or how many people read it. Performance also has to do with how influential a story is and how many people emerge from it more informed, although these outcomes are impossible to track.
But what Belam meant by high performance, and what it generally means across the web, is that more people clicked on it. As Belam wrote to me in an email, he was referring to the number of “people on the page.” This would be a demoralizing measure of success even if it were accurate, but it’s made worse by the fact that pageviews tell us almost nothing about readers’ behavior.
Pageviews are meaningless
“Page views don’t matter anymore,” writes Julia Greenberg at Wired. “But they just won’t die.” This is the paradox of internet metrics: There’s an emerging consensus that the pageview is a terrible way to measure a story’s performance, but no one has been able to dethrone it once and for all.
In the halcyon days of the web, the pageview was a relatively simple metric. A human user navigated her browser to a URL, which caused the publisher’s server to deliver some HTML (the story) to the browser. The browser then put the story on the page. At that last stage, the publisher would be able to register a new pageview.
But improvements in computer processing power and web technology complicated the relationships between a user’s browser, the URL, and the server. We put layers of JavaScript between the server and the user’s browser, so much so that sometimes it’s hard to say what should count as a “view” of a “page.”
Consider this: When you complete this article and scroll all the way down to the bottom, you’ll find more articles waiting for you. Quartz isn’t alone here; infinite scroll graces us on sites of Time, the New Republic, the Intercept, the Marshall Project, and many more.
What does a “pageview” mean on a site with infinite scroll? Should we count a pageview as soon as the HTML for the article is on the page, which may appear before the reader can actually view the article? Perhaps we should count a pageview as soon as the article actually comes into view—so, one pixel? One hundred pixels?
Once, a click—that is, navigating to a URL—meant that a user was seeing a piece of content. But in our brave, JavaScripted world, URLs don’t necessarily mean the same thing across sites. Since Quartz and Vox display articles in different ways, for example, what counts as a “page” being “viewed” differs between sites. And that’s a problem, because metrics aren’t really useful if they mean different things across the web.
Speaking of Vox, here’s a pageview on the site:
If you were to arrive upon this page, would you be viewing the article—a solid, meaty piece about lead poisoning in American cities—or car ad? The URL—and the pageview metric—says you’ve arrived (huzzah—content!). But you haven’t actually seen the article yet.
Indeed, there’s no guarantee that clicks and pageviews mean that you’ve actually seen any content at all. They certainly don’t mean that you read the article—let alone that you understood or appreciated it. And yet, when we talk about “popular” content, we’re probably talking about an article that garners a lot of pageviews. Popularity on the web is a lot like popularity in high school: empty.
The search for the “God metric”
Tony Haile, the founding CEO of the web analytics company Chartbeat, would argue that the problem is that we just picked the wrong metric. Under Haile’s tenure, Chartbeat pushed “attention” metrics: the time that you spent on a page actually reading an article.
In my interview with Haile, he pointed out that, in 2015, the piece we paid the most attention to (with some caveats) was the Atlantic’s longform feature, “What ISIS really wants.” A 10,000 word, scholarly analysis of Islamic State is hardly the clickbait we’ve come to expect to top the lists of best-performing articles. That’s because pageview metrics don’t distinguish between users who close the window in disgust after several seconds and readers who complete the article. Attention metrics do.
But attention metrics aren’t perfect either. When publications rely on any single metric, they open themselves up to real danger. BuzzFeed co-founder and CEO Jonah Peretti, in a wide-ranging interview with Felix Salmon, says: “what you see in the industry now is people jumping around and trying to find the God metric for content.”
The problem, he continues, is that publishers inevitably start trying to game metrics as soon as they settle upon them as accurate measures. Peretti explains: “If you create a long, meandering, boring story that’s just good enough to keep people reading, they might spend more time on that story than the short, condensed one that just tells you what you need to know.” In this kind of case, the attention metric “will tell you to do the wrong thing.”
Indeed, there’s already been some backlash against “engaged time” analytics. Nieman Lab’s Joshua Benton found this gem in Cory Bergman’s Mobile Media Memo: “Part of the problem is the industry’s fixation on ‘time spent’ as an engagement metric.” He notes:
It’s often better to maximize “time saved” rather than time spent, especially on a per session basis. Imagine, for example, that you can get the nugget of a 2-minute video in a 24-second clip, or 80% of the value in 20% of the time. For most mobile users, that’s more delightful than watching the full 2 minutes.
But Haile stands by attention. “Everyone kind of nods wisely when you say ‘there is no God metric’, but it doesn’t really reflect the reality of how people make money in this business,” he told me. While Haile acknowledges the obvious truth that reality is too complex for a single metric, he also subscribes to the business truth that reality has to be dumbed down in order for capitalism to work.
“You don’t have a carpenter with just one tool in their tool bag,” he says. “That’s totally true. However, it’s also not usefully true, because the fact of the matter is that our businesses run on currencies which tend to be singular metrics.”
And that brings us back to the question of how publishers can strike a balance between capitalism and humanism. Haile acknowledges that metrics are only one piece of a larger puzzle—“I’m not crazy,” he says. But what does it say about the state of journalism when we run our businesses on metrics that don’t match reality?
“I’m just as bad as anyone else,” Haile says. “You get a couple of pints in me, and I start kind of thinking about the future of metrics and all the wonderful metrics that we could use, and then what brings me back down to earth is not only the hangover but also the understanding that we have to be able to adapt these metrics to the world we live in.”
Haile’s logic is tempting—but it’s also circular. Metrics describe the world around us, and our choice of metrics also determines the reality we see. If you compromise on your metrics in order to fit the capitalist model of the world, then that cold, inhospitable picture will be all that’s reflected back at you.
Breaking Urizen’s compass
Humans love to make progress in the way we measure things. We figured out how to measure distance with the length of our feet and wound up with digital calipers. We went from analyzing the size of an object with our naked eye to tunneling electron microscopes. And we began measuring a publication’s success by its circulation rates and ended with cookies and attention-trackers.
There’s utility, to be sure, in discovering these ever-more sophisticated and precise ways of measuring. But we also need a good dose of humility. What history has taught us is that the more we measure, the more we realize how little we know.
In The Givenness of Things, a book of essays on faith excerpted at the Nation, Marilynne Robinson writes that “the fundamental character of time and space is being called into question.” Next to what we know now, she says, “Einstein’s universe seems mechanistic in comparison. Newton’s, the work of a tinkerer. If Galileo shocked the world by removing the sun from its place, so to speak, then this polyglot army of mathematicians and cosmologists who offer always new grounds for new conceptions of absolute reality should dazzle us all, freeing us at last from the circle of old Urizen’s compass. But we are not free.”
Instead, we persist in imagining the world as a closed system made up of solvable equations. It’s true that we have more data than ever before about how readers interact with journalism. But the dumb, wooden ruler is just as effective as the most sophisticated AI algorithm in telling us how valuable a piece of writing is—which is to say not very effective at all.
Haile believes that as long as engineers are told to optimize for the wrong metrics, “you’ll see this kind of tension between the mission and the metrics.” But he’s hopeful about finding a happy compromise between capitalism and humanism.
“I don’t think there’s a natural tension between engineers and journalists,” he tells me. We simply need “to choose the right performance indicators.” It’s seductive—and convenient—that Chartbeat can provide both a better performance indicator and the tool to track it.
Readers versus users
I’ve been using the words “reader” and “user” more or less synonymously, as if they weren’t indicators of two dramatically different paradigms. In truth, a reader, like a citizen, is a very different kind of person than a user or a taxpayer. As Robinson writes in a different essay on the fate of public universities, “While the Citizen can entertain aspirations for the society as a whole and take pride in its achievements, the Taxpayer, as presently imagined, simply does not want to pay taxes.”
Similarly, while the Reader is an active participant in the social contract between journalism and the people it describes, the User, as presently imagined, merely wants to watch watermelons explode. The User’s fickle tastes demand obsequiousness. Publishers loathe their stupidity and fear their power.
Contrast this with mutual respect and value. Richard Rodriguez, also writing in Harper’s, says that “In the nineteenth-century newspaper, the relationship between observer and observed was reciprocal: the newspaper described the city; the newspaper, in turn, was sustained by readers who were curious about the strangers that circumstance had placed proximate to them.” These papers were founded on the idea of a reciprocal relationship between journalism and its reader.
To be sure, there’s a sick kind of symbiosis involved in so-called metrics-driven journalism. Content farms produce what the metrics say users want, and users give their attention, against which content creators can sell ads. There’s even a quasi-populist argument to make here: Data are better than media elites at deciding what readers want! But if you suggest to these self-styled populists that we ought to put the question directly to readers, then get ready for the quick reply, “Don’t listen to users!” Perhaps this attitude makes sense if you’re trying to design an app; the way that users behave matters more than what they say they want. But it makes a lot less when you’re deciding what stories are worth telling. The “populist” argument relies upon a hollow definition of populism.
And so it’s no surprise that when publications treat readers as users, they find what they expect to see: vapid, venal, flaky masses who constitute a collective problem to be solved by the data wizards of Silicon Valley.
But readers aren’t the problem. Readers are the solution. If publications can reclaim the reciprocal relationship between themselves and the people for whom they tell stories, then they can nurture a different kind of growth. It would not be the fast, social media-driven pageview growth that we see from venture capital-backed media upstarts. It would not be wide growth. Rather, it would be deep growth: fewer users but more loyalty and impact.
Haile insists that advertisers need a metric in order to decide to which publications they ought to give money: “People are going to ask you for some kind of singular currency so that someone can make capital allocation decisions.” I don’t doubt advertisers, venture capitalists, and “serious” media thinkers will ask this. But it is a failure of imagination to assume that we can’t create new systems that help us decide what kinds of writing is worthwhile.
Right now, credible publications know instinctively that there is something inherently valuable about good writing on the web. But we can’t seem to find a good way to prove it, and so we’re stuck asking the wrong questions: How can we sustain good journalism without caving to clickbait? Personally, I think it would be a lot more interesting if publishers stopped focusing on the problem and started thinking about the possibilities: How can journalism leave a greater impact on our readers?
After all this, I have to confess that I do believe there is some place for metrics in journalism. Figuring out how our readers navigate the page, for example, is a valuable endeavor. Tracking can help us there. But we need to remember that metrics are mirrors, not lenses. In their reflection, we see ourselves and our biases just as much as the world around us.