Posts about engagement

Good metrics, bad metrics

Screenshot 2014-03-10 at 10.28.01 AMChartbeat CEO Tony Haile writes an important piece about bad media metrics online. He pokes holes in the value of the click as the be-all-and-end-all of media measurement. He reveals that sharing turns out to be a bad measurement of engagement and value because we often don’t read what we “like” or share (we just bother other people with it). He deflates the value of native advertising, demonstrating with hard data that readers understand the difference between real content and — let’s call it what it is — advertising and they quickly abandon it.

The bottom line of Tony’s data is bad news for cynical publishers who have tried to manipulate readers with link-bait headlines and lists, and who are trying to pull the wool over advertisers’ eyes by selling them link-bait listicles and so-called native advertising. Certain emperors have no clothes. The readers know it. The advertisers will wake up and realize it.

But that’s the bad news.

Where we should turn the discussion next is to what the right metrics for media should be. As they say, you get what you measure. So what should we measure? How do we create positive feedback loops that improve the news, not degrade it as unique users, pageviews, and other relics of mass media have done?

I’ll start with the most important and most difficult thing to measure: outcomes. Were people more informed because of what we gave them? Did they accomplish what they wanted as individuals (Sally got new health insurance and saved money) or as communities (Riverdale cleaned up that messy park)? I just had breakfast with Robert Rosenthal of the Center for Investigative Reporting and he told me they start the process of reporting by considering impact and they end by trying to measure it. Why deal in bad proxies for good journalism, based on popularity, when we could get to the reason journalism should exist: to improve the world?

In his book News: A User’s Manual, Alain de Botton says that news has “the power to assemble the picture that citizens end up having of one another; the power to dictate what our idea of ‘other people’ will be like; the power to invent a nation in our imaginations.” And it has the power to help us get there. (Many more quotes in my post about the book, here.) Mark Zuckerberg says that platforms, including news, should offer communities “elegant organization.” These are higher aspirations than mere exposure.

On a tour of technology companies in Silicon Valley a few weeks ago with my dean, we talked about metrics and found different measurements being used for different platforms with different goals. Ev Williams’ Medium values total time spent reading. That is appropriate for a platform that wants to get people to explore ideas in depth — and I find I’m spending more time there reading more posts; it’s working.

Attention, in the form of time spent, is used by many in media as a measure of engagement. But that’s not always the case. Attention can also be another egocentric media metric: how many people come to look at my stuff; how many pages of my stuff do they look at; how much time do they spend with my stuff? No, sometimes, the less time spent the better. What if news were more efficient? Sometimes, spending less time to get what I want is the right metric. That metric doesn’t serve the old media business model of delivering as many eyeballs to as many ads as possible. That is why Yahoo shifted from — in the words of cofounder Jerry Yang — getting you in and out with the answer you needed as quickly as possible to instead trying to bombard you with content and keep you around as long as possible to show you as many ads as possible. Attention, in the wrong hands, can also be a corrupting metric.

Cir.ca has a fascinating metric: follows. When a reader follows a story, she is telling Cir.ca, “Please bother me and let me know when something new happens here.” That is a measure of true interest.

Similarly, Flipboard keeps track of how many people subscribe to a publication — and even to an advertiser’s publication. It also watches what people “flip” or save to read later, which strikes me as a much better indication of interest than sharing.

Google has long valued links as a digital version of citation. That has served search well. Google News also uses citations to try to infer which news organization created or is staying on top of a story — if everyone writing about Walter Reed Army Medical Center quotes the Washington Post then there’s a good chance it’s the Post’s story.

Repost.US and YouTube and now Getty Images track embeds — how many people truly want to share a video or an article because they repost it in their own space on the web. The problem with just “liking” or “sharing” on Twitter and Facebook is that there turns out to be no cost for those transactions; it’s too easy to just keep passing things on. Embedding uses my space and affected my reputation with you. I would like to see more such higher friction means of sharing that really do impute engagement.

What is engagement? It’s likely not one measure of one method of interacting with content. It could be that I spend time with something, that I interact with it or the people gathered around it (though don’t we know that comments are no indication of quality), that I save it, that I take action based on it.

We want to find good proxies for engagement in the hopes that they will lead us to indications of quality, which in turn should tell us something about the authority of the creator and the trust the public has in her. None of these is easy to measure, like “likes.”

Another word for engagement is relationship. I have been arguing that we in news should stop seeing ourselves as content factories and start seeing ourselves as members of our communities who are in the relationship business, who use what we know about people to better serve them. Thus, I ask media companies how many relationships they have with the people they serve and what they know about them — what signals they have, enabling them to improve relevance and thus value and often impact. Those are metrics that start with the public rather than with media. Those are metrics that matter.

Why not a reverse meter?

As I ponder the future of The New York Times, it occurred to me that its pay meter could be exactly reversed. I’ll also tell you why this wouldn’t work in a minute. But in any case, this is a way to illustate how how media are valuing our readers/users/customers opposite how we should, rewarding the freeriders and taxing — and perhaps turning away — the valuable users.

So try this on for size: Imagine that you pay to get access to The Times. Everyone does. You pay for one article. Or you pay $20 as a deposit so you’re not bothered every time you come. But whenever you add value to The Times, you earn a credit that delays the next bill.
* You see ads, you get credit.
* You click: more credit.
* You come back often and read many pages: credit.
* You promote The Times on Twitter, Facebook, Google+, or your blog: credit. The more folks share what you’ve shared, the more credit you get.
* You buy merchandise via Times e-commerce: credit.
* You buy tickets to a Times event: credit.
* You hand over data that makes you more valuable to The Times and its advertisers (e.g., revealing where you’re going on your next trip): credit.
* You add pithy comment to articles that other readers appreciate: credit.
* You take on tasks in crowdsourced journalistic endeavors: credit.
* You answer a reporter’s question on Twitter and the reporter uses your information: credit.
* You correct an error in a story: credit.
* You give a news tip or an idea for an article The Times publishes: credit.
Maybe you never pay for The Times again because The Times has gained more value out of its relationship with you. If, on the other hand, you hardly do any of those things, then you have to pay for using The Times.

I’ve been thinking about this, too, in light of a few other trends I’ve seen with newspapers online. First, some that are trying meters are finding that very, very few readers ever hit the wall (which papers are setting at anywhere from 1 to 20 pages). That so few hit the wall is frightening. It means that most readers don’t use these sites much. That’s nothing to brag about. Engagement is criminally low. Second, I’ve seen many sites that get a surprising proportion of their traffic from out of their markets — traffic that is valueless (or even costly, in terms of bandwidth) to sites that sell only local ads. This comes from following a goal of pageviews, pageviews, pageviews — brought in with search-engine optimization — rather than valued relationships.

After hearing a few such stories, I suggested that a site with a meter might want to reward local readers by giving them more free content and charge out-of-market readers by charging them sooner.

You see, that values the local reader over the remote reader. My idea for the reverse meter values the engaged reader over the occasional reader — and even rewards greater engagement. And therein lies, I think, the key strategic skill for news businesses online: understanding that all readers are not equal; knowing who your more valuable readers are; getting more of them; and making them more valuable.

Now I’ll tell you why my reverse meter won’t work: When I spoke with all our journalism students at CUNY about their business ideas on Friday, I asked how many had hit the Times pay wall — many — and how many had paid — few. Abundance remains the enemy of payment. There’s always someplace else to get the news. The Times can make its present meter work because (a) it’s that good [the Steve Jobs exception that proves the rule], (b) it’s still sponsoring — that is, giving a free ride — to its most valuable readers, though that is supposed to end soon, and (c) its engagement is still too low and thus many readers don’t even confront the wall (that needs to change).

So never mind the idea of the reverse meter, but retain the lesson of it: Value should be encouraged, not taxed. Readers bring value to sites if the sites are smart enough to have the mechanisms to recognize, exploit, and reward that value, which comes in many forms: responding to (highly targeted and relevant) ads; buying merchandise; contributing information, content, and ideas; promoting the site…..

The key strategic opportunity for news sites is relationships — deeper, more valuable relationships with more (but not too many) people. Engagement.

The money graph

A new Pew study on the economics of news does not give comfort to news sites planning pay schemes. It also does not give me comfort that we’re wasting precious time futzing over walls when we should be paying attention to the big problems we have — one of which this Pew study points out: dreadful engagement and loyalty — and should be looking at other ways to give and gain value in our relationships with the public. The Pew data:

Over all, the evidence suggests the outlook is difficult both for paywalls and for online display advertising. While most people have not been asked to pay for content, even among the most avid news consumers online, only about one in five at this point say they would be willing to pay, and this does not include less voracious news consumers. At the same time, the vast majority of those online, 8 out of 10, say they basically ignore online ads.

In short, a good deal must change, the data suggests, before the digital age will begin to sustain itself.

About 71% of internet users, or 53% of all American adults, get news online today, a number that has held relatively steady in recent years.

Most of these online news consumers graze across multiple sites without having a primary one that they rely on. Only 35% of online news consumers have a favorite site.

To put it another way, 65% of online news consumers do not have a site that is so important to them that it stands out in their minds above all other sites they visit.

The users who do have a favorite site are pretty faithful. Some 65% of them check in with that favorite site at least once a day.

Yet even among these most loyal news consumers, only a minority (19%) said they would be willing to pay for news online, including those who already do so and those who would be willing to if asked.

Instead, a large majority – 82% – of those with a favorite site said they would find somewhere else to get the news.

Because so few online news consumers even have a favorite site this translates to only 7% of all people who get news online having a favorite online news source that they say they would pay for.

This is a sign of just how much initial difficulty the movement toward pay walls could have.

In sum, there appears to be only a very small cohort of voracious news consumers who have to have their news from a particular site, even if they have to pay for it. The vast majority of online news consumers, though, seem willing to browse for news from many sites, do not have a favorite online news source, and even if they do, are not willing to pay for that site’s content.

This is not to say that resistance might breakdown over time. . . .

All these findings speak to the natural disadvantage of news content: Most news is covered by more than one organization and people do not place enough value on the difference between the various reports. In other words, if a user had to pay for a New York Times article on Haiti, evidence suggests that he or she would just look for another source that could provide the basic information. The nuances of depth or breadth in the pay story may not be valued enough to induce payment over a free alternative.

Thus, if the news industry is going to make headway with pay-walls, they are going to have to break through what for now appears to be continuing reluctance, even among its most avid consumers.