Posts about measurement

The need for a measurement summit

Comscore and Federated Media (which sells some ads on this blog) have teamed up to try to improve measurement in the long tail of social and niche media online. And that’s good.

Except I argue that the panel means of measurement is doomed to miserable failure in the mass of niches. You cannot possible build a panel large and varied enough to get reliable measurement of the audience and traffic of millions — even thousands — of sites, especially when we get the means to tie together lots of those small sites into networks.

What I hope they do is honestly and harshly look at their stats from their panel versus the server stats of the sites — especially the smaller sites, not the much-easier-to-measure big boys like Digg and BoingBoing — and realize that the panel just doesn’t work.

What we need, I’ve long argued, is standard metrics reported from the sites’ servers or from snoopers on pages and verified by a service such as Comscore or Nielsen. Old methods will not work in this new world. The same goes for Nielsen, which is buying the rest of Netratings.

And whilel we’re at it, let’s figure out the new measurements that capture the unique value of this new medium: authority, speed, connectedness… The page view is dead.

I think it’s time for a measurement summit: Bring together the measurement companies, the advertisers and their agencies (buyers), the sites’ reps (sellers), the media sites, and technology companies and let’s hammer out some standards and methods for measurement. This will only work if we have open standards with analytics (like Comscore and Nielsen) building value atop that common data. Otherwise, we end up in a world that will continue to confuse and scare advertisers — and their money — away.

Guardian column: Death of the page view

My Guardian column this week is a sanded and polished version of recent posting and linking about the death of the page view and, with it, the mass. (Nonregistration version here.)

Just when we were getting used to it, the page view has been declared dead. There are many reasons for its passing, having to do with how web pages are now made and how web content is now distributed. But there is one seismic implication to this – in media, mass is over. Size doesn’t matter.

It was only about a decade ago that I sat on a dreadfully boring committee of the American Audit Bureau of Circulations debating how to define a page view (rather than a hit) as the elemental measurement of new media. This body, which blesses the circulation counts of print products, tried to replicate its world-view online, verifying the circulation – that is, the audience and traffic – of internet properties. But as it turned out, no advertiser or publisher wanted these audits. All marketers cared about was verifying whether they got what they paid for: views, people, clicks. You see, overall circulation mattered only when you and your ads were stuck in the same pages with many other advertisers and you all got the same audience, whether that audience gave a damn about you or not. But now, online, you could find better ways to reach just the people you wanted or who wanted you. Thus, travel advertisers needn’t care about the circulation of Guardian Unlimited, only about who saw their ads on travel pages.

In recent times, the situation has grown more complicated because, on the web, a page is no longer a page. Video can be served on a page, but it is measured in time, not space. Flash and Ajax technology can make any individual page many levels deep, allowing users to interact with content – navigating maps, ordering merchandise, viewing slideshows, chatting – without ever leaving the page. So the activities that once would have added up to a dozen page views will now count as only one. This is having a significant impact on businesses such as Yahoo, which are using these technologies to improve the user experience, reducing clicks in the hopes of increasing time on the site or satisfaction or loyalty. But this reduces page views and with that, bragging rights and, in some cases, revenue.

Now add to this the widgetisation of the web. Content may be displayed not only on your pages but also in widgets – boxes, gadgets and applications – that are embedded in pages elsewhere. This is how much of MySpace is built and how YouTube spreads video all over the internet. The audience becomes the distributor. How do you count that?

And consider Google AdSense modules that are spread all across the web, from NYTimes.com to my humble blog. Shouldn’t each of those be counted as Google page views since Google revenue is attached? Doesn’t that make Google look even more gigantic than it already is? What this really means is that in the new distributed media economy, owning a site doesn’t matter so much as enabling a network. This, in a nutshell, is why Yahoo, the centralised media property, is at a disadvantage versus Google, the distributed network.

Things get even crazier when you consider that if you make a good commercial, the public will distribute it for you on YouTube – advertising becomes content. Now that is really an upside-down world.

Finally, consider the impossibility of the old means of measurement. TV ratings were based on a sampling that determined the proportion of the audience watching, say, channel 2 or 4. But in this new niche world, no sample can possibly be large enough to measure millions of blogs or online TV shows.

But more fundamental than all this is, again, that size doesn’t matter anymore, not in media. Oh, yes, the movie with the biggest box office or the book with the biggest sales still makes the most money – for now at least. But in more and more of the media, mass measurements are obsolete because we are now fragmented into the mass of niches. And the truth is that we, the audience, never cared how many more people were watching what we watched. And advertisers don’t care so much what we’re watching so long as we’re watching them.

In a world of so many choices, the audience care about trust, taste, relevance, usefulness, not ratings. And advertisers care more about targeting, efficiency, engagement, branding and return on investment. These are better measurements than print circulation or broadcast ratings or online page views. And so now, publishers, advertisers and technologists must catch up and change their yardsticks for success yet again. It is time to measure quality over quantity.

Size doesn’t matter: The distributed media economy

No, size does not matter, not in media, not anymore.

I know that’s counterintuitive and counter everything we’ve assumed about mass media. But today what matters is reaching the right people by the right means. That has always been the case. Only now, thanks to connected, collaborative media, it’s finally possible.

I’ll pull together a lot of links around this topic below. But most of them are still trying to measure mass: the new pageview, the
new audience count, the new click. I say the change we’re facing is much bigger than just the obsolescence of the pageview, much more fundamental: Size doesn’t matter. Relevance, credibility, and attraction do.

Instead of measuring quantity, we have to measure quality. And only when we do that will the true value of these new media be unlocked for everyone.

Some of the discussion that is boiling up out there:

* The end of the pageview: Steve Rubel has been doing a good job hammering on the anti-meme that the pageview is over: “The page view does not offer a suitable way to measure the next generation of web sites. These sites will be built with Ajax, Flash and other interactive technologies that allow the user to conduct affairs all within a single web page – like Gmail or the Google Reader. This eliminates the need to click from one page to another.” See, for example, Yahoo grappling with the impact of the unpage. This is not entirely new; it was a problem I grappled with on refreshing chat pages a decade ago. But the phenomenon is growing in both an Ajaxed web and a Flashed video world: What’s a page now? What’s a view? What’s a viewer? A decade ago, I spent months on a tortuous committee of the Audit Bureau of Circulations answering just those questions. Now, it doesn’t matter, or at least, it matters less and less.

* Targeting and verification matter more than size: When I sat in those endless ABC committee meetings, our aim was to come up with the standards against which to audit the circulation or audience — old terms — of web sites. But that effort was eventually abandoned because advertisers didn’t care about verifying the size of a site; only publishers cared about those bragging rights and not enough to pay for auditing them. As it turned out, advertisers cared only about auditing their own flights of ads: ‘Did I get what I paid for (whether that was people or views or clicks or actions or demos or branding)?’ You see, circulation mattered only when you were stuck in the same pages as all the advertisers and you all got the same audience whether that audience gave a damn about you or not. But online, you could find ever-better ways to reach just the people you wanted or who wanted you. Travel advertisers didn’t need to care about the circulation of NewYorkTimes.com, only about who saw its ads in the travel section. Oh, yes, advertisers are still buying the old way, but that’s because it’s more convenient — albeit far less efficient — to buy us in bulk. But the mass is gone. Size doesn’t matter.

* The widgetization of the web: Niall Kennedy called it back in July: Pages are now made up of widgets that operate like multiple pages themselves. But this is about more than adding cool stuff to your site. See how MySpace is built with widgets from elsewhere and how Flickr is spread via widgets. This is a new means of distribution.

* The people are your distributors: One day this week, seven of the top 10 viral videos — determined by links and embeds, that is, by recommendations rather than just traffic — were performances by James Brown, following his death. (By the way, I didn’t see these videos on the most-viewed lists on YouTube; those are the old, mass lists people still look at but they’re pretty much meaningless). This tells me that the people will distribute your stuff if given a chance; they jumped onto word of Brown’s death and they served relevance. Note well that you don’t need everyone doing this; even as Wikipedia’s content is made by the fabled 1 percent of its users, so can the new networks of information be driven by 1 percent of their members.

* The distributed media economy is taking over: This last week, Google was said to surpass Yahoo as the second most visited site on the internet after Microsoft; this comes after MySpace surpassed Yahoo in pageviews. But the truth is that Google surpassed them all long ago, for Google is not a site; it is a platform. Every piece of Google — like the ads on this page — counts as another pageview, if you’re still counting them. This is why I keep saying that Yahoo is the last old-media company, relying on controlling content and marketing to attract an audience to see ads, but Google is the first distributed platform, no longer making the people come to it but going to the people wherever they are. And see James Brown: The people will take you to the people, if you’re good and if you’re lucky and if you let them.

Add to this the notion that advertising can be content (the viral ad) and that ad creative gets tangled up again with media and distribution (being on MySpace is itself the brand statement) and you continue to unravel all the old assumptions about the media economy; see Scott Karp arguing that the page view will be dethroned by innovation in advertising.

And mess things up even more when you start tearing apart the methodologies of measurement. Fred Wilson, on the board of Comscore, begins to address this. I argue that any paneled, sampled measurement scheme will simply not work in this new world. Full stop. The Nielsen method of putting together a sample of people who represented the rest of us cannot work in the mass of niches, for you cannot have a sample that is ever large enough to measure the tail. You can’t measure quantity.

So pageviews are obsolete already, thanks to Ajax and other unpage technologies and to the widgetization of content, funtionality, and branding: Again, what’s a ‘page’? Audience measurements are obsolete, at last, thanks to the fact that the former consumer is now also the creator and distributor: What’s an ‘audience’? Mass measurements are dead, thank God, because we are now joyfully fragmented into the mass of niches: Who’s a ‘user’?

The truth is that we, the former audience, have long paid only scant attention to the old, quantitative measurements: Box office numbers and Nielsen ratings were curiosities. We have always measured, instead, relevance, trust, usefulness, interest, attraction, action, value. Those are the measurements that matter, always have been, only now media must catch up to us. And when media and marketers do, they will give us greater value and get more in return.

This is about far more than the damned pageview.

See also:
* Scott Karp on the fallout for media and creative ad agencies.
* Michael Parekh.
* Evan Williams last summer on the dying pageview.

Viral Nielsens

I’ve been wanting to know what viral videos had infected the most conversations and was thinking about trying to do something that looked at links and insertions into blogs and such. Well, somebody just went and did it. Scott Button unveils the Viral Video Chart. What’s interesting about this is that it isn’t purely traffic. It is about conversation. They explain:

We scan several million blogs a day to see which online videos people are talking about the most. We count the number of times each video is linked to and the number of times each video is embedded. Every morning, after we’ve had a cup of coffee, we publish a list of the 25 videos that generated the most buzz over the previous day. We reckon this is a pretty good yardstick of what’s hot and what’s not. At the moment we only look for references to videos on the three most influential video sharing sites: YouTube, Google Video and MySpace. We tried looking for references to videos on some other sites for a while, but nothing ever made the top 25 so we stopped.

Today’s top video, Keith Olberman slapping Bush around:

Stand up and be counted…. and counted

The Audit Bureau of Circulations — the agency that verifies the copies sold by newspapers and magazines — just announced a new effort to combine print and online audience numbers.

But their logic is off. The Times says that for their first test case, Advertising Age, they “arrived at a figure by combining the publication’s print readership with a day’s worth of its Web site traffic.” The Times also reports that newspapers say this isn’t accurate because it doesn’t account for people who read both the paper and its online site. And I say that measuring only one day is ridiculous, for as many studies have found, very few people come back to online sites every day (and the truth is that they probably didn’t read papers every day, either; it’s just that they were delivered every day). If the goal is to measure total audience — total reach — then this will inevitably fall way short.

I toiled for way too long on the ABC committee that first tried to measure online (I suffered through endless meetings debating the question, ‘What’s a page view?’) As it turned out, advertisers didn’t really care about auditing the audience of online sites — and thus publishers didn’t want to pay for the service. Advertisers care only about auditing their own flights of ads; the wanted to verify that they got what they paid for. And that makes sense. For in an online site, you don’t really care how big the whole site it; you care about how many people your ads reach how many times.

The ABC and other old media players keep trying to use old math with new media. It won’t compute.