Posts about curation

Content farms v. curating farmers

Tweet: Content farms v curating farmers: Deeper insights in Demand Media’s model & finding opportunity in finding quality.

I spent an hour on the phone the other day with Steven Kydd, exec VP of Demand Studios, to understand their model—using algorithms to assign content creation based on search and advertising demand and to minimize cost and maximize revenue—because I wanted to learn a deeper layer of lessons than I think we’re hearing in the discussion of Demand’s allegedly evil genius.

The talk thus far misses their key insight and the opportunities they create. Much of what I see online is fear that Demand Media—with the slightly rechristened “Aol.” following fast behind—will cheapen content and flood the internet—that is, search results—with crap that’s just good enough to fool algorithms. Some also fear that while putting content creators to work they will put better content creators out of work: the dreaded deprofessionalization and deflation of media.

Michael Arrington marks the end of “hand-crafted content” (somewhere I hear Nick Carr and Andrew Keen cackling maniacally). And Read Write Web’s Richard MacManus worries that the web’s quality will suffer.

They may be right. But then again, the internet has always been filled with crap. So the challenge has always been how you find the cream. That’s where opportunities lie. That’s what Google saw. The new question is whether Google can keep ahead of the content farms and continually find new and better ways to find better stuff. I’ll bet on Google over crap-creators. But they better get cracking.

This is why, when I proposed an X prize to solve media’s key problems at Yale symposium, Clay Shirky responded with a call for work on what he called “algorithmic authority.” A few of my students’ proposals in my entrepreneurial journalism class tackled just this problem with discovering and prioritizing content for us: one using humans aided by algorithms, one using algorithms aided by humans; neither operated like a one-size-fits-all search engine (but then, soon, Google won’t either).

I think we may see search fall as the sole or even key means of discovery and filtering of quality content. I see three rings of discovery today: search (Google); algorithms (see: Google News, Daylife); and humans (see: Twitter). Note again that Bit.ly alone causes as many clicks a month—one billion—as Google News. Human power rises again. That’s what Fred Wilson says today when he argues that social beats search, because “it’s a lot harder to spam yourself into a social graph.” As search becomes more personal and no longer universal, SEO as a dark art and as the fertilizer for content farms will diminish and the social graph — our own circles of authority — will become more important in search as well. So I have faith that there are solutions to stem any rising tide of crap.

This is how I put it in a tweet: “algorithm-aided human writing will meet human-aided algorithmic curation; quality will rise.”

In all of this, I caution us not to miss Demand’s key insight: that the public should assign the creators, including journalists. The public often knows what it wants to know. I learned this lesson when I consulted at About.com and saw how they monitor search queries to see where there are questions for which the don’t have answers. When that happens, they go write answers; Demand automates the process. Makes sense.

This is not how we have operated in media: We decided which questions to answer because we asked them. What hubris! Today, I teach my students to find conversations on the internet and add journalism to them in the form of answers, corrections, reporting, explanations. In 2007, my students in a seminar at Burda in Munich and in my class at CUNY asked why the public doesn’t assign us and my entrepreneurial students in two classes have worked on that problem. Jay Rosen just started playing with this notion at ExplainThis.org, creating a platform for the public to ask reporters to report their questions. Demand and About are doing the same thing, only through search queries. Jeff Sonderman compiles some more examples. Where appropriate, reversing the assignment pipe is a good idea.

Demand is also creating a system they say will find the best writer for each assignment. We are free to disagree with their methods and results, but there’s insight here, too. Two students in my entrepreneurial journalism class won a grant to create a platform to do just this with local and hyperlocal news assignments (note that Kydd told me Demand isn’t touching news); I’ll report more on their project as it gets closer to launch. Can’t news organizations learn and steal some lessons from Demand? What if you wanted to create a content asset — say, a complete travel guide — and you opened up the process and its discreet tasks to a marketplace of paid contributors, enabling you to do larger projects at lower cost than before?

I always tell my students: Wherever you see a problem, look for the opportunity. That’s Arrington’s point: The next generation of content creation is here; deal with it. If you don’t like what Demand et al are doing, see the opportunity in it to surface quality content and to create competitive quality stars whose creations rise not just through algorithmic search cynicism but through human recommendation. Dig to the next layer.

* * *

I got lots of details from Kydd about the Demand method. In their view, they have combined content-creation and social-media platforms to enable content creators with “spare cycles”—his nerdy words—to earn money.

Kydd says 11 community members contribute to each article by fulfilling the discreet functions Demand identified: writing, copy-editing, copy-chiefing, reviewing titles, managing topic pages, checking facts. That is done by freelancers. The staff directs, edits, curates, and manages them. The algorithm makes all this more effective as it tracks content and ad demand and writes headlines for pieces it says will get traffic and earn money. Editors are 1.5 times more effective in creating assignments that will generate traffic, Kydd said, but the algorithm is 4.9 times better than creators.

Kydd said Demand pays from $0 (with revenue sharing) to $100 per piece; it averages at $20. Copy editors make $2.50-$3 per piece, which works out to $15-20 an hour. He said these people like to wake up and know there’s work they can do—there are 100k assignments waiting for takers right now—while they wait for old, human editors to respond to pitches. He said they also like being paid twice a week. Kydd said Demand employed 4,500 creators (text and video) and 400 copy editors in the last 30 days.

What amazed me most is that Demand uses its method not only for service content but for jokes at Cracked.com. Could an algorithm and social network replace Jay Leno? Easy.

: LATER: See also Doc Searls on junk food and chefs.

: Paul Marcum tweeted today: “Prediction: increasing clutter from algo content farms + mobile app convenience will have even @jeffjarvis paying for news by 2011.” I responded seeing the irony here: that value will come from aggregation and curation of quality content. But imagine then if the aggregators become more valuable than the creators and start charging; the creators (i.e., Murdoch) will go batshit. I’ve argued that in the link economy, there are two creations of value around content: from those who make the content and from those who bring together the public around it. Where is there greater value? We’ll see….

: LATER STILL: See Upendra Shardanand (founder of Daylife, where I’m a partner) on the need for new tools to create new handcrafted content. Problem is, he says, we’re using old text tools. See my related posts on storytelling and post-page media.

The new news

Josh Young writes a fascinating and nicely written essay about the shape of news and competition around it in the Google (read: internet) age, but I think it badly needs a clear lede summarizing his point to prove his point.

So I’ll summarize: He’s saying that Google is causing news to be reshaped so it can be found, now that it has been unbundled from the products we used to have no choice but to buy: our newspapers. He says that news is an “experience good” we can’t really know until we taste it. He says we need a new experience of news and it ain’t Google. I will argue, though, that this very post, the one you are reading now, is the antidote to what he sees, for I experienced his essay and I recommend it to you without Google while also giving you the search-engine-and-browsing-friendly summary – a reason to read – that we now expect before investing in content online. And there’s my point.

Young argues that Google causes the structure of news to change. We agree about the change but we disagree about the cause. Even I don’t think that the all-powerful Oz Google is behind everything that happens on and because of the internet.

In fact, news is one of the areas where Google has little influence – despite the wails and whining of newspaper people – for Google is bad at current content, at the live web, at news. It needs content to ferment with links and clicks and context before it can figure out what it is. There’s no time for that in news. And GoogleNews itself isn’t an answer for it only makes the vastness of news vaster. We don’t find the latest news through search. We find it through recommendations, still, either from editors (going to a packaged site, a [cough] newspaper.com) or from peers (in blogs and RSS for years now and more lately in Twitter; this is why Twitter matters and why Google recognizes that it complementary).

So I’ll argue that we already have the beginnings of the news experience Young wants. Through this quote (which comes at the end of Young’s essay but would have been better as his lede, I think… I often find that to be the case when I write a post), please replace the word “search” with news, for “search” has become synonymous with Google and that’s not what we’re talking about. Young writes:

“We need a What we need is a search [news] experience that let’s us discover the news in ways that fit why we actually care about it. We need a search [news] experience built around concretely identifiable sources and writers. We need a search [news] experience built around our friends and, lest we dwell too snugly in our own comfort zones, other expert readers we trust…. We need a search [news] experience built around beats and topics that are concrete—not hierarchical, but miscellaneous and semantically well defined. We need a search [news] experience built around dates, events, and locations. We need a search [news] experience that’s multi-faceted and persistent. Ultimately, we need a powerful, flexible search [news] experience that merges [automation] and human judgment—that is sensitive to the very particular and personal reasons we care about news in the first place.

I think we’re seeing the beginning of what Young wants in blogs, Twitter, aggregation, better automated targeting, geotagging, and the move to human curation and I hope we’ll see people build other pieces of it in the ecosystems of news that will replace the papers that die (or don’t). I’m working with folks who are trying to build that now – with beats and organization and social recommendation – associated with the New Business Models for News Project. It’s just starting to come together, I think, and Young will be glad to know it’s not from Google; Google’s only a part.

Something like that, Young and I agree, will be the structure of the experience of finding – searching, broadly defined – and using and spreading news. As I said, we also agree that the structure of news will also change – but not just because of Google.

I argue in this post and in slides 6-11 here that the basic building block of news will no longer be the article – a creation and necessity of the means of production of newspapers – but instead the topic or the flow with many elements: process (think: blog), updates (feed), snapshot of current knowledge (wiki), perspective (comments, links), curation (links), and narration (the article still has its place). Yes, it is SEO-friendly. And, yes, Marissa Mayer gave a similar vision to John Kerry’s Senate hearings – of a “living story” that is updated at a permalink – but that doesn’t mean she decreed it. The greater functionality of the internet is shifting news to this structure because it is also link-friendly, blog-friendly, Twitter-friendly, feed-friendly, conversation-friendly, distribution-friendly….

If we invented news today - and we are – this is how it will look, not because Google replaces paper as the medium but because we are not limited to either.

(By the way, I’m probably wrong about Young’s lede. Even without it, because his essay was so deftly written, I read through to the end and took the trouble of reacting to it and recommending it to you here. I’ll also confess that I found it through Google search but only because Young kindly linked to me and mentioned my book. So the link was human, conversational, contextual, targeted, everything Young wants. Google just helped.)

: LATER: Another neat essay today, this one by Kim Pearson, on bringing computational thinking to journalism. I think it stretches the point just a bit (I don’t see how slideshows are particularly compuational) but the larger point is intriguing.

Death of the curator. Long live the curator.

For a long time now, I’ve been pushing hard the idea of journalist-as-curator. It appears that curators are looking at journalists and worrying about their loss of control, as evidenced by this post about the death of the curator, inspired by journalists – the Guardian – and curators – the Saatchi Gallery – enabling the great unwashed to help curate a show:

Museum curators and print journalists have a lot in common, in that it is their skills that turn an amount of information into something worth giving a damn about. There are plenty of other places to find out about the DefCon of journalism, especially the ever increasing problem of how to get paid. At this current moment in time, the museum curator is “safe”, got nothing to worry about. “It will all blow over”. The fact that there is a gallery in London who are going to offer thousands of people, many without art history degrees, the ability to choose what goes on the wall. The first step new media did to try to kill old media was to make the skills unimportant under the banner of “democratising”. “Everybody can get involved!” also means “It doesn’t matter what you know!”. Suddenly, your art history or archaeology degree isn’t looking so important, your museum post-grad may not be enough and your years of experience don’t mean much in the world of facemuseumtube when your job can be done by a thousand unpaid contributors. Curators may be safe now, but they would do well to look over their shoulders to their destitute journalist buddies.

Every priesthood, it seems, is having a fit over loss of its centralized control: How dare people pick what they like without history degrees or share what they know without journalism degrees! The nerve!

Except the irony in this comparison is that journalists need to learn better curatorial skills. Yes, in a sense, they’ve always curated information, collecting it, selecting it, giving it context in their stories. But now they have to do that across a much vaster universe: the internet. I hear all the time about the supposed problem of too much information online. Wherever you see a problem, I advise, seek the opportunity in it. There is a need to curate the best of that information (and even the people who gather it). We have many automated means to aggregate news (including Daylife, where I’m a partner). Curation is a step above that, human selection. It’s a way to add value.

I think that curators have things to teach journalists and that’s why I’m planning a symposium on curation at CUNY, bringing together museum curators, event curators, possibly even sommeliers to share their views of the value they add to collections of things, people, information – or wine. Note that one of the suggestions I make in What Would Google Do? is to capture the data of dining room – which wines went well with which dishes, according to diners – to crowdsource the job of the sommelier. Yes, every priesthood is vulnerable to the crowd.

[via Das Kulturemanagement blog]

Planning an event on curating news

Mindy McAdams does a wonderful job extending the definition and discussion of curation in the journalistic sense. It’s a word I’ve used a lot lately because I think it well describes the key role for journalists in a world of links and networks, selecting and organizing the best reports and best reporters. Mindy breaks out these roles: 1. Selection of the best representatives. 2. Culling. 3. Provide context. 4. Arrangement of individual objects. 5. Organization of the whole. 6. Expertise. 7. Updating.

I’m planning to hold an event at CUNY on curation and journalism (no date set; just planning). I will have a museum curator there and someone who curates events — any other ideas — with an editor and a link bloggers to compare worldviews and help illuminate this function in journalism.

I think it’s important now to bring in people from other disciplines to listen to their worldview of news and journalism and how they would go about it (and in many cases are going about it). At the New Business Models for News Summit, Tom Evslin made a point of saying he’s not a journalist and then did a wonderful job presenting network economics in a way that opened many eyes. At Davos last January, I ran a session with newspaper editors and technology CEOs (John Chambers, Reid Hoffman, Joe Schoendorf) who slapped the Eeyoreing editors out of their funk and made them see the opportunities they have.

Yesterday, I had coffee with Jay Rosen talking about his exciting (if competitive) Studio 20 program at NYU. As he gave his vision of how his students and their partners would work together on a big project, I said I thought it sounded a lot like agile development in technology and it occurred to me that such a developer could advise the project.

We’ve had too few new perspectives in journalism over the years because we thought our method was set. But today, as journalism changes or tries to, those new perspectives would be invaluable. So what would you add to the list here — curators, technologists, technology executives, agile developers…. Who else would have a valuable perspective on how the functions of news? Teachers, now that we have to be more generous sharing knowledge? Artists on creativity? Meeting facilitators on bringing out a group’s ideas? Hostage negotiators in negotiating? Researchers in navigating the value of peer review? Restaurateurs in gauging taste? Hoteliers in making strangers comfortable? Retailers in creating navigation? Cops in handling trolls? Who else?

They all know news. They were all what we used to call consumers. Some of them even write today. Journalism is in their hands. So I think it would do journalists a world of good to hear how they view news.