Posts about search

Google’s synchronicity

December 18, 2009 by Jeff Jarvis
, , , , , , ,

On the latest This Week in Google, we talked about many of Google’s product announcements and enhancements and though none on its own was earthshattering, as we added them up, I started to see synchronicity approaching — all the moreso last night when TechCrunch reported that Google’s negotiating to buy Yelp.

I see a strategy emerging that has Google profoundly improve search by better anticipating our intent and then moving past search to build hegemony in local and mobile (which will come to mean the same thing).

Add up Google’s recent moves in local/mobile:

* Yelp would bring Google a scalable platform to get information and reviews about every local business using community. Yelp enhances Google’s place pages. Place pages enhance Google Maps. Google Maps are our pathway to local information on what we still mobile phones but will soon see as our constant connectivity devices.

* Google distributed 190,000 QR codes for local businesses to paste on their front windows. Take a picture of it and Google will give you information about the place (see: above). Businesses have another reason to advertise on and be found through Google and its business center.

* TechCrunch also speculates that we could use these QR codes to check in to Foursquare, Gowalla, et al. Local is social.

* Google Goggles goes the next step and lets you take a picture of a place — or object (or soon, person) — and use that as a search request to get local information — or leave it.

Thus Google becomes a doorway to the annotated world. Everyplace has information swirling around it; Google organizes it and motivates and enables us to create more information for it to organize (more on this idea of the annotated world in another post).

* Google’s reported phone is said to have a “weirdly large camera.” If that camera becomes a key to visual search, that makes sense, eh? That also gives us a better way to take more geo-tagged photos, which better annotates the world and gives Google more information to serve back to us.

* Google is trying to get better at recognizing speech to prepare for a voice-controlled (read: mobile) web world. That, say Chris Anderson and Tim O’Reilly, is why they give away GOOG411 for free: to learn our voices. And now note that Google is asking people to donate their voicemails to Google’s effort to improve its own transcription. Search will become visual and aural (read: mobile).

* Google Earth is coming to the cockpit of the new Audi, giving drivers rich geographic data about where they are and where they’re going.

* GoogleMaps on Android will now tell you what’s nearby.

* Let’s not forget that Google will make money on local — Eric Schmidt said on CNBC a year and a half ago that Google will eventually make more on mobile than the web (which, to me, doesn’t mean phones; it means our constantly on connection devices). This is why Google bought mobile ad leader AdMob for $750 million.

That’s mobile. Now look at some of its search enhancements to better intuit intent:

* Go to the Google home page. Start typing “Weather in Lon” and stop there. Google will not only suggest that you want weather in London, it will give you the forecast for London right there in the search box. You didn’t even finish typing in what you wanted to ask and Google gave you the answer without you even having to click and go to a site.

Google search

Google’s holy grail, they’ve long said, is to anticipate your intent. That explains, I think, some of Google’s other moves.

* Google DNS is supposed to speed up the web for you (speed is a big Google cause these days) but it also gives Google an invaluable source of data about web usage: who goes where when and before and after what sites looking for what. Now, your ISP knows that. But with DNS, Google could know that. It makes Google smarter about the web and its content as a whole, certainly, and so long as it is careful about privacy, it can enable Google to target to us better.

I see a day when search (like news) is no longer one size fits all. Search will be customized, personalized, and targeted to us and our contexts: who we are and where and when we are asking for something. This, I think, could mean the slow death of the dark art of SEO.

* How will Google get us to use its DNS? Well, I’ll bet it will be the default in computers equipped with Google Chrome OS. And I wouldn’t be surprised of the Google Chrome browser can provide some of this data to Google.

* Google launches social search. This creates more context and gives Google another clue to intent.

Now add back in all the mobile developments above. This gives Google more context to anticipate our intent.

But that’s not all. I’ve said for sometime that Google is behind in battles for the live and social web and was going to say here that it was bypassing those strategies to concentrate on mobile/local. But as I wrote the post, I saw more threads in both live and social.

* Google added Twitter to its search results. That’s pretty much a BFD. But it shows they’re trying to grapple with the live web. And that’s why there are never-ending rumors about Google buying Twitter.

* Wave is an important shift in the metaphor for content creation, making it collaborative (read: social) and live. Google added social tools to Google Docs. It make Docs a path to publishing (and being found via search) on the web. Creation itself is a social act once it enables us to connect.

* Add in the social bits above: Yelp is a community tool; QR codes and visual search will let us talk about places and things and find each other and meet; Foursquare and Gowalla make local social and Google could help them.

Last night, after the Yelp report, I tweeted this: “Yelp + GoogleMaps + StreetView + PlacePages + GOOG411 + Google Goggles + Android + AdSense = Google synchronicity”. Om Malik piped in: “@jeffjarvis I love your unrelenting belief in google. I think u need to start look at world in a non-search context.” But then I said – and others agreed: “I also think Google is starting to look at the world in a non-search context (i.e., local, live, mobile)”.

I believe that’s what we’re seeing here: the start of Google’s view of itself after search. Not that search will go away but it will become less important in the shifting mix of out rings of discovery. And if search is going to stay preeminent, it had better update itself profoundly.

: See also Gina Trapani’s excellent roundup of Google’s amazing 2009 developments.

: LATER: Kara Swisher says Google is also eying real-estate search Trulia.

Content farms v. curating farmers

December 14, 2009 by Jeff Jarvis
, , , , , , ,

Tweet: Content farms v curating farmers: Deeper insights in Demand Media’s model & finding opportunity in finding quality.

I spent an hour on the phone the other day with Steven Kydd, exec VP of Demand Studios, to understand their model—using algorithms to assign content creation based on search and advertising demand and to minimize cost and maximize revenue—because I wanted to learn a deeper layer of lessons than I think we’re hearing in the discussion of Demand’s allegedly evil genius.

The talk thus far misses their key insight and the opportunities they create. Much of what I see online is fear that Demand Media—with the slightly rechristened “Aol.” following fast behind—will cheapen content and flood the internet—that is, search results—with crap that’s just good enough to fool algorithms. Some also fear that while putting content creators to work they will put better content creators out of work: the dreaded deprofessionalization and deflation of media.

Michael Arrington marks the end of “hand-crafted content” (somewhere I hear Nick Carr and Andrew Keen cackling maniacally). And Read Write Web’s Richard MacManus worries that the web’s quality will suffer.

They may be right. But then again, the internet has always been filled with crap. So the challenge has always been how you find the cream. That’s where opportunities lie. That’s what Google saw. The new question is whether Google can keep ahead of the content farms and continually find new and better ways to find better stuff. I’ll bet on Google over crap-creators. But they better get cracking.

This is why, when I proposed an X prize to solve media’s key problems at Yale symposium, Clay Shirky responded with a call for work on what he called “algorithmic authority.” A few of my students’ proposals in my entrepreneurial journalism class tackled just this problem with discovering and prioritizing content for us: one using humans aided by algorithms, one using algorithms aided by humans; neither operated like a one-size-fits-all search engine (but then, soon, Google won’t either).

I think we may see search fall as the sole or even key means of discovery and filtering of quality content. I see three rings of discovery today: search (Google); algorithms (see: Google News, Daylife); and humans (see: Twitter). Note again that Bit.ly alone causes as many clicks a month—one billion—as Google News. Human power rises again. That’s what Fred Wilson says today when he argues that social beats search, because “it’s a lot harder to spam yourself into a social graph.” As search becomes more personal and no longer universal, SEO as a dark art and as the fertilizer for content farms will diminish and the social graph — our own circles of authority — will become more important in search as well. So I have faith that there are solutions to stem any rising tide of crap.

This is how I put it in a tweet: “algorithm-aided human writing will meet human-aided algorithmic curation; quality will rise.”

In all of this, I caution us not to miss Demand’s key insight: that the public should assign the creators, including journalists. The public often knows what it wants to know. I learned this lesson when I consulted at About.com and saw how they monitor search queries to see where there are questions for which the don’t have answers. When that happens, they go write answers; Demand automates the process. Makes sense.

This is not how we have operated in media: We decided which questions to answer because we asked them. What hubris! Today, I teach my students to find conversations on the internet and add journalism to them in the form of answers, corrections, reporting, explanations. In 2007, my students in a seminar at Burda in Munich and in my class at CUNY asked why the public doesn’t assign us and my entrepreneurial students in two classes have worked on that problem. Jay Rosen just started playing with this notion at ExplainThis.org, creating a platform for the public to ask reporters to report their questions. Demand and About are doing the same thing, only through search queries. Jeff Sonderman compiles some more examples. Where appropriate, reversing the assignment pipe is a good idea.

Demand is also creating a system they say will find the best writer for each assignment. We are free to disagree with their methods and results, but there’s insight here, too. Two students in my entrepreneurial journalism class won a grant to create a platform to do just this with local and hyperlocal news assignments (note that Kydd told me Demand isn’t touching news); I’ll report more on their project as it gets closer to launch. Can’t news organizations learn and steal some lessons from Demand? What if you wanted to create a content asset — say, a complete travel guide — and you opened up the process and its discreet tasks to a marketplace of paid contributors, enabling you to do larger projects at lower cost than before?

I always tell my students: Wherever you see a problem, look for the opportunity. That’s Arrington’s point: The next generation of content creation is here; deal with it. If you don’t like what Demand et al are doing, see the opportunity in it to surface quality content and to create competitive quality stars whose creations rise not just through algorithmic search cynicism but through human recommendation. Dig to the next layer.

* * *

I got lots of details from Kydd about the Demand method. In their view, they have combined content-creation and social-media platforms to enable content creators with “spare cycles”—his nerdy words—to earn money.

Kydd says 11 community members contribute to each article by fulfilling the discreet functions Demand identified: writing, copy-editing, copy-chiefing, reviewing titles, managing topic pages, checking facts. That is done by freelancers. The staff directs, edits, curates, and manages them. The algorithm makes all this more effective as it tracks content and ad demand and writes headlines for pieces it says will get traffic and earn money. Editors are 1.5 times more effective in creating assignments that will generate traffic, Kydd said, but the algorithm is 4.9 times better than creators.

Kydd said Demand pays from $0 (with revenue sharing) to $100 per piece; it averages at $20. Copy editors make $2.50-$3 per piece, which works out to $15-20 an hour. He said these people like to wake up and know there’s work they can do—there are 100k assignments waiting for takers right now—while they wait for old, human editors to respond to pitches. He said they also like being paid twice a week. Kydd said Demand employed 4,500 creators (text and video) and 400 copy editors in the last 30 days.

What amazed me most is that Demand uses its method not only for service content but for jokes at Cracked.com. Could an algorithm and social network replace Jay Leno? Easy.

: LATER: See also Doc Searls on junk food and chefs.

: Paul Marcum tweeted today: “Prediction: increasing clutter from algo content farms + mobile app convenience will have even @jeffjarvis paying for news by 2011.” I responded seeing the irony here: that value will come from aggregation and curation of quality content. But imagine then if the aggregators become more valuable than the creators and start charging; the creators (i.e., Murdoch) will go batshit. I’ve argued that in the link economy, there are two creations of value around content: from those who make the content and from those who bring together the public around it. Where is there greater value? We’ll see….

: LATER STILL: See Upendra Shardanand (founder of Daylife, where I’m a partner) on the need for new tools to create new handcrafted content. Problem is, he says, we’re using old text tools. See my related posts on storytelling and post-page media.

Microsoft’s Sneakerphone

May 22, 2008 by Jeff Jarvis
, , ,

Microsoft’s effort to bribe/reward/cajole ecommerce search business away from Google with customer rebates is the product of dubious business economics. It’s a trap: a customer acquisition cost that becomes a habit hard to break. It’s just like premiums given by magazines to get you to subscribe. When I was at Time Inc. in the ’80s, Sports Illustrated had a big hit on its hands — or so they thought — with the Sneakerphone, free with your subscription. Time and other of the company’s magazines followed with clocks and other geegaws. In the end, though, they found that people weren’t subscribing to the magazines; they wanted the Sneakerphone and when it came time to renew, because they already had what they wanted, they canceled — and renewals are where magazines begin to see a return from their marketing to acquire subscribers. Advertisers eventually realized that they weren’t talking to readers; the magazines were the premium. The Sneakerphone turned out to be a very expensive problem. It took painful effort for Time Inc. to ween itself and its subscribers from the expectation of freebies with subscriptions.

Microsoft’s fees are a marketing cost, pure and simple. The company could pay to advertise it search or it can pay consumers to search. I’m glad to see money going into the pockets of consumers — the internet dividend strikes again. But I doubt that these economics are sustainable; this is just an effort to poke Google in the kidneys and I doubt that the giant will even notice. This is akin to Mark Cuban’s sillyass idea to pay/bribe/reward/cajole advertisers into leaving Google.

Michael Arrington has a well-done analysis of the Microsoft gambit. He concludes that it could increase Microsoft’s share of valuable commerce search:

A year ago Microsoft basically did a trial run of Live Search CashBack with Live Search Club, which lured searchers to Microsoft with offered of prizes to users for using Live Search. Microsoft went from 10.3% to 13.2% market share in a month, a nearly 30% rise. Live Search CashBack, which gives a much more straightforward payout to users, should see significantly better results.

But earlier in his post, I think he defeated that argument when he said, quite rightly:

This is a winner-take-most market: Having 9% of search doesn’t mean Microsoft has 9% of search marketing dollars. Far from it – publishers go to Google to partner on ads, which means advertisers must go there to get inventory, and a very healthy auction system pushes up prices. So not only does Microsoft (and Yahoo, and everyone else) have much fewer queries than Google, they are also generating much less revenue per query as well.

Right. So Microsoft pays heavily to raise it share but still doesn’t get critical mass. Then let’s say that Cuban gets on the board of Yahoo and convinces them to follow his plan and they lower their profit margin by paying advertisers, forcing Microsoft to do likewise. And what will they be left with? A warehouse filled with sneakerphones.

Is there a way to defeat the Google beast at search? Not this way. How about the Mahalo or Wikio method? I’m not sure about them either. They all have to try to change what is already a well-ingrained consumer habit and a critical mass of advertiser participation. Does this mean that search is Google’s forever? Well, I still don’t see anything to topple them — certainly not Microsoft’s plan. It has been tried before. Remember IWon.com? I barely did.

The segregated web

April 10, 2008 by Jeff Jarvis
, ,

I’m not sure what I think about the Barry Diller/IAC announcement of a black-oriented search engine and content site, Rushmore Drive. I get the content part, of course. I also understand specialized search engines based on need or interest — jobs v. homes v. medicine, and so on. But isn’t there a danger in creating a search engine segregated along racial lines? Does it create more separation? Does it create a new sort of echo chamber? Does it limit the world reached by the search? I would never want to use a search engine aimed at middle-aged, suburban white guys like me; I want the world. And how do they know what is black-oriented content? It almost smacks of reverse red-lining, possibly pandering: How can you tell that a given article would be of more interest to an African-American than others and who’s to say that all African-Americans would look at it the same way? Perhaps I need to hear the problem stated clearly before I can judge this as a solution.

It has been said that tools like blogging and Facebook are disproportionately white; is that the problem? Is the solution, then, a search engine that gives them more traffic? Well, perhaps. But it’s a rather indirect one; there are other ways to encourage more creation and send them more audience.

Or is the problem search itself and an inability to find some content? I did a search on teen pregnancy because I recalled a recent survey saying that the incidence is much higher in the African-American community, and on Rushmore Drive, the fourth and fifth results are about just that. Not so on Google, though adding “African-American” to the search query comes up with very good results.

I would regret seeing the open prairie of the web marred with fences. On the internet, nobody knows you’re a dog or a cat, black or white, man or woman, young or old, unless you chose to say so and unless it’s relevant. I would have hoped that we could use this vast openness to break down some of these separations, not build them back up.

DLD: Wales v. Calacanis

January 21, 2008 by Jeff Jarvis
, , , ,

Jason Calacanis makes the case for human v. algorithmic (SEOed or spammed) search and then shows off his apple pie and his new social network. Jimmy Wales says what we need is more transparency and openness in search. “This is something that has benefited us across the internet,” he says. David Kirkpatrick, the moderator, says that given Google’s incredible growth, people are satisfied with search. Calacanis mentions a study that says 60 percent are dissatisfied (anybody have that citation?). Jimmy says that the leading search engines — Google, Yahoo, Ask — return essentially equivalent-quality results. So he argues that they compete on brand.

Marissa Meyer of Google is sitting in the front row right in front of them. I’m betting she’s not scared. She’s the fourth member of the panel just by her presence.

Jimmy says that Wikia will have not only the functions Jason’s Mahalo has but also user-submitted algorithms. This goes to yesterday’s theme of software being content — and open software requires open algorithms. Jason says he doesn’t think Jimmy will be unsuccessful trying to “jump in front of the Google train” in algorithms.

Jason says that he has a “pretty good suspicion” that Google is using people to clean up results. Marissa shakes her head no.

Jason says he has 60 fullitme people and 400 freelancers. “I believe in paying people for work,” he says; his zing on Jimmy. “I’m a writer by trade and I take offense when people try to devalue writing.” Do you think this way about Wikipedia, Kirkpatrick asks.

“There’s a very strange element to it, cult-like some people would say, that they want content to be free.” Jimmy responds: “Nobody works for free… What people do for free is have fun…. We don’t look at basketball games and people playing on the weekends and say these people are really suckers doing this for free.”

Jason has his teeth in the leg. He says it’s hypocritical that “everybody in the Web 2.0 industry should become millionaires and billionaires except for the writers.”

Jimmy: “To me this like the debate we got over a long time ago because it was so stupid that open-source software is communism.”

Yes. It’s about aligned interests.

Kirkpatrick pushes Jimmy and says he has a chicken-egg problem: Wikia is getting “terrible reviews” and he needs people to build the product but he needs the product to attract the people. Jimmy agrees about the chickena and the egg and says the bad reviews also hatched ore audience.

“If you’re going to come in the search space you need to invest at least $50 million to do this… This is not for the faint of heart. A major of search companies fail.”

Jimmy says it will take at least two years to get to industry-standard results.

Marissa from the front row says that it’s a mistake to return to the human-created directory. she agrees with them that there’s a false-dichotomy to look at this as all algorithmic or all human and, as Jimmy pointed, out, Pagerank is edited by humans — engineers, he says, but humans. She pushes back on Jason’s model of trust and says that may not be the right model; expertise matters.

“I agree with everything you said,” Jason says. “Can I go work for you?”

Esther Dyson says the problem with their model is the long tail. If she were starting a company she’d do somethign else. “Ok, I’m going to kill myself now,” Jason said. Punch line. He says that the fat tail will be human, the medium tale social, the long tail algorithmic. And he says that the advertising interest is in the fat tail.

Michael Arrington says he was very tough on Wikia when it started but has “promising aspects.” He returns to the playing basketball for fun question and says “that there are very few examples of for-profit companies getting people to do their work.” Not sure I buy that as a rule. He acknowledges Digg. I’m seeing networked journalism. I do believe in sharing revenue (a better model than staff work given the business realities today). Jimmy responds that “if we ask people to do work we will fail” but if he provides tools they want to use for their purposes they will succeed.

At the end, I ask about advertising and whether they will concentrate on the fat-tail brand, display ads and Google will have the scale to do automated advertising. Turns out that Jason will likely turn over all his inventory to Google to start and then, as with similar companies, will grow sold advertising. As the last word, Jimmy says, amazingly, that he hasn’t thought about advertising.