Bias is not a number and measurement is not the cure

Following up on NY Times public editor’s claim that he could measure bias in the paper — and found none — now Chicago Tribune ombudsman Tim McNulty measures his paper and finds little.

I asked Tribune researchers to catalog all front-page headlines, lead paragraphs and photos with each of the three candidates’ names and images over the past 12 months.

Here are the raw numbers: Obama was cited first in 93 front-page stories in the last year, Clinton in 80 stories and McCain in 39 stories.

So, how much does that tell us? Just a little. It certainly suggests there was more interest in Obama as the campaign heated up. Most stories in the last year focused on daily campaign coverage or reports on larger issues that included the other candidates.

Here’s Jay Rosen on the fallacy of measuring newsroom bias, critiquing Hoyt’s defense of the Times:

It is rare that a single article advances American press think. In fact, it is rare for American press think to advance at all, which is one of the reasons our press is so vexed these days. Take Clark Hoyt’s latest effort as New York Times public editor. It goes like this:

Many readers have complained to me that the Times is not “shooting down the middle” in its coverage of the 2008 campaign. But I’ve been monitoring and grading the coverage myself, and I have a surprise for some of you. “The Times has not been systematically biased in its news coverage, even if it has occasionally given ammunition to those who claim otherwise.”

Ta-da… An unbiased press! Now I do not doubt his word. Clark wouldn’t cook the books. But this is a conversation that’s savagely stuck, gamed not to go anywhere– for all sides. Professional journalists do not improve the situation when they double down on their neutrality and present objectivity as a truth claim about their own work. It is this kind of claim that compels people to furnish–furiously–more chapter and verse in the very bad and very long book of media bias. Which then causes Hoyt to speak lines like, “Bias is a tricky thing to measure, because we all bring our biases to the task.”

The only exit from this system is for people in the press to start recognizing: there is a politics to what they do. They have to get that part right. They have to be more transparent about it.

  • Highgamma

    It is extraordinarily hard to get the “right test” for bias and John Lott, for all of his controversial ways, has come closest to measuring newspaper bias. He looked at the coverage of economic events by newspapers and found that the headlines associated with these events tend be be covered more negatively when a Republican is president than when a Democrat is president. You can argue about sample size or the way he measures negativity of a headline; however, he provides one of the first objective methodologies that I have seen for measuring media bias.

    The key feature is that the measurement of “negativity” of the headlines is done SEPARATELY from determining who was president. That is, the criteria used to rate headlines was done by people who did not know if the headline was during a Republican presidency or a Democrat presidency. His results are statistically significant and his method appears scientifically valid. It’s worth a read.

    http://ww2.lafayette.edu/~econ_bus/nerds_papers/2008/lott_mediabias_1_24_08.pdf

  • chico haas

    As accurate as a health effects study by Philip Morris.

  • http://www.thefutureofnews.com Steve Boriss

    The best way I have found to measure bias is to study the premise of the article, not its content. Who are the good guys and who are the bad guys (or who has been placed on the defensive)? Good institutions, bad institutions? Good policies, bad policies? When you do that, the fact that mainstream news has a center-left bias should be obvious. And, usually it is obvious by reading the headline — you don’t even have to read the first word of the article. (Steve Boriss, The Future of News)

  • http://robertdfeinman.com/society robertdfeinman

    There are several types of “bias” in the Times. There is the bias of some columnists to favor one of the candidates, but, I believe the Times’ policy prohibits outright support. So you see someone like Krugman who favors Clinton being bashed by the Obama supporters for lack of fairness.

    More serious is the trend, which is becoming more evident, that the Times business viewpoint has been taken over by libertarians. The evidence keeps mounting. I’m not talking about the usual Main Street Republicanism that was it’s hallmark for generations, but the support for ideas which have no standing in the scholarly world (especially outside the US).

    The evidence includes two libertarian columnists from the same liberatarian economics department (George Mason) – Cowan and Tabbarok. I don’t see this amount of space being given to contributors from more highly respected universities.

    Then there are the prominent places given to Brooks, Friedman and Kristol. They aren’t just neo-cons, they are libertarian inflected neo-cons. Then there is the space given to the libertarian clown Ben Stein – who doesn’t even have any credentials (scientific or economic), but feels free to make a film debunking Darwinism. Does the Times not have any criteria for the credibility of the people it gives space to?

    Finally there is the subtle shift in the way “news” and analysis articles are written. Here’s an example from today.

    Today’s paper has a front page article:
    “Parties Differ on Whom Economic Aid Should Help”

    There is only one “economist” cited in the article, and he’s from Cato Institute:

    ‘ “Democrats are more likely to propose protecting individuals, and Republicans are more likely to propose protecting markets,” said William A. Niskanen, chairman of the Cato Institute, a libertarian research group in Washington that champions smaller government.’

    The meaningless phrase “protecting markets” is discussed further as if it meant something other than protecting the wealthy. There are no “markets”; there are people who participate in markets, and there are firms who participate as proxies for them.

    Why would any (non-biased) columnist even think of soliciting a quote from somebody at Cato?

    Liberal economists keep puzzling as to why the press goes with the misleading stories over the projections for Social Security and Medicare. If the “press” is seen as the libertarian press, everything becomes clear.

    It’s not only the lack of objectivity that the press failed at over Iraq and the “war on terror”, but there is a similar ideological bias when it comes to economics and related social policies.

    Papers have always been biased, that was why many were established, so their publishers could promote their viewpoints (Hearst, McCormick, etc.), but they made their position clear.

    Over the past 50 years the majors have taken on a theme of impartiality which hasn’t been reflected in their actions. The myth that the viewpoint is restricted to the editorial page is another reason they are distrusted.

  • Highgamma

    I agree that you can “see” bias in an article. (My personal favorite was a gun control article that quoted the NRA, which everyone knows is pro-gun, and also quoted a scholar, whom most people whom would not know was pro-gun control. The scholar was not identified as pro-gun control and the NRA was not identified as pro-gun. We heard from both sides but most people would discount the NRA person as biased while the academic would be considered unbiased. By the way, I think gun control is a good idea, but I know bias when I see it.)

    However, how do we “measure” bias. While John Lott might not be the best source, his methodology is straight-forward to replicate. Economic news should just be economic news, but if the headlines are biased to be negative during Republican administrations and positive during Democratic ones, someone else should be able to verify that.

  • http://tomgrey.motime.com Tom Grey

    Why not look at NYT front pages for the last year, and note all the stories about scandals or negative news on elected people.
    Then note whether, and where, the party identification is.

    The facts will be in this form: X% of one party is identified in the first paragraph, but only Y% of the other party is identified in the first paragrah.

    If X-Y > (10%)*X, I claim bias. (Let’s define how much bias is bias!)

    I’m not sure if, under this definition, the NYT is biased — but Don Surber keeps talking about “Name that Party”. I’m pretty sure “Democrat” is often missing from stories about Spitzer, for instance.

  • http://www.shootingbynumbers.com peter

    The Nazi party consistently gets bad press,

    Philanthropists on the other hand normally get good press.

    - a long way to go before we can claim a bias-free media, but we are getting there!

  • http://www.pressthink.org Jay Rosen

    Never was there a more doomed subject than this.

  • Dick Eagleson

    ideas which have no standing in the scholarly world (especially outside the US).

    Given that much of the “scholarly world” suffers from the apparently incurable mental dysfunction that is left-wing politics, as does much of the world “outside the U.S.” – which I take to be your euphemism for the decaying societies of Western Europe – it is unsurprising that they join you in finding “libertarian” ideas so discomfiting. In less than a century Europe has gone from owning the world and inventing much it to dicing with arriviste tribal barbarians for control of their own national territories. The only thing of consequence Europe exports these days are subsidized aircraft, bad philosophy and – oh, yes – Europeans.

    There are no “markets”

    Utter twaddle. Yeah, there are markets. They’re everywhere. The word is not reserved exclusively to describe that neo-classical pile of stone in which stocks are traded in New York. And if you think the latter is just for “the wealthy,” we sure have a lot of “wealthy” then as over 50% of American adults have equity holdings of some kind. If you want to stalk imaginary animals often referred to in political discourse but not findable in nature, try “social justice,” “the common good,” and all the other unicorns of leftist cant.

    Still, it’s interesting that at least a part of the reason for the coming collapse of the moth-eaten newspaper business is that at least a few fools out there find that it is not left-wing biased enough.

  • http://www.pressthink.org Jay Rosen

    We’ll let the Newspaper Association of American, the publisher’s trade group, know that they should put “leftist cant, especially in media blog comment threads…” on their list of factors causing the demise of their industry. I am sure they will be interested to hear of it.