Doesn’t add up

Doesn’t add up

: Reporters and editors need more training in how to handle numbers.

Dan Okrent has a good column on the topic today. He says it is a refreshingly equal-opportunity sin; readers from left and right complain about numbers.

On my ride back from Boston yesterday, Joe Trippi took me through the numbers for the last election in a way I hadn’t heard before. It was said that the youth didn’t come out — that was the accepted wisdom starting on election night because of a flawed interpretation of the numbers and, unfortunately, it sticks. I would give you Joe’s analysis but I didn’t take notes and don’t want to get it wrong. The truth is, he said, that younger and older voters came out while voting by those in the middle ages declined. That misinterpretation spread by media will affect political strategy and as a result public policy. It’s not a small mistake.

A few weeks ago, I had the pleasure of chatting with Steve Shepard, the editor-in-chief of Business Week and soon to be the head of CUNY’s new journalism school, and he said — understandably, considring the magazine he edits — that reporters are notoriously bad at numbers and need education in stats.

Whether on a paper or via weblogs, it would also be great to have people who are trained in stats available to fix or at least question the flawed analyses that turn into accepted wisdom.

  • http://myblahg.blogspot.com Robert McClelland

    It was said that the youth didn’t come out — that was the accepted wisdom starting on election night
    No, Jeff. That was only the accepted wisdom on the right. The rest of us have known since a day or two after the election that the youth did in fact turn out.

  • Rob Schneider

    Req’d reading for journalists should be “Dammed Lies and Statistics”, by Joel Best.

  • Rob Read

    Or the DailyAblution blog where the Grauniads dodgy stats and stories get eviscerated daily!

  • http://sisypheanmusings.blogspot.com/ Sisyphus

    Jeff,
    A while back, Andrew Cline and Jay Manifold started the 411 blog, which I thought was a great and powerful idea. Like many great and powerful ideas, it’s been backburnered for more mundane endeavors – like earning a paycheck.
    Anyway, maybe some attention/support from you?
    Here’s an example of what you’re looking for from 411: 519.5 Statistical Mathematics

  • paladin

    Hee!Hee! I think GWB gets it too. Why do you think he’s bringing social security reform to the front now? Because “the youth” are skeptical that there will be anything left for them when they retire. This is important to them. But predictably, what do the dim Dems do? Hollar that there is no problem with SS, that GWB is “creating” a crises, that the right is trying to “dismantle” the New Deal (and we all know the 1930′s are “just like” the 21st Century), and if anything, we need to INCREASE benefits. It’s just like Wiley Coyote and the Roadrunner. Beep!Beep!

  • http://homepages.nyu.edu/~mtz206/ Michael Zimmer

    Whether on a paper or via weblogs, it would also be great to have people who are trained in stats available to fix or at least question the flawed analyses that turn into accepted wisdom.
    Such as with your post here.

  • Observer

    Is this Jeff “Blogging doesn’t require a business model” Jarvis who’s decrying the sad state of business? Or is it Jeff “Anything that has to do with the Old Media” (which only pays my salary)” Jarvis talking?

  • thibaud

    The examples of NYT journalists’ innumeracy are beyond shocking; they’re a disgrace. How can anyone with a college degree not know the difference between real and nominal prices?
    Okrent shows us why MSM journalism is such a joke. Not only are journalists not professionals, they’re not even minimally numerate. Good riddance to these incompetents.

  • http://bigdirigible.rubberdinosaurs.com big dirigible

    Yes indeed, reporters in general are baffled by numbers. However, so are most readers.
    There are plenty of educated Americans who know more than most people could possibly want to know about statistics, and they’ve known that the polls are bogus since they started to appear in the news decades ago. Now in the blog era, some of these mathematical types are online and anyone with the stamina can read up on the tedious details. Early efforts of mine are at
    http://bigdirigible.rubberdinosaurs.com/archive1.htm#040907
    and
    http://bigdirigible.rubberdinosaurs.com/archive1.htm#040928c

  • http://www.havecoffeewillwrite.com Jeff Hess

    Back in the Dark Ages of my undergraduate years, one of the reasons that students at Ohio University chose Journalism was that it was the ONLY degree that did not require any math. That got fixed in 1984 (fittingly) when the freshman class of ’85 was informed that it would have to take Stats I and Stats II to graduate. I’ve always wondered if the predicted drop in class size occurred.

  • http://shinobi42.blogspot.com Shinobi

    When I was in the process of getting my Statistics degree we would regularly take articles from the NY Times and openly mock them during class.

  • http://www.lexicillin.com/blog.htm Bill Koslosky

    Jeff:
    Most major newspaper writers usually miss the distinction between causality and correlation when they report on published medical studies. The recent report showing a *correlation* between the moderate use of alcohol and decrease in decline of cognitive skills among women, was reported by the WSJ thusly:
    “But a new study, published in today’s New England Journal of Medicine, found that consuming a small amount of alcohol–a drink per day or less–can help prevent cognitive decline in women as they age.”
    The accompanying editorial in the same issue of the NEJM warned that this study doesn’t prove that the alcohol is causing this effect, but seems to be linked. To prove this, you need to do a random, double-blinded study of women, where they don’t know if they’re drinking an alcoholic drink or not, and then follow them over the years, checking their scores on tests of cognition. This is not likely to be done any time soon.
    We blog also:
    http://www.lexicillin.com/2005/01/causality-vs-correlation-medical.htm
    BK

  • Angus Jung

    “The rest of us have known since a day or two after the election that the youth did in fact turn out.”
    …for Bush. Whoops!

  • mamapajamas

    To me, the worst misuse of stats is the infamous opinion poll. To this day, the US is mocked worldwide because “40% believe Saddam caused 9/11.” This is never challenged by anyone except in blogs like this.
    I recall that the NYT opinion poll that carried the question along the lines of “Do you believe that Saddam had anything to do with 9/11″ used the results as proof that the “right” is stupid. However, the question asked if Saddam had anything to do with 9/11, and I believe that, yes, he did have SOMETHING to do with it via his terrorist connections. I do not believe that he was sitting at the planning table with OBL, or even that he had any foreknowledge of the plan, but I believe that Salman Pak was where the terrorists who did the hijacking trained.
    The point is that the question was extremely vague, asking if Saddam had “anything” to do with 9/11. And, yes… most of the right DOES think that.
    It’s opinion, and not an outrageously off-the-wall opinion, either. Too bad the NYT didn’t look at it that way. They ran with the “look how stupid the right is!” angle, and that was picked up world-wide.

  • J. Peden

    Journalists seem self-selected in a way which results in a virtual illiteracy in math, science, and logic rampantly infecting what they produce. Whatever the cause really is, this condition is probably not remediable within individuals having the defect.

  • http://www.glcq.com paul_lukasiak

    However, the question asked if Saddam had anything to do with 9/11, and I believe that, yes, he did have SOMETHING to do with it via his terrorist connections. I do not believe that he was sitting at the planning table with OBL, or even that he had any foreknowledge of the plan, but I believe that Salman Pak was where the terrorists who did the hijacking trained.
    Another example of “left wing” media bias. It wasn’t the “failure of the youth vote to turn out” or “the values vote” that put Bush in the White House. It was the stupid vote—but the “left wing” media was so afraid of the “eastern elitist” label they were afraid to point out that Bush voters were overwhelmingly ignorant of the facts on the issues that concerned them most.

  • http://www.blogads.com henry

    Journalists are HORRIBLE when it comes to numbers. Most are liberal arts grads who failed a couple of math courses and climbed to safety aboard the life-raft of journalism. When editing a business newspaper in the 90s, I’d always ask aspiring reporters — “btw, what’s 20% of 500?” Half of them couldn’t do it and, worse, were insulted anyone dared ask.

  • http://www.kstengel.us Karl Stengel

    A while back, I put together a site on the misuse of statistics. It’s supposed to be directed at high school graduates, or college students, with no special math training. I cover
    Correlation does not prove causation;
    When determining the effect of one variable, you have to keep all other relevant variables the same, or be able to account for their effect;
    A dose-response relationship is a minimum requirement for proving a cause-and-effect relationship;
    A study must show substantial differences for an effect to be real;
    When in doubt, do a longitudinal study;
    Ensure the effects you see are substantially different from chance.
    Beware the base rate fallacy.
    The site is at:
    http://home.earthlink.net/~kstengel226/sci_tech/stat/
    - Karl

  • Ric Locke

    Feh. “Journalists” are not merely innumerate and ignorant of statistics; they are uttelry ignorant of science, technology, and reality in general. The difference between a “reporter” and a “journalist” is that a reporter may sometimes regret this or feel that it might impact his work; a journalist is aggressively proud of it — peons take care of that stuff. Hewers of wood and drawers of water are well below the Status they assume themselves to occupy, and actually knowing anything about what such pariahs do is suffering intolerable contamination.
    I once posted on Jay Rosen’s site that I had been involved in a number of items that made the “news,” and in every single case what appeared in print or on screen bore so little resemblance to anything that actually occurred as to give rise to the question of what reality the “journalists” inhabit. He and his sycophants sneered at me. Effum. It’s true anyway.
    The proper method for handling “journalists” is to say as little as possible, lie when convenient, and use a few jawcracking words. What will actually appear will be made up out of leftoid prejudice and invincible ignorance, keyed off of whatever terms you might have used that caught their imaginations, i.e. anything that can be made into a “conflict;” why spoil it with facts, which they won’t allow and can’t comprehend anyway?
    This is why both Left and Right complain that “the media” are prejudiced against them. They’re both right — and they’re both wrong. In reality, our visitors from Planet Journo are absolutely convinced that the only thing people want to hear or see is car wrecks and their equivalent, so if there isn’t one they’ll manufacture it, and if there is one that’s the only aspect of any importance. Of course, they’re so ignorant that they entirely miss what the controversy is all about, being able only to connect with a strictly limited number of concepts, so their account of arguments completely ignores the subject and concentrates on the personalities.
    And they wonder why they’re losing influence.
    Regards,
    Ric Locke