Should you trust news reports about studies?

by MooshiMooshi

You know how everyone complains that studies get reported in the media which then get contradicted a year or two later? Well, it turns out there are reasons for this.

Study: half of the studies you read about in the news are wrong

It turns out that reporters tend to report on initial studies, which are more likely to be contradicted in one or more ways later on. In the world of science, inital studies are just that: initial.

Besides the attention grabbing headline, this article has a good critique of the reasons why initial studies tend to be reported instead of the later metareviews which are more likely to be correct.

This is a real problem. People learn about science mainly through the media, and if it feels like everything reported turns out to be wrong, people start distrusting science. If reporters were more careful to publicize the later, more complete studies, people might develop more faith in science. I think reporters, too, should spend more time explaining the process of science to their readers, rather than just pushing out headlines and brief explanations of what may be very small and very tentative studies.

Good science and financial reporters are in terribly short supply, And given the fragile state of the field of journalism these days, I don’t see it improving. But these are two areas that impact everyone. People have to make decisions about both science and financial information all the time, including when they vote. How can we improve public understanding?

Advertisements

How Those Crazy Studies Make the News

Both Honolulu Mother and Rocky Mountain Stepmom sent in posts about this article.

I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.


by Honolulu Mother

We’ve all noticed how contradictory the conclusions from “scientific” studies in the news can be on many topics — what foods are good or bad, what type of exercise is effective or injurious, what parenting choices have good or bad effects. Sometimes this may be the result of a better understanding of a subject over time — surely the fact that eggs and butter, so reviled a generation ago, are now good for you again is an example of the arc of the moral universe bending toward justice. But most of the time, contradictions multiply and it’s never clear from the reporting what the studies were even based on.

Now a hero of our time has provided the explanation: it’s because the news media will publish any piece of crap study that sounds authoritative and has a headline-worthy conclusion, as outlined in the article linked above.

But notwithstanding the flaws in the study identified by its own author, I’m going to stick with his conclusion and make sure to get my chocolate every day. Because science!


by Rocky Mountain Stepmom

The article linked above is about the way science journalism works, or doesn’t work. One of its main points is that journalists are lazy. That resonated with me because of two work experiences I’ve had.

In 1983, I finished a Masters of Library and Information Studies degree at Berkeley. World’s easiest degree, but that’s not important right now. I had an internship at the KPIX news library. KPIX was and maybe still is the CBS affiliate in San Francisco. I got to see the local news produced every evening, and it was…startling. The reporters and producers just trolled popular magazines for stories they could regurgitate. I fetched Glamour magazine articles for the reporters to crib from. They stole from every conceivable source. It was disheartening.

My first job after library school was as an indexer for what was then Information Access Corp. (Remember Reader’s Guide to Periodical Literature? It was like that.) We indexed popular magazines, trade magazines, and five newspapers: New York Times, Chicago Tribune, LA Times, Christian Science Monitor, and the Wall Street Journal. We sat in front of our Apple 2e computer and read every single article in the papers and assigned index terms. It was a very peaceful way to make a living. But one thing I learned very quickly was that the newspapers all stole from each other regularly. The same article, with just a few changes, would appear in all the papers, and no, those articles weren’t from UPI or AP. They were by-lined by staff writers. There was clearly no independent verification going on. See article, crib article, print article. Again, very distressing to naive little me. The worst was the “end of year wrap-up” stuff. You might as well just burn all the papers from about Dec. 10 to Jan 5, because unless Manhattan gets nuked, there will be NO actual news whatsoever.

Totebaggers, do you trust the news? What’s your preferred source? Are you as skeptical as you would like to be? Or do you tend to believe stuff just because it’s in print? And do you believe that chocolate can accelerate weight loss?