Should you trust news reports about studies?

by MooshiMooshi

You know how everyone complains that studies get reported in the media which then get contradicted a year or two later? Well, it turns out there are reasons for this.

Study: half of the studies you read about in the news are wrong

It turns out that reporters tend to report on initial studies, which are more likely to be contradicted in one or more ways later on. In the world of science, inital studies are just that: initial.

Besides the attention grabbing headline, this article has a good critique of the reasons why initial studies tend to be reported instead of the later metareviews which are more likely to be correct.

This is a real problem. People learn about science mainly through the media, and if it feels like everything reported turns out to be wrong, people start distrusting science. If reporters were more careful to publicize the later, more complete studies, people might develop more faith in science. I think reporters, too, should spend more time explaining the process of science to their readers, rather than just pushing out headlines and brief explanations of what may be very small and very tentative studies.

Good science and financial reporters are in terribly short supply, And given the fragile state of the field of journalism these days, I don’t see it improving. But these are two areas that impact everyone. People have to make decisions about both science and financial information all the time, including when they vote. How can we improve public understanding?

‘Opposition to Galileo was scientific, not just religious’

by WCE

This article detailing an alternative to Copernicus’ view that planets travel around the sun intrigued me, because correct hypotheses in science are usually the ones we learn and remember. It reminded me of the modern controversy over short term global cooling and warming trends and how to interpret the past 50 years of planetary temperature data, in light of limited historical data. Had you heard of Locher? Does he remind you of any other scientist? Does the controversy remind you of any other scientific controversy?

Opposition to Galileo was scientific, not just religious

String Theory

by Honolulu Mother

String theory has always had the problem of being essentially unfalsifiable. I’ve wondered myself if it’s just particle physics’s version of epicycles. Thus, I was very intrigued by the suggestion in this Atlantic article on string theory (I know, that reputed science journal, the Atlantic) that insofar as string theory produces testable hypotheses, they’re being borne out:

Using the physical intuition offered by strings, physicists produced a powerful formula for getting the answer to the embedded sphere question, and much more. “They got at these formulas using tools that mathematicians don’t allow,” Córdova said. Then, after string theorists found an answer, the mathematicians proved it on their own terms. “This is a kind of experiment,” he explained. “It’s an internal mathematical experiment.” Not only was the stringy solution not wrong, it led to Fields Medal-winning mathematics. “This keeps happening,” he said.

Do you have an opinion on string theory, or any other cutting edge field of science? Or failing that, do you support the level of public spending necessary to, say, prove the existence of the long-predicted Higgs-Boson particle?

She Blinded Me With S̶c̶i̶e̶n̶c̶e̶ Funding

by Rocky Mountain Stepmom

Here’s a discouraging article from the Chronicle of Higher Ed on the way funding agencies can manipulate science. From the article:

I am very concerned about the culture of academia in this country and the perverse incentives that are given to young faculty. The pressures to get funding are just extraordinary. We’re all on this hedonistic treadmill — pursuing funding, pursuing fame, pursuing h-index — and the idea of science as a public good is being lost.

I wouldn’t expect private funding sources to be any better than the government ones. Is there a solution?

The Water Next Time: Professor Who Helped Expose Crisis in Flint Says Public Science Is Broken

How Those Crazy Studies Make the News

Both Honolulu Mother and Rocky Mountain Stepmom sent in posts about this article.

I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.

by Honolulu Mother

We’ve all noticed how contradictory the conclusions from “scientific” studies in the news can be on many topics — what foods are good or bad, what type of exercise is effective or injurious, what parenting choices have good or bad effects. Sometimes this may be the result of a better understanding of a subject over time — surely the fact that eggs and butter, so reviled a generation ago, are now good for you again is an example of the arc of the moral universe bending toward justice. But most of the time, contradictions multiply and it’s never clear from the reporting what the studies were even based on.

Now a hero of our time has provided the explanation: it’s because the news media will publish any piece of crap study that sounds authoritative and has a headline-worthy conclusion, as outlined in the article linked above.

But notwithstanding the flaws in the study identified by its own author, I’m going to stick with his conclusion and make sure to get my chocolate every day. Because science!

by Rocky Mountain Stepmom

The article linked above is about the way science journalism works, or doesn’t work. One of its main points is that journalists are lazy. That resonated with me because of two work experiences I’ve had.

In 1983, I finished a Masters of Library and Information Studies degree at Berkeley. World’s easiest degree, but that’s not important right now. I had an internship at the KPIX news library. KPIX was and maybe still is the CBS affiliate in San Francisco. I got to see the local news produced every evening, and it was…startling. The reporters and producers just trolled popular magazines for stories they could regurgitate. I fetched Glamour magazine articles for the reporters to crib from. They stole from every conceivable source. It was disheartening.

My first job after library school was as an indexer for what was then Information Access Corp. (Remember Reader’s Guide to Periodical Literature? It was like that.) We indexed popular magazines, trade magazines, and five newspapers: New York Times, Chicago Tribune, LA Times, Christian Science Monitor, and the Wall Street Journal. We sat in front of our Apple 2e computer and read every single article in the papers and assigned index terms. It was a very peaceful way to make a living. But one thing I learned very quickly was that the newspapers all stole from each other regularly. The same article, with just a few changes, would appear in all the papers, and no, those articles weren’t from UPI or AP. They were by-lined by staff writers. There was clearly no independent verification going on. See article, crib article, print article. Again, very distressing to naive little me. The worst was the “end of year wrap-up” stuff. You might as well just burn all the papers from about Dec. 10 to Jan 5, because unless Manhattan gets nuked, there will be NO actual news whatsoever.

Totebaggers, do you trust the news? What’s your preferred source? Are you as skeptical as you would like to be? Or do you tend to believe stuff just because it’s in print? And do you believe that chocolate can accelerate weight loss?