A basic of science

I often find myself reminded of a post I made on just the third day in the life of FTSOS. It was about a media report on a recent study that said a certain pesticide found in anti-bacterial soaps may actually contribute to the development of antibiotic resistance by bacteria. It was a fine study, but it was far from conclusive. (The news article wasn’t so cautious in its assertions.) Perhaps it would be best if people only used regular old soap, what with that not really qualifying as a real sacrifice, but as for the science, I was far from ready to say that that pesticide was a contributor to antibiotic resistance among bacteria in any significant way in the given environment.

And the reason is quite simple: science does not rely upon individual studies. Of course, we may be able to point back to the results from one lab or one group of researchers as published in a single study as the linchpin that opened up a whole new branch of study. But that doesn’t mean we believe that paper as being conclusive on its own. It only works when we have a body of evidence. In most cases that means a number of studies looking at the same or a similar problem and coming to the same or very similar conclusions. For a single paper that proves itself a linchpin, that means we need a number of other studies which use its findings as their basis. For instance, green fluorescent protein, or GFP, was shown to work as a marker of gene expression in a pretty definitive study. It has about a bajillion (rough estimate) other studies on it, but no one needed to reproduce the study which won one research team the Nobel Prize in chemistry. But people did use that study as a basis for about a gagillion (rough estimate again) studies. If the original study was wrong or faked or otherwise limited, we would be well aware of that by now because of all those subsequent studies. That is one way to compose a body of evidence.

To put this another way, take the studies on intercessory prayer and its efficacy. We have some that show positive results. Look, God is here to help! But then we have others that show negative results. Oh, no! God must be angry! And then we have a whole bunch which shows a null result. Uh…God must be indifferent. So how do we interpret these results?

Remember, we need to be looking at the evidence as a body. As one of those intolerant, bigoted, hate-filled evilutionist atheists, I would find it humorous if prayer gave negative health results. But I don’t get to have that laugh. Instead, I have to conclude that prayer has no detectable effect on health. None of the studies are conclusive; they suffer from bias, or are statistically insignificant in either direction, or just show a blatant null result. The most likely conclusion is that prayer does nothing. No study has convinced me otherwise, and most of the studies have shown prayer to be inconsequential to the well being of people anyway.

What I hope this post enables readers to do is recognize a fundamental aspect of how science works so that next time they see a study which concludes a link between this or that, they know what to think. That doesn’t mean it is okay to just dismiss a non-bias confirming study (i.e., a study that doesn’t give a result one likes). It just means that it is always necessary to look at the entire body of evidence before drawing a conclusion.

2 Responses

  1. It just means that it is always necessary to look at the entire body of evidence before drawing a conclusion.

    It is difficult for most people to do that. We all have our biases, some ideological, some from laziness, others from ignorance or cultural brainwashing, etc.

  2. I’m so bloody glad there is someone out there who is doing this… ‘defending’ science and explaining some of the basics of it, in order to better explain how it works.

    I was having an argument with a mate of mine just last night about how ”natural medicine” and religion are not ”alternatives” to science but rather just offshoots from our collective human past that we should be able to shake off by now, to the point where they no longer hold any sway. But, alas, they do, and very strongly at that.

    Anyway, just wanted to say AWESOME WORK on this website (particularly enjoyed the one of naturopaths are dangerous quacks’).

    I’m always astounded at how often science comes under attack, from everyday people, the commercial media, and organised relgion. It’s just so damn tiresome. Science is the only way to seriously view and understand (and appreciate!) the world. Why oh why can’t people be thankful of science and what is has provided so far, and spend their time learning about the AMAZING stuff that is happening in the world of science rather than attacking it and coming up with hotch-potch ”alternatives”?

    Rant over. Sorry.

    PS I climbed Kili in July 2010! Nice pic.

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: