Good Science, Bad Science... Who can you really trust?

I've read over a few posts lately regarding research papers and how to interpret the results of these studies and I thought I'd summarize everything here and include an awesome presentation by Tom Naughton on the subject (and maybe make you laugh a little too).





With so many different studies getting published by the media these days, each one competing for the biggest headline, it really is hard to believe anything health related. Picking out the applicable research from the junk is a tough job. I've completely blocked out commercials, they seem to be the worst and on top of that it's nearly impossible to track down the sources (if there are any). I cringe ever time an Activia commercial comes on... among others.

So what can we do about this? We can't stop the headlines or the commercials, but what we can do is ask three important questions:

1) Animal or Human Study?
2) Observational vs. Clinical Trial?
3) Who funded the research?

Animal or Human Study?
I think animal studies are useful in some cases, but you should always be careful when researchers make the leap from results of mice studies to studies on humans in their conclusions. Humans and mice may have a lot of genetic similarities, but there are also major differences and other factors that can greatly affect the results. Like Tom mentioned in his presentation, mice did not evolve to eat saturated fat in large quantities (or straight casein for that matter) and therefore trials where high saturated fat diets were given to mice are completely useless as an analogue to human saturated fat consumption (humans have been eating saturated fat for millions of years). Careful examination of the diet is key in these studies, especially since mice studies are notorious for feeding mixtures of fructose, casein and vitamin/mineral blends to rats and expecting healthy results... yeah right...

In human studies you need to make sure that the demographic studied was truly a good representation of the general population. Did they include all ages? All races an socioeconomic backgrounds? Is it really a random sampling? I realized that no sampling will be perfect, but these are just a few things to keep in mind.

Observational vs. Clinical Trial
It's as important to know what kind of trial you're reading as it is to know who the test subjects were. Observational studies (aka. epidemiological, cohort or prospective studies) are really only hypothesis, and therefore any conclusions made from them are hypothesis and nothing more. Most of the headline science today is observational. Thus, this only proves a correlation and not causation! Firemen are often found where there are fires, however, it would be incorrect to assume that firemen caused fires because of this... Firemen and fires are correlated, but as it turns out many times the actual cause is quite different.

The Clinical Trials are the best source of health and nutrition research, although these studies can still be effected by confounding factors (unknown drug interactions, resulting in changing multiple variables at once instead of one) and manipulation of the data (leaving out data that "doesn't fit").

Who Funded the Research?
This brings us to the last factor to consider, where the funding came from for the study... Is this a study showing that whole grains can lower your risk of heart disease and you check the funding and it was sponsored by General Mills (Cheerios), you might want to be a little more critical of the results than if it was done by an independent research company. Scientists are suppose to be honest and not skew/influence the results, but when your paycheck comes from the company that's looking for a certain result you tend to look for that result too...


So, the main point is to be critical of what you hear and read, look deeper than the flashy headline and then make up your own mind using common sense... Here's some further reading on the subject if you're interested, here and here.