Understanding the results of human health studies: a primer

At least once every week I open the newspaper to see a headline trumpeting the results of the latest human health studies. Studies linking egg yolks, red meat and alcohol (to name but a few), with increased mortality and morbidity.

results of human health studies

results of human health studies

results of human health studies

The headlines are purposefully provocative (“Egg yolks almost as unhealthy as cigarettes“), the articles replete with scientific and statistical terms that mean nothing to the reader and very rarely is there sufficient information about the original study to allow proper interpretation of the results.

Frequently, this week’s study contradicts the results of previous studies on the same topic, making it difficult for the educated lay person to know which study is correct and hence, whether they should reduce their consumption of eggs, eliminate red meat from their diet completely, drink more, drink less or not at all.

Long before I became a fitness writer, I studied the behaviour, ecology and evolution of fish. Data analysis and experimental design were things I excelled at. Much of what I learned is directly applicable to understanding the results of human health studies. Shall I share?

Rather than risk the ire of readers by using their favourite food as an example, let’s investigate the relationship between snorkleberries and snorkleberry-beri (a highly contagious disease that renders the fingers and toes a fluorescent pink, leading to social stigmatization and shunning).

When you read the results of human health studies you need to ask four questions:

1. Is the study observation or experimental?

Observational (or population) studies are those conducted on a large group (or population) of individuals. Data is often obtained via questionnaire. For example, individuals might answer questions about how frequently they eat snorkleberries , how often they exercise, whether there’s a history of heart disease in their family and whether they smoke. Measurements such as weight, height, body fat and blood snorkleberry levels might be taken as well.

Multivariate analysis of the data is conducted to identify relationships between variables; for example, individuals who ate more snorkleberries were more likely to suffer from snorkleberry-beri. Observational studies can identify correlations, but cannot conclude that consequence B is a direct result of action A.

Correlation does not equal causation. Why? It just might be that individuals who eat more snorkleberries also consume large amounts of enkelbird eggs.

Experimental studies are exactly what they sound like. Experiments.

Subjects were (or should have been) randomly assigned to test and control groups, assigned a particular treatment (for example, eat 100 snorkleberries per week, eat 50 snorkleberries per week, eat 0 snorkleberries per week) and monitored for adherence to the protocol (this is a huge challenge with human subjects :) ).

Data are analyzed by either comparing average responses between groups or, in the case of an experiment in which each subject experiences each treatment (again, in random order), the average of the difference between each subject’s response to the various treatments. Confusing, ‘eh?

Experimental studies can demonstrate a causal relationship between action A and consequence B, given appropriate experimental controls and data analysis. In this case, snorkleberries eaters suffered more bouts of snorkleberry-beri than those who abstained.

2. Is the sample size adequate?

In general, the more participants in the study, the better. Most data sets contain a lot of noise (variation between individuals). Biologically significant effects may not reveal themselves to the researcher if too few individuals are included in the study. Observational studies tend to have much larger sample sizes than experimental studies, in part, because it’s a lot easier to get people to fill out questionnaires than it is to get them to adhere to strict research protocols. But also, because of the ethics involved in conducting controlled health studies on humans. (Who would voluntarily sign up to eat 100 snorkleberries each week if they knew there was a possibility that they’d contract snorkleberry-beri?).

3. Is the effect revealed by the study biologically meaningful?

While having a large sample size is generally thought to be beneficial, due to the magic of statistics, it also makes very small effects more easily detectable. Just because a statistically significant effect is found does not mean that the effect itself is biologically important. A 300% increase in the risk of contracting snorkleberry-beri is not biologically significant if the initial risk itself was only 1 in one million.

4. Was the study free of bias?

While most scientific researchers aim to eliminate all forms of bias from their studies, human subjects are notorious for failing to follow experimental protocols to the letter (“I didn’t like the taste of the snorkleberries so I substituted blueberries instead”). When questionnaires are administered, we often (unconsciously) under-report activities that we perceive as negative and even slant our answers in the direction that we think the researcher wants to hear.

And of course, a study’s source of funding should always be considered when deciding how much value to give to its results. Naturally, the Snorkleberry Grower’s Association is happy to fund (and acknowledge funding) of studies indicating no relationship between snorkleberry consumption and snorkleberry-beri.

If you’ve made it to the end of the post, congratulations! You’ve just passed Stats 101. Enjoy some snorkleberry juice, on me; I promise, there’s no evidence that it causes snorkleberry-beri!

Are you frequently alarmed by the results of human health studies?

Do you greet their conclusions with skepticism or believe what’s written and immediately change your behaviour?

 

 

Comments

  1. Well put and interesting throughout! I made it to the end without anything causing snorkleberry-beri. But are you sure there is no correlation? Hang onto this post as one of your classics and most helpful for sure!
    KymberlyFunFit recently posted..Challenge Yourself to a Healthier YouMy Profile

    • Thanks Kymberly! Glad you didn’t catch snorkleberry-beri! (That was the hubs’ idea; he was thinking it sounded Seussian…)

  2. Being a former data analyst and math nerd, I loved this post. Great information and I hope it gets lots of views. Thanks, Tamara.
    Elle recently posted..Wordless WednesdayMy Profile

    • Math nerds unite! I have always loved numbers and statistics was one of my favourite courses in university. Hoping that I made a rather complex subject transparent enough to be useful…

  3. This was such a great post! Josh’s masters minor is in statistics so he’s always telling me to be more critical when reading health studies. It’s crazy how things can be distorted!
    Erin @ Girl Gone Veggie recently posted..Buying My Wedding RingMy Profile

  4. Now I’m hungry for some snorkleberries, but that might just be a confounding variable, Confound It!
    AlexandraFunFit recently posted..Challenge Yourself to a Healthier YouMy Profile

  5. Snorkelberries!!!!! I think I might have had one of those & that is my prob! ;-) Great post Tamara – AS ALWAYS!!! We really have to read & research just like those labels on food & the verbiage on the package – what is behind all that!
    Jody – Fit at 54 recently posted..Breast Cancer Awareness; PB Recall ExpandedMy Profile

  6. I don’t believe everything I read at first glance or ‘listen’. I want to be informed and look further into what is published or ‘advertised’, before letting it impact me or my behaviour towards something :)

    Great info! Thanks!

    p.s. I don’t think I’ve ever had me snorkleberries! lol
    Kierston recently posted..WIAW: Fitness Competitor Friendly Potluck!My Profile

    • My hubby just made snorkleberries up! I didn’t want to vilify any particular food group, since the jury seems to be out on just about everything these days!
      Thanks for checking in!

  7. i work in the news business and we hear all of the crazy studies the other day one came out that wearing a helmet when riding a bike can help you not hurt your head when you fall… really? there are some crazy ones out there and they all contradict another one!
    Danielle @ itsaharleyyylife recently posted..Fall PickingMy Profile

    • Sounds like that authors of that study forgot to wear their helmets while biking! Somebody took one to the head…

  8. Im such a skeptic.
    I consume all written things with an eye to WAIT WHATWHAT?
    MIzMiz recently posted..THE MIZFIT WORKOUT SKIRTS ARE HERE!!My Profile

  9. Great post. I’ve been a skeptic every since the hubby took a stats class and came home to inform me that you could use statistical data to prove that increased drowning deaths were caused by an increased consumption of ice cream across the population.

    Plus, if I changed my behavior with every study what would I eat? :)

    • Thanks Rhonda! Stats are so easily misused. That’s for sure. And really, what food isn’t bad for you if you eat too much of it…

  10. Ooh, great topic, Tamara! I couldn’t believe the latest one I saw about the egg yolk – the title of the study said “Egg Yolks as Dangerous as Smoking Cigarettes!” and then the study basically had NOTHING to back that statement up. Ugh.
    Paige @ your trainer Paige recently posted..Fave New Running Shoes, Newton’s Law, & Dartfish {Thoughts on That Thursday}My Profile

    • Thanks Paige! That headline drove me to write this post! I worry about all the people who don’t have the background to really read and interpret newspaper versions of scientific studies.

Trackbacks

  1. [...] if you want to really rock out at sorting through information overload flotsam and jetsam, read Understanding Studies from our friend and fellow FitFluential Ambassador, Tamara [...]

Speak Your Mind

*

CommentLuv badge