The media publishes a flood of medical-related articles that capitalize on people's interest in health and related topics. With the deluge of information, it can be hard to figure out what is meaningful and what is not. Unfortunately, the media leads people astray by violating a basic principle of science: one study is not the basis for fact, theory or conclusions.
The basis of the scientific method that clinical studies use is to create logical experiments that follow a defined hypothesis. It also relies on a method that is repeatable so peers can verify the findings. Taking the results of one study to heart violates this principle.
Scientific Method
An experiment should follow the clearly defined steps of the scientific method. It begins with a hypothesis. Then, an experiment is developed to test the hypothesis. Ideally, it is random and controlled. Anything that can contaminate the results fouls the evidence.
Then, the experiment is carried out with unbiased observations. These are compiled and from which, a conclusion is drawn. No conclusion is just as important as a definitive yes or no. The goal is the test of the hypothesis, not the results.
Types of Studies
When studies are published, the first thing to consider is the type of study. A research paper may present an observational study based on the input of participants. It may be a review of past studies. It may also be a randomized, controlled study to eliminate extrinsic factors to discern the truth. Knowing the basis for a question is important to weight its value.
Factors of Good Data
The first criterion is that studies should be evaluated by peers in the field. An unbiased view can often illuminate problems or inconsistencies of a study. Fortunately, this is a criterion for most journals as part of the publication process.
Ideally, the study group should be large and diverse to find commonalities. A small sample group may not represent the population as a whole The important thing to remember is that scientific studies rely on statistical data for conclusions. The number in a study needs to be large enough to make sound conclusions.
The question then concerns observational studies versus controlled trials. An observational study often includes a large number of participants, making it statistically viable. However, it also relies on participant data. This may or may not be valid, depending upon the source. Other factors also weigh in, especially if participants are questioned about habits over the long term. They may simply forget details that are important to a particular study.
Randomized Controlled Studies
Randomized controlled studies can provide the best evidence because distracting factors are limited. Participants may eat the same foods, exercise the same amount or do the same things in order to get to the facts. Of course, this represents a burden to the participants, especially in long-term studies. However, it is the best way to eliminate other factors that can influence or bias the outcome.
Published Studies
The problem with studies published in the media is that a journalist will often report on one study and its conclusions. In science, conclusions are not drawn from the results of one study. Rather, it is the body of evidence that supports a theory.
Yet, the media reports it in such a way that one study is fact. Nothing could be further from the truth, especially if it deviates from other studies. The only way to validate divergent findings is to repeat the experiment and come out with similar results. The media does a gross disservice to the general public by giving the impression that one study is definitive. It confuses people and is misleading.
The best way to view studies published in the general media is with a bit of skepticism. One study does not foul the results of dozens of studies before. It merely opens up a new avenue of investigation. The one truth about science is that it is ever-evolving. Science changes as life changes. It is the only absolute truth.
Sources:
- Linus Pauling Institute, Are Multivitamins Useless?
- Science Buddies, Steps of the Scientific Method.
- Gerard Tortora and Bryan H. Derrickson. Principles of Anatomy and Physiology, Wiley, 2008.
