Making Sense of Contradictory Claims

If you’ve ever felt confused by contradictory nutrition advice, you’re not alone.

The recommendation to drink red wine with dinner has morphed into the proclamation that even an occasional drink elevates our risk for cancer, diabetes, liver disease and cognitive decline. A stern warning that soy causes breast cancer has been replaced by praise from the American Cancer Society of soy as a healthy low-fat, high-fiber alternative to meat.

Why Studies Contradict Each Other

Scientific experiments begin with a hypothesis. Next, researchers design a study to test the hypothesis. Often their methods rely on precise measurements. They employ strategies to account for variables that could skew the results. Finally, the data is analyzed to determine whether there is a statistically signifiant indication that that the hypothesis is true.

Data Collection Issues

The importance of precise measurements is one reason that nutrition is a tricky subject to study. Since few people are eager to volunteer to live in a laboratory, most studies either experiment on animals or rely on self-reported data, where participants are asked to remember what they ate over days, weeks, or even years.

This approach is notoriously unreliable, as memory can be faulty and dosage (amount consumed) varies from person to person. Do you remember what you ate last Thursday? How about the Thursday before?

To compensate for faulty memories, researchers will test their hypotheses on animals confined to a lab. While it is easier to control the diets of mice in a cage, the results may not necessarily apply to people.

A significant limitation of the initial soy study is that mice metabolize soy differently than humans. In addition, the mice were fed unusually high amounts. Even a tofu lover like me, doesn’t eat it every day.

Nonetheless, headlines like “Soy May Raise Breast Cancer Risk” spread quickly, creating a persistent but misleading narrative. Further research has shown that moderate soy consumption is not only safe but may even offer protective benefits against certain cancers and heart disease, directly contradicting the early alarmist findings.

Yet, even these studies are limited by variables such as the form of soy consumed—tofu, soy milk, edamame, tempeh or soy supplements—and where it was grown.

Infinite Variables

Consider for a moment—what would you measure if you wanted to test the health impacts of a particular food? Perhaps the gender, age, weight and health history of the participants?

What variables might you want to account for? What about their activity level or exposure to environmental toxins? Since stress can impact digestion, what might you want to know about their emotional health?

As we covered in the last blog post, insufficient sleep can interfere with the glymphatic system. If your study considers the neurological effects of the target food, the quality of subjects’ sleep might be relevant.

The more variables collected, the more difficult it may be to ensure that the test subjects are similar enough to draw meaningful conclusions from the data.

Furthermore, some variables are extremely difficult to factor in. Individual responses to foods vary widely, influenced by gut microbiomes, genetics, and even undiagnosed food allergies.

Beyond the Headlines

Making sense of contradictory claims requires decoding the data beyond the headlines. If possible, review the study cited (this is not always feasible as some research articles reside behind a pay wall). If the study is available, consider the following:

• Sample Size: How many people participated? For example “N=200” indicates 200 participants. A conclusion based on 24 people is less reliable than a study of 24,000 people.

  
  

• Duration: How long were the participants observed? Especially where diet is concerned, short-term effects may not reflect long-term outcomes as many serious health issues arise after decades of poor nutrition. There’s a difference between chowing down on french fries every day for a month and every day for a decade.

  
  

• Study Type: How was the study structured? A randomized controlled trial (RCT) is considered the gold standard in research. A RCT randomly assigns participants to different groups (such as an intervention and a control group) to isolate the effect of a particular variable, reducing bias.

  
  

• Blind and Double-Blind Studies: In a blind study, participants do not know whether they are receiving the treatment or a placebo, helping reduce psychological bias.

  
  

  In a double-blind study, neither the participants nor the researchers know who is receiving the treatment and who is in the control group, minimizing both participant and researcher biases.

  
  

• Peer Review: Was the study published in a reputable, peer-reviewed journal? Peer reviews ensure that researchers followed appropriate protocols and reached a reasonable conclusion based on the data collected.

  
  

  While publication is a vital marker of quality, it is not uncommon to encounter a study that was later withdrawn or substantially re-written after wider scrutiny.

  
  

• Meta-Analysis: In a meta-analysis, or review, researchers compare results across multiple similar studies to identify overall trends and reduce the impact of outliers. For example, subsequent studies analyzing the relationship between soy and cancer in humans concluded that moderate soy consumption may actually reduce breast cancer risk.

  
  

• Funding Sources: Who paid for the research? Financial interests can sometimes influence outcomes. Ask yourself who stands to benefit (or lose) if the public shuns (or embraces) a particular product.

  
  

• Special Interest Influence: Are there special interest groups involved? Just as tobacco manufacturers once suppressed research linking smoking to cancer and heart disease, some food industry groups have been known to highlight benefits while ignoring or downplaying potential harms.

  
  

  For example, on their website the Florida Citrus Growers cite a 4-week study of 24 men funded by the European Fruit Juice Association touting the immunity-boosting benefits of nutrients in orange juice. However, the website doesn’t mention that a whole orange delivers the same nutritional benefits plus fiber, which moderates blood sugar levels—a crucial concern for people with metabolic issues such as diabetes. Juice is a lucrative product for citrus producers since there is on average three oranges in a glass of OJ.

  
  

• The Pitch: Be especially wary of the sales pitch disguised as nutrition information. Often these posts begin by sharing ground-breaking revelations about a particular substance before pivoting to a specially formulated powder or pill.

  
  

  Dietary supplements are not regulated by the Food and Drug Administration (FDA). It is not uncommon for independent third-party testing to reveal that the contents differ from the ingredients listed on the label.

  
  

Digging into the research can be a bit like Alice’s tumble down the rabbit hole. Unless you are a nutrition wonk like me, it may not be an appealing use of your time and energy—and that’s ok. Instead you may want to season your consumption of sensational nutritional claims with a liberal sprinkling of skepticism.