In this episode, we discuss:
- The problems with nutritional research
- Why correlation isn’t causation
- The Bradford Hill criteria
- The healthy-user bias
- Why diet quality matters
- The type of diet that can impact gut flora
- Why you should take nutritional studies with a grain of salt
- “Will a Low-Carb Diet Shorten Your Life?” by Chris Kresser
- Spurious Correlations, by Tyler Vigen
- “Red Meat and TMAO: Cause for Concern, or Another Red Herring?” by Chris Kresser
- “RHR: You Are What Your Bacteria Eat: The Importance of Feeding Your Microbiome—With Jeff Leach”
- The American Gut Project
- “RHR: Is a Disrupted Gut Microbiome at the Root of Modern Disease?—With Dr. Justin Sonnenburg”
- “The Challenge of Reforming Nutritional Epidemiologic Research,” by John Ioannidis
Hey, everybody, Chris Kresser here. For the past 50 years we’ve been told that eating meat, saturated fat, and cholesterol is unhealthy. Recently, a growing number of people are turning to a vegetarian or vegan diet with the goal of improving their health.
But is it really true that meat and fat are bad for us? And are vegetarian and vegan diets a good choice for optimizing health and extending lifespan? If not, what is the optimal human diet? Join me on the Joe Rogan Experience on Thursday, September 27th, as I debate these questions with vegan doctor Joel Kahn. You can tune in live at 1pm PT (***Updated time!***) at JoeRogan.live or, youcan catch the recording at podcasts.joerogan.net, on YouTube, or in iTunes or Stitcher.
If you’d like to receive updates about the debate, including links to the recording and new articles and information I’ve prepared on this topic, go to Kresser.co/Rogan. That’s Kresser.co/Rogan and put your email in the box.
Okay, now onto the show.
Hey, everybody, Chris Kresser here. Welcome to another episode of Revolution Health Radio. This week, we’re actually going to do a reader-submitted question. I don’t know when the last time I did one of these. It’s been quite a while. We’ve had a lot of fascinating interviews with guests that I hope you’ve enjoyed, but today, we’re going to take a question from Carrie, so let’s give that a listen.
“Hi, Chris, Carrie Bennett from Carrie B Wellness. I have a question when it comes to a high-fat diet and the health of the gut microbiome when I’m coaching my nutrition clients. After reading the Sonnenburgs’ book about gut health and the gut microbiome, it seems to be that their recommendation is that fat can change the gut microbiota in a negative way. I’m just wondering how you view this in light of a Paleolithic/primal-style diet. I’m wondering if the studies that they’re referencing are utilizing poor fat sources as opposed to high-quality, pastured or grass-fed animal fats, and just your take on it. Is high fat healthy for the gut microbiota? Thanks for any information you can provide.”
Okay, great question, Carrie. Thanks so much for sending it in. I’m going to answer that specifically, but I also want to use it as a springboard for a general discussion of problems with nutritional research.
The Problems with Nutritional Research
The first issue is that most of the research related to a high-fat diet and the effects on the gut microbiome have been done in rodents. While rodent research is certainly valuable and there’s a lot that we can learn from it, the results don’t necessarily apply to humans, especially in the case of a high-fat diet. In the rodent research, the most common high-fat diet uses highly processed and refined fats like corn and soybean oil. Now that very well may apply to many Americans who are eating a high-fat diet because if they’re eating out at restaurants or even stores with prepared foods or they’re eating a lot of packaged and processed and refined foods, then many of those are going to contain industrial seed oils like corn, soybean, cottonseed, safflower and sunflower oils, and those oils have a different effect on the gut microbiome than some traditional fats would.
Is a high-fat diet bad for your gut health? Not necessarily. Check out this episode of RHR for more on how fat impacts your gut, and find out why you can’t always trust the latest research.
In humans, the research correlating high-fat diets with changes to the gut microbiome is almost exclusively observational, as far as I know, which means that it suffers from many of the issues that I detailed in my recent article critiquing the study, which claims to find that low-carb diets shorten our lifespan. That was just a train wreck of a study, and I went through it kind of point by point. If you haven’t had a chance to check that out, you can probably just Google “kresser low-carb diet shortens lifespan” and you’ll find it. But I’ll just briefly highlight some of the main points from that article as it relates to this question that Carrie sent in.
Why Correlation Isn’t Causation
The first is the most obvious, at least if you’ve been following my work or this field for any length of time, and that is that correlation is not causation. An observational study just follows groups of people and examines certain variables like their diet or their lifestyle, and then they look for associations between those variables.
For example, in this case, they would look and see that people who ate a high-fat diet tended to have worse gut microbiota. But the first thing you learn if you take a research methodology class in college or grad school is that when two variables are associated together, that doesn’t mean that one is causing the other. That’s what the phrase “correlation is not causation” means, and there are a lot of kind of silly examples that are used to make this point.
If you consider the statement, “The more firefighters that are sent to a fire, the more damage gets done,” that’s obviously a ridiculous statement, right? That’s not how it works. It’s not that more firefighters are causing the damage. It’s that when fires are worse, more firefighters are required to fight the fire so, in that example, the causation would be reversed.
Another one is children who get tutored get worse grades than children who don’t get tutored. Again, the causality there is reversed. The children who are not getting good grades in the first place are the ones that are more likely to hire a tutor, or at least their parents will.
There are a lot of examples like that just to make the point that you have to be very careful with assuming that two variables that are correlated together are, one is causing the other. There’s actually a whole website that’s dedicated to kind of mocking this assumption that unfortunately is very common in the mainstream media, and I’ll come back to that in a second. It’s a page on someone’s website, Tyler Vigen, I’m not sure if I’m pronouncing that right. It’s called “spurious correlations” and I’m not sure where he found all this stuff, but he went and found correlations that are just ridiculous.
For example, U.S. spending on science, space, and technology correlates with suicides by hanging, strangulation, and suffocation at 99.8 percent, which is almost a perfect correlation, 100 percent being perfect. I think we can all agree it’s pretty obvious that U.S. spending on science, space, and technology is not causing suicides by hanging, strangulation, or suffocation or vice versa.
Another one of my favorites is the divorce rate in Maine correlates with per capita consumption of margarine at 99.3 percent. Again, it’s obvious that there is no causality, there even though those two variables are highly correlated.
This is really critical to keep in mind when you’re looking at any observational study because observational studies were never designed to prove causality or to show that one variable is causing another. To do that, we need a randomized controlled trial.
The Bradford Hill Criteria
There are certain situations where it’s not practical to do a randomized controlled trial, and we have to do our best to apply certain criteria to the observational or epidemiological data to at least increase the probability of determining whether there’s a causal relationship, and these are referred to as Bradford Hill criteria.
These were originally published in 1965, if I remember correctly, and they include a number of different criteria, like the strength of the relationship, whether the relationships are connected together in time, and several other factors that when you put them all together, if those criteria are met, it significantly increases the likelihood that there is a causal relationship. But that’s not done in many of these studies. In fact, many epidemiologists, especially 10, 20, 30 years ago, were sounding the alarm and saying, “Look, if we do observational study, if the risk is not at least two or three times higher…,” let’s say we’re talking about a high-fat diet microbiota, “…if you don’t see at least two- or three- or even higher-fold risk, that a high-fat diet is contributing to worse microbiota,” then we can’t really even—we might as well just not even pay attention, that these small relative increases in risk, like 15 percent or 30 percent or even 50 percent higher relative risk in observational research, are not strong enough to really show us that there’s anything that we should be investigating. You want to see at least 200 or 300 percent higher risk.
I think in the cigarette smoking–lung cancer studies, it was like 3,000 percent higher. If you see variables correlated and one is leading to a 3,000 percent higher risk of something else, then yes, that’s something that you’re going to pay a lot of attention to and ideally do a randomized controlled trial for. That’s not ethical with smoking and that’s a good example where you use things like the Bradford Hill criteria to try to suss out whether there is a causal relationship. But in many of these observational studies that are being published, you’ll see references to like a 10, 20, 30 percent increase, and that’s just almost impossible to differentiate from the noise in a big sample of people.
The Healthy-User Bias
Any observational studies that have showed that people who eat high-fat diets have inferior gut microbiota or just showing correlation between those two variables, they do not prove that their high fat intake is the cause of the poor gut microbiota. Why? Well, because of the phenomenon known as the healthy-user bias, which I’ve written about and talked about on the show quite a bit.
People who engage in a behavior that’s perceived as healthy are more likely to engage in other behaviors that are also perceived as healthy and vice versa. Because fat has been perceived as unhealthy for so many years, on average, people in studies that eat more fat—and this is especially true in the 80s, 90s and the 00s—people that eat more fat will be more likely to:
- Drink too much
- Not exercise
- Eat too much sugar
- Not eat fresh fruits and vegetables, etc.
There have been many studies that have shown that to be true, that, on average, if you take a baseline population of people who have a higher fat intake or higher red meat intake or higher saturated fat intake specifically, and you compare them to another group at baseline who has lower fat intake or lower intake of red meat, you will see that, yes, that actually is true, that the group with the lower fat intake tends to smoke less, drink less, exercise more, and engage in other behaviors that are perceived as healthy because we’ve been told for 50 years that fat is bad for us and meat is bad for us.
In an ideal world, an observational study would control for all of those potential confounding factors like smoking or drinking too much or not exercising. But in the real world, that rarely happens, and even when some factors are controlled for, there are inevitably factors that are not controlled for and may not even be on the radar of the researchers.
For example, we know that people’s gut microbiota actually may affect how they process, digest, and absorb red meat and what the effects of red meat might be. A few years back, you might remember, there was a lot of brouhaha about TMAO, which is a compound that is associated with a higher risk of heart disease, and some studies found that people who eat red meat have higher levels of TMAO because it’s made from carnitine, which is in red meat. When I looked more deeply at that study, and I wrote about this, it became clear that there was no attempt to control for the gut flora of the people who were eating the meat.
If you take someone who is on a Paleo type of diet who’s eating a lot of nutrient-dense, whole foods, fresh vegetables and fruits, nuts and seeds, starchy tubers that are all high in fiber, they would have a phenomenal gut flora in that case versus someone whose intake of meat consists of burgers, KFC, sandwiches on white bread, and the typical American Diet, their gut flora is probably going to be seriously impaired. The way that those two different people respond to red meat and then subsequent TMAO production will be completely different. And that is not being controlled for in any studies that I know of.
Why Diet Quality Matters
The other problem is that these studies often don’t consider diet quality. I alluded to this before. If you consider two hypothetical people, a person who’s on a low-carb diet that eats primarily highly refined fats like industrial seed oils found in processed foods and foods cooked in restaurants, or a person who’s on a low-carb diet that eats primarily natural fats from fresh whole foods like meat and fish, avocados, nuts, seeds, etc., that are prepared at home, is it logical to predict that these two people will enjoy the same health, the same protection from disease in the same lifespan? Of course not. Yet that is exactly what most of these observational studies assume. They all suffer from what is called “nutritionism,” which is a reductionist idea that a nutrient is a nutrient, a carbohydrate is a carbohydrate, a fat is a fat, a protein is a protein, a calorie is a calorie. Virtually every study, with some notable exceptions, doesn’t pay any attention to the quality of the nutrients that are being consumed.
In order to really determine whether a high-fat diet affects the health of the gut microbiome independently of any of the other factors that we’ve mentioned, ideally you’d recruit a group of people, you’d lock them up in a metabolic ward where you can control all of the other variables, and you would get them to follow a nutrient-dense, whole-foods diet, so they would all be eating fresh, whole, nutrient-dense foods. But one group would eat a lower-fat diet and one group would eat a higher-fat diet, and then you would observe them over a period of whatever period was deemed appropriate, and you would watch for changes. In that example, you’d really be more comparing apples to apples and you’d be isolating the variable of fat intake, and ideally you would have them consuming the same kinds of fats, so that didn’t become a confounding factor. But as you might suspect, that would be an extremely expensive study. It’s almost certainly never going to happen because most of those kinds of studies at this point are funded by drug companies or from pharmaceutical companies. There’s no reason that they would do that, and public health funding for studies like that, especially very expensive studies like that, is declining, not increasing.
The Type of Diet That Can Impact Gut Flora
Having said all of that, I will point out that I think there are at least some lines of evidence suggesting that, at least in some cases, a very-low-carb or ketogenic diet may not be optimal for the gut flora over the long term. I talked about this with Jeff Leach I think several years ago now. He is one of the scientists that works on the American Gut Project, which is one of the first projects to really set out to sequence the gut microbiome, and he sent me some early results from people who had sent in stool samples to be analyzed by American Gut who were self-identified with a very-low-carb or ketogenic diet. He had noticed that there were some patterns of gut flora in those people that were not optimal.
Now you have to take this with a huge grain of salt because these were not controlled studies. The same sort of observational confounding factors could be present here. There could be a selection bias. I mean, this is not a controlled study at all, but as I discussed with Dr. Justin Sonnenburg from Stanford on my podcast a few years back, we know what feeds the beneficial gut microbiota. They’re called microbiota-accessible carbohydrates, or MACs, as he calls them, and certainly non-starchy vegetables contain some of those fibers that feed beneficial bacteria. But sodas, starchy plants, and fruits, which would be eliminated on a keto or a very-low-carb diet, and so do some foods like legumes and whole grains, which have their own issues which he’s talked about. But the point being that on a very-low-carb or keto diet, in some cases for some people, depending on how they go about it, their fermentable fiber intake would be lower and they may actually experience some consequences from that in terms of their gut flora.
Oftentimes with patients in the clinic, if they’re on a keto or very-low-carb diet, we might actually suggest that they do whatever they can to increase their intake of fermentable fiber that’s compatible with the keto approach and/or we might also suggest that they take some supplemental fermentable fibers, noncaloric prebiotic supplements, things like lacto-oligosaccharides, FOS, non-starch polysaccharides, soluble fibers, and resistant starches, all of which can help feed the beneficial gut bacteria.
Why You Should Take Nutritional Studies with a Grain of Salt
I think the takeaway here, as it so often is with these kinds of studies, is take them with a huge grain of salt because in the case of nutritional epidemiology or observational research, they are so problematic that some very prominent, well-respected, even famous epidemiologists like Dr. John Ioannidis at Stanford Medical School have come out recently, and he’s been sounding this alarm for years. He wrote a recent editorial called “The Challenge of Reforming Nutritional Epidemiologic Research.” This was published in JAMA just now in late August of 2018, and I’m just going to read you a couple little choice bits.
The first sentence is, “Some nutrition scientists and much of the public often consider epidemiologic associations of nutritional factors to represent causal effects that can inform public health policy and guidelines.” When you see a study that says, “Low-carb diet increases your risk of death by 30 percent,” that’s what he’s talking about here. “However, the emerging picture of nutritional epidemiology is difficult to reconcile with good scientific principles. The field needs radical reform.” And he goes on to explain why nutritional epidemiology or observational research should really never be used to inform public policy because as I said before, it’s just a way of generating hypotheses. It’s not a way of proving causality, and every researcher knows this.
Any science reporter should know it, and when researchers report their results to the media, most do say, “This doesn’t indicate that there’s a causal relationship.” And yet, the media doesn’t like that nuance. They want headlines that are flashy and will get people to click on them, so the way it’s reported inevitably and wrongly makes it seem like a causal relationship has been established, and then you get people changing their diet based on those headlines. You even get public policy and dietary guidelines for whole countries being based on these observational studies that were never designed to inform public policy.
I think this is a huge issue. I’ll be writing, actually, in the near future a lengthy critique of observational data in nutrition and how it has, really, I think, caused a lot of harm over the last 30 to 40 years in terms of creating confusion amongst the general public and medical professionals and leading us astray with our dietary guidelines and actually encouraging a focus on nutritionism and reductionism, where we just think in terms of macronutrients and quantity of them, like how many carbs and how much protein and how much fat versus the quality of the diet that we should be eating. That’s really what is most important.
If we all just ate real food—forget about Paleo, keto, high-carb or low-carb anything. If we all just ate real food that didn’t come out of a bag or box, we would not be in the position that we’re in today. This is real harm that this approach has caused over the past several decades, and I think we really need to address that in order to move forward.
Okay, everybody, thanks for listening. I hope this was helpful. Carrie, thanks again for sending in your question, and those of you who would like to send a question in, that’s chriskresser.com/podcastquestion, and we’ll see you next time.