Sherlock Holmes is back on the screen. A master of deduction and a firm believer in science, he is showing us the devil is in the details. But even data acquired through vigorous scientific studies need to be exposed to the same meticulous scrutiny worthy of Holmes. For the devil does seem to reside comfortably in the details.
Recently, a study written by Frank Sacks and colleges was published in renowned New England Journal of Medicine. It examined the effects of four diets differing in macronutrient composition. The study was to a large degree flawed by bad design decisions. The diets were too similar at baseline and ended up looking close to identical. In addition, the researchers chose to do an “intent to treat” analysis witch made it impossible to see the exact effect of following the prescribed diets. Reading Sacks et al we find this, “A smaller group of studies that extended the follow-up to 1 year did not show that low-carbohydrate, high-protein diets were superior to high-carbohydrate, low-fat diets.” The statement is immediately followed by a group of good looking references which supposedly have shown that low carb dieting for one year is no more effective than low fat dieting. But this is not what the referenced articles show us. It is what they try to tell us, but their data tells a different story. Reading the entire articles, looking closely at the details we find that the statement of Sacks et al is in fact false.
Two of the referenced articles are Foster et al 2003 and Stern et al 2004. Both studies compared a diet low in fat and calories with one low in carbohydrates over one year. Main outcome was (or was supposed to be) weight change.
Foster et al tells us the low carb group lost on average 4.4kg while the low fat group lost 2.5kg. This difference of 1.9kg is not significant and he authors cannot rightly claim that there was a difference. But these numbers also include the subjects in the study who for various reasons didn’t follow their prescribed diet. If we look at the difference between the two groups when we only include those who actually stayed on the diet for a full year the numbers are 7.3kg for the low carb group and 4.5kg for the low fat group. The difference increases to 2.8kg. It might still not be significantly different, but it’s hard to say the diets are the same. In the discussion section Foster et al writes: “The lack of a statistically significant difference between the groups at one year is most likely due to greater weight regain in the low-carbohydrate group and the small sample size.” But weight regain in the low carbohydrate group appeared when carbohydrates were reintroduced to the diet. The underlying conclusion is that decreasing carbohydrates makes you lose weight and increasing carbohydrate intake makes you regain weight.
Similarly, in the study by Stern et al the average weight loss at one year was 5.1kg in the low carb group and 3.1kg in the low fat group. A 2kg difference. The conclusion in their abstract is, “Weight loss was similar between groups…” Once again, if only the subjects who completed one year of dieting were examined the low carb group lost on average 7.3kg and the low fat group lost 3.7kg. A 3.6kg difference. Once again the actual effect of following the prescribed diet is covered up by a statistical tool which includes all participants in the study, whether they actually stayed on the diet or not. In addition, Stern et al also reintroduced carbohydrates to the low carb group thus slowing the weight loss and causing weight regain.
The best part of the article from Stern et al is this, “Persons on the low-carbohydrate diet who dropped out lost less weight than those who completed the study (change, -0.2 ± 7.6 kg vs. -7.3 ± 8.3 kg, respectively; mean difference, -7.1 kg [CI, -11.6 kg to -2.8 kg]; P = 0.003).”
This tells us that the subjects who followed the low carb diet for one year lost on average 7.3kg while the subjects who didn’t complete the diet, but was still a part of the final analysis lost on average 0.2kg of weight. And it is natural to assume that following a diet gives a different result from not following the diet. But, when looking at the low fat group in the same study we find that, “In contrast, weight loss was not significantly different for those on the conventional diet, whether they dropped out or completed the study (change, -2.2 ± 9.5 kg vs. -3.7 ± 7.7, respectively; mean difference, -1.5 kg [CI, -5.7 kg to 2.7 kg]; P > 0.2).”
Translated, the subjects who completed one year of low fat dieting lost on average 3.7kg of weight while the subjects who didn’t follow the diet lost 2.2kg. What this last part implies is that following a low fat diet is no more effective than not following a diet at all.
In conclusion two of the studies wildly cited for not showing a difference between low fat versus low carb diets, actually showed that reducing carbohydrate intake is more effective than reducing fat intake, that decreasing dietary carbohydrates make you lose weight while increasing intake of carbohydrates slows weight loss, and that following a low fat diet is about as effective as not following a diet. The rest of the references in Sacks et al show a similar pattern.
The devil is tapping his hoof to the beat of his fiddle. Salvation is apparently found in the overexcited use of statistical tools, and Sherlock… Sherlock drugged himself silly to get away from all the nonsense.
If you want more details, read this:
this:
and this:
http://www.annals.org/content/140/10/778.long
0 Response to "The devil is in the details"
Post a Comment