Tuesday, December 04, 2018

An exchange of half bricks

I would guess that everyone is aware of the study by Ebbeling et al, Ludwig's group, looking at the metabolic effect of low carbohydrate diets on total energy expenditure (TEE, all graphs show kcal/d) in the aftermath of weight loss on a conventional diet.

Effects of a low carbohydrate diet on energy expenditure during weight loss maintenance: randomized trial

I'd like to summarise their data using numbers taken from Tables 2 and 3 which, with a little arithmetic, allows me to produce this graph of TEE at various time points. These are as follows: when the subjects walk off the street (Pre on the graph), after a period of semi-starvation on a conventional diet (Start) and then during weight stability on a high, medium or low carbohydrate diet (End). The plot looks like this:

In the study they compared the change from the Start TEE to the End TEE, ie they used these data points:

They took the absolute changes from Start to End thus and got a resultant p of less than 0.05

This, obviously, is completely unacceptable. Well, it is if you are Kevin Hall. So now we have this

No Significant Effect of Dietary Carbohydrate versus Fat on the Reduction in Total Energy Expenditure During Maintenance of Lost Weight

What Ludwig's group did wrong (amongst the many other things pointed out by Hall and Guo) is that they used the wrong data points.

Recall the original graph:

According to Hall: If you want to ask about the effect of low carbohydrate diets on the depression in TEE produced by conventional semi-starvation you should NOT compare the semi-starved TEE (as in Start) to the TEE on a high, medium or low carbohydrate diet (End). You should instead use the TEE expenditure at randomisation (Pre on the graph). Like this:

Using Pre as your anchor point you can draw the same data thus:

Which obviously gives us p greater than 0.05 and all of the benefits of low carbohydrate diets are lost. Phew. Happy Hall. But why should anyone use the Pre values as an anchor point?

Now, no one is an unbiased researcher. Hall is, surprisingly, no exception. Hence the current exchange of half bricks in the BMJ.

As I see it the Ebbeling paper looks at the effect of LC eating on the damage done to TEE by conventional dieting.

What Hall wants the analysis to do instead is to look at the overall effect of damage done to TEE by conventional semi-starvation combined with partial rescue during weight-stable LC eating vs the combined damage done by conventional semi-starvation followed by maintained damage done by HC weight-stable eating. As he writes:

"However, the final analysis plan was modified to make the diet comparisons with the TEE measurements collected in the immediate post-weight loss period rather than at the pre-weight loss baseline"

To me Hall is stating that Ebbeling et al almost did make the "Hall" mistake of using the "Pre" TTE as anchor point but corrected this at the 11th hour, still before blinding was unmasked. What puzzles me is how Ebbeling could have ever even considered using the "pre weight loss baseline" as the anchor point in the original study design.

The massive benefit to Hall of including the conventional semi-starvation active weight loss period along with the intervention weight stability period is to dilute the remedial biological effect of LC eating out of statistical significance.

The core information which the study provides is about the remedial effect of LC eating on correcting the damage done by a conventional semi-starvation period. That effect only happens between "Start" and "End", which is when carbohydrate restriction is applied.

That's one of the MASSIVE problems with carbohydrate restricted eating. It only provides benefit when you don't eat carbohydrate!

Including data from "Pre" right through to "End" dilutes the clearly demonstrable biological effect of carbohydrate restriction on reduced TEE post conventional dieting.

So what doe the title and text of Hall's rebuttal tell us? Either about Hall or about TEE? Don't over think it!

I would also declare that my own biases are a conflict of interest but if you need me to say that then you have probably arrived here by accident, you know where the back button is.

However I would say that I am ambivalent about the importance of the TEE changes, though I suspect they do happen. What really matters to me is what happened in Aberdeen over a decade ago.



cavenewt said...

Hall should not have included the "Pre" run-in diet period—it's right there in the very title of his paper: "No Significant Effect of Dietary Carbohydrate versus Fat on the Reduction in Total Energy Expenditure DURING MAINTENANCE OF LOST WEIGHT."

valerie said...

Do you have any idea why there is such a great variation in the decrease in TEE between the Pre and Start points between the three groups?

I would expect semi-starvation to have a more consistent effect on TEE.

Or maybe this is just a chance effect, and it illustrates the pitfalls or randomization without stratification?

Peter said...

valerie, Yes, I think it's random. There will be lots of reasons but none which we could pull out a marker and measure to predict... Much as some people get in to ketosis far more easily than others etc

cave, absolutely. The paper is looking at correcting the damage done, it's not comparing LC or HC against a random population. Hall has a very blotted copy-book. Last of the dinosaurs! Hall has never hidden the depths of his bias. At least he is honest in that detail, but he still does damage. I attended a rawfeeding conference recently where Alex German was an invited non-rawfeeding contributor. Straight "starve and force exercise, count every calorie" sort of chap from Liverpool Uni vet school. Citing Hall's metabolic chamber study to show that all calories are equal...


raphi said...

i haven't yet read the study. although im not sure what's left to discover after pointing out the "Pre", "Start" and "End shennanigans

Peter said...

raphi, I doubt I'd have blogged the paper if Hall's rebuttal hadn't been so grossly misleading, and wrong of course. The other idea which has occurred to me since is that it possibly does relate to the Aberdeen study. The reason anyone might refuse to eat 2000kcal is that they do have excess calories available, as in hypoinsulinaemia. It also starts to germinate thoughts about what is meant by weight loss, from a metabolic flux point of view, ie access to fat calories. And there is that paper Mike Eades tweeted where some folks are looking at metabolic flux the Ag neurones in the VMH as an appetite regulator. They looked at glucose but you can't blame them too much for that. But the VMH is also full of CD36 receptors... Of course I doubt they were thinking about superoxide, but they should be!


Passthecream said...

Peter, obviously, the correct procedure is to take the start to end change values and extrapolate them back to the pre point by adding them to the inital values. Looks slightly better, the whole graph then relates to carb intake and p values be darned.


cavenewt said...

I just thought it was amusing that Hall's rebuttal was completely contradicted by his own title.

Display Name said...

Ebbeling Study Question:
-How does weight maintainence affect EE?
Hall Study Question:
-How does weight loss and maintainence affect EE?

These are 2 DIFFERENT research questions. It appears Hall is demanding that HIS research question is answered because he does not like the results. Either question is valid to study, but the Ebbeling question is more rigorous because:

-Weight loss and maintainance have very different effects on EE and these phases must be separated.
-The baseline EE test should be performed close to randomization (as in Ebbeling's study question, not much earlier as in Hall's)
-Previous studies have already been conducted combining these phases (eg Hall). Why repeat them ad naseum?
-Even if there is only a small benefit in EE during weight loss, it's important to know what the effect is on the maintenance phase, particularly since this phase should last far longer than the weight loss phase and there is more opportunity for higher EE to provide a benefit over time (we were limited to 5 months of maintenance).

The EE outcome is a mean, which means that some people may benefit from LC more than others when we consider the range and standard deviation. Hall claims a small EE of 100 kcals/day is not significant, but some are benefiting more and we need to identify these people.

It's especially important mechanistically that effect modification by insulin resistance was discovered by Ebbeling. This suggests that the weight maintainence phase may need to be targeted in people with IR who lose weight by any means (possibly even including those with IR after bariatric weight loss). Only Ebbeling's work provides this important lead about IR.

RCTs routinely make changes to their protocols without any objection, but now that these results are in conflict with the CICO hypothesis, there are implications and allegations of research fraud with NO SUPPORTING EVIDENCE.

Hall's "reanalysis" using his study question ironically SUPPORTS Ebbeling's results. How can the Ebbeling data be flawed if the results were similar for the study question demanded by Hall? The real question is whether Hall has the data to repeat the Ebbeling analysis. So far, Hall has NOT responded to my question about this.

I find it rather stunning that a CICO proponent has actually argued that an increase in EE due to LC would be detrimental to longevity based on this study. This is a good example of how things have devolved in the wake of Ebbeling- desparation has ensued.

The bottom line is that Hall's "reanalysis" is an inappropriate demand for his study question to be answered by Ebbeling. Hall's results do ABSOLUTELY NOTHING to diminish the impact of Ebbeling's results.

By First Name

donny said...

The Mediterranean dieters don't seem to have gotten the memo about dietary "quality" mattering more than macronutrient ratios.

donny said...

If I was going to do the Mediterranean diet for maintenance, using Hall's start points, would I be better off going low carb, or high carb? Looks like there might be a statistically significant advantage to the switch to low carb, but not to low fat. Choosing what to compare things to is fun.

Last time I posted multiple times in a row here, I had a manic episode. So I'll stop for now.

Peter said...

cave, I should chill a little. I do see what you mean!

Hi First, in the original post I had a rambling aside about how you sometimes submit a paper and the scrutineers want you to have done a different study to the one you have done. I see exactly what you are saying…

I’ve had some info via Tucker as to how the initial protocol started off as using the design Hall considers to be correct and I consider completely unacceptable. All I can say is kudos for realising there was a mistake and for correcting it before it was too late.

Re 100kcal vs 270kcal. Clearly 270kcal is the correct figure and 100kcal is incorrect. So no need to worry about how clinically significant a 100kcal improvement is when the data show the actual improvement is 270kcal….

What is clear is that 30% calorie reduction for six days with forced exercise and very careful food source choices can prop up the (extreme) low fat protocol (which sounds like Denise Minger's term "carbosis" btw). That was its purpose, I suspect.

Looking at the remedial effect of modest LC eating on the the damage done by two months of conventional dieting is, I agree, a totally non-related study. One of significantly greater impact to real life than six days in a metabolic chamber.

Point scoring vs practical application.

Hi donny, personally I think both might matter.


Peter said...

donny, crossed postings. I have many ideas which drive me to LC eating... And modest protein restriction. Mostly related to the insulin/GH/IGF-1 axis.


Passthecream said...


Paul L said...

Peter, as always, thanks for your clarity. I confess I struggle to keep up with these analyses, since neither nutrition nor statistics have ever been my fields (French and music, instead), but I do appreciate the digs against Keven Hall, who seems a right . . . (fill in the blank with your favorite insult).

Btw, the new rats are charming. Hope yours are doing well, too.

Peter said...

Love it pass, xkcd is one of my frequently used bookmarks. Dilbert is top and Jesus and Mo second, xkcd third. Hadn't seen that one!

Paul, great to hear it. Our two ratties have turned out to be stars too. Really lucky to get this pair, both very confident and outgoing. Though I have to say my wife has been feeding them cheese through the cage bars so I got nipped in a flurry of overexcitement when I went to open the cage door last night. Humph!


cavenewt said...

My son had a pair of rats when he was younger. They were charming and interactive pets. They were also prolific tumor factories, which finally did them in, despite several successful surgeries. Does that happen to all of them?

Peter said...

cave, yes, they do tend to, especially when fed on CIAB. The one's we've low-ish carbed have tended to go with skin problems rather than tumours. They've been fed human level protein intake so never even looked at ketosis. None of them have had amazingly long lifespans. But they are very cute!


donny said...

Just eating low carb, not worrying about fat vs. protein brought me down to 170 from 190. For my height, that brings me from barely obese to barely overweight by bmi. I know bmi can be misleading, but for me, the mirror and tape measure tell me that obese and overweight were accurate in my case. I could get leaner by eating less, which I usually did by upping protein, lowering fat. Which worked, I got leaner, but the leaner I got, the more prone to binges I got. A pound of cheese, or 700 gram bag of nuts. Last time I got down to 154 this way, I was having binge problems. We were storing some non-food items for some neighbours who were snowbirding to Indonesia, I ended up bingeing on some of that. Maple syrup, honey. Keto was just getting off, I got curious about what it would be like to eat the 4:1 diet, so I tried that. The tendency to binge went away. I didn't stay that high fat, but it got me more wary of protein, since then my "easy maintenance" weight has gone down about ten pounds to 160, but I can keep it closer to 150 on a tighter ratio, without the issues with binges and low energy. Maybe coincidentally, for 25 years or so I'd been hearing voices. It started as a constant thing, typical schizophrenic sort of stuff, but was down mostly to the illusion that people were yelling when I operated loud machinery, anyways, I noticed operating a gas wood splitter that always set me off that that was gone when I went to 4:1, it never came back.

Peter said...


The loss of voices is really nice to hear about. You do know there is a similar case report, of remission of very severe schizophrenic voices? The lady only went in to ketosis for weight control, not schizophrenia management. She'd had severe voices for over 60 years.There are also reports from Abraham Hoffer of tidying up a mixed macro diet and adding B3? Took many months to take effect whereas ketosis is less than a week...


Passthecream said...

Ketone bodies - seems that the medium is the message: https://www.cell.com/trends/endocrinology-metabolism/fulltext/S1043-2760(13)00156-2

George Henderson said...

Reading all the BMJ responses, it looks like the Ebbeling study might be the inverse of the Aberdeen study.
The authors don't deny that their low carbers might be moving more. They don't insist that the TEE salvation is all about uncoupling or other metabolic advantage, because they didn't control movement.
If there is in fact no MAD, then low carb either kills your appetite, or it gives you ants in your pants. Helped, I would say, by a reduction in joint pain after grain avoidance.
Either way it works well enough to tie Hall up in interesting knots of his own design.

valerie said...

In the Ebbeling study, they also measure REE in a metabolic chamber. The REE was not different between the 3 diets.

One interpretation is that people on the low-carb diet moved more (unintentionally). Their REE did not increase but their TEE did. OK. Maybe.

Another interpretation is that the doubly labeled water measure of TEE is not reliable. I asked Google about it. I found this.


"...there was a consistent overestimation of rCO2 by the DLW [doubly labeled water] technique in the HF [high-fat]-fed animals..."

Moreover, the raw data is available for the Ebbeling study. Looking at participant's data one at a time (not averaged), it appears that nearly half the participants have implausible TEE. For instance, quite a few of them have higher TEE right after weight loss (on the low-calorie diet) than at the start of the study.

The authors even acknowledge that they removed some data points from their analysis because they judged them implausible. I thought doubly labeled water would give super reliable measurements of TEE. That is obviously not the case here.

In my view, the most plausible explanation is that doubly labeled water is not all it's cracked up to be.