Too many times have physicians, from my point of view, fall short of common agreement with regards to my health, especially when it comes to nutrition. I tend to be skeptical of their opinion on the matter, more even so since I switched to a paleo diet and literally dropped the weight that I was supposed to be "genetically" stuck with. So have several of my friends around me.
I doubt the answer is as easy as some would like it to be, but empirically it seems low carb diets have tremendous benefits. I'd recommend reading the Primal Blueprint (http://primalblueprint.com/) for a more thorough explanation of the theory behind it, I'll stick with with my current diet as all indicators (including regular doctor checks) show a drastic increase in my health.
Doctor here. I agree with you about being skeptical of physicians' nutritional advice; we generally aren't nutrition experts and internists, especially, are jaded by the vast number of hospitalized patients we see who have simply abused their bodies with food and drink for decades.
That said, you echo an argument that's as useless as me telling patients "calories in < calories out": that you switched to X diet and dropped weight, just like "several of your friends." That's doesn't provide utility to any public health stance. Of course, I will freely admit that I don't have the answer, either.
What I tell my fat patients who are starting out is this (and I use these exact words): eat the same shit you're eating now, just cut it in half. For educated, motivated people who aren't in denial (which I'm convinced represents < 10% of my patients), we have a more in-depth discussion about dieting ideas that might specifically work for them.
I lost about 50lbs this year. If someone told me to cut the shit I was eating in half, no way in the world a "diet plan" would have worked. Explain to your fat patients that a diet isn't eating less, its eating differently. And three words to end this: slow carb diet.
I'm pretty sure the calories in/out model works to a certain extent. But the paleo diet helped me structure my diet in a way that lets me focusing on simple, healthy food, instead of quantities.
Thing is , we don't know if that is due to paleo or due to being mindful of our eating and thus not overeating. So in a way, the diet can mask what really makes you lose weights, I.e. eating less and working out (if you do)
I think the more popular diets stick around because they are balanced enough to not make you sick and complex enough to make you watch how much you eat. The rest is due to personal preferences.
I was a vegetarian for 3 years before that and have been very careful with my diet for all of my adult life. I've cooked most of my meals in the past 15 years and have felt a striking contrast in health since I cut carb out of my diet, including weight loss.
I spend most of my days in an office indeed, as I work in tech. That being said, I've always worked out regularly (especially cardio) and biked to work for the past 7 years. I would say switching to "lifting heavy things" instead of focusing on running probably helped too (though it precedes my paleo diet).
I doubt the answer is as easy as some would like it to be, but empirically it seems low carb diets have tremendous benefits. I'd recommend reading the Primal Blueprint (http://primalblueprint.com/) for a more thorough explanation of the theory behind it, I'll stick with with my current diet as all indicators (including regular doctor checks) show a drastic increase in my health.