Milk has unique properties that, for better or worse, makes it not like any other product solid or liquid. Removing the fat destroys some of them - but it doesn't make it Sugar Water, again, for better or worse.

Also, carbs aren't evil - they are only a problem if that's all you have. Same for seed oils.

The problem is most people get 90% of their calories from carbs and seed oils, whereas it should be more like 20-30% from carbs and maybe 2-3% from seed oils, and that includes nuts ( which are basically whole food seed oils ).