Not all vegans and vegetarians are so for altruistic reasons. Some simply choose it for health reasons. Same applies to different diets.

But yes, I agree with you that it is more ethical to consume what you grow, raise or hunt, locally or in the wild.