Half of all crop farming goes to feed animals anyway, so just as many small animals will be killed harvesting crops for the cow as harvesting for humans. And the fact that the harvested food will be able to feed several times more people than the flesh of the cow because of how inefficient the cow is at turning grain/grass/feed into meat.

The goal of Veganism is to mitigate suffering as much as possible, and there aren't many other decisions that can reduce one's impact by 75% so easily. Total elimination of harm in food production is very difficult.