Ok, usually it means stick to your area of expertise, but I don't think the treatment of parents of obese kids is anyone's private purview.

Generally the phrase is used as a lazy way to shutdown unwanted opinions. It is an appeal to an unknown authority. I'd rather see a good argument refuting the idea. "Not your business" is a decent start.