I personally think Adam Scott (the actor) is a perfect example of this. Looks much better with beard than without. Your theory holds up.
Discussion
Once he hit his 40s, yeah. Earlier he was in the phase where it can go either way.
If a guy has an unusually strong jaw (eg Brad Pitt), then the age with which it can go either way tends to be greatly extended.
I hear the sluttiest thing a man can do is grow a mustache.
I’d like to see how Scott Adams would look with a bear🤷