I do believe that making excuses for your fitness because of your age is an easy mistake to make. Never say things like, “I’m forty, I can have a little belly,” or some such nonsense.
If you’re going to quit on yourself, just quit. Don’t blame it on your age.