My cousins got their huge size from their mother's side. She was Russian and these guys are 6'3" plus , 300 lbs, size 16 shoes. Their Dad was 5'8 and husky, sorta average.

By age 3-4 they can tell with 90% certainty though your son can push it by eating extremely healthy, playing sports and maintaining great posture.

All 3 of my sons are taller than me by at least an inch.