Contest may already be over... But the speaker is referencing his own son. The speaker has no siblings, but if THAT man's father, is the speaker's own father's son, then THAT man must be the grandson of the speaker's father. Since he himself had no siblings, it must be his own son.