He's not being defiant, he's just telling you that his patience with you is coming to an end and you should leave.