Prior to making these comments, Jesus takes a denarius and asks, "whose face is in this coin?" to which the pharasees respond with "Caesars" to which Jesus says, "Give to Caesar what is Caesars and give to God what is God's."

Nothing of this world is eternal and will fall away. It is a better use of time to focus on the eternal. I.e, give to God what is God's.