The IRS can just say you owe them a large some of money if they don't like your transaction history, but assuming don't have any trades then they can't confiscate your net worth.