That's the only logical conclusion. Otherwise, they'd only charge you the tax you deferred. The penalty is just that. It's a punishment for pulling out of their scheme.