Because the house is registered into a government database. If you do not pay taxes the house goes poof even if you have the keys.
Discussion
that's right. my ownership of the house isn't about my house keys, but by the fact it's recognized by the central authority which has the monopoly on violence. not a good analogy with bitcoin keys