"Perfectly" scarce is a bit of a stretch. It is scarce by consensus, and yes that is a pretty strong convention. Perfect, maybe not. It's OK for people to have doubts. Consensus can be gamed. Especially if we are not careful. Indeed the chain could split due to conensus gaming. I am quite sure there are those that would like to see that happen. And it has happened before, almost fatally. We need to stay humble and avoid being complacent on these issues.
Discussion
It’s simpler than that though. Bitcoin is 21 million and it’s perfectly scarce because of that explicit limit. I run a node, therefore no one can say that anything is different.
Bitcoin’s consensus is and will be defined by the intolerant minority. I run a node that defines it as 21 million, and anything incompatible with that is invalid.
I think accepting the notion that this is NOT set and determined is itself a mental concession. One that I refuse to give. There is no way to “game consensus” and make bitcoin not 21 million, because I will never call anything that changes that Bitcoin. If we accept that this can be gamed, we literally give up leeway to game it. From the POV of my node and my bitcoin, it can’t be.