You need wall time to be tethered to the real world, but it's also not terribly important. Hash is way more important, which is why a newer (higher) block can have an OLDER timestamp than the previous (lower) one, in terms of wall time.
Miners are paid by the network, in part, to provide a more-or-less "accurate" timestamp, which is used to, via averaging across 2016 blocks, to keep a constant 10 minute block time. Always.
The timechain is a purely digital arrow of time, and it is built by the Bitcoin network itself, which is (in some weird sense) a clock.