Are you pointing out there's a difference between algos reducible to known hardness assumptions (let's say Schnorr not ecdsa for simplicity), and thus computationally hard at best, vs information theoretic security like a one time pad? If so, good point to raise, but i trust both of the much more than the hardware substrate which is, to my mind, the biggest weak spot to worry about, and where it's the most important to worry about CPOFs.