It's pretty bad that the prefetcher dereferences anything that looks like a pointer and let's any other process see the address. That's ripe for language theoretical security vulnerabilities.

Is this about losing your device? From what I understood, you should be careful about what other software you're running while your M1/M2 is performing cryptographic operations.