True. This was only half serious.

Only 2 weeks ago I did a similar experiment and noted how it was kinda dangerous; you can prompt it to go in entirely the wrong direction and then it will very confidently do the wrong thing, which is not ideal in cryptography!