"People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work. They are right to be concerned: this lack of understanding is essentially unprecedented in the history of technology. "
Worth a read from Dario Amodei, we're sailing somewhat un-chartered waters now.
https://www.darioamodei.com/post/the-urgency-of-interpretability