It is possible they could do this, yes, but would be very difficult.

From the perspective of openAI, all of our user's queries are bunched together as one super user "PPQ" which is our business account. But if individual users are sharing PII in the chat then technically they (openAI) could analyze the content and attempt to tie things together.

But even if a user said "I am john smith" in their queries, I don't think it could ever be brought to court in that way since anyone can say "I am John Smith" in a chat. It doesn't PROVE that the person who queried is actually that person since the credit card or email is never connected.