That downside might be too much. Most people on Nostr use signal. It seems like adding a message with the handle to the lightning transaction directly to their lnurl might be the easiest way. I just don't know how private that is.
Discussion
I've done that by DMing a unique random email alias then they email me their signal handle.