its mainly that the peers learn your IP address.

This can partially be addressed with a VPN or Tor, but I'm not sure how sophisticated network layer identity management could work, like a new IP for each group etc.