Does wavlake support lists? People could just follow you by putting you into one for their lists. Then they don't need to see your posts, just see what you are listening to. Either way, the list should be theirs, not yours.
You can already use the current lists we have to manage access controls. That is easy. You just need to code the control itself (check if the key is in the list before downloading). I think Blossom has a version of that already.