Coracle uses your follow and mute lists this way. The problems are 1. Private follows, which could become more popular in the future, and 2. Bootstrapping new users. That could be done in a lot of ways, for example with PoW, or an artificial trust rank where services verify the pubkey via captcha, payment, or something else.