If clients supported it you could use a single encryption key that rotates when you change your follow list and posts a DM-like event with a new Kind for each of the people who can read them now. Actually you only need to rotate when you unfollow