Fetching from relays of people you follow doesn't have to be inefficient. How are you doing it? I guess this question can't be answered without code so nevermind.
Discussion
Jumble currently uses a greedy algorithm to ensure that each people you follow is covered by at least two relays while minimizing the total number of relays used. But it still ends up needing to fetch from dozens of relays, so loading the following feed is quite slow, and some websocket connections even get blocked by the browser.