damus already works this way for the global feed, relays are selected depending on the view (home/global/notifications/dms) I only have the option for global for now to reduce confusion, I could show the filters for the other views but will require careful design work.
I think having relay sets so you can switch between a private and public set globally might be interesting, but yeah the design is really important since this can get confusing quickly.