I was just thinking for your own events, but I could see how it would be worthwhile to also have all the events you've tagged in your notes in one way or another.
Discussion
Getting your own events is straightforward and mostly a series of paginated calls. Depending on relay, may also need to AUTH.
Getting context of events you replied to, and those replying to you is a Big-O type problem requiring additional queries for each event pulled back in the initial set. Doable, but a higher cost is appropriate, and for best performance and reduced network consumption should be offered as a service by the relay operator