It's so fast because it uses a caching server that gathers events from the top 100 public and paid relays. So rather than having to read from a dozen or more relays it only reads from one -- the caching server. The server also aggregates and indexes the data in a way that relays don't, making it much faster to gather metrics about notes (# of likes/replies) and also allows for filtering that other clients don't. Your configured relays for writing are still posted to for all write events (notes, reactions, profile updates etc).

I'll be expanding the number of relays it reads from soon and it's on the roadmap to also allow using NostrGram without the caching server for folks who want to use it for custom/private relays. Lots of stuff on the roadmap atm!