basically queries just return pointers to memory aligned structs stored on disk. so no copying data in out. lmdb queries are so fast (its like an in-memory btree lookup), so this gives you an "infinitely" large external memory for nostr notes, powered by the operating systems virtual memory infrastructure.

Reply to this note

Please Login to reply.

Discussion

Ok Ive been toying with a similar implementation but this one is all in ram, doesn’t touch disk. Can store about 3-6 million notes easily

when you use something like lmdb you can let the OS reclaim old pages when under memory pressure. so its a bit nicer and you don't have to worry about the size of the database