yay, the new pubkey index compression scheme i have built is now in testing phase, should be a few more hours and i'll have it committed and going live on wss://mleku.realy.lol

this is a hella compression algorithm for saving resources for handling nostr follow/mute events, it will achieve savings of between 75-87.5% of the storage cost of pubkeys in the database meaning the relay can provide a LOT more usable data about social graph and will effectively enable a fairly precise census of nostr users in combination with a network spider

i was at first only going to apply it to these hefty events but then i realised it's probably no less performant to use it on all event kinds which will shave another 24-72 bytes off typical kind 1 events, for instance, which is somewhere around 15% improvement in storage

just gotta trawl through some stack traces to figure out where i have forgot to allocate memory and check bounds and it will be ready