Replaceable doesn’t always mean it will use the same space on disk, the nature of lmdb and append-only dbs like nostrdb means that if we didn’t have rate limiting on replaceable events then it would be an easy way to fill a disk, especially since nostrdb treats replaceable events as a version log.
Discussion
You should have a policy to delete old versions after a threshold. Keeping them all will not be sustainable on most relays. And most relays already delete them anyway, so it wouldn't be a surprise to users if you do delete them as well.
deleting them doesn’t clear up space, this is just how lmdb and strfry works
That seems like a massive problem, then. Regardless of drafts.
The changes to contact lists alone will fill it up very quickly. Especially now that we are incentivizing more adds and removes from the follow list to avoid getting it so big.
This is why rate limiting is so important, people can fill the disk quickly otherwise. I can always nuke the relay when it gets too big, so it’s not a big deal.
writing a GC for event and enabling the relay operator to define a storage limit was the first thing i did for replicatr... gonna be nice when i finally get it production ready (you can use it but i promise you it will burn your CPU after 24 hours)
you need a garbage collector :)
Standard way to do it is a compact step which rebuilds the DB skipping all the deleted stuff. I still need to make this a feature in nostrdb
Looks like Bitcoin found a use finally
Grrr replied to the wrong note?? What is going on.
it's already available in badger, and it's faster than stink... with batching i wrote a thing that could measure utilization in 8 seconds on 15gb of event data (it was only traversing keys, but that is how you get it to work)