Hey nostr:nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqy0emt2wlpsezcnmxtyrpf33qe7gwy5u8yzssvv6uw53em0k32t7qz32cpy 👋 What do you think about this "new" kv-store? It seems to me that is is good but doesn't support, by default, durability on point-write, RMW operations as the operations are done in memory. The kv-store answers the user directly and without appending to a WAL. But it seems strange to me, maybe I missed something. I'm particularly curious about the reason why LMDB is not listed in the benchmarks. https://www.vldb.org/pvldb/vol18/p4910-kanellis.pdf

Hey nostr:npub1y0emt2wlpsezcnmxtyrpf33qe7gwy5u8yzssvv6uw53em0k32t7q7smm9n, we want to, zero-downtime swap an #LMDB environment that comes from an S3 with one that is already on disk (and opened). My first idea was to open the incoming env, open a write txn on the dst one and clear and copy the entries into the existing one but I fear about the MDB_TXN_FULL error. The second idea is to close the env (awaiting running txns) and swap the env files. This can produce a downtime. Do you recommend anything else?

nostr:npub1zea4m6qyckzcajvy9ad768paqe688j9zd4uultz52a5u9zuwewlqsmrzh5 nostr:npub15s57gd2l4asmta5xzh96zmkltt898cd880zc9aeqwgcclxveagxswfxszk Unfortunately, lmdb-rkv is always above heed on lib.rs, even if it's no more maintained... https://lib.rs/search?q=lmdb

nostr:npub1acg6thl5psv62405rljzkj8spesceyfz2c32udakc2ak0dmvfeyse9p35c nostr:npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s Would be better if reproduced from scratch in C but if you know the last operation that corrupts the database maybe do this last one in C if possible. Providing the pre-corrupted database and the last operations breaking it could probably help diagnosis.

nostr:npub15s57gd2l4asmta5xzh96zmkltt898cd880zc9aeqwgcclxveagxswfxszk Just go with #LMDB. Only a single file, faster than SQLite for key-value storage and zerocopy read. I maintain a Rust wrapper called heed and Meilisearch is running on too of that, works perfectly 🎉

I recently updated the Wikipedia page. https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database

nostr:npub1acg6thl5psv62405rljzkj8spesceyfz2c32udakc2ak0dmvfeyse9p35c nostr:npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s

I worked on bringing the DUPSORT LMDB feature to heed today. I will continue with the DUPFIXED later. I will release a new alpha version later next week, probably 🙂 https://github.com/meilisearch/heed/pull/200

nostr:npub1acg6thl5psv62405rljzkj8spesceyfz2c32udakc2ak0dmvfeyse9p35c nostr:npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s If you need anything, tell me, create an issue and I will see if I have time to work on it. I am working on this big v0.20 alpha refactor so let’s break everything 🏺

nostr:npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s nostr:npub1acg6thl5psv62405rljzkj8spesceyfz2c32udakc2ak0dmvfeyse9p35c I advise you to use heed that I maintain. We use it in Meilisearch. You should try the latest v0.20-alpha versions. https://lib.rs/crates/heed