14000 kind 1 notes is about 7MB in nostrdb. Heaviest is contact lists. 4000 lists are about 70mb [1]. These are stored in a compact binary format so they are pretty space efficient, but it’s amazing how many notes there are floating around. I hit 14000 text notes after a few minutes of use.

Working on #nostrdb pruning logic so that #damus will use less space. We probably don’t need to store everything, so I am working on making sure only the important stuff sticks around the longest (profiles, etc). Storing text notes is nice for when we add note searching.

nostr:npub13v47pg9dxjq96an8jfev9znhm0k7ntwtlh9y335paj9kyjsjpznqzzl3l8 has been working on more aggressive image cache pruning. Images make up most of the storage space, so we’re making sure damus as space efficient as possible for the next release.

It’s nice to be back hacking on damus 🤗 whats everyone working on today?

[1] https://cdn.jb55.com/s/64418a4c0e35247e.txt

Reply to this note

Please Login to reply.

Discussion

About images: There are a lot of duplicates (e.g. memes). I imagine they could be easily sorted out to save space.

Nice job Will and team!

Work is good. Good work is cathartic.

Made content taken from our ( nostr:npub1afne208f0mlwl9aktfkss0xs6zd7ga6w4e5hx3dh3lg24vfl8ddswzqdtu ) Nostrasia talk, now scheduling the next meet ups.

Be well and build

🤙🏼

- Streaming custodial zaps to my desktop, tablet and laptop, using gamestr replaceable events, and my personal relay running using fonstr

- set up fonstr with a public https cert, while just a test relay, its worked amazingly well for me and ive never had to debug it

- two client side apps to measure my work and productivity in realtime with streaming zaps, one hourly averages, one daily pie chart and progress bars

- created a single page app that gives you a tarot card each day, created some artwork, and history page ( https://tarotask.com/ )

- working with new AI code, text and image models, ( llama2 70B, realistic vision 3.0 )

- working on single celled autonomous AI agents, with nostr keys, called "Ameba"s

- now hooking up AI agents, to my daily journals, to multi AI inferencing AIs, that are 5x cheaper/faster than GPT, with open models (hopefully finished this evening)

454k sats streamed to me, during the day, in almost 1000 tx.

You know it’s serious when will starts .txt posting. 👀

Amazing!

Since images use the most storage space: Are images being reused, or does every time someone uploads a duplicate image it takes up additional space? Perhaps images can be hashed and matched so when an exact duplicate is added the original is linked instead of wasting more space.

And then apply this same logic to videos and all other media as well.

could damus use an LRU and expose a slider for the user to choose cache size?

nostrscript cache policy?!