I could try to adapt nostrdb to use indexeddb as a backend, still feel like a native js solution might be better.
Discussion
Might be better
When has JavaScript ever been better?! 😂😂 kidding, sort of.
I’m sure we can get a lot more out of the dexie (indexedDB) cache adapter. Just tough to find the time.
nostrdb has some other crazy optimizations like multithreaded ingester that will find the ID and check for its existence in the DB before continuing json parsing and checking signatures. So some web nostr db could steal those.
IndexedDB is slow as hell. Chrome implemented IndexedDB on top of LevelDB and Firefox on top of SQLite. Almost all fast-enough web clients cache events in JS memory instead of query from IndexedDB on every read.
This is also why I intentionally do not implement storage layer for nostr.ts.
Browser is not designed for local-first, client driven applications.
If you implement NostrDB on top of IndexedDB, you effectively implemented it on top of SQLite/LevelDB which is exactly what you planned to avoid at the first place.
Thoughts on using redis or Postgres instead? Obviously you have the network overhead but feels like it could actually be much simpler given 2 decades of web apps being built like that.
At least in Blowater, my current solution is to treat IndexedDB as a huge string store. I don't query from it at all. I just load all events to memory on init and do manual map/filter/reduce on a huge event array. The performance is 100X better than querying from IndexedDB, even for 100K+ events.
🤯 that’s insane. It’s actually faster to pull in everything and operate on huge arrays?!
Yes. O(n) in memory is faster than nlogn in disk for small data such as 100K events. That said how horrible IndexedDB is. Loading the whole table in indexed db is also faster than select 1 by index. My guess is that V8 team and other chrome component teams are at different levels. V8 is so fast that most computation heavy thing can happen in JS now.
nostr:npub1zuuajd7u3sx8xu92yav9jwxpr839cs0kc3q6t56vd5u9q033xmhsk6c2uc If this web client is SSR, when it makes a lot of sense. But it doesn't make sense for pure client driven SPAs or native apps. Effectively it becomes a different relay interface. So why not extend the query interface of relays?
The smart client dumb relay setup with Nostr is great for encouraging decentralization. Which is why I think it’s great that we don’t extend that relay search interface too far.
That said, I believe we’re likely going to see a layer of caching services start to pop up to serve client devs but I’m not sure how I feel about that long term yet.
alternatively could do something like nip07 and expose a nostrdb relay interface as an extension 🤔
Wait extensions are just js as well right? Not sure what native capabilities they have.
Their capabilities are pretty much the same, just running in another V8 isolate (Chrome)
I think the only win you can really get then is using a compact note format like nostrdb’s to fit as many notes into memory as possible and use that as the note representation instead of js objects. Still pretty horrible though.
That could be a benefit if the client chooses to keep GBs of events in-memory. Blowater actually plans to "never" delete locally.
Many web clients still don't treat on-device storage the main storage and query from relays everytime.
I believe local-first approach is the only way to have decentralization product-wise and to have fast client tech-wise. Therefore, I like nostrdb's motivation a lot.
But what will be nostrdb's query interface? Function calls? Query languages? Can it answer questions such as what's all my notes which have been replied within 2 hours of creation? I would love to see a nostr "database" that provides graph algorithm queries.
Right now it is just function calls (lookup profiles by pubkey, notes by id) but the plan is to full nostr query support.
Then i will eventually remove all my websocket code from damus and talk to nostrdb directly, it will become a caching/multiplexing relay of sorts. Any data it doesn’t have it can use negentropy to fetch from strfry relays. This will all be transparent under the hood so clients won’t have to deal with the complexity.
I am exploring Tauri, it's like a better Electron. The native process is Rust instead of NodeJS. Maybe we can have 80% of the features in Browser and the user can choose to download a native app for the rest 20%. But this approach basically says goodbye to PWA mobile.
Mobile is just too different. An attempt to cross-platform mobile is very hard.
Yeah. Tauri looks awesome. Haven’t had time to really dig into the weeds there though.
Seems like you could just put nostrdb directly in the browser like these people did with SQL. https://github.com/sql-js/sql.js