But I want fast sync access
Discussion
Negentropy sync? It can be added there.
I mean synchronous access in code (no futures/promises)
That feels so wrong. Everything on Android these days is async. The sync stuff is all wait calls on async procedures.
Does it feel wrong to access memory synchronously?
It all depends on the i/o operation, where and for how long it's blocking. Nostrdb should be crazy fast, way faster than sqlite3 whose most popular bindings are...also sync.
Sync allows for API design that async does not, and I want that flexibility at this lower level. You can always turn sync into async but not the other way round.
I am very confused on what you are calling sync. Sync I/O is not the same as promises/futures that you mentioned before. Regarless, any Lmdb-based citrine-like option can reach similar performance levels as an embed lmdb option because the cost of communication is negligible compared to the I/O.
nostr query is as fast as an in memory btree lookup. Out of process relay will never compare, especially with the serialization overhead
nostrdb*
Only if the data happens to be already mapped to memory, right? If it needs to go to the disk, performance drops. Even mmap is slow compared to any serialization.
I have never noticed a page fault
How much memory are you loading usually?
Can be large but its not heap memory https://lmdb.readthedocs.io/en/release/#:~:text=Diagnostic%20tools%20often,at%20any%20moment
actually i do notice page faults when I run “ndb stat” which walks every key/value in the database and can take a few seconds for a large db the first time you run it. Every subsequent run is instant due to the page cache (https://en.m.wikipedia.org/wiki/Page_cache ), since mmap leverages that. the db persists in memory even after the app closes.
This is why notedeck startup time is stupid fast, the db can be already in memory when you open it
I mean, if the DB is already in memory, then it's not the real start up time, right? It feels like we are comparing apples to oranges.
Sure, if users have enough memory to play all the apps and come back, it works well. But, at least on mobile, user's LOVE to kill apps (swipe up) every time they leave them. And that's for a reason. The performance hit of having multiple apps running at the same time is very visible to users even though most apps are not doing anything in the background.
You might argue that if users kill apps all the time, there is more memory for nostrdb pages to stay live, but what happens if 3-5 apps have embedded nostrdb at the same time? Are they all going to stay? What happens if it is 10-20 apps using nostrdb.
This is very similar to how Amethyst works and why Amethyst is fast: we just keep the memory live when the app goes to the background. When the users come back, all the tabs, all the feeds are already pre-loaded. The strategy works until everybody starts doing the same and then no apps stays live and everything is slow.
> You might argue that if users kill apps all the time, there is more memory for nostrdb pages to stay live, but what happens if 3-5 apps have embedded nostrdb at the same time? Are they all going to stay? What happens if it is 10-20 apps using nostrdb.
From what i understand the page cache maintains an LRU cache of pages (4096 byte slices of the db). Even if you have lots of nostrdbs, the OS can intelligently evict pages that haven’t been touched in a while.
This is the great thing about virtual memory mapping, the OS should make sure to have pages that are frequently accesses available while pages that don’t get touched that often can be reclaimed.
You don’t need all of the data in a DB all the time, the OS will make sure there is enough memory for apps that need it based on access patterns.
The OS should always use all available memory for the page cache, on my desktop arc (zfs thing) is using 16gb right now.
Also the comparison to citrine is weird. This is not a replacement for a local relay, it’s for a high performance in-process relay. Thats a different thing.
You can’t have local relays on iOS. Think of this as the nostr version of indexeddb in web browsers.
I agree it is different. But we can do local relays on Android. So, to me the question really is what is the DB that the local relay is using and how to make that relay load from disk as fast as possible. Sure, the apps will still have their own cache, but it can be an in-memory only cache. If the relay is fast, the apps don't need do manage any event storage by themselves.
The in-memory caches are not evictable, so it’s doing a shittier job that the OS could be doing automatically.
But they are. Java has a gargabe collector that knows way more than the OS does and can optimize for when to call it. Then the two objects you use are WeakReference and SoftReference, Weak is used to tell the GC to clear as soon as no one else is using this object and Soft indicates that the GC should leave the object in memory and only clear on memory contention.
These are all theoretical without much substance to back it up. The real benefits is not just automatic memory management, it’s the not needing the in memory cache at all. no async code and things being out of sync :
They are not theorical man... Efficient garbage collection without the dev even thinking about it is literally the whole reason Java exists. Every app uses those things all the time. For many apps, the cache is just the list objects for the feed itself. We do some crazy things on top of it, but I don't expect anyone to have to do it for smaller apps. The rest is just regular loads using Nostr filters.
you’ve chosen to optimize in an area I’m avoiding entirely. I am just happy i don’t have to deal with garbage collection and inefficient use of memory.
You are not avoiding it. You are making an entire DB to mimic garbage collection because you don't have garbage collection. I fell like I am the one avoiding everything you went through to make it.
And I am happy you are doing it. I really do. All I am asking is this discussion is to provide performance indicators that allows us to compare the complete performance with other stacks.
If I compare my cache with your DB already in memory, the performance is exactly the same. Mine might be slower because it is full thread safe for 1000s of reads and writes in the same millisecond. But that's it. Basically its the difference between B+ trees vs hash tables.
So, if I follow 500 people and use 5 Nostr apps, do you truly think duplicating 500 profiles in memory on each of those apps is an "efficient" use of memory?
In your example it is not, but in the notedeck usecase (nostr browser) of potentially hundreds of apps in a single process is very efficient and faster than having in memory caches of duplicate data from a local relay across multiple processes.
And yes if you have something in memory is likely the same or faster when using a hashmap, but performance isn’t a one dimensional thing like comparing btree to hashmap performance. It’s the entire system as a whole. For instance, the binary note format is cpu cache friendly. Even just switching to that in damus ios helped perf immensely.
A lot of the computational things involved in note processing is amortized by having it done up front in the nostrdb and stored forever. Things like note content parsing (contents are chopped up into blocks, these are just offsets into the contents tagged with a type (url, hashtag, etc), stat counting, etc. i’m also going to add and store minimal perfect hashmaps for efficient mute word checking.
Just having these data structures near the note in memory and available to any app without duplicate processing was another motivation.
notedeck is a realtime system, i can’t have it do any work on the render thread. The render path has to be as fast as possible, it has forced me to move all of the computational upfront work into the nostrdb ingester pool.
This results in a smooth scrolling experience because the render thread is just accessing pointers to data that has already been processed.
Anyway, just trying to point out that hyper fixating on one component of the system would be misleading, as it was specifically designed for performance at every level.
If you want to check performance just run notedeck, you will see the results.
Agree, we don't use the render path either. Though I lazy load/compute the additional data structure (like the text parsing) only when the event is being placed in the same page that will be rendered. And those can be discarded individually by the GC after the event is old/not used anymore. Same for decryptions.
Don't get me wrong. I think notedeck is fast. My only concern is "at what cost" (increased disk usage, increased memory usage). Exactly by how much is very unclear right now.
Keep pushing.
> Though I lazy load/compute the additional data structure (like the text parsing) only when the event is being placed in the same page that will be rendered
damus ios works the same way, but it is really annoying, leading to needing to recalculate the view after the parsing was done, which was janky. Now i don’t have to worry about it. If it’s in the db then it has everything it needs to render.
It definitely uses more disk space, but i just need to optimize the db on startup by removing old notes not in web of trust. I think nostur does something like this as well.
“Increased memory usage” is kind of misleading. It uses a lot of memory in the sense the page cache uses a lot of memory, but it does that anyway when reading from disk from anywhere. you can’t compare heap memory to a pagecache-backed virtual memory. They are not really the same thing. it’s more accurate to say it uses a small amount of fixed memory for the subscription queues and thats about it.
There is middle ground too, where nostrdb is used to cache certain kinds (vs everything) and the local relay is used for certain other kinds (vs always)
The main issue with the local relay is that you can't count on it so you need a database anyway
Sync: blocking, async non-blocking. I'm interested in this for a specific API design. Communicating with other Android apps is necessarily async. But I'll think about it.
I see the advantage of using a device relay but I'm unsure of the UX. Are you going to ask your users to download and continually run citrine in order to use your app? It will also depend on the overlap - for Amethyst and Zapstore it's probably none