Global Feed Post Login
Replying to Avatar Vitor Pamplona

Two weeks of hardcore performance work on Quartz, but now we can import 3GB containing 2.1 million contact list events in 9.4 minutes, *single-threaded* on a Pixel 9, using just a cool, calm, and collected 15MB (yes, MB) of RAM.

That's up to 10,000 events per second per CPU for lighter follow lists and 3,000 events per second on heavier ones.

By import, I mean receive, unzip, parse, string intern, verify, save, and index (SQLite) with individual transactions on write-ahead logging to guarantee DB consistency and match real use cases.

JSON parsing w/ string interning is now pretty much capped by IO and interning at 120,000 events per second per CPU or ~50MB/s per CPU.

JSON Parsing + interning + verifying runs at ~20,000 events per second per CPU or ~30MB/s per CPU.

None of these use SIMD instructions, which are still poorly supported on mobile CPUs.

Avatar
lkraider 1w ago 💬 2

I don’t want to have to spend 10 minutes waiting.

Wait, what is Quartz?

Reply to this note

Please Login to reply.

Discussion

Avatar
Vitor Pamplona 1w ago

Kotlin library for nostr

Thread collapsed
Avatar
Vitor Pamplona 1w ago

Also. Don't worry we are not downloading 3GB of events on Amethyst yet :)

Thread collapsed