nostr:nprofile1qythwumn8ghj7mtvv44h2tnwdaehgu339e3k7mf0qyghwumn8ghj7mn0wd68ytnzv9hxgtcqypxgqqjh5ky2s2zf6pyczlptm2kesje953ddnak66ehy05a50caj75ukn0y here you go: https://github.com/mleku/orly/pull/4
I originally ran the benchmark on v0.4.8, but when I pulled your recent changes from 0.4.14, I saw large performance improvement. Great job!
very nice. merged. i figure this tool can be pointed at other relays for comparison also.
i'm surprised how fast it's saying the events were published tho. if you use the import tool on a trove of events of around 117k, it takes about 15 seconds to swallow the whole lot, which means it's more like 8000 events/s
but maybe you have faster, next generation hardware, my pc has like 3 years ago tech in it. i've used a hetzner server that was about 3-4x faster at everything in my recent previous project with a large database iteration (comparing a set of 1000 records to each other, N(N-1) operations, something like, what was it, 400,000 operations in total. this included standard library JSON decoding. i was going to optimize that stuff with a decoder cache but it was working well enough for processing one row at a time.
What's your hardware? I'm running on an NVIDIA RTX 3060 with i7 CPU and 32 GB RAM.
# System Details Report
---
## Report details
- **Date generated:** 2025-08-05 18:53:35
## Hardware Information:
- **Hardware Model:** Primux Tech Primux_PTIOX
- **Memory:** 64.0 GiB
- **Processor:** AMD Ryzen™ 5 PRO 4650G with Radeon™ Graphics × 12
- **Graphics:** AMD Radeon™ RX 7800 XT
- **Disk Capacity:** 4.0 TB
## Software Information:
- **Firmware Version:** F19c
- **OS Name:** Ubuntu 24.04.2 LTS
- **OS Build:** (null)
- **OS Type:** 64-bit
- **GNOME Version:** 46
- **Windowing System:** X11
- **Kernel Version:** Linux 6.14.0-27-generic
yeah, i underestimated how old it was
came out in 2020. the video card came out end of 2023
obviously if i upgrade my motherboard/memory/cpu i'll probably see a big jump in performance, at least double if not more.
Those benchmarks are just for encoding/decoding in memory. The 8k/s import speed includes database writes, which are way slower than pure codec operations.
nostr:nprofile1qythwumn8ghj7mtvv44h2tnwdaehgu339e3k7mf0qyghwumn8ghj7mn0wd68ytnzv9hxgtcqypxgqqjh5ky2s2zf6pyczlptm2kesje953ddnak66ehy05a50caj75ukn0y if helpful, I can add a note in the benchmark docs clarifying this.
what i was more so interested in was a tool that you run the relay and it hurls events at it and then queries it and times the full operation. such a tester could be pointed at any relay implementation then. certainly then you could add a script that builds and installs khatru and relayer, and probably since the secp256k1 library install script in scripts puts all the C/C++ things in, probably can also do strfry and the rust relay as well, by adding a rustup installer
i might do it myself at some point but feel free to do it if you like. it shouldn't be hard to add this to it.
seeing a side by side comparison of all of the relays would be great for seeing how good performance is but also how good each different relay is in comparison with each other on the same test workloads.
That's interesting... I'll work on this.
nostr:nprofile1qythwumn8ghj7mtvv44h2tnwdaehgu339e3k7mf0qyghwumn8ghj7mn0wd68ytnzv9hxgtcqypxgqqjh5ky2s2zf6pyczlptm2kesje953ddnak66ehy05a50caj75ukn0y I put up an initial PR for this, let me know what you think. https://github.com/mleku/orly/pull/8
i couldn't get it to all run, anyway, as we discussed, when it's all wrapped in a docker it will work for everyone.
now i've removed all those pesty serialization calls in the logging into closure logs, when logging is at default, it will go as fast as it can go.
i'm really curious to see how that works. i'm also curious how it comes to be that relayer is so freakin slow. and lol strfry. really. my goodness it's awful, and it's one of the most commonly deployed relays? wow.
probably lucky we haven't had the massive influx that some keep on complaining about.
ah.
still, that's incredibly fast. i knew they were fast because i already wrote a benchmark for the json and binary codecs and they take tens of microseconds on my hardware.
damn, now i dang want an upgrade of my mobo/ram/cpu
