Subnostr

Yesterday I modded btcd to use a SIMD SHA256 hash function and greatly increased its parallel threads to 3 per physical thread and set to run with 300% normal garbage collector heap allocation.

so, in short terms, it preallocates a lot more memory and it runs triple the normal number of threads (which is set to the number of CPU threads, which is half the number of cores, so that's 4 cores on my lenovo ideapad3 with ryzen 5, meaning 8 threads and 24 parallel goroutines).

the first of these is especially good but the three of them show the performance boost in initial block download and validating the transactions and putting them into the database is way faster:

2023-10-25 07:48:38.126 [INF] SYNC: Processed 11 blocks in the last 10.66s (47083 transactions, height 797983, 2023-07-09 15:53:47 -0100 -01)

2023-10-25 07:48:49.143 [INF] SYNC: Processed 7 blocks in the last 11.01s (20232 transactions, height 797990, 2023-07-09 17:01:00 -0100 -01)

2023-10-25 07:48:59.190 [INF] SYNC: Processed 8 blocks in the last 10.04s (27234 transactions, height 797998, 2023-07-09 18:01:45 -0100 -01)

47k transactions in 10 seconds means it's doing 4700 tx/s, which is nearly 2.5% the vanilla configuration with threads set to limit at standard and GC set to 100.

clearly also the use of the SIMD SHA256 is helping as well.

some more, coming through now:

2023-10-25 07:54:15.237 [INF] SYNC: Processed 10 blocks in the last 10.04s (39948 transactions, height 798219, 2023-07-11 04:35:03 -0100 -01)

2023-10-25 07:54:26.682 [INF] SYNC: Processed 9 blocks in the last 11.44s (37835 transactions, height 798228, 2023-07-11 05:38:01 -0100 -01)

2023-10-25 07:54:36.739 [INF] SYNC: Processed 6 blocks in the last 10.05s (21489 transactions, height 798234, 2023-07-11 06:21:06 -0100 -01)

2023-10-25 07:54:46.868 [INF] SYNC: Processed 7 blocks in the last 10.12s (20932 transactions, height 798241, 2023-07-11 07:22:57 -0100 -01)

as you can see, it's keeping well above 2000tx/s throughput, hitting 4000tx/s often.

this is nearly a doubling of performance. 100% worth it and only took about 60 lines of code to change it this way.

i also had proposed to have part of this (not the SIMD SHA256 library) added to btcd over a year ago.

nobody cares, apparently.

Reply to this note

Please Login to reply.

Discussion

mleku 2y ago

out of curiosity i increased the threads, because as i understand it, the number of threads can increase the parallel execution of searches of chain history for transactions during validation.

i set the threads to 256 this time, and now it is running between 25-30ktx/s

probably worth bumping the memory use too since i have 20gb memory.

mleku 2y ago

haha. oops, forgot to actually enable these things in the configuration, didn't notice the comment in front.

immediately the CPU usage went up to almost double normal just starting up,

well, it sure looks like it's working harder but yeah, 2000tx/s...

not sure i've really gained much over the 24 threads/300% GC heap allocation but i'll leave it that way anyway, since i'm not gonna run it full time, it's a dev node, which i've not got a direct use for at the moment.

mleku 2y ago

2023-10-25 08:05:26.364 [INF] SYNC: Processed 9 blocks in the last 17.17s (29870 transactions, height 798533, 2023-07-13 12:21:29 -0100 -01)

2023-10-25 08:05:38.744 [INF] SYNC: Processed 3 blocks in the last 12.37s (11243 transactions, height 798536, 2023-07-13 12:34:47 -0100 -01)

2023-10-25 08:05:40.443 [INF] SYNC: New valid peer 103.99.170.210:8333 (outbound) (/Satoshi:0.21.2(@wiz)/)

2023-10-25 08:05:40.646 [INF] SYNC: New valid peer 198.244.167.179:8333 (outbound) (/Satoshi:24.0.1/)

2023-10-25 08:05:40.688 [INF] SYNC: Lost peer 198.244.167.179:8333 (outbound)

2023-10-25 08:05:50.350 [INF] SYNC: Processed 6 blocks in the last 11.6s (17808 transactions, height 798542, 2023-07-13 13:44:22 -0100 -01)

2023-10-25 08:06:00.434 [INF] SYNC: Processed 6 blocks in the last 10.08s (18691 transactions, height 798548, 2023-07-13 16:02:05 -0100 -01)

2023-10-25 08:06:11.642 [INF] SYNC: Processed 6 blocks in the last 11.2s (18807 transactions, height 798554, 2023-07-13 16:45:27 -0100 -01)

2023-10-25 08:06:18.577 [INF] SYNC: Lost peer 54.253.15.33:8333 (outbound)

2023-10-25 08:06:22.013 [INF] SYNC: Processed 7 blocks in the last 10.37s (19734 transactions, height 798561, 2023-07-13 18:17:21 -0100 -01)

2023-10-25 08:06:32.756 [INF] SYNC: Processed 6 blocks in the last 10.74s (16995 transactions, height 798567, 2023-07-13 19:13:40 -0100 -01)

2023-10-25 08:06:37.157 [INF] SYNC: New valid peer 70.161.65.159:8333 (outbound) (/Satoshi:22.0.0(FutureBit-Apollo-Node)/)

2023-10-25 08:06:44.691 [INF] SYNC: Processed 6 blocks in the last 11.93s (15897 transactions, height 798573, 2023-07-13 20:09:51 -0100 -01)

2023-10-25 08:06:55.634 [INF] SYNC: Processed 8 blocks in the last 10.94s (24698 transactions, height 798581, 2023-07-13 22:59:08 -0100 -01)

2023-10-25 08:07:05.744 [INF] SYNC: Processed 7 blocks in the last 10.1s (17245 transactions, height 798588, 2023-07-14 00:04:34 -0100 -01)

2023-10-25 08:07:11.855 [INF] SYNC: New valid peer 168.119.68.43:8333 (outbound) (/Satoshi:0.16.3/)

2023-10-25 08:07:11.855 [INF] SYNC: Lost peer 168.119.68.43:8333 (outbound)

2023-10-25 08:07:16.082 [INF] SYNC: Processed 6 blocks in the last 10.33s (13299 transactions, height 798594, 2023-07-14 01:21:14 -0100 -01)

2023-10-25 08:07:26.566 [INF] SYNC: Processed 6 blocks in the last 10.48s (15132 transactions, height 798600, 2023-07-14 02:26:26 -0100 -01)

2023-10-25 08:07:37.472 [INF] SYNC: Processed 8 blocks in the last 10.9s (22857 transactions, height 798608, 2023-07-14 04:01:09 -0100 -01)

2023-10-25 08:07:48.002 [INF] SYNC: Processed 8 blocks in the last 10.53s (28166 transactions, height 798616, 2023-07-14 05:19:14 -0100 -01)

2023-10-25 08:07:51.567 [INF] SYNC: New valid peer 20.29.44.86:8333 (outbound) (/Satoshi:25.0.0/)

2023-10-25 08:07:58.681 [INF] SYNC: Processed 5 blocks in the last 10.67s (15913 transactions, height 798621, 2023-07-14 06:10:17 -0100 -01)

2023-10-25 08:08:09.307 [INF] SYNC: Processed 5 blocks in the last 10.62s (20392 transactions, height 798626, 2023-07-14 06:54:57 -0100 -01)

2023-10-25 08:08:20.591 [INF] SYNC: Processed 7 blocks in the last 11.28s (28490 transactions, height 798633, 2023-07-14 07:45:01 -0100 -01)

still going pretty fast but perhaps it takes time for it to cache up all the chain DB that would mean it will get closer to its peak the longer it runs.