I need to write some multithreaded code and I’m dragging my feet.
Discussion
What language/OS?
Did you get it done?😁
Ruby/Linux. It was pretty simple and I got the code written and tested in about an hour. Just needed to do some simple database updates concurrently to finish a data migration from MySQL -> PostgreSQL in a timely fashion.
Doing them single threaded would have taken ≈24 hours. Doing them in parallel spread over about 170 threads on 64 core servers got it done in about 30 minutes.
Very cool.😁
My cousin used to tell me how much he liked Ruby, but I never tried it...
Yeah, I love Ruby and when I need perf it’s easy enough to write something in C or Rust and call it via FFI. But this code spends the vast majority of its time waiting on the network.
I was able to run the same code single threaded on my M1 Mac against a local copy of the PostgreSQL DB and it only took 1 hour. But adding the network slowed it down by a factor of 24, hence the need for multiple threads.
It’s always easy to forget just how slow even the fastest network is until you run head first into the problem again. 🤣
In my personal playbook, design wins, structure wins, and tool wins *always* trump high-performance hardware wins...
I'm coming from the very early days (ever heard of the Kim-1 single board computer?) when hardware was so limited, we had to optimize for program memory space. It was all ASM (hand-assembled) and unbelievably compact (chess programs in 1.2k of RAM?)...
I own an M1 Macbook Air now, and find it alarming that contemporary coding practices big *it* down!😳