the best thing since sliced bread: https://capnproto.org/
Discussion
best bread: the sliced thing since https://capnproto.org/
no lies detected.
It's a little tricky if you're coordinating with a large ecosystem, but the speed is fantastic. One thing to keep in mind is copying small values out when you're "done" with the full record. Otherwise it's easy for trivial values to keep a huge amount of data in memory, which is a performance drag of its own
Things that blew my mind optimizing some Java: massive per-thread JDBC result sets hanging around in memory, and 250 MB of the strings "Y" and "N" because they weren't interned
So ideally with this, you have small-medium messages and it acts like an in-memory struct but safe and almost as fast.
After you are done processing a message, free it, and it’s done. Usually you would not need to keep one in memory for long, even parts of it (say HTTP request bodies, or logs that get processed and written to a file)
For when you need a subset that is not worth keeping the message around for, copy it, agreed
Performance optimization is an amusing mix of vital and irrelevant, and isn't until you're successful that you find out which. It can pay amazing dividends though: being able to scale an order of magnitude or two without additional hardware and associated administration is 🔥
Depending on the language you're using, there might not be an explicit way to free memory