CPU Mitigations noticeably affects performance on Linux machines. Booting with mitigations=off greatly improves performance. LUKS also greatly affects Disk performance, even on SSD's. If you combine both of these configurations, you loose a lot of performance
Discussion
You can do luks ssh decrypt with mitigations=off. Performance impact is negligible and mainly at boot.
Regular Disk IO still will suck for servers that need decent performance using Luks. It was noticably faster after I left luks. Even windows bitlocker terribly degrades performance. I never heard of any organization using encryption for server operating system partitions.
VeraCrypt with an encrypted partition was way faster than Luks....and if a server uses debian and btrfs, apt performance is slowwwwww because of know issue with how apt/dpkg writes to disk with.