It’s possible to get a semi-decent result by using Adiantum. Try this on a Raspberry Pi 4:

cryptsetup benchmark -c xchacha20,aes-adiantum-plain64

I think it takes advantage of vector extensions such as NEON. The encryption acceleration stuff coming form the RISC-V folks might be doing something similar.