15-25% advantage on tokens/s

The higher % on big 405B models

The lower % on 70B models

I feel with my set up, I have an extra 3090 card almost, because of them.