"rust-analyzer.checkOnSave.allTargets": false

this might help, but there are many options to fine tune, what and when the rust-analyser checks.