I turned off every microphone permission on every app I don't actually need it in. It gives some peace of mind at least.