It’s definitely a risk/benefit analysis one has to do. I let my tool handles dry for a week or more before using them, and that’s after wiping down.
Discussion
Have you considered boiled linseed oil for tool handles? Much nicer and it smells good too! That's what I use.
I have, both the chemical kind and the actually boiled linseed oil, Ballistol, beeswax, beeswax and Ballistol mixed together, don’t like any of them as well as oil and they don’t preserve like the oil does