Well it's possible to avoid by only starting to mine after you get the block template but a delay by 2s also means you could lose out on the block reward.
Discussion
Even in a low fee environment, I think it can be much longer than 2sec. There is latency sending msgs between miners<>pool.
I heard wiz and others saying getblockemplate itself was taking >1min on default node configurations in recent fee environment. It runs fast on my MacBook, but haven’t tested my RasPi 🤔