This is the exact usecase for assumeutxo. You can use it in Bitcoin Core v29 but it's not the most user friendly experience.