Yes, that's pretty much my understanding too, as long as I dont become a UK tax resident for atleadt 5 years.

In my circumstances (everyone needs to check their own circumstances) I can visit the UK for upto 90 days / year otherwise risk becoming a UK tax resident again.

I will need get professional advice to make sure plan of action is clear. You scared me when you said we need to pay 24% CGT if leaving the UK tax system.

If we were to leave UK tax system, they can change the rules retroactively. So I'm not sure how they could make life worse for non UK tax residents?

I suppose they could in theory change rules and say we not allowed to hold UK passport. So its essential to have 2nd passport incase this happens. But unlikely for this to happen in the near future.