You probably can tweak everything into oblivion and break it, but xfce within the usual configuration options should not affect stability IMO. Everything else 🤷‍♂️