I do feel bad, they are beautiful creatures if you take time to study them. We had a nest in our grounds (underground) last year. Our village WhatsApp group were encouraging us to destroy it, but we left it alone.
When it comes to the house, though, I have no choice. I can't let them invade our home.