They're predators that attack other pests. but, I understand, they have to go.
DE (diamatecous earth), dusted around the entrance will be effective, but it will kill any insects, including beneficial bees & butterflies that come in contact.
#smij doesn't like man-made pesticides. they poison everything