Literally everyone wants this. Just, like, a negative-list. Editable anti-follow list.
Discussion
Or just allow private comments on profiles. Maybe you even want to note something positive (This is a great guy and he knows all about hiking trails in the Ozarks.)