That type of flag would be excellent. Something you could see in your follow list, privately
Discussion
Literally everyone wants this. Just, like, a negative-list. Editable anti-follow list.
Or just allow private comments on profiles. Maybe you even want to note something positive (This is a great guy and he knows all about hiking trails in the Ozarks.)