Certainly an option. If I were to build this feature, I would probably start with the most simple option - manual whitelisting, and then feel out via usage what additional filtering might be useful. I do like the idea of manual control, as it puts the user fully in control. But there are probably some limits to that as well.