People seem to think that just because you're a peaceful man, that you're harmless.

Those two things are not exclusive to each other; at all.