Harassment is an action beyond speech. It's one thing to speak, it's another thing to take a way someone's right to get away from you.

Same with the "shouting fire in a crowded theater". It's not the shouting, it the harm by deception. Same with fraud. It's not the speaking, it's the theft by deception. Defamation, etc.

Some words have consequences, but the words themselves are not the illegal act, just the means.

It's always going to be a sticky distinction, but it's an important one to make.