A society needs a shared narrative to function. Absolute freedom of speech is a danger to that (no judgement here, it is what it is). Running down the narrative will cause it to be useless but the chaos will entail. Then the strongest peacemaker will emerge. And it might be "peace" or in other words a very disputable order of things. And freedom of speech would most certainly be gone by then. Therefore I don't think freedom-of-speech lovers are working to their own best interest
But then again. I have a fallacy to prefer known evil (which can be gamed) over not known one