Here's what I've collected. I feel like it boils down to this.
For:
1. Users want to degen and they are creating unprunable dust.
2. An OP_RETURN limit removal might incentivise them to use prunable utxos.
3. Unofficial relay alternatives already exist for non standard transactions and may continue to develop, reducing the transparency of bitcoin, (making it harder to identify attempts of real censorship?)
Against:
1. Filters are decentivising degen in general.
2. Removal might increase non-monetary usage of our Bitcoin nodes.
3. This does not prevent spammers from intentionally creating unspendable dust.
4. Non standard transactions has always been a thing, we can add to the standard list or expand the definition of standard when we need, but we don't have them for no reason. Why not make any valid transaction standard?
I put a part of it in brackets because I haven't heard anyone explicitly say this, I've heard "it will make it less decentralized" which is a sweeping statement IMO. But I feel like this is what the devs are pointing out, they're just not saying it. Regardless, it is something to consider.