Sipa actually talks about the reasons why the size limit on arbitrary data was removed in a mailing list conversation. I have to find it somehow.

I'm talking about the size limit removal that people falsely claim enabled ordinals and inscriptions while in reality, it was always possible previously to add arbitrary data in transactions albeit to a lesser extent.

Maybe listening to the Chaincode podcast episodes he did with Tim Ruffing about Signature aggregation, Schorr sigs, etc. can provide insights as well.

I suspect the reasoning behind it was to enable devs to do 'interesting things' with the Bitcoin script, but what these interesting things are, I haven't taken the time to understand deeply.