But the OP_RETURN data is part of the main transaction hash, and by extension the block. You can’t “download a block but not the OP_RETURNs” without losing the ability to verify against doublespends and inflation.

You can however not download witnesses and the only thing you lose is verifiability of the execution path (and some data like HTLC preimages). But this data is not relevant to 99% of people, and even LN node impls, after 3 months.

We can use an assumevalid-type approach to skip witness verification .