Or rather, without the shared secret that also unlocks your other DMs to each other.
Maybe decryption with shared secret could be attempted by default unless another secret is present in the link.
But link-specific secret is useful if you want to share it to multiple recipients.
Even if you share the link publicly, the file host won't see the file content unless someone sends them the link. I wonder if file hosts would love or hate that 😄
One consideration is that the file host needs to accept encrypted files, unlike some hosts that only accept valid images and may resize or convert them.