It works really well for files under 100KB. We are waiting for new Relay codebases to better manage large blobs. I think this is inevitable. But it doesn't work well on any of the current relay implementations.
What ever happened to these discussions around storing files directly on relays (those who choose to accept file upload events)?
nostr:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6 nostr:npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s nostr:npub1gcxzte5zlkncx26j68ez60fzkvtkm9e0vrwdcvsjakxf9mu9qewqlfnj5z nostr:npub1acg6thl5psv62405rljzkj8spesceyfz2c32udakc2ak0dmvfeyse9p35c nostr:npub1l56z85xvaplh3033l8h2x9thypamu0d2hyqnxkw0ujuntqm8asaqh99sza nostr:npub12262qa4uhw7u8gdwlgmntqtv7aye8vdcmvszkqwgs0zchel6mz7s6cgrkj nostr:npub1skfmmcet2244yx8xn5xjfcmwg3u4mcjdndpd2gcjkq4qfycm6xtql0xa7d nostr:npub1jlrs53pkdfjnts29kveljul2sm0actt6n8dxrrzqcersttvcuv3qdjynqn
Discussion
I’ve got an experimental implementation of multipart nip-95 files working. Hope to cleans it up and have it on my server for folks to check out this week.
Nice! Make it super optimized for blobs and people will pay attention. We can have specialized relay implementations (Like a Preferred relay list for blobs) to be used only with these event types if necessary.
It’s not yet super optimized. Making sure the rules around accepting such messages and reassembling them are working well first.
Optimization of the implementation itself can come in a later iteration, without changing the public interface.
Plus, any implementation details there will probably be specific to the relay implementation. Happy to optimize mine to be an example to follow, but I don’t imagine it’ll just be a copy/paste to add it to other relays.
The ideas I have for optimization are basically: deconstruct the kind 1064 JSON messages so you can store the blob as binary instead of base64. (This is done transparently on the server. It still has to reconstruct and serve text JSON events when they’re requested, since Nostr protocol requires it.)
You can do that at the event level and save some space. And it’s the simpler implementation. But, if you expect people might re-post the same content, an implementation that can reassemble the full file and store that in a content-addressable store would be even better. But that gets more complicated with multipart files. (Can’t decode the whole blob and verify and store it until all messages are present.)
Alright, #nostr / #nostrdev folks, I've got my implementation of multipart NIP-95 files up here:
https://github.com/NfNitLoop/yastr/blob/main/src/server/nip95.rs
(Yes, it's very messy. I just saw all the TODOs which I've already done. 😆)
And I've got a little tool to upload multi-part files here: https://github.com/nfnitloop/nostr-cli/ (The `nt upload` command.)
Since clients might not know how to put together multipart files yet, my relay (wss://www.nfnitloop.com/nostr/) also makes them available via HTTP. For example:
There is room for space-saving in the storage implementation. If this takes off I'd want to store the BLOBs in binary instead of Base64. And I'd probably want to dedupe whole files so that we don't end up having to store multiple copies of them if multiple people upload them. BUT, that can come later without changing the public interface.
Before *that*, I want to implement HTTP range requests to show that the blockSize metadata allows a client or server to efficiently fetch bytes from a known offset. That way you could, for example, scrub to a certain position in a video without having to download all the bytes up to that point.
And now I’ve added HTTP Range support.
❤️ to the axum-range crate which made this much simpler. (Though I did have to delve deeper into Rust async than I have before. But I was happy to learn more there.)
Here’s a sample video which I saw making the rounds on Mastodon earlier this week:
https://nfnitloop.com/nostr/files/fb8e82bed22cf9c8ee39ef2898970beee6fec7ca9068e1506a061f22f5ec1ae7/
Note that only the first part of the video is fetched. You can jump past that and the server will start loading from the exact point you jump to.
The algorithm the server uses to answer these range requests could also be implemented client-side. (Say, as the Blob interface in the browser.) I’m just not implementing a client. (Yet. 😉)