Nostr Web Client

A rough draft of a BUD-10 for multi-part uploads to blossom servers. not sure if this is a good idea so I'm looking for feedback

https://github.com/hzrd149/blossom/pull/67

The biggest issue comes from the fact that blossom relies on the sha256 hash of the whole blob. so there isn't a way to verify that each uploaded chunk is part of a whole until the all the chunks are uploaded.

I think this issue can be mitigated slightly if the client includes `x` tags in the auth event for each chunk its uploading. so that the server can verify that each chunk was created by the client.

In the case of payments, I think it could be possible for the client to pay for each chunk. although this might require more requests or the server to define an x sats/bytes pricing.

Inspiration taken from https://tus.io/

Thanks to nostr:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6 for making me aware of it

Reply to this note

Please Login to reply.

Discussion

arkinox 8mo ago

This is necessary for large uploads isn't it? 👍

hzrd149 8mo ago

Yes, I think. Most clients cant keep a long running http requests for uploads, and mobile clients need a way to retry parts when the network fails or times out

arkinox 8mo ago

I'm pretty sure we had problems with uploading audio so yeah that checks out. Nice work

elsat 8mo ago

nostr:npub1ye5ptcxfyyxl5vjvdjar2ua3f0hynkjzpx552mu5snj3qmx5pzjscpknpr would you cap chunks at e.g. 25 MB max?

Shawn 8mo ago

Wasn’t nostr:npub1t89vhkp66hz54kga4n635jwqdc977uc2crnuyddx7maznwfrpupqwra5h9 working on something like this? Hornet?

Colby Serpa 8mo ago

Yes we’re still working. Merkle trees are the proper way to do this, not jerry rigging it.

Brett Phillips 8mo ago

Thanks nostr:nprofile1qqszv6q4uryjzr06xfxxew34wwc5hmjfmfpqn229d72gfegsdn2q3fgprfmhxue69uhhq7tjv9kkjepwve5kzar2v9nzucm0d5hszythwden5te0dehhxarj9emkjmn99u0t7re4 as always. I’m happy to take a look and give some feedback. You should consider the GET side of this using range requests. This would assist in downloading larger files.

Freakoverse 8mo ago

Would it be possible for there to be a client side blob creation first, send/upload that hash ID first, then begin the chunk upload process? (if I understood that that the blob hash is used to check the chunk hash ID)

CanopyKev 8mo ago

I have a working demo at https://canop2p.com just look for the Chunked uploader section. It will chunk to IPLD standard and produce a IPFS compatible CID. It will publish the chunks to a Nostr relay. It also has a visual representation of the chunks and what's already uploaded. So partial chunk resumes/completions already work.

CanopyKev 8mo ago

Oh, and the website above is stored in Blossom as a chunked dag-pb set and served up in realtime from the blossom server.

Freakoverse 8mo ago

Appreciate the progress in this!

Will most likely use this for nostr:nprofile1qy2hwumn8ghj7erfw36x7tnsw43z7un9d3shjqpq7jl3ldd6305rnacvwvchx03snauqsg4nz8mruq0emj9thdpglr2sym646a when the time comes, as a lot of creators will most likely attempt to upload large files on it (hundreds of MBs to double-digit GBs)

Appreciate what nostr:npub1t89vhkp66hz54kga4n635jwqdc977uc2crnuyddx7maznwfrpupqwra5h9 is doing as well on their end.

hzrd149 8mo ago

Depending on how large some of the mods get it might be worth attempting to use torrents. since they are much better for larger files.

IMO the only reason to build on top of blossom instead of torrents is if you know the files are going to be small (cat pictures) and they NEED to work in the web browser. otherwise if you building a desktop app it would probably be much better with torrents

Freakoverse 8mo ago

Makes sense.

I guess I need to consider that down the line then.

I'm basically thinking both now and in the future. And the future being a lot of people having very convenient home servers with Blossom Drive, and start mirror mods and providing their links to it to be discovered, for the mods they like, to support/enhance their availability and censorship resistance.

I guess testing these things out and seeing what comes as the best setup solution