Global Feed Post Login
Replying to Avatar Laeserin

File sizes have actually been getting steadily bigger, as compression algorithms are more effective on a large document and the sum total of the data set requires less storage and is faster to query. People storing lots of data on one hosted instance, with one entry point, save money, RAM, computation, and human effort, if they keep the files large.

But a distributed, eventually-consistent storage system, like Nostr, has the opposite problem: it benefits for having lots of little files that are spread around, slowly and thinly, and then gathered and assembled, quickly, when used.

Avatar
Daniel Wigton 11mo ago

This is why I harp on format. Small files suffer more from format and metadata overhead.

Reply to this note

Please Login to reply.

Discussion

Avatar
Laeserin 11mo ago

That's why it's best to keep publication metadata only in the index and have the content only in the sections, so that there is less redundancy.

Thread collapsed