Did you digest the math in that note?
The more events a relay contains, the shorter the time in which it need administer those events (the faster it can delete them) and the more it can be a true "relay" (passing things on), rather than an "event store" (saving things).
The idea to have a few gigantic relays storing everyone's notes in perpetuity destroys this efficiency because you end up needing gigantic data centers and that shit is expensive and is an attack vector.
Nobody else can do this.
Doesn't this result in the largest relays acting more like search engine indexers than anything else?
Querying a large relay will be like Googling something; its main job is to take you someplace where the information you're looking for is stored more permanently.
With the mailbox model, yes. Especially profile and relay metadata needs to be easy to find.
You need some relays that you can traverse, to get from one data pool to the next, so they need to hold enough notes long enough, that people can find friends and set up hops with their frens.
And then your WoT is your public square and the squares overlap.
I don't need regular Access to all of the notes being generated in Mongolia or Argentina. I just need a way to connect with my two bros in Mongolia and Argentina, so that I can download their notes to my preferred hosted relays or my cell phone.
Rather than one gigantic data pool (a la Twitter), you have lots of seas, lakes, and ponds connected by rivers and streams. It's slightly slower, but not noticeably so, and the data moving between the different bodies creates some redundancy.
That's what I meant by "proximity to the end user", in the other note.
The end user should think about how he stores his events, just like he thinks about how he stores his word documents, e-mails, or whatnot. He can pay someone to archive them, he can download them, he can have a relay on his PC or a privately hosted relay, etc. Or he can just be like, I don't care about my GM note from Oktober 2022.
Most of the data stored on these gigantic servers is obsolete. People don't care if they delete it, but they feel obliged to save everything and they have your spam folder and the Powerpoint presentation you did 5 years ago copied to 5 different cloud megaservers and mirrored in 15 data centers worldwide and advertisers pay for that.
It's completely stupid and environmentally nuts.
Simple text notes are like daily chit-chat; no one remembers. Notes that get a bunch of interaction are probably worth saving, just like someone will remember an insightful comment made in a conversation.
The more effort that goes into a piece of content, the more likely it should be archived forever. Blog posts, git repos, books, videos, etc. that are intended to be more timeless should stick around.
Yes, if I want to find it, later, long-form is better, for instance.
A good search algorithm, then, needs a way to remember, on a per-npub basis, which relays are part of that npub's "public square."
If you go looking for someone, you start in the town square. If you can't find the fella in the square, you ask around, and people point you to this or that part of town where you might find the person.
It's six degrees of Kevin Bacon but for the internet. You don't need a direct link to everyone, you just need a large enough network that you can get to anyone else in six hops or less. And "large enough" isn't actually that large.
I bet one solution will be for search algorithms to live client-side. It needs to be attuned to each npub's network of connections, in order to more quickly find what that npub is probably looking for. There will be server-side algorithms as well, but those will probably solve a different problem.
The clients will collect mailbox-lists, I suppose, like a WoT of relays.
We also don't actually need to limit the largest relays as much, if we don't use them as de facto archives. Someone with a good hardware setup or a nice cloud contract might have a couple terrabytes of data and a top of the line server to retrieve them.
That's a lot of notes.
