Nostr

It's def cool tech, but I dunno how critical it is.. so far I've only seen it used by crawlers that grab all my relays events and then go confuse users by posting them to relays that they weren't targeted for. And search moguls that don't opensource their code.

Also, it causes massive CPU spikes.

But all in all, still pretty cool just not sure if it's critical. I like organic relays.

mleku 2y ago

it uses a lot of CPU because it's doing a massive amount of hashing of content to ask other strfry relays if they have "this hash" and "that hash".

this is the inefficiency that HORNET Storage is aiming to eliminate.

i dunno if posting content to other relays that "didn't ask for them" is entirely a good strategy but it's because of this promiscuity that you can find all the posts in a thread where users are outside your relay set, but in the set of those in the thread.

it would be better if relays had some system to quickly identify missing content that users are requesting and cache enough of it ahead of time that the user is none the wiser. Negentropy is one effort in this direction, but it is very primitive. HORNET Storage FOREST algorithm reduces the computation and network traffic requirement by the log of the number of accounts involved.

without this nostr would rapidly silo up on the most popular relays who would then either fail due to insufficient capacity or insifficient resources to pay for capacity, and only the big ones would stay operational and provide network data. and then the only thing we have is yet another facebook, yet another twitter. before you know it there will be ads forced on you and KYC, the whole nine yards. it's naive to think that the big guys aren't seeing this as a way to rebrand themselves as "open" with very little resources spent to take it over, and no real difference, since they can apply their algorithms to squash all the "offensive" content, shadowban those users by not relaying their posts, and colluding with other big relays to maintain a unified front, and keep their revenue stream from government and corporations.

this is the problem that we are warning people about that nobody seems to want to acknowledge who are supposedly the "wizards of nostr". anyone with a fragment of knowledge about distributed systems database state replication can tell you this is a looming problem.

it would be better if it was driven by an efficient algorithm that identifies relevant users content and could efficiently acquire the data after it has been requested, to proactively get more of that content from nearby it in the graph.

this will struggle to happen while everyone's watching the big swinging dicks at the top of the trending graph and paying no attention to the gradual attrition of small relays on the margin as the data set gets beyond the resources of volunteers.

i dunno how to explain it any more concisely, but due to what i see going on in the protocol and the way the social control is trumping the market needs, someone needs to start a call for more work to be done on building a scalable consensus.

personal relays really are just caches, they are an important measure in the efforts but also the easiest part to solve. cache syncing, data reference schemes, and ancilliary content hosting are all bigger, much harder problems to solve that are rapidly converging on cloud solutions and you know where that leads.

Reply to this note

Please Login to reply.

Discussion

cloud fodder 2y ago

Hornet does sound cool.

My question is this though.. why not gossip model? The thing that irks me right now about relays syncing, is that clients don't know the event went somewhere else. Someone sees the event on another relay and replies, the OP will not see the reply.

In a multi reply gossip scenario, your client should be posting the replies to the inbox of everyone being tagged...then relay syncing isn't necessary.

What u think? Is there a flaw in that you can see?

mleku 2y ago

the gossip model will have such a large amount of traffic it will use up relays VPS bandwidth allocations quickly and the delay with which the network synchronise would get longer with the log (2) of the number of users posting to it. keep in mind that this outbound traffic will also compete with users posting traffic as well.

even if you make that more efficient with a kademlia DHT based partition resistant broadcast, the thing is that relay users are not necessarily going to request all the content that is sent to them in this manner, and this is going to consume resources needlessly and ultimately slow down the synchrony of the network (time between publishing and all readers getting their subscriptions delivered), as well.

FOREST enables a publish/subscribe model to efficiently enumerate and request demanded notes from other relays, and not just notes, it can enable also media and other files, and not just the whole files, but pieces of them.

IPFS could achieve the same thing but its merkle trees are massive and people would jam them so full , and then it would also have the very same broadcast propagation inefficiency, and the same high cost of making missing piece requests.

the rockstar devs who think they are gods at the top of the trending feeds think that FOREST is the same as IPFS, they literally said as much and this is why they are shitting all over Robin and Colby's work, even resorting to straight up harassment and trolling.

it's not. it enables publish-subscribe efficiency without centralisation. think like fully open Kafka or Dynamo.

as a cloud dev you would be familiar with these things and why achieving them in a decentralised way is a big deal for Nostr.

cloud fodder 2y ago

🤔

Ok so my simple brain is trying to think through some gossiping..

Client sees an event. Client queries the inbox(s) of the event publisher, for all the replies. Done.

You could query more relays, perhaps you want to crawl the tree and do this for every reply event too, grab the replies from all the inboxes that you can find. If all clients were using gossip, then you would not find any additional replies *unless these replies were getting nerf'd by the OP's inbox relay. If this is the case, then you could confirm this by what, crawling all relays in existence? If someone notices their replies are rejected for whatever reason, and they really really want to troll the OP or followers of the thread with their uncensored opinion, they could boost/quote it into the current relay or another relay as a top post. Or, since they found the event on 'current relay' they can simply reply on 'current relay' and whoever sees it sees it. This would be no worse than what happens without gossip.

🤔

I will think this over more in the AM with more braincells.. I am truly curious about all this stuff, I assumed clients would use gossip eventually, pretty much anything would be better than what it is right now. (The bandwidth used by the flagship mobile clients is already enormous, relays already using way too much bandwidth, I don't think it could be less efficient if it tried).

And with that, my phone battery is 1% from dead. gn sir! Thanks for the info.

mleku 2y ago

hehe, gn!

you'll get it that an effective indexing strategy is the key after your brain does its night time thing.