Avatar
karl Black
f96ff2d9688602ef172ac050d4a426bdef384e2b2012fe6ff21e3ae017f30ee5
bitcoin node runner. small-time home miner. drinker of stouts and scotch. builder of (semi)worthless web applications. player of video games. engineer of electromagnetics

I bought Mastering Lightning by Andreas after getting about a third through Mastering Bitcoin.

What do you all think of the Lightning book?

A good Guinness and the good boy

# Implementing The Gossip Model

version 2 (2024-03-23)

## Introduction

### History

The gossip model is a general concept that allows clients to dynamically follow the content of people, without specifying which relay. The clients have to figure out where each person puts their content.

Before NIP-65, the gossip client did this in multiple ways:

1. Checking kind-3 contents, which had relay lists for configuring some clients (originally Astral and Damus), and recognizing that wherever they were writing our client could read from.

2. NIP-05 specifying a list of relays in the `nostr.json` file. I added this to NIP-35 which got merged down into NIP-05.

3. Recommended relay URLs that are found in 'p' tags

4. Users manually making the association

5. History of where events happen to have been found. Whenever an event came in, we associated the author with the relay.

Each of these associations were given a score (recommended relay urls are 3rd party info so they got a low score).

Later, NIP-65 made a new kind of relay list where someone could advertise to others which relays they use. The flag "write" is now called an OUTBOX, and the flag "read" is now called an INBOX.

The idea of inboxes came about during the development of NIP-65. They are a way to send an event to a person to make sure they get it... because putting it on your own OUTBOX doesn't guarantee they will read it -- they may not follow you.

The *outbox model* is the use of NIP-65. It is a subset of the *gossip model* which uses every other resource at it's disposal.

### Rationale

The gossip model keeps nostr decentralized. If all the (major) clients were using it, people could spin up small relays for both INBOX and OUTBOX and still be fully connected, have their posts read, and get replies and DMs. This is not to say that many people *should* spin up small relays. But the task of being decentralized necessitates that people *must be able* to spin up their own relay in case everybody else is censoring them. We must *make it possible*. In reality, congregating around 30 or so popular relays as we do today is not a problem. Not until somebody becomes very unpopular with bitcoiners (it will probably be a shitcoiner), and then that person is going to need to leave those popular relays and that person shouldn't lose their followers or connectivity in any way when they do.

A lot more rationale has been discussed elsewhere and right now I want to move on to implementation advice.

## Implementation Advice

### Read NIP-65

[NIP-65](https://github.com/nostr-protocol/nips/blob/master/65.md) will contain great advice on which relays to consult for which purposes. This post does not supersede NIP-65. NIP-65 may be getting some smallish changes, mostly the addition of a private inbox for DMs, but also changes to whether you should read or write to just some or all of a set of relays.

### How often to fetch kind-10002 relay lists for someone

This is up to you. Refreshing them every hour seems reasonable to me. Keeping track of when you last checked so you can check again every hour is a good idea.

### Where to fetch events from

If your user follows another user (call them jack), then you should fetch jack's events from jack's OUTBOX relays. I think it's a good idea to use 2 of those relays. If one of those choices fails (errors), then keep trying until you get 2 of them that worked. This gives some redundancy in case one of them is censoring. You can bump that number up to 3 or 4, but more than that is probably just wasting bandwidth.

To find events tagging your user, look in your user's INBOX relays for those. In this case, look into *all* of them because some clients will only write to *some* of them (even though that is no longer advised).

### Picking relays dynamically

Since your user follows many other users, it is very useful to find a small subset of all of their OUTBOX relays that cover everybody followed. I wrote some code to do this as (it is used by gossip) that you can look at for [an example](https://github.com/mikedilger/gossip-relay-picker).

### Where to post events to

Post all events (except DMs) to *all* of your users OUTBOX relays. Also post the events to *all* the INBOX relays of anybody that was tagged or mentioned in the contents in a nostr bech32 link (if desired). That way all these mentioned people are aware of the reply (or quote or repost).

DMs should be posted only to INBOX relays (in the future, to PRIVATE INBOX relays). You should post it to your own INBOX relays also, because you'll want a record of the conversation. In this way, you can see all your DMs inbound and outbound at your INBOX relay.

### Where to publish your user's kind-10002 event to

This event was designed to be small and not require moderation, plus it is replaceable so there is only one per user. For this reason, at the moment, just spread it around to lots of relays especially the most popular relays.

For example, the gossip client automatically determines which relays to publish to based on whether they seem to be working (several hundred) and does so in batches of 10.

### How to find replies

If all clients used the gossip model, you could find all the replies to any post in the author's INBOX relays for any event with an 'e' tag tagging the event you want replies to... because gossip model clients will publish them there.

But given the non-gossip-model clients, you should also look where the event was seen and look on those relays too.

### Clobbering issues

Please read your users kind 10002 event before clobbering it. You should look many places to make sure you didn't miss the newest one.

If the old relay list had tags you don't understand (e.g. neither "read" nor "write"), then preserve them.

### How users should pick relays

Today, nostr relays are not uniform. They have all kinds of different rule-sets and purposes. We severely lack a way to advice non-technical users as to which relays make good OUTBOX relays and which ones make good INBOX relays. But you are a dev, you can figure that out pretty well. For example, INBOX relays must accept notes from *anyone* meaning they can't be paid-subscription relays.

### Bandwidth isn't a big issue

The outbox model doesn't require excessive bandwidth when done right. You shouldn't be downloading the same note many times... only 2-4 times depending on the level of redundancy your user wants.

Downloading 1000 events from 100 relays is in theory the same amount of data as downloading 1000 events from 1 relay.

But in practice, due to redundancy concerns, you will end up downloading 2000-3000 events from those 100 relays instead of just the 1000 you would in a single relay situation. Remember, per person followed, you will only ask for their events from 2-4 relays, not from all 100 relays!!!

Also in practice, the cost of opening and maintaining 100 network connections is more than the cost of opening and maintaining just 1. But this isn't usually a big deal unless...

### Crypto overhead on Low-Power Clients

Verifying Schnorr signatures in the secp256k1 cryptosystem is not cheap. Setting up SSL key exchange is not cheap either. But most clients will do a lot more event signature validations than they will SSL setups.

For this reason, connecting to 50-100 relays is NOT hugely expensive for clients that are already verifying event signatures, as the number of events far surpasses the number of relay connections.

But for low-power clients that can't do event signature verification, there is a case for them not doing a lot of SSL setups either. Those clients would benefit from a different architecture, where half of the client was on a more powerful machine acting as a proxy for the low-power half of the client. These halves need to trust each other, so perhaps this isn't a good architecture for a business relationship, but I don't know what else to say about the low-power client situation.

### Unsafe relays

Some people complain that the outbox model directs their client to relays that their user has not approved. I don't think it is a big deal, as such users can use VPNs or Tor if they need privacy. But for such users that still have concerns, they may wish to use clients that give them control over this. As a client developer you can choose whether to offer this feature or not.

The gossip client allows users to require whitelisting for connecting to new relays and for AUTHing to relays.

## See Also

[The Gossip Model: Outstanding Issues](https://habla.news/u/mike@mikedilger.com/gossip-model-outstanding-issues)

Wait how did you do this? What client?

This guy loves a blanket like no other dog I’ve met

We’re doing this or what, anon?

#Zapathon nostr:note164s0qw78hpkah3x97vpp0y5374rp8kv45s7dwe28zmpmnstnw78sl6edqu

An overly complicated system to pump water from under my father’s house out to the street 😂. The crawlspace will flood without the pump in the first picture. It pumps up and into a channel that hits my retention system. Still need to put vapor barrier down in some spots and insulate the walls again.

I installed the 600 gallon retention system, but the clay soil doesn’t allow the rain water to dissipate off fast enough if we have consistent rain. So I picked the low spot and installed a 1hp sump to pump it up to the street. Still need to dig the trench, run actual pipe instead of the blue hose, and cut through the curb to finish it up.

It’s been…fun. The city has not made my job easy.

I really like the idea of this implementation. Might it slow down adoption of Nostr though? Maybe that’s the point 🤔

Sump pump installed and stormwater management system upgrades in progress

GM - plan for today:

> install new sump pump in crawlspace so it doesn’t flood again

> work fiat job from home

> leg day at the gym

> continue fleshing out design of MPPT solar charge controller for bitaxe

Thanks for the pointer! Yeah I don’t think the AC-DC conversion will be a challenge. Pretty standard.

Just need to decide on the battery chemistry (I’m thinking LiFePO), design the solar charge controller, step down the DC to 5VDC while allowing for ~ 2A draw.

It’s going to be a fun project.

Replying to Avatar Derek Ross

We've had a lot of talk about relays over the past day here on Nostr. This may be a bit confusing if you're new to Nostr.

What's going on with this discussion and why are many people making comments, jokes, or memes about relays?

The argument boils down to centralization and decentralization. On Nostr, we decentralize our infrastructure by using a potential wide variety of servers. Many of us are reading this note right now that's been retrieved from a variety of servers or as we call them, relays. Some relays are more popular than others. This leads to the questions surrounding decentralization. If many users are only using a top subset of relays, then we're not as decentralized as we could be or maybe even as decentralized as we should be.

I have various opinions on this subject. Even if Nostriches mostly only use ten relays, we're still magnitudes better than any existing social platform where only one entity runs the server infrastructure. We can do better though and we should do better. We have the tools to and the capabilities to make Nostr even more decentralized.

Now, this gets us into the next discussion. Which method of decentralization is best? For that, again, in my opinion, I'm a fan of the Gossip model. Simple put, Gossip makes relay selection not overly matter all that much. You can use any relay that you'd like and your follower's clients essentially fetch the notes and events from the relays that are being used without the need to be utilizing the same relay for communication to flow.

What should you do? Nothing, IMO. Relay selection and discovery aren't that great on most clients. I'd say keep using the same relays that you're using for now. Maybe, at some point, you'll update your relays to use smaller, community relays. For now, this is a developer problem to solve. You just keep creating notes and sending zaps. It will all work out in the end. Our developers LOVE to have problems to solve. Pura vida!

Community focused relays sounds like a fantastic idea. Host deeper conversations on specialized topics. Retain the ability to write to larger relays if you desire.

That’s the plan.

I also want to develop an automatic transfer switch so it can switch over to 120VAC in the event there is no power from the solar panel and the battery pack is depleted. Need to make it so power is not lost in the transition from DC to AC.

GM - plan for the day:

> take dad to 👁️ appointment

> fiat job this morning and later evening (I’ve got a deadline to make)

> hack away on bitaxe solar PSU. Had to break this guy out for a review of DC-DC converters

success on getting the #bitaxe Swarm up and running!

with the Ultra and Supra running I'm coming in at a little over 1 Th/s and around 24 Watts.

Successfully self-hosting these on Public Pool

🙏

unfortunately this usually translates into me getting less sleep that I need…

GM - plan for the day:

> work fiat job this morning

> set up my #bitaxe Supra this evening!

> Figure out the Swarm functionality in AxeOS (hopefully nostr:npub1vwf2mytkyk22x2gcmr9d7ktprakh6llwpzxqlke8rlv5j0qyx2esf2lxtw has some content on that)

> Learn more about using coldcard mk4 in air gapped config

Tried a ZYN for the first time today. Definitely not my thing. Never did chewing tobacco or anything like that before. Smoke a cigar every once in a while; about one per year.

Felt a little light headed and nauseated within five minutes. Then dialed in cognitively. Kind of reminds me of adderall in that regard. Won’t be doing it again.