Avatar
So Tachi
6a72db8ef3f3b9ee5ecd808ed6d0631d1e4dda5c5dadf07887104d33957eba48
building nostr:nprofile1qqs8wfkyxlx00y0kmmvhmwkps3hxyqv7t77jfapwnke0vs8jx8pupxsx8yfv2

the fist million is always the most difficult

So pumped to see nprofile1qqs8wfkyxlx00y0kmmvhmwkps3hxyqv7t77jfapwnke0vs8jx8pupxsx8yfv2 in the list

The logo needs a rework

Waking up a snowy morning to this 100k milestone and seeing the memes out there in Nostr feels like Christmas

Replying to Avatar hodlbod

# #lastword

A few weeks ago, Mike proposed the addition of a feature to nostr. The content of the proposal itself isn't important, but the resulting conversation illustrated something important about nostr development that I wanted to draw attention to.

If you're interested, you can find the issue [here](https://github.com/nostr-protocol/nips/issues/1204). The idea was basically a tag that disabled comments to a reply, for when you wanted to gracefully exit a conversation that had outlived its usefulness.

While I definitely sympathize with the experience of getting stuck in an unproductive argument and being unable to leave because you have to have the last word, I do think it's better to take responsibility for leaving the conversation, rather than make other people do it for you. You can either outlast your opponent, let them have the last word, or tell them "I don't want to talk about this any more, I'm not going to reply".

This is just my opinion, and it's really not important whether I'm right or not. What was interesting was how Vitor responded:

> I am not sure if the NIP review process should consider "what's good for the user" in the discussion. That kind of nanny state thinking is what went wrong with regular social media in the first place.

# Permaculture and Ethics

What it sounds like (although I have a hard time believing this is actually his position), is that Vitor is dismissing the relevance of an ethical framework in designing a protocol, preferring to stick to the mechanics of what is being suggested. As Vitor says, "Clients can do whatever they want, of course." This is true, but ignores the question of why a client developer might want to do any particular thing.

In a recent [Thank God for Nostr episode](https://fountain.fm/episode/30uEXC25615Ze2ELjY2p), I interviewed Scott Mann of the [Permaculture Podcast](https://thepermaculturepodcast.com/). When I asked him his opinion on how to manage decentralized protocol design and build effective consensus, here's what he had to say:

> [You] don't need to frame it as a software development project, or even a protocol. I would look at it as a distributed community-based and community-supported project, whatever that is. Because permaculture is such a large umbrella, I like to go up to that 50,000 foot view and pull away from what the details are. Because the details are what we're going to build our solutions _from_.

>

> And that's one of the things I didn't mention earlier — there's kind of a hierarchy within permaculture that goes, at the top are the ethics of permaculture: earth care, people care, fair share. Beneath that are the principles, then usually we have strategies and techniques.

>

> But there's a dividing line between ethics and principles and the strategies and techniques, that we start at that top, and use the ethics to decide whether or not we're even going to launch a project.

He goes on to say:

> I'm going to use the principles and see how can I apply those principles to my research and original design. To make sure that whatever I'm creating creates some kind of a surplus, to have a refinement process in place before I even launch, like what is that going to look like, even if I have to change it later, just having some of these building blocks in place.

>

> And then once I've done that and have gone deep into my research into what this might look like, how I might launch it, that's where I would start going into strategies and going ok, how do I want to market this? How do I want to get this out to the people who are going to use it, how do I want to maintain this, how do I want to do distributed decision making, and then as I start to think about distributed decision making, looking at what is the form that I want to use for that?

Scott's thesis is that ethics and principles should be in the front of your mind both as you're considering a project, and as you continue to build it out. This not only makes sense as something to do if you want to succeed, it's categorically true. Action can't happen without agency, and it is your agency that informs what you choose to do, and how you plan to go about it.

The word "ethics" comes from the Greek _ēthos_, meaning "moral character". In other words, your ethic is _who you are_. Your values, hopes, preferences, faults all factor in to your actions.

This is not always clear, because in fiat-world, many people suspend their values in order to "get something done". If you want to protect your savings, you invest it in index funds, propping up the stock price of companies that hate you. If you want to make money, you go work at a job where you're berated quarterly for being racist. People think that they can exercise their own values in their private life, while actively undermining those values with the majority of their time and purchasing power.

But this is not how people with integrity act. And I think if you can say one thing about nostr — both its developers and the community at large — is that they have a very high level of integrity. In other words, their actions are clearly informed by their _ēthos_.

This is a very good thing. What is the point of building an entirely new internet if we're not going to impose our values upon it? What an absolute waste of effort.

# A Nostr Manifesto

This is not to say that any one developer has the right to imposing his own vision on the protocol because of his own personal values and reasons for contributing, which is what I think Vitor was being cautious about. But there are lines I think we can draw as a community that can't really be crossed without excluding yourself from what I might call the "nostr group ethic".

So what are those boundaries? What _is_ a nostrich? Here are some values I've observed to be generally shared among nostr developers and users. Not everyone would full agree with these (including myself), but I think they're a fair characterization of the community.

1. Free speech absolutism. No central entity should be able to globally censor any content. This comes with the trade-off of objectively evil content continuing to exist. This trade-off is acceptable, both because of the value of free speech, but also because evil will continue to exist regardless of attempts to suppress it.

2. Empower individuals over institutions. No centralized entity can be trusted to safeguard the interests of the individual. Institutional incentives are asymmetric and easily corrupted. Better to have many subjective views of the world, than a single, centrally managed view of the world.

3. Advertising-based business models should be viewed with great skepticism and caution. Advertising is a system of incentives that is central to the institutional corruption we see around us. Broadly, this includes paid ads, monetization of engagement, public/private partnerships, and "crypto".

There are others that are shared by many within nostr, although not as widely agreed upon. Two I can think of are:

- Economic activity should be voluntary. Software and content should be free, and creators should be amply rewarded via zaps or other value-for-value models.

- Social media should support real life community and relationships, not detract from them. We should all take time to touch grass.

I'm personally skeptical of the first of these, and strongly in support of the second. Much of my energy as a nostr developer has gone into attempting to subvert and reform traditional patterns of social media to not only support, but also resemble relationships that exist in the analog world God made, and placed us in.

This particular principle is the one at play in the conversation I linked to at the top of this post. My comments weren't an accusation that anyone was acting "unethically" in a universal sense, only that the proposed feature was incompatible with my vision for what nostr should be.

But of course, my vision is not shared by everyone, and the principle of "support real life" is clearly subservient to core ethic #2, which admits the value of a diverse set of opinions about the world. I have no right (or ability) to invalidate anyone else's core principles. But by the same token I'm free to express my own, and attempt to convince other people to share them. This is the basic value proposition of freedom of speech itself.

# Ethical Cohesion

I would go further, and say that not only is it permissible to talk about ethical reasons for building one thing or another on nostr, it's essential. By having these conversations we fuse our individual ethics into a shared ethic. By calibrating our moral compasses to point in (roughly) the same direction, we also decrease the friction involved in getting something done.

I think this was a significant part of the idea behind Sovereign Engineering — get a bunch of people in a room together sharing meals and going on hikes, and the work will accelerate! This is also the way a church works. By meeting weekly together we strengthen our shared identity and build one another up through our activity. In fact, this is the basic definition of a community as Scott Mann puts it. In his words, a community can provide:

> a series of connections, and a knowledge base, and a skillset that we can't fulfill as an individual, while having a social relationship with people in such a way that we can call on them for help.

So maybe, as I've said before, the real protocol is the friends we made along the way. Disagreement and discussion is a healthy thing for a community to have, and we should never stop asking "why?"

Interesting, this subject naturally brings me to how bitcoin is being developed. The protocol is definitely ossifying more and more over time.

Bitcoin and Nostr are both protocols serving different needs, one is money, the other is information, or speech.

The real world analogy for Bitcoin would be gold, which is a very physically and chemically stable metal, hence its successful use as money.

What would be the real world analogy for Nostr? Perhaps politics.

Unlike Bitcoin, there is no single direction for Nostr, nor a concept of a hard fork.

NIPs might increasingly resemble a legal framework. But one were you can ignore the part that you disagree with, not change them.

Replying to Avatar hodlbod

Nostr is a mess. It always has been and will always be. That's part of the appeal! But it's important that users be able to navigate the rolling seas of this highly partition-tolerant network of kaleidoscopically-interwoven people, bots, topics, relays, clients, events, recommendations, lists, feeds, micro-apps, macro-apps, Chinese spam, and "GM"s.

In order to do this, users must be able to articulate "what" they are looking for, and clients must be able to articulate "how" to find that thing. This "how" is divided into two parts: building a request that will match the desired content (very easy), and selecting a relay that is able to serve that content to the user requesting it (very very hard).

# Why guessing isn't good enough

As a concrete example, let's say the user wants to find everyone in their "network" who is using a particular topic. The process would look something like this:

1. The user clicks the "network" tab and types in the topic they want to browse. This is the "what".

2. The client then translates the term "network" to a list of public keys using whatever definition they prefer (Follows? WoT? Grapevine?), and builds a filter that might look something like this: `[{"authors": pubkeys, "#t": ["mytopic"]}]`. Any relay will happily accept, understand, and respond to that filter.

3. The client then has to decide which relays it should send that filter to. This is the `???` stage of the outbox model, which immediately precedes:

4. Profit

It may not be immediately obvious why selecting the correct relays might be difficult. Most people post to relay.damus.io, and most people read from relay.damus.io, so in most cases you should be good, right?

This approach to relay selection has historically worked "well enough", but it depends on a flawed definition of success. If you only want to find 90% of the content that matches your query, using the top 10 relays will suffice. But nostr is intended to be censorship-resistant. What if those 10 hubs have banned a particular public key? Nostr clients should (at least in theory) be 100% successful in retrieving requested content. Even if someone only posts to their self-hosted relay, you should be able to find their notes if their account is set up properly.

# A naive solution to fixing the FOMO

A 90% hit rate results in a feeling of flakiness, even if users aren't completely aware of what isn't working. Feeds will be incomplete, quoted notes will be missing, replies will be orphaned, user profiles won't load. The natural response to the FOMO this creates is for users to "try harder" by adding more relays.

On the read side, this means clients open more connections, resulting in much higher data transfer requirements, with massively diminishing returns, since there's no reason to expect that a randomly chosen relay will have a substantially different data set.

One the publish side, this means that clients end up publishing more copies of their data to more relays. This approach has been automated in the past by services like Blastr, which don't store a copy of events published to the relay, but instead forward events to the top 300 relays in the network. This results in a two-orders-of-magnitude increase in storage required, and only makes the read side of the problem worse, since it reduces the uniqueness of the data set each relay stores. This in turn means that more duplicates are retrieved when querying relays.

Both halves of this approach are equivalent to guessing. On the read side, users are guessing which relays will have any arbitrary content they might ask for in the future. On the write side, users are guessing which relays other people might use to find their notes. It is a brute-force method for finding content.

# Randomness results in centralization

In theory, random relay selection would result in a perfect distribution of content across all relays in the network. But in practice, this method of selection isn't random at all, but is strongly influenced by user bias in what constitutes a "good" relay. While some users may check [nostr.watch](https://nostr.watch) for ping times, geographical proximity, or uptime, most will choose relays based on familiar names or other people's recommendations.

In either case, these biases are entirely orthogonal to achieving a higher content retrieval hit rate, _except when bias in relay selection results in clustering_ — i.e., centralization. In other words, the kind of randomness exhibited by users when selecting relays actually results in pretty much everyone picking the same few relays. We see this same effect when people try to come up with passwords or seed phrases — human-provided randomness is anything but random.

Clustering improves the hit rate when requesting events (slightly), but it results in nearly as much centralization as if only a single relay was used — and a lot more duplicate events.

# Something (anything) other than randomness

In early 2023, Mike Dilger [introduced NIP 65](https://github.com/nostr-protocol/nips/pull/218) (now known as the "Outbox Model") with a problem statement in the spirit of the original description of nostr: "Nostr should scale better. People should be able to find what they want."

_Historical note: NIP 65 was formerly known as the "Gossip Model", derived from the name of Mike's [desktop nostr client](https://github.com/mikedilger/gossip), called "Gossip". This unfortunately created a lot of confusion, since [gossip protocols](https://en.wikipedia.org/wiki/Gossip_protocol) work very differently from how nostr tends to work, hence the re-brand._

Before NIP 65, an informal standard existed in which `kind 3` user contact lists also included a list of relays that clients could use as something similar to Mastodon's "home servers". This list included the option to only read or write from a given relay. Unfortunately, it wasn't really clear what the semantics of this relay list were, so different clients handled them differently (and many clients ignored them). Usually this amounted to user-provided static relay configurations, which resulted in the naive relay selection approach described above.

NIP 65 used a very similar format (a list of relay urls with optional "read" or "write" directives), but with a very important semantic difference: relays listed in a user's `kind 10002` were intended to "advertise to others, not for configuring one's client." In other words, these relay selections were intended as a signal to other users that they should use certain relays when attempting to communicate with the author of the relay list.

I highly recommend reading the [entire NIP](https://github.com/nostr-protocol/nips/blob/master/65.md), which is very short and easy to read. But the mechanics of the spec are very simple:

> When seeking events **from** a user, Clients SHOULD use the WRITE relays of the user's `kind:10002`.

>

> When seeking events **about** a user, where the user was tagged, Clients SHOULD use the READ relays of the user's `kind:10002`.

>

> When broadcasting an event, Clients SHOULD:

>

> - Broadcast the event to the WRITE relays of the author

> - Broadcast the event to all READ relays of each tagged user

For the first time, we had a way to differentiate relays in terms of _what content could be found where_.

When looking for a note by a particular user, a client could now look up the author's `write` relays according to their `kind 10002` event, and send its query there. The result is a much higher hit rate with much lower data transfer requirements, and fewer connections per query.

# Making Outbox Work

There are of course some assumptions required to make this work.

First, the user must know which author they're looking for. This isn't always true when looking up a quote or parent note, but context and [pubkey hints](https://github.com/nostr-protocol/nips/pull/1171) solve this difficulty in most cases.

The author must also publish a `kind 10002` event. This may not always be the case, but clients should prompt users to set up their relay list correctly. This isn't really a flaw in the Outbox Model, just in implementations of it.

Additionally, the user's client must be able to find the author's `kind 10002` event. This is the "bootstrapping" phase of the Outbox Model, during which the mechanisms the system provides for finding events aren't available. This requires us to fall back to randomly guessing which relays have the content we're looking for, which as we saw above doesn't work very well.

Other than guessing, there are a few different ways a client might find the relay selection event in question, each of which is applicable in different circumstances. In most cases, using one of a handful of indexer relays like [purplepag.es](wss://purplepag.es) or [relay.nostr.band](wss://relay.nostr.band) is a simple and efficient way to find user profiles and relay selections.

However, if an author's content has been aggressively purged from these indexers due to censorship, they obviously can't be relied upon. Even though the author in question hasn't been deplatformed from nostr itself (since he can always self-host a publicly accessible relay to store his content), he has been effectively shadow-banned.

To get around this, relay selections have to be communicated in some other way. Nostr has a few different mechanisms for this:

- If the author's NIP 05 address is known and properly configured (it may not be), clients can look up the author's NIP 05 endpoint to find some reasonable relay hints. Unfortunately, these are often neglected, and usually custodial, so they can run into the same problems.

- If the author's pubkey is found in another signed event found on nostr, [relay hints](https://github.com/nostr-protocol/nips/blob/fade0164f52033314bf0a5ef9bd63c2483afae9b/10.md#marked-e-tags-preferred) can be a way to propagate relay selections through the network. This relies on implementations picking reliable relay hints which can be difficult, and hints do tend to become less reliable over time. However, this strategy is very effective in resisting censorship because it makes banning viral — if a relay wants to completely purge a particular pubkey from their database, they have to purge every event that references it, since events are tamper-proof.

- In extremis, relay recommendations can always be communicated out-of-band. This can be done using manual input, QR codes, DHTs, jsonl torrents full of `kind 10002` events, or any other mechanism client developers choose to resort to.

Another, more technical assumption is that any given query can be fulfilled by few enough relays that a client can actually make all the connections needed, without running into resource limits. If you're trying to request content from 10,000 users across 1,000 relays, you're going to have a bad time. This was [pointed out](https://coracle.social/nevent1qythwumn8ghj76twvfhhstnwdaehgu3wwa5kuef0qyv8wumn8ghj7cm9d3kxzu3wdehhxarj9emkjmn99uq3samnwvaz7tmrwfjkzarj9ehx7um5wgh8w6twv5hsqgrn7l6zj7ht6ruyk76vvvtkfs4xrhyzc3tm64l3eyfvd40y26sz0gshmunh) to me by Mazin of [nostr.wine](https://nostr.wine). He makes a good point, and it's definitely something to keep in mind. There are some mitigating factors though.

The first is that the current topology of the network probably won't persist forever. Because nostr is largely populated by self-hosting enthusiasts, the number of "tiny" relays is proportionally much higher than it will be if adoption picks up, even if the total number of relays grows. The trajectory is that nostr will drift toward fewer, larger relays, reducing the number of connections needed to fulfill any given query.

This is "centralizing", but it's important to understand that this isn't necessarily a bad thing. As long as there are more than one or two large hubs, there is user choice. And as long as it's possible to run a new relay, there is always an escape hatch. Nostr, like bitcoin, has no hard dependency on the biggest player in the network.

The other thing to consider is that there are lots of other techniques we can use to overcome the limits of the lowest-common denominator's limitations (mobile browser clients), including self hosted or third-party relay proxies. The trade-off here is that a little trust (aka centralization) can go a long way to reducing resource requirements needed to fulfill queries using the Outbox model.

If you're interested in more details on this topic, see [this blog post](https://habla.news/u/hodlbod@coracle.social/sfwV1rNaoQXd65PbIMYgm).

That was a long digression, but there is one other thing that the Outbox model assumes to be the case. Even if the correct relays are found and connected to, they still may not return all desired content, either because they don't have it, or because they refuse to return it to the user requesting it.

This can happen if the publishing client isn't following the Outbox Model, if the author had migrated from one relay set to another without copying their notes over, or if the relay in question chose not to retain the author's content for some reason.

The first two issues can be fixed by improving implementations, but the question of policy is a little more interesting.

# Relativistic relays

The Outbox Model is a mechanical process; it's only as useful as user relay selections are. In order for it to work, users have to be able to make intelligent relay selections.

Every relay has trade-offs, depending on its policy. [140.f7z.io](wss://140.f7z.io) would not be useful for long-form content, for example. Some relays might have a content retention policy that changes depending on whether you're a paying user. If you don't pay, you might find out too late that your content has been deleted from the relay.

So what makes a relay "good" for a particular use case? Well, it's complicated. Here are a few factors that go into that calculus:

- Is the relay in the same geographical as the user? Proximity reduces latency, but jurisdictional arbitrage might be desired. Users should probably have a variety of relays that fit different profiles.

- Will the relay ban the user? Do the operators have a history of good behavior? Is the relay focused on particular types of content? Is the relay's focus consistent with the user's goal in adding that relay to their list?

- What are the relay's retention policies? A user might want to set up an archival relay for her old content, or a multi-availability-zone relay so her notes are immediately accessible to the rest of the network.

- Does the relay require payment? Paid relays are more aligned with their users, but obviously come at a financial cost.

- Does the relay have policies for read-protecting content? If so, other users might not be able to find your posts published to that relay. On the other hand, some relays are configured to work as inboxes for direct messages, which can help preserve privacy.

- Does the relay request that users authenticate? Authentication can help manage spam, but it also allows relays to correlate content requests with users, reducing user privacy.

- Is the relay you use hosted by your client's developer? If so, you're in danger of getting banned from your client and your relay at the same time.

- Is the relay a hub? Using hubs can help smooth out rough areas in Outbox Model implementations, at the cost of centralization.

- Is the relay used by anyone else? One-off relays can be useful for archival purposes, but often won't be used by clients following the Outbox Model, depending on how they optimize requests.

There are lots of ways to approach the problem of helping users select relays, but it's an inherently complex problem which very few people will have the patience to properly address on their own. Relay selection is a multi-dimensional problem, and requires satisfying multiple constraints with a limited number of relay selections.

In the future, special-purpose clients might be used to help people build relay sets. Clients also might provide curated "relay kits" that users can choose and customize. Or, we might see an increase in hybrid solutions, like smarter relay proxies or client-local relays that synchronize using other protocols or platforms.

# The Limitations of Outbox

Outbox is not a complete solution, not because of any of the caveats listed above, but because NIP 65 per se only addresses the question of how to index content by pubkey in a broadcast social media context. But there are many other scenarios for relay selection that Outbox does not solve:

- Community, chat, and group posts might be best posted to relays dedicated to that context.

- Direct messages shouldn't follow the same contours as public social media content.

- Topic-oriented relays, or relays serving a custom feed might be useful independent of who uses them.

- Relays focused on serving a particular kind of event, like music, long-form content, or relay selections, are useful independent of who reads from or writes to them.

- Certain clients might need to fulfill particular use cases by using relays that support certain protocol features, like search, count, or sync commands.

- Some events might not make sense to publish to relays, but should instead be shared only directly, out of band.

Some of these use cases might be solved by new specifications similar to Outbox that prescribe where certain data belongs — for example, [NIP 17](https://github.com/nostr-protocol/nips/blob/master/17.md) requires users to publish a different relay list before they can receive direct messages, while [NIP 72](https://github.com/nostr-protocol/nips/blob/master/72.md) places community relay recommendations directly into the group's metadata object. A reasonably complete list of different relay types can be found in [this PR](https://github.com/nostr-protocol/nips/issues/1282), very few of which have a canonical way to manage selections.

Other use cases might be supported more informally, either by relays advertising their own value proposition, or via third-party [NIP 66](https://github.com/nostr-protocol/nips/pull/230) metadata. Still others might be supported by scoping the network down to only certain relays through explicit relay selection — this is how white-labeled [Coracle instances](https://coracle.tools/) work.

The basic idea here is that there are categories of events that don't have anything to do with where a particular person puts his or her "tweets". For every "what" on nostr, there should be a "how".

# Keep nostr weird

Whatever additional systems we end up adopting for helping with relay selection, one thing is certain — people will continue to discover new, creative uses for relays, and we will always be playing catch up. This is one of the coolest things about nostr!

But it does mean that users will have to adapt their expectations to a network that partitions, re-configures, and evolves over time. Nostr is not a "worse" experience than legacy social media, but it is a version of social media that has itself been set free from the stagnant walled-garden model. Nostr is in many ways a living organism — we should be careful not to impose our expectations prematurely, leaving room to discover what this thing actually is, or can be.

If you enjoyed this post but want more take a look at the talk I gave at [Nostrasia](https://www.youtube.com/live/Nz15SyiwQFk?t=2751s) last year. I also wrote up a [blog post](https://habla.news/u/hodlbod@coracle.social/1700155417145) at about the same time that addresses some of the same issues, but focuses more on privacy concerns around relays and nostr groups. Finally, I recently wrote [this comment](https://github.com/nostrability/nostrability/issues/69#issuecomment-2310524841), which includes some details about challenges I've faced putting Outbox into Coracle.

It feels like some of the bootstrapping logic could be offloaded to extensions.

Relay hints should be immediately available to you the moment you follow a profile. If those information are kept in your login extension and available to any client this would boost the bootstrapping process.

For custom feed, they are "custodian" feeds and pushed to you by the clients, so the relay hints is not the extension responsibility anymore.

I was hoping for yet another cashu wallet