I created a quick diagram to illustrate the Primal cache issue in a simplified way that may help some people understand what is going on and help with choices about which client to use. There are many ways to build clients and there are advantages and disadvantages of each.

In Primal’s case, notes are served from Primal’s cache server which gets notes from the relays rather than the client getting them directly from the relays. The advantage to this is that much of the processing can be done on the server side resulting in a very smooth and consistent experience for users.

The down side to this is that it becomes centralised as, without the server infrastructure operated by Primal themselves, the client completely stops working (the cache is open source but I’m not aware of any other instances of the Primal cache being run and most users wouldn’t use a different one anyway). The issues with this centralisation are that it make Primal much more prone to outages and it would be very easy for ISPs to block Primal just by DNS blocking of the Primal cache server or blocking the associated IPs. Primal could also potentially filter or censor the feed from their cache although I’m not aware of whether they actually do this currently.

For me personally, I want a client that talks directly to the relays and does the processing locally but others may be comfortable with the limitations and loss of decentralisation and find that Primal provides a good experience.

One of the best things about Nostr is that you can take your key and move to a different client. It is worthwhile experimenting with different clients to find the experience that best suits you. 💜🫂

Reply to this note

Please Login to reply.

Discussion

This is a great post. I sometimes find Primal cache very annoying. Check this note below and click on the image, you’ll understand. You meed to be on Primal to see the issue. Once in a while an API fails to deliver data whether a techincal issue or it gets rate limited momentarily but even if it gets fixed on my server after a minute or so, the cache prevents the update being pushed into the client. I don’t see this issue happening on Damus as it talks directly to the relays.

https://primal.net/e/note13z83kw6dr8dykan2cwck6ht69px39tde53hvmrh7wzha7lpqfycstgpl3s

I will post this answer again that I think is relevant to the cash server approach

It is no different then twitter.

https://yakihonne.com/notes/nevent1qqsflz35p85f77nlazyj8x4c8yld26tdmxkrk2a7wzgvatfu0llvueszyqewrqnkx4zsaweutf739s0cu7et29zrntqs5elw70vlm8zudr3y2qcyqqqqqqgn96mkx

The "it's no different from Twitter" is a gross overstatement, though I do sympathize with Will's view. I certainly prefer to select the relays I read from and have the client I am using pull notes from those relays directly.

That said, I also understand Primal's choice to use a caching relay for the sake of optimizing for new users who barely understand what a relay is, let alone how to select ones that will achieve their goals. New users don't yet HAVE goals they want to accomplish in their selection of relays.

On Twitter, you don't own your identity. On Nostr, you absolutely do, even on Primal. Yes, Primal COULD put fake notes that appear to be from your npub, but the fact that they weren't actually signed by your nsec would be immediately apparent when they cannot be accessed from any other client, and cannot be rebroadcast out to any other relays.

On Twitter, you cannot post your tweets to any server other than Twitter's centralized server. On Primal, you absolutely still can maintain your censorship resistance by posting to multiple relays, because relays reject notes that aren't signed by the correct private key.

On Twitter, you can only read tweets that were posted to their centralized server. On Primal, you may only be able to read from their caching relay, BUT those aren't just posts that were written TO their relay. You are still seeing posts that were originally written to a wide variety of relays.

On Twitter, you are a slave to their proprietary algorithm. On Primal, you get to choose your algorithm, or create one that suits your own needs.

Now, do I think this is ideal to only read from a caching relay? Absolutely not. Do I think Will has a lot of good points about why this is a compromise of Nostr's values? Yes, I do.

But it's not "no different from Twitter" by a long shot.

The "no different from twitter" is a bit of an exaggeration perhaps, at least on some aspects of Nostr, I agree.

I am personally a big user of primal as well despite it having centralised tendency.

But we must be open about it and clear and voice our concerns so that the devs steer development in the correct direction.

And by the way the "use other clients" is not a good argument, if primal becomes the biggest client no one will build other clients. What we should do is use the decentralised clients and improve the protocol for better experience instead of going centralised.

Wait... "Use other clients is not a good argument," but "what we should do is use the decentralized (other) clients"?

I don't share your concern that "if Primal becomes the biggest client no one will build other clients." Primal simply does not and will not ever do all the things that users want. They are absolutely optimizing for the newbie.

Thing about that is newbies don't remain newbies for long. Eventually they want to try other things, or they see a note quoting some event kind that Primal doesn't display and they ask how they can see that note.

There will be other clients that do some things better. Primal doesn't have live-streams, but Amethyst and noStrudel do, and zap.stream is specialized to ONLY that kind of content. Same thing goes for other types of content, such as photos. Olas is optimizing for an image feed, so is slidestr.net in a different way. Different users are going to prefer each approach. Amethyst has integrated the same event kind as Olas, but I happen to prefer Olas' UI for browsing images over that of Amethyst.

The point is, different clients will ALWAYS have their strengths and weaknesses, so there will almost certainly be a few major clients on each platform (iOS, Android, Desktop, Web) and then a MASSIVE number of specialty clients that focus on a particular type of content or use-case.

Even if Primal gains (or maybe has gained?) the largest user-base by simply being the easiest to use, other clients will still have a decent market-share because they provide other things users value that Primal has opted not to include.

I am not sure it is true that people will move away when they are not newbies anymore, people tend to stick to the first app they use.

Now imagine if primal has a big chunk of the network and it is taken down by gov, what will this do to Nostrs reputation? I am not sure myself maybe the damage won't be big because people will just download a more decentralised client, but then why not use a decentralised client from the start?

Why not just focus on the decentralised direction and improve that? If we can't create a good user experience that way we might as well just give up.

Ummm, most Nostr users I am aware of use a plethora of clients on the daily. One of the biggest selling points of Nostr is specifically the fact that you can take your identity, your content, and your social graph with you to ANY client. So, while there are probably a contingent of folks who ONLY use Primal, I would wager that is a very small subset of Nostr users overall, because one of the reasons we came to Nostr in the first place is because we can use our Nostr ID to sign into all sorts of different clients.

You point about Primal potentially being shut-down very easily, and what that might do to Nostr's reputation, is definitely a valid one. I think that would be the case if any major Nostr client was shut down, though. The fact that Primal only reads from its single caching relay just makes it the most likely first target.

That said, I don't see that as much different than the Bitcoin FUD we used to have surrounding a majority of the hash-rate being located in China. "China could attack the network." "Could you imagine what would happen if China banned mining? We may never find another block because the difficulty would be too high to get to the next adjustment!" "What would that do to Bitcoin's reputation?" Well, China did ban mining, and it did have a short-term effect on Bitcoin. However, Bitcoin is stronger than ever because of it.

I think the same would be the case with Primal. Some people would leave Nostr over it. Others would migrate to other clients and complain that the UX sucks compared to what they had. But a year later those who left will be shocked that Nostr hasn't died after all, and is actually stronger than ever.

I also agree with you that it would be better for users to focus on the more decentralized clients. That's why I don't use Primal much at all. I mainly use #Amethyst, or #Coracle, or #noStrudel, or #Ditto, or... The list goes on and on.

Primal is making a trade-off. Users who are on Primal should be aware of that trade-off. Absolutely agree with that. I also think that caching services may become more and more necessary for UX as Nostr grows. Heck, one of my favorite relays to read from (nostr.wine) is effectively a paid caching service. As is any web-of-trust relay. Primal's may be the only major one we know of right now, but I think that list is going to grow by necessity. Trying to read from 100 different relays on-the-fly to get all of your friends' notes is going to be a bad time, let alone discovering new content.

I don't think that means we have failed, either. So long as we are writing to multiple relays and CAN fall-back to those relays when the caching relay is unavailable, I think we'll still be able to deliver on the promise of a censorship resistant and decentralized social internet and also deliver outstanding UX along with it.

What happens to notes on relays they don't cache? Even though a user may select that relay as one they want to read from they won't see the notes in Primal

I would expect that their cache ingests from a fairly large number or relays so hopefully in most cases there would be an overlap between the set of relays that the note is written to. It could be a problem if you are trying to use some special purpose relays. I don’t know how they handle things like the Fediverse bridge relays for example.

It looks like I can see Fediverse accounts in Primal so that does seem to work.

Any notes that are visible ONLY on relays Primal doesn't cache from would not be visible on Primal, regardless of what relays you select. In Primal, you only WRITE to the relays you select, and you only READ from their caching relay.

That said, the likelihood of any notes only being on a relay that Primal isn't caching from is relatively small. Most people are posting their notes to multiple relays with at least one that Primal is pulling notes in from.

I wonder if it’s possible to make it so that primal will read directly from a relay if it isn’t cached by the caching relay.

Is damus' nostrdb a better alternative to primal caching? Seems so, since, as I understand, it's running on the client, no?

I don’t know if it’s the same but there is something called indexedDB that runs client side / in browser that caches a lot of stuff locally from your own relays.

nostrdb support filter queries, full text search, note stats, multithreaded ingester with efficient note de-dupe by a quick exit json parser, flatbuffer-like binary note format for zero serialization (queries return pointers to virtual memory) . It’s like an embedded strfry. IndexedDB isn’t really comparable.

So yeah its like a caching relay in your phone

Is nostrdb open source? Is there documentation to implement it?

https://github.com/damus-io/nostrdb

https://docs.rs/nostrdb/latest/nostrdb/

docs aren’t the best, still need to improve that

Cool!

notedeck is completely powered by this, which is why its crazy fast. Still in the process of moving Damus iOS to it, but it is in damus iOS in a limited capacity (profile search, tagging, fulltext search, walking threads). Damus iOS just lacks using it for all queries atm.

Another cool thing that notedeck will do is host many different nostr apps at once. Like a nostr OS/browser. This means all your nostr apps can interact with each other while you’re offline.

Interesting

tangent but I would actually like a lightning app with this primal architecture

im not convinced running a node on my phone is always the best

all the load/syncing times, failed payments etc

If the Lightning node is not running on your phone, you only have a few other options:

1. You run the node on your own computer, such as a Start9 or Umbrel, or even an Alby Hub instance installed to your PC, so long as you keep it online 24/7.

2. You run the node on someone else's computer, such as Alby's cloud option for running Alby Hub, or Voltage.

3. Someone else runs the node on their computer, such as if you have an Uncle Jim you trust who could host your wallet, or use a custodial option such as CoinOS or MiniBits.

The "Primal achetecture" option would be most like option 3, where someone else runs the node on their computer.

Breez has been working pretty well for me

ok cool will check it out

Appreciate the simple breakdown without all the drama 😆

Thanks, I was trying to keep it fairly balanced and avoid the drama.

Is there a way for #primal to make the caching server opt-in, or possibly a premium feature? Or does that completely break their app?

Anything is possible but I suspect the Primal team made the design decision to use a cache for their app so probably won’t change it.

I think it could make sense to implement a fail back to local processing of notes directly from the relays in the case that the cache is unreadable. At least that way the app would keep functioning if there was a cache outage, although that would likely be with a more limited set of features than with the cache.

I could see it being a design decision since that probably simplifies a lot of client code and they're building several clients.

But given that they're trying to make a profit somehow that seems like exactly the kind of thing you'd try to charge for.

Primal is optimizing for new users so that they have to configure as little as possible for the app to just work. The caching relay set as the ONLY relay the client is reading from fits right into this objective. Users don't have to wonder why they don't see notes on Primal that are visible elsewhere, because chances are the caching relay has pulled them in from other relays. Users don't have to go and make sure they are reading from the relays that their friend who told them about Nostr is writing to for the same reason.

That being the case, I don't think they would ever make the caching relay an optional service that is only available to premium subscribers. It is too fundamental to their user-friendly UX.

Yes nostr:npub16c0nh3dnadzqpm76uctf5hqhe2lny344zsmpm6feee9p5rdxaa9q586nvr confirmed they intend on adding a fallback

note1kdqwf24qu3cur6gsmynmqy2tzu9z4mnl8ymnwgrrgequssurl85sj9y498

Thanks, I missed seeing that comment but good to know!

Yeah I got the idea from ChatGPT so I figured I would just ask him.

I’d like to see Primal reading directly from a relay if the relay isn’t cached by the caching service

Fantastic explanation! Thank you!

Suggestion: slight modification to the packet leaving the primal server compared to other clients. Maybe a slightly different color?

I did think about colour coding it but it was just a pretty quickly made up diagram!

Great work though! Happy New Years!

Thanks! Happy New Year

Couldn't they easily get around this by having the option for relays to replicate amongst one another?

#realy already does this for user profile data and follow and mute lists and deletes and whatnot

i already built a "layer 2" interface for it and have some example implementations that use shitcoins and second databases for testing the space limiting garbage collecting feature i built into the event store, it would be simple to add a layer2 that forwards queries out to other relays and caches the answer for other users as well as returning the results to the client

also, the question/problem of consistency on the nostr network is one that many minds are working on

one really important thing to note about this is that the lack of a prescription for a consensus to create consistency is intentional, because every type of replication strategy has tradeoffs that are only suited to some use cases and not others

There are relays that already aggregate notes from other relays which provides similar benefits to the cache. I’ve previously used the nostr.wine paid relay which does this and works well.

https://docs.nostr.wine/filter/readme

Primal intends on building a fallback where the client will read directly from the relays in the case of a caching server outage.

note1kdqwf24qu3cur6gsmynmqy2tzu9z4mnl8ymnwgrrgequssurl85sj9y498

As much as I want to understand the diff, 100 i do not 😅Goodnight Nostur 🐯#goodnight #GN