> Layered protocol (separate core from applications)
What do you mean by this? How would it work?
# Nostr 2?
## Breaking Changes in Nostr
Nostr was a huge leap forward. But it isn't perfect.
When developers notice a problem with nostr, they confer with each other to work out a solution to the problem. This is usually in the form of a NIP PR on the nips repo.
Some problems are easy. Just add something new and optional. No biggie. Zaps, git stuff, bunkers... just dream it up and add it.
Other problems can only be fixed by breaking changes. With a breaking change, the overall path forward is like this: Add the new way of doing it while preserving the old way. Push the major software to switch to the new way. Then deprecate the old way. This is a simplification, but it is the basic idea. It is how we solved markers/quotes/root and how we are upgrading encryption, among other things.
This process of pushing through a breaking change becomes more difficult as we have more and more existing nostr software out there that will break. Most of the time what happens is that the major software is driven to make the change (usually by nostr:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6), and the smaller software is left to fend for itself. A while back I introduced the BREAKING.md file to help people developing smaller lesser-known software keep up with these changes.
## Big Ideas
But some ideas just can't be applied to nostr. The idea is too big. The change is too breaking. It changes something fundamental. And nobody has come up with a smooth path to move from the old way to the new way.
And so we debate a bunch of things, and never settle on anything, and eventually nostr:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6 makes a post saying that we don't really want it anyways 😉.
As we encounter good ideas that are hard to apply to nostr, I've been filing them away in a repository I call "nostr-next", so we don't forget about them, in case we ever wanted to start over.
It seems to me that starting over every time we encountered such a thing would be unwise. However, once we collect enough changes that we couldn't reasonably phase into nostr, then a tipping point is crossed where it becomes worthwhile to start over. In terms of the "bang for the buck" metaphor, the bang becomes bigger and bigger but the buck (the pain and cost of starting over) doesn't grow as rapidly.
## WHAT? Start over?
IMHO starting over could be very bad if done in a cavalier way. The community could fracture. The new protocol could fail to take off due to lacking the network effect. The odds that a new protocol catches on are low, irrespective of how technically superior it could be.
So the big question is: can we preserve the nostr community and it's network effect while making a major step-change to the protocol and software?
I don't know the answer to that one, but I have an idea about it.
I think the new-protocol clients can be dual-stack, creating events in both systems and linking those events together via tags. The nostr key identity would still be used, and the new system identity too. This is better than things like the mostr bridge because each user would remain in custody of their own keys.
## The nitty gritty
Here are some of the things I think would make nostr better, but which nostr can't easily fix. A lot of these ideas have been mentioned before by multiple people and I didn't give credit to all of you (sorry) because my brain can't track it all. But I've been collecting these over time at https://github.com/mikedilger/nostr-next
* Events as CBOR or BEVE or MsgPack or a fixed-schema binary layout... anything but JSON text with its parsing, it's encoding ambiguity, it's requirement to copy fields before hashing them, its unlimited size constraints. (me, nostr:npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s)
* EdDSA ed25519 keys instead of secp256k1, to enable interoperability with a bunch of other stuff like ssh, pgp, TLS, Mainline DHT, and many more, plus just being better cryptography (me, Nuh, Orlovsky, except Orlovsky wanted Ristretto25519 for speed)
* Bootstrapping relay lists (and relay endpoints) from Mainline DHT (nostr:npub1jvxvaufrwtwj79s90n79fuxmm9pntk94rd8zwderdvqv4dcclnvs9s7yqz)
* Master keys and revocable subkeys / device keys (including having your nostr key as a subkey)
* Encryption to use different encryption-specific subkeys and ephemeral ones from the sender.
* Relay keypairs, TLS without certificates, relays known by keypair instead of URL
* Layered protocol (separate core from applications)
* Software remembering when they first saw an event, for 2 reasons, the main one being revocation (don't trust the date in the event, trust when you first saw it), the second being more precise time range queries.
* Upgrade/feature negotiation (HTTP headers prior to starting websockets)
* IDs starting with a timestamp so they are temporally adjacent (significantly better database performance) (Vitor's idea)
* Filters that allow boolean expressions on tag values, and also ID exclusions. Removing IDs from filters and moving to a GET command.
* Changing the transport (I'm against this but I know others want to)
## What would it look like?
Someone (Steve Farroll) has taken my nostr-next repo and turned it into a proposed protocol he calls [Mosaic](https://github.com/SteveFarroll/mosaic-spec). I think it is quite representative of that repo,
as it already includes most of those suggestions, so I've been contributing to it. Mosaic spec is rendered [here](https://stevefarroll.github.io/mosaic-spec/).
Of course, where Mosaic stands right now it is mostly my ideas (and Steve's), it doesn't have feedback or input from other nostr developers yet. That is what this blog post is about. I think it is time for other nostr devs to be made aware of this thing.
It is currently in the massive breaking changes phase. It might not look that way because of the detail and refinement of the documentation, but indeed everything is changing rapidly. It probably has some bad ideas and is probably missing some great ideas that you have.
Which is why this is a good time for other devs to start taking a look at it.
It is also the time to debate meta issues like "are you crazy Mike?" or "no we have to just break nostr but keep it nostr, we can't dual-stack" or whatever.
Personally I think mosaic-spec should develop and grow for quite a while before the "tipping point" happens. That is, I'm not sure we should jump in feet first yet, but rather we build up and refine this new protocol and spend a lot of time thinking about how to migrate smoothly, and break it a lot while nobody is using it.
So you can just reply to this, or DM me, or open issues or PRs at [Mosaic](https://github.com/SteveFarroll/mosaic-spec), or just whisper to each other while giving me the evil eye.
> Layered protocol (separate core from applications)
What do you mean by this? How would it work?
That is an organisational comment, as to how the protocol definition (the NIPs) are organised (or not organised). Nostr NIPs have no structure in terms of which NIPs apply to everybody (it is not just 1 anymore) at the base protocol, and what is an application riding on top of nostr. It isn't clear without reading or being around that NIP-10, NIP-18, NIP-22(?), NIP-11, NIP-09, NIP-51, NIP-59, and more... these are all kind of core facilities that applications can use. But NIPs 54 (wiki), C7 (chats) 64 (chess), 35 (torrents), etc, these are all applications that ride on top of a core.
IMHO the NIPs repo should just register application kinds so that applications don't clobber each other with their kinds, but not be defining all these applications. There could be a sister repo that aggregates lots of nostr app specs, it wouldn't have to be pushed out to separate owners.
I see. I thought you were talking about a way to disentangle all the relay-picking quibbles from the development such that it could be all isolated in a simple library. It would be great if such a thing was possible but I can't imagine it. Every app developer today needs to keep relays in mind to some extent and I don't see how that could be different. In fact ideally users have to also.
If it's just an organizational comment then we can do that today, but I'm not really seeing what you're describing so I don't know.
My first reaction is that changing the way we organize stuff will not make it less messy, we will just move the mess around in different shapes.
What I really think is missing is a good documentation page that distills the knowledge required for writing Nostr apps. It would have to be something that doesn't follow the NIPs closely, but just lays a meaningful story on top of them and other real-world experience.