if someone can come up with a plan for migrating to a new wire encoding that would be great. i've thought about it. mac users might be familiar with the concept of "fat binaries" and i can imagine a new format that includes the old message in the old format also signed as a transitional step, that allows a relay or client to propagate messages that a transitional client or relay can recognise and know to extract the encapsulated message for legacy clients,

ultimately eventually deprecate the json encoding. my feeling is to use line structured, sentinel based, and we need to get rid of kinds because they are redundant if you can wrap them into tags anyway, and the escaping scheme of json, ok, nostr has a simplified one that only has like 8 escaped characters but you can trim that down to two if you use linebreaks as separators of event fields.

i also think that websockets are a dumb idea, all of the functions can be done without sockets. SSE covers subscriptions and you just open one to a relay to get push messages and everything else you send http requests.

i was going to build out a protocol i was provisionally naming "manifold" using that line structured encoding i mentioned, but i decided to switch gears and build a layer cake relay where you have the core nip-01 (and maybe eventually add delete, search and a few other things that kinda fit within that box) and then wrap that in a proxy that does the nip-42 and can also handle nip-98 auth as well, and a yet to be defined authed proxy protocol that allows a client to get a relay to proxy content fetched with their authorization from auth-required relays.

I like the fat binary idea. Newer clients sign the JSON event and also the new format. Mosaic has been going with a "clean break" and a new cryptosystem, in which this idea isn't sufficient for dealing with replies... but I'm getting closer to the point of just accepting that we simply have to stick with secp256k1 and simply have to keep working with existing nostr.... still a fight goes on in my head.

Line structured data (like HTTP) is reasonable. But even HTTP after they went with line-based added compression. The compressed data is clearly binary, the thing everybody shutters about, but nobody seems to mind when HTTP uses compression. I also want what is digitally signed to be all lined up ready to sign and not need to be copied and shuffled first, but that is a minor point.

As for kind, I'm of the opinion that we have a 64-bit kind number, where 5 bytes are the application ID, 2 bytes are used within the application, and 1 byte is flags telling the relay how to handle it (ephemeral? duplicates? serve only to author? etc). Then applications (like zaps, kanban boards, git, etc) are out of scope and specified by anybody who wants to write an app that is now strictly on top. App IDs are just handed out to anybody who wants one with no debate.

HTTP/WebSockets doesn't add anything on top of streams except for framing (which is easy). So I'm for direct on top of QUIC, and for Tor support which can't handle UDP direct on top of TCP with TLS.

I'm keen on using client-side certificates in TLS for auth. The only downside is that your connection is either AUTHed or not, so you can't conditionally upgrade it, meaning you have to reconnect if something let you know it is time to auth. But reconnecting on QUIC is trivial and highly performant. Putting AUTH inside of nostr caused some state and order related problems... maybe we solved them all I'm not sure. But TLS auth I'm quite sure is well researched and secure.

Reply to this note

Please Login to reply.

Discussion

personally i like BIP-340 secp256k1 signatures. but the simple fact is that they are basically the same as ed25519 except with a group that has a couple more roots than 19. even the modulo multiplication of signatures is almost identical, basically schnorr. which was technically out of patent when satoshi dealt with it. and also, secp256k1 and ed25519 are both "nothing up my sleeve" curves and what's better about the bitcoin curve is you can use the exact same group to do ECDH where you have to use curve25519 to do ECDH in edwards universe. which is a problem.

compression should really be considered to be a transport layer thing, what is received by the client should be the uncompresssed binary anyway.

we don't need kinds. that's what tags can be used for. and we don't need multiple value fields, because the only reason why nostr has multiple fields is because of the "only single alpha character is indexable" idea. if you throw away that silly idea, then you see that kinds becomes a tag, which simplifies the database indexing and simplifies search notation as well. and it also means we can use standardised mimetypes, like email and http.

i like quic too but i think that for mostly non-interactive stuff it's a waste of time using sockets. you have requests, and you have push on the other side.

it would be zero change to most code to enable listening with quic protocol and connecting with it as well. and since it's all http, it could be in the headers to request transport upgrades other than websockets, like quic, i think that's already pretty much standardised.

TLS auth basically works on the same principle as nip-42, nip-98 and JWT anyway. signature, timestamp, challenge.

i thought you might appreciate my counterpoints on those.

i'm not against edwards curves at all but the support for ECDH to do symmetric encryption is very clumsy right now in comparison.

My interest in ed25519 isn't really the cryptography technicals. You make a good point on that. I'm only interested in wider compatibility. You could use ed25519 within TLS, or to store bootstrapping information in bittorrent's DHT, or even as OpenPGP keys (or openssh, or wireguard, etc). But you can't do any of that with secp256k1 (why? probably no good reason but they just didn't add it).

The difference between kind being a tag and kind being a separate field is that when it is a separate field it is required. As a tag it might be left out. And I think it must always be specified. So long as it is always specified, how it is encoded doesn't seem to make a functional difference. I'd prefer a separate field so it is never forgotten. I've written database indexing at least 3 times now on the KV database level. You are right it would simplify it without a separate kind. And the filter would be simpler to just have tags and not a separate kind. But I still think it should be separate for reason I mentioned.

I'm good with standard mime types. There is the concern that if lots of mime types are used, clients wont know how to deal with many of them. But I think that is workable.

QUIC is really just a reimplementation of TCP+TLS. HTTP/3 is built on top of that. Request-response HTTP (1.1 or 2.0) is built on top of sockets itself, just the spec closes it down after the server responds, instead of leaving it open for more messages later.

from a database implementation perspective, the kind field creates an extra factor in a factorial combination of fields that the user can search on, that's why i want to do away with it.

kind specifies a protocol - the combination of an application, and an encoding, right? all messages need this anyway. mimetype: text/plain would be all you would need to describe kind 1 notes. other things might be more complex, like long form, you would want to have text/markdown or text/asciidoc or text/pdf or something, but again, you would have to put these somewhere anyway, it's not like it would be logical to leave out the document type. most event kinds are just document types, some include encoding, some include application/protocol stuff.

so, i'm just saying, kind is mostly redundant, and even already some event kind specs already are redundant by having a kind, as well as these more detailed things in the tags that are part of the kind definition.

65536 possible encodings/document types/applications is very restrictive and not at all future proof.

also, regarding quic, and TCP itself - this is for short message interactive protocols predominantly. not even as slow as IRC chat messages, faster than this. think collaborative document protocols.

for threaded forums, completely useless and irrelevant

for chats, pretty much not relevant to use this kind of low latency interactive socket transport

it's only for control interfaces between servers, really.

i mean, sure, there is no problem to use HTTP/3 over QUIC transparently in many languages already anyway. i think it should be just negotiated in the http headers, or better still, different ports/schemes (same thing, really).

Couldn't you just have a tiny translation layer where you move the kind into the tags before you do the indexing, and before you do the searching by filter? That is, index the kind in the same index that you use for indexing the tags. Maybe make the tag key something unique and nutty like7SFOIU_l, to avoid collisions.

I think kinds have a lot more meaning than describing how to interpret the content. They imply how to find the events you care about (e.g. outbox model?), if replies/threads make sense, if they are ephemeral, multiple versions or replacement, and maybe much more. I wouldn't want to presume we know all the other things kind might mean in the next new innovative idea. It provides hard separation between very different application types, which I very much like.

well, you'd have to interpret the meaning of each kind to decide how they would be tagged. some might be one, some two, some three facets of the design.

like what nostr:npub1l5sga6xg72phsz5422ykujprejwud075ggrr3z2hwyrfgr7eylqstegx9z has been doing building indexes and document segments has done with several types, there is a case of several kinds that are really part of one document.

and in other cases, kinds can actually pertain to several kinds of documents, all connected to the one kind, but they have several formats related to them.

yeah, simple example:

kind 1 events

you have the root second field in the tag for the OP

you have the mentions (p tags) that point to past known events related to the thread

so, you already have two distinct types of documents in the kind 1, the root, and the reply type.

then, additionally, you can also refer to these in other 1 events that are either root or reply kind 1s, that refer to other kind ones, called "quotes" and "reposts".

We are imagining different things. I'm trying to build an isolated layer wherein the content is opaque, and there is a kind that specifies whose social media specification protocol this record belongs to. If someone wants mime-multipart content or whatever, that is the next layer up from what I care about, opaque at the layer I'm concerned with. Now if any application has specific needs from the lower layer and they are general and apply to multiple applications, then I need to think about that.

kinds are not a singular thing, and they are not descriptive. they are a stupid number that tells you nothing and forces you to refer to some document that probably will change and refers to applications that are still alpha.

what the programmer who is building tools to generate and parse content needs:

1. encoding

2. semantics

3. protocol sequence

kind is all three of these in one, cloaked in a stupid number, and hosted very often inside a PR on a poorly managed specification repository. it's not descriptive, it's just cryptic.

Using just ws:// for both one-off and live communications and the Nostr event structure being JSON sure attract devs. This part isn't the problem. (Though, why .content isn't a regular tag? Why is .id included?)

Regarding nostr keys/signature/structure, if changing them wouldn't make things so fast that a new use case would be possible, I guess it isn't worth it. And its good to have something like frost available.

But Nostr has some annoying inefficiencies on relays that won't ever change cause it would take additions to NIP-01 that, let's face it, will never be made.

The main inefficiencies are:

- The relay can't apply different event size restrictions by event kind on EVENT messages. Using a ws binary message with the event kind at the beginning for that would be great.

- Why is sending the event id (client-to-relay and relay-to-client) required when it would have to be recalculated anyway to make sure its correct?

- Relay can't announce it's custom features/config within the ws connection: https://github.com/nostr-protocol/nips/pull/1969

- The AND operator won't ever make it to NIP-01 (new NIPs can't use it for new event kinds cause relay support would have to be broad): https://github.com/nostr-protocol/nips/pull/1365

I guess if someone has the time and willpower to invest into making a nostr-v2, they would have to spin-up a big nostr-v2 free relay to attract client devs and index events from v1's main relays to have initial content. Good luck for that brave guy or maybe this will be Google or Meta at some point.

damn Coracle bizarre threading ui

I replied to a msg a bit deeper on the thread than I wanted =s