Nostr Web Client

That thought had crossed my mind multiple times. I think it is definately something to explore. I mainly see relays as having an expectation of uptime, whereas p2p tends to only work if your computer is on right now. But clearly people can also run servers behind NAT. And as you say, people can use servers not behind NAT (relays) to solve all kinds of problems.

I'm very open to this and I don't have any firm beliefs about it yet, so I'm impressionable and you could steer me quite easily right now.

Also, I really like the choices the Iroh guys made (rust, quinn, QUIC, even BLAKE3).

When I tried using Iroh, it worked, but it was going through their stream relay (they have 3 relays I think worldwide in case holepunching fails).

I'm also a bit concerned that Iroh is a software stack, not an open specification, or at least that there isn't a separate alternate implementation.

Tekkadan, ゲロゲロ! 🐸 6mo ago

https://rsocket.io/ ?

Hmm. I feel like we have all the right ingredients. We just need the right team. I've been mucking with a spec for 8+ years and every time you open your mouth, I raise an amen. This indicates to me that, given the convergence of ideas, there is a correct answer.

The problem is that everything needs to be fixed at once. There are lots of pre-existing projects that are mostly correct for some aspect, but this is a case where mostly correct doesn't get you there, largely because the pre-existing solutions over solve some aspect making them, as you mentioned, software rather than a spec.

i think that mistakes were made but the core principles are corrrect.

i am working now on buliding relays as multiple components, so, the spec has many things, and some naturally cluster functionally, and other things are able to be separated.

i'm currently rewriting realy to be a bare naked unauth simple relay, and then i'm going to build a relay auth proxy, it will be set up between the client and the relay to control access, thus simplifying the build of both parts

the same thing could then be done at a third layer with network transport, giving you a p2p type networking

i can think of other layers to add too, like proxy caches, that are dumb and can only comprehend "do you have event ID X" - sorta like blossom but basically nostr query only for event IDs, or for pubkeys, or for the newest thing.

actually, i started building something inspired by chatter with semisol, an HTTP API call that lets you just poll a relay for its internal reference numbers (monotonic sequence numbers of each event) and a peer then only has to keep a latest sequence number state value related to another relay and the relay can then deliver all the newer events to it no muss no fuss.

i do think the nostr protocol has a load of ugly bits in it but so long as everyone mainly respects the nip-01 rule about string escaping i'm good. all the other things are copable.

It is my long term goal to eventually convert you to something a bit lower level than nostr. Years from now once I finish this spec and a rust reference implementation, I won't be happy till someone makes competing implementations in other languages.

We are probably all a bit too curmudgeony to ever work together, but we'd make lovely competitors. Flame wars and such.

yeah, i want to remove kinds, and json, and websockets. tags can do kinds, line structured text is simpler to parse and easier to read for humans, and http/sse serve perfectly fine, modern network stacks don't need liveness pings to report tho the client or server that the connection has dropped.

shame about the language tho. rust is ugly, and overly complicated, unlike what you are trying to achieve.

Nostr has a lot of bright devs that could all work together to build the next iteration. I certainly don't have it all in my head. I've focused on the low-level stuff, and other devs are far more aware of the problems with the nostr high-level stuff. For example, if you asked me how zaps should have been done, I have no idea. But of course among the low level stuff we have a lot of different ideas and thoughts. I'm keen to dig into other similar work like nostr:npub1fjqqy4a93z5zsjwsfxqhc2764kvykfdyttvldkkkdera8dr78vhsmmleku realy, dpc/rostra, iroh, nostr:npub1l3cgtsurhfchg4cyhhqudm70074sr96srhje330xc5m6czej5n9s9q6vs2 spec fork, any spec you are working on, etc, and try to pull the best stuff into a consistent spec. I was intending to finish Mosaic and submit it as "this is how I would do it" and then tear it to shreds with the feedback it got. I only made it detailed to prove to myself that the ideas would work (in that proof I have changed them quite a bit so it was worthwile)... even though that seems wasteful to get so detailed so early without feedback yet.

if someone can come up with a plan for migrating to a new wire encoding that would be great. i've thought about it. mac users might be familiar with the concept of "fat binaries" and i can imagine a new format that includes the old message in the old format also signed as a transitional step, that allows a relay or client to propagate messages that a transitional client or relay can recognise and know to extract the encapsulated message for legacy clients,

ultimately eventually deprecate the json encoding. my feeling is to use line structured, sentinel based, and we need to get rid of kinds because they are redundant if you can wrap them into tags anyway, and the escaping scheme of json, ok, nostr has a simplified one that only has like 8 escaped characters but you can trim that down to two if you use linebreaks as separators of event fields.

i also think that websockets are a dumb idea, all of the functions can be done without sockets. SSE covers subscriptions and you just open one to a relay to get push messages and everything else you send http requests.

i was going to build out a protocol i was provisionally naming "manifold" using that line structured encoding i mentioned, but i decided to switch gears and build a layer cake relay where you have the core nip-01 (and maybe eventually add delete, search and a few other things that kinda fit within that box) and then wrap that in a proxy that does the nip-42 and can also handle nip-98 auth as well, and a yet to be defined authed proxy protocol that allows a client to get a relay to proxy content fetched with their authorization from auth-required relays.

I like the fat binary idea. Newer clients sign the JSON event and also the new format. Mosaic has been going with a "clean break" and a new cryptosystem, in which this idea isn't sufficient for dealing with replies... but I'm getting closer to the point of just accepting that we simply have to stick with secp256k1 and simply have to keep working with existing nostr.... still a fight goes on in my head.

Line structured data (like HTTP) is reasonable. But even HTTP after they went with line-based added compression. The compressed data is clearly binary, the thing everybody shutters about, but nobody seems to mind when HTTP uses compression. I also want what is digitally signed to be all lined up ready to sign and not need to be copied and shuffled first, but that is a minor point.

As for kind, I'm of the opinion that we have a 64-bit kind number, where 5 bytes are the application ID, 2 bytes are used within the application, and 1 byte is flags telling the relay how to handle it (ephemeral? duplicates? serve only to author? etc). Then applications (like zaps, kanban boards, git, etc) are out of scope and specified by anybody who wants to write an app that is now strictly on top. App IDs are just handed out to anybody who wants one with no debate.

HTTP/WebSockets doesn't add anything on top of streams except for framing (which is easy). So I'm for direct on top of QUIC, and for Tor support which can't handle UDP direct on top of TCP with TLS.

I'm keen on using client-side certificates in TLS for auth. The only downside is that your connection is either AUTHed or not, so you can't conditionally upgrade it, meaning you have to reconnect if something let you know it is time to auth. But reconnecting on QUIC is trivial and highly performant. Putting AUTH inside of nostr caused some state and order related problems... maybe we solved them all I'm not sure. But TLS auth I'm quite sure is well researched and secure.

personally i like BIP-340 secp256k1 signatures. but the simple fact is that they are basically the same as ed25519 except with a group that has a couple more roots than 19. even the modulo multiplication of signatures is almost identical, basically schnorr. which was technically out of patent when satoshi dealt with it. and also, secp256k1 and ed25519 are both "nothing up my sleeve" curves and what's better about the bitcoin curve is you can use the exact same group to do ECDH where you have to use curve25519 to do ECDH in edwards universe. which is a problem.

compression should really be considered to be a transport layer thing, what is received by the client should be the uncompresssed binary anyway.

we don't need kinds. that's what tags can be used for. and we don't need multiple value fields, because the only reason why nostr has multiple fields is because of the "only single alpha character is indexable" idea. if you throw away that silly idea, then you see that kinds becomes a tag, which simplifies the database indexing and simplifies search notation as well. and it also means we can use standardised mimetypes, like email and http.

i like quic too but i think that for mostly non-interactive stuff it's a waste of time using sockets. you have requests, and you have push on the other side.

it would be zero change to most code to enable listening with quic protocol and connecting with it as well. and since it's all http, it could be in the headers to request transport upgrades other than websockets, like quic, i think that's already pretty much standardised.

TLS auth basically works on the same principle as nip-42, nip-98 and JWT anyway. signature, timestamp, challenge.

i thought you might appreciate my counterpoints on those.

i'm not against edwards curves at all but the support for ECDH to do symmetric encryption is very clumsy right now in comparison.

My interest in ed25519 isn't really the cryptography technicals. You make a good point on that. I'm only interested in wider compatibility. You could use ed25519 within TLS, or to store bootstrapping information in bittorrent's DHT, or even as OpenPGP keys (or openssh, or wireguard, etc). But you can't do any of that with secp256k1 (why? probably no good reason but they just didn't add it).

The difference between kind being a tag and kind being a separate field is that when it is a separate field it is required. As a tag it might be left out. And I think it must always be specified. So long as it is always specified, how it is encoded doesn't seem to make a functional difference. I'd prefer a separate field so it is never forgotten. I've written database indexing at least 3 times now on the KV database level. You are right it would simplify it without a separate kind. And the filter would be simpler to just have tags and not a separate kind. But I still think it should be separate for reason I mentioned.

I'm good with standard mime types. There is the concern that if lots of mime types are used, clients wont know how to deal with many of them. But I think that is workable.

QUIC is really just a reimplementation of TCP+TLS. HTTP/3 is built on top of that. Request-response HTTP (1.1 or 2.0) is built on top of sockets itself, just the spec closes it down after the server responds, instead of leaving it open for more messages later.

from a database implementation perspective, the kind field creates an extra factor in a factorial combination of fields that the user can search on, that's why i want to do away with it.

kind specifies a protocol - the combination of an application, and an encoding, right? all messages need this anyway. mimetype: text/plain would be all you would need to describe kind 1 notes. other things might be more complex, like long form, you would want to have text/markdown or text/asciidoc or text/pdf or something, but again, you would have to put these somewhere anyway, it's not like it would be logical to leave out the document type. most event kinds are just document types, some include encoding, some include application/protocol stuff.

so, i'm just saying, kind is mostly redundant, and even already some event kind specs already are redundant by having a kind, as well as these more detailed things in the tags that are part of the kind definition.

65536 possible encodings/document types/applications is very restrictive and not at all future proof.

also, regarding quic, and TCP itself - this is for short message interactive protocols predominantly. not even as slow as IRC chat messages, faster than this. think collaborative document protocols.

for threaded forums, completely useless and irrelevant

for chats, pretty much not relevant to use this kind of low latency interactive socket transport

it's only for control interfaces between servers, really.

i mean, sure, there is no problem to use HTTP/3 over QUIC transparently in many languages already anyway. i think it should be just negotiated in the http headers, or better still, different ports/schemes (same thing, really).

Couldn't you just have a tiny translation layer where you move the kind into the tags before you do the indexing, and before you do the searching by filter? That is, index the kind in the same index that you use for indexing the tags. Maybe make the tag key something unique and nutty like7SFOIU_l, to avoid collisions.

I think kinds have a lot more meaning than describing how to interpret the content. They imply how to find the events you care about (e.g. outbox model?), if replies/threads make sense, if they are ephemeral, multiple versions or replacement, and maybe much more. I wouldn't want to presume we know all the other things kind might mean in the next new innovative idea. It provides hard separation between very different application types, which I very much like.

well, you'd have to interpret the meaning of each kind to decide how they would be tagged. some might be one, some two, some three facets of the design.

like what nostr:npub1l5sga6xg72phsz5422ykujprejwud075ggrr3z2hwyrfgr7eylqstegx9z has been doing building indexes and document segments has done with several types, there is a case of several kinds that are really part of one document.

and in other cases, kinds can actually pertain to several kinds of documents, all connected to the one kind, but they have several formats related to them.

yeah, simple example:

kind 1 events

you have the root second field in the tag for the OP

you have the mentions (p tags) that point to past known events related to the thread

so, you already have two distinct types of documents in the kind 1, the root, and the reply type.

then, additionally, you can also refer to these in other 1 events that are either root or reply kind 1s, that refer to other kind ones, called "quotes" and "reposts".

We are imagining different things. I'm trying to build an isolated layer wherein the content is opaque, and there is a kind that specifies whose social media specification protocol this record belongs to. If someone wants mime-multipart content or whatever, that is the next layer up from what I care about, opaque at the layer I'm concerned with. Now if any application has specific needs from the lower layer and they are general and apply to multiple applications, then I need to think about that.

kinds are not a singular thing, and they are not descriptive. they are a stupid number that tells you nothing and forces you to refer to some document that probably will change and refers to applications that are still alpha.

what the programmer who is building tools to generate and parse content needs:

1. encoding

2. semantics

3. protocol sequence

kind is all three of these in one, cloaked in a stupid number, and hosted very often inside a PR on a poorly managed specification repository. it's not descriptive, it's just cryptic.

arthurfranca 6mo ago

Using just ws:// for both one-off and live communications and the Nostr event structure being JSON sure attract devs. This part isn't the problem. (Though, why .content isn't a regular tag? Why is .id included?)

Regarding nostr keys/signature/structure, if changing them wouldn't make things so fast that a new use case would be possible, I guess it isn't worth it. And its good to have something like frost available.

But Nostr has some annoying inefficiencies on relays that won't ever change cause it would take additions to NIP-01 that, let's face it, will never be made.

The main inefficiencies are:

- The relay can't apply different event size restrictions by event kind on EVENT messages. Using a ws binary message with the event kind at the beginning for that would be great.

- Why is sending the event id (client-to-relay and relay-to-client) required when it would have to be recalculated anyway to make sure its correct?

- Relay can't announce it's custom features/config within the ws connection: https://github.com/nostr-protocol/nips/pull/1969

- The AND operator won't ever make it to NIP-01 (new NIPs can't use it for new event kinds cause relay support would have to be broad): https://github.com/nostr-protocol/nips/pull/1365

I guess if someone has the time and willpower to invest into making a nostr-v2, they would have to spin-up a big nostr-v2 free relay to attract client devs and index events from v1's main relays to have initial content. Good luck for that brave guy or maybe this will be Google or Meta at some point.

arthurfranca 6mo ago

damn Coracle bizarre threading ui

I replied to a msg a bit deeper on the thread than I wanted =s

I think this is the way. You can get to a kind of Facebook like experience. Seeing content from people you know. You do trade off many things like broad discovery, but what you get in return is a network that is extremely resilient to outside censorship and hard to spam, even without filters.

The reason that it should be done is that, even if it isn't very performant you do get something that is 100% aligned with the users.

From there you can add more robust nodes to people that better uptime / wider reach. That will have some centralizing effects, but if those fail then ordinary people still have a fallback that doesn't make them susceptible to all the problems of a wide open relay.

Imagine that we created a way for relays to live behind NAT and be hole-punched into (using Iroh or otherwise) but they were still separate from clients. Then imagine we forced every client to also be a relay, and conversely every relay to also be a client. Do we then end up with the circumstance you are envisioning or have I missed something?

Because in that scenario, while I like the 1st step, I don't see the point of the 2nd step. it seems more flexible to leave relays and clients separate.