Nostr Web Client

a lot of early nostr apps started with server-as-the-only-datastore but now many of us are making nostr apps local-first + set reconciliation with remote only when you need it.

the flexibility of choice here is super powerful. notedeck is local-first by default for any app you build inside it (its a nostr browser). local network or remote syncing is more of a business logic choice for the app.

Nuh 5mo ago

I understand that, it is not unique to Nostr, it has been a thing since Git, and Peergos for example while it puts lots of emphasis on data integrity using authenticated data structures (as opposed to separate events) it still acts as local first, and tries to sync with the server with CAS like api.

My point is that eventually you are going to run out of apps that don't require a clear and consistent answer to the question "did I update the state yet?" Or "What is the canonical state that others eventually will see".

There is only so many apps that can tolerate the lost update problem, which is very common in systems that don't have a datastors.

My point is; data stores are not a centralisation force, unlike say search indexes and "global feeds", so we shouldn't give up on them, they can be as easy as letting the user choose one of their devices as a home server, and all other devices wait for it to acknowledge updates, and show things in pending state otherwise...

But also, if your notes can be found, then a note about your "Outbox" can be found, and then that home server being cloud hosted is not really a "shitcoin", because you can't exit it on a whim.

Reply to this note

Please Login to reply.

Discussion

jb55 5mo ago

the behavior is up to the app to decide how they want the data synced, not the protocol. its true this logic might diverge between apps, but if looking at it on a per app basis then it will work fine.

if cross-app compatibility is important enough (i think it is for microblogging), then we will converge to things like outbox and other methods, but we are not tied to it and its open to be improved even more. its bottom up not top down. its the bazaar not the cathedral.

the SSB, matrix and even git guys already tried the method of heavily perscribing the state syncing in a top-down way, and it just sucks for app development.

Nuh 5mo ago

I respect that. My experience is usually that devs are frustrated when I put in their hand and API that doesn't block until the data is canonically stored, so I am always surprised when Nostr OG devs argue that something like webdav or S3 would suck... but I guess you have definitely earned your opinion.

I just struggle to imagine what can be done with Nostr that a remote storage API with a tree over the data, just like LMDB would make impossible or suck, for example when I don't want need LMDB atomicity I build my own buffer, it doesn't get in my way.

I would still be interested in learning what would you hate about something like LMDB but with remote api?

jb55 5mo ago

I'm not sure if nostr:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6 intended this when designing the protocol, but it affords use cases that do not depend on a remote api. and that is really useful. the protocol dictates a uniform query interface that doesn't necessarily depend on location. so this enables local-first apps with optional remote replicas.

not *needing* an api is huge. the query language is the universal api. this means a local app would work exactly the same as one with data stored on a remote relay.

you *do* have blocking write confirmations, I added it to the protocol in the form of command results (OK). you just don't have transactions, but that can be implemented at a local layer before replicating.

I imagine building a distributed transaction model would be complicated, dbs like tigerbeetle and foundationdb are very complicated. the only other project I can think of tackling this in an interesting way is https://simon.peytonjones.org/verse-calculus/ via a deterministic logic language for building data models in a metaverse context, but that is also complicated.

nostr avoids this transaction complexity by an append-only graph style way of coding apps... if you ignore replaceable events, which I very much try to do at all costs, except where I can't at the moment (profiles, contact lists)

Nuh 5mo ago

Fiatjaf is definitely a strong believer in RPC as superior to HTTP based API.

However, I think this is a week point, you can send HTTP queries over Bluetooth, HTTP headers are not inherently inferior to JSON, i argue it is better, if only because it allows for using curl and all the infrastructure of talking to servers.

I will however agree that a query language that is simple enough, and thanks to Fiatjaf discipline; resistant to creep and complexity is a great feature... but this is not special to Nostr either, nor does it get embedded by a canonical source of truth or a key value stor api.

You can name your keys in ways that make them effectively lookup indexes, which already covers many of what Nostr queries do.

The append only way is also entirely possible in a system like Peergos, and if you ONLY create new files. Then the CAS will never fail, but then you have extra features that Nostr doesn't have, like being able to see exactly what is the current state, and GC older leaves (which is what LMDB does as well) and everything we take for granted when we have a reliable database.

I am not supporting decentralised databases to the contrary, I am supporting a single writer (home server) so that we don't have to fuck around with decentralised transactional databases.. too hard. But the price of that, is a server, usually in the cloud.

My hope is that smart builders stop perceiving that as a bad thing, as long as the migration between home servers is smooth and permissionless.

Either way, I trust that people would converge into that eventually, and we would have something like Proton Drive (but open and self hostable) as the base layer of free web development, especially with decentralised PKI to make migration better than what it is in say Mastodon. I hope I did enough work to enable that.

semisol 5mo ago

There’s also the question of do you need transactions.

There is transactions for consistency, and there’s transactions to ensure correctness of the system state (like indexes)

Many Nostr use cases actually do not need strict consistency. There is only some level of correctness required.

CRDTs and conflict resolution fixes this. A notes app for example can be represented as a set of diffs on top of each other, and two updates to the same note can be merged.

This is also why Dynamo and eventually consistent databases exist. You could also have slightly smarter relays that can do slightly smarter queries if you want.

WasBuddhaMaybe 5mo ago

Hi semi

Nuh 5mo ago

I am too familiar with CRDTs... they don't generalise and more importantly they are very hostile towards deletion and storage reclamation as each deletion has to forever take space, the best you can do is what the Willow project does (deleting entire folders with one tombstone)...

Then of course migrating to other hosts or just backup becomes an optimistic operation where you can never know if you are done.

Seriously the only downside of authenticated data structures is that you have to block concurrent writes to serialise them, and I think that is such a simple thing to ask I don't understand why resist it.

Even Negentropy (Range Base set reconciliation) has now been proven to be trivial to do with any authenticated data structure with normal hash function (by the author of RBSR).

So I am constantly struggling to understand why not accept the limitation we accept with LMDB, and we know in our bones that it is worth it.

Nuh 5mo ago

I also want to note, whenever I am arguing about design decisions I don't talk about Nostr usecas, I talk about the general usecase of web 1.0 (excluding Web 2.0 with its aggregation and indexing, because i am skeptical we can fix that).

So when someone smart says that the cloud is a shitcoin, my reaction is, hold on why go that far? Servers are cool, people like to know their data is safe? Why give up on these parts of the web usecases? Not everything is a broadcasting app, we need data integrity too.

jb55 5mo ago

i mean there is no shortage of cloud bullshit. the local first, distributed app idea is criminally under explored. mostly because of the web which heavily encouraged the client and web server model of app development

semisol 5mo ago

remember couchdb?

Nuh 5mo ago

It is not criminally under explored, there are tons of work done on that side of the equation, almost none leveraged by Nostr so far, for example I haven't seen anyone working with Automerge or engaging with the ink and switch people, most likely because all that ecosystem has little to no intersection with Bitcoin.

But these are not mutually exclusive concepts and giving up on open data stores is a step back from the old days of Email and WebDav... why take steps back?

Niel Liesmons 5mo ago

Lol, you keep dropping interesting tech I've never heard of.

Happen to have a list of Nostr-conpatible tech by any chance ? 😜

semisol 5mo ago

Your following list seems to have gotten nuked.

https://nostr.land/restore

Niel Liesmons 5mo ago

I did that all by myself sir #bigboy

Niel Liesmons 5mo ago

Don't use it anymore.

semisol 5mo ago

Lists?

Nuh 5mo ago

Seriously this is one of the things I enjoy doing; connecting ideas across communities.

I had luck with Range-Based-Set-Reconciliation/Negentropy (I guess because people trusted Strfry developer, fair enough), not much with the rest of my suggestions :D

Nostr is not special, anything that people tried for local first and set reconciliation and "bring your own background" etc. applies to Nostr and any open web protocol.

Fun fact, Peergos is based on Cryptree which was developed like 15 years ago.

There are very few new ideas, just ego preventing us from using old ones.

Niel Liesmons 5mo ago

> There are very few new ideas, just ego preventing us from using old ones.

:scoresoccer:

Niel Liesmons 5mo ago

Looking into Peergos rn.

Trying to see how it can map onto private Nostr events and groups.

Niel Liesmons 5mo ago

Where do you work currently btw? If its not Synonym any longer.

jb55 5mo ago

A handful of projects noone has heard about and not related to data-interoperable, local first apps is still criminally under explored to me. You keep referencing webdav like it isn’t the worst possible api in the history of apis… happy to leave those behind.