Nostr Web Client

How about we fix the protocol first, instead?

These side quests are only going to make it harder to get to the core fixes that the nostr protocol needs to expand at scale and subsume the current internet in total.

Vitor Pamplona 10mo ago

Like what?

The nips are a mess. This needs to be rectified. The first step needs to be a community effort to GTFO github. Documentation and pass fail tests need to be standardized for all accepted nips and a policy of full documtation and pass/fail testing before a nip is formally accepted needs to be enforced. Basically, before things get out of hand, the nips need to get tidied up as there is a lot... Non-clarity.

The current relays can't scale. See nostr:nprofile1qqs99d9qw67th0wr5xh05de4s9k0wjvnkxudkgptq8yg83vtulad30gpz4mhxue69uhks6tnwshxummnw3ezumrpdejqzxrhwden5te0wfjkccte9ehx7umhdpjhyefwvdhk6qgdwaehxw309ahx7uewd3hkcd5u7te and nostr:nprofile1qqsyeqqz27jc32pgf8gynqtu90d2mxztykj94k0kmttxu37nk3lrktcpzemhxue69uhk6mr9dd6juun9v9k8jtnvdakz7qg4waehxw309a6x2um59eex2ctv0yhxcmmv9uq3kamnwvaz7tm5dpjkvmmjv4ehgtnwdaehgu339e3k7mf0f6aqk5 and nostr:nprofile1qqs8eseg5zxak2hal8umuaa7laxgxjyll9uhyxp86c522shn9gj8crspz9mhxue69uhkummnw3ezuamfdejj7qgjwaehxw309ahx7um5wgerztnrdakj7qgkwaehxw309a3x2an09ehx7um5wgcjucm0d5hsvlnggv and others for solutions and suggestions.

Interoperability is barely there. Sure, it kinda works, but, just about every client is broken for basic expected functionality. (Though, to your credit, amethyst is the king of all nostr clients.)

AUTH. Now. This is non-negotiable. It's exceptionally stupid that this isn't standard yet.

The mindset of trumpeting about new features while ignoring the broken mess that's surrounding the core simplicity of what the nostr protocol should be (and is meant to be, IMO) also needs to stop.

New methods of revenue generation are also necessary. The VC BS needs to stop as it is distorting the market incentives and only makes the stupid feature creep and fragmentation worse since it decouples devs from the market and lets them write really crappy code without consequences.

Vitor Pamplona 10mo ago

Those are all good. But many of them are not solvable or it is not on our power to solve. If you want decentralization, everything will always be a mess.

We all want to get out of GitHub, but no one has ever made a better tool than GitHub that works right now that we can migrate. We are not going to work on that tool ourselves. We don't have the time. So you have to find a person that can do that, likely for the next 3 years, in order to fully replace GitHub.

Relays can't scale because no one is coding one that does scale. We literally don't have a single dev working on real scaleable architecture. You have to find the person first. None of us are good for that type of work.

And systems will interoperate, but they will always have parts where they don't. There is no way to force everyone to implement the same thing unless you are on a dictatorship. There is no fix for it. The best we can do is to slowly make things more interoperable.

Auth is very dangerous. It allows user tracking, including geo-location tracking, and kills all privacy-preserving features of Nostr.

Revenue generation is not be a protocol problem. It's a community problem and people can solve it separatedly from devs. Again, no one is really working on that side. Devs, and product builders in general, are not skilled revenue generators.

> Relays can't scale because no one is coding one that does scale. We literally don't have a single dev working on real scaleable architecture. You have to find the person first. None of us are good for that type of work.

Yup :( No one can seem to just implement mysql like their app will catch a disease or something. Make stateless apps again.

> We all want to get out of GitHub, but no one has ever made a better tool than GitHub that works right now that we can migrate. We are not going to work on that tool ourselves. We don't have the time. So you have to find a person that can do that, likely for the next 3 years, in order to fully replace GitHub.

This is a bit of a fallacy though. I don't use GitHub (not that it matters because I don't have collaborators) but there is plenty of competing software available, even better than GitHub in terms of actual product management, integration and deployment. The only advantage GitHub has is a popular website, devs that already have accounts, and social media like gimmicks. OneDev, GitLab, ForgeJo, GitTea, and probably a bunch of others have more than enough features. But no one wants to invest the time into running the servers. Which I get. I spend like 10-20hrs/week just doing server bullshit. Just say that, instead of there is nothing as good as a multi billion dollar company offering a SaaS product. GitHub is far from a "better" product, it's just convenient and free.

Vitor Pamplona 10mo ago

True on the other softwares... But the issue is not that nobody wants to run them, the issue is that by using them you just switch your overlords, from Microsoft to whoever is running those servers. It's virtually no improvement to the situation.

If they where on Nostr itself, though... Then we can talk. But we don't have good clients for that yet.

I think projects should get to own their infrastructure. Wouldn't call it "overlord" and that still not exactly an issue GitHub challenges or fixes.

It's not difficult to mirror anyone's repo either. I mirror dozens of projects, mostly for cache reasons, but just to offer mirrors. I can spare a few MB to mirrors repos for other people. It takes almost no time if you have git, any generic http server, (cron if you want to stay up to date) and some free disk space. The "overlords" can't take that away from us unless they go closed source and well that's not even what were talking about.

I still think that's a bit of a cop out though. Fighting for perfect when we can have good enough for now running our own servers and discussing issues and making proposals on nostr. Many of us already do that.

Exactly. This can and should have been done much sooner.

Then build one. You're a client dev. I know that git via nostr has been discussed and can be done if given priority.

Basically, stop telling other people to do it when you could do it. I can't, so I'm just whinging about it until it happens as I see the current situation as untenable long term and something needs to be done about it sooner rather than later.

running a gitea is easy, it even has some nice extras now that can do things while it is still one of the most svelte git http hosting implementations there is

i'm not sure how big a userbase has to get to actually need better than a like, 4 core, 16gb memory with a 500gb SSD - i have written a GC that enables you to make sattelite event store/relays and they could be linked to any kind of other data storage system including bigger, "master" relays that have yuge filesystems but are not built for high load (so they act as a big cache for smaller relays that have contained storage

i could build out something like that in a month or two if i had all that time free, i've got most of teh pieces already

also, SQL is a stupid way to store events when you don't need all that advanced search logic nor that many tables, plus it would be space inefficient due to all the extra indexes that come by default that are probably not possible to turn off even though the query logic doesn't need it

> also, SQL is a stupid way to store events when you don't need all that advanced search logic nor that many tables, plus it would be space inefficient due to all the extra indexes that come by default that are probably not possible to turn off even though the query logic doesn't need it

Sure, but it scales. I can stand up a dozen relays and load balance the databases with replicas. Might not be the most efficient, but I can make up for that with more of them. We need to be able to run dozens of relay instances off the same backend.

nosql and kv stores also have scaling strategies too, and use one of the several common replication protocols, you just been in this a while ;)

You might be right. But no relays are using scalable KV services though? At least none that i've seen?

nope but i know i could add a two tier fewer larg many smaller caching relays strategy it just requires writing one implementation of a library i wrote already and tested for a different second level "master" cache

I would really like to be able to "load balance" relays meaning they are stateless and use a backend that I can administrate on my own. I don't want to rely on the relay software itself to admin the database. Again where SQL wins. As an admin I can do optimization outside of the relay environment.

this is a bit like the debate about why bother enabling JWT bearer tokens (for dumb old epaper readers and postman btw) versus just use nip-98 (which doesn't have an extra expiry field)

i'm not sure what you mean by "stateless" relays, unless you mean they are dumb stores and the "master" pushes and pulls data from them, ie, it would subscribe to all new events from the slave relays, and then push them to the others

the tools to do this are not created but thety are also quite trivial, and can be built as separate pieces, you can have a replicator server that just subs on all the relays and pushes to the master, and the satellites can just forward queries when they come in and push the newly added updated events that came in from other satellites over the subscriptions that clients opened up for matching events

i think it would be better if you made them all simply replicators so each of the load balance relays simply pushes new events to the centre master and the master broadcasts them to all the others, instead of the master pulling them with subscriptions

subscriptions are cool and all, but broadcast is coolerer

it's all teh same to me though, point being that a relay IS a database server already

strfry can scale quite large.. larger than what anyone needs right now. you can setup replicas and load balance between them. if you wanted to you could shard them, but you would need a sharding proxy.

either way, i think nostr scales with more people running relays and using outbox, so thats my focus (not as much mega scale single relay DB because i dont want to be like bluesky)

Pleb relays is the way, but some of us are going to have to handle thousands of concurrent users (hopefully)

strfry uses and embedded db doesnt it? lmdb iirc?

How can I run multiple strfy instances from the same db?

It's also not just for scaling, but for redundancy too. Were talking experimental and poorly tested at best, software here.

yes, it uses lmdb. you can run multiple processes of strfry that use the same db directly on disk if you wanted that. to scale horizontally you can replicate the db and have a load balancer in front. sure its not transactional like mysql is, but it is eventually consistent like a larger nosql db would be.

well i'd have to share it across a nfs or cifs share which historically has been shit for things like that. Server disk space is expensive. I would much rather have "centralized" db servers in a cluster and have them available for services to connect to them.

Id like the instances to be deployable in an HA environment so the disk can't be shared since it's not on the same server.

you would not want to share the db this way no, you would use strfry protocol (negentropy/stream/router) to keep the replicas in sync. it would be ha. every node the same..

But that means the db would be duplicated on every node and I would be relying on strfry to keep things in sync. Outside of my ability to interact with it using standard tools like weve been using for decades with sql.

My issue is relying on a service that is designed to be a relay, also trying to be a database system. I think it's just too much to ask and can't be done well.

i don't know what you mean by "load balanced" to be honest

nostr relays are really just a database server in themselves

do you want to replicate the database, do you want to demand cache it, do you want to shard it? it's quite important what strategy you have in mind and whether that actually fits the use case

sharding is probably gonna need to use social graph and maybe combine that with geofencing

caching is the simplest, dumbest idea, where the relays that people use don't actually store much data but they proactively fetch new data and expire old data quickly to keep the space available

caching was the way i envisioned working it... you could extend it with broadcast so that new data is shared around immediately and then with the garbage collection, rapidly purged from the store when it gets no traffic

how you design that optimization depends entirely on how the data is going to be used, and propagated

inherently, replicated, RAFT/Paxos/pBFT databases ARE exactly replicas, you even used the word replica, you didn't mean replica because you just clearly said you didn't mean replica

replica is like a bitcoin node with the same blockchain, that's a replica, all replicated database protocols make replicas, and individual replicas don't have the option to decide not to store data arbitrarily

what you are talking about is a caching strategy, which means you want to do garbage collection and broadcast

there *are relays that use SQL.. like khaturu or ditto.

i know gleason spends lots of time and probably money on servers, performance tuning the queries 😁 nostr will absolutely slam sql databases and is typically much more expensive to run a sql backend than lmdb.

ditto uses postgress last i checked and seems like it still does. Closer I guess. khaturu seems to be the best option, although I have to manage a build myself if I want to connect it to my infra. Midly acceptable.

yeah, people seem obsessed with postgres, i think they must enjoy the pain. unsure if they can be adapted to mysql, probably not once theyre done tuning it.

there is also relays using mongodb..

Yes grain does. truely not a fan of mogo myself though. I guess I'm alone on SQLServer world XD. I use it for everything, but maria db is close second except it's column search sucks huge ass compared to sqlserver.

katuru has pluggable backends so it may not be too hard to add one..

yeah, except you get to have fun with fiatjaf's shitty concurrency and slow ass json codec along with khatru

i've firmly decided that after i finish this JWT bearer token stuff and implement the HTTP endpoints that all features cease after that and i build out a modular scheme for them all , and untangle all the entanglement

oh and i forgot to mention fiatjaf's shitty event store interface which assumes you want to deal with channels and several more goroutines than you need, which make an utter hellish mess

but have fun with htat anyway

i will try and build out a fully simple architecture to make it all easy to include or not include, very soon, it's driving me nuts, i made the first part nice, now feature adding, ok, too much, i'm getting claustrophobia