Nostr Web Client

There's a lot of variation on how events are connected in Nostr. It'll take a lot of work just to get the basics down.

This is just using "e" and "p" tags as references, and I already found, just from a random query, that Gossip references both the root and the direct parent with "e" and has a value at the end of the tag to specify which is which. Keep Nostr weird I guess.

Not to mention references that only exist in the content via nostr: or @ constructs. But hey, as long as the data is there, Neo4j can handle it.

nostr:nprofile1qqsw2feday2t6vqh2hzrnwywd9v6g0yayejgx8cf83g7n3ue594pqtcpr3mhxue69uhkummnw3ezucnfw33k76twv4ezuum0vd5kzmqprfmhxue69uhhyetvv9ujuarpwpjhxarj0yhxu6twdfssz9mhwden5te0dehhxarj9enx6apwwa5h5tnzd9aqhdwwcj also, I've written a batch unwind import in Go and found that it's so fast with the right batch size and indexes that you're only limited by your IO speed in getting the events over the wire.

Reply to this note

Please Login to reply.

Discussion

Jay 10mo ago

Also nostr:nprofile1qqsp3yzapfwkyw4cr2vt4xx9s27474lj2pkxhqyfqh79n826pv3fkzqpz4mhxue69uhks6tnwshxummnw3ezumrpdejqzynhwden5te0wp6hyurvv4cxzeewv4eszenhwden5te0ve5kcar9wghxummnw3ezuamfdejj7mnsw43rzun6vuunv7n2v9mxwct5wdur2cmgxfm8var3x3shgct5d3un2unkv3mhzem2wqc82arcwu6r27n90fh8v7t3vejxk7rkv5lkyun0v9jxxctnws7hgun4v5vdu9e6 that selected node is you. Fun fact.

Jay 10mo ago

This is what you look like as a node.

node 10mo ago

This is cool!

Signed means events that I signed? What do referended mean?

Jay 10mo ago

Signed means that you authored the note. And references are when my event references yours in a tag, like with this reply. This kind of thread is what's being shown in the picture.

Your note there is this one:

nostr:nevent1qqsdm28u9e3rpw6hu8rp4xygun940d207qs88ctyfcdc9a69r02qvmghkjnwq

node 10mo ago

Interesting. What will this be used for?

Jay 10mo ago

This is going to be a web of trust relay, eventually; where you trust the people you most interact with.

And the way the data is stored, with these nodes and relationships, makes that calculation possible.

node 10mo ago

Ah similar to utxo’s relays. Cool

david 10mo ago

I believe we’re going to see the idea of a nostr personalized webs of trust relay evolve as we develop more and more sophisticated tools to calculate our webs of trust. (“Webs” = plural!)

First generation, the state of the art: WoT = my follows + their follows (with maybe some simple tweaks)

Second generation: centrality algorithms like PageRank. Powerful, but not contextual.

Third generation: centrality algorithms designed for context-based trust-weighted averages and summations (I call these “GrapeRank”)

Jay 10mo ago

And I'm just skipping straight to 4th gen which is web of trust as calculated solely by engagement.

Btw, just because this just happened on my screen, but around 1.6M events ingested in just over 3 minutes. That amounts to 2.2M nodes and 5.6M relationships. But it'll be many more relationships as I add more per event.

david 10mo ago

You loaded 2.2M nodes and 5.6M edges into neo4j in 3 minutes??!?!? 🤯

Jay 10mo ago

Yeah haha. Each batch transaction is in the ms range, but that's because there's some very good presorting going on and the right indexes. Right now my repo is just on my private server, but I'll mirror it on GitHub too 👍

david 10mo ago

How many edge types are you currently maintaining? FOLLOW, MUTE, REPORT? Which is a tiny bit more complicated bc you’re not just adding data, you sometimes have to remove relationships too. Whether that impacts the efficiency of the database updates, I don’t know.

Jay 10mo ago

Only very base spec stuff so far, with no abstractions like replies or quotes and such:

User -SIGNED- Event

Event -TAGGED- Tag

Event -REFERENCES- Event

Event -REFERENCES- User

I'll have to handle removals eventually too but only in relation to ephemeral and addressable events. You have to understand that I'm approaching this from a completely different direction. I'm not going to base anything in the web of trust calculation on follows or mutes, and the calculation will be locally incremental rather than global like pagerank. I probably won't even need graph data science for the MVP.

david 10mo ago

Pagerank can be calculated globally or personalized, and personalized PageRank is what I’m calculating at grapevine-brainstorm.vercel.app

GrapeRank likewise can be personalized or global, and personalized is the way to go.

Personalized means that one (or in theory can be more than one) node is assigned privileged status. The idea of using centrality algos for a personalized WoT relay is that all of the centrality algos will typically be personalized using the end-user’s npub.

I would classify your trust score calculations as personalized centrality algorithms, if I understand correctly what you’re doing.

david 10mo ago

I want to see LOTS of different methods used to calculate WoT. Your method using engagement being one of them. Which is why I like the terms “webs of trust,” emphasis on the plural: webs, because there are LOTS of ways to calculate WoT. There are different types of “trust,” different applications for the scores we calculate, and different people have different ideas on how to calculate scores for any given application.

Jay 10mo ago

I also have a powerful PC, but even on low end hardware, it shouldn't be more than an order of magnitude slowdown (30 minutes), which would be my guess as to the upper bound on how slow it should be.

david 10mo ago

👀 thread nostr:npub10npj3gydmv40m70ehemmal6vsdyfl7tewgvz043g54p0x23y0s8qzztl5h

david 10mo ago

Do you have the code up on GitHub?

I’m writing bash scripts that update follows, mutes and reports as they come in. For starters, the graph will have NostrUsers and that’s it, just to keep things simple. Then neo4j graph data science to calculate PageRank, which will feedback into strfry plugins and filters by periodically updating files of whitelists (and blacklists once we move beyond PageRank).

I think we’re going to need specialized scripts to organize the data into a nostr knowledge graph (NKG), and I think they’re going to need to be pretty sophisticated.

I propose we start to think deeply about the rules for organization of a personalized nostr knowledge graph. We’re going to need code to do it, and the code will need to be open source. Let’s think about putting together one or more repos for this purpose.

We can call the repos: BRAINSTORM!

🧠 🍇 ⚡️ 💜 😃

Anders 10mo ago

"root" and "reply" marked e tags are part of the spec

https://github.com/nostr-protocol/nips/blob/master/10.md