nostr:npub1u5njm6g5h5cpw4wy8xugu62e5s7f6fnysv0sj0z3a8rengt2zqhsxrldq3 I'm trying to understand grapevine, but https://brainstorm.ninja/#/grapevine/calculateInfluenceScores gives me errors:
Discussion
Во первых - это красиво
I’ve refactored it - try here:
https://grapevine-brainstorm.vercel.app
(You’re reminding me I need to update this in my profile)
The original (brainstorm.ninja) downloads kinds 3 and 10000 events on the fly and stores them in the browser. The updated site (grapevine-brainstorm.vercel.app) maintains those events in sql on the back end.
I have long believed that graph databases and WoT are a match made in heaven, so my next iteration will use neo4j on the back end.
If you can get grapevine-brainstorm.vercel.app to work for you, it should generate and store on the back end two “WoT networks”: one defined as any user less than N hops away (by follows) a user is from you, and one defined as any user whose Grapevine WoT score is above some threshold.
The site will allow you to explore these two WoT networks in the form of a table that shows the Grapevine WoT Score (“influence”) and the DoS (degrees of separation score, ie number of hops away from you) for each pubkey. You can filter and sort which means you can do things like filter on all pubkeys 5 hops away and order them by influence score, as in the second image below.
My WoT network has just under 160k pubkeys and goes as far out as 8 hops away:
So you're saying that Nostr has about 160k users?
Yup, with caveats: my existing backend database is probably incomplete and is not actively scraping new kind 3 and 10000 notes. Once I get the next iteration running (with neo4j) and actively scraping data, we’ll have a better picture.
And of course, my data only includes pubkeys connected to me. I have no idea (but am very curious to know) whether there are other disconnected nostr networks of significance — small towns in China, for example?
176473 users.
But what if one of these accidentally follows a bot, then that bot follows 10000 other bots, will the number of "users connected to me" grow by 10000?
What is a minimum viable "influence score" to filter out bots and other crap?
Filtering by minimum of 0.1 gives me 65839 users.
Probably the best way to define your WoT network will be to set some low influence score threshold - maybe 0.1, maybe even lower. Something we can play with. The bots like in your hypothetical probably have a score much less than 0.1 although I won’t know for sure until my next iteration which I hope will be much more performant than what you see now.
The GrapeRank algorithm is kinda like PageRank, except that the PageRank score has no upper limit, whereas GrapeRank produces a score that maxes out at 1. Imagine mapping PageRank from the interval [0, infinity) to the interval [0,1] where it approaches 1 asymptotically.
The advantage of the Grapevine WoT score is that it is ideally suited to be used as a weight in calculations like weighted averages or voting. This is not true of the PageRank score.
My current roadmap:
Product 1. “Google for nostr; PageRank for pubkeys.” Similar in some ways to nostr:npub1xdtducdnjerex88gkg2qk2atsdlqsyxqaag4h05jmcpyspqt30wscmntxy ‘s nostr.band’s keyword search. Maintains a centralized nostr cache that uses neo4j (a FOSS graph database) and calculates personalized PageRank scores, GrapeRank scores, and DoS scores.
Product 2. Personalized Grapevine WoT relay. Alternative to nostr:npub1utx00neqgqln72j22kej3ux7803c2k986henvvha4thuwfkper4s7r50e8 ‘s WoT relay, except that the WoT network is defined using the GrapeRank algo, ie: your network = all pubkeys with influence score above 0.01 or whatever threshold works for you. Will use #neo4j and will borrow a lot of code from Product 1. Two options: 1. Fork nostr:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6 ‘s relay written in go (which is what utxo did I think) or 2. make a plugin for strfry. (Maybe try both?)
Product(s) 3. Make tools to make it easy for devs to crank out new interpretation protocols and craft new Worldviews, as presented here: https://grapevine-brainstorm.vercel.app/#/worldviews
#wot
#grapevine
Products 2 and 3 are FOSS and all-out full-blown cypherpunk, imho. But VCs will support them because from a VC perspective:
Product 1 is a centralized service and will be how the VC makes money.
Products 2 and 3 will add value to nostr and increase the nostr user base, thus increasing revenue from product 1.
You don't have to fork khatru, you use it as a library that does the annoying plumbing and you just sprinkle in your magic querying on top to build custom relays.
You could probably do in 5 min what would take me a month 😅
I'm also interested in using grapevine in some experiments here. I'll try to implement it myself soon.
I would love to see that! lmk what I can do to help 💜 🍇
lmk if you have questions about how the GrapeRank algo works.
Lately I’ve been thinking that a potentially faster way to calculate something similar to GrapeRank would be:
1. Calculate PageRank
2. Map the PageRank score from [0, infinity) to [0, 1) — see the graph in my habla article below.
I don’t think that would be identical to my existing GrapeRank algo, but it would probably be a good approximation. Haven’t thought it all the way through. This would be easier to implement if you’re already familiar with PageRank and have access to software that implements it; and may be computationally faster too, since GrapeRank is iterative.
You speak as if I knew how to calculate PageRank and that was supposedly easier than Grapevine.
In my understanding you can calculate Grapevine by just iterating through Nostr connections sequentially through levels, right? Meanwhile PageRank involves some advanced matrix math I don't understand.
GrapeRank requires calculations that are about the same level of complexity as PageRank.
There are several ways to implement PageRank, one of which uses matrix algebra. The conceptually easier (but less computationally efficient — which doesn’t really matter for our purposes) doesn’t require any advanced math, although it is tedious to code.
The way I currently calculate GrapeRank is the same: tedious to code but nothing more mathematically complex or advanced than weighted averages.
And you’re correct: to calculate GrapeRank, you just do lots and lots of iterations. For each iteration, you calculate the GrapeRank score of one pubkey. You have to iterate through every pubkey multiple times until the scores don’t change, or the changes are below some threshold (I think 10 or 12 iterations was enough in my most recent implementation).
nostr:npub1xdtducdnjerex88gkg2qk2atsdlqsyxqaag4h05jmcpyspqt30wscmntxy uses something very much like PageRank to stratify search results for nostr.band, and it works pretty well imho, as he discusses here:
I’m not aware of anyone else using PageRank in nostr. (Although surely there’s gotta be someone?)
maybe even iris, https://graph.iris.to/
another interesting thing
Cool, ty for the links.
Although those don’t use PageRank, do they? Unless I just missed it …
I am trying right now.
btw nostr:npub10npj3gydmv40m70ehemmal6vsdyfl7tewgvz043g54p0x23y0s8qzztl5h and nostr:npub1manlnflyzyjhgh970t8mmngrdytcp3jrmaa66u846ggg7t20cgqqvyn9tn have each independently built their own implementations of GrapeRank.
Independent is a strong word. We’ve been in touch and collaborating the whole time.
The GrapeRank engine I’ve made is intended as a library for other search engines and clients to implement. It has Interpreter, Calculator, and Storage components.
- The Interpreter requests content (like follows, mutes, likes, ect…) from any network (Nostr or others) and normalizes these for calculation, as per existing (and extendable) protocols. It receives configs for this from the engine request.
https://github.com/Pretty-Good-Freedom-Tech/graperank-nodejs/tree/main/src/Interpreter/Nostr
- The Calculator is where the algo lives. At its heart is a ScorecardCalculator class with sum and calculate function for each user or content to be “scored”. Sum collects values from interpreted content and calculate returns the final score.
https://github.com/Pretty-Good-Freedom-Tech/graperank-nodejs/blob/main/src/Calculator/index.ts
- The Storage component can be extended to use any storage backend (including Nostr events for relays, once we settle on a standard schema and NIP proposal). It stores and retrieves calculated lists of users (or content) and list configurations (called worldviews) for quick retrieval.
The “WoT” base level “grapevine” list (of a users entire “not spam or bot” network) takes about 5 minutes to calculate.
As we move forward, “worldviews” configured by users will be able to input their grapevine (or any other) list to create more specific lists that will be able to be dynamically updated.
- most recommended News bots
- Nostr developers
- popular clients
- Nostr artist
- ect…
My Grapevine is an evolving demo of some of this functionality.
nostr:npub10npj3gydmv40m70ehemmal6vsdyfl7tewgvz043g54p0x23y0s8qzztl5h implemented the GrapeRank algo in go as a first step towards a personalized Grapevine WoT relay.
Fascinating thread. Devs talking devvy.
I'm keeping this for good old times. Also the game fiatjaf is playing...cool!
Click on the edited word.
How is that even possible?