Sorry I find this extremely exciting, I have a few questions if you have time:
- Did you create the pre-computed graph using the iris frontend itself?
- What's the size of the included dataset and what's the size of a typical dataset a user would crawl? (1 MB, 100 MB?)
- Any feeling for how many entries you need to make it useful?
- Have you thought about "web of comments"? Or "web of retweets"? Could you combine different datasets into a big graph with different edge types?
I quickly checked the crawl data in my browser before but couldn't figure out what the entries mean. I'd *love* to analyse the data.
- Do you think it's feasible to crawl a global graph instead of "local" (for a certain user)?
The graph was crawled using https://github.com/mmalmi/nostr-social-graph . I used a node.js script from the repo, but the same stuff can be generated in Iris on the social graph settings page.
The repo has a Dockerfile for a server which keeps an up-to-date version of the graph, runs here https://graph-api.iris.to/ . On Iris build time, a 2 MB social graph dataset is requested from there. Should probably refresh on 1st iris.to open as well.
The graph up to approximately 3+ degrees is currently 200K users, 6M follows, 80K mutes. ~50MB. My macbook can easily handle it, but mobile phone will crash. Native apps on mobile should easily handle it.
The exported file contains uniqueIds: public keys mapped to integers for shorter representation. Then follow lists which are encoded using these integers. I want to do a binary export format: it would cut the file size approximately in half.
To crawl a graph you always need some follow list as entry point, but the graph by itself is "global", you can easily change the root point and recalculate distances.
If you crawl up to infinite distance from entry point, you might get lots of spam accounts that follow each other. Interesting topic for research and visualisation.
With a fresh 2MB graph, new user who follows 1 user from the graph gets this
Maybe you could combine multiple metrics such as likes, zaps, follows and comments into a numeric score for the edge
