So I ended up dropping Neo4j because DVMs should move to encrypted setups with NIP17 or NIP-EE, and I was using Neo4j for a global event debugger for DVM flows (which isn’t possible if everything’s encrypted).
But you can check out the representation I made and the code to insert and query Nostr data, could be helpful if you’re deciding how to structure Nostr data into graphs.
The main challenge as I see it is to ingest a high volume of events in real time. I’d love to see a general purpose nostr graph db relay, although I don’t think it will ever be as performant as non-graph relays for routine relay functionality.
Did you ever have any difficulties with real time data ingestion? Could you process them one at a time or did you need to batch them together at regular intervals? Or perhaps there was never a high enough volume of DVMs for this to be an issue?
Dude. Yes. Had to batch them basically. I was running on the low end of hardware, maybe just 1gb of ram, which is stupid expensive if you use the aura hosted version. Probably only worth it if you host Neo4j yourself on better hardware. I ended up doing a hybrid Postgres + Neo4j setup, so that I only needed to use Neo4j for queries that could benefit from the graph structure
What do you think of a hybrid solution? Last weekend I researched a few graphs to use. I settled on Cozo + sqlite. I think it solves the overhead you described. There's no need for batching. I vibed this once I picked what I thought was the optimal solution.
Cool. How big of a social graph have you achieved with it? I sync with wss://WoT.grapevine.network whenever I spin up a new brainstorm instance. It keeps track of kinds 0, 3, 1984 and 10000 just for this purpose. Should yield ~ 250k pubkeys connected via ~ 24 million follows.
Looks like what you’ve vibe coded is pretty comprehensive. All you’d have to add is NIP-85 support and you’d have a fully fledged Personalized Trust Metric Service Provider! Excellent entry for the WoTathon!🔥
I see that CozoDB supports PageRank, but looks like it only supports global, not personalized. Neo4j, memgraph and ArangoDB are the open source graph dbs that I know of that support personalized PR. Have you looked at ArangoDB? That’s one I have not looked into.
I'll have to try out a real world test this weekend and report back. I have a lot of Nostr catch up to do. So far the relay works fine but I have a bug in the sync to sort out. When is the deadline for the hackathon? I'm more interested in listening to presentation than submitting.
Our first community call will be Nov 20. Probably aim to have teams and projects lined up by mid January and run through until April.
btw I like the idea of a hybrid solution. Brainstorm is essentially a hybrid, with strfry and neo4j on a single server.
