Avatar
Mike Dilger ☑️
ee11a5dff40c19a555f41fe42b48f00e618c91225622ae37b6c2bb67b76c4e49
Author of Gossip client: https://github.com/mikedilger/gossip Dual National (USA / New Zealand) My principles are Individualism, Equality, Liberty, Justice and Life

nostr:npub1l2vyh47mk2p0qlsku7hg0vn29faehy9hy34ygaclpn66ukqp3afqutajft, I'm adding a relay usage called "discovery" to gossip where you can mark a relay as a directory for kind-10002 events. In testing it, I set wss://purplepag.es/ to discovery. When I ran it, it subscribed to everybody I follow, kind 10002. I got back just 1 event (hodlbod's) and then EOSE. I'm presuming purplepag.es isn't ready yet, is this correct? Maybe you were waiting on a client that could use it? BTW I have 57 relay lists (kind 10002) in my local client database for the 132 people I follow.

I'm getting pushback. Rather than reply to individual comments, my point was that they don't have free speech and so you can't be sure if they are telling you the truth, or saying what they need to so they don't get arrested.

I made a similar comment about Ukraine right before the parent post.

I'm getting pushback. Rather than reply to individual comments, I will say that my point was only that they don't have free speech, and if they aren't free to express their actual opinion, asking them isn't necessarily going to get you their actual opinion. For example nostr:npub1pyzvmpujlpcy9whydlcay3gkm02wu0flehua3afdwqt22yqt33cqe6k0h8 gave a 50/30/20 breakdown where ZERO people expressed the illegal opinion.

I made the same point about Russia in a post right after the parent post.

I have a feeling the average I.Q. in Oamaru just went up. It might become more techy.

I tried to change too much code at once, and now I have a mess. It is hard to split up because it is a database change that necessarily changes lots of code. 😕

Nobody actually knows what Russians think about the war between Russia and Ukraine.

It is illegal for Russians to criticize Russian military operations in foreign areas.

Nobody actually knows what Ukrainians think about the war between Ukraine and Russia.

It is illegal in Ukraine to freely express an opinion on the topic. Your opinion is dictated by the state. A journalist was just arrested in Ukraine for posting online justifications for the armed aggression of the Russian Federation against Ukraine.

When posting an event and getting back a success, new code is recording that the event is on that relay. So the 'seen on' information visible by hovering over the eye will include all the relays it successfully posted to. Previously it was only getting that data if the event came in via a subscription.

There was a race condition between writing the event and writing the event_relay (seen on) data that caused a FOREIGN KEY failure. The way the code is structured I cannot easily fix it, so I instead set a PRAGMA to disable foreign key checks for this, in case it saves the event_relay record before it saves the event record.

Took a few tries to get it right.

Replying to 21823843...

Got it, makes sense. I think if you make sure the event is written before updating the DB then that would be safe, as you describe. It still might be worthwhile to check if there is any performance difference between the parallel file and storing the validated event JSON in sled DB along with the indices. If it works like LMDB then retrieving it will be functionally identical (an offset into an mmap), and an event write will require only one fsync, not two. It also might save you a lot of trouble later on, when it comes to backups and periodic rewrites you mention.

About the indices: No, I don't think it's a dumb algorithm. It's actually a really difficult problem, to figure out which index or indices to use in a scan. strfry has all the indices you mention, also composite with created_at. Additionally, there is a (pubkey,kind,created_at) index, which I think is useful because I believe queries like "get contact list of user X" are quite common.

strfry currently never joins two indices together or does a set intersection as you describe -- its algorithm is even dumber. It only ever uses 1 index. If the query can be fully satisfied by the index then it doesn't need to load the flatbuffers (most queries are like this). But if it does need to load the flatbuffers this is when it's important that accessing them and applying filters are fast. Optimisations for this code path are also beneficial for the live events filtering (ie post EOSE).

Your approach sounds like it might be quite effective. The only thing I'd watch out for is if your IO patterns are excessively random. The nice thing about working on one index at once is that the access is largely contiguous data. Even several contiguous streams is probably OK, but some queries have like hundreds of pubkeys in them (for instance).

I'm sure you are right about just using sled instead of doing my own mmap. But I feel pride that it seems to work so I'm gonna leave that code for a while. Once it bites me, I'll do what you suggest. I've got a compiler fence so it doesn't reorder the writes, but I had forgotten to put in the msyncs!

Ok that seems to have worked.

fourth time's a charm?

Use the latest one. As nostr:npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s says, people have to be able to remove data, and you should presume that is what happened rather than presuming it was done in error.

Replying to Avatar Andrew

nostr:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6 here's bsky team member Paul Frazee's response to your blogpost on Bluesky, if you're curious

I just read fiatjaf's recent blog post, and then this. If these posts were supposed to make bluesky sound better than fiatjaf made it out to be, well, they sure didn't.

Nostr is my home. There will be bridges I'm sure.

Back in 1994, having been excited about public-key cryptosystems and into collecting Magic the Gathering cards, I conceived of a computer game where people would own something similar to collectible magic cards, and trade them, but they would actually be files on your computer. The file would include the game description, have a unique ID, and be digitally signed by the manufacturer. People would play by bringing their files to a LAN and play on a PC game which would contact the manufacturer's online service to validate if someone really still had an item. I stay "still" because you could trade (the manufacturer would track ownership) if you lost the duel, you would lose a random item to your opponent, maybe your pauldrons, or a WIS+2 ring (I was imagining D&D type items, not MtG cards, I didn't dain to steal their game). In order to make them feel more real, I imagined a Java-based animated image, instead of just an image, and you had to have the file so people would collect the files themselves and it would feel more like a real-world asset rather than something on some MMORPG server somewhere. You load them into the game client, it validates them and marks the bad ones (the ones you lost or traded away), lets you inspect and animate the ones you own. Then you pick your setup and dual.

That was all before blockchain. When blockchain became a thing, I thought about reviving my idea, but I didn't.

I always thought the thing had to have some feeling of value, like being valuable in a competitive game like MtG cards are. I never imagined that NFTs and inscriptions without any other sense of value would become a thing. If I thought it was going to be that easy I would have done it long ago.