a Relay Test Suite is in the works...

I've already got ideas for 94 tests and as I write these tests they expand out into even more tests.

I hoping that exposing relay behavior will drive relay software towards consistently supporting the same way of doing certain things, and that this will help nostr compatibility.

Tests include things like:

* Can the public write events?

* Can an authenticated person write somebody else's events?

* Does the relay respond to minimally constrained filters?

* Does limit 0 give an EOSE?

* When you write an event and get OK-true, can you read it directly back?

* Does limit behave properly when a filter has multiple pubkeys and multiple kinds?

* Does it require AUTH to read DMs and giftwraps?

* When submitting a replaceable event, are the prior events replaced?

* When submitting a replaceable event with newer ones already present, is it rejected or ignored?

* Does deletion of a replaceable event preserve the events that are newer than the deletion event?

* Does it prompt for auth initially?

* How large of an event can it handle?

* Does it verify signatures?

* How does it react to invalid UTF8?

* Does it handle all JSON escape codes? What about surrogate pairs?

* Does it preserve field order in events?

* Does it preserve fields in events not defined by the nostr spec?

* How many subscriptions does it allow simultaneously?

* Do events with negative created_at values work? What about exponential notation? What about numbers that won't fit in a u64?

... that kind of thing.

If you want to help contribute, even if you just have an idea of what to test, come on over to https://github.com/mikedilger/relay-tester and give feedback, open an issue, or better yet submit a PR.

What about the other things I'm working on?

* Gossip is on pause while we wait for the UI developer to handle some personal affairs, then we will have annotations and NIP-17 DMs.

* Chorus is stable and upgraded to use Pocket

* Pocket has been completed and is stable too, but I haven't tried to make it useful outside of the scope of a relay yet.

This is amazing.

Reply to this note

Please Login to reply.

Discussion

This "will be" amazing.

Those tests aren't written yet. It only does 18 tests so far, the rest are placeholders, and I'm still doing major refactors which are much easier with less tests.

The fact that you have a list of things is already amazing!

I waited about a year for somebody else to do it and eventually got tired of waiting. Then when I started thinking about tests I couldn't stop. I couldn't sleep I kept getting up and writing down more test ideas.