lol

what you don't understand is that CBOR has a lot of extra metadata in it and we have exactly two primary data types, the event and filter, and the envelopes, which are mostly just a few fields and an event or filter or array of such

the extra data wasted by this and the processing time required on both ends of the process could be left out

i say that if we have a binary wire ercoding for nostr, it should be one that is designed precisely for our purpose

you can literally encode events and filters without any metadata fields because they are such simple structures

you probably know what the structure for a canonical event array, the `[0,"pubkey",...]` structure, this is the event without the detachable elements

just make the same thing except in hex, put a versioned magic string at the front, and the fields are either varints for numbers, binary for hex, and length prefix for strings, and you know exactly what they are... the tags the first value is a varint representing the number of tags, and then each tag has a number of fields prefix, and each field has a number of characters prefix

it's so damn simple it's insane to use some complicated nonsense that is going to be 100x slower and use more data on the wire than a simple binary format

Reply to this note

Please Login to reply.

Discussion

https://github.com/mleku/realy/blob/dev/event/binarymarshal.go

this is my binary encoder and this is the benchmarks for it:

https://github.com/mleku/nostrbench

goos: linux

goarch: amd64

pkg: github.com/mleku/nostrbench

cpu: AMD Ryzen 5 PRO 4650G with Radeon Graphics

BenchmarkEncodingEasyJSON-12 687991 1790 ns/op 1631 B/op 6 allocs/op

BenchmarkDecodingEasyJSON-12 676521 1777 ns/op 1413 B/op 16 allocs/op

BenchmarkEncodingGob-12 184216 5982 ns/op 4872 B/op 43 allocs/op

BenchmarkDecodingGob-12 55344 21500 ns/op 10061 B/op 236 allocs/op

BenchmarkEncodingFiatjafBinary-12 99108 11383 ns/op 73789 B/op 1 allocs/op

BenchmarkDecodingFiatjafBinary-12 1370816 863.9 ns/op 769 B/op 10 allocs/op

BenchmarkMlekuMarshalJSON-12 1000000 1149 ns/op 0 B/op 0 allocs/op

BenchmarkMlekuUnmarshalJSON-12 609270 1951 ns/op 684 B/op 13 allocs/op

BenchmarkMlekuMarshalBinary-12 2982892 402.2 ns/op 600 B/op 2 allocs/op

BenchmarkMlekuUnmarshalBinary-12 1571326 767.5 ns/op 1096 B/op 14 allocs/op

as you can see, it is as fast as it can be and you can see reading the encoder that it's 250 lines of code to implement it

you can also see that my custom made JSON codec is only about 3x slower and faster than the easyjson that has a massive 1000 lines of generated source code to enable it

when you understand those numbers there you can see why i am skeptical there is really any need for it, it's the least expensive operation

but what i will point out is that between the json and my binary encoding, part of the reason why it is so fast is because my runtime encoding also uses binary instead of hex, there is only a hex en/decode step to go to the wire

This is really cool. I see exactly what you're saying now.

This (as I think you said earlier) could just be a simple optional NIP that relays could signal support for and they'd handle the data however they want on their side right?

We can have a version field also.

Version 1: just normal Nostr events, signed as usual

Version 2: fields can now contain non-UTF8 content, new signing scheme