Nostr Web Client

omg. fucking sometimes i really hate the #golang devs.

so, they make this json library. it FORCES output to replace < & and > with unicode escapes becos muh XSS security.

you can't disable it no way no how if you want your calling functions to work with common nostr event encodes because they can contain these characters and then they get mangled by json.Marshal in such a way that it destroys the integrity of the event ID when converted to canonical form.

it can only be done by creating a new json.Encoder with the html escaping disabled. the standard library version that you get when you call json.Marshal breaks any events with any of those above characters.

it was mainly irking me because i was concerned that maybe https://huma.rocks did not handle this correctly, but it does seem that they have got that covered in the outputs so in theory it should then be possible to use my custom event type (which is very custom, everything is []byte except for kind and created_at) which json encoder normally mangles as base64 encoded data. even though it is actually json compliant unicode.

so i think this means i can use the nice openapi library, and pass it events, and it will use my encoder and it uses json.NewEncoder with html escaping disabled. good, because shit, i mean, probably the large majority of go devs working with JSON are NOT feeding these to be displayed on a web page (and exposing a probably archaic XSS vulnerability), they are sending it to custom written code that - like nostr clients, is probably automatically doing this transformation if it is necessary to prevent the stupid html parser from letting people do XSS attacks using json events.

i do really wonder whether this XSS vulnerability even exists in any browser version for data tagged as application/json mimetype at all anymore. but it's still baked into the goddamn json library in go.

i may have to investigate the v2 library that comes with go 1.25. maybe they finally fucking fixed that. they probably didn't change it because of their API compatibility promise, but a v2 would allow them to do whatever they like.

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ 4mo ago

i ran into this issue ages back and forgot about it and anyway just went ahead and created my own custom marshal/unmarshal API that was designed to append output to provided slices, so they could be spliced together such as a simple array encoder that spews its results onto a buffer with multiple other encoders with the same API.

you can't do that with the standard json.Marshal/Unmarshal anyway. in fact the tag codec i wrote, each tag must be constructed using this method to mesh with the custom parser state machine i made (this mainly affects unmarshaling) but underneath the level of implementing the UnmarshalJSON (and MarshalJSON) it already bypasses these things, and i can trivially implement these anyhow except of course using them on tags that contain these characters will not work with json.Marshal anyway.

oof. the main encoder available without further dependencies in Go has a serious flaw that impedes usage for nostr. this is why you will notice that all of the #golang #nostr relays use some kind of custom json encoder.

ah well, it's not so bad. this is just one of the unfortunate issues that preventing user generated XSS attacks in json sent to a Go server that renders the json into a browser due to FUCKING RETARDED BROWSERS AND THEIR ABSURD COMPLEXITY) so whatever. you can't use json.Marshal with nostr. the end. and they all lived happily ever after.

Reply to this note

Please Login to reply.

Discussion

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ 4mo ago

also, it was just a bad design, locking json library into being for embedding into HTML like this. they refuse to change it but they don't mark it deprecated. it should definitely be deprecated.

no idea if they have fixed this mistake in v2 but i'm betting they did. meh, anyway, so i put some loud warnings into the documentation, i don't want someone to have this issue trying to use my fast custom json encoder and wondering why it doesn't behave properly.