Nostr Web Client

Ok, so if RSS and nostr have some similarities, the discussion becomes, why can't we just host all the feeds as nostr notes?

You could, but what are the unintended consequences, and at what point does nostr end up using the solutions that podcasting has already figured out?

For example, there's over 4 million podcasts, 400,000 active ones.

Search and discoverability is a big deal, how does nostr solve that without having some sort of searchable Index of all of the podcasts that are spread out amongst all of the relays?

An index of 4 millions podcasts costs about $900 a month, so if it's going to be decentralized, each index is spending $900 per month.

The question for me isn't so much how can nostr replace RSS, but how can RSS adopt the ideas of nostr to be more decentralized and robust than it already is. I think it's easier to improve something with 20 years of precedence and reliability and infrastructure than it is to try and replace it with something else that will bring it's own unique set of problems.

Remember, there are no solutions, only tradeoffs.

I'm not fully versed but as I understand for search there's two components? There is the index list and the actual file hosting?

Can we simply start by indexing on nostr? It's just a glorified text file no? That will immediately decentralize that portion of things. A single index provider can spin up an npub and just publish the latest list anytime there's a change. This can be further decentralized with multiple indexes doing the same thing.

A single master indexer could write to multiple PC2.0 specific relays for redundancy. You could have several indexes doing the same thing if you wanted to be more immutable. Similar to how bitcoin's ledger is replicated across all the nodes.

End user facing apps simply need to connect to these relays to access the feeds?

I'm saying all of this will little technical knowledge of what's truly going on in the background so I could be entirely full of shit. But this seems to be a simple starting point?

An index is just a database that has the links to the RSS files. It has other stuff like podcast title, author, etc. to make searching for a particular podcast easier, but it really is just a searchable list that points to all of the RSS feeds that are spread out amongst thousands of servers.

So yes, you could start by indexing all of the notes that are feeds on nostr, having the title, author, etc to make searching for a particular note easier, but the issue becomes when that single index is now Indexing 4 million notes, and costs $900 per month. It can be further decentralized with multiple indexes doing the same thing, each at $900 per month. But it this point, it sure looks just like a traditional podcast index, just pointing to notes instead of RSS feeds.

OK so... an RSS feed is basically a big tagged text file. Nostr notes can certainly do that. So rather than storing individual feeds on a centralized server, they can be distributed by nostr relays. Specific PC2.0 relays can be created to house this data and multiple relays provide redundancy. You just have to make sure you're publishing to these pc2 specific relays.

For example if I sign up for rssblue, i hook up my npub or spin up a new one. That's my artist account. All input fields remain the same. When I publish, it pushes a nostr note of the rss feed I just created, to the pc2 relays.

At that point you wouldn't need a single master index since everything lives on the relays. Any application wanting to access that data just needs to connect to the relays

Am I missing something fundamental here?

Running an entire network of pc2 relays I imagine would be significantly cheaper than $900/mo.

The Podcast Index costs them $900 a month to index and serve all of the podcasts. The amount of data being sent with 4 million feeds is surprisingly expensive. Perhaps sending it across relays is different, but I suspect bandwidth is bandwidth whether it's across a websocket or http.

But in this instance it's not a single provider taking on all the cost. With a sufficient number of reliable relays in the network, each relay only has to house a portion of the data.

The total cost can be distributed across that network, lowering costs for each individual entity involved. Apps would only need to pull the requested data adhoc

I'm not sure how it would work. That's ignorance on my part, not that it's a bad idea, it's a great one if it works. If I'm searching for Ford Pintos, which relay is responsible for storing the podcasts on Ford Pintos, and how do I know which relay to search? Do I just search on every single relay and hope one of them sends me the data I needs? If you need a master relay to direct all the other relays, is that master relay now a single point of failure?

Again, not bashing or disagreeing with the idea, just working with it.

I'd imagine you only really need a handful of relays to do all of this. I'm connected to around 20 relays for normal nostr notes. I can't imagine you need that many across the rss ecosystem. How many notes are served across the 20 largest nostr relays? (Idk the answer but finger in the air I'd imagine it's comparable to total # of rss feeds?)

If podcastindex were to sponsor say 5 relays with that 900/mo, they could set it up as "primary" relays each with a different genre. Individuals can self host their own relay and can probably filter feeds they wish to include, almost like playlist curation if they wanted to get granular like that. Any app could connect to any relay. Someone can easily run a simple website posting a running list of all relays if you wanted to.

This would help organization and lower data costs as you could spin up a lightning thrashes app that only connects to metal relays for example.

The downside is you might not have a single full accurate picture of the entire ecosystem, but there are very little practical instances where a user needs access to the entire library at once.

Yeah, feeds aren't stored across a centralized server, they're stored on thousands of servers all over the world. You could have a podcast feed on an Raspberry Pi in your closet. The index doesn't store the feed, it points to the location of the feed. It's like a phone book for feeds. It lets you look up where to find the actual feed on the internet.

So, even if you info was on a relay instead of a server (relays and servers are very similar, both transmit and store data, just a different delivery mechanism), you still have the problem of how does anyone find your info. If you're doing a podcast on Ford Pintos, and they don't know you, how do they find you? That's what an Index is for.

OK i see. So this index is just a running list of available feeds and their locations?

Rather than serving the full list, can't that be pushed to apps adhoc? So if I'm on a pc2 app, I want to look up Ford pinto podcast, i query that term. The app then goes to the relays its connected to and searches for that keyword and returns results adhoc. No need to serve the entire list and no need for a single index as long as the app is connected to the relays. Maybe you just need an index of relay addresses which would cost basically nothing.

Yep, that's all an Index is doing.

And yes, it could search relays instead of the index server. It needs to know which relays to search. It's not highly efficient. I search across 100 relays, I may get 100 results back, and I have to parse through all those results, combine them, and remove any duplicates. It's all doable, but it introduces another set of problems.

Duplicates would be easy to remove as the notes would be digitally signed and can be uniquely identified. Idk how it happens behind the scenes but I'm sure amethyst and damas and all other existing client apps do this

They do, and it's inefficient and wasteful. If you want to see a note, you ask for it from 20 servers. Those 20 servers will send you all the same note. The bandwidth on 19 of them was wasted. Bandwidth adds up to real money. That's just a side effect of how nostr works. Maybe the payoff is worth it, but I think it should at least be understood as one of the tradeoffs, and perhaps there's a solution to it worth exploring.

I wonder if the relays can be tiered? So perhaps... client connects to just 3 main relays. Each of those 3 relays basically sort and filter from 10 each. Yes there is still duplication but less in total from a per-relay point of view.

I've been playing around with building a nostr client so I can understand nostr better, and when I'm searching for a particular note, I do a serial search in which I search the first relay, and if it returns the note, I stop the search, otherwise search the next relay, so there are ways around it to one degree or another. That's the 'fun' part of being an engineer of sorts is seeing problems, thinking of solutions, then seeing the problems those solutions created and seeing which of those you can 'fix'.

I've also thought about that, a smart relay that connects to other relays and dedupes the data before sending it to the client. At that point though, it starts to move away from dumb relays, so may be moving away from one of the core ideas of nostr.

The way it's sort of framed in my head is this....

Back in the brick and mortar days you had individual record stores. Some were general music, others were specialized. I had to jump in my car and visit these stores. I certainly couldn't find the entire library of all music in a single store, but if I visited every record store in the cou try, I probably could get close.

The record stores are the relays hosting feeds. My car is whatever client app I'm using, connecting me to various stores/relays.

It's not the most efficient nor does it provide a whole picture. But as a user, I can only listen to one song at a time. I don't want access to everything all at once. I want to find specific songs or shows upon demand, or else sort and shuffle by genre. I think this enables it without providing more than what's realistically necessary from a end user standpoint.

The key difference is what you mentioned, the signature. It allows Nostr events to live everywhere since thier authenticity is proven by the signature

RSS does not have this so it relies on the domain to prove its authenticity

Besides that the are functionally the same, except that RSS has a long proven track record and 10x more integrations than Nostr

That said I don't think the solution is to just copy RSS into Nostr, or replace RSS with nostr. There are ways Nostr can be integrated with RSS without replacing it and breaking all the existing RSS interactions

I also have to point out that RSS is XML, which any self respecting dev should shun like the plague :D

nostr:npub1v5ufyh4lkeslgxxcclg8f0hzazhaw7rsrhvfquxzm2fk64c72hps45n0v5 is a good example of how nostr integrate with RSS and make it better

Using nostr for comments and sharing (social) that is visible across apps.

I also think there is a huge opportunity to save "subscribed" feeds and playlists as nostr events

I agree, and the signature part is really exciting to me, and I don't know why it can't be added to a feed so verify authenticity, at which point the feed could also live anywhere.

I'm also not convinced copying RSS into Nostr is a good solution, especially if the feed can live on multiple servers and be authenticated. Nostr could be used to point to the various feed locations. And I 100% agree, integration, not replacement is what I'm thinking about.

And yes, I'm no fan of XML, end up converting it all to JSON then back to XML so I can work with it. But I'm stuck working with the ideas of others.

Just think, all the new devs 20 years from now are going to be talking about how old nostr is and how it sucks here, here, and here, and needs to be replaced.

https://github.com/nostr-protocol/nips/blob/master/84.md

If you wanted to have signatures in RSS feeds you would have to introduce a standard way to serialize them and get a hash. and unfortunately the word standard and XML don't mix 😅

And yeah, I want to see a lot more integration before we attempt to replace RSS with nostr. there is a ton of ways nostr can help RSS without replacing it.

for example I think nostr:npub1l2vyh47mk2p0qlsku7hg0vn29faehy9hy34ygaclpn66ukqp3afqutajft mentioned NIP-84 (highlights) could support highlighting mp3 files. so it could be used to create a nostr native podcast clip that could be shared and commented on

Does the feed really need to be serialized and hashed, or just have the most basic of signatures?

Here's an b64 encoded signature. Are you able to decode it and verify that nostr:npub1yvgrrzf4dnmu30qfhw95x87ruu0g2kpv3a64h8hpvqsre8qeuspsgd6pv9 signed this feed? https://www.base64decode.org/

eyJjcmVhdGVkX2F0IjogMTczOTkwMzE0MiwiY29udGVudCI6ICIiLCJraW5kIjogMzMzMzMsInRhZ3MiOiBbXSwicHVia2V5IjogIjIzMTAzMTg5MzU2Y2Y3YzhiYzA5YmI4YjQzMWZjM2U3MWU4NTU4MmM4Zjc1NWI5ZWUxNjAyMDNjOWMxOWU0MDMiLCJpZCI6ICJiMTVkNzg4NGJkMDkyODQ1NDczYWVlODgxYjhiMTNkNWJjMjI1NmNiOWE0Yzk0N2UxZDIyMTM1ZmU4YzcyZWRjIiwic2lnIjogIjcyMDIxNTIwYzIxNDUzNDEyN2NiMDNmMDM2MzE2ZjkwZWRlMTU4YzM0YzQ4YTUwMGE0Mzk1MzM3YjhjNzU3NTFhMGMzMWM5OGQ0OWI1MzU5OTliNDNkZjA1NDZlZmFiODg3MjIxMDE4MmIzOWI5MjNhMzgzZjZiZTQ0MjBmZjBiIn0=

If this is in the feed, isn't that enough for a server to know that the feed has been authorized by the npub to be updated. https://raw.githubusercontent.com/thebells1111/bible-song-band/refs/heads/main/feed.xml

Basically, a server gets a request for a feed update, looks up the old feed, check the npub in the old feed, verify the new feed has the same npub, check the signature, if it's legit, then update the old feed with the new feed. This can be done across multiple servers for redundancy.

https://npub1ye5ptcxfyyxl5vjvdjar2ua3f0hynkjzpx552mu5snj3qmx5pzjscpknpr.nsite.lol/feeds/example.xml

That's a signature of nothing though. sure the pubkey signed that event but it only proves that they signed a k:33333 event, it doesn't relate to the feed at all

For example I could easily copy that feed, modify it, host it on my own site and say its your feed

The difficulty of cryptographic signatures is that they only work if your able to hash and sign the content itself. otherwise they are kind of pointless

If the k:33333 signed the feed GUID then that at least would prove that you say you "own" the GUID, but still the feed could be modified and there is know way to know what the original one is 😞

With podcasting, pirating a feed has always been possible. Although that should be addressed, the problem I'm thinking about isn't pirating so much as it's redundancy. If you, me, and Spencer all had servers, I want to host yours and his feed as well as my own as a backup, and you guys do the same for me. I'm trying to think of a way that I can prove to your server that it's me that's actually requesting the feed update, and not someone else. Perhaps the way to do it isn't the signature in the feed, but I encode the whole feed, sign it, then send it to your server. I'm in your list of approved npubs, so you verify it's me, decode the feed and replace the old one.

I guess this still doesn't solve the problem of how to verify my feed hasn't been tampered with when being stored on someone else's server. You could very easily replace all my value addresses with your own prior to publishing, so I'd have to trust anyone I was using as a redundancy. Seems signing the whole feed is necessary as you were pointing out.

https://raw.githubusercontent.com/thebells1111/bible-song-band/refs/heads/main/nostr.xml

What about something like this?

When I publish, I can hash the feed, sign the hash, add the signature to the feed, then publish the feed.

A podcast app removes the signature from the feed, verifies the signature matches the npub in the feed, hashes the feed (with the signature removed) and verifies the hash matches the hash in the signature.

https://svelte.dev/playground/134809f1af4d4651a35516502432cf3b?version=5.20.2

The issue is hashing the feed, whats the standard way to turn an XML file into a string? do you take the whole file or do you format it in a very specific way?

If you take the whole file as a raw text string, then it becomes invalid as soon as you add the signature to it because you've changed it. and if podcast apps where smart enough to remove the signature from the XML file before verifying, they would have to be specific about what they removed. because one missed char or white space and the file will be invalid :(

My idea is the .xml file is read it as a simple string. The host or feed builder would minify the string using a standardized function (this helps to remove extra spaces, new lines, etc that would create a different hash), then hashes it, signs the hash, then add that signature to the xml file.

Adding the signature to the file of course breaks the hash.

So to verify, an app would first remove the signature from the xml file, then minify the string using the same function the host did, then hash then hash it. The string should match the string that was originally hashed because the signature was removed first, and the hashes should match.

And yes, there would have a specific tag to add and remove to ensure nothing extra was added, but as long as everyone was using the same inserting and removal functions, that 'should' guarantee consistency across multiple apps and hosts.

The other would be a function that every app and host uses that parses the xml to a json file, remove the signature tag from the json file, then hash the JSON.stringified object, sign that hash, add it back to the JSON signature tag, then convert back to xml. As long as there's a consistent way to remove the signature before hashing, I think it would work.

Now, I don't think there's a way around preventing piracy, that is someone from taking your feed, changing the value block and npub to their own, and signing it and posting it on their server.

There's never been a way to prevent that, and I'm not sure signatures can, any more than they can stop either of us from signing a note that says "I rock" and claiming to be the original author. But signatures can definitely help with the problem of redundancy and ensuring the feed hasn't been tampered with.