Media hosting is a hard problem to solve. The CDN costs with other providers are significantly higher compared to Cloudflare.

Moderation is hard. You wouldn’t want to know the type of content that has to be filtered every day.

There is a lot of work going on in the background so that someone uploading a 10MB image gets optimized down to a few hundred KB, strip metadata, and have the media file load fast, etc.

And a lot more other costs and work exist, while no one properly pays for hosting media, so no one wants to work on hosting media. Nostrimg died partially for the reason of CSAM spam.

I don’t fully get the concerns about CF either in some cases. Sure, if this is a sensitive application then it could matter.

But if it is media that is already publicly available, what changes?

Everyone complains but no one wants to fund switching to an alterntive reliable CDN (some “cheap” options bring significant drawbacks as well)

On Nostr.build we also have backups in a completely different provider.

Reply to this note

Please Login to reply.

Discussion

It's not a hard problem to solve. I host my own stuff, and people are willing to pay for hosting via nostr build.

The people who don't want to host their own stuff or pay for it, well they are not important. Freeloaders don't belong on nostr.

The difference with hosting using a VPS is that you pay with your time and using someone else’s infra still. So in the end nothing is free.

Some things also don’t exist like moderation because it’s only your media, and tradeoffs like no CDN.

I have a dedicated server and it's quite cheap compared to a VPS.

Yes you are right, nothing is free. I am paying for my dedicated server unless I host my own stuff from my home, but even then I pay for my ISP, but that is what makes nostr great is that there will be not a lot of freeloaders like X and Facebook. It's more organic.

don't support freeloaders!

Remind me what Nostr was for again. Between WoT, auth, PoW, paid relays, paid media hosts, and filters, I’m starting to forget… Something about decentralization and privacy, right? 🤔

it's about telling freeloaders to f off

The era of paid protocols and commercial networks has been dead for about 30 years now.

it's not dead, most webmasters just don't know how to make money.

Hosting media for untrusted users is a very hard problem to solve. It can be "easy" for very small numbers of users and viewers, but scale slightly beyond that and guaruntee %99 uptime and it's a nightmare.

It’s not even uptime but abuse. All sorts of things happen.

Yeah, you're right I guess I was throwing that into availability. I don't have a good solution except for aggressive rate limiting, but that doesn't fix ddos, or poorly configured clients that can hammer endpoints while loading.

Yeah, spam media too/CSAM.

About bandwidth, you need a front-box somewhere that takes the majority of the load with caching, and having that onprem is expensive due to it becoming DC uplink level pricing.

Yup. I've been thinking of a tiered arrangement to keep cost down as well. Smaller boxes on edge, larger boxes behind and big ass bulk and cool caches locally.

It's only hard for admins that allow freeloaders, if you stop allowing freeloaders then it becomes easy. It's the admins that are complicating things themselves

I've talked about hosting a time or two, got crickets for replies, not sure anyone actually want's to pay for media hosting. Not really looking for general public either, more interested in B2B or partnering with applications.

Anyway, on the CF thing. It's more because most of the web goes through them, they almost have a monopoly on monitoring web traffic without the user's permission. When visiting your site, I had to go through CF before I could even get to your terms of service once the site loads. CF does a lot of work, but they also block nearly every website I would like to access. They do IP trust rating and categorizing, and require browser fingerprinting and javascript before I can access anything.

They can see almost everything a user does and almost make it a point to monitor users so they can build the analytics and "defense" techniques. But beyond that, don't see me a free speech service but require me to show my browser fingerprint and fund a "trustworthy" IP address in order to use your service. And that's just an assumption, I can't know what Cloudflare requires in order to let me have access to the things it protects.

B2B and integrations will be even worse. Almost no project on Nostr has the money to fund this infrastructure or the will to as long as they don’t notice the effects.

The few that do, are not willing to fund this infrastructure, ask me how I know.

Almost every CDN service does the things you have listed, and the other option is build your own which is really expensive.

Or host media out of a basement/VPS and accept that it will be pretty bad and still expensive for the bandwidth.

In the end, there are tradeoffs, and the majority of the people do not want to tolerate the cost and impact of changes.

And out of the concerns you have listed, only a subset of them apply to a media host. (as I specified in my original post)

Ya basement hosting is key IMO. We need more of it. But then there's the security issue of self hosting on your personal network.

It seems like CDN and algo could be solved in one fell swoop via some variation of DVM.

-A locally available DVM solves the CDN issue. It's local.

-It runs an AI scout (discovery algo) which sifts the network for relevant content. Since it's constantly looking, it effectively acts as a miner.

-Then it caches a chain/string of relevant posts, with a 50-100 post head start, cached locally for fast serving.

-Then multiply and incentivize these DVM/algo/miners for decentralization.

Solves CDN due to locality.

Solves speed via caching.

Solves decentral with multi users.

Solves discovery via algo.

Solves the serving layer, without being bogged by the lifetime storage layer.

Well the only reason I believe it could work is that I was willing to front load the cost myself. The idea is a slow process (im not in a rush for cash) and I already have _most_ of the infra for my other work, just not at a very large scale.

I'm not saying cloudflare specifically is the issue, in this case though they're the largest and fully block websites/applications. If a cdn wants to block media because they don't like my IP address fine, but when 1/20 results on a search engine I can't see because they use Cloudflare or Vercel, something has to change.

That sounds like a good plan but I do not expect it to be self-sustaining at all or to be funded by grant orgs unfortunately due to how this works.

We need more CF alternatives.

The problem is on Nostr people expect things for free and complain about ads, which makes it unreasonably expensive (relative to income) to use other solutions, and none of the big grant orgs are willing to provide enough funding to get a CF-free alternative off the ground. "Just enough" to make it work.

At the same time, CF can subsidize smaller customers which may become larger enterprise customers, and can work on massive economies of scale.

(On Nostr.land there is no CF except for DNS (registration was done there but will move out), and on Nostr.build everything except media serving is not through CF)

(I should probably mention Nostr.land event cannon uses CF Workers, but the main reason is that it is not a sensitive workload and it makes it easier to deal with improper relay rate limiting schemes.)

Mind you I'm also a huge hypocrite in this scenario for the same reasons. I've used some very shitty registrars and domains started getting unaffordable and CF was the only provider I've worked with that offered stable costs. I got sick of spending 100s/year on registration and migrations and privacy and other service fees. Then also still needing a DNS provider and paying 100s/year for an actually commercial service as well. CF has been the most professional to work with and get support from. Now I only use them for registration and DNS as well.

That said, yeah DNS and domain registration is also hard... possibly even harder than this discussion.

That's a follow. Only interested in meat/potato people here. The puff-balls on here are annoying.

Good points on CDN.

It's almost like a makeshift CDN needs building. Maybe. Not a small task. But I have an idea here that may aid this end. Need to think more.

Do any FOSS CDN initiatives exist? Does IPFS have any CDN component?

Yeah that was kind of the idea, my plate is too full to really invest in good UI and the costs of moderation, so it would have to start small until that built up.

There are some foss cdn software options, but the N in CDN is the part you have to put real effort in where there are no instructions lol.

This is enlightening man.

The CSAM problem would be brutal. And any censorship that exists for a legitimate reason, can then be used for overreach. Tricky balance.

But media seems like the biggest opportunity for Nostr marketing. TikTok grew solely from cross-platform watermarked content.

A cool videos with a TikTok endings flooded other networks. Then after seeing enough cool videos, users of that other network went "this TikTok thing is cool, I should just go there to get all these bangers". And so they did.

Fully organic and content led marketing.

Media servers hold one of the strongest marketing angles on Nostr. To drive Nostr growth and monetize a business.

I second this observation.

I’m not aware of any one thing. Most bounties are just ad-hoc posts or comments on GitHub. There was supposed to be one as a part of bitcoinbounties.org I guess, but nobody ever used it.

May have been because the owner is anti-FOSS to some extent

Yeah, everyone pretty much abandoned that whole thing a couple of years back.

I say, start a new one!

Thanks man. I think that's the one I remember. At least whatever is left of it.

I'll think on how to do it right.

Does multi-sig lightning exist? Or only on-chain? Would prob need a committee-esc mechanism to approve actions. Multi-sig would serve that purpose well. Need to think....

I think what you are looking for is a viable escrow system on Lightning, where bounties can be pre-funded and paid out based on a set of conditions. I know some people have discussed and maybe attempted to build this, but I’m not sure what the state of that is at the moment.

escrow is hard and risky

Both valid points. It's hard to set a rigid smart contract condition I think, and requires a bunch of building. Plus you never know what the breach will be.

Prob just easier to have trusted arbiters. Ya I gotta think. I'm not super passionate about this. But I think it's an important solution. nostr:nprofile1qqswum4p82uluhz2dr40nvdrflspffntgqghc58w9fs57nx6jkdkuaqpz4mhxue69uhks6tnwshxummnw3ezumrpdejqzyrhwden5te0dehhxarj9ekxzmny5dhgt8

And when you have an arbiter you have custodial risk and being a custodian is MUCH HARDER in terms of risk.

Exactly, nobody wants to be the custodian. Too many targets on their back.

Trusting a custodian is hard. Being one is harder.

Especially when the target market is penetration testers. Particularly the most skilled ones.

Sometimes we just gotta be courageous and do what needs to be done tho. That's what trailblazing is about.

This is why the ETH crowd loves smart contracts and DAOs so much. But that won’t ever fly on Nostr. I’ve ever seen a single “DeFi on bitcoin” product to date that I would consider using other than just to play around with and forget.

*never