is nostr's huge data consumption a solvable problem?
Discussion
Does it use a lot of up/down data? Or storage space?
Bigger hard drives and zip flies
I always turn off media previews when I’m traveling out of the country and don’t have unlimited data
I recently did that and 2 days of amethyst are 4gb data
I use Damus and it doesn’t use much data with media previews turned off for me
I don't think efficiency is on the radar for Amethyst. At least not from what I can tell. The focus seems to be on shipping features at any cost.
Amethyst is specially costly. Would be amazing if users have an interface to tweak default subscriptions, deactivate drafts, kind 1 edits etc
we've got both Moore's law for compute and Nielsen's Law for bandwidth going for us.
for users with unlimited 5G+ or wired access and fair use unmetered plans, I'm not sure it needs to be solved for them
for users on metered plans and 3G/4G mobile networks, perhaps. this would impact usage in developing nations the most
i think it's possibly solvable with relay focused, interoperable NIPs that clients can adopt to make traffic more efficient
"Unlimited plans" arnt actually Unlimited.
Mobile plans usually cap you at 35-50gb then you get no service.
(Speaking from experience from when I forgot to use nostr on wifi for a week and couldn't doordash for a month lol)
What would nostrs data consumption be if every client and relay strictly synced using negantropy?
Fiatjaf fixed with hyperloglog but we aren't using it
Solution 1: Smarter Data Transmission
One way to ease the load is by reducing bandwidth use. While each Nostr event is cryptographically signed (meaning its content can’t be altered without invalidating the signature), we can still compress data during transmission. Imagine a client requesting a batch of events from a relay: the relay could zip them up into a compressed package, send it over, and the client unzips it upon arrival. The original, signed events stay intact on the relay, preserving security, while bandwidth usage drops. This is a practical tweak that works within Nostr’s existing framework.
Solution 2: Optimizing Storage
On the storage front, relays can get clever too. Nostr events are currently formatted in JSON, which is readable but bulky. While switching to a more compact binary format might require big protocol changes, relays could instead use databases designed to handle JSON efficiently—ones that support built-in compression. Picture this: events get stored in a compressed state, and when needed, the relay decompresses only what’s requested. Modern hardware can handle this extra processing, trading a bit of CPU effort for significant storage savings.
Solution 3: Distributing the Load
Nostr’s decentralized nature is a built-in advantage. Not every relay needs to store every event. Users and clients can pick relays based on their needs—some might focus on recent posts, others on specific communities, like a relay for musicians or coders. This natural distribution spreads the data burden. Sure, it might mean connecting to multiple relays to get the full picture, but that’s a fair trade-off for keeping individual relays lean and manageable.
Solution 4: Client-Side Caching
Clients—your phone or computer running a Nostr app—can shoulder some responsibility too. By caching events locally and only fetching updates since the last sync, they cut down on redundant data transfers. Since Nostr events are immutable (barring rare deletion requests), cached data stays valid indefinitely. Smart caching, already used by many Nostr clients, slashes bandwidth needs without changing the protocol.
Solution 5: Handling Media Wisely
What about big files like images or videos? In Nostr, it’s common to link to external hosting services rather than embedding media directly in events. This keeps event sizes small—usually just metadata and a URL—offloading the heavy lifting to platforms better suited for it. Encouraging this practice ensures relays don’t drown in oversized data.
Solution 6: Pruning Old Data
Relays could also prune less critical data, like dropping events older than a year, based on their own policies. Users needing history could seek out “archive” relays that keep everything. This flexibility lets operators tailor storage to their resources, while the network as a whole retains access to all data somewhere.
most of it is video/images so I’d say yes solvable
I have a couple other ideas as well 😅
What is data
Relay-as-a-Pointer with Decentralized Storage
What It Is: Instead of storing full copies of every event, relays only store tiny cryptographic hashes or pointers (like URLs) to the actual data, which lives on a decentralized storage system such as IPFS (InterPlanetary File System).
How It Works: When a client posts an event, it uploads the data to IPFS and sends the relay a hash. The relay keeps the hash and metadata (e.g., event ID, timestamp), while the heavy content (text, images) stays on IPFS. Clients fetch the data directly from IPFS using the pointer.
Why It’s New: While IPFS is used in other decentralized projects, pairing it with Nostr relays to offload storage entirely is unexplored territory.
Why It Helps: Relays need almost no storage—just a small database of pointers—making them cheap and easy to run on minimal hardware. Decentralization stays intact because IPFS is peer-to-peer, and no single relay holds all the data.
Isn't this pretty much how it works already sans the ipfs parts? This is definitely how atproto works (hell, atproto even uses CIDs/JSON-LD for all the content).
Seems like a huge difference to me, but still maybe not optimal. Text isn't really very heavy, distributing it p2p isn't really any harder than distributing the network's whole structure p2p by my understanding
yes
I think some of the responses represent the bigger problem with Nostr adoption. I know that power users don’t care about data consumption, but if we want nostr to take off we need to stop thinking like that. That’s why Nostr lacks diverse content, and it will stay that way, cause only knowleadgable power users are here.
Yes, data consumption is a problem for regular users, the users we want to join.
#asknostr #data #clients #nostr
Pretty sure it’s boiling the oceans
Big YES.
If a client connects to 6 relays and downloads the same 1000 notes from all the 6 relays every time you open it then it's not a Nostr problem, it's a client problem.
Something NDK can fix?
Every single client I used does this. Bad clients are a nostr problem.
Clients can't be blamed because there is no way to avoid duplicate downloads in nostr.
blame game does not work, butbutt, "bad clients can't be blamed"reinforces 1st point t *Y* gr8wayz from the dark - atm - PNW *_*
We have to blame someone. It's either the protocol or the clients.
If it's the clients we can fix it, if it's the protocol we should give up and go use Bluesky.
that fella is a bot
It is technically a Client problem.
But Nostr is not simple if you solve that problem.
You break everything simple about Nostr when you solve it.
So, either Nostr is complicated as fuck to code OR this is a protocol problem.
that's why my data plan goes through the roof.
Of course.
good sourcing, digi, on point. /*\ o_O'
thank you 🤙📈
thot_ 
add uP?
i C mal adjusted over time/
Hard to really trust any stats on exchange reserves as long as people consider it normal to use exchanges that don't really do proof of reserves 🤙
client has a cache server or relay that aggregates stuff, I think primal does this, or and I think it's possible, clients are smarter and instead of reading from N relays and get all the data, the connect to the specific required relays from the npubs you follow and just get the notes from that/those npubs.
My average Nostr data consumption is 70 GB per month, and i only open it once a day... Insane.
I need to check my other phone because I only got primal on this one and I've been testing other apps on the other. 8GB of data from 13 hours of use from primal on this phone.
Yes.
The wss://theforest.nostr1.com reads and writes from some of the largest relays, and I use clients that have implemented outboxes. That basically solves the problem.
yes but the cargo cult of strfry and the in-crowd of opensats beneficiaries have a kooky system of beliefs that basically make almost every reasonable use of nostr for real world things impossible
a bit like your stupid fucking rugpull tech pimping
there's something wrong with your brain, that prevents you from using your capacity for reason, but don't mind me
it can be improved a lot by not downloading a bunch of events in the first place.
By using DVMs this burden can be moved to service providers, which benefit from better infra and economies of scale.
An example is using Vertex vs downloading 1000 kind:3s to compute WoT things.
smarter relays are the key
idk why they made up this DVM thing, it's just a fucking relay
can i get more free money from odell if i say I'm a DVM developer?
It is a relay, yes, however the output is a new event kind which depends on the inputs of the request.
Yes, the same can technically be achieved by abusing a REQ filter, buuut it's different enough from the normal use that I am okay in using a new thing.
Why would it be "abusing" the REQ?
What's the difference in terms of conditions that you can put on the request?
Both can have a price.
Both can ID the requester.
nip-01 protocol is a mess that literally mashes 3 query APIs into one, incoherently
fetch events by id
search events by filter
search events by content keyword search
it's three API methods that are not clearly separated and this confusion is very obvious at least between the "ids" field and the rest except the "search" field
can we combine SEARCH and FILTER together? maybe yes, but that's a separate logic, and it is actually doable with a small enough result set out of the filter before you do a full text contains search
the nip-01 protocol was not designed as a proper API, and we aren't going to progress this protocol's capabilities without adopting age old "API" semantics
Hehe, I think nostr:npub1wf4pufsucer5va8g9p0rj5dnhvfeh6d8w0g6eayaep5dhps6rsgs43dgh9 and nostr:npub176p7sup477k5738qhxx0hk2n0cty2k5je5uvalzvkvwmw4tltmeqw7vgup would agree on that last point, yes.
> nip-01 protocol is a mess that literally mashes 3 query APIs into one, incoherently
fetch events by id
search events by filter
search events by content keyword search
Yes, If I were to re-make nip-01, I would use these 3 APIs + a general request/response API (DVM like).
However, as much as I don't like messy things, I don't think this is a big deal.
yeah, you only say that because you don't care as much as i do about auth and the private/paid relay use case, which i see as the main avenue of funding for development
We tried and discovered that DVMs do have a place
You could do REQs with AUTH otherwise
also, can you tell me why everything needs a websocket upgrade?
I've been insisting with plain HTTP endpoints for a while but nostr CEO told me that won't happen. Would add too much complexity
yeah, it is a temporary phase, that's how this works, after that the websockes for everyone phase is finally fading and only retards still use it two years later
Websockets are important for feeds
no, they are only important for subscriptions, everything else is in the past and in the event store
IMO this is the number one reason why clients blow up relays sockets management systems - and with Go that means coroutines as well, that manage the pings - because client devs are web devs and are not familiar with this retardation called "websockets that flip from response to subscription when they receive an EOSE" and i doubt that is very scalable either
i'm sure other languages may be slightly more efficient at managing thousands of open sockets that are basically idle but it's not gonna be that big a difference, since the coroutine is 2kb and the socket has minimum 4kb buffer allocated, and that's just the read buffer, not counting all the state
it's just wrong to do request/response with sockets, not only is it extra initial overhead and a round trip, it's also those threads, and the ping pong waste, and the biggest waste is giving sockets to web devs who barely understand http requests
What the funk is a DVM?
for every user, a relay (or a few).
your client talks to **only** this relay.
Yes, less paranoia fixes it
No more GM/GN? That will save big.
Its the media
We need smarter media Nostr providers
choose layer 0 or 1 when you want to optimize for traffic. choose higher layers if you want to help with decentralization.
nostr:naddr1qvzqqqr4gupzp8lvwt2hnw42wu40nec7vw949ys4wgdvums0svs8yhktl8mhlpd3qqxnzdejxqmnqde5xqeryvej5s385z