So apparently utxo set size has now already hit 160M. This is a huge blowup, presumably due to "stamps" and similar projects. I wonder if there is a way in which we can finesse by having a flavor of bitcoin node which is "optimistic", stores (in fast access let's say) only utxos above a certain value (somewhere above dust), and has a kind of slower recovery mode to deal with spends of utxos below that filter. I don't know if that makes any sense, especially outside of "blocksonly", and after all we can't have all nodes be blocks only! (or can we? err I just remembered that someone around here may disagree :) ).
Discussion
Please keep pulling on this thread. Would be great to not worry about decentralization. At first, the ord enjoyoors were also running nodes, but I'm not sure they are anymore since they have some purpose built wallets and exchanges that run the node for them. Need more nodes. Need nodes to be able to stay cheap as possible.
We can already just run knots and prune Op_return data over a certain size.
I think the point here is to store in fast memory only transferable utxos for waxwings project which is join markets.
Also, I do agree with you that not storing such data on one's nodd is appropriate and undermining the garantee that JPEGs will be findeable would froce them to run full nodes.
Core prunes op_return since 0.10 something
It's not about pruning data. It's about UTXOs that have to be held in memory and increasing the resources needed to run a node. Or maybe i missed the point too.
What I meant is that core will not store those in the utxo set at all
Indeed, important to mention OP_RETURN, but iirc Stamps followed the example of Counterparty and did not use it, specifically to avoid being pruned (though the history of this is complex and controversial).
Yes, they do use bare p2ms, which is not pruned. But the last time I've checked, the ordinals was the actual problem, because people crate a ton of small amount (10k sats) UTXOS to "hold" inscriptions.
If you roll the numbers, small size p2tr dominates this recent surge in UTXOs
Right, i see. I guess what I'm saying in the OP applies mostly to them.
This seems to be an optimization problem. Certain statistics will probably predict the probability of a UTXO being used in the next x days again like amount, age, prevout, etc and maybe those UTXOs above a threshold probability could have a favorable spot in memory and others live on disk. This would have to be a very heuristic and rule-based approach, statistical models would have to be kept up to date etc... which iss all not really ideal. Maybe there's a better way?
The real fix seems to be utreexo which feels a few years away still.
Is utreexo even still being developed? The github repo seems pretty dead sadly
Ah there is another, really active one :)
It is, we actually have two clients under active development. Check out https://github.com/utreexo/utreexod and https://github.com/Davidson-Souza/Floresta
I'm unsure, but isn't the Witness unusually large no ?
Can't we just prune using a value to Witness data size ?