Hi ZmnSCPxj,
Allow me to reply to your post in mixed order (fraud proofs first): >But peers can be set up to allow you to hear of all chains while denying you >proof of the invalidity of some UTXO. I don't believe this is fundamentally different. In either scenario you end up on the wrong chain if all your peers are lying to you. One happens by omission of a fraud proof, while the other happens by omission of a valid longest chain. >This is precisely the "data unavailability claim" that shot down the previous >fraud proofs The "data unavailability" issue I was referring to, and which I believe is the reason why fraud proofs were abandoned, is the following: - Alice downloads a block with her full node, but the block is incomplete (e.g. a transaction is missing). - Alice reports this to Bob's SPV fraud proof client, who verifies this by requesting the transaction from the network. - If Bob can't download it, he rejects the block. - If Bob can download it, either Alice was malicious, or a miner was temporarily withholding the data. - Since Bob can't be certain Alice was being malicious, Bob can't ban her, which results in a DoS vector where SPV fraud proof clients can be forced to download all blocks. We circumvent the data unavailability problem here completely, since we are only questioning the validity of blocks which are involved in a fork (expensive and/or rare), and we are simply always downloading them in full. If my arguments above hold up, we can use fraud proof commitments as described in segwit BIP141 [0] instead of UTXO set commitments, which seems like the more elegant way to achieve the desired outcome. >Perhaps in combination with BIP157/158 it may be possible, if the filters >contain UTXO spends and a BIP158 filter was committed to on-chain. Then a >proof of absence could be done by revealing all the BIP158 filters from the >UTXO creation to the block being validated, as well as the blocks whose BIP158 >filters matched the UTXO and revealing that no, they actually do not spend the >UTXO. Yes, I mentioned something similar to Laolu, but it does seem computationally expensive to run every input in a block through the filter of every past block. The fact that BIP157/158 can function without commitments is also why I suspected we may not necessarily need UTXO set commitments. >UTXO sets can only be validated by actually running the entire blockchain, >i.e. fullnoding. It seems to me you can validate uncommitted UTXO sets by comparing them. Download and compare UTXO set hashes from multiple peers. If they disagree on a certain block, download that block and the relevant merkle path(s) from the previous block's UTXO set, and then verify who is right. Ban the peer who lied. Note that unlike fraud proofs, it is not possible to lie by omission, but it does assume one of your peers is honest. Of course this does nothing to dispute your earlier point that this may not be all that efficient (e.g. full nodes keeping merkle paths of all prior states). >What BIP157 does is summarize data that is within a block, thus validating >them can be done simply by downloading the block in question. >UTXO sets summarize data in the entire blockchain, hence proper validation >requires downloading the entire blockchain. Thus it cannot be a comparison point. It's still possible to lie by omission. Let's say a miner spends some coins in block N, and spends the exact same coins again in block N+1, making block N+1 invalid. If the filter for block N is maliciously constructed, you won't notice the spend in block N, causing you to think block N+1 is valid. In short, you're still relying on one of your peers to give you a correct filter. If all your peers lie, you can always be deceived. >Tangentially, we cannot just magically commit to anything on the blockchain. >[...] if you are adding new information to be committed, that may increase the >resource usage of fullnodes. [...] This is probably still better than BIP37 >but we should still be aware the additional load on fullnodes. I agree with all this. To summarize, this is my current understanding of our options for enabling light clients to verify a single block in isolation: 1. UTXO set commitments (complex, more resource usage to full nodes) 2. BIP157/158 commitments (expensive for clients to check all filters to get exclusion proofs) 3. BIP141 fraud proof commitments (assumes fraud proofs will be passed on to the SPV client) The debate is still open on whether the options above can be done without actually committing them into blocks via a soft fork. My current hunch is "yes" for 1 and 2, and "no" for 3, which would be unfortunate, because 3 currently seems to me like the more elegant solution. -- Ruben Somsen [0] https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Compact_fraud_proof_for_SPV_nodes >UTXO sets can only be validated by actually running the entire blockchain, >i.e. fullnoding. >What BIP157 does is summarize data that is within a block, thus validating >them can be done simply by downloading the block in question. UTXO sets summarize data in the entire blockchain, hence proper validation requires downloading the entire blockchain. Thus it cannot be a comparison point. > > This makes no sense > > or you trust that every peer you have is not omitting the proof. > > It's the latter, you trust every peer you have is not omitting the > proof. It requires one honest peer. The reason this is acceptable is > because you're already making that assumption. If none of your peers > are honest, you have no guarantee of hearing about the chain with the > most PoW. But peers can be set up to allow you to hear of all chains while denying you proof of the invalidity of some UTXO. This is precisely the "data unavailability claim" that shot down the previous fraud proofs (i.e. absence of proof is not proof of absence, and proof of UTXO validity was defined by proof of absence of any intervening spend of the UTXO). Perhaps in combination with BIP157/158 it may be possible, if the filters contain UTXO spends and a BIP158 filter was committed to on-chain. Then a proof of absence could be done by revealing all the BIP158 filters from the UTXO creation to the block being validated, as well as the blocks whose BIP158 filters matched the UTXO and revealing that no, they actually do not spend the UTXO. -- Tangentially, we cannot just magically commit to anything on the blockchain. Header blocks commit to block data and commit to some other header block. All those header blocks and the block data need to be stored and transmitted over the network somehow, even though they are "only" being committed to. Thus, if you are adding new information to be committed, that may increase the resource usage of fullnodes. So if UTXO set commitments, or utreexo commitments, or BIP158 filter digests, etc. are committed to in the coinbase, they have to be stored somehow in fullnodes the entire UUTXO set, or the actual utreexo structure, or the actual BIP158 filter, etc. at each block. Otherwise it would be pointless to store those commitments since it would not be possible to somehow acquire the data being committed to after-the-fact. This is probably still better than BIP37 but we should still be aware the additional load on fullnodes. Regards, ZmnSCPxj On Sat, Apr 20, 2019 at 6:45 AM ZmnSCPxj <zmnsc...@protonmail.com> wrote: > > Good morning Ruben, > > > Hi ZmnSCPxj, > > > > > There is no safe way to use UTXO sets without identifying who is telling > > > you those sets are valid, or making it expensive to lie > > > The first option requires trust and is weaker than SPV, the second > > > requires committing to a proof-of-work > > > > Olaoluwa Osuntokun's BIP157 manages to function without a commitment: > > "If the client receives conflicting filter headers from different > > peers for any block and filter type, it SHOULD interrogate them to > > determine which is faulty." > > > > I am wondering if the same logic can be applied to UTXO sets or the > > fraud proofs I just described. > > UTXO sets can only be validated by actually running the entire blockchain, > i.e. fullnoding. > > What BIP157 does is summarize data that is within a block, thus validating > them can be done simply by downloading the block in question. > > UTXO sets summarize data in the entire blockchain, hence proper validation > requires downloading the entire blockchain. > Thus it cannot be a comparison point. > > > > > This makes no sense > > > or you trust that every peer you have is not omitting the proof. > > > > It's the latter, you trust every peer you have is not omitting the > > proof. It requires one honest peer. The reason this is acceptable is > > because you're already making that assumption. If none of your peers > > are honest, you have no guarantee of hearing about the chain with the > > most PoW. > > But peers can be set up to allow you to hear of all chains while denying you > proof of the invalidity of some UTXO. > This is precisely the "data unavailability claim" that shot down the previous > fraud proofs (i.e. absence of proof is not proof of absence, and proof of > UTXO validity was defined by proof of absence of any intervening spend of the > UTXO). > > Perhaps in combination with BIP157/158 it may be possible, if the filters > contain UTXO spends and a BIP158 filter was committed to on-chain. > Then a proof of absence could be done by revealing all the BIP158 filters > from the UTXO creation to the block being validated, as well as the blocks > whose BIP158 filters matched the UTXO and revealing that no, they actually do > not spend the UTXO. > > -- > > Tangentially, we cannot just magically commit to anything on the blockchain. > Header blocks commit to block data and commit to some other header block. > All those header blocks and the block data need to be stored and transmitted > over the network somehow, even though they are "only" being committed to. > Thus, if you are adding new information to be committed, that may increase > the resource usage of fullnodes. > > So if UTXO set commitments, or utreexo commitments, or BIP158 filter digests, > etc. are committed to in the coinbase, they have to be stored somehow in > fullnodes the entire UUTXO set, or the actual utreexo structure, or the > actual BIP158 filter, etc. at each block. > Otherwise it would be pointless to store those commitments since it would not > be possible to somehow acquire the data being committed to after-the-fact. > > This is probably still better than BIP37 but we should still be aware the > additional load on fullnodes. > > Regards, > ZmnSCPxj _______________________________________________ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev