Re: [bitcoin-dev] Reorgs on SigNet - Looking for feedback on approach and parameters
> On Sep 12, 2021, at 00:53, Anthony Towns wrote: > > On Thu, Sep 09, 2021 at 05:50:08PM -0700, Matt Corallo via bitcoin-dev wrote: >>> AJ proposed to allow SigNet users to opt-out of reorgs in case they >>> explicitly want to remain unaffected. This can be done by setting a >>> to-be-reorged version bit [...] >> Why bother with a version bit? This seems substantially more complicated >> than the original proposal that surfaced many times before signet launched >> to just have a different reorg signing key. > > Yeah, that was the original idea, but there ended up being two problems > with that approach. The simplest is that the signet block signature > encodes the signet challenge, But if that was the originally proposal, why is the challenge committed to in the block? :) > So using the RECENT_CONSENSUS_CHANGE behaviour that avoids the > discourage/disconnect logic seems the way to avoid that problem, and that > means making it so that nodes that that opt-out of reorgs can distinguish > valid-but-will-become-stale blocks from invalid blocks. Using a versionbit > seems like the easiest way of doing that. Sure, you could set that for invalid block signatures as well though. It’s not really a material DoS protection one way or the other. >>> The reorg-interval X very much depends on the user's needs. One could >>> argue that there should be, for example, three reorgs per day, each 48 >>> blocks apart. Such a short reorg interval allows developers in all time >>> zones to be awake during one or two reorgs per day. Developers don't >>> need to wait for, for example, a week until they can test their reorgs >>> next. However, too frequent reorgs could hinder other SigNet users. >> I see zero reason whatsoever to not simply reorg ~every block, or as often >> as is practical. If users opt in to wanting to test with reorgs, they should >> be able to test with reorgs, not wait a day to test with reorgs. > > Blocks on signet get mined at a similar rate to mainnet, so you'll always > have to wait a little bit (up to an hour) -- if you don't want to wait > at all, that's what regtest (or perhaps a custom signet) is for. Can you explain the motivation for this? From where I sit, as far as I know, I should basically be a prime example of the target market for public signet - someone developing bitcoin applications with regular requirements to test those applications with other developers without jumping through hoops to configure software the same across the globe and set up miners. With blocks being slow and irregular, I’m basically not benefited at all by signet and will stick with testnet3/mainnet testing, which both suck. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] TAPLEAF_UPDATE_VERIFY covenant opcode
Sorry for the lack of clarity, sometimes it sounds easier to explain ideas with code. While MERKLESUB is still WIP, here the semantic. If the input spent is a SegWit v1 Taproot output, and the script path spending is used, the top stack item is interpreted as an output position of the spending transaction. The second top stack item is interpreted as a 32-byte x-only pubkey to be negated and added to the spent internal pubkey. The spent tapscript is removed from the merkle tree of tapscripts and a new merkle root is recomputed with the first node element of the spending control block as the tapleaf hash. From then, this new merkle root is added as the taproot tweak to the updated internal pubkey, while correcting for parity. This new tweaked pubkey is interpreted as a v1 witness program and must match the scriptPubKey of the spending transaction output as the passed position. Otherwise, MERKLESUB returns a failure. I believe this is matching your description and the main difference compared to your TLUV proposal is the lack of merkle tree extension, where a new merkle path is added in place of the removed tapscript. Motivation is saving up the one byte of the new merkle path step, which is not necessary for our CoinPool use-case. > That would mean anyone who could do a valid spend of the tx could > violate the covenant by spending to an unencumbered witness v2 output > and (by collaborating with a miner) steal the funds. I don't think > there's a reasonable way to have existing covenants be forward > compatible with future destination addresses (beyond something like CTV > that strictly hardcodes them). That's a good catch, thanks for raising it :) Depends how you define reasonable, but I think one straightforward fix is to extend the signature digest algorithm to encompass the segwit version (and maybe program-size ?) of the spending transaction outputs. Then you add a "contract" aggregated-key in every tapscript where a TLUV/MERKLESUB covenant is present. The off-chain contract participant can exchange signatures at initial setup committing to the segwit version. I think this addresses the sent-to-unknown-witness-output point ? When future destination addresses are deployed, assuming a new round of interactivity, the participants can send the fund to a v1+ by exchanging signatures with SIGHASH_ALL, that way authorizing the bypass of TLUV/MERKLESUB. Of course, in case of v1+ deployment, the key path could be used. Though this path could have been "burnt" by picking up an internal point with an unknown scalar following the off-chain contract/use-case semantic ? > Having the output position parameter might be an interesting way to > merge/split a vault/pool, but it's not clear to me how much sense it > makes sense to optimise for that, rather than just doing that via the key > path. For pools, you want the key path to be common anyway (for privacy > and efficiency), so it shouldn't be a problem; but even for vaults, > you want the cold wallet accessible enough to be useful for the case > where theft is attempted, and maybe that's also accessible enough for > the ocassional merge/split to keep your utxo count/sizes reasonable. I think you can come up with interesting contract policies. Let's say you want to authorize the emergency path of your pool/vault balances if X happens (e.g a massive drop in USDT price signed by DLC oracles). You have (A+B+C+D) forking into (A+B) and (C+D) pooled funds. To conserve the contracts pre-negotiated economic equilibrium, all the participants would like the emergency path to be inherited on both forks. Without relying on the key path interactivity, which is ultimately a trust on the post-fork cooperation of your counterparty ? > Saving a byte of witness data at the cost of specifying additional > opcodes seems like optimising the wrong thing to me. I think we should keep in mind that any overhead cost in the usage of a script primitive is echoed to the user of off-chain contract/payment channels. If the tapscripts are bigger, your average on-chain spends in case of non-cooperative scenarios are increased in consequence, and as such your fee-bumping reserve. Thus making those systems less economically accessible. If we really envision having billions of Bitcoin users owning a utxo or shards of them, we should also think that those users might have limited means to pay on-chain fees. Where should be the line between resource optimizations and protocol/implementation complexity ? Hard to tell. > I don't think that works, because different scripts in the same merkle > tree can have different script versions, which would here indicate > different parities for the same internal pub key. Let me make it clearer. We introduce a new tapscript version 0x20, forcing a new bit in the first byte of the control block to be interpreted as the parity bit of the spent internal pubkey. To ensure this parity bit is faithful and won't break the updated key path, it's committed in
Re: [bitcoin-dev] Proposal: Auto-shutdown as 5-year fork window
You can do that kind of change in your own Bitcoin-compatible client, but you cannot be sure that other people will run that version and that it will shut down when you want. Many miners use their own custom software for mining blocks, the same for mining pools. There are many clients that are compatible with consensus, but different than Bitcoin Core. Also you should notice that Bitcoin community make changes by using soft-forks, not hard-forks. Backward compatibility is preserved as often as possible and there is no reason to change that. Any change can be deployed in a soft-fork way, even "evil soft-forks" are possible, as described in https://petertodd.org/2016/forced-soft-forks. I think that kind of soft-fork is still better than hard-fork. On 2021-09-12 21:16:00 user James Lu via bitcoin-dev wrote: If MTP-11 is greater than 5 years after the release date of the current software version, the full node should shut down automatically. This would allow writing code that gives the community ~5 years to upgrade to a version that executes a new hard fork while keeping everyone in consensus, provided the change is non-controversial.___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[bitcoin-dev] Proposal: Auto-shutdown as 5-year fork window
If MTP-11 is greater than 5 years after the release date of the current software version, the full node should shut down automatically. This would allow writing code that gives the community ~5 years to upgrade to a version that executes a new hard fork while keeping everyone in consensus, provided the change is non-controversial. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Reorgs on SigNet - Looking for feedback on approach and parameters
> Sometimes that reorg could be deeper if you would be lucky enough to get a block with more work than N following blocks combined Each block is credited for its contribution to total chainwork by the difficulty target, not the hash of the block itself. On Sun, Sep 12, 2021 at 10:42 PM vjudeu via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > > - 1 block reorgs: these are a regular feature on mainnet, everyone >should cope with them; having them happen multiple times a day to >make testing easier should be great > > Anyone can do 1 block reorg, because nonce is not signed, so anyone can > replace that with better value. For example, if you have block > 0086d6b2636cb2a392d45edc4ec544a10024d30141c9adf4bfd9de533b53 with > 0x0007f4cc nonce, you can replace that with 0x00110241 nonce and get > 00096a1c4239d994547185c80308a552cba85d5bd28a51e9dc583ae5eadb block, > where everything is identical, except the nonce. > > Sometimes that reorg could be deeper if you would be lucky enough to get a > block with more work than N following blocks combined. > > On 2021-09-08 09:59:29 user Anthony Towns via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > > On Tue, Sep 07, 2021 at 06:07:47PM +0200, 0xB10C via bitcoin-dev wrote: > > The reorg-interval X very much depends on the user's needs. One could > > argue that there should be, for example, three reorgs per day, each 48 > > blocks apart. > > Oh, wow, I think the last suggestion was every 100 blocks (every > ~16h40m). Once every ~8h sounds very convenient. > > > Such a short reorg interval allows developers in all time > > zones to be awake during one or two reorgs per day. > > And also for there to reliably be reorgs when they're not awake, which > might be a useful thing to be able to handle, too :) > > > Developers don't > > need to wait for, for example, a week until they can test their reorgs > > next. However, too frequent reorgs could hinder other SigNet users. > > Being able to run `bitcoind -signet -signetacceptreorg=0` and never > seeing any reorgs should presumably make this not a problem? > > For people who do see reorgs, having an average of 2 or 3 additional > blocks every 48 blocks is perhaps a 6% increase in storage/traffic. > > > # Scenario 1: Race between two chains > > > > For this scenario, at least two nodes and miner scripts need to be > > running. An always-miner A continuously produces blocks and rejects > > blocks with the to-be-reorged version bit flag set. And a race-miner R > > that only mines D blocks at the start of each interval and then waits X > > blocks. A and R both have the same hash rate. Assuming both are well > > connected to the network, it's random which miner will first mine and > > propagate a block. In the end, the A miner chain will always win the > race. > > I think this description is missing that all the blocks R mines have > the to-be-reorged flag set. > > > 3. How deep should the reorgs be on average? Do you want to test > >deeper reorgs (10+ blocks) too? > > Super interested in input on this -- perhaps we should get optech to > send a survey out to their members, or so? > > My feeling is: > > - 1 block reorgs: these are a regular feature on mainnet, everyone >should cope with them; having them happen multiple times a day to >make testing easier should be great > > - 2-3 block reorgs: good for testing the "your tx didn't get enough >confirms to be credited to your account" case, even though it barely >ever happens on mainnet > > - 4-6 block reorgs: likely to violate business assumptions, but >completely technically plausible, especially if there's an attack >against the network > > - 7-100 block reorgs: for this to happen on mainnet, it would probably >mean there was a bug and pools/miners agree the chain has to >be immediately reverted -- eg, someone discovers and exploits an >inflation bug, minting themselves free bitcoins and breaking the 21M >limit (eg, the 51 block reorg in Aug 2010); or someone discovers a >bug that splits the chain, and the less compatible chain is reverted >(eg, the 24 block reorg due to the bdb lock limit in Mar 2013); >or something similar. Obviously the bug would have to have been >discovered pretty quickly after it was exploited for the reorg to be >under a day's worth of blocks. > > - 100-2000+ block reorgs: severe bug that wasn't found quickly, or where >getting >50% of miners organised took more than a few hours. This will >start breaking protocol assumptions, like pool payouts, lightning's >relative locktimes, or liquid's peg-in confirmation requirements, and >result in hundres of MBs of changes to the utxo set > > Maybe it would be good to do reorgs of 15, 150 or 1500 blocks as a > special fire-drill event, perhaps once a month/quarter/year or so, > in some pre-announced window? > > I think sticking to 1-6 block reorgs initially is a fine way to
Re: [bitcoin-dev] Reorgs on SigNet - Looking for feedback on approach and parameters
> - 1 block reorgs: these are a regular feature on mainnet, everyone should cope with them; having them happen multiple times a day to make testing easier should be great Anyone can do 1 block reorg, because nonce is not signed, so anyone can replace that with better value. For example, if you have block 0086d6b2636cb2a392d45edc4ec544a10024d30141c9adf4bfd9de533b53 with 0x0007f4cc nonce, you can replace that with 0x00110241 nonce and get 00096a1c4239d994547185c80308a552cba85d5bd28a51e9dc583ae5eadb block, where everything is identical, except the nonce. Sometimes that reorg could be deeper if you would be lucky enough to get a block with more work than N following blocks combined. On 2021-09-08 09:59:29 user Anthony Towns via bitcoin-dev wrote: > On Tue, Sep 07, 2021 at 06:07:47PM +0200, 0xB10C via bitcoin-dev wrote: > The reorg-interval X very much depends on the user's needs. One could > argue that there should be, for example, three reorgs per day, each 48 > blocks apart. Oh, wow, I think the last suggestion was every 100 blocks (every ~16h40m). Once every ~8h sounds very convenient. > Such a short reorg interval allows developers in all time > zones to be awake during one or two reorgs per day. And also for there to reliably be reorgs when they're not awake, which might be a useful thing to be able to handle, too :) > Developers don't > need to wait for, for example, a week until they can test their reorgs > next. However, too frequent reorgs could hinder other SigNet users. Being able to run `bitcoind -signet -signetacceptreorg=0` and never seeing any reorgs should presumably make this not a problem? For people who do see reorgs, having an average of 2 or 3 additional blocks every 48 blocks is perhaps a 6% increase in storage/traffic. > # Scenario 1: Race between two chains > > For this scenario, at least two nodes and miner scripts need to be > running. An always-miner A continuously produces blocks and rejects > blocks with the to-be-reorged version bit flag set. And a race-miner R > that only mines D blocks at the start of each interval and then waits X > blocks. A and R both have the same hash rate. Assuming both are well > connected to the network, it's random which miner will first mine and > propagate a block. In the end, the A miner chain will always win the race. I think this description is missing that all the blocks R mines have the to-be-reorged flag set. > 3. How deep should the reorgs be on average? Do you want to test >deeper reorgs (10+ blocks) too? Super interested in input on this -- perhaps we should get optech to send a survey out to their members, or so? My feeling is: - 1 block reorgs: these are a regular feature on mainnet, everyone should cope with them; having them happen multiple times a day to make testing easier should be great - 2-3 block reorgs: good for testing the "your tx didn't get enough confirms to be credited to your account" case, even though it barely ever happens on mainnet - 4-6 block reorgs: likely to violate business assumptions, but completely technically plausible, especially if there's an attack against the network - 7-100 block reorgs: for this to happen on mainnet, it would probably mean there was a bug and pools/miners agree the chain has to be immediately reverted -- eg, someone discovers and exploits an inflation bug, minting themselves free bitcoins and breaking the 21M limit (eg, the 51 block reorg in Aug 2010); or someone discovers a bug that splits the chain, and the less compatible chain is reverted (eg, the 24 block reorg due to the bdb lock limit in Mar 2013); or something similar. Obviously the bug would have to have been discovered pretty quickly after it was exploited for the reorg to be under a day's worth of blocks. - 100-2000+ block reorgs: severe bug that wasn't found quickly, or where getting >50% of miners organised took more than a few hours. This will start breaking protocol assumptions, like pool payouts, lightning's relative locktimes, or liquid's peg-in confirmation requirements, and result in hundres of MBs of changes to the utxo set Maybe it would be good to do reorgs of 15, 150 or 1500 blocks as a special fire-drill event, perhaps once a month/quarter/year or so, in some pre-announced window? I think sticking to 1-6 block reorgs initially is a fine way to start though. > After enough testing, the default SigNet can start to do periodical > reorgs, too. FWIW, the only thing that concerns me about doing this on the default signet is making sure that nodes that set -signetacceptreorg=0 don't end up partitioning the p2p network due to either rejecting a higher work chain or rejecting txs due to double-spends across the two chains. A quick draft of code for -signetacceptreorg=0 is available at https://github.com/ajtowns/bitcoin/commits/202108-signetreorg Cheers, aj
Re: [bitcoin-dev] Reorgs on SigNet - Looking for feedback on approach and parameters
On Thu, Sep 09, 2021 at 05:50:08PM -0700, Matt Corallo via bitcoin-dev wrote: > > AJ proposed to allow SigNet users to opt-out of reorgs in case they > > explicitly want to remain unaffected. This can be done by setting a > > to-be-reorged version bit [...] > Why bother with a version bit? This seems substantially more complicated > than the original proposal that surfaced many times before signet launched > to just have a different reorg signing key. Yeah, that was the original idea, but there ended up being two problems with that approach. The simplest is that the signet block signature encodes the signet challenge, so if you have two different challenges, eg " CHECKSIG" "0 SWAP 1 2 CHECKMULTISIG" then while both challenges will accept a signature by normal as the block solution, the signature by "normal" will be different between the two. This is a fairly natural result of reusing the tx-signing code for the block signatures and not having a noinput/anyprevout tx-signing mode. More generally, though, this would mean that a node that's opting out of reorgs will see the to-be-reorged blocks as simply invalid due to a bad signature, and will follow the "this node sent me an invalid block" path in the p2p code, and start marking peers that are following reorgs as discouraged and worth disconnecting. I think that would make it pretty hard to avoid partitioning the network between peers that do and don't accept reorgs, and generally be a pain. So using the RECENT_CONSENSUS_CHANGE behaviour that avoids the discourage/disconnect logic seems the way to avoid that problem, and that means making it so that nodes that that opt-out of reorgs can distinguish valid-but-will-become-stale blocks from invalid blocks. Using a versionbit seems like the easiest way of doing that. > > The reorg-interval X very much depends on the user's needs. One could > > argue that there should be, for example, three reorgs per day, each 48 > > blocks apart. Such a short reorg interval allows developers in all time > > zones to be awake during one or two reorgs per day. Developers don't > > need to wait for, for example, a week until they can test their reorgs > > next. However, too frequent reorgs could hinder other SigNet users. > I see zero reason whatsoever to not simply reorg ~every block, or as often > as is practical. If users opt in to wanting to test with reorgs, they should > be able to test with reorgs, not wait a day to test with reorgs. Blocks on signet get mined at a similar rate to mainnet, so you'll always have to wait a little bit (up to an hour) -- if you don't want to wait at all, that's what regtest (or perhaps a custom signet) is for. I guess it would be super easy to say something like: - miner 1 ignores blocks marked for reorg - miner 2 marks its blocks for reorg, mines on top of the most work block - miner 2 never mines a block which would have (height % 10 == 1) - miner 1 and miner 2 have the same hashrate, but mine at randomly different times which would mean there's almost always a reorg being mined, people that follow reorgs will see fewer than 1.9x as many blocks as non-reorg nodes, and reorgs won't go on for more than 10 blocks. Cheers, aj ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
