[Bitcoin-development] Some PR preparation
I'm sure it won't be long before Slashdot and a variety of sources start reporting on this event. Bitcoin has been in the media a lot lately, so this story is likely to get some attention. The blowback of this event is mostly psychological, so I think it would be exceptionally wise to start preparing PR comments that can be posted on articles immediately after they go public. This event is likely draw much more negative attention than it deserves, and getting some positiveinformed comments posted up front will potentially make a difference in the way the story is received. Undoubtedly, many articles (and especially commenters) will shape this into the end of Bitcoin. I would describe it as there was a short and mostly-harmless lapse in the ability of the network to reach a consensus, causing transactions to get delayed by a few hours. It *really* needs to be emphasized that coins are safe, and nothing anyone has/could do will change that. And that it would've been extremely difficult to exploit for gain. Transactions got delayed while a bug was fixed. End of story. Hell, someone here should submit their own slashdot article about it! 100% chance this hits slashdot -- it might as well be written by someone who understands it. Similarly, we could be sending sources information to pre-empt misinformation being spread about it. Unfortunately, I have to go to bed, so I can't really do much. I just wanted folks to be on the lookout and be ready to respond to the crazy stuff that's going to hit the media in the next 12 hours. -Alan -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
Just so we're all on the same page, can someone confirm my understanding - are any of the following statements untrue? BDB ran out of locks. However, only on some 0.7 nodes. Others, perhaps nodes using different flags, managed it. We have processed 1mb sized blocks on the testnet. Therefore it isn't presently clear why that particular block caused lock exhaustion when other larger blocks have not. The reason for increasing the soft limit is still present (we have run out of space). Therefore transactions are likely to start stacking up in the memory pool again very shortly, as they did last week. There are no bounds on the memory pool size. If too many transactions enter the pool then nodes will start to die with OOM failures. Therefore it is possible that we have a very limited amount of time until nodes start dying en-masse. Even if nodes do not die, users have no way to find out what the current highest fees/bids for block space are, nor any way to change the fee on sent transactions. Therefore Bitcoin will shortly start to break for the majority of users who don't have a deep understanding of the system. If all the above statements are true, we appear to be painted into a corner - can't roll forward and can't roll back, with very limited time to come up with a solution. I see only a small number of alternatives: 1) Start aggressively trying to block or down-prioritize SatoshiDice transactions at the network level, to buy time and try to avoid mempool exhaustion. I don't know a good way to do this, although it appears that virtually all their traffic is actually coming via blockchain.infos My Wallet service. During their last outage block sizes seemed to drop to around 50kb. Alternatively, ask SD to temporarily suspend their service (this seems like a long shot). 2) Perform a crash hard fork as soon as possible, probably with no changes in it except a new block size limit. Question - try to lift the 1mb limit at the same time, or not? On Tue, Mar 12, 2013 at 2:01 AM, Pieter Wuille pieter.wui...@gmail.com wrote: Hello again, block 015c50b165fcdd33556f8b44800c5298943ac70b112df480c023 (height=225430) seems indeed to have cause pre-0.8 and 0.8 nodes to fork (at least mostly). Both chains are being mined on - the 0.8 one growing faster. After some emergency discussion on #bitcoin-dev, it seems best to try to get the majority mining power back on the old chain, that is, the one which 0.7 accepts (with 01c108384350f74090433e7fcf79a606b8e797f065b130575932 at height 225430). That is the only chain every client out there will accept. BTC Guild is switching to 0.7, so majority should abandon the 0.8 chain soon. Short advice: if you're a miner, please revert to 0.7 until we at least understand exactly what causes this. If you're a merchant, and are on 0.8, stop processing transactions until both sides have switched to the same chain again. We'll see how to proceed afterwards. -- Pieter On Tue, Mar 12, 2013 at 1:18 AM, Pieter Wuille pieter.wui...@gmail.com wrote: Hello everyone, Í've just seen many reports of 0.7 nodes getting stuck around block 225430, due to running out of lock entries in the BDB database. 0.8 nodes do not seem to have a problem. In any case, if you do not have this block: 2013-03-12 00:00:10 SetBestChain: new best=015aab28064a4c521d6a5325ff6e251e8ca2edfdfe6cb5bf832c height=225439 work=853779625563004076992 tx=14269257 date=2013-03-11 23:49:08 you're likely stuck. Check debug.log and db.log (look for 'Lock table is out of available lock entries'). If this is a widespread problem, it is an emergency. We risk having (several) forked chains with smaller blocks, which are accepted by 0.7 nodes. Can people contact pool operators to see which fork they are on? Blockexplorer and blockchain.info seem to be stuck as well. Immediate solution is upgrading to 0.8, or manually setting the number of lock objects higher in your database. I'll follow up with more concrete instructions. If you're unsure, please stop processing transactions. -- Pieter -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the
[Bitcoin-development] Changing the fee on already sent transactions
We can allow for transaction replacement for the purpose of adding fees
to existing transactions safely, and while not increasing the risk of
double-spends by taking advantage of the stubbed out replacement code.
Specifically the replacement code allows for the replacement of a
transaction if a transaction spending the tx that is being replaced is
not in the mempool. Specifically:
664 // Check for conflicts with in-memory transactions
665 CTransaction* ptxOld = NULL;
666 for (unsigned int i = 0; i tx.vin.size(); i++)
667 {
668 COutPoint outpoint = tx.vin[i].prevout;
669 if (mapNextTx.count(outpoint)){
Followed by the actual replacement logic. We could change this logic to
instead evaluate if the candidate replacement does not remove or
decrease the value of any existing outputs. Adding outputs is ok.
Changing the set of inputs is also ok, provided that there are no
conflicts with other spent transactions. DoS attacks would be prevented
by only forwarding/accepting into the mempool replacements that increase
the fees paid by at least MIN_RELAY_TX_FEE * size - essentially the same
decision to allow the broadcast of the transaction in the first place.
Because a transaction can not be replaced if another transaction already
depends on it the change would not increse double-spend risks for
unconfirmed transactions.
Along with this change code can be added to clients to examine the
mempool and recent blocks to determine what fee would be required to get
a transaction confirmed in what time.
Of course, considering our recent fun last night, I'll be the first to
admit that this change needs a lot of testing and careful thought.
However the ability to increase fees on already broadcast transactions
would be very valuable to users as competition for blockchain space
increases.
--
'peter'[:-1]@petertodd.org
signature.asc
Description: Digital signature
--
Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester
Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the
endpoint security space. For insight on selecting the right partner to
tackle endpoint security challenges, access the full report.
http://p.sf.net/sfu/symantec-dev2dev___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
On Tue, Mar 12, 2013 at 10:10:15AM +0100, Mike Hearn wrote: There are no bounds on the memory pool size. If too many transactions enter the pool then nodes will start to die with OOM failures. Therefore it is possible that we have a very limited amount of time until nodes start dying en-masse. Note that nodes dying en-mass due to OOM failures is a relatively benign failure mode as the point as which any particular node would die is uncorrelated with other nodes - it won't cause a network fork. Implementing a simple and stupid while [ true ] do ; ./bitcoind ; done loop combined with ulimit to keep total memory usage to something sane is a perfectly acceptable hack until proper mempool code with expiration can be written. Gavin can talk more about his ideas in that regard. -- 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
However, most nodes are not running in such a loop today. Probably almost no nodes are. I suppose you could consider mass node death to be more benign than a hard fork, but both are pretty damn serious and warrant immediate action. Otherwise we're going to see the number of nodes drop sharply over the coming days as unattended nodes die and then don't get restarted. -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
Yes, 0.7 (yes 0.7!) was not sufficiently tested it had an undocumented and unknown criteria for block rejection, hence the upgrade went wrong. More space in the block is needed indeed, but the real problem you are describing is actually not missing space in the block, but proper handling of mem-pool transactions. They should be pruned on two criteria: 1. if they gets to old 24hr 2. if the client is running out of space, then the oldest should probably be pruned clients are anyway keeping, and re-relaying, their own transactions and hence it would mean only little, and only little for clients. Dropping free / old transaction is a much a better behavior than dying... Even a scheme where the client dropped all or random mempool txes would be a tolerable way of handling things (dropping all is similar to a restart, except for no user intervention). Following that, increase the soft and hard limit to 1 and eg 10MB, but miners should be the last to upgrade. /M On 12/03/2013, at 10:10, Mike Hearn m...@plan99.net wrote: Just so we're all on the same page, can someone confirm my understanding - are any of the following statements untrue? BDB ran out of locks. However, only on some 0.7 nodes. Others, perhaps nodes using different flags, managed it. We have processed 1mb sized blocks on the testnet. Therefore it isn't presently clear why that particular block caused lock exhaustion when other larger blocks have not. The reason for increasing the soft limit is still present (we have run out of space). Therefore transactions are likely to start stacking up in the memory pool again very shortly, as they did last week. There are no bounds on the memory pool size. If too many transactions enter the pool then nodes will start to die with OOM failures. Therefore it is possible that we have a very limited amount of time until nodes start dying en-masse. Even if nodes do not die, users have no way to find out what the current highest fees/bids for block space are, nor any way to change the fee on sent transactions. Therefore Bitcoin will shortly start to break for the majority of users who don't have a deep understanding of the system. If all the above statements are true, we appear to be painted into a corner - can't roll forward and can't roll back, with very limited time to come up with a solution. I see only a small number of alternatives: 1) Start aggressively trying to block or down-prioritize SatoshiDice transactions at the network level, to buy time and try to avoid mempool exhaustion. I don't know a good way to do this, although it appears that virtually all their traffic is actually coming via blockchain.infos My Wallet service. During their last outage block sizes seemed to drop to around 50kb. Alternatively, ask SD to temporarily suspend their service (this seems like a long shot). 2) Perform a crash hard fork as soon as possible, probably with no changes in it except a new block size limit. Question - try to lift the 1mb limit at the same time, or not? On Tue, Mar 12, 2013 at 2:01 AM, Pieter Wuille pieter.wui...@gmail.com wrote: Hello again, block 015c50b165fcdd33556f8b44800c5298943ac70b112df480c023 (height=225430) seems indeed to have cause pre-0.8 and 0.8 nodes to fork (at least mostly). Both chains are being mined on - the 0.8 one growing faster. After some emergency discussion on #bitcoin-dev, it seems best to try to get the majority mining power back on the old chain, that is, the one which 0.7 accepts (with 01c108384350f74090433e7fcf79a606b8e797f065b130575932 at height 225430). That is the only chain every client out there will accept. BTC Guild is switching to 0.7, so majority should abandon the 0.8 chain soon. Short advice: if you're a miner, please revert to 0.7 until we at least understand exactly what causes this. If you're a merchant, and are on 0.8, stop processing transactions until both sides have switched to the same chain again. We'll see how to proceed afterwards. -- Pieter On Tue, Mar 12, 2013 at 1:18 AM, Pieter Wuille pieter.wui...@gmail.com wrote: Hello everyone, Í've just seen many reports of 0.7 nodes getting stuck around block 225430, due to running out of lock entries in the BDB database. 0.8 nodes do not seem to have a problem. In any case, if you do not have this block: 2013-03-12 00:00:10 SetBestChain: new best=015aab28064a4c521d6a5325ff6e251e8ca2edfdfe6cb5bf832c height=225439 work=853779625563004076992 tx=14269257 date=2013-03-11 23:49:08 you're likely stuck. Check debug.log and db.log (look for 'Lock table is out of available lock entries'). If this is a widespread problem, it is an emergency. We risk having (several) forked chains with smaller blocks, which are accepted by 0.7 nodes. Can people contact pool operators to see which fork they are on? Blockexplorer and blockchain.info seem to be stuck as well.
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
On Tue, Mar 12, 2013 at 11:10:47AM +0100, Mike Hearn wrote: However, most nodes are not running in such a loop today. Probably almost no nodes are. I suppose you could consider mass node death to be more benign than a hard fork, but both are pretty damn serious and warrant immediate action. Otherwise we're going to see the number of nodes drop sharply over the coming days as unattended nodes die and then don't get restarted. I'm sure if mass node death becomes an issue miners will have plenty of incentive to temporarily, or permanently, setup some high-memory and high-bandwidth nodes to accept transactions. The DNS seeds sort by reliability so it won't be long before nodes are connecting to them. My home machine has 16GB of ram, bigger than the whole blockchain. -- 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
On Tue, Mar 12, 2013 at 11:13:09AM +0100, Michael Gronager wrote: Following that, increase the soft and hard limit to 1 and eg 10MB, but miners should be the last to upgrade. We just saw a hard-fork happen because we ran into previously unknown scaling issues with the current codebase. Why follow that up immediately with yet another jump into unknown scaling territory? I suspect the PR fallout from another chain split, let alone multiple splits, will be far damaging to Bitcoin than stories along the lines of Gee, actually it'd kinda expensive to do a Bitcoin transaction these days due to all the competition. I dunno, I guess it must be really popular and valuable or something? Lets let the issue rest for a while, and we can all have some time to work on our various approaches to solving the problem. The worst that will happen is growth temporarily slows - hardly a disaster I think. -- 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
clients are anyway keeping, and re-relaying, their own transactions and hence it would mean only little, and only little for clients. Not all end-user clients are always-on though -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
We just saw a hard-fork happen because we ran into previously unknown scaling issues with the current codebase. Technically, it with the previous codebase ;) -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Some PR preparation
On Tuesday, March 12, 2013 7:03:54 AM Alan Reiner wrote: I'm sure it won't be long before Slashdot and a variety of sources start reporting on this event. Bitcoin has been in the media a lot lately, so this story is likely to get some attention. The blowback of this event is mostly psychological, so I think it would be exceptionally wise to start preparing PR comments that can be posted on articles immediately after they go public. This event is likely draw much more negative attention than it deserves, and getting some positiveinformed comments posted up front will potentially make a difference in the way the story is received. Undoubtedly, many articles (and especially commenters) will shape this into the end of Bitcoin. I would describe it as there was a short and mostly-harmless lapse in the ability of the network to reach a consensus, causing transactions to get delayed by a few hours. It *really* needs to be emphasized that coins are safe, and nothing anyone has/could do will change that. And that it would've been extremely difficult to exploit for gain. Transactions got delayed while a bug was fixed. End of story. I think we should be careful not to downplay the reality either. For a number of hours, transactions could have received up to N confirmations and then still been reversed. While we could contact the bigger payment processors, I saw people still trying to buy/sell on OTC, whom could have been scammed even by taking standard precautions. Luke -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
I'm not even sure I'd say the upgrade went wrong. The problem if anything is the upgrade didn't happen fast enough. If we had run out of block space a few months from now, or if miners/merchants/exchanges had upgraded faster, it'd have made more sense to just roll forward and tolerate the loss of the older clients. This really reinforces the importance of keeping nodes up to date. On Tue, Mar 12, 2013 at 12:44 PM, Pieter Wuille pieter.wui...@gmail.com wrote: On Tue, Mar 12, 2013 at 11:13:09AM +0100, Michael Gronager wrote: Yes, 0.7 (yes 0.7!) was not sufficiently tested it had an undocumented and unknown criteria for block rejection, hence the upgrade went wrong. We're using 0.7 as a short moniker for all clients, but this was a limitation that all BDB-based bitcoins ever had. The bug is simply a limit in the number of lock objects that was reached. It's ironic that 0.8 was supposed to solve all problems we had due to BDB (except the wallet...), but now it seems it's still coming back to haunt us. I really hated telling miners to go back to 0.7, given all efforts to make 0.8 signficantly more tolerable... More space in the block is needed indeed, but the real problem you are describing is actually not missing space in the block, but proper handling of mem-pool transactions. They should be pruned on two criteria: 1. if they gets to old 24hr 2. if the client is running out of space, then the oldest should probably be pruned clients are anyway keeping, and re-relaying, their own transactions and hence it would mean only little, and only little for clients. Dropping free / old transaction is a much a better behavior than dying... Even a scheme where the client dropped all or random mempool txes would be a tolerable way of handling things (dropping all is similar to a restart, except for no user intervention). Right now, mempools are relatively small in memory usage, but with small block sizes, it indeed risks going up. In 0.8, conflicting (=double spending) transactions in the chain cause clearing the mempool of conflicts, so at least the mempool is bounded by the size of the UTXO subset being spent. Dropping transactions from the memory pool when they run out of space seems a correct solution. I'm less convinced about a deterministic time-based rule, as that creates a double spending incentive at that time, and a counter incentive to spam the network with your risking-to-be-cleared transaction as well. Regarding the block space, we've seen the pct% of one single block chain space consumer grow simultaneously with the introduction of larger blocks, so I'm not actually convinced there is right now a big need for larger blocks (note: right now). The competition for block chain space is mostly an issue for client software which doesn't deal correctly with non-confirming transactions, and misleading users. It's mostly a usability problem now, but increasing block sizes isn't guaranteed to fix that; it may just make more space for spam. However, the presence of this bug, and the fact that a full solution is available (0.8), probably helps achieving consensus fixing it (=a hardfork) is needed, and we should take advantage of that. But please, let's not rush things... -- Piter -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
Well a reversed upgrade is an upgrade that went wrong ;) Anyway, the incident makes it even more important for people to upgrade, well except, perhaps, for miners... Forks are caused by rejection criteria, hence: 1. If you introduce new rejection criteria in an upgrade miners should upgrade _first_. 2. If you loosen some rejection criteria miners should upgrade _last_. 3. If you keep the same criteria assume 2. /M On 12/03/2013, at 13:11, Mike Hearn m...@plan99.net wrote: I'm not even sure I'd say the upgrade went wrong. The problem if anything is the upgrade didn't happen fast enough. If we had run out of block space a few months from now, or if miners/merchants/exchanges had upgraded faster, it'd have made more sense to just roll forward and tolerate the loss of the older clients. This really reinforces the importance of keeping nodes up to date. On Tue, Mar 12, 2013 at 12:44 PM, Pieter Wuille pieter.wui...@gmail.com wrote: On Tue, Mar 12, 2013 at 11:13:09AM +0100, Michael Gronager wrote: Yes, 0.7 (yes 0.7!) was not sufficiently tested it had an undocumented and unknown criteria for block rejection, hence the upgrade went wrong. We're using 0.7 as a short moniker for all clients, but this was a limitation that all BDB-based bitcoins ever had. The bug is simply a limit in the number of lock objects that was reached. It's ironic that 0.8 was supposed to solve all problems we had due to BDB (except the wallet...), but now it seems it's still coming back to haunt us. I really hated telling miners to go back to 0.7, given all efforts to make 0.8 signficantly more tolerable... More space in the block is needed indeed, but the real problem you are describing is actually not missing space in the block, but proper handling of mem-pool transactions. They should be pruned on two criteria: 1. if they gets to old 24hr 2. if the client is running out of space, then the oldest should probably be pruned clients are anyway keeping, and re-relaying, their own transactions and hence it would mean only little, and only little for clients. Dropping free / old transaction is a much a better behavior than dying... Even a scheme where the client dropped all or random mempool txes would be a tolerable way of handling things (dropping all is similar to a restart, except for no user intervention). Right now, mempools are relatively small in memory usage, but with small block sizes, it indeed risks going up. In 0.8, conflicting (=double spending) transactions in the chain cause clearing the mempool of conflicts, so at least the mempool is bounded by the size of the UTXO subset being spent. Dropping transactions from the memory pool when they run out of space seems a correct solution. I'm less convinced about a deterministic time-based rule, as that creates a double spending incentive at that time, and a counter incentive to spam the network with your risking-to-be-cleared transaction as well. Regarding the block space, we've seen the pct% of one single block chain space consumer grow simultaneously with the introduction of larger blocks, so I'm not actually convinced there is right now a big need for larger blocks (note: right now). The competition for block chain space is mostly an issue for client software which doesn't deal correctly with non-confirming transactions, and misleading users. It's mostly a usability problem now, but increasing block sizes isn't guaranteed to fix that; it may just make more space for spam. However, the presence of this bug, and the fact that a full solution is available (0.8), probably helps achieving consensus fixing it (=a hardfork) is needed, and we should take advantage of that. But please, let's not rush things... -- Piter -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
On Tue, Mar 12, 2013 at 2:10 AM, Mike Hearn m...@plan99.net wrote: BDB ran out of locks. However, only on some 0.7 nodes. Others, perhaps nodes using different flags, managed it. We have processed 1mb sized blocks on the testnet. Therefore it isn't presently clear why that particular block caused lock exhaustion when other larger blocks have not. Locks are only mostly related to block size, once I heard what was happening I was unsurprised the max sized test blocks hadn't triggered it. Therefore it is possible that we have a very limited amount of time until nodes start dying en-masse. Scaremongering much? Egads. On Tue, Mar 12, 2013 at 5:27 AM, Michael Gronager grona...@ceptacle.com wrote: Forks are caused by rejection criteria, hence: 1. If you introduce new rejection criteria in an upgrade miners should upgrade _first_. 2. If you loosen some rejection criteria miners should upgrade _last_. 3. If you keep the same criteria assume 2. And ... if you aren't aware that you're making a change ??? -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
On 3/12/2013 5:18 AM, Jorge Timón wrote: A related question...some people mentioned yesterday on #bitcoin-dev that 0.5 appeared to be compatible with 0.8. Was that only for the fatal block and would have forked 0.8 later too or is it something else? I'm having a hard time understanding this 0.5 thing, if someone can bring some light to it I would appreciate it. Thanks in advance It was reported that not all 0.7 died from the BDB error either. This will likely take a post-mortem to determine exactly what build environments and versions are incompatible, by feeding each the bloated block (hopefully there are lots of snapshots of the bad chain being the best height for testing; I forgot to get one). -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
Forks are caused by rejection criteria, hence: 1. If you introduce new rejection criteria in an upgrade miners should upgrade _first_. 2. If you loosen some rejection criteria miners should upgrade _last_. 3. If you keep the same criteria assume 2. And ... if you aren't aware that you're making a change ??? then only half should upgrade :-P Well I thought I covered that by 3... But, question is of course if we could have been in a situation where 0.8 had been the one rejecting blocks? So miners could go with a filtering approach: only connect to the network through a node of a version one less than the current. That would still have caused block 225430 to be created, but it would never have been relayed and hence no harm. (and if the issue had been in 0.8 the block would not even have been accepted there in the first place). Downside is some lost seconds. /M -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Some PR preparation
On Tue, Mar 12, 2013 at 8:10 AM, Luke-Jr l...@dashjr.org wrote: I think we should be careful not to downplay the reality either. For a number of hours, transactions could have received up to N confirmations and then still been reversed. While we could contact the bigger payment processors, I saw people still trying to buy/sell on OTC, whom could have been scammed even by taking standard precautions. I don't want to misrepresent what happened, but how much of that was really a risk? The block was rejected, but the transactions were not. Any valid transactions to hit the network would get added to everyone's memory pool and mined in both chains. Thus all nodes would still reject double-spend attempts. As far as I understood it, you would've had to have majority mining power on one of the chains (and both had non-negligible computing power on them), so double-spending still required an exceptional amount of resources -- just not the normal 50% that is normally needed. Perhaps... 10%? But how many people can even have 10%? In addition to that, a victim needs to be found that hasn't seen the alert, is willing to execute a large transaction, and is on the wrong side of the chain. Is this incorrect? Yes, there was less resources needed to execute an attack -- but it still required a very powerful attacker, way outside the scope of regular users. -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Some PR preparation
Can some enterprising soul determine if there were any double-spend attempts? I'm assuming no, and if that's the case, we should talk about that publicly. Either way, I think it's generally another test well done by everyone; people pitched in on PR, tech, communication, yay Bitcoin! On Tue, Mar 12, 2013 at 9:55 AM, Alan Reiner etothe...@gmail.com wrote: On Tue, Mar 12, 2013 at 8:10 AM, Luke-Jr l...@dashjr.org wrote: I think we should be careful not to downplay the reality either. For a number of hours, transactions could have received up to N confirmations and then still been reversed. While we could contact the bigger payment processors, I saw people still trying to buy/sell on OTC, whom could have been scammed even by taking standard precautions. I don't want to misrepresent what happened, but how much of that was really a risk? The block was rejected, but the transactions were not. Any valid transactions to hit the network would get added to everyone's memory pool and mined in both chains. Thus all nodes would still reject double-spend attempts. As far as I understood it, you would've had to have majority mining power on one of the chains (and both had non-negligible computing power on them), so double-spending still required an exceptional amount of resources -- just not the normal 50% that is normally needed. Perhaps... 10%? But how many people can even have 10%? In addition to that, a victim needs to be found that hasn't seen the alert, is willing to execute a large transaction, and is on the wrong side of the chain. Is this incorrect? Yes, there was less resources needed to execute an attack -- but it still required a very powerful attacker, way outside the scope of regular users. -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- -- [image: CoinLab Logo]PETER VESSENES CEO *pe...@coinlab.com * / 206.486.6856 / SKYPE: vessenes 811 FIRST AVENUE / SUITE 480 / SEATTLE, WA 98104 -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Some PR preparation
On Tue, Mar 12, 2013 at 9:55 AM, Alan Reiner etothe...@gmail.com wrote: I don't want to misrepresent what happened, but how much of that was really a risk? The block was rejected, but the transactions were not. Some but not much. If someone flooded a bunch of duplicate concurrently announcing both spends to as many nodes as they could reach they would almost certainly gotten some conflicts into both chains. Then both chains would have gotten 6 confirms. Then one chain would pop and anyone on the popped side would see 6 confirm transactions undo. This attack would not require any particular resources, and only enough technical sophistication to run something like pynode to give raw txn to nodes at random. The biggest barriers against it were people being uninterested in attacking (as usual for all things) and there not being many (any?) good targets who hadn't shut down their deposits. They would have to have accepted deposits with 12 confirms and let you withdraw. During the event an attacker could have gotten of their deposit-able funds. On Tue, Mar 12, 2013 at 10:35 AM, Peter Vessenes pe...@coinlab.com wrote: Can some enterprising soul determine if there were any double-spend attempts? I'm assuming no, and if that's the case, we should talk about that publicly. There were circulating double-spends during the fork (as were visible on blockchain.info). I don't know if any conflicts made it into the losing chain, however. It's not too hard to check to see what inputs were consumed in the losing fork and see if any have been consumed by different transactions now. I agree it would be good to confirm no one was ripped off, even though we can't say there weren't any attempts. -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Some PR preparation
On Tue, Mar 12, 2013 at 11:09 AM, Gregory Maxwell gmaxw...@gmail.com wrote: On Tue, Mar 12, 2013 at 10:35 AM, Peter Vessenes pe...@coinlab.com wrote: Can some enterprising soul determine if there were any double-spend attempts? I'm assuming no, and if that's the case, we should talk about that publicly. [snip] I agree it would be good to confirm no one was ripped off, even though we can't say there weren't any attempts. https://bitcointalk.org/index.php?topic=152348.msg1616747#msg1616747 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Some PR preparation
Just a quick and dirty check if something bad actually happened. 430 transactions that were confirmed in the alt-chain, are not confirmed in the true blockchain. The good news is that as far as I can tell most of them are low volume transactions destined for SD. 7 transactions were true double spends, or to be more precise transactions in which an conflicting transaction was confirmed in the new chain (with their respective amount): 12814b8ad57ce5654ba69eb26a52ddae1bff42093ca20cef3ad96fe7fd85d195 261 BTC cb36ba33b3ecd4d3177d786209670c9e6cdf95eb62be54986f0b49ca292714af 0.06 BTC 7192807f952b252081d0db0aa7575c4695b945820adaf7776b7189e6b3d86f96 0.01 BTC 355d4ea51c3b780cf0b10e8099a06a31484e0060bc140b63f3d6e5fb713ace5e 0.05 BTC b961bc0c663a46893afd3166a604e7e2639533522d9fec61fdb95eb665e86f5a 0.61 BTC 138063e4bdb76feaa511f1e7f9c681eb468ef9140c141671741c965e503b84c6 1.62 BTC a10bd194cdbf9aa4c12eb0b120056998a081a9b0d93d70570edff24dec831f90 0.81 So the one transaction that really hurt was the one published on BitcoinTalk. We're not yet out of the woods as some of the 423 transactions still have a chance of being doublespent, but looks like it's not that bad after all. Cheers, Chris P.S.: For a complete list of transactions see http://pastebin.com/wctJU3Ln -- Christian Decker On Tue, Mar 12, 2013 at 7:39 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Tue, Mar 12, 2013 at 11:09 AM, Gregory Maxwell gmaxw...@gmail.com wrote: On Tue, Mar 12, 2013 at 10:35 AM, Peter Vessenes pe...@coinlab.com wrote: Can some enterprising soul determine if there were any double-spend attempts? I'm assuming no, and if that's the case, we should talk about that publicly. [snip] I agree it would be good to confirm no one was ripped off, even though we can't say there weren't any attempts. https://bitcointalk.org/index.php?topic=152348.msg1616747#msg1616747 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Some PR preparation
Thanks Chris. Yep, looks like an honest-ish user managed to accidentally get one tx into one chain and another into the other. I think I'd cautiously say that if OKPay gets their cash back, or freezes his balance nobody is out BTC for last night, (instead just time and effort). I'm doing a little FUD-fighting right now, but will try and pick up a bit more if necessary tonight after my flight lands. I think this is mostly over the heads of a lot of our typical media contacts, though. Peter On Tue, Mar 12, 2013 at 12:53 PM, Christian Decker decker.christ...@gmail.com wrote: Just a quick and dirty check if something bad actually happened. 430 transactions that were confirmed in the alt-chain, are not confirmed in the true blockchain. The good news is that as far as I can tell most of them are low volume transactions destined for SD. 7 transactions were true double spends, or to be more precise transactions in which an conflicting transaction was confirmed in the new chain (with their respective amount): 12814b8ad57ce5654ba69eb26a52ddae1bff42093ca20cef3ad96fe7fd85d195 261 BTC cb36ba33b3ecd4d3177d786209670c9e6cdf95eb62be54986f0b49ca292714af 0.06 BTC 7192807f952b252081d0db0aa7575c4695b945820adaf7776b7189e6b3d86f96 0.01 BTC 355d4ea51c3b780cf0b10e8099a06a31484e0060bc140b63f3d6e5fb713ace5e 0.05 BTC b961bc0c663a46893afd3166a604e7e2639533522d9fec61fdb95eb665e86f5a 0.61 BTC 138063e4bdb76feaa511f1e7f9c681eb468ef9140c141671741c965e503b84c6 1.62 BTC a10bd194cdbf9aa4c12eb0b120056998a081a9b0d93d70570edff24dec831f90 0.81 So the one transaction that really hurt was the one published on BitcoinTalk. We're not yet out of the woods as some of the 423 transactions still have a chance of being doublespent, but looks like it's not that bad after all. Cheers, Chris P.S.: For a complete list of transactions see http://pastebin.com/wctJU3Ln -- Christian Decker On Tue, Mar 12, 2013 at 7:39 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Tue, Mar 12, 2013 at 11:09 AM, Gregory Maxwell gmaxw...@gmail.com wrote: On Tue, Mar 12, 2013 at 10:35 AM, Peter Vessenes pe...@coinlab.com wrote: Can some enterprising soul determine if there were any double-spend attempts? I'm assuming no, and if that's the case, we should talk about that publicly. [snip] I agree it would be good to confirm no one was ripped off, even though we can't say there weren't any attempts. https://bitcointalk.org/index.php?topic=152348.msg1616747#msg1616747 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- -- [image: CoinLab Logo]PETER VESSENES CEO *pe...@coinlab.com * / 206.486.6856 / SKYPE: vessenes 811 FIRST AVENUE / SUITE 480 / SEATTLE, WA 98104 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Blocking uneconomical UTXO creation
Instead of thinking in terms of blocking uneconomical transactions (how would a node even determine what's economical?), what about thinking in terms of paying for a feed of economical (i.e. profitable) transactions? There is a market for fee bearing, profitable transactions...if there is no one willing to pay to receive a transaction, then no one will bother propagating it. Such a system would make it possible to determine the probability of confirmation in a given timeframe for a given fee. On Tue, Mar 12, 2013 at 3:49 AM, Peter Todd p...@petertodd.org wrote: On Sat, Mar 09, 2013 at 11:31:55PM -0500, Peter Todd wrote: As discussed endlessly data in the UTXO set is more costly, especially in the long run, than transaction data itself. The fee system is per KB in a block, and thus doesn't properly capture the long-term costs of UTXO creation. There's been a lot of discussion about this issue, and many people have asked that Bitcoin not arbitrarily block interesting potential uses of provably unspendable txouts for data applications, and similarly spendable txouts representing assets. I've changed my hardline position and now think we should support all that stuff. However, there is one remaining class of txout not yet talked about, unspendable but not provably so txouts. For instance we could make the following a standard transaction type: scriptPubKey: OP_HASH160 20 byte digest OP_EQUALVERIFY data scriptSig: data Of course, usually the 20 byte digest would be picked randomly, but it might not be, and thus all validating nodes will always have a copy of the data. With the 10KB limit on script sizes you can fit 9974 bytes of data per transaction output with very little waste. A good application is timestamping, with the advantage over coinbase/merkle tree systems in that you don't have to wait until your timestamp confirms, or even store the timestamp at all. Another application, quite possible with large block sizes and hence cheap or free transactions, is secure data backups. In particular such a service, perhaps called Google Chain Storage, can offer the unique guarantee that you can know you're data is secure by simply performing a successful Bitcoin transaction. -- 'peter'[:-1]@petertodd.org -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Stephen Pair, Co-Founder, CTO Does *your* website accept cash? bitpay.com [image: bitpay-small] ABC6 C11B BF75 9E2B FC6A B3E0 7B96 40B2 CAC0 C158 image001.png-- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
