Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-14 Thread Jean-Pierre Rupp
Jean-Pierre Rupp from Haskoin here.

I support a hard fork to fix consensus bugs.  The Bitcoin protocol should 
eventually get to a state where it is documented in a clear and understandable 
fashion.  Bugs are bugs, and are the enemy.  We should not attempt to live with 
them.  We should be opening a process of thoroughly documenting and reparing 
consensus bugs on a separate branch, and eventually schedule a hard fork.

There are two good things that will come out of that:

1. Known bugs will be gone, and
2. We will have a process in place to get rid of future bugs in eventual future 
hard forks.

We do not need to become paranoid about the ramifications of a hard fork, or 
how it will open the door for unwanted changes in the protocol.  We are 
discussing about removing bugs, and bugs that could be used to exploit the 
network in ways that may not be immediately obvious.

There are 144 blocks generated per day by groups of miners that are mostly 
identified.  It is not going to be a titanic task to get consensus from the 
main mining pools on fixing this at the mining level.  We must address how the 
fixes for some of these bugs affect other types of software such as wallets.  I 
can think that fixing the bug where OP_CHECKMULTISIG pops an extra value from 
the stash could be more traumatic, since it requires anything that creates and 
validates multi-signature transactions to change the way it works.  Hardware 
wallets could be impacted.  But most of the consensus bugs would not affect the 
way the vast majority of bitcoin transactions that are currently created.  
Therefore it should not be traumatic at all for users, but only really affect 
mining pools, who would only need to be convinced to upgrade their bitcoind 
well in advance, which seems to me that it is not an issue at all.

We should not compare doing a Bitcoin hard-fork with doing something like 
deploying IPv6 world-wide or enforcing TLS and SPF on every SMTP connection.  
We should not conflate Bitcoin with other network protocols.  The Bitcoin 
protocol is actually relatively easy to upgrade at this point.  Let's take 
advantage of this fact.

On 06/11/14 15:36, Justus Ranvier wrote:
> Because Bitcoin has a extra consensus requirements, requirements which
> are really rare in engineering, the necessity of fixing bugs is even
> greater.

-- 
Be Happy :)



0x310A8A5B.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature
--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-07 Thread Mike Hearn
>
> > Who benefits from not fixing bugs in Bitcoin?
>
> We can bring up politics if you want.


No, please don't. That question was rhetorical, not an invitation for you
to try and convince bystanders that anyone who disagrees with you is a
shadowy Agent Of Centralisation or an idiot. You use that tactic way too
much: it's obnoxious and you need to stop it.

Hard forks vs soft forks are *purely* about whether you drag along old
nodes in a quasi-broken state. They do not reduce total work needed by the
community one iota. Non-miners who wish to reject a soft fork can easily
run a node that does so, if they wanted to - the voting mechanism still
boils down to "which side of the fork do I accept in my economic activity".
It's certainly garbage to claim that the reason to want to avoid soft forks
is being an Evil Centralised Foundation:  this is about a set of
engineering tradeoffs only.
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-07 Thread Wladimir
On Fri, Nov 7, 2014 at 12:30 PM, Clément Elbaz  wrote:
> Thinking out loud here : would it make sense to separate the consensus code
> into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project
> that could be used by anyone ?

Yes, we're moving in that direction. First with a script verification
library in 0.10, which will be extended to other parts of the
consensus by 0.11 and after that.

Wladimir

--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-07 Thread Peter Todd
On Fri, Nov 07, 2014 at 11:30:22AM +, Clément Elbaz wrote:
> Thinking out loud here : would it make sense to separate the consensus code
> into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project
> that could be used by anyone ?

That's a pretty old idea, and we're working on it. First step is a
stand-alone script verification library:

https://github.com/theuni/bitcoin/blob/da18a0266c4de76c2a461cc2984aa2fa066c42f5/src/script/bitcoinconsensus.h

#ifndef H_BITCOIN_BITCOINCONSENSUS
#define H_BITCOIN_BITCOINCONSENSUS

#if defined(BUILD_BITCOIN_INTERNAL) && defined(HAVE_CONFIG_H)
#include "config/bitcoin-config.h"
  #if defined(_WIN32)
#if defined(DLL_EXPORT)
  #if defined(HAVE_FUNC_ATTRIBUTE_DLLEXPORT)
#define EXPORT_SYMBOL __declspec(dllexport)
  #else
#define EXPORT_SYMBOL
  #endif
#endif
  #elif defined(HAVE_FUNC_ATTRIBUTE_VISIBILITY)
#define EXPORT_SYMBOL __attribute__ ((visibility ("default")))
  #endif
#elif defined(MSC_VER) && !defined(STATIC_LIBBITCOINCONSENSUS)
  #define EXPORT_SYMBOL __declspec(dllimport)
#endif

#ifndef EXPORT_SYMBOL
  #define EXPORT_SYMBOL
#endif

#ifdef __cplusplus
extern "C" {
#else
#include 
#endif

/** Script verification flags */
enum
{
bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NONE  = 0,
bitcoinconsensus_SCRIPT_FLAGS_VERIFY_P2SH  = (1U << 0), // evaluate 
P2SH (BIP16) subscripts
};

/// Returns true if the input nIn of the serialized transaction pointed to by
/// txTo correctly spends the scriptPubKey pointed to by scriptPubKey under
/// the additional constraints specified by flags.
EXPORT_SYMBOL bool bitcoinconsensus_verify_script(const unsigned char 
*scriptPubKey, const unsigned int scriptPubKeyLen,
const unsigned char *txTo, const 
unsigned int txToLen,
const unsigned int nIn, const unsigned int 
flags);

EXPORT_SYMBOL unsigned int bitcoinconsensus_version();

#ifdef __cplusplus
} // extern "C"
#endif

#undef EXPORT_SYMBOL

#endif // H_BITCOIN_BITCOINCONSENSUS

-- 
'peter'[:-1]@petertodd.org
01208038fd7130083ff118147890dbb37913ffa83c1f48cd


signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-07 Thread Clément Elbaz
Thinking out loud here : would it make sense to separate the consensus code
into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project
that could be used by anyone ?

Bitcoin Core (and any other application wishing to do so) could be based on
it.

The kernel would just contain the absolute minimum code for reaching
consensus, leaving every other aspects of the implementation to the
applications built with it.

It would be stateless : it would provide an interface to submit a
block/transaction to be validated, including the context needed to validate
it (the previously validated blocks referenced by this block/transaction).

What do you think ?

Clément

Le Fri Nov 07 2014 at 9:49:05 AM, Peter Todd  a écrit :

On Fri, Nov 07, 2014 at 09:07:47AM +0100, Tamas Blummer wrote:
> > Peter,
> >
> > forking would work best with a freeze of the consensus code. Do you see
> any chance for that?
>
> To a first approximation the consensus code *is* frozen; if we introduce
> any consensus changes into it at this point it's due to a mistake, not
> intentionally.
>
> Of course, that's not including the two serious soft-fork proposals in
> the air right now, Pieter Wuille's BIP62 and my CHECKLOCKTIMEVERIFY.
> However dealing with proposed changes like those in an environment where
> the competing implementations all use essentially the same
> consensus-critical code is much easier than in an environment where they
> don't; I say this on both a technical and political level.
>
> --
> 'peter'[:-1]@petertodd.org
> 0c901eb1b6b765519b99c3afd7a9062ff4cfa29666ce140d
> 
> --
> ___
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-07 Thread Peter Todd
On Fri, Nov 07, 2014 at 09:07:47AM +0100, Tamas Blummer wrote:
> Peter,
> 
> forking would work best with a freeze of the consensus code. Do you see any 
> chance for that?

To a first approximation the consensus code *is* frozen; if we introduce
any consensus changes into it at this point it's due to a mistake, not
intentionally.

Of course, that's not including the two serious soft-fork proposals in
the air right now, Pieter Wuille's BIP62 and my CHECKLOCKTIMEVERIFY.
However dealing with proposed changes like those in an environment where
the competing implementations all use essentially the same
consensus-critical code is much easier than in an environment where they
don't; I say this on both a technical and political level.

-- 
'peter'[:-1]@petertodd.org
0c901eb1b6b765519b99c3afd7a9062ff4cfa29666ce140d


signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-07 Thread Tamas Blummer
Peter,

forking would work best with a freeze of the consensus code. Do you see any 
chance for that?

Tamas Blummer


On Nov 7, 2014, at 1:03 AM, Peter Todd  wrote:
> Forking the codebase, rather than rewriting it, best
> ensures that your code actually implements the protocol properly, is
> safe to use for mining, and actually gets used.
> 
> Rewriting Bitcoin Core is a fun project, but it's terrible politics.



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Gregory Maxwell
On Thu, Nov 6, 2014 at 11:12 PM, Peter Todd  wrote:
> BIP62 does make life easier for wallet authors as they don't have to
> deal with malleability - maybe! -

Yes, I agree for most contract purposes CTLV is what you want to be
using, instead of refund transactions beyond being more clearly
correct, it shrinks the protocol state machine by one step.

Though BIP62 also achieves the secondary goal of making required
implementation behaviour more explicit (e.g. the parts enforced in all
transactions), and that shouldn't be discounted.

They're somewhat orthogonal, somwhat complementary things.

--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Peter Todd
On Thu, Nov 06, 2014 at 05:36:55PM -0600, Justus Ranvier wrote:
> This explanation is completely incoherent.
> 
> Because Bitcoin has a extra consensus requirements, requirements which
> are really rare in engineering, the necessity of fixing bugs is even
> greater.
> 
> There are two general ways to fix bugs: either as part of a
> controlled, planned, and managed process, or as a response to an
> immediate disaster.
> 
> The alternative to scheduling and planning the upgrades which are
> necessary to fix the bugs in the protocol, where such fixes can be
> written, tested, and documented at leisure, is to wait for some crisis
> and slap on another bandaid when the network breaks again (like it did
> March of last year).

The protocol is what the protocol is; the bugs are when you don't match
the protocol.

> Who benefits from not fixing bugs in Bitcoin?

We can bring up politics if you want.

In the current model, the specification *is* the protocol, and the
Bitcoin Core team is scared to death of changing anything; they've got
very little real power. Soft-forks are the minimum-viable way of making
changes to the protocol, and it's very clear how they get adopted:
minerr consensus. They're also a fundemental way of changing the
protocol that is impossible to prevent, so you might as well use it.

Hard-forks require political consensus to achieve, and the way you
create that political consensus is by creating committes, groups,
associations... Foundations. Every last one of those things requires
centralization and political power.

You know, the smartest thing the Bitcoin Foundation could do if they
wanted to cement their place in the Bitcoin ecosystem as a power broker
would be to setup a program of periodic hard-forks, say every year or
two, and then manage the committees that decide what goes into those
hard-forks. That they haven't suggested that yet is a sign that they're
either not evil, or they don't understand Bitcoin very well.

I think programmers find this reality hard to accept, because they're
mostly interested in writing code that'll get widely used. To them it's
hard to accept that the Bitcoin protocol *is* a few thousand lines of
C++ code, and they're not good enough to write their own implementation
and make it match; if we replaced programmers with writers we might get
the equally bizzare and pointless situation of people taking perfectly
good RFCs and rewriting them in their own words.

If you do care about keeping the politics of Bitcoin development free
from centralized control you should do what I advised the Dark Wallet
team to do a year ago: fork Bitcoin Core and change the
non-consensus-critical code that implements policy. I've done this
myself in a minor way with my replace-by-fee(1) version. Luke-Jr has
also done this with his Eligius branch, a fork that something like 30%
of the Bitcoin hashing power appear to run. (Discus Fish has been mining
non-standard transactions(2) lately)

Multiple *forks* of the Bitcoin Core reference client that are actually
getting used by miners and other users ensures that no one group
maintaining such a fork has the ability to change anything without
strong consensus. Forking the codebase, rather than rewriting it, best
ensures that your code actually implements the protocol properly, is
safe to use for mining, and actually gets used.

Rewriting Bitcoin Core is a fun project, but it's terrible politics.

1) https://github.com/petertodd/bitcoin/tree/replace-by-fee-v0.9.3
2) 
https://blockchain.info/tx/e24a4085c54a6362e615f8eab758c12d80e488b73757e6d2b8ab6bfc8be7007e

-- 
'peter'[:-1]@petertodd.org
08f2290924a6882928d4566f487f33cc57203a6535795201


signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11/06/2014 05:26 PM, Peter Todd wrote:> For the same reason we
don't do hard-forking upgrades of basically every
> protocol on the planet on a regular basis, even when we don't have 
> consensus problems to worry about.
> 
> Flag days are really rare in engineering, and for good reason.


This explanation is completely incoherent.

Because Bitcoin has a extra consensus requirements, requirements which
are really rare in engineering, the necessity of fixing bugs is even
greater.

There are two general ways to fix bugs: either as part of a
controlled, planned, and managed process, or as a response to an
immediate disaster.

The alternative to scheduling and planning the upgrades which are
necessary to fix the bugs in the protocol, where such fixes can be
written, tested, and documented at leisure, is to wait for some crisis
and slap on another bandaid when the network breaks again (like it did
March of last year).

Who benefits from not fixing bugs in Bitcoin?

- -- 
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k
-BEGIN PGP SIGNATURE-

iQEcBAEBAgAGBQJUXAYVAAoJEMP3uyY4RQ21YxMH/3O+vFK2jDXV5V8IIsJnU/u1
D4gYyVm89E0zmGTyLAUYCJGj0eg5tMyUUzu62hOECOeQuKdVi+mbkLi4TiF0sHIb
8k25MgqJgzH/021eoI2g2w1FrDlZut02LNX/V09+owd1yhp+SEXZ3/+HlqsZXhsv
/u9u5OayzhGlzS6apQtrosl5P+KIquHqIbtwBtPOb2rvlL0miJ6sRcAH2JCXCBDT
HMcswMtIGZbgqL/K7e/6vH7dUWdp0866RZXvt7aWGNUgvxHbGMs+zsnxp4nNslxM
wqL71gTmtMnLcw0GtqmXPjcjo+adrPnqp45xc9lSt23PGjxxfR0FKYIPb2uZdq8=
=9GOY
-END PGP SIGNATURE-


0x38450DB5.asc
Description: application/pgp-keys
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Peter Todd
On Thu, Nov 06, 2014 at 04:48:54PM -0600, Justus Ranvier wrote:
> Why not schedule protocol upgrades every two years for the foreseeable
> future?

For the same reason we don't do hard-forking upgrades of basically every
protocol on the planet on a regular basis, even when we don't have
consensus problems to worry about.

Flag days are really rare in engineering, and for good reason.

-- 
'peter'[:-1]@petertodd.org
08f2290924a6882928d4566f487f33cc57203a6535795201


signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Peter Todd
On Thu, Nov 06, 2014 at 11:11:52PM +0100, Jeff Garzik wrote:
> IMO, CHECKLOCKTIMEVERIFY should be included in that list, too.
> 
> RE soft fork vs. hard fork:  It's about this time at Mike Hearn will
> chime in, on the side of hard forks.  Hard forks are in a sense much
> cleaner, and permit solving problems not otherwise solvable with a
> hard fork.  However, hard forks clearly have risks, notably the Big
> Risk akin to a US Constitutional Convention:  once you open the door,
> anything can happen, any rule no matter how "sacred" can be changed.

I think people in this community often miss the serious political and
legal ramifications of hard-forks. Being in the social position of being
able to succesfully pull off hard-forks, particularly for new features,
is clear evidence that you have de-facto control over the system.
Regulators around the world appear to be going in directions that would
make that control subject to regulation and licensing, e.g. the European
Banking Association proposals, and initial Bitlicense proposals.

Equally, look how hard-forks - known as flag days elsewhere - are
generally considered to be dangerous and worth avoiding in other
contexts due to simple engineering reasons. It's just easier to upgrade
systems in backward compatible ways, especially when they incorporate
features specifically to make that possible. (as does bitcoin!)

> Soft forks are not without their own risks, e.g. reducing some things
> to SPV levels of security.

This is a misconception; you can't prevent soft-forks from happening, so
you always have an SPV level of security by that standard.

People put *way* too much trust in small numbers of confirmations...

-- 
'peter'[:-1]@petertodd.org
0094d543907eaf0f94f4ff5f4c760b3552d84ff811cd9053


signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Peter Todd
On Thu, Nov 06, 2014 at 10:05:54PM +, Matt Corallo wrote:
> Depends, without BIP62 a /lot/ of the even basic contracts that people
> want to use today (or wanted to use 18 months ago) are unusable, in
> fact, without BIP62, the atomic swaps suggested as important for
> sidechains are not secure. While redoing Bitcoin in a hardfork is nice,
> its a very long-term thing, so I'm not sure about making people wait for
> a large hardfork just to use payment channels.

BIP62 is a less-than-ideal way of making contracts secure against
malleability as it relies on a "whack-a-mole" approach to security that
is insecure if any flaw is missed. If you only wanted to make contracts
secure, you'd either implement a new SignatureHash() that could leave
out the prevout field in favor of hashing the previous input's CTxOut()
structure, and/or implement the significantly more limited
CHECKLOCKTIMEVERIFY.

Equally BIP62 fails at making more complex types of contracts secure.
For instance suppose I had a multi-step protocol that required more than
two transactions:

tx1: Alice -> (Alice, Bob)
tx1_refund: (Alice, Bob) -> Alice

tx2: (Alice, Bob) -> Charlie
tx2_refund: (Alice, Bob) -> Bob

tx1 can only be modified by Alice, so tx1_refund is secure. However the
second stage, where the output of tx1 is spent by tx2, with a refund
transaction giving the funds back to Bob, can't be made secure as BIP62
can't prevent Alice from changing her signature, getting tx2' mined
instead, and making tx2_refund invalid.

OTOH a new form of signature hash that was a signature on tx2.vout
structure rather than it's txid would be secure, as tx2_refund would be
valid regardless of tx2's actual txid.

Obviously there are good reasons to not use such signature hashes in the
general case, as they imply you can't reuse scriptPubKeys securely, but
that's a minor problem for purpose-built contract protocols. It's
certainly a much more minor problem then the huge number of holes
possible with BIP62.

BIP62 does make life easier for wallet authors as they don't have to
deal with malleability - maybe! - but for contracts it's a bad design.

> Also, I echo the difficulty of writing consensus-compatible code and
> highly suggest anyone with money behind an implementation that is doing
> script verification in code that isnt Bitcoin Core rethink that decision.

FWIW I've done due-dilligence reviews for investors on projects and
companies that have re-implemented Bitcoin Core consensus-critical code,
and every time my review lists doing so as a major red flag.

-- 
'peter'[:-1]@petertodd.org
166801ed3959dde6b7d979735c290e7c4271ae3cf75ced63


signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Justus Ranvier
On 11/06/2014 04:11 PM, Jeff Garzik wrote:
> RE soft fork vs. hard fork:  It's about this time at Mike Hearn will
> chime in, on the side of hard forks.  Hard forks are in a sense much
> cleaner, and permit solving problems not otherwise solvable with a
> hard fork.  However, hard forks clearly have risks, notably the Big
> Risk akin to a US Constitutional Convention:  once you open the door,
> anything can happen, any rule no matter how "sacred" can be changed.

Yes, there are risks, but those risks could be managed with appropriate
effort. Major players could publicly commit to a set of ground rules vis
a vis what categories of changes are and are not acceptable.

Maybe at some point there could even be something that resembles project
management for the Bitcoin protocol.

Why not schedule protocol upgrades every two years for the foreseeable
future?

Spend one year achieving broad consensus regarding what changes to make
in the next upgrade, then spend one year in feature freeze (all future
proposals postponed for the next cycle) then execute the upgrade.

The top priority should be fixing bugs that make specifying and
re-implementing the protocol nearly impossible. Those kinds of changes
should have little difficulty achieving near-unanimous consensus.

There shouldn't be any problems separating obviously-needed changes from
the ones that let third parties blacklist coins, or a majority of miners
vote to confiscate block rewards from minority, tamper with the issuance
schedule, etc.

-- 
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k


0x38450DB5.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Jeff Garzik
IMO, CHECKLOCKTIMEVERIFY should be included in that list, too.

RE soft fork vs. hard fork:  It's about this time at Mike Hearn will
chime in, on the side of hard forks.  Hard forks are in a sense much
cleaner, and permit solving problems not otherwise solvable with a
hard fork.  However, hard forks clearly have risks, notably the Big
Risk akin to a US Constitutional Convention:  once you open the door,
anything can happen, any rule no matter how "sacred" can be changed.

Soft forks are not without their own risks, e.g. reducing some things
to SPV levels of security.

Leaning towards soft fork, but it is a good discussion to have.  A
poorly implemented soft fork may potentially require a hard fork to
fix rollout bugs.


On Thu, Nov 6, 2014 at 11:05 PM, Matt Corallo  wrote:
> Depends, without BIP62 a /lot/ of the even basic contracts that people
> want to use today (or wanted to use 18 months ago) are unusable, in
> fact, without BIP62, the atomic swaps suggested as important for
> sidechains are not secure. While redoing Bitcoin in a hardfork is nice,
> its a very long-term thing, so I'm not sure about making people wait for
> a large hardfork just to use payment channels.
>
> Also, I echo the difficulty of writing consensus-compatible code and
> highly suggest anyone with money behind an implementation that is doing
> script verification in code that isnt Bitcoin Core rethink that decision.
>
> Matt
>
> On 11/06/14 21:58, Tamas Blummer wrote:
>> Thanks Peter,
>>
>> Having tried to write a bug-for-bug compatible code with Satoshi, I can only 
>> second that it is rather close to impossible.
>>
>> The aim of BIP62 is noble, still it does not feel right for me to increase 
>> the complexity of the code with e.g. soft-fork-ready versioning.
>> Freezing the consensus code, studying its bugs appears more appropriate to 
>> me. What we learn could define a hard fork or a better
>> chain we migrate to as discussed by blockstream.
>>
>> Tamas Blummer
>
> --
> ___
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development



-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/

--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Matt Corallo
Depends, without BIP62 a /lot/ of the even basic contracts that people
want to use today (or wanted to use 18 months ago) are unusable, in
fact, without BIP62, the atomic swaps suggested as important for
sidechains are not secure. While redoing Bitcoin in a hardfork is nice,
its a very long-term thing, so I'm not sure about making people wait for
a large hardfork just to use payment channels.

Also, I echo the difficulty of writing consensus-compatible code and
highly suggest anyone with money behind an implementation that is doing
script verification in code that isnt Bitcoin Core rethink that decision.

Matt

On 11/06/14 21:58, Tamas Blummer wrote:
> Thanks Peter,
> 
> Having tried to write a bug-for-bug compatible code with Satoshi, I can only 
> second that it is rather close to impossible. 
> 
> The aim of BIP62 is noble, still it does not feel right for me to increase 
> the complexity of the code with e.g. soft-fork-ready versioning.
> Freezing the consensus code, studying its bugs appears more appropriate to 
> me. What we learn could define a hard fork or a better
> chain we migrate to as discussed by blockstream.
> 
> Tamas Blummer

--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Tamas Blummer
Thanks Peter,

Having tried to write a bug-for-bug compatible code with Satoshi, I can only 
second that it is rather close to impossible. 

The aim of BIP62 is noble, still it does not feel right for me to increase the 
complexity of the code with e.g. soft-fork-ready versioning.
Freezing the consensus code, studying its bugs appears more appropriate to me. 
What we learn could define a hard fork or a better
chain we migrate to as discussed by blockstream.

Tamas Blummer


signature.asc
Description: Message signed with OpenPGP using GPGMail
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug

2014-11-06 Thread Peter Todd
Recently wrote the following for a friend and thought others might learn
from it.


> Nope, never heard that term.  By "bug-for-bug" compatibility, do you mean
> that, for each version which has a bug, each bug must behave in the *same*
> buggy way?

Exactly. tl;dr: if you accept a block as valid due to a bug that others reject,
you're forked and the world ends.

Long answer... well you reminded me I've never actually written up a good
example for others, and a few people have asked me for one. A great example of
this is the SIGHASH_SINGLE bug in the SignatureHash() function:

uint256 SignatureHash(CScript scriptCode, const CTransaction& txTo, 
unsigned int nIn, int nHashType)
{



else if ((nHashType & 0x1f) == SIGHASH_SINGLE)
{
// Only lock-in the txout payee at same index as txin
unsigned int nOut = nIn;
if (nOut >= txTmp.vout.size())
{
printf("ERROR: SignatureHash() : nOut=%d out of range\n", nOut);
return 1;
}


}



// Serialize and hash
CHashWriter ss(SER_GETHASH, 0);
ss << txTmp << nHashType;
return ss.GetHash();
}

So that error condition results in SignatureHash() returning 1 rather than the
actual hash. But the consensus-critical code that implements the CHECKSIG
operators doesn't check for that condition! Thus as long as you use the
SIGHASH_SINGLE hashtype and the txin index is >= the number of txouts any valid
signature for the hash of the number 1 is considered valid!

When I found this bug¹ I used it to fork bitcoin-ruby, among others.
(I'm not the first; I found it independently after Matt Corallo) Those
alt-implementations handled this edge-case as an exception, which in
turn caused the script to fail. Thus they'd reject blocks containing
transactions using such scripts, and be forked off the network.

You can also use this bug for something even more subtle. So the
CHECKSIG* opcode evaluation does this:

// Drop the signature, since there's no way for a signature to sign itself
scriptCode.FindAndDelete(CScript(vchSig));

and CHECKMULTISIG* opcode:

// Drop the signatures, since there's no way for a signature to sign itself
for (int k = 0; k < nSigsCount; k++)
{
valtype& vchSig = stacktop(-isig-k);
scriptCode.FindAndDelete(CScript(vchSig));
}

We used to think that code could never be triggered by a scriptPubKey or
redeemScript, basically because there was no way to get a signature into a
transaction in the right place without the signature depending on the txid of
the transaction it was to be included in. (long story) But SIGHASH_SINGLE makes
that a non-issue, as you can now calculate the signature that signs '1' ahead
of time! In a CHECKMULTISIG that signature is valid, so is included in the list
of signatures being dropped, and thus the other signatures must take that
removal into account or they're invalid. Again, you've got a fork.

However this isn't the end of it! So the way FindAndDelete() works is as
follows:

int CScript::FindAndDelete(const CScript& b)
{
int nFound = 0;
if (b.empty())
return nFound;
iterator pc = begin();
opcodetype opcode;
do
{
while (end() - pc >= (long)b.size() && memcmp(&pc[0], &b[0], 
b.size()) == 0)
{
pc = erase(pc, pc + b.size());
++nFound;
}
}
while (GetOp(pc, opcode));
return nFound;
}

So that's pretty ugly, but basically what's happening is the loop iterates
though all the opcodes in the script. Every opcode is compared at the *byte*
level to the bytes in the argument. If they match those bytes are removed from
the script and iteration continues. The resulting script, with chunks sliced
out of it at the byte level, is what gets hashed as part of the signature
checking algorithm.

As FindAndDelete() is always called with CScript(vchSig) the signature
being found and deleted is reserialized. Serialization of bytes isn't
unique; there are multiple valid encodings for PUSHDATA operations. The
way CScript() is called the most compact encoding is used, however this
means that if the script being hashed used a different encoding those
bytes are *not* removed and thus the signature is different.

Again, if you don't get every last one of those details exactly right, you'll
get forked.

...and I'm still not done! So when you call CScript(vchSig) the relevant code
is the following:

class CScript : public std::vector
{



explicit CScript(const CScriptNum& b) { operator<<(b); }



CScript& operator<<(const std::vector& b)
{
if (b.size() < OP_PUSHDATA1)
{
insert(end(), (unsigned char)b.size());
}
else if (b.size() <= 0xff)
{
insert(end(), OP_PUSHDATA1);
insert(end(),