Re: [bitcoin-dev] Selfish Mining Prevention

[Eric Voskuil via bitcoin-dev]

> Once hashrate gets large enough, no new miners (additional
hashrate) will want to join since their share of the hashrate is too
small to make a profit.

The share (hash power) of a miner is proportional to capital investment, not 
the newness of that investment. The efficiency of a new mine (inclusive of 
pooling pressures) can certainly be sufficient to outperform other miners, 
resulting in the departure of the latter, thus not preventing the entry of the 
former.

The point you should be making here is that energy consumption is regulated by 
the cost of capital (in addition to reward value and the cost of energy).

Note that higher efficiency mining does not reduce energy consumption, nor does 
variation in the necessary cost of mining hardware. The total energy cost is 
the control, not the hash rate. This is of course why proof-of-memory (space) 
is pointless. It simply shifts most of the energy cost to hardware manufacture, 
shipment, etc.

e

On Sep 17, 2018, at 06:18, Andrew  wrote:

>> I see what you say, however, since the proposal as I have read it says "And 
>> this will keep happening as long as hashrate keeps rising," - what actually 
>> happens in the case hashrate stagnates or falls?
> 
> In general, a target hashrate can be set by users (can be less or more
> than the peak hashrate). As long as hashrate is rising and still
> didn't reach the target, miners will incrementally get the reserve
> fees. Once the "contract" times out, the remaining part can be used as
> fees by the users who created the reserve fee "contract". So if
> hashrate remains the same or falls, then users get the reserve fees
> back.
> 
> I agree that we can't stop people from being greedy. If they are not
> Bitcoin mining, they will try to put their energy to earn in some
> other way...The hashrate is related the demand for Bitcoin (price) and
> the amount of fees/subsidies the miners will get paid. For every level
> of mining rewards (based on demand) there exists a limit on the
> hashrate. Once hashrate gets large enough, no new miners (additional
> hashrate) will want to join since their share of the hashrate is too
> small to make a profit.
> 
> Also with merge mining and proof of space we can be quite efficient in
> the future. But of course I sympathize with the "don't be greedy"
> philosophy, and it can be good to educate people to use less resources
> than they need, just I think it's a bit outside of the scope of what
> the Bitcoin software protocol does.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Removal of reject network messages from Bitcoin Core (BIP61)

[Eric Voskuil via bitcoin-dev]

>> On Mar 7, 2019, at 19:09, Wilmer Paulino via bitcoin-dev 
>>  wrote:
>> ...
>> Nodes on the network can not generally be trusted to send valid ("reject") 
>> messages, so this should only ever be used when connected to a trusted node.
> 
> Nodes in the network generally rely on the assumption that they are connected 
> to at least one honest peer, so we can actually converge on the set of honest 
> peers and ban/disconnect any who send an invalid reject message for a valid 
> transaction.

This implies the reject message is valid only when it is expected (i.e. the 
sender is knowingly sending invalid transactions), which is presumably useful 
only in a local development environment.

e
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] New BIP - v2 peer-to-peer message transport protocol (former BIP151)

[Eric Voskuil via bitcoin-dev]

On 03/22/2019 02:04 PM, Jonas Schnelli via bitcoin-dev wrote:
> Proposal:
> 
> 
>   BIP: ???
>   Layer: Peer Services
>   Title: Version 2 Peer-to-Peer Message Transport Protocol
>   Author: Jonas Schnelli 
>   Status: Draft
>   Type: Standards Track
>   Created: 2019-03-08
>   License: PD
> 
> 
> == Abstract ==
> 
> This BIP describes a new Bitcoin peer to peer transport protocol with 
> opportunistic encryption.
> 
> == Motivation ==
> 
> The current peer-to-peer protocol is partially inefficient and in plaintext.
> 
> With the current unencrypted message transport, BGP hijack,
> block delay attacks 
> and message tempering are inexpensive and can be executed in a covert way 
> (undetectable MITM)[https://btc-hijack.ethz.ch/files/btc_hijack.pdf 
> Hijacking Bitcoin: Routing Attacks on Cryptocurrencies - M. Apostolaki, A. 
> Zohar, L.Vanbever].

This proposal does not provide mitigation for BGP hijacking, message
tampering or delaying, between anonymous peers.


> Adding opportunistic encryption introduces a high risk for attackers of
> being detected. Peer operators can compare encryption session IDs


This is only possible if the peers have access to a secure/trusted side
channel between them. In other words, this does not benefit anonymous
peers. It also seems like quite a stretch to consider it creating "high
risk" for the attacker, since the chances of any given pair of peers
actually comparing session IDs over a secure channel seems extremely remote.


> or use other form of authentication schemes  name="bip150">[https://github.com/bitcoin/bips/blob/master/bip-0150.mediawiki 
> BIP150] to identify an attack.


Authentication helps mitigate attacks by requiring the identity of the
peer (based only on the presumption that a trusted peer wouldn't
attack). This provides no benefit to anonymous peers.

Data communicated between peers is entirely public. Unlike other systems
that maintain data integrity through encryption, Bitcoin relies on
validation. Encrypting public data between anonymous peers is pointless,
and thus counterproductive from an engineering and software security
standpoint.

More importantly Bitcoin system security *requires* widespread anonymous
participation. It's generally not a good idea to implement features that
backfire if they actually get widespread use. While we cannot prevent
people from using VPNs, incorporating them into the protocol is
counterproductive from a system security standpoint.


> Each current version 1 Bitcoin peer-to-peer message uses a double-SHA256 
> checksum truncated to 4 bytes. Roughly the same amount of computation power 
> would be required for encrypting and authenticating a peer-to-peer
> message with ChaCha20 & Poly1305.


The proposal overlooks the simple alternatives of (1) not validating the
checksum, which is never necessary, and (2) proposing a protocol change
to drop the checksum altogether. The former requires no protocol change
and the latter can allow the checksum to be dropped in all messages
except "version" given a simple protocol version number increment (i.e.
no need to consume a service bit), saving not only the CPU resource but
also network bandwidth.


> Additionally, this BIP describes a way how data manipulation (blocking or 
> tempering commands by an intercepting TCP/IP node) would be identifiable
> by the communicating peers.


The only such method described is manual comparison of session ID's
between trusted parties over a secure side channel.


> Encrypting traffic between peers is already possible with VPN, tor,
> stunnel, 
> curveCP or any other encryption mechanism on a deeper OSI level,
> however, most 
> of those solutions require significant knowhow in how to setup such a
> secure 
> channel and are therefore not widely deployed.


Yet this is exactly what a secure side channel is. Furthermore, being
manual, not only would it also suffer from not being widely deployed,
but also widely ignored.


> == Specification ==
> 
> 
> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
> "SHOULD",
> "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this document are
> to be
> interpreted as described in RFC
> 2119[https://tools.ietf.org/html/rfc2119 
> RFC 2119].
> 
> 
> A peer that supports the message transport protocol as defined in this
> proposal 
> MUST accept encryption requests from all peers.
> 
> Both communication direction share the same shared-secret but have
> different 
> symmetric cipher keys.
> 
> The encryption handshake MUST happen before sending any other messages
> to the 
> responding peer.
> 
> If the responding peer closes the connection after sending the handshake 
> request, the initiating peer MAY try to connect again with the v1
> peer-to-peer 
> transport protocol. Such reconnects allow an attacker to "downgrade" the 
> encryption to plaintext communication and thus, accepting v1 connections
> MUST 
> not be done when the Bitcoin peer-to-peer network uses almost only v2 
> communi

Re: [bitcoin-dev] Making Electrum more anonymous

[Eric Voskuil via bitcoin-dev]

Hi Thomas,

The scheme is essentially onion routing. The set of {M} are entry nodes
and the set of {S} are exit nodes. The weaknesses are as you would see
in an analogous TOR implementation:

(1) The lack of relay nodes {R} make collaboration between any subset of
{M} and {S} trivial.

(2) OR is a mixnet, so the size of the network matters a lot.

(3) The directory is a perpetual weakness.

(4) Content is visible to the exit node (or the final service). This
means each address must be passed via a distinct route to prevent
correlation.

e

On 07/22/2015 08:51 AM, Thomas Voegtlin via bitcoin-dev wrote:
> Hello,
> 
> Although Electrum clients connect to several servers in order to fetch
> block headers, they typically request address balances and address
> histories from a single server. This means that the chosen server knows
> that a given set of addresses belong to the same wallet. That is true
> even if Electrum is used over TOR.
> 
> There have been various proposals to improve on that, but none of them
> really convinced me so far. One recurrent proposal has been to create
> subsets of wallet addresses, and to send them to separate servers. In my
> opinion, this does not really improve anonymity, because it requires
> trusting more servers.
> 
> Here is an idea, inspired by TOR, on which I would like to have some
> feedback: We create an anonymous routing layer between Electrum servers
> and clients.
> 
> * Each server S publishes a RSA public key, KS
> * Each client receives a list of available servers and their pubkeys
> * For each wallet address, addr_i, a client chooses a server S_i, and a
> RSA keypair (K_addr_i, k_addr_i)
> * The client creates a list of encrypted requests. Each request contains
> addr_i and K_addr_i, and is encrypted with the pubkey KS_i of S_i
> * The client chooses a main server M, and sends the list of encrypted
> requests to M
> * M dispatches the client's requests to the corresponding servers S_i
> (without the client's IP address.)
> * Each server decrypts the requests it receives, performs the request,
> and encrypts the result with K_addr_i
> * M receives encrypted responses, and forwards them to the client.
> * The client decrypts the encrypted response with k_addr_i
> 
> What do you think? What are the costs and benefits of such an approach?
> 
> (Note: this will not work if all servers, or a large fraction of them,
> are controlled by the same entity that controls M)
> 
> 
> Thomas
> ___



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Making Electrum more anonymous

[Eric Voskuil via bitcoin-dev]

I should add that the obvious resolution to this set of problems is to
use a distinct Tor route for each Bitcoin address, not to reinvent Tor
and reproduce its community. So ultimately this is easy to implement,
but the downside is performance.

But it's important to keep in mind that poor-performing perfect privacy
for address monitoring is trivial to achieve - just sync the full
blockchain.

Presumably if you don't trust a server to protect your privacy, you also
don't trust it with your money. So any robust privacy optimization would
at least be designed to support partial (SPV) chain clients. It would
also need to support wallet restoration from backup.

The level of privacy will always be a performance trade-off. The ideal
solution would allow a client to balance privacy against performance.

e

On 07/22/2015 09:30 AM, Eric Voskuil wrote:
> Hi Thomas,
> 
> The scheme is essentially onion routing. The set of {M} are entry nodes
> and the set of {S} are exit nodes. The weaknesses are as you would see
> in an analogous TOR implementation:
> 
> (1) The lack of relay nodes {R} make collaboration between any subset of
> {M} and {S} trivial.
> 
> (2) OR is a mixnet, so the size of the network matters a lot.
> 
> (3) The directory is a perpetual weakness.
> 
> (4) Content is visible to the exit node (or the final service). This
> means each address must be passed via a distinct route to prevent
> correlation.
> 
> e
> 
> On 07/22/2015 08:51 AM, Thomas Voegtlin via bitcoin-dev wrote:
>> Hello,
>>
>> Although Electrum clients connect to several servers in order to fetch
>> block headers, they typically request address balances and address
>> histories from a single server. This means that the chosen server knows
>> that a given set of addresses belong to the same wallet. That is true
>> even if Electrum is used over TOR.
>>
>> There have been various proposals to improve on that, but none of them
>> really convinced me so far. One recurrent proposal has been to create
>> subsets of wallet addresses, and to send them to separate servers. In my
>> opinion, this does not really improve anonymity, because it requires
>> trusting more servers.
>>
>> Here is an idea, inspired by TOR, on which I would like to have some
>> feedback: We create an anonymous routing layer between Electrum servers
>> and clients.
>>
>> * Each server S publishes a RSA public key, KS
>> * Each client receives a list of available servers and their pubkeys
>> * For each wallet address, addr_i, a client chooses a server S_i, and a
>> RSA keypair (K_addr_i, k_addr_i)
>> * The client creates a list of encrypted requests. Each request contains
>> addr_i and K_addr_i, and is encrypted with the pubkey KS_i of S_i
>> * The client chooses a main server M, and sends the list of encrypted
>> requests to M
>> * M dispatches the client's requests to the corresponding servers S_i
>> (without the client's IP address.)
>> * Each server decrypts the requests it receives, performs the request,
>> and encrypts the result with K_addr_i
>> * M receives encrypted responses, and forwards them to the client.
>> * The client decrypts the encrypted response with k_addr_i
>>
>> What do you think? What are the costs and benefits of such an approach?
>>
>> (Note: this will not work if all servers, or a large fraction of them,
>> are controlled by the same entity that controls M)
>>
>>
>> Thomas
>> ___
> 



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Making Electrum more anonymous

[Eric Voskuil via bitcoin-dev]

This is a good point. I didn't delve into the specifics of
implementation due to the larger issues that I raised. Libbitcoin Server
uses CurveZMQ, an implementation of CurveCP.

http://curvecp.org
http://curvezmq.org
https://en.bitcoin.it/wiki/Libbitcoin_Server

e

On 07/22/2015 04:11 PM, gb via bitcoin-dev wrote:
> Why RSA?
> 
>>
>> Here is an idea, inspired by TOR, on which I would like to have some
>> feedback: We create an anonymous routing layer between Electrum servers
>> and clients.
>>
>> * Each server S publishes a RSA public key, KS
>> * Each client receives a list of available servers and their pubkeys
>> * For each wallet address, addr_i, a client chooses a server S_i, and a
>> RSA keypair (K_addr_i, k_addr_i)
>> * The client creates a list of encrypted requests. Each request contains
>> addr_i and K_addr_i, and is encrypted with the pubkey KS_i of S_i
>> * The client chooses a main server M, and sends the list of encrypted
>> requests to M
>> * M dispatches the client's requests to the corresponding servers S_i
>> (without the client's IP address.)
>> * Each server decrypts the requests it receives, performs the request,
>> and encrypts the result with K_addr_i
>> * M receives encrypted responses, and forwards them to the client.
>> * The client decrypts the encrypted response with k_addr_i



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Bitcoin Core and hard forks

[Eric Voskuil via bitcoin-dev]

On 07/22/2015 05:13 PM, Eric Lombrozo via bitcoin-dev wrote:
> Only being partly serious - I strongly am in favor of a sufficiently
modularized codebase that swapping out consensus rules is fairly
straightforward and easy to test...

We (libbitcoin) have taken the time to publish and maintain bitcoind's
"libbitcoinconsensus" source files as an independent C++ library (with
Java and Python bindings).

https://en.bitcoin.it/wiki/Libbitcoin_Consensus

It can be easily verified against bitcoind sources and in builds of
libbitcoin-blockchain it can be swapped out for libbitcoin's native
consensus checks.

https://en.bitcoin.it/wiki/Libbitcoin_Blockchain#Consensus_Validation

So there is really no reason to consider the original client synonymous
with consensus. I initially argued for this library to be natively
isolated from bitcoind, but that didn't seem to be in the cards so we
did it independently.

In any case I agree with your stated need for this isolation (if not the
means) for the reasons you state. The community needs to move beyond a
largely singular and monolithic codebase that is holding that position
in part due to fear about consensus bug forks.

To make choice regarding consensus an actual choice (and thereby actual
consensus) the modularity you suggest is essential. One must be able to
take new developments without having to take consensus changes. The
option to fork the codebase is not reasonable for most people. At this
point there is no defensible reason for coupling consensus checks with
other features.

e




signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Electrum Server Speed Test

[Eric Voskuil via bitcoin-dev]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Does "to process into the index" include time for transport and/or
block validation (presumably by bitcoind) or this this exclusively the
time for Electrum Server to index a validated block?

e

On 07/23/2015 08:56 AM, Slurms MacKenzie via bitcoin-dev wrote:

> Similar to the Bitcoin Node Speed Test, this is a quick
> quantitative
look at how the Electrum server software handles under load. The
Electrum wallet is extremely popular, and the distributed servers
which power it are all hosted by volunteers without budget. The server
requires a fully indexed Bitcoin Core daemon running, and produces
sizable external index in order to allow SPV clients to quickly
retrieve their history.
> 
> 
> 3.9Gelectrum/utxo 67M electrum/undo 19G electrum/hist 
> 1.4Gelectrum/addr 24G electrum/
> 
> 
> Based on my own logs produced by the electrum-server console, it
takes this server (Xeon, lots of memory, 7200 RPM RAID) approximately
3.7 minutes per megabyte of block to process into the index. This
seems to hold true through the 10 or so blocks I have in my scroll
buffer, the contents of blocks seem to be of approximately the same
processing load. Continuing this trend with the current inter-block
time of 9.8 minutes, an electrum-server instance running on
modest-high end dedicated server is able to support up to 2.64 MB
block sizes before permanently falling behind the chain.


-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJVsTCdAAoJEDzYwH8LXOFOLpgIAIAe5QBTTgpe2kyYrqamUKNI
uasljmNQi3In/rdxNm9Ga+vwTsBg7f4v/xHVrQYAGxdjjZxs0h9bOmAvuc+p2fad
ZeHC/vewrtLxB2aVj5JVbSk5ik57ePyk3SmghTpGMAhgTIWkceIrxR+Fq7TFOlqn
NGnTuBSSsGL9nY57hIFwMJb2CKdPwMVLL/0gjVQ9Llqt+Fu31eSRhDzHOvJnkofO
xwVrVGgST2GE73np3WA0jvzRHjFgsPnQknhjLGQiTgDKsKL0BywE9/Vke2zVNyP7
cAlTQoScEal++9u0h3D475lsxv43V0Rxafc7W0a/OyfXujJ2AlixgV8PlLwhvaY=
=DK5K
-END PGP SIGNATURE-
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Making Electrum more anonymous

[Eric Voskuil via bitcoin-dev]

I've looked at IT-PIR for Libbitcoin. It's certainly more elegant than
onion routing for query privacy, but doesn't improve on the collusion
problem. As a result the related directory problem would also remain.

"This protocol sacrifices some level of privacy to gain robustness.
Because of this  we need to assume that there is no collusion between
some number of servers. In some  settings, it is unclear how this
requirement can be enforced or detected. This uncertainty  may make this
protocol less desirable than others with different privacy guarantees."

http://www.cypherpunks.ca/~iang/pubs/hybridpir-pets.pdf

From the same source, regarding aforementioned options:

"An assumption used in many PETs, including mix networks, secret
sharing, onion routing and some voting protocols, is that no more than
some threshold of agents are colluding against the user to discover the
private information."

Another option is computationally-bounded CPIR, but from the same source:

"The main problem with the CPIR protocols already  discussed is that
they do not generally perform queries faster than the trivial protocol."

Where the "trivial protocol" in our problem is full blockchain download.

WRT an alternative CPIR proposed in 2007:

"The idea behind their protocol is to add noise to the query in a way
that the server  cannot discover which record the client is interested
in, but with the secret information  that the client has, she can remove
the noise from the server’s response."

This is the idea behind stealth prefix queries:

https://github.com/libbitcoin/libbitcoin-explorer/wiki/bx-fetch-stealth

From our perspective, another important objective of query privacy is
allowing the caller make the trade-off between the relative levels of
privacy and performance - from absolute to non-existent. In some cases
privacy is neither required nor desired.

Prefix filtering accomplishes the client-tuning objective. It also does
not suffer server collusion attacks nor is it dependent on computational
bounds. The primary trade-off becomes result set (download) size against
privacy.

e

On 07/23/2015 05:23 AM, Stefan Richter wrote:
> This looks like a prime application for this PIR
> library: http://percy.sourceforge.net/apidocs/index.html
> 
> Eric Voskuil via bitcoin-dev  <mailto:bitcoin-dev@lists.linuxfoundation.org>> schrieb am Do., 23. Juli
> 2015 um 02:07 Uhr:
> 
> This is a good point. I didn't delve into the specifics of
> implementation due to the larger issues that I raised. Libbitcoin Server
> uses CurveZMQ, an implementation of CurveCP.
> 
> http://curvecp.org
> http://curvezmq.org
> https://en.bitcoin.it/wiki/Libbitcoin_Server
> 
> e
> 
> On 07/22/2015 04:11 PM, gb via bitcoin-dev wrote:
> > Why RSA?
> >
> >>
> >> Here is an idea, inspired by TOR, on which I would like to have some
> >> feedback: We create an anonymous routing layer between Electrum
> servers
> >> and clients.
> >>
> >> * Each server S publishes a RSA public key, KS
> >> * Each client receives a list of available servers and their pubkeys
> >> * For each wallet address, addr_i, a client chooses a server S_i,
> and a
> >> RSA keypair (K_addr_i, k_addr_i)
> >> * The client creates a list of encrypted requests. Each request
> contains
> >> addr_i and K_addr_i, and is encrypted with the pubkey KS_i of S_i
> >> * The client chooses a main server M, and sends the list of encrypted
> >> requests to M
> >> * M dispatches the client's requests to the corresponding servers S_i
> >> (without the client's IP address.)
> >> * Each server decrypts the requests it receives, performs the
> request,
> >> and encrypts the result with K_addr_i
> >> * M receives encrypted responses, and forwards them to the client.
> >> * The client decrypts the encrypted response with k_addr_i
> 
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> <mailto:bitcoin-dev@lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Making Electrum more anonymous

[Eric Voskuil via bitcoin-dev]

On 07/23/2015 08:42 PM, Slurms MacKenzie via bitcoin-dev wrote:
>> From: "Eric Voskuil via bitcoin-dev"
>>
>> From our perspective, another important objective of query privacy is
>> allowing the caller make the trade-off between the relative levels of
>> privacy and performance - from absolute to non-existent. In some
>> cases privacy is neither required nor desired.
>>
>> Prefix filtering accomplishes the client-tuning objective. It also
>> does not suffer server collusion attacks nor is it dependent on
>> computational bounds. The primary trade-off becomes result set
>> (download) size against privacy.
>
> Keep in mind this is the similar premise as claimed to be offered by
> BIP37 bloom filters, but faulty assumptions and implementation
> failure in BitcoinJ have meant that bloom filters uniquely identify
> the wallet and offer no privacy for the user no matter what the
> settings are.

Yes, quite true. And without the ability to search using filters there
is no private restore from backup short of downloading the full chain,
rendering the idea rather pointless.

This is why privacy remains a significant issue. Privacy is an essential
aspect of fungibility. This is a central problem for Bitcoin. The
correlation of addresses within transactions is of course problematic.
Possibly zero knowledge proof will at some point come to the rescue. But
the correlation of addresses via search works against the benefits of
address non-reuse, and the correlation of addresses to IP addresses
works against the use of private addresses.

Solving the latter two problems can go a long way to reducing the impact
of the former. But currently the only solution is to run a full chain
wallet. This is not a viable solution for many scenarios, and getting
less so.

This is not a problem that can be ignored, nor is it unique to Electrum.
The Bloom filter approach was problematic, but that doesn't preclude the
existence of valid solutions.

> If you imagine a system where there is somehow complete
> separation and anonymization between all requests and subscriptions,
> the timing still leaks the association between the addresses to the
> listeners.

Well because of presumed relationship in time these are not actually
separated requests. Which is why even the (performance-unrealistic)
option of a distinct Tor route for each independent address request is
*still* problematic.

> The obvious solution to that is to use a very high latency
> mix network, but I somehow doubt that there's any desire for a wallet
> with SPV security that takes a week to return results.

Introducing truly-random timing variations into the mixnet solutions can
mitigate timing attacks, but yes, this just makes the already
intolerable performance problem much worse.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Bitcoin Core and hard forks

[Eric Voskuil via bitcoin-dev]

https://twitter.com/petertoddbtc

Thanks for the triggering warning, if not for that I may have gone into
seizures.

e


On 07/27/2015 10:05 AM, Alice Larson via bitcoin-dev wrote:
> The twitter teenage nonsense from Todd is ridiculous:
> https://twitter.com/playatodd (warning, triggering)



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Libconsensus separated repository (was Bitcoin Core and hard forks)

[Eric Voskuil via bitcoin-dev]

On 07/23/2015 07:30 AM, Jorge Timón wrote:
> On Thu, Jul 23, 2015 at 2:49 AM, Eric Voskuil via bitcoin-dev wrote:
>> On 07/22/2015 05:13 PM, Eric Lombrozo via bitcoin-dev wrote:
>>> Only being partly serious - I strongly am in favor of a sufficiently
>>> modularized codebase that swapping out consensus rules is fairly
>>> straightforward and easy to test...
>>
>> We (libbitcoin) have taken the time to publish and maintain bitcoind's
>> "libbitcoinconsensus" source files as an independent C++ library...

> I think there were some misunderstandings in our previous conversation
> about this topic.
> I completely agree with having a separated repository for libconsensus
> (that's the whole point, alternative implementations can be
> consensus-safe by using it, and in the event of a schism fork[1], they
> can fork just that smaller project without having to relay on Bitcoin
> Core [satoshi] at all).
> But I thought you also wanted Bitcoin Core to use libconsensus instead
> of just having a subtree/subrepository like it currently does with
> libsecp256k1.

libsecp256k1 has it's own repository, libbitcoinconsensus doesn't.  A
separate repository was what I considered as a requirement for us to use it.

> I'm not sure if that would ever be accepted, but in any case we're
> certainly far away from that goal.

If it's not certain whether this would even be accepted, the commitment
to a community consensus library is too weak to take a strong dependency
on. But for us it's moot, as we have made the already accomplished that
goal.

> Here are some things that need to
> happen first:
> 
> 1) Finish encapsulating consensus code so that it can be built without
> any (we've done it only with script-related code so far). Here are
> some related PRs (other people have done other things that help with
> this as well):
...
> 2) Finish libconsensus's API: expose more things than VerifyScript, at
> the very least, also expose VerifyTx, VerifyHeader and VerifyBlock.
> Feedback from alternative implementations like libbitcoin is extremely
> valuable here. Some related closed-for-now PRs:

In our earlier discussion I believe you said that the library would not
be undergoing significant change or feature creep. If this is the very
least that's projected it would seem that constraint will not hold.

> 3) Move libconsensus to a separate repository as a
> subtree/subrepository of Bitcoin Core.
> 
> Only after all that we can discuss whether Bitcoin Core itself should
> include libconsensus' code or just use its API directly.

I don't think it's a question of whether it *should* use its own library
as it is published for others - this is a practically self-evident
conclusion.

> I hope that after all this, libbitcoin also reconsiders whether to
> reimplement its own libconsensus or use the "official" one directly
> instead.

We use a fork of libsecp256k1 and would probably do the same with the
consensus library if it was cleanly isolated.

>> In any case I agree with your stated need for this isolation (if not the
>> means) for the reasons you state. The community needs to move beyond a
>> largely singular and monolithic codebase that is holding that position
>> in part due to fear about consensus bug forks.
> 
> I completely agree. That's the goal of libconsensus (and an
> alternative implementation like libbitcoin being able to use it
> without sacrificing any of its current or future design differences
> from Bitcoin Core would be a sign of success in this reward).

It's a performance sacrifice, and then there's the OpenSSL dependency,
but these are both optional within our stack - so the application
developer has the option. So the only downside is that we are
maintaining the conditional compilation.

> Unfortunately any changes that touch consensus code are risky and
> therefore slow. And when consensus encapsulation changes conflict with
> other changes (not because the other changes need to change consensus
> but because consensus code is still coupled with policy and other
> bitcoind-specific code), refactors are never prioritized. Ironically,
> you need to encapsulate the consensus code to avoid such conflicts,
> which would make all non-consensus changes far less risky (reducing
> the consensus-critical review development bottleneck).
> 
> Unfortunately and ironically again, safer, small and incremental
> changes are less interesting for reviewers.
> For example, I've been trying to move consensus code to the consensus
> folder for a long time. The correctness of a MOVEONLY change is
> trivial to review for anyone who knows how to copy/paste in its
> favorite editor and how to use git di

Re: [bitcoin-dev] Libconsensus separated repository (was Bitcoin Core and hard forks)

[Eric Voskuil via bitcoin-dev]

On 07/28/2015 02:58 AM, Jorge Timón wrote:
>> I haven't looked at any of these commits, but I'll make some time to at
>> least give a cursory review.
> 
> Great. I mean, I wasn't asking about reviewing the commits themselves
> (which is also great if you do), but rather on answering the questions
> I'm making there, ie: what to expose next (ie VerifyTx or
> VerifyHeader)?

Oh, I misunderstood your ask then. I don't have a preference on
prioritizing VerifyTx vs VerifyHeader.

> would this be an acceptable way to expose
> VerifyHeader?

I'm not sure how you mean to expose it, could you clarify?

> Which of he step-checks functions is worth exposing too (Bitcoin
> Core is currently using some to prevent DoS attacks, for example)?

I don't see any reason to expose checkpoints in this library. They are
trivial to implement and are not part of consensus.

>>> Would you agree that asking people to fork an independent libconsensus
>>> project instead of having to fork the full Bitcoin-qt is much more
>>> reasonable?
>>
>> Yes, of course. We've already done it. For each release of the satoshi
>> client since we made libbitcoin-consensus I've copied the sources. It's
>> pretty much automated and easy to visually verify that the sources
>> match. That would be quite a bit more difficult if there wasn't an
>> independent build.
> 
> Well, neither libbitcoincosnensus nor libbitcoin-consensus implements
> all the consensus rules.
> That's what makes them different from future-libconsensus.
> But great, we're confirming more views that we share.

Nothing can eliminate all consensus risk, not even a common full node
implementation.

>> Useful specifications often have two reference implementations. It's the
>> idea that there can be only one legitimate implementation that's
>> problematic.
> 
> Well, this is where I fear we will never agree. I think "Bitcoin is
> different" in this reward and you disagree.
> Maybe Pieter's explanation is more convincing to you:
> https://youtu.be/PxW5D9xCIsc?t=769
> Otherwise, I think I'll stop trying convincing you.

Maybe I wasn't sufficiently explicit. It is problematic. That is the
core issue we are dealing with. That doesn't mean I disagree with the
objectives of an independent community consensus library.

The premise of the "one true library" idea is that there is *no way* to
sufficiently test for consensus bugs in any software release. That of
course means that each release of the satoshi client poses a significant
risk to the network. This risk is presently greater than that posed by
other implementations simply because of adoption. That is the basis of
the red herring argument:

https://blog.conformal.com/the-bitcoin-consensus-red-herring

The bottom line is that nobody has control over this process. There are,
and will always be, a multitude of consensus implementations that intend
to target the same coin. Presently there are multiple versions of the
satoshi client, and this has produced forks, and will continue to do so.

Isolating the satoshi consensus checks to an independent library serves
not to eliminate that risk, but can reduce it somewhat. Importantly it
will allow various implementations to overcome a perception problem,
which will improve implementation diversity and developer participation.

>>> I believe that's the only point where we fundamentally disagree, but
>>> it shouldn't be a barrier in our common goal of taking "power" away
>>> from Bitcoin Core development. If we're successful Bitcoin Core won't
>>> have any privileged position with regards to, say, libbitcoin when it
>>> comes to deciding consensus rules changes.
>>
>> I don't think we disagree on anything fundamental. My issues with the
>> library were (1) the lack of isolation, (2) the fact that the satoshi
>> client wouldn't actually use the library, and (3) backtracking to use
>> OpenSSL, which we had recently removed from libbitcoin.
> 
> 1) Working on it

For the sake of clarity, this is now a non-issue for us.

> 2) The Satoshi client has been using all along and it will use it
> forever (maybe not through the API, but I don't get what the problem
> with that is).

Again, I consider this a requirement for us to link directly to it as a
library. If the sources are isolated into an independent repo, but the
satoshi client is embedding its own copies, one must continue to diff
the client sources against the library sources. We are doing this
already, so the benefit to having the independent repo is in no longer
having to do this.

There are also differences in the build system that can affect outcome.
Comparing those differences across repos can be more challenging. For
this reason I consider it important to your objective that the satoshi
client actually use the library - as I assume it will at some point.

If the satoshi client folks are to maintain a consensus library for the
community it's also important to show a commitment to its independence.
Dogfooding is of course a software engineering best

Re: [bitcoin-dev] Incentivising full nodes by having SPV nodes to pay for data requests

[Eric Voskuil via bitcoin-dev]

The typical interpretation of the "tragedy of the commons" scenario is
based on the presumption that a limited resource is controlled by the state.

In a free market there is no such idea, a limited resource becomes
property and is therefore allocated by customer preference and
maintained by self-interest.

Validation is not a resource controlled by the state, which is of course
the point of Bitcoin.

It is inaccurate to think of Bitcoin validation a limited resource. The
scenario in which fewer people validate does not reduce the amount of
validation available. It increases the risk of loss to those who fail to
validate. The gain to those who do validate comes from avoiding that loss.

e


On 08/03/2015 10:06 AM, Luv Khemani via bitcoin-dev wrote:
> The current block size debate has brought up an important, albeit often
> neglected observation. Full nodes suffer from a tragedy of the commons
> problem and therefore are likely to continue decreasing as a percentage
> of total Bitcoin nodes. This also results in a vicious circle as more
> and more people use SPVs, the burden on existing full nodes will
> increase even without a block size increase, which will further reduce
> the number of full nodes . A few people have mentioned it in blogs or
> reddit, but the topic is generally quickly overshadowed by posts along
> the lines of  "RAISE the blocksize already!".
> 
> Full nodes bear the full cost of validating/relaying/storing the
> blockchain and servicing SPV clients but gain nothing financially from
> it, yet they serve an important role in validating transactions and
> keeping miner dishonesty in check. If there were few independent full
> nodes, it would be possible for 3-4 of the biggest mining pools to
> collude and do literally whatever they wanted with the protocol,
> including inflating the money supply, freezing funds or even
> confiscating funds, because who would know? And even if someone running
> a full node did voice out, the majority of users on SPV/Coinbase/etc..
> would be powerless to do anything about it and would likely bear with
> the changes to protect status quo, just as is the case with current fiat
> regimes where people bear with QE/Inflation/bail outs because they are
> so dependent on the current system that they would rather tolerate any
> injustice than to have the system go down and bring them with it. 
> This is the primary reason why many in the technical community are
> against drastic blocksize increases, as this will only worsen the
> problem of decentralization as this cost increases. And as long as full
> nodes are running on charity, i'm fully in agreement with the
> conservative block size camp. 
> 
> However, it is important to note that this seems to be an economic
> problem instead of a technical one. I cannot deny the argument from the
> big block side that technically, the hardware/bandwidth required to run
> full nodes supporting considerably larger blocks (4MB-8MB) is not out of
> reach of many individuals around the globe. The core issue in my opinion
> is that of incentive, because at the end of the day, running a full node
> is not free and at larger blocks costs will not be trivial. In my
> opinion, its perhaps our insistence that full nodes cant be incentivised
> that contributes to centralization pressures and discourages increasing
> of blocksize even though the technology exists to support it.
> 
> Technically, existing hardware is capable of validating/processing
> blocks in the region of an order of magnitude larger than the current
> limit. Bandwidth requirements for running a validating full node are
> also not very high if you are not mining, as you can afford to wait a
> couple of minutes to download your block. This is obviously not the case
> for miners who need to download new blocks asap to avoid idle hash power
> or as has been seen in the recent fork, SPV mining (which is extremely
> undesirable for the network). IBLT should help greatly in reducing the
> propagation time of new blocks and ease peak bandwidth requirements. But
> im not worried about the miners, they are after all being financially
> compensated for what they are doing and investing in more
> bandwidth(either locally or running a full node remotely) can be seen as
> a cost of the business as long as the cost of running a full node is
> insignificant to the cost of hashing equipment to keep barriers to
> mining low. 
> 
> Before the concept lightning, there did not seem to be any trustless way
> of feasibly paying small micropayments to full nodes for their services.
> However, with payment channels and lightning, this may no longer be an
> issue. A node could advertise it's rates to a SPV nodes upon connection
> and the SPV could either agree or look for another node with lower fees.
> If implemented, fees are likely to be trivial(few satoshis per request)
> as competition will drive down fees close to the cost of running a full
> node. This should spur an increase in the 

Re: [bitcoin-dev] Off-chain transactions and miner fees

[Eric Voskuil via bitcoin-dev]

Hi Anthony,

No belief can be shown to be universally held, and an appeal to
authority is also a logical fallacy for good reason.

The blog you quote is littered with flawed economic ideas. It's become a
pet peeve of mine that people refer to mining (and/or validation) as a
"tragedy of the commons" problem, or a "public good" subject to a "free
rider" problem. This betrays a fundamental misunderstanding of both
money and Bitcoin.

I'm not commenting on the other merits of your argument or others in
this thread, I mean just to dispute the validity of this particular
reference. Even the portion you quoted is quite absurd:

>> "We’re not spending so much on mining because we really need it.
>> It’s because printing money distorts behaviour."

We don't "really need" to prevent "printing money" - Bitcoin could
somehow get by without that constraint? Preventing the printing of money
is the only reason that Bitcoin exists.

The tragedy of the commons scenario properly applies only to property
controlled by the state. In the quoted blog the analogy is so misapplied
that it fundamentally misrepresents the forces at work in Bitcoin.

Bitcoin is not at all "like a lighthouse". State run lighthouses are
financed via taxation. That may be taxation of anything, whether or not
related to the shipping the lighthouse purports to protect. It may in
fact protect no shipping at all, since payment is generally completely
divorced from benefit, and the benefits may be completely divorced from
shipping. For example, preservation of jobs for lighthouse keepers and
the Coast Guard, or even nostalgia. Just as with a private grazing
field, a truly private lighthouse would not have a "commons problem" at all.

Bitcoin mining is financed by a fixed schedule of inflation and
transaction fees. State inflation is a tax on all holders of currency
and a form of default on state debt. This and other taxes fund
lighthouses. A tax is the seizure of someone else's property through
force. Bitcoin inflation is predictable, so the inflation cost is
factored in to its value before it is acquired, according to the
depreciation schedule, just like bond valuation for example. This means
it is NOT a tax, is merely a cost that is paid to miners for use of
their security services.

Bitcoin transaction "fees" are not fees in the state use-fee (taxation)
sense, since the fees are priced based on voluntary trade. The blog
misinterprets who is paying the cost of securing a transaction when it
claims, "it's the sender who pays." Both parties to a transaction bear
the cost of using any given medium of exchange. If the receiver is
concerned about double spending risk, it's the sender who will have to
compensate with time and/or money. But this is just as much a cost to
the receiver as it has raised the effective price of his sales with the
difference in money accruing to the third party.

Finally, transaction fees *are* mining contracts. Creating *another*
system of mining contracts initiated by a receiver would do nothing to
change the economics, but it would significantly complicate the
implementation (raising costs generally). The cost of paying a mining
contract would of course be paid by the sender, in terms of increased
price charged by the receiver.

I believe that a fundamental misunderstanding of the important
distinction between voluntary trade and state-controlled trade is
underpinning a lot of confusion and misunderstanding with respect to the
block size debate. Bitcoin does not have a commons problem specifically
because it's designed to resist state control. It's only in the loss of
that independence that such a problem would arise (and effectively kill
Bitcoin altogether).

Ironically the desire to fix a non-existent commons problem in Bitcoin
seems to be a driving force behind what may in fact weaken its only
defence against eventually becoming a commons.

e


On 08/10/2015 11:50 AM, Anthony Towns via bitcoin-dev wrote:
> On Mon, Aug 10, 2015 at 12:20:36AM +0200, info--- via bitcoin-dev wrote:
>> one argument I often read on this mailing list is that it's essential to
>> reward miners with transaction fees at some point to secure the network.
> 
> That's not a universally held belief. See for example:
> 
>  https://en.bitcoin.it/wiki/Funding_network_security#Alternatives
>  https://bitcointalk.org/index.php?topic=157141.0
> 
> It's also not clear to me what amount of security people actually "want".
> In late May, Mike Hearn wrote:
> 
>>  "Currently the Bitcoin community is being effectively taxed about
>>   $832,000 per day ... just to support mining! [...]
>> 
>>   We’re not spending so much on mining because we really need it. It’s
>>   because printing money distorts behaviour."
> 
>   -- https://medium.com/@octskyward/hashing-7d04a887acc8



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mai

Re: [bitcoin-dev] Fees and the block-finding process

[Eric Voskuil via bitcoin-dev]

Hi Michael,

> One of the key characteristics toward that is Bitcoin being
> inexpensive to transact.

What you seem to be missing is *why* bitcoin is better money. Have you
considered why is it comparatively inexpensive to transact in a medium
that is based on such a highly inefficient technology?

You might want to consider that these two considerations are not
independent. The reduced cost of transacting (and carrying) Bitcoin is a
direct consequence of its trustless nature. Any compromise in that
nature will eliminate that advantage, and therefore Bitcoin.

Bitcoin is designed to solve only one problem that other systems do not.
To accomplish this it makes significant compromises in other areas. The
benefit of this solution is that it cannot be effectively controlled by
the state. As a result, all of the associated overhead is eliminated.
Hence the net cost benefit despite high technical costs.

So this is a case where you should be careful what you wish for.

e

On 08/11/2015 02:18 PM, Michael Naber via bitcoin-dev wrote:
> The only reason why Bitcoin has grown the way it has, and in fact the
> only reason why we're all even here on this mailing list talking about
> this, is because Bitcoin is growing, since it's "better money than other
> money". One of the key characteristics toward that is Bitcoin being
> inexpensive to transact. If that characteristic is no longer true, then
> Bitcoin isn't going to grow, and in fact Bitcoin itself will be replaced
> by better money that is less expensive to transfer.
> 
> So the importance of this issue cannot be overstated -- it's compete or
> die for Bitcoin -- because people want to transact with global consensus
> at high volume, and because technology exists to service that want, then
> it's going to be met. This is basic rules of demand and supply. I don't
> necessarily disagree with your position on only wanting to support
> uncontroversial commits, but I think it's important to get consensus on
> the criticality of the block size issue: do you agree, disagree, or not
> take a side, and why?
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Bitcoin XT 0.11A

[Eric Voskuil via bitcoin-dev]

[cross-posted to libbitcoin]

I applaud this effort not for the merits of the hard fork but on the
effects of the code fork. We have been witnessing the self-destruction
of Bitcoin's central authority. This is a necessary outcome.

Understandably, many are concerned that if consensus settles on a larger
block size Bitcoin will suffer greater centralization. The point of
Bitcoin however is that nobody is in control of consensus. If a
consensus decision leads to centralization, the consensus will move.

Yes, when this happens people who bet on the losing fork will lose
money. This is how Bitcoin works. One bets not on personal desires but
on what others will accept. That is how consensus is built.

Some fret that if this process evolves Bitcoin will suffer a
catastrophic loss of "value" and not recover. This may come to pass, but
there is no avoiding the possibility.

The ease with which consensus can move when it wants to is important. In
other words, friction is weakness and a single overly complex codebase
is high friction.

Amir saw this coming before most. While we are posting manifestos, I
thought it more than appropriate to quote from the libbitcoin manifesto:

> If development is too centralised, with a small core infrastructure,
> then businesses will put real pressure to have features that destroy
> the integrity of the Bitcoin network.
> ...
> The possible malicious scenarios are endlessAt the other end of
> the spectrum, is putting development effort into diversifying the
> ecosystem to protect against censorship... That's where developers
> who believe in Bitcoin should devote time to.
> ...
> A diversified Bitcoin of many wallets and implementations is a strong
> and pure Bitcoin. To protect the integrity of the network, we need to
> eliminate single points of failure. An inbred Bitcoin with the same
> software code everywhere shares the same weaknesses, and is
> susceptible to the same attacks...
>
> The implications of a diversified Bitcoin is a Bitcoin difficult to
> control. It also sets the protocol in stone, as nobody has sole power
> over the standard. Consensus from many parties is the way forwards.
>...
> Viewed over a longer time period, extra or unnecessary features seem
> to creep into the system beyond the initial goals and the small code
> of 15,000 lines set by Satoshi. The result will be a Bitcoin that
> becomes increasingly difficult to understand or implement without a
> huge initial investment of resources, time and people. No single
> person will fully understand Bitcoin anymore, and development
> monopolies will be further enforced.

http://libbitcoin.dyne.org/libbitcoin-manifesto.pdf

e

On 08/15/2015 10:02 AM, Mike Hearn via bitcoin-dev wrote:
> Hello,
> 
> As promised, we have released Bitcoin XT 0.11A which includes the bigger
> blocks patch set. You can get it from
> 
>  https://bitcoinxt.software/
> 
> I feel sad that it's come to this, but there is no other way. The
> Bitcoin Core project has drifted so far from the principles myself and
> many others feel are important, that a fork is the only way to fix things.
> 
> Forking is a natural thing in the open source community, Bitcoin is not
> the first and won't be the last project to go through this. Often in
> forks, people say there was insufficient communication. So to ensure
> everything is crystal clear I've written a blog post and a kind of
> "manifesto" to describe why this is happening and how XT plans to be
> different from Core (assuming adoption, of course).
> 
> The article is here:
> 
> https://medium.com/@octskyward/why-is-bitcoin-forking-d647312d22c1
> 
> It makes no attempt to be neutral: this explains things from our point
> of view.
> 
> The manifesto is on the website.
> 
> I say to all developers on this list: if you also feel that Core is no
> longer serving the interests of Bitcoin users, come join us. We don't bite.
> 
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] libconsensus assertion fails if used in multiple threads

[Eric Voskuil via bitcoin-dev]

On 08/18/2015 03:31 AM, Tamas Blummer via bitcoin-dev wrote:
> The performance of libconsensus is surprisingly close to the java one. 
...
> Another nice demonstration why one should not trade in advances
> of languages for the last decades for a marginal gain of performance
> with C/C++...

If performance was the only trade-off with Java, and if C++ was frozen
in time, I would say you have a point.

Thanks for submitting, and Cory, thanks for working this out.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Bitcoin XT Fork

[Eric Voskuil via bitcoin-dev]

[cross-posted to libbitcoin]

On 08/19/2015 03:00 PM, Jorge Timón via bitcoin-dev wrote:> On Wed, Aug
19, 2015 at 10:04 PM, Eric Lombrozo  wrote:
>> But the consensus code should NOT be subject to the same commit
policies…and we should make an effort to separate the two clearly. And
we should find a way to communicate the difference succinctly and
clearly to laypeople (which is something I think the XT opponents have
been horrible at doing so far).
>
> I think that effort is in progress (again, much slower that I would
> like it to be) and it's called libconsensus.
> Once we have libconsensus Bitcoin Core it's just another
> implementation (even if it is the reference one) and it's not "the
> specification of the consensus rules" which is a "privileged" position
> that brings all sorts of misunderstandings and problems (the block
> size debate is just one example).

Jorge,

I applaud your efforts and objectives WRT libconsensus independence. But
as you know I differ with you on this point:

> Once we have libconsensus Bitcoin Core it's just another
> implementation

I do not consider Bitcoin Core just another implementation as long as
libconsensus is built directly out of the bitcoind repository. It's a
finer point, but an important one. Eric makes this point emphatically as
well:

>> But the consensus code should NOT be subject to the same commit
policies...and we should make an effort to separate the two clearly.

As you have implied, it's not likely to happen in the Bitcoin Core repo.
Taking a dependency on Bitcoin Core is a metaphorical deal with the
devil from our perspective. So my question is, how do you expect other
implementations to transition off of that repository (and commit
policies)? Or do you expect the dependency to be perpetual?

In our discussion leading up to libbitcoin building libbitcoin-consensus
we disagreed on whether intentional hard forks would (or even could)
happen. I think that issue is now settled. So my question remains how do
stakeholders (users/miners) maintain consensus when it's their
individual intent (the first objective of libconsensus), and diverge
when intended (which a direct dependency on libconsensus makes harder)?
IMO it's unreasonable to operate as if this won't happen, given that it has.

There are a very small number of implementations that rely on consensus
(fewer that aren't also forks of Bitcoin Core). I think it's time we
discuss how to work together to achieve our mutual goal. I assume you
have been in contact with all of us. If you would like to facilitate
this I'd be happy to join in an offline discussion.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Bitcoin XT Fork

[Eric Voskuil via bitcoin-dev]

On 08/19/2015 04:27 PM, Jorge Timón wrote:
> On Thu, Aug 20, 2015 at 1:07 AM, Eric Voskuil  wrote:
>> [cross-posted to libbitcoin]
>>
>> On 08/19/2015 03:00 PM, Jorge Timón via bitcoin-dev wrote:> On Wed, Aug
>> 19, 2015 at 10:04 PM, Eric Lombrozo  wrote:
 But the consensus code should NOT be subject to the same commit
>> policies…and we should make an effort to separate the two clearly. And
>> we should find a way to communicate the difference succinctly and
>> clearly to laypeople (which is something I think the XT opponents have
>> been horrible at doing so far).
>>>
>>> I think that effort is in progress (again, much slower that I would
>>> like it to be) and it's called libconsensus.
>>> Once we have libconsensus Bitcoin Core it's just another
>>> implementation (even if it is the reference one) and it's not "the
>>> specification of the consensus rules" which is a "privileged" position
>>> that brings all sorts of misunderstandings and problems (the block
>>> size debate is just one example).
>>
>> Jorge,
>>
>> I applaud your efforts and objectives WRT libconsensus independence. But
>> as you know I differ with you on this point:
>>
>>> Once we have libconsensus Bitcoin Core it's just another
>>> implementation
>>
>> I do not consider Bitcoin Core just another implementation as long as
>> libconsensus is built directly out of the bitcoind repository. It's a
>> finer point, but an important one. Eric makes this point emphatically as
>> well:
>>
 But the consensus code should NOT be subject to the same commit
>> policies...and we should make an effort to separate the two clearly.
>>
>> As you have implied, it's not likely to happen in the Bitcoin Core repo.
>> Taking a dependency on Bitcoin Core is a metaphorical deal with the
>> devil from our perspective. So my question is, how do you expect other
>> implementations to transition off of that repository (and commit
>> policies)? Or do you expect the dependency to be perpetual?
> 
> No, as previously explained, once libconsensus is complete it can be
> moved to a separate repository like libsecp256k1.

I don't see this happening any time soon, and I'm not sure why we should
wait for it.

> At first it will need to be a subtree/subrepository of Bitcoin Core
> (like libsecp256k1 currently is), but I still don't undesrtand how
> that can possibly be a problem for alternative implementations (they
> can use a subtree as well if they want to). Depending on a separated
> libconsensus doesn't "make Bitcoin Core a dependency" more than
> depending on libsecp256k1 currently does.
>
>> In our discussion leading up to libbitcoin building libbitcoin-consensus
>> we disagreed on whether intentional hard forks would (or even could)
>> happen. I think that issue is now settled. So my question remains how do
>> stakeholders (users/miners) maintain consensus when it's their
>> individual intent (the first objective of libconsensus), and diverge
>> when intended (which a direct dependency on libconsensus makes harder)?
>> IMO it's unreasonable to operate as if this won't happen, given that it has.
> 
> I believe the simplest option...

You might consider this as feedback from your customer base.

> would be to fork the libconsensus
> project and do the schism/controversial/contentious hardfork there.
> But of course modifying libconsensus will be much easier than
> modifying Bitcoin Core (if anything, because the amount of code is
> much smaller).

That's a false dichotomy. We never would have considered forking Bitcoin
Core, and still wouldn't. Why would we set ourselves up for this
disruption, which would inevitably lead to us factoring the consensus
portions of libconsensus out of /bitcoin at the 11th hour?

We have to operate as if it can happen at any time. Otherwise we have
relinquished control of this vote and failed our users. Given that
operating assumption, it is much safer for us to have already done this
work (which we did). [It also provides a forcing function for us to
review in detail any consensus changes that get pushed out.]

My question is why you would not embrace an independent consensus
repository? Your work to evolve it doesn't change.

>> There are a very small number of implementations that rely on consensus
>> (fewer that aren't also forks of Bitcoin Core). I think it's time we
>> discuss how to work together to achieve our mutual goal. I assume you
>> have been in contact with all of us. If you would like to facilitate
>> this I'd be happy to join in an offline discussion.
> 
> Unfortunately I only directly contacted libbitcoin because I was
> subscribed to the list at the time (maybe I'm still subscribed, not
> really sure).
> The other attempts to get feedback from other alternative
> implementations have been just mostly-ignored threads in bitcoin-dev.
> So, no, I cannot facilitate such a discussion, but I'm more than happy
> to collaborate to achieve our mutual goal.

OK

e
___
bitcoin-de

Re: [bitcoin-dev] Dynamically Controlled Bitcoin Block Size Max Cap [BIP 1xx - Draft]

[Eric Voskuil via bitcoin-dev]

> just as in democracy in general

One should be clear that Bitcoin is by no possible measure a democracy.

The proposed vote is on what goes into a particular github repository.
The outcome is ultimately controlled by those with control of that
repository.

Bitcoin is an anarchy by design. People will use whatever they want.

e

On 08/25/2015 02:14 PM, Matt Whitlock via bitcoin-dev wrote:
> On Tuesday, 25 August 2015, at 1:37 pm, Peter Todd wrote:
>> On Tue, Aug 25, 2015 at 04:26:23PM -0400, Matt Whitlock wrote:
>>> On Tuesday, 25 August 2015, at 1:16 pm, Peter Todd via bitcoin-dev
wrote:
 What would you think of an approach like John Dillon's proposal to
 explicitly give the economic majority of coin holders a vote for
the max
 blocksize? Miners could still vote BIP100 style for what max they were
 willing to use, limited in turn by the vote of the economic majority.
>>>
>>> What fraction of coin holders do you suppose will vote? And, of
those, what fraction have the technical knowledge to make an informed
vote? It would be like polling Toyota truck owners to see whether the
2017 Tacoma should increase its engine's cylinder displacement by 10%.
Ordinary users just aren't going to be able to vote meaningfully, and
most won't respond to the poll at all.
>>
>> Note that you can make the % of voters required adapt dyanmically to
voter
>> interest. Also, your example is rather misleading, as car buyers *do*
>> make those kinds of decisions though various market mechanisms.
>
> Yes, car buyers do make those kinds of decisions through market
mechanisms. An equivalent process for Bitcoin would be that the max
block-size limit (and other fundamental economic parameters) would be
determined via a process of forking off altcoins (such as Bitcoin XT
will do) and allowing the market to decide which coin is most valuable.
This is the "default" mechanism for change (because it's what naturally
happens when there is a lack of internal consensus), but it's not the
least painful mechanism.
>
> My point still stands that — just as in democracy in general — the
voters are really in no position to cast informed votes, nor should they
be (cf. "rational ignorance" [1]). I do not oppose opening up the
determination of the max block-size limit to a popular "check" via
stakeholder vote — actually, I think this is an important check on
miners' power — but I do argue that the vote is likely to have
drastically little participation and very low-quality results.
>
> [1] Rational Ignorance: «Ignorance about an issue is said to be
"rational" when the cost of educating oneself about the issue
sufficiently to make an informed decision can outweigh any potential
benefit one could reasonably expect to gain from that decision, and so
it would be irrational to waste time doing so.»





signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Your Gmaxwell exchange

[Eric Voskuil via bitcoin-dev]

On 09/01/2015 09:51 AM, Monarch via bitcoin-dev wrote:
> On 2015-09-01 15:59, Dave Collins via bitcoin-dev wrote:
>> I'd be interested to know about these supposed btcd mainnet forks that
>> have occurred due to a consensus failure since it came out of alpha.
>> I'll go ahead and save you some research time - there hasn't been one.
>> I'm not claiming there will never be one as that would be just as
>> foolish as claiming Bitcoin Core won't have any more either.
>
> For the purposes of the conversation this was only brought up to re-
> enforce my claim that this is outrageously difficult software
> development, irrespective of the quality of the code being produced in
> alternate implementations.  Sorry if that came across as an attack
> against your software in particular, it wasn't intended.

Whether intended or otherwise this is an attack on the idea of
decentralized bitcoin development. The option to fork or roll your own
is open source, not decentralization. Decentralization requires
*actually doing so*. One step down that path, even for a fork, is a
major commitment.

Common consensus check code is now available in several bitcoin
implementations. The claim that this is outrageously difficult is
misleading. It's just engineering work that needs to get done if Bitcoin
is to survive.

>> On the other hand, Bitcoin Core has had actual forks on mainnet during
>> the same period.  I'm not casting stones at Bitcoin Core here, because
>> as I've said many times, none of us are perfect.  No matter how careful
>> everyone is, it is bound to happen from time to time.
> 
> The point I was trying to make is that this is simply a hard
> development situation to be working in, we don't know what behavior is
> inferred by the use of CPP and even more so OpenSSL (as the DER
> encoding consensus failure made abundantly clear).  There's almost
> certainly more problems lying around given how generally dusty a lot
> of the transaction environment is, it's very easy to get off the
> beaten track with Bitcoin script.

These are issues that affect the satoshi client as much as other
implementations, and therefore don't support your premise.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP-38 issue and altchain support

[Eric Voskuil via bitcoin-dev]

In the integration of BIP-38 into libbitcoin we ran into two issues.

First, the scenario that justifies the "confirmation code" is flawed. We
have implemented full support for this, but have also marked it as
deprecated.

I am seeking counter arguments, in case there is some scenario that we
haven't imagined where it might be useful. Details here:

[TLDR: the confirmation code cannot prove anything about the owner's
ability to spend from the public-key/address that it confirms.]

https://github.com/libbitcoin/libbitcoin/wiki/BIP38-Security-Considerations

Second, BIP-38 envisions altchain integration but doesn't specify it. We
have implemented the capability, documented here:

[TLDR: incorporate the payment address version into the last byte of the
encoded encrypted key prefixes, with backward compatibility]

https://github.com/libbitcoin/libbitcoin/wiki/Altchain-Encrypted-Private-Keys

If there is sufficient support I'll write up a Proposal that modifies
BIP-38.

Thanks to Neill Miller for the libbitcoin and bx BIP-38 pull requests.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Scaling Bitcoin conference micro-report

[Eric Voskuil via bitcoin-dev]

On 09/18/2015 11:06 PM, NxtChg via bitcoin-dev wrote:
>> While to many of us that sounds crazy, if you're threat model assumes
>> Bitcoin is a legal/regulated service provided by a highly trusted
>> mining community it's a reasonable design.
>
> There is a large, grey area all the way to "legal/regulated service
> provided by a highly trusted mining community". Painting the worst
> looking picture is either a defect in thinking or intentional FUD.

The state is the threat in the Bitcoin threat model. You comments below
acknowledge it. The assumption of hostile state actors is the only
rational starting point. That which is regulated (and regulatable) in
Bitcoin is the attack surface.

While of course there are various degrees of weakness, the reference to
"legal/regulated service provided by a highly trusted mining" as the
threat is by no means irrational or misdirecting. This threat represents
the difference between Bitcoin and Fedcoin.

I found Mike's threat model downright disturbing. All benefits of
Bitcoin arise from its resistance to this threat. Anyone investor in
this space should be paying attention... the apparent benefits of
Bitcoin will vaporize with regulation.

>> Mike Hearn recently posted his threat model, which specifically
>> argues we should assume governments are not a threat.
>
> There are two ways to fight governments:
>
> 1. either you become too big to close, so political repercussions
> become unacceptable

This is extremely naive. At a minimum, getting popular/successful (and
regulated) is the formula for regulatory capture.

> 2. or you become too tiny to hunt, in which case you are much better
> off with a specialized alt-coin, designed specifically for that
> purpose.

I assume you are referring some marginal and largely irrelevant effort.

False dichotomy.

[cross-posted to libbitcoin]



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Scaling Bitcoin conference micro-report

[Eric Voskuil via bitcoin-dev]

On 09/19/2015 12:27 AM, NxtChg wrote:
>> This is extremely naive. At a minimum, getting popular/successful (and 
>> regulated) is the formula for regulatory capture.
> 
> Let me give you an example.
> 
> Suppose you are a regular guy, say Peter Todd, and you are faced with 10 
> policemen in anti-riot gear.
> 
> You can fight them in two ways:
> 
> 1. become stronger, so you could provide an adequate response, either by 
> turning into Hulk or by getting another 30-50 Peter Todds.

Your vision of censorship resistance is to become such a strong central
authority that you can resist it in direct physical confrontation. If
you succeed at this, you are the threat.

> 2. lose some fat, learn a few parkour tricks and move around mostly by night 
> behind dumpsters.

And your alternative is to lurk in dark corners.


The inability to see another option is the inability to understand what
Satoshi created.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Scaling Bitcoin conference micro-report

[Eric Voskuil via bitcoin-dev]

On 09/19/2015 12:57 AM, NxtChg wrote:>
>> Your vision of censorship resistance is to become such a strong
>> central authority that you can resist it in direct physical
>> confrontation. If you succeed at this, you are the threat.
>
> My vision is a strong _decentralized_ system, which is:
>
>   a) too important to close,

Your argument is that the state is not a threat to a system designed to
deprive the state of seigniorage, because the state will see that system
as too important?

Bitcoin cannot be both decentralized and reliant on being, "too
important to close". If it can be closed there is insufficient
decentralization.

I was concerned that this was going off topic for a technical forum.
However this is the central technical issue of Bitcoin. If one does not
understand the threat then one cannot model it or design systems to
defend against it. On the other hand, this is unfortunately not new
territory, so I'll leave it at this, which is also not news to most of us...


>   b) able to provide adequate response to governments, like EFF or
Google do.

"The National Security Agency paid millions of dollars to cover the
costs of major internet companies involved in the Prism surveillance
program after a court ruled that some of the agency's activities were
unconstitutional, according to top-secret material passed to the Guardian.

The technology companies, which the NSA says includes Google..."

http://www.theguardian.com/world/2013/aug/23/nsa-prism-costs-tech-companies-paid

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] summarising security assumptions (re cost metrics)

[Eric Voskuil via bitcoin-dev]

On 11/05/2015 03:03 PM, Adam Back via bitcoin-dev wrote:
> ...
> Validators: Economically dependent full nodes are an important part of
> Bitcoin's security model because they assure Bitcoin security by
> enforcing consensus rules.  While full nodes do not have orphan
> risk, we also dont want maliciously crafted blocks with pathological
> validation cost to erode security by knocking reasonable spec full
> nodes off the network on CPU (or bandwidth grounds).
> ...
> Validators vs Miner decentralisation balance:
> 
> There is a tradeoff where we can tolerate weak miner decentralisation
> if we can rely on good validator decentralisation or vice versa.  But
> both being weak is risky.  Currently given mining centralisation
> itself is weak, that makes validator decentralisation a critical
> remaining defence - ie security depends more on validator
> decentralisation than it would if mining decentralisation was in a
> better shape.

This side of the security model seems underappreciated, if not poorly
understood. Weakening is not just occurring because of the proliferation
of non-validating wallet software and centralized (web) wallets, but
also centralized Bitcoin APIs.

Over time developers tend to settle on a couple of API providers for a
given problem. Bing and Google for search and mapping, for example. All
applications and users of them, depending on an API service, reduce to a
single validator. Imagine most Bitcoin applications built on the
equivalent of Bing and Google.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] summarising security assumptions (re cost metrics)

[Eric Voskuil via bitcoin-dev]

On 11/06/2015 03:41 PM, Chris Priest wrote:
>> The bigger point however, which Erik explained, was: widespread use of
>> APIs as a sole means of interfacing with the blockchain also
>> contributes to reducing network security for everyone, because it
>> erodes the consensus rule validation security described under
>> "Validators" in the OP.
> 
> I completely disagree with this. You are implying that there is some
> sort of ideal ratio of full nodes to 'client only' nodes that the
> network must maintain. You seem to be implying that if that ideal
> ratio is to somehow be disrupted, then security suffers. My question
> to you is what is that ideal ratio and what methodology did you use to
> come up with it?

Nobody has advocated a golden ratio.

> The way I see it, the security of the system is independent on ratio
> between full nodes and lightweight nodes.
> 
> In other words, if there are 100,000 lightweight wallets to 100 full
> nodes, then you have the same security profile as one with 100,000
> full nodes to 100 lightweight wallets.

This is a false dichotomy. Both scenarios are poor for security, as
nobody with a wallet is validating. It's entirely possible, even
probable, that one person controls all of the nodes.

> I think most 'big blockers' think the same way I do, hence the rub
> between the two camps.
> 
> Small block people need to make a better case as to how exactly full
> node ratio relates to network security (especially the 'for everyone'
> part), because the link is not clear to me at all. Small block people
> seem to take this simple fact as self evident, but I just don't see
> it.

Fewer people independently validating their own transactions means trust
is placed in fewer people. The degenerate case of one validator and
everyone trusting it is dispositive, and equates roughly to the Federal
Reserve.

> On 11/6/15, Adam Back  wrote:
>> You're right that it is better that there be more APIs than fewer,
>> that is less of a single point of failure.  It also depends what you
>> mean by APIs: using an API to have a second cross-check of information
>> is quite different to building a wallet or business that only
>> interfaces with the blockchain via a 3rd party API.  There are
>> different APIs also: some are additive eg they add a second signature
>> via multisig, but even those models while better can still be a mixed
>> story for security, if the user is not also checking their own
>> full-node or checking SPV to make the first signature.
>>
>> Power users and businesses using APIs instead of running a full-node,
>> or instead of doing SPV checks, should be clear about the API and what
>> security they are delegating to a third party, and whether they have a
>> reason to trust the governance and security competence of the third
>> party: in the simplest case it can reduce their and their users
>> security below SPV.
>>
>> The bigger point however, which Erik explained, was: widespread use of
>> APIs as a sole means of interfacing with the blockchain also
>> contributes to reducing network security for everyone, because it
>> erodes the consensus rule validation security described under
>> "Validators" in the OP.
>>
>> Adam
>>
>>
>> On 6 November 2015 at 09:05, Chris Priest  wrote:
>>> On 11/5/15, Eric Voskuil via bitcoin-dev
>>>  wrote:
>>>> On 11/05/2015 03:03 PM, Adam Back via bitcoin-dev wrote:
>>>>> ...
>>>>> Validators: Economically dependent full nodes are an important part of
>>>>> Bitcoin's security model because they assure Bitcoin security by
>>>>> enforcing consensus rules.  While full nodes do not have orphan
>>>>> risk, we also dont want maliciously crafted blocks with pathological
>>>>> validation cost to erode security by knocking reasonable spec full
>>>>> nodes off the network on CPU (or bandwidth grounds).
>>>>> ...
>>>>> Validators vs Miner decentralisation balance:
>>>>>
>>>>> There is a tradeoff where we can tolerate weak miner decentralisation
>>>>> if we can rely on good validator decentralisation or vice versa.  But
>>>>> both being weak is risky.  Currently given mining centralisation
>>>>> itself is weak, that makes validator decentralisation a critical
>>>>> remaining defence - ie security depends more on validator
>>>>> decentralisation than it would if mining decentralisation was in a
>>>>> better shape.
>>>>
>>>> This side of

Re: [bitcoin-dev] Libconsensus phase 2

[Eric Voskuil via bitcoin-dev]

Jorge, first, thanks again for your work on this.

Without creating and using a public blockchain interface in phase 2, how
will you isolate the database dependency from consensus critical code?
Is it that the interface will exist but you will recommend against its use?

This work presumes that the users of the library reject the argument
that the database implementation is consensus critical code. Faithful
reproduction of stored data is a prerequisite for a validity. But a
common store implementation is only slightly more reasonable for this
library than a common RAM implementation.

e

On 01/12/2016 09:53 AM, Jorge Timón wrote:
> After talking to some people about libconsensus in the last Hong Kong
> conference I realized that my initial plan of exposing one more thing
> at a time would actually probably slow things down.
> 
> There's still a promised pdf with pictures that will be released, and
> actually drafting the UML pictures helped realize that the whole
> explanation could be much simpler if #7091 was merged first as the
> last step in phase 1 (that phase has so many contributors that I will
> probably never get finished documenting it). Matt Corallo's idea of
> exposing VerifyScript() through a C API certainly helped a lot in
> cementing the more-minimal-than-earlier dependencies (thanks to Cory
> Fields among many other people before him) that are not part of the
> incomplete but existing libbitcoinconsensus library.
> 
> Given this success in protecting encapsulation by exposing things in a
> new library, my instinct was to expose more things: VerifyHeader(),
> VerifyTx() and VerifyBlock() [in that order].
> But all those three new functions depend on storage in one way or
> another. That was part of my reasoning to expose VerifyHeader() first,
> because I believe there will be less discussion on a common interface
> for the stored longest chain than for the utxo view (which may depend
> on other transactions spent within the same block).
> In any case, I realized we should finish putting all the consensus
> critical code in the libconsensus lib and then worry about its "final"
> API.
> 
> Therefore I changed the goal of the phase 2 in my libconsensus
> encapsulation planning from "expose VerifyHeader() in the existing
> libconsensus library" to "build all the consensus critical code within
> the existing libconsensus library, even if we don't expose anything
> else". I believe this is much feasible for a single Bitcoin Core
> release cycle and also more of a priority. Other implementations
> experimenting with libconsensus like
> https://github.com/libbitcoin/libbitcoin-consensus will have the
> chance to compare their reimplementations with the future complete
> libbitcoinconsensus without having to worry about the C API, which
> ideally they will help to define.
> 
> I repeat, the goal of phase 2 in my upcoming libconsensus
> encapsulation plan is to fully decouple libconsensus from Bitcoin
> Core.
> In phase 3, we can refine the storage interfaces and focus on a
> quasi-final C API.
> In phase 4, we can refine and take out code that doesn't belong in
> libconsensus like CTxOut::GetDustThreshold() in
> primitives/transaction.h and move all those consensus files to the
> consensus folder before creating a separate sub-repository like for
> libsecp256k1. Note that most of the file-moving work can be in
> parallel to phases 2 and 3 and, in fact, by any new developer that is
> willing to exchange rebase-patience for meaningless github stats (I'll
> do it if nobody else wants, but I'm more than happy to delegate there:
> I have more than enough github meaningless stats already).
> 
> As said, the document with pictures and the update to #6714 are still
> promised, but until they're ready, merging/reviewing #7091, #7287,
> #7310 and #7311 could do a great deal to make later steps in
> libconsensus phase 2 more readable.
> Most reviewers probably don't need to see any "big picture" to tell
> whether certain functions on Bitcoin Core are consensus-critical or
> not, or whether consensus critical code needs to depend on util.o or
> not.
> But I wouldn't be writing to the mailing list without a plan with
> further words nor pictures if I didn't had what I believe is a
> complete implementation of what I just defined as "libconsensus phase
> 2".
> 
> Phase 3 should finish long pending discussions like "should
> libconsensus be C++14 or plain C" which should NOT delay phase 2.
> Phase 4 should be mostly trivial: rename files to the target dir and
> move the remaining unused code out of libconsensus.
> Phase 5 should make Bitcoin Core eat its own dog food and use
> libbitcoinconsensus oonly by its generic C API (I'm sorry if this
> looks too far away for me to even think about detailing it).
> 
> The work in progress branch (but hopefully being finished, nit and
> merged within the 0.12.99 cycle) can be found in:
> https://github.com/jtimon/bitcoin/commits/libconsensus-f2
> 
> Before sipa asks, signing 

Re: [bitcoin-dev] Libconsensus phase 2

[Eric Voskuil via bitcoin-dev]

On 01/13/2016 12:37 AM, Jorge Timón wrote:
> On Tue, Jan 12, 2016 at 8:17 PM, Eric Voskuil  wrote:
>> Jorge, first, thanks again for your work on this.
>>
>> Without creating and using a public blockchain interface in phase 2, how
>> will you isolate the database dependency from consensus critical code?
>> Is it that the interface will exist but you will recommend against its use?
> 
> The interface will exist but it will be a C++ interface that fits
> better with Bitcoin Core internals.
> See an initial draft of what could be the storage interface:
> https://github.com/jtimon/bitcoin/blob/1717db89c6db17ea65ddbd5eb19034315db0b059/src/consensus/storage_interfaces_cpp.h
> 
> Phase 3 will consist on discussing and refining that interface to also
> define the C interfaces using structs of function pointers instead of
> classes (see 
> https://github.com/jtimon/bitcoin/blob/2bcc07c014e5dd30e666be047dfa11f54c10/src/consensus/interfaces.h
> for an early draft) that is needed for the "final" C API.
> But since I think there will be more discussion and work defining
> those interfaces, I would rather start with ANY interface that allows
> us to decouple the consensus code from chain.o and coins.o, which we
> don't want to be built as part of the consensus building package
> (which is used for building both libbitcoinconsensus and Bitcoin
> Core).

Okay.

> Future potential users are more than welcomed to draft their own C
> APIs and that experience should be useful for phase 3.
> I was expecting you, for example, to include the whole consensus code
> (even if it lacks a C API) in
> https://github.com/libbitcoin/libbitcoin-consensus for better testing
> of the equivalent code in libbitcoin. You are kind of taking the C API
> part out already, so this time you will just have less things to
> delete/ignore.

Generalization of the store interface may be more challenging than you
anticipate, but the objective makes sense.

>> This work presumes that the users of the library reject the argument
>> that the database implementation is consensus critical code. Faithful
>> reproduction of stored data is a prerequisite for a validity. But a
>> common store implementation is only slightly more reasonable for this
>> library than a common RAM implementation.
> 
> This is a concern that has been risen repeatedly.
> I am aware that faithful reproduction of the stored data is a
> prerequisite for consensus validity. On the other hand, my presumption
> is that a libbitcoinconsensus that forces its users to a given unifrom
> storage will likely had much less users and any alternative
> implementation that wants to implement its own custom storage would
> have to necessarily reimplement the consensus validation code.
> Doing it this way is more flexible. We can relatively easily implement
> another library (if I remember correctly, last time we talked about it
> we reffered to it as "libconsensus plus", but there's probably better
> names) also takes care of storage for the users that don't want to
> take the risks of reimplementing the storage (probably just using
> Bitcoin Core's structures).
> 
> Unlike me, Luke Dashjr, for example, advocated for the
> storage-dependent version, but I believe that implementing both
> versions was an acceptable solution to him.
> It is certainly an acceptable solution for me. I don't want to force
> anyone that doesn't want or need to take the risks reimplementing the
> consensus storage part to do so. But at the same time I really believe
> that it would be a mistake to not allow it optionally.
> 
> Does that make sense?

We would not offer libbitcoinconsensus integration if it required us to
incorporate the store. These are distinct logical components, as are p2p
networking and client-server networking (e.g. RPC), for example. I would
not think of these as multiple versions of libbitcoinconsensus but
instead as distinct components of a bitcoin node. It doesn't make sense
to me that you would ship this as two consensus variants. I would work
toward shipping independent component libraries (i.e. consensus and store).

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Hardfork to fix difficulty drop algorithm

[Eric Voskuil via bitcoin-dev]

> A 6 month investment with 3 months on the high subsidy and 3 months on low 
> subsidy would not be made…

 

Yes, this is the essential point. All capital investments are made based on 
expectations of future returns. To the extent that futures are perfectly 
knowable, they can be perfectly factored in. This is why inflation in Bitcoin 
is not a tax, it’s a cost. These step functions are made continuous by their 
predictability, removing that predictability will make them -- unpredictable.

 

Changing these futures punishes those who have planned properly and favors 
those who have not. Sort of like a Bitcoin bail-in; are some miners are too big 
to fail? It also creates the expectation that it may happen again. This infects 
the money with the sort of uncertainty that Bitcoin is designed to prevent.

 

e

 

From: bitcoin-dev-boun...@lists.linuxfoundation.org 
[mailto:bitcoin-dev-boun...@lists.linuxfoundation.org] On Behalf Of Tier Nolan 
via bitcoin-dev
Sent: Wednesday, March 2, 2016 10:08 AM
Cc: Bitcoin Dev 
Subject: Re: [bitcoin-dev] Hardfork to fix difficulty drop algorithm

 

On Wed, Mar 2, 2016 at 4:27 PM, Paul Sztorc via bitcoin-dev 
mailto:bitcoin-dev@lists.linuxfoundation.org> > wrote:

For example, it is theoretically possible that 100% of miners (not 50%
or 10%) will shut off their hardware. This is because it is revenue
which ~halves, not profit.

 

It depends on how much is sunk costs and how much is marginal costs too.

If hashing costs are 50% capital and 50% marginal, then the entire network will 
be able to absorb a 50% drop in subsidy.

50% capital costs means that the cost of the loan to buy the hardware 
represents half the cost.

Assume that for every $100 of income, you have to pay $49 for the loan and $49 
for electricity giving 2% profit.  If the subsidy halves, then you only get $50 
of income, so lose $48.  

But if the bank repossesses the operation, they might as well keep things 
running for the $1 in marginal profit (or sell on the hardware to someone who 
will keep using it).

Since this drop in revenue is well known in advance, businesses will spend less 
on capital.  That means that there should be less mining hardware than 
otherwise.

A 6 month investment with 3 months on the high subsidy and 3 months on low 
subsidy would not be made if it only generated a small profit for the first 3 
and then massive losses for the 2nd period of 3 months.  For it to be made, 
there needs to be large profit during the first period to compensate for the 
losses in the 2nd period.

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Hardfork to fix difficulty drop algorithm

[Eric Voskuil via bitcoin-dev]

On 03/02/2016 12:08 PM, Paul Sztorc wrote:
> On 3/2/2016 2:01 PM, Eric Voskuil via bitcoin-dev wrote:
>>> A 6 month investment with 3 months on the high subsidy and 3 months on
>> low subsidy would not be made…
>>
>> Yes, this is the essential point. All capital investments are made based
>> on expectations of future returns. To the extent that futures are
>> perfectly knowable, they can be perfectly factored in. This is why
>> inflation in Bitcoin is not a tax, it’s a cost. These step functions are
>> made continuous by their predictability, removing that predictability
>> will make them -- unpredictable.
> 
> The Ministry of Truth is taking job applications in the doublespeak
> department...

Not sure how you interpret a tautology as doublespeak.

>> Changing these futures punishes those who have planned properly and
>> favors those who have not. Sort of like a Bitcoin bail-in; are some
>> miners are too big to fail? It also creates the expectation that it may
>> happen again. This infects the money with the sort of uncertainty that
>> Bitcoin is designed to prevent.
> 
> Coinbase-smoothing can be done via soft fork (soft forks typically only
> move "one way" toward stability).

I'm addressing the hard fork proposal (see subject line).

> Moreover, the effect *costs* miners,
> it does not benefit them. Finally, it can be done so that the economic
> impact on miners is minimized.

Changes to consensus rules change the value of coins, which are property
of their owners. Nobody owes a miner a promise of consistent revenue for
future work. Cost or benefit to miners is relevant only to the extent
that those who hold money believe it will affect their value and
therefore consider it in their decision to consent.

> You'll just have to weigh the risks -- some vague, tiny effect on
> expectations today, vs the need for a small group of experts to
> emergency hard fork once every four years.

How is the small group of experts today different from the small group
of experts tomorrow?

> I'm sure those experts are completely reliable, and won't get threatened
> or assassinated!

This is precisely the issue. The precedent of hard-forking to "fix" the
money is a precedent for establishing authority over the money.

>> A 6 month investment with 3 months on the high subsidy and 3 months on
>> low subsidy would not be made if it only generated a small profit for
>> the first 3 and then massive losses for the 2nd period of 3 months.  For
>> it to be made, there needs to be large profit during the first period to
>> compensate for the losses in the 2nd period.
> 
> The word "loss" is of utmost importance here...if they are operational
> losses, it should be obvious to everyone that the best "compensation for
> losses in the 2nd period" is to just shut them off (thus reducing losses
> to zero).

But of course the losses would not be entirely operational, since
hardware (at a minimum) does not depreciate to zero because of a
halving. The ability to plan does not change this fact. There are
certainly similar considerations for labor, bandwidth, space and even
electrical/cooling costs (contracts). To the extent that these costs are
sunk (as Tier said) *any* earnings are better than none.

> So you must be arguing that miners have made an investment 3 months
> prior, knowing that it would pay for itself despite the reward halving.

Of course, how could they not?

> That's nice, but it ignores the fact that, if that investment is made
> everyone, by all miners, the *difficulty* will have increased 2 weeks
> afterward...such that operating profits are tending *immediately* toward
> zero, and will be zero by the time the first set of 3 months is over.

... which also ignores fees.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Hardfork to fix difficulty drop algorithm

[Eric Voskuil via bitcoin-dev]

On 03/02/2016 03:02 PM, Peter Todd wrote:
> On Wed, Mar 02, 2016 at 11:01:36AM -0800, Eric Voskuil via bitcoin-dev wrote:
>>> A 6 month investment with 3 months on the high subsidy and 3 months on low 
>>> subsidy would not be made…
>>
>> Yes, this is the essential point. All capital investments are made based on 
>> expectations of future returns. To the extent that futures are perfectly 
>> knowable, they can be perfectly factored in. This is why inflation in 
>> Bitcoin is not a tax, it’s a cost. These step functions are made continuous 
>> by their predictability, removing that predictability will make them -- 
>> unpredictable.
> 
> You know, I do agree with you.
> 
> But see, this is one of the reasons why we keep reminding people that
> strictly speaking a hardfork *is* an altcoin, and the altcoin can change
> any rule currently in Bitcoin.
> 
> It'd be perfectly reasonable to create an altcoin with a 22-million-coin
> limit and an inflation schedule that had smooth, rather than abrupt,
> drops. It'd also be reasonable to make that altcoin start with the same
> UTXO set as Bitcoin as a means of initial coin distribution.
> 
> If miners choose to start mining that altcoin en-mass on the halving,
> all the more power to them. It's our choice whether or not we buy those
> coins. We may choose not to, but if 95% of the hashing power decides to
> go mine something different we have to accept that under our current
> chosen rules confirmations might take a long time.
> 
> Of course, personally I agree with Gregory Maxwell: this is all fairly
> unlikely to happen, so the discussion is academic. But we'll see.
> 
I agree, this is a perfectly rational interpretation. I also agree that
this particular instance is academic. But I see more to this than
accepting what is possible.

In the case of Federal Reserve Notes the gold obligation was abrogated.
This was (at least) a contract default, implemented by force of arms.
This contentious hard fork was clearly an attack.

But in a system with no authority and in which nobody has formed a
contractual obligation with anyone else, what would constitute an attack
on the money? There is no difference between state attacks on (or
collusion with) miners and miners acting on self interest.

One answer is that nothing is an attack, it's up to the market to
decide. But to the extent that there can be an attack on the money, the
attempt to move the value of the coin to an altcoin (hard fork) is it.
Though the choice of the term "attack" isn't essential.

The importance of recognizing an attack is that it affords one the
opportunity to defend against it. People holding "dollars" in 1933 were
ill equipped to defend against a system level attack (monetary policy),
in part because many did not recognize it as such, and in part because
there was insufficient preparation by those who did.

I see us building the tools and awareness necessary for defense. As you
say, nobody has to buy into an altcoin forked from their coin. This much
is simple to achieve. The more difficult problem is preserving the
utility of the original coin. Clearly the purpose of a hard fork (as
opposed to a new coin) is to transfer this value.

We've all seen arguments for contentious hard fork deployment that
explicitly depend on the fear of monetary loss to drag people to
acceptance. While this may be the nature of the technology, it's
important that we develop effective defense against it.

Ultimately the only defense is individual validation. The collusion of
banks (web wallets) with miners in attacking consensus is obvious. But
even without active collusion, the surrender of validation leaves people
just as defenseless as *being* unarmed while retaining a right to
*become* armed.

Even if every person mines at the same level, the system amounts to
little more than majority rule if validation is not decentralized. There
are people perfectly willing to exploit this weakness.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] p2p authentication and encryption BIPs

[Eric Voskuil via bitcoin-dev]

On 03/23/2016 01:36 PM, Tom via bitcoin-dev wrote:
> On Wednesday 23 Mar 2016 16:24:12 Jonas Schnelli via bitcoin-dev wrote:
> * why would you not allow encryption on non-pre-approved connections?

Agree

> * we just removed (ssl) encryption from the JSON interface, how do you 
> suggest 
> this encryption to be implemented without openSSL?

CurveCP

> * What is the reason for using the p2p code to connect a wallet to a node?
> I suggest using one of the other connection methods to connect to the node. 
> This avoids a change in the bitcoin protocol for a very specific usecase.

Agree, P2P and client-server protocols are distinct use-cases. Missing
this distinction is the root cause of problems with the bloom filters
feature.

> * Why do you want to do a per-message encryption (wrapping the original)? 
> Smaller messages that contain predictable content and are able to be matched 
> to the unencrypted versions on the wire send to other nodes will open this 
> scheme up to various old statistical attacks.

Privacy cannot currently be achieved unless the server is trusted. In
most wallet scenarios that's not a reasonable assumption unless one
controls the full node. So this is only useful in the case where the
wallet is trusting a remote server, and as you point out - message
encryption is weak in this case. In a trustless server scenario
encryption would be unnecessary overhead.

>> Responding peers must ignore (banning would lead to fingerprinting) the 
> requesting peer after 5 unsuccessfully authentication tries to avoid resource 
> attacks.
> 
> Any implementation of that kind would itself again be open to resource 
> attacks.
> Why 5? Do you want to allow a node to make a typo?

Agree, denial of service protection can and should be much more flexible
than this. It's not necessary to incorporate DoS protection into a
protocol. I think maybe this stems from the ill-advised attempt at
messaging reliability.

>> To ensure that no message was dropped or blocked, the complete communication 
> must be hashed (sha256). Both peers keep the SHA256 context of the encryption 
> session. The complete enc message (leaving out the hash itself) 
> must be added to the hash-context by both parties. Before sending a 
> enc command, the sha256 context will be copied and finalized.
> 
> You write "the complete communication must be hashed" and every message has a 
> hash of the state until it is at that point.
> I think you need to explain how that works specifically.

Also, this gets into the area of messaging reliability. This is
certainly not something I would recommend for a P2P protocol optimized
for maintaining a cache of public data.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

I haven't seen much discussion here on the rationale behind BIP 151. Apologies 
if I missed it. I'm trying to understand why libbitcoin (or any node) would 
want to support it.

I understand the use, when coupled with a yet-to-be-devised identity system, 
with Bloom filter features. Yet these features are client-server in nature. 
Libbitcoin (for example) supports client-server features on an independent port 
(and implements a variant of CurveCP for encryption and identity). My concern 
arises with application of identity to the P2P protocol (excluding Bloom filter 
features).

It seems to me that the desire to secure against the weaknesses of BF is being 
casually generalized to the P2P network. That generalization may actually 
weaken the security of the P2P protocol. One might consider the proper 
resolution is to move the BF features to a client-server protocol.

The BIP does not make a case for other scenarios, or contemplate the 
significant problems associated with key distribution in any identity system. 
Given that the BIP relies on identity, these considerations should be fully 
vetted before heading down another blind alley.

e
-BEGIN PGP SIGNATURE-
Version: oPenGP 6.0 on iOS

iQEVAwUBV3IkYjzYwH8LXOFOAQg+iggAkFShi/ibZXiVv3A3z1a1SMd+4ar0kiZk
mCkBBZaatoW8tXVZmuv5xzLnj3ali9Y4jp/3h2nUJ1B4ov2kcB0kZIKE/a1DTFwb
4X3uSzgu0lEAqSZormOvt7Op46NPn6KJ+/wTtP4lUFU72lSd7qrVKMlCVc88VE7/
pMloKSc69nAeFIkyWbOUi/zDzefu/5tarfif85jumooYjPmAwJnkgiPCqpqBbuga
5lBdS1r47KK+SaDFl6Cbn4i/c6tBPLTnu+TR7TEKOW5vwVA7eUqb6SOK7pETWJGK
0Ii4ZWYt7MOPSEda381CMjWEwtsCNp0eI4GPZAAz+jNLo4G1+PAbaw==
=Balw
-END PGP SIGNATURE-

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

Hi Jonas, I'll follow up in your second reply as well. Responses inline:

> On Jun 28, 2016, at 10:26 AM, Jonas Schnelli via bitcoin-dev 
>  wrote:
> 
> Hi Eric
> 
>> I haven't seen much discussion here on the rationale behind BIP 151. 
>> Apologies if I missed it. I'm trying to understand why libbitcoin (or any 
>> node) would want to support it.
>> 
>> I understand the use, when coupled with a yet-to-be-devised identity system, 
>> with Bloom filter features. Yet these features are client-server in nature. 
>> Libbitcoin (for example) supports client-server features on an independent 
>> port (and implements a variant of CurveCP for encryption and identity). My 
>> concern arises with application of identity to the P2P protocol (excluding 
>> Bloom filter features).
>> 
>> It seems to me that the desire to secure against the weaknesses of BF is 
>> being casually generalized to the P2P network. That generalization may 
>> actually weaken the security of the P2P protocol. One might consider the 
>> proper resolution is to move the BF features to a client-server protocol.
>> 
>> The BIP does not make a case for other scenarios, or contemplate the 
>> significant problems associated with key distribution in any identity 
>> system. Given that the BIP relies on identity, these considerations should 
>> be fully vetted before heading down another blind alley.

> In my opinion, the question should be "why would you _not_ encrypt".

1) creation of a false sense of security
2) as a tradeoff against anonymity
3) benefit does not justify cost

> 1) Transaction censorship
> ISPs, WIFI provider or any other MITM, can holdback/censor unconfirmed
> transactions. Regardless if you are a miner or a validation/wallet node.
> 
> 2) Peer censorship
> MITM can remove or add entries from a "addr" message.
> 
> 3) Fingerprinting
> ISPs or any other MITM can intercept/inject fingerprinting relevant
> messages like "mempool" to analyze the bitcoin network.

Encryption alone cannot protect against a MITM attack in an anonymous and 
permissionless network. This is accepted in the BIP (and your follow-up reply).

> 4) SPV
> For obvious reasons regarding BF (see BIP or above).
> 
> 5) Goundwork for a "client-server" model over the P2P channel
> Fee estimation, bloom-filters, or any other message type that requires
> authentication.

I do not challenge the usefulness and appropriateness of encryption with 
authentication in a client-server blockchain protocol.

> I would not reduce BIP151 to only solve the BF privacy/censorship problem.
> 
> If we agree that censorship-resistance is one of the main properties of 
> Bitcoin,

We do.

> then we should definitively use a form of end-to-end encryption between 
> nodes. Built into the network layer.

This is the assumption that I'm questioning.

> There are plenty of other options to solve this problem. stunnel,
> Bernsteins CurveCP, VPN, etc. which are available since years.
> But the reality has shown that most bitcoin traffic is still unencrypted.

The question arises from concern over the security of the network in the case 
where encryption (and therefore authentication) is pervasive.

As you point out, anyone can set up a private network of nodes today. These 
nodes must also connect to the permissionless network to maintain the chain. 
These nodes constitute a trust zone within Bitcoin. This zone of exclusion 
operates as a single logical node from the perspective of the Bitcoin security 
model (one entity controls the validation rules for all nodes).

Widespread application of this model is potentially problematic. It is a 
non-trivial problem to design a distributed system that requires authentication 
but without identity and without central control. In fact this may be more 
challenging than Bitcoin itself. Trust on first use (TOFU) does not solve this 
problem.

In my opinion this question has not received sufficient consideration to 
warrant proceeding with a network encryption scheme (which concerns me as well, 
but as I consider it premature I won't comment).

> Example: IIRC non of the available SPV wallets can "speak" on of the
> possible encryption techniques. Encrypting traffic below the application
> layer is extremely hard to set up for non-experienced users.

Bloom filters can (and IMO should) be isolated from the P2P protocol. Also, if 
the proposal creates an insecurity its ease of deployment is moot.

> On top of that, encryption allows us to drop the SHA256 checksum per p2p
> message which should result in a better performance on the network layer
> once BIP151 is deployed.

I would not consider this a performance enhancing proposal. Simply dropping the 
checksum seems like a better option. But again, it is moot if it creates an 
insecurity.

> I agree that BIP151 _must_ be deployed together with an authentication
> scheme (I'm working on that) to protect again MITM during encryption
> initialization.

At a minimum I would propose that you modify BIP151 to decla

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

continued from previous post...

> On Jun 28, 2016, at 2:13 PM, Jonas Schnelli via bitcoin-dev 
>  wrote:
> 
> Hi Eric
> 
> Sorry for not directly addressing your points.

No problem. Thanks for the detailed replies.

> I try to be more precise in this follow up email:
> 
>> I understand the use, when coupled with a yet-to-be-devised identity system, 
>> with Bloom filter features. Yet these features are client-server in nature. 
>> Libbitcoin (for example) supports client-server features on an independent 
>> port (and implements a variant of CurveCP for encryption and identity). My 
>> concern arises with application of identity to the P2P protocol (excluding 
>> Bloom filter features).
> 
> I think the bloom filter SPV usecase is not "pure client-server". SPV
> clients could request from/broadcast to multiple "trusted nodes".

I have referred to the Bloom filters messages. These are clearly asymmetric in 
nature. Despite being possible it is not a valid use case for a full node to 
make BF requests to another node.

One client to multiple servers is still client-server for the sake of this 
discussion. The nature of the P2P protocol is synchronization of content 
between all nodes/peers. If the protocol is asymmetric the semantics, and 
therefore use cases, are different.

FWIW posting a transaction to the network can be done using the P2P protocol, 
connecting for a short period of time. But this is also a client-server 
scenario and is a hack when done (full disclosure, bx provides both P2P and 
client-server commands for tx posting). Broadcasting is naturally the behavior 
of a full node.

> Trusted nodes could be nodes where the operators have shared identities/keys 
> in advance over a different channel.

Yes, this is necessarily the case in order to prevent a MITM attack. This is 
the basis of my concern.

> Further private p2p extensions (lets say a p2p form of the estimatefee
> command) are something which needs to be discussed first and not
> something that is encouraged or outlined in BIP151.

Sure, but then let us not make assumptions about it in the context of this 
discussion. Libbitcoin provides fee estimation by monitoring broadcast 
penetration using a client-server protocol with an optional subscription 
mechanism.

>> It seems to me that the desire to secure against the weaknesses of BF is 
>> being casually generalized to the P2P network. That generalization may 
>> actually weaken the security of the P2P protocol. One might consider the 
>> proper resolution is to move the BF features to a client-server protocol.
> 
> I don't see reasons why BIP151 could weaken the security of the P2P network. 
> Can you point out some specific concerns?

TOFU cannot prevent MITM attacks (the goal of the encryption). Authentication 
requires a secure (trusted) side channel by which to distribute public keys. 
This presents what I consider a significant problem. If widespread, control 
over this distribution network would constitute control over who can use 
Bitcoin.

The effort to prevent censorship could actually enable it. I don't think it 
would get that far. Someone would point this out in the process of vetting the 
authentication BIP, and the result would be the scrapping of BIP151.

>> The BIP does not make a case for other scenarios, or contemplate the 
>> significant problems associated with key distribution in any identity 
>> system. Given that the BIP relies on identity, these considerations should 
>> be fully vetted before heading down another blind alley.
> 
> BIP151 does not rely on identities. BIP151 does not use persisted keys
> (only ephemeral keys).

BIP 151 is incomplete without authentication.

> The authentication/identity system needs to be described in a another BIP.
> But correct, BIP151 without a form of authentication/identity management
> is vulnerable to all sorts of MITM attacks and that's why I think BIP151
> must be deployed together with an p2p authentication scheme.

Agree, but my problem is that I do not believe we can assume this is a solvable 
problem.

> Scope creeping and the risks of overspecifying is the main reason to
> focus on the "pure encryption part" in BIP151.

Understood, yet this is the basis of my blind alley comment.

e

> Thanks
> ---
> 
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

Hi Peter,

What in this BIP makes a MITM attack easier (or easy) to detect, or increases 
the probability of one being detected?

e

> On Jun 28, 2016, at 8:22 PM, Peter Todd  wrote:
> 
> On Tue, Jun 28, 2016 at 06:45:58PM +0200, Eric Voskuil via bitcoin-dev wrote:
>>> 1) Transaction censorship
>>> ISPs, WIFI provider or any other MITM, can holdback/censor unconfirmed
>>> transactions. Regardless if you are a miner or a validation/wallet node.
>>> 
>>> 2) Peer censorship
>>> MITM can remove or add entries from a "addr" message.
>>> 
>>> 3) Fingerprinting
>>> ISPs or any other MITM can intercept/inject fingerprinting relevant
>>> messages like "mempool" to analyze the bitcoin network.
>> 
>> Encryption alone cannot protect against a MITM attack in an anonymous and 
>> permissionless network. This is accepted in the BIP (and your follow-up 
>> reply).
> 
> Being able to easily detect MITM attacks is a _huge_ step forward that
> shouldn't be underestimated; even if 99% of users aren't in a position to
> detect the MITM you only need a small subset of users that do the necessary
> checks to alert the wider community, who can then respond with stronger
> security measures. Those measures are likely to be more costly - authenticated
> systems are significantly harder than not - so better to save your efforts
> until the need for them is more obvious.
> 
> Also the fact that an attack has a reasonable probability of detection is a 
> big
> disincentive for many types of attackers - note how one of the things revealed
> in the Snowden leaks was the fact that the NSA generally tries quite hard to
> avoid tipping off targets to the fact that they're being surveilled, with a
> myriad of carefully scripted policies to control when and how exploits are 
> used
> against targets.
> 
> -- 
> https://petertodd.org 'peter'[:-1]@petertodd.org
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

> On Jun 28, 2016, at 10:14 PM, Peter Todd  wrote:
> 
>> On Tue, Jun 28, 2016 at 08:35:26PM +0200, Eric Voskuil wrote:
>> Hi Peter,
>> 
>> What in this BIP makes a MITM attack easier (or easy) to detect, or 
>> increases the probability of one being detected?
> 
> BIP151 gives users the tools to detect a MITM attack.
> 
> It's kinda like PGP in that way: lots of PGP users don't properly check keys,

PGP requires a secure side channel for transmission of public keys. How does 
one "check" a key of an anonymous peer? I know you well enough to know you 
wouldn't trust a PGP key received over an insecure channel.

All you can prove is that you are talking to a peer and that communications in 
the session remain with that peer. The peer can be the attacker. As Jonas has 
acknowledged, authentication is required to actually guard against MITM attacks.

> so an attacker won't have a hard time MITM attacking those users. But some
> users do check keys, a labor intensive manual process, but not a process that
> requires any real cryptographic sophistication, let alone writing any code.
> It's very difficult for widescale attackers to distinguish the users who do
> check keys from the ones that don't, so if you MITM attack _any_ user you run
> the risk of running into one of the few that does check, and those users can
> alert everyone else.
> 
> The key thing, is we need to get everyones communications encrypted first: if
> we don't the MITM attacker can intercept 99% of the communications with 0% 
> risk
> of detection, because the non-sophisticated users are trivially 
> distinguishable from the sophisticated users: just find the users with 
> unencrypted
> communications!
> 
> -- 
> https://petertodd.org 'peter'[:-1]@petertodd.org
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

> On Jun 28, 2016, at 10:36 PM, Peter Todd  wrote:
> 
>> On Tue, Jun 28, 2016 at 10:29:54PM +0200, Eric Voskuil wrote:
>> 
>> 
 On Jun 28, 2016, at 10:14 PM, Peter Todd  wrote:
 
 On Tue, Jun 28, 2016 at 08:35:26PM +0200, Eric Voskuil wrote:
 Hi Peter,
 
 What in this BIP makes a MITM attack easier (or easy) to detect, or 
 increases the probability of one being detected?
>>> 
>>> BIP151 gives users the tools to detect a MITM attack.
>>> 
>>> It's kinda like PGP in that way: lots of PGP users don't properly check 
>>> keys,
>> 
>> PGP requires a secure side channel for transmission of public keys. How does 
>> one "check" a key of an anonymous peer? I know you well enough to know you 
>> wouldn't trust a PGP key received over an insecure channel.
>> 
>> All you can prove is that you are talking to a peer and that communications 
>> in the session remain with that peer. The peer can be the attacker. As Jonas 
>> has acknowledged, authentication is required to actually guard against MITM 
>> attacks.
> 
> Easy: anonymous peers aren't always actually anonymous.
> 
> A MITM attacker can't easily distinguish communications between two nodes that
> randomly picked their peers, and nodes that are connected because their 
> operators manually used -addnode to peer; in the latter case the operators can
> check whether or not they're being attacked with an out-of-band key check.

An "out of band key check" is not part of BIP151. It requires a secure channel 
and is authentication. So BIP151 doesn't provide the tools to detect an attack, 
that requires authentication. A general requirement for authentication is the 
issue I have raised.

e
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

> On Jun 28, 2016, at 11:36 PM, Gregory Maxwell  wrote:
> 
> On Tue, Jun 28, 2016 at 9:22 PM, Eric Voskuil via bitcoin-dev
>  wrote:
>> An "out of band key check" is not part of BIP151.
> 
> It has a session ID for this purpose.

Passing the session ID out of band is authentication. As this is explicitly not 
part of BIP151 it cannot be that BIP151 provides the tools to detect a attack 
(the point at issue).

>> It requires a secure channel and is authentication. So BIP151 doesn't 
>> provide the tools to detect an attack, that requires authentication. A 
>> general requirement for authentication is the issue I have raised.
> 
> One might wonder how you ever use a Bitcoin address, or even why we might 
> guess these emails from "you" aren't actually coming from the NSA.

The sarcasm is counterproductive Greg. By the same token I could ask how you 
ever use Bitcoin given that the P2P protocol is not encrypted or authenticated.

It doesn't matter who I am, maybe I am the NSA. I don't argue from a position 
of authority. Signing my emails while traveling on holiday with only my phone 
gets a little tedious.

The blockchain and mempool are a cache of public data. Transmission of a 
payment address to a payer is not a comparable scenario.

The possibility that authentication may become required to participate in this 
trustless network is a legitimate concern, and one that has not been addressed.

e
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

Hi Cameron, good to hear from you!

> On Jun 28, 2016, at 11:40 PM, Cameron Garnham  wrote:
> 
> Unauthenticated link level encryption is wonderful! MITM attacks are 
> overrated; as they require an active attacker.

This is not really the case with Bitcoin. A MITM attack does not require that 
the attacker find a way to inject traffic into the communication between nodes. 
Peers will connect to the attacker directly, or accept connections directly 
from it. Such attacks can be easier than even passive attacks.

> Stopping passive attacks is the low hanging fruit. This should be taken first.
> 
> Automated and secure peer authentication in a mesh network is a huge topic. 
> One of the unsolved problems in computer science.
> 
> A simple 'who is that' by asking for the fingerprint of your peers from your 
> other peers is a very simple way to get 'some' authentication.  Semi-trusted 
> index nodes also is a low hanging fruit for authentication.

It is the implication of widespread authentication that is at issue. Clearly 
there are ways to implement it using a secure side channels.

> However, let's first get unauthenticated encryption. Force the attackers to 
> use active attacks. (That are thousands times more costly to couduct).
> 
> Sent from my iPhone
> 
>> On 29 Jun 2016, at 00:36, Gregory Maxwell via bitcoin-dev 
>>  wrote:
>> 
>> On Tue, Jun 28, 2016 at 9:22 PM, Eric Voskuil via bitcoin-dev
>>  wrote:
>>> An "out of band key check" is not part of BIP151.
>> 
>> It has a session ID for this purpose.
>> 
>>> It requires a secure channel and is authentication. So BIP151 doesn't 
>>> provide the tools to detect an attack, that requires authentication. A 
>>> general requirement for authentication is the issue I have raised.
>> 
>> One might wonder how you ever use a Bitcoin address, or even why we
>> might guess these emails from "you" aren't actually coming from the
>> NSA.
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

Your description of the two scenarios reduces to one. They both require 
authentication, and if you intend to connect to potentially evil nodes you 
aren't securing anything with link level security except the knowledge that 
your potentially evil node connection remains so.

e

> On Jun 29, 2016, at 12:33 AM, Cameron Garnham  wrote:
> 
> 
> There are two different topics mixed up here.
> 
> 1. Link-level security (secure connection to the node we intended to connect 
> to).
> 
> 2. Node-level security (aka; don't connect to a 'evil node').
> 
> The fist requires link-level encryption and authentication.
> 
> The second requires identity authentication.
> 
> You described the 'evil node' attack; that indeed needs an identity system to 
> stop. However BIP151 doesn't intend to protect against connecting to evil 
> Bitcoin Nodes.
> 
> It is important not to mixup link-level authentication and node-level 
> authentication.
> 
> When your client picks random nodes to connect to, you are not considered 
> whom in particular runs them. (Rather that you have a good random sample of 
> the network).
> 
> If you manually add a friends node; at this point you wish to have node-level 
> authentication.  However, this may (and probably should) happen out-of-band.
> 
> 
> Sent from my iPhone
> 
>> On 29 Jun 2016, at 01:07, Eric Voskuil  wrote:
>> 
>> Hi Cameron, good to hear from you!
>> 
>>> On Jun 28, 2016, at 11:40 PM, Cameron Garnham  wrote:
>>> 
>>> Unauthenticated link level encryption is wonderful! MITM attacks are 
>>> overrated; as they require an active attacker.
>> 
>> This is not really the case with Bitcoin. A MITM attack does not require 
>> that the attacker find a way to inject traffic into the communication 
>> between nodes. Peers will connect to the attacker directly, or accept 
>> connections directly from it. Such attacks can be easier than even passive 
>> attacks.
>> 
>>> Stopping passive attacks is the low hanging fruit. This should be taken 
>>> first.
>>> 
>>> Automated and secure peer authentication in a mesh network is a huge topic. 
>>> One of the unsolved problems in computer science.
>>> 
>>> A simple 'who is that' by asking for the fingerprint of your peers from 
>>> your other peers is a very simple way to get 'some' authentication.  
>>> Semi-trusted index nodes also is a low hanging fruit for authentication.
>> 
>> It is the implication of widespread authentication that is at issue. Clearly 
>> there are ways to implement it using a secure side channels.
>> 
>>> However, let's first get unauthenticated encryption. Force the attackers to 
>>> use active attacks. (That are thousands times more costly to couduct).
>>> 
>>> Sent from my iPhone
>>> 
>>>> On 29 Jun 2016, at 00:36, Gregory Maxwell via bitcoin-dev 
>>>>  wrote:
>>>> 
>>>> On Tue, Jun 28, 2016 at 9:22 PM, Eric Voskuil via bitcoin-dev
>>>>  wrote:
>>>>> An "out of band key check" is not part of BIP151.
>>>> 
>>>> It has a session ID for this purpose.
>>>> 
>>>>> It requires a secure channel and is authentication. So BIP151 doesn't 
>>>>> provide the tools to detect an attack, that requires authentication. A 
>>>>> general requirement for authentication is the issue I have raised.
>>>> 
>>>> One might wonder how you ever use a Bitcoin address, or even why we
>>>> might guess these emails from "you" aren't actually coming from the
>>>> NSA.
>>>> ___
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev@lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

On Jun 28, 2016, at 10:06 PM, Jonas Schnelli  wrote:

>>> In my opinion, the question should be "why would you _not_ encrypt".
>> 
>> 1) creation of a false sense of security
> 
> False sense of security is mostly a communication issue.
> BIP151 does focus on encryption (not trust).
> 
> Are users aware of the fact that ISP/WiFi-Providers can track their
> bitcoin spending (if they use SPV+BF) and link it with other internet
> traffic or sell the data to anyone who is interested to do correlation?

The relevant question would be to ask whether encryption would prevent an ISP 
from doing so (which it would not). This is a good example of false sense of 
security.

> Are node operators aware of the possibilities that ISPs/Data-Centers,
> etc. can hold back peers, etc.?
> 
> If there is a false sense of security/anonymity, then we are already
> deep into this territory.
> BIP151 was designed as a puzzle-pice towards better security and better
> censorship resistance. You shouldn't project all sorts of "false sense
> of security" into BIP151. Is a stepping stone towards greater security.

FWIW I was just answering your question comprehensively. Relationship to BIP151 
is incidental (though apparently applicable).

Keep in mind my specific concern is not with the design of BIP151, it is with 
the implication of its dependency on an unspecified authentication proposal.

>> 2) as a tradeoff against anonymity
> 
> Can you point out the tradeoffs?
> BIP151 does not introduce fingerprinting possibilities.

The security tradeoff would arise from widespread deployment of authentication 
- which is necessary to make encryption useful against envisioned MITM attacks. 
See my previous discussion of trust zones below.

>> 3) benefit does not justify cost
> 
> Can you elaborate the costs?
> [Extremely simplified]: we need 300 lines of code from openssh
> (ChaCha20-Poly1305@openssl) and some ECDH magic (already in
> Bitcoin-Cores codebase) together with two or three (maybe payed)
> cryptoanalysis once the implementation is done.

Simply put, any code that is unnecessary does not justify its cost.

>>> There are plenty of other options to solve this problem. stunnel,
>>> Bernsteins CurveCP, VPN, etc. which are available since years.
>>> But the reality has shown that most bitcoin traffic is still unencrypted.
>> 
>> The question arises from concern over the security of the network in the 
>> case where encryption (and therefore authentication) is pervasive.
>> 
>> As you point out, anyone can set up a private network of nodes today. These 
>> nodes must also connect to the permissionless network to maintain the chain. 
>> These nodes constitute a trust zone within Bitcoin. This zone of exclusion 
>> operates as a single logical node from the perspective of the Bitcoin 
>> security model (one entity controls the validation rules for all nodes).
>> 
>> Widespread application of this model is potentially problematic. It is a 
>> non-trivial problem to design a distributed system that requires 
>> authentication but without identity and without central control. In fact 
>> this may be more challenging than Bitcoin itself. Trust on first use (TOFU) 
>> does not solve this problem.
> 
> Yes. There is no plan to adopt a TUFO scheme. Bip151 does not use TUFO
> it does not cover "trust" (It just encrypt all traffic).

TOFU (trust on first use) was a reference to what was discussed on IRC as a 
potential solution to the (deferred) authentication problem. I didn't mean to 
imply that it was part of BIP151.

> Imaging Bip151 together with a simple form of preshared EC key
> authentication (nonce signing or similar). You could drastically
> increase the security/censor-resistance-properties between nodes where
> owners have preshared identity keys (with nodes I also mean SPV/wallet
> nodes).

This is a restatement of what I have accepted as a premise - that 
authentication, and as such, key distribution, will be a necessary part of 
making any encryption scheme effective. "Preshared" implies a secure side 
channel for key distribution.

> And I guess there are plenty of awesome identity management system ideas
> tied or not tied to the Bitcoin blockchain out there.
> This is also a reason to not cover trust/authentication/identity in BIP151.
> It is  possible to have multiple authentication schemes.

Whether or not there are multiple schemes is not relevant to the point I have 
raised. The issue is that authentication is necessary.

>> In my opinion this question has not received sufficient consideration to 
>> warrant proceeding with a network encryption scheme (which concerns me as 
>> well, but as I consider it premature I won't comment).
> 
> Yes. I think nobody have started implementing BIP151. It's a draft BIP
> and I think it's still okay and great that we have this discussion.
> 
> BIP151 hopefully has started some brainwork in how encryption and
> authentication could work in Bitcoin and I'm happy to deprecate BIP151 if we 
> ha

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

On Jun 28, 2016, at 9:55 PM, Gregory Maxwell via bitcoin-dev 
 wrote:

>> I understand the use, when coupled with a yet-to-be-devised identity system, 
>> with Bloom filter features. Yet these features
> 
> This is a bit of a strawman, you've selected a single narrow usecase which 
> isn't proposed by the BIP and then argue it is worthless. I agree that 
> example doesn't have much value (and I believe that
> eventually the BIP37 bloom filters should be removed from the protocol).

I don't follow this comment. The BIP aims quite clearly at "SPV" wallets as its 
justifying scenario.

> Without something like BIP151 network participants cannot have privacy for 
> the transactions they originate within the protocol against network observers.

And they won't get it with BIP151 either. Being a peer is easier than observing 
the network. If one can observe the encrypted traffic one can certainly use a 
timing attack to determine what the node has sent.

> Even if, through some extraordinary effort, their own first hop is encrypted, 
> unencrypted later hops would rapidly
> expose significant information about transaction origins in the network.

As will remain the case until all connections are encrypted and authenticated, 
and all participants are known to be good guys. Starting to sound like PKI?

> Without something like BIP151 authenticated links are not possible, so
> manually curated links (addnode/connect) cannot be counted on to provide 
> protection against partitioning sybils.

If we trust the manual links we don't need/want the other links. In fact 
retaining the other links enables the attack you described above. Of course 
there is no need to worry about Sybil attacks when all of your peers are 
authenticated. But again, let us not ignore the problems of requiring all peers 
on the network be authenticated.

> Along the way BIP151 appears that it will actually make the protocol faster.
> 
>> Given that the BIP relies on identity
> 
> This is untrue. The proposal is an ephemerally keyed opportunistic
> encryption system. The privacy against a network observer does not depend on 
> authentication, much less "identity".  And when used with authentication at 
> all it makes interception strongly detectable after the fact.

Maybe I was insufficiently explicit. By "relies on identity" I meant that the 
BIP is not effective without it. I did not mean to imply that the BIP itself 
implements an identity scheme. I thought this was clear from the context.

>> The BIP does not [...] contemplate the significant problems associated with 
>> key distribution in any identity system
> 
> Because it does not propose any "identity system" or authorization (also, I 
> object to your apparent characterization of authentication as as an 'identity 
> system'-- do you also call Bitcoin addresses an identity system?).

Please read more carefully what I wrote. I did not characterize authentication 
as an identity system. I proposed that key distribution has significant 
problems, and used identity systems as an example of systems with such 
problems. I could just have easily written "authentication systems", (and 
probably should have).

> That said, manually maintaining adds nodes to your own and somewhat trusted 
> nodes is a recommend best practice for miners and other high value systems 
> which is rendered much less effective due to a lack of
> authentication, there is no significant key distribution problem in that case

This is the only legitimate scenario that I am aware of. Doing this by IP 
address (as we do) is weak if there is no VPN.

Yet this scenario is very different than general authentication. This scenario 
is a set of nodes that is essentially a single logical node from the 
perspective of the Bitcoin security model. One entity controls the validation 
rules, or is collaborating with another entity to do so.

My concern is that a general authentication requirement expands this single 
logical node and gives control over if to the entity that controls key 
distribution - the hard problem that hasn't been addressed.

If there is no such entity restricting access to the network (which hopefully 
we can assume) then there is no reason to expect any effective improvement, 
since nodes will necessarily have to connect with anonymous peers. Anyone with 
a node and the ability to monitor traffic should remain very effective.

> and I expect the future auth BIP (Jonas had one before, but it was put aside 
> for now to first focus on the link layer encryption)
> to address that case quite well.

Defining an auth implementation is not a hard problem, nor is it the concern I 
have raised.

e
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

>> On Jun 29, 2016, at 12:22 AM, Gregory Maxwell  wrote:
>> 
>> On Tue, Jun 28, 2016 at 9:59 PM, Eric Voskuil  wrote:
>> Passing the session ID out of band is authentication. As this is explicitly 
>> not part of BIP151 it cannot be that BIP151 provides the tools to detect a 
>> attack (the point at issue).
> 
> It provides the ID, the rest is meat.

The rest is "authentication".

> Users can compare session IDs
> via whatever communications channels they already use after the fact
> and discover if they were or are being MITMed.
> 
 It requires a secure channel and is authentication. So BIP151 doesn't 
 provide the tools to detect an attack, that requires authentication. A 
 general requirement for authentication is the issue I have raised.
>>> 
>>> One might wonder how you ever use a Bitcoin address, or even why we might 
>>> guess these emails from "you" aren't actually coming from the NSA.
>> 
>> The sarcasm is counterproductive Greg. By the same token I could ask how you 
>> ever use Bitcoin given that the P2P protocol is not encrypted or 
>> authenticated.
> 
> I think I was unclear. A bitcoin address needs to be sent over a secure 
> channel, which we do not provide. Yet sending funds to addresses instead of 
> anyone_can_spend is pretty useful.
> 
> Similarly, I can guess that messages claiming to you are probably from you 
> when many people can independently check, even if they don't usually. The 
> fact tampered messages might be detected is a big disincentive from trying.

You were perfectly clear. Did I give some indication that I did not understand 
what you meant?

>> The blockchain and mempool are a cache of public data. Transmission of a 
>> payment address to a payer is not a comparable scenario.
> 
> The precise timing and ordering of transactions being relayed is _not_
> public data.

Posting txs to the network is a client-server scenario. The set of txs arriving 
at an arbitrary node, including the order of arrival, is by definition public 
information. The only possible way it could be considered private is if the 
entire network was private.

So where does the private timing become public? First hop, second, third?

Encryption and authentication cannot prevent timing attacks against a person 
posting txs to the network unless the entire network is "secured". That is not 
possible without centralized access control.

Encrypting the P2P network doesn't resolve this problem, nor does 
authentication, nor does Tor. I would prefer we advance an actual solution to 
this significant problem than advance a false sense of security while creating 
both complexity and the likely evolution of node identity.

e
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

On Jun 29, 2016, at 3:01 AM, Gregory Maxwell  wrote:
> 
>> On Tue, Jun 28, 2016 at 11:33 PM, Eric Voskuil  wrote:
>> I don't follow this comment. The BIP aims quite clearly at "SPV" wallets as 
>> its justifying scenario.
> 
> It cites SPV as an example, doesn't mention bloom filters.. and sure-- sounds 
> like the bip text should make the

"MOTIVATION:
The Bitcoin network does not encrypt communication between peers today. This 
opens up security issues (eg: traffic manipulation by others) and allows for 
mass surveillance / analysis of bitcoin users. Mostly this is negligible 
because of the nature of Bitcoins trust model, however for SPV nodes this can 
have significant privacy impacts [1] and could reduce the censorship-resistance 
of a peer."

This is not an example, this is the exception that is described as 
"significant" in comparison to the other issues, which are described as 
"negligible".

The Bloom filters messages are of course the unique aspects of the protocol as 
it pertains to "SPV".

The RISKS section declares that the BIP cannot prevent MITM attacks and that 
"identity authentication" will  be defined in a forthcoming BIP.

The obvious implication (accepted by the author) is that authentication is 
required to prevent a MITM attack, and furthermore establishment of identity 
will be required to ensure that the authenticated party is not a bad actor.

>>> Without something like BIP151 network participants cannot have privacy for 
>>> the transactions they originate within the protocol against network 
>>> observers.
>> 
>> And they won't get it with BIP151 either. Being a peer is easier than 
>> observing the network.
> 
> Not passively, undetectable, and against thousands of users at once at low 
> cost.

This is a straw man, as the BIP does not state that its objective is to 
moderately raise the cost of passive attack against large numbers of users.

It is also a red herring, as passivity is not itself a benefit. It implies that 
the attack is easier and therefore less costly. But a trivial active attack may 
be a larger security problem than a complex passive attack. Attacks against 
privacy under this BIP (and with authentication) can be carried out by 
passively monitoring traffic and operating one or more nodes. Operating a node 
may be considered "active" because the node communicates, but technically it is 
not. In either case the activeness itself hardly raises the difficulty, 
especially for a global (thousands of users) passive attacker.

Depending on the attacker, cost may not be an issue at all, so raising it can 
have zero effect. Certainly we are not talking about prohibitive 
(cryptographically hard) cost. Raising the cost *any* amount is not likely a 
reasonable cost-benefit tradeoff.

Privacy attacks would remain entirely undetectable under this proposal, and 
under any additional proposal that required authentication in the absence of 
identity. Only with all users of the network identified as "good" would such 
proposals be effective. Until that point any bad actors can become an integral 
part of the network. I will investigate the question of identity in a follow-up 
to an independent post.

>> If one can observe the encrypted traffic one can certainly use a timing 
>> attack to determine what the node has sent.
> 
> Not against Bitcoin Core, transactions are batched and relayed in
> sorted order.  (obviously there are limits at what this provides;
> ironically, the lack of link encryption has been used to argue against
> privacy preserving relay behavior)

It cannot be both impossible ("not against Bitcoin Core") and limited in 
effectiveness ("obviously there are limits").

We should be clear at this point that the transaction-posting security provided 
against a privacy attack, based on the assumption of "good" (identified) peers 
in the first few hops, derives entirely from the ability of the good peers to 
break the timing attack, which is itself "limited".

This is a compound pair of weak assumptions, that to be made stronger will 
require widespread use of identity (not just authentication).

The proliferation of node identity is my primary concern - this relates to 
privacy and the security of the network. Secondarily I am concerned about users 
operating under a false assumption about the strength of privacy. Thirdly I am 
concerned about the risk of vulnerability introduced by the integration into 
the P2P network layer of an totally new network security scheme. Fourthly I'm 
concerned about the cost of the above based on the belief that the benefit may 
not be material and that it may lead to increased centralization.

>>> Even if, through some extraordinary effort, their own first hop is 
>>> encrypted, unencrypted later hops would rapidly
>>> expose significant information about transaction origins in the network.
>> 
>> As will remain the case until all connections are encrypted and 
>> authenticated, and all participants are known to be good guys. S

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

Hi Alfie,

Yes, this is exactly what I meant. The complexity of the proposed construction 
is comparable to that of Bitcoin itself. This is not itself prohibitive, but it 
is clearly worthy of consideration.

A question we should ask is whether decentralized anonymous credentials is 
applicable to the authentication problem posed by BIP151. I propose that it is 
not.

The core problem posed by BIP151 is a MITM attack. The implied solution (BIP151 
+ authentication) requires that a peer trusts that another is not an attacker. 

Authentication of an anonymous peer cannot achieve this objective, since the 
peer may be anyone and an attack on privacy can be undetectable. The identity 
of a peer must be known to the relying peer, either directly or transitively.

DAC is applicable in cases where identity is never required.  The prime example 
in the paper is that of first-come-first-served name registration. No identity 
is required in that scenario, just proof that a party in question is the 
original registrant. All participants are presumed to be "good".

I believe that a distributed anonymous system is fundamentally at odds with 
isolation of "good" vs. "bad" participants who comply with protocol rules (DoS 
considerations aside), and that any attempt to resolve this conflict will 
result in the system no longer allowing anonymous participation.

I may be mistaken, but I haven't found a way out of this realization.

e

> On Jun 29, 2016, at 1:17 PM, Alfie John  wrote:
> 
> On Tue, Jun 28, 2016 at 06:45:58PM +0200, Eric Voskuil via bitcoin-dev wrote:
>>> then we should definitively use a form of end-to-end encryption between
>>> nodes. Built into the network layer.
>> 
>> Widespread application of this model is potentially problematic. It is a
>> non-trivial problem to design a distributed system that requires 
>> authentication
>> but without identity and without central control. In fact this may be more
>> challenging than Bitcoin itself. Trust on first use (TOFU) does not solve 
>> this
>> problem.
> 
> Maybe the following paper can feed into this discussion:
> 
> "Decentralized Anonymous Credentials" by Christina Garman, Matthew Green, Ian 
> Miers
>   https://eprint.iacr.org/2013/622.pdf
> 
> Alfie
> 
> -- 
> Alfie John
> https://www.alfie.wtf
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

> On Jun 30, 2016, at 2:20 PM, Jonas Schnelli  wrote:
> 
> 
>> Yes, this is exactly what I meant. The complexity of the proposed 
>> construction is comparable to that of Bitcoin itself. This is not itself 
>> prohibitive, but it is clearly worthy of consideration.
>> 
>> A question we should ask is whether decentralized anonymous credentials is 
>> applicable to the authentication problem posed by BIP151. I propose that it 
>> is not.
>> 
>> The core problem posed by BIP151 is a MITM attack. The implied solution 
>> (BIP151 + authentication) requires that a peer trusts that another is not an 
>> attacker.
> 
> BIP151 would increase the risks for MITM attackers.
> What are the benefits for Mallory of he can't be sure Alice and Bob may
> know that he is intercepting the channel?

It is not clear to me why you believe an attack on privacy by an anonymous peer 
is detectable.

> MITM is possible today, it would still be possible (though under higher
> costs) with BIP151.
> 
> With BIP151 we would have the basic tool-set to effectively reduce the
> risks of being MITMled.
> 
> IMO we should focus on the risks and benefits of BIP151 and not drag
> this discussion into the realm of authentication. This can and should be
> done once we have proposals for authentication (and I'm sure this will
> be a heated debate).
> 
> The only valid risk I have on my list from you, Eric, is the false sense
> of security.
> 
> My countermeasure for that would be...
> - deploy BIP151 together with the simplest form of authentication
> (know_hosts / authorized_keys file, no TOFU only editable "by hand")
> - make it more clear (in the BIP151 MOTIVATION text) that it won't solve
> the privacy/MITM problem without additional authentication.
> 
> Or could you elaborate again – without stepping into the realm of
> authentication/MITM (which is not part of the BIP or possible already
> today) – why BIP151 would make things worse?
> 
> 
> 
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

Pieter, these are in my opinion very reasonable positions. I've made some 
observations inline.

> On Jun 30, 2016, at 3:03 PM, Pieter Wuille  wrote:
> 
> On Thu, Jun 30, 2016 at 11:57 AM, Eric Voskuil via bitcoin-dev
>  wrote:
>> The proliferation of node identity is my primary concern - this relates to 
>> privacy and the security of the network.
> 
> I think this is a reasonable concern.
> 
> However, node identity is already being used widely, and in a very
> inadvisable way:
> * Since forever there have been lists of 'good nodes' to pass in
> addnode= configuration options.
> * Various people run multiple nodes in different geographic locations,
> peering with each other.
> * Various pieces of infrastructure exist that relies on connecting to
> well-behaving nodes (miner relay networks, large players peering
> directly with each other, ...)

Yes, libbitcoin also provides these options on an IP basis.

> * Several lightweight clients support configuring a trusted host to connect 
> to.

I explicitly exclude client-server behavior as I believe the proper resolution 
is to isolate clients from the P2P protocol. Libbitcoin does this already.

> Perhaps you deplore that fact, but I believe it is inevitable that different 
> pieces of the network will make different choices here. You can't prevent 
> people from create connections along preexisting trust lines. That does not 
> mean that the network as a whole relies on first establishing trust 
> everywhere.

Of course, the network operates just fine without universal trust. My concern 
is not that it is required, but that it may grow significantly and will have a 
tendency to gravitate towards more effective registration mechanisms for what 
is a "good" peer. Even an informal but pervasive web of trust may make it 
difficult for untrusted parties to connect.

> And I do think there are advantages.
> 
> BIP 151 on its own gives you opportunistic encryption. You're very right to 
> point out that this does not give you protection from active attackers, and 
> that active attacking is relatively easy through sybil attacks. I still 
> prefer my attacker to actually do that over just listening in on my 
> connection.

We agree, and the ease of this attack must be acknowledged. And given that the 
protection is weak it is not unreasonable to consider the potential downside of 
creeping node identity.

> And yes, we should also work on improving the privacy nodes and wallets have 
> orthogonal to encryption, but nothing will make everything perfectly private.

I agree, and I doubt this proposal will have much impact on an advanced 
persistent threat, or even lesser threats. People should understand that there 
is both a risk and a limited benefit to this proposal.

> BIP 151 plus a simple optional pre-shared-secret authentication extension can 
> improve upon pure IP-based authentication, as well as simplify things like 
> SSL tunnels, and onion addresses purely used as identity. This will still 
> require explicit configuration, but not more than now.

I agree - I consider tunneling the legitimate use case for this proposal. Yet 
when nodes become closely coupled they are not fully independent. I have a 
concern with these practices being promoted for general use while at the same 
time being strongly implemented.

> BIP 151 plus a non-leaking public key authentication scheme (where peers can 
> query "are you the peer with pubkey X?" but don't learn anything if the 
> answer is no) with keys specific to the IP addresses can give a TOFU-like 
> security. Nodes already remember IP addresses they've succesfully interacted 
> with in the past, and ban IP addresses that misbehave. Being able to tell 
> whether a node you connect to is the same as one you've connected to before 
> is a natural extension of this,

With this I disagree. There is no way to know that a node is one you have 
connected to previously unless that node wants you to know (apart from relying 
on the IP address). This is of no value in detecting misbehaving nodes that do 
not want to be detected. Ones that don't care (eg broken nodes) can be 
sufficiently managed by IP address.

> and does not require establishing any real-world identity beyond what we're 
> already implicitly relying on.
> 
> Perhaps these use cases and their security assumptions should be spelled out 
> more clearly in the BIP.

Absolutely.

> If there is a misunderstanding, it should be clearly stated that BIP 151 is 
> only a building block for further improvements
> 
>> Secondarily I am concerned about users operating under a false assumption 
>> about the strength of privacy.
> 
> This is a widespread problem, but it exists far outside the scope of this 
> proposa

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

> On Jun 30, 2016, at 2:43 PM, Jonas Schnelli  wrote:
> 
 The core problem posed by BIP151 is a MITM attack. The implied solution 
 (BIP151 + authentication) requires that a peer trusts that another is not 
 an attacker.
>>> 
>>> BIP151 would increase the risks for MITM attackers.
>>> What are the benefits for Mallory of he can't be sure Alice and Bob may
>>> know that he is intercepting the channel?
>> 
>> It is not clear to me why you believe an attack on privacy by an anonymous 
>> peer is detectable.
> 
> If Mallory has substituted the ephemeral keys in both directions, at the
> point where Alice and Bob will do an authentication, they can be sure
> Mallory is listening.

I understand the mechanics of a tunnel between trusting parties that have a 
secure side channel. But this assumes that no other peer can connect to these 
two nodes. How then do they maintain the chain?

The "middle" in this sense does not have to be the wire directly between these 
two peers. It can be between either of them and any anonymous connection they 
(must) allow.

Of course this creates pressure to expand their tunnel. Hence the problem of 
expanding node identity in an effort to preserve privacy. The protection will 
remain weak until the entire network is "secure". At that point it would 
necessarily be a private network.

As Pieter rightly observes, there are and always will be tunnels between 
trusting nodes. Often these are groups of nodes that are in collaboration, so 
logically they are one node from a system security standpoint. But if people 
become generally reliant on good node registration, it will become the 
registrar who controls access to the network. So my concern rests I this 
proposal becoming widely adopted.

> Simple dummy example:
> 1.) Encryption setup with ECDH with ephemeral keys after BIP151
> 2.) Mallory is MITMling the connection. He is substituting both direction 
> with its own keys
> 3.) Connection is successfully MITMled
> 4.) Alice tells Bob "prove me that you are Bob, please sign the session-ID 
> with your identity key"
> 5.) Bob signs the sessionID (ECDH secret) with his identity key which
> will be unusable for Mallory who has a substituted sessionID in both 
> directions.
> 6.) Alice has successfully detected the Mallory
> 
> Disclaimer: 4) and 5) are _not_ authentication proposals :-)
> 
> 
> 
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

> On Jun 30, 2016, at 6:52 PM, Peter Todd  wrote:
> 
>> On Thu, Jun 30, 2016 at 05:22:08PM +0200, Eric Voskuil via bitcoin-dev wrote:
>> 
>>> On Jun 30, 2016, at 2:43 PM, Jonas Schnelli  wrote:
>>> 
>>>>>> The core problem posed by BIP151 is a MITM attack. The implied solution 
>>>>>> (BIP151 + authentication) requires that a peer trusts that another is 
>>>>>> not an attacker.
>>>>> 
>>>>> BIP151 would increase the risks for MITM attackers.
>>>>> What are the benefits for Mallory of he can't be sure Alice and Bob may
>>>>> know that he is intercepting the channel?
>>>> 
>>>> It is not clear to me why you believe an attack on privacy by an anonymous 
>>>> peer is detectable.
>>> 
>>> If Mallory has substituted the ephemeral keys in both directions, at the
>>> point where Alice and Bob will do an authentication, they can be sure
>>> Mallory is listening.
>> 
>> I understand the mechanics of a tunnel between trusting parties that have a 
>> secure side channel. But this assumes that no other peer can connect to 
>> these two nodes. How then do they maintain the chain?
>> 
>> The "middle" in this sense does not have to be the wire directly between 
>> these two peers. It can be between either of them and any anonymous 
>> connection they (must) allow.
>> 
>> Of course this creates pressure to expand their tunnel. Hence the problem of 
>> expanding node identity in an effort to preserve privacy. The protection 
>> will remain weak until the entire network is "secure". At that point it 
>> would necessarily be a private network.
>> 
>> As Pieter rightly observes, there are and always will be tunnels between 
>> trusting nodes. Often these are groups of nodes that are in collaboration, 
>> so logically they are one node from a system security standpoint. But if 
>> people become generally reliant on good node registration, it will become 
>> the registrar who controls access to the network. So my concern rests I this 
>> proposal becoming widely adopted.
> 
> To be clear, are you against Bitcoin Core's tor support?
> 
> Because node-to-node connections over tor are encrypted, and make use of onion
> addresses, which are self-authenticated in the exact same way as BIP151 
> proposes.

BIP151 is self-admittedly insufficient to protect against a MITM attack. It 
proposes node identity to close this hole (future BIP required). The 
yet-to-be-specified requirement for node identity is the basis of my primary 
concern. This is not self-authentication.

> And we're shipping that in production as of 0.12.0, and by default Tor onion 
> support is enabled and will be automatically setup if you have a recent 
> version of Tor installed.
> 
> Does that "create pressure to expand node identity"?

The orthogonal question of whether Tor is safe for use with the Bitcoin P2P 
protocol is a matter of existing research.

e
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 151

[Eric Voskuil via bitcoin-dev]

> On Jun 30, 2016, at 9:06 PM, Peter Todd  wrote:
> 
> On Thu, Jun 30, 2016 at 08:25:45PM +0200, Eric Voskuil wrote:
>>> To be clear, are you against Bitcoin Core's tor support?
>>> 
>>> Because node-to-node connections over tor are encrypted, and make use of 
>>> onion
>>> addresses, which are self-authenticated in the exact same way as BIP151 
>>> proposes.
>> 
>> BIP151 is self-admittedly insufficient to protect against a MITM attack. It 
>> proposes node identity to close this hole (future BIP required). The 
>> yet-to-be-specified requirement for node identity is the basis of my primary 
>> concern. This is not self-authentication.
>> 
>>> And we're shipping that in production as of 0.12.0, and by default Tor 
>>> onion support is enabled and will be automatically setup if you have a 
>>> recent version of Tor installed.
>>> 
>>> Does that "create pressure to expand node identity"?
>> 
>> The orthogonal question of whether Tor is safe for use with the Bitcoin P2P 
>> protocol is a matter of existing research.
> 
> I don't think you answered my question.
> 
> Again, we _already have_ the equivalent of BIP151 functionality in Bitcoin
> Core, shipping in production, but implemented with a Tor dependency.
> 
> BIP151 removes that dependency on Tor, enabling encrypted connections
> regardless of whether or not you have Tor installed.
> 
> So any arguments against BIP151 being implemented, are equally arguments
> against our existing Tor onion support. Are you against that support? Because
> if you aren't, you can't have any objections to BIP151 being implemented

Neither Tor nor Bitcoin Core are part of this BIP (or its proposed dependency 
on node identity).

But again, given that node identity is not part of the Bitcoin Core Tor 
integration, my objection to the presumption of node identity by BIP151 is 
unrelated to Bitcoin Core's Tor integration.

e
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Completing the retirement of the alert system

[Eric Voskuil via bitcoin-dev]

ACK

libbitcoin defines the message and includes the public key but only for 
completeness and reference purposes. It has never been used in the node.

e

> On Sep 9, 2016, at 5:42 PM, Gregory Maxwell via bitcoin-dev 
>  wrote:
> 
> The alert system was a centralized facility to allow trusted parties
> to send messages to be displayed in wallet software (and, very early
> on, actually remotely trigger the software to stop transacting).
> 
> It has been removed completely in Bitcoin Core after being disabled for a 
> while.
> 
> While the system had some potential uses, there were a number of
> problems with it.
> 
> The alert system was a frequent source of misunderstanding about the
> security model and 'effective governance', for example a years ago a
> BitcoinJ developer wanted it to be used to control fee levels on the
> network and few months back one of Bloq's staff was pushing for a
> scheme where "the developers" would use it to remotely change the
> difficulty-- apparently with no idea how abhorrent others would find
> it.
> 
> The system also had a problem of not being scalable to different
> software vendors-- it didn't really make sense that core would have
> that facility but armory had to do something different (nor would it
> really make sense to constantly have to maintain some list of keys in
> the node software).
> 
> It also had the problem of being unaccountable. No one can tell which
> of the key holders created a message. This creates a risk of misuse
> with a false origin to attack someone's reputation.
> 
> Finally, there is good reason to believe that the key has been
> compromised-- It was provided to MTGox by a developer and MTGox's
> systems' were compromised and later their CEO's equipment taken by the
> Japanese police.
> 
> In any case, it's gone now in Core and most other current software--
> and I think it's time to fully deactivate it.
> 
> I've spent some time going around the internet looking for all
> software that contains this key (which included a few altcoins) and
> asked them to remove it. I will continue to do that.
> 
> One of the facilities in the alert system is that you can send a
> maximum sequence alert which cannot be overridden and displays only a
> static key compromise text message and blocks all other alerts. I plan
> to send a triggering alert in the not-distant future (exact time to be
> announced well in advance) feedback on timing would be welcome.
> 
> There are likely a few production systems that automatically shut down
> when there is an alert, so this risks some small one-time disruption
> of those services-- but none worse than if an alert were sent to
> advise about a new system upgrade.
> 
> At some point after that, I would then plan to disclose this private
> key in public, eliminating any further potential of reputation attacks
> and diminishing the risk of misunderstanding the key as some special
> trusted source of authority.
> 
> Cheers,
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Start time for BIP141 (segwit)

[Eric Voskuil via bitcoin-dev]

If somebody is not "running their own validation code"  then they aren't 
actually using Bitcoin, so their ease in transition is irrelevant. For all they 
know they are accepting random numbers.

e

> On Oct 16, 2016, at 9:35 AM, Gavin Andresen via bitcoin-dev 
>  wrote:
> 
>> On Sun, Oct 16, 2016 at 10:58 AM, Tom Zander via bitcoin-dev 
>>  wrote:
>> The fallow period sounds wy to short. I suggest 2 months at minimum
>> since anyone that wants to be safe needs to upgrade.
> 
> I asked a lot of businesses and individuals how long it would take them to 
> upgrade to a new release over the last year or two.
> 
> Nobody said it would take them more than two weeks.
> 
> If somebody is running their own validation code... then we should assume 
> they're sophisticated enough to figure out how to mitigate any risks 
> associated with segwit activation on their own.
> 
> -- 
> --
> Gavin Andresen
> 
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

NACK

Horrible precedent (hardcoding rule changes based on the assumption that large 
forks indicate a catastrophic failure), extremely poor process (already 
shipped, now the discussion), and not even a material performance optimization 
(the checks are avoidable once activated until a sufficiently deep reorg 
deactivates them).

e

> On Nov 14, 2016, at 10:17 AM, Suhas Daftuar via bitcoin-dev 
>  wrote:
> 
> Hi,
> 
> Recently Bitcoin Core merged a simplification to the consensus rules 
> surrounding deployment of BIPs 34, 66, and 65 
> (https://github.com/bitcoin/bitcoin/pull/8391), and though the change is a 
> minor one, I thought it was worth documenting the rationale in a BIP for 
> posterity.
> 
> Here's the abstract:
> 
> Prior soft forks (BIP 34, BIP 65, and BIP 66) were activated via miner 
> signaling in block version numbers. Now that the chain has long since passed 
> the blocks at which those consensus rules have triggered, we can (as a 
> simplification and optimization) replace the trigger mechanism by caching the 
> block heights at which those consensus rules became enforced.
> 
> The full draft can be found here: 
> 
> https://github.com/sdaftuar/bips/blob/buried-deployments/bip-buried-deployments.mediawiki
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

Actually this does nothing to provide justification for this consensus rule 
change. It is just an attempt to deflect criticism from the fact that it is 
such a change.

e

> On Nov 15, 2016, at 9:45 AM, Btc Drak  wrote:
> 
> I think this is already covered in the BIP text:-
> 
> "As of November 2016, the most recent of these changes (BIP 65,
> enforced since December 2015) has nearly 50,000 blocks built on top of
> it. The occurrence of such a reorg that would cause the activating
> block to be disconnected would raise fundamental concerns about the
> security assumptions of Bitcoin, a far bigger issue than any
> non-backwards compatible change.
> 
> So while this proposal could theoretically result in a consensus
> split, it is extremely unlikely, and in particular any such
> circumstances would be sufficiently damaging to the Bitcoin network to
> dwarf any concerns about the effects of this proposed change."
> 
> 
> On Mon, Nov 14, 2016 at 6:47 PM, Eric Voskuil via bitcoin-dev
>  wrote:
>> NACK
>> 
>> Horrible precedent (hardcoding rule changes based on the assumption that
>> large forks indicate a catastrophic failure), extremely poor process
>> (already shipped, now the discussion), and not even a material performance
>> optimization (the checks are avoidable once activated until a sufficiently
>> deep reorg deactivates them).
>> 
>> e
>> 
>> On Nov 14, 2016, at 10:17 AM, Suhas Daftuar via bitcoin-dev
>>  wrote:
>> 
>> Hi,
>> 
>> Recently Bitcoin Core merged a simplification to the consensus rules
>> surrounding deployment of BIPs 34, 66, and 65
>> (https://github.com/bitcoin/bitcoin/pull/8391), and though the change is a
>> minor one, I thought it was worth documenting the rationale in a BIP for
>> posterity.
>> 
>> Here's the abstract:
>> 
>> Prior soft forks (BIP 34, BIP 65, and BIP 66) were activated via miner
>> signaling in block version numbers. Now that the chain has long since passed
>> the blocks at which those consensus rules have triggered, we can (as a
>> simplification and optimization) replace the trigger mechanism by caching
>> the block heights at which those consensus rules became enforced.
>> 
>> The full draft can be found here:
>> 
>> https://github.com/sdaftuar/bips/blob/buried-deployments/bip-buried-deployments.mediawiki
>> 
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> 
>> 
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> 
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

This sort of statement represents one consequence of the aforementioned bad 
precedent.

Are checkpoints good now? Are hard forks okay now?

What is the maximum depth of a reorg allowed by this non-machine consensus?

Shouldn't we just define a max depth so that all cruft deeper than that can 
just be discarded on a regular basis?

Why are there activation heights defined by this hard fork if it's not possible 
to reorg back to them?

The "BIP" is neither a Proposal (it's been decided, just documenting for 
posterity), nor an Improvement (there is no actual benefit, just some tidying 
up in the notoriously obtuse satoshi code base), nor Bitcoin (a hard fork 
defines an alt coin, so from Aug 4 forward it has been CoreCoin).

e

> On Nov 16, 2016, at 5:29 AM, Jameson Lopp  wrote:
> 
> Since "buried deployments" are specifically in reference to historical 
> consensus changes, I think the question is more one of human consensus than 
> machine consensus. Is there any disagreement amongst Bitcoin users that BIP34 
> activated at block 227931, BIP65 activated at block 388381, and BIP66 
> activated at block 363725? Somehow I doubt it.
> 
> It seems to me that this change is merely cementing into place a few 
> attributes of the blockchain's history that are not in dispute.
> 
> - Jameson
> 
>> On Tue, Nov 15, 2016 at 5:42 PM, Eric Voskuil via bitcoin-dev 
>>  wrote:
>> Actually this does nothing to provide justification for this consensus rule 
>> change. It is just an attempt to deflect criticism from the fact that it is 
>> such a change.
>> 
>> e
>> 
>> > On Nov 15, 2016, at 9:45 AM, Btc Drak  wrote:
>> >
>> > I think this is already covered in the BIP text:-
>> >
>> > "As of November 2016, the most recent of these changes (BIP 65,
>> > enforced since December 2015) has nearly 50,000 blocks built on top of
>> > it. The occurrence of such a reorg that would cause the activating
>> > block to be disconnected would raise fundamental concerns about the
>> > security assumptions of Bitcoin, a far bigger issue than any
>> > non-backwards compatible change.
>> >
>> > So while this proposal could theoretically result in a consensus
>> > split, it is extremely unlikely, and in particular any such
>> > circumstances would be sufficiently damaging to the Bitcoin network to
>> > dwarf any concerns about the effects of this proposed change."
>> >
>> >
>> > On Mon, Nov 14, 2016 at 6:47 PM, Eric Voskuil via bitcoin-dev
>> >  wrote:
>> >> NACK
>> >>
>> >> Horrible precedent (hardcoding rule changes based on the assumption that
>> >> large forks indicate a catastrophic failure), extremely poor process
>> >> (already shipped, now the discussion), and not even a material performance
>> >> optimization (the checks are avoidable once activated until a sufficiently
>> >> deep reorg deactivates them).
>> >>
>> >> e
>> >>
>> >> On Nov 14, 2016, at 10:17 AM, Suhas Daftuar via bitcoin-dev
>> >>  wrote:
>> >>
>> >> Hi,
>> >>
>> >> Recently Bitcoin Core merged a simplification to the consensus rules
>> >> surrounding deployment of BIPs 34, 66, and 65
>> >> (https://github.com/bitcoin/bitcoin/pull/8391), and though the change is a
>> >> minor one, I thought it was worth documenting the rationale in a BIP for
>> >> posterity.
>> >>
>> >> Here's the abstract:
>> >>
>> >> Prior soft forks (BIP 34, BIP 65, and BIP 66) were activated via miner
>> >> signaling in block version numbers. Now that the chain has long since 
>> >> passed
>> >> the blocks at which those consensus rules have triggered, we can (as a
>> >> simplification and optimization) replace the trigger mechanism by caching
>> >> the block heights at which those consensus rules became enforced.
>> >>
>> >> The full draft can be found here:
>> >>
>> >> https://github.com/sdaftuar/bips/blob/buried-deployments/bip-buried-deployments.mediawiki
>> >>
>> >> ___
>> >> bitcoin-dev mailing list
>> >> bitcoin-dev@lists.linuxfoundation.org
>> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> >>
>> >>
>> >> ___
>> >> bitcoin-dev mailing list
>> >> bitcoin-dev@lists.linuxfoundation.org
>> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> >>
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

I would suggest that, before discussing how best to fork the chain to meet this 
objective, we consider the objective.

The implementers have acknowledged that this does not represent a performance 
improvement. Especially given that this was apparently not initially 
understood, that alone is good reason for them to reconsider.

The remaining stated objective is reduction of code complexity. Let us be very 
clear, a proposal to change the protocol must be considered independently of 
any particular implementation of the protocol. While the implementation of 
BIP34 style activation may be hugely complex in the satoshi code, it is 
definitely not complex as a matter of necessity.

Activation constitutes maybe a dozen lines of additional code in libbitcoin. 
The need to hit the chain (or cache) to obtain historical header info will 
remain for proof of work, so this change doesn't even accomplish some sort of 
beneficial isolation from blockchain history.

So, at best, we are talking about various ways to introduce a consensus fork so 
that a well designed implementation  can remove a tiny amount of 
already-written code and associated tests. In my opinion this is embarrassingly 
poor reasoning. It would be much more productive to reduce satoshi code 
complexity in ways that do not impact the protocol. There are a *huge* number 
of such opportunities, and in fact activation is one of them. Once that is 
done, we can talk about forking to reduce source code complexity.

These fork suggestions actually increase *necessary* complexity for any 
implantation that takes a rational approach to forks. By rational I mean 
*additive*. Deleting rules from Bitcoin code is simply bad design. Rules are 
never removed, they are added. A new rule to modify an old rule is simply a new 
rule. This is new and additional code. So please don't assume in this 
"proposal" that this makes development simpler for other implementations, that 
is not a necessary conclusion.

e

> On Nov 16, 2016, at 1:01 PM, Peter Todd via bitcoin-dev 
>  wrote:
> 
>> On Wed, Nov 16, 2016 at 09:32:24AM -0500, Alex Morcos via bitcoin-dev wrote:
>> I think we are misunderstanding the effect of this change.
>> It's still "OK" for a 50k re-org to happen.
>> We're just saying that if it does, we will now have potentially introduced
>> a hard fork between new client and old clients if the reorg contains
>> earlier signaling for the most recent ISM soft fork and then blocks which
>> do not conform to that soft fork before the block height encoded activation.
>> 
>> I think the argument is this doesn't substantially add to the confusion or
>> usability of the system as its likely that old software won't even handle
>> 50k block reorgs cleanly anyway and there will clearly have to be human
>> coordination at the time of the event.  In the unlikely event that the new
>> chain does cause such a hard fork, that coordination can result in everyone
>> upgrading to software that supports the new rules anyway.
>> 
>> So no, I don't think we should add a checkpoint.  I think we should all
>> just agree to a hard fork that only has a very very slim chance of any
>> practical effect.
> 
> So, conceptually, another way to deal with this is to hardcode a blockhash
> where we allow blocks in a chain ending with that blockhash to _not_ follow
> BIP65, up until that blockhash, and any blockchain without that blockhash must
> respect BIP65 for all blocks in the chain.
> 
> This is a softfork: we've only added rules that made otherwise valid chains
> invalid, and at the same time we are still accepting large reorgs (albeit 
> under
> stricter rules than before).
> 
> I'd suggest we call this a exemption hash - we've exempted a particular
> blockchains from a soft-forked rule that we would otherwise enforce.
> 
> -- 
> https://petertodd.org 'peter'[:-1]@petertodd.org
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

On 11/16/2016 03:58 PM, Jorge Timón via bitcoin-dev wrote:
> On Wed, Nov 16, 2016 at 3:18 PM, Thomas Kerin via bitcoin-dev
>  wrote:
>> BIP30 actually was given similar treatment after a reasonable amount of time
>> had passed.
>> https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L2392
> 
> This is not really the same. BIP30 is not validated after BIP34 is
> active because blocks complying with BIP34 will always necessarily
> comply with BIP30 (ie coinbases cannot be duplicated after they
> include the block height).

This is a misinterpretation of BIP30. Duplicate transaction hashes can
and will happen and are perfectly valid in Bitcoin. BIP34 does not
prevent this.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

No, BIP30 prevents duplicate tx hashes in the case where the new tx hash
duplicates that of a preceding tx with unspent outputs.

There was one such case that had already become buried in the chain at
the time, so it was exempted from validation. There was another case of
a duplicate hash, but it's predecessor was spent so it complied with the
new rule.

Both of these cases resulted from exact duplicate txs, which BIP34 now
precludes. However nothing precludes different txs from having the same
hash.

e

On 11/16/2016 04:06 PM, Jorge Timón wrote:
> On Thu, Nov 17, 2016 at 1:00 AM, Eric Voskuil  wrote:
>> This is a misinterpretation of BIP30. Duplicate transaction hashes can
>> and will happen and are perfectly valid in Bitcoin. BIP34 does not
>> prevent this.
> 
> Sorry for moving the topic, but isn't duplication of tx hashes
> precisely what BIP30 prevents?
> That was my undesrtanding but should read it again.
> Since regular txs take inputs, the collision is extremely unlikely
> (again, this is my understanding, please correct me when wrong), the
> worrying case is coinbase txs (which don't have input to take entropy
> from). By introducing the committed height, collisions on coinbase txs
> are prevented too.
> 
> If I'm wrong on any of this I'm more than happy to learn why.
> 



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

On 11/16/2016 06:18 AM, Thomas Kerin wrote:
> BIP30 actually was given similar treatment after a reasonable amount of
> time had passed.
> https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L2392

BIP30 was the resolution to a catostrophic protocol flaw that would
impact any block whether above or below the point where the rule was
applied. Applying it to all future blocks, regardless of whether there
is a reorg back to genesis, was the only option as far as I can tell. So
the comparison to an unnecessary fork is hardly apt.

> You are also missing BIP50: 'March 2013 Chain For Post-Mortem', which
> neither benefited nor improved bitcoin, but did document an event for
> posterity.

BIP50 documents the release of an "unexpected" hard fork to a large
number of users. Given that Core code is considered by some to be the
*definition* of the true protocol, this led to two "legitimate" Bitcoin
chains. Leveraging the centralized state of Bitcoin mining, the
development team was able to kill the newer chain. This was simply an
altcoin that didn't survive because people stopped using it.

Anyone can create an altcoin - the question here is specifically, why
would we want to do so in this case.

> This is not a hard fork. Removing ISM just means we've committed to
> those soft-forks only locking into the chain we use now.

There didn't seem to be any confusion among the implementers that it is
a hard fork.

I will correct one implication I made below. The heights in the proposal
are required in the absence of BIP34-style activation so that the soft
fork validation rules can be properly enforced at those points (whether
or not a deep reorg happens).

e

> On 11/16/2016 01:58 PM, Eric Voskuil via bitcoin-dev wrote:
>> This sort of statement represents one consequence of the
>> aforementioned bad precedent.
>>
>> Are checkpoints good now? Are hard forks okay now?
>>
>> What is the maximum depth of a reorg allowed by this non-machine
>> consensus?
>>
>> Shouldn't we just define a max depth so that all cruft deeper than
>> that can just be discarded on a regular basis?
>>
>> Why are there activation heights defined by this hard fork if it's not
>> possible to reorg back to them?
>>
>> The "BIP" is neither a Proposal (it's been decided, just documenting
>> for posterity), nor an Improvement (there is no actual benefit, just
>> some tidying up in the notoriously obtuse satoshi code base), nor
>> Bitcoin (a hard fork defines an alt coin, so from Aug 4 forward it has
>> been CoreCoin).
>>
>> e
>>
>> On Nov 16, 2016, at 5:29 AM, Jameson Lopp > <mailto:jameson.l...@gmail.com>> wrote:
>>
>>> Since "buried deployments" are specifically in reference to
>>> historical consensus changes, I think the question is more one of
>>> human consensus than machine consensus. Is there any disagreement
>>> amongst Bitcoin users that BIP34 activated at block 227931, BIP65
>>> activated at block 388381, and BIP66 activated at block 363725?
>>> Somehow I doubt it.
>>>
>>> It seems to me that this change is merely cementing into place a few
>>> attributes of the blockchain's history that are not in dispute.
>>>
>>> - Jameson
>>>
>>> On Tue, Nov 15, 2016 at 5:42 PM, Eric Voskuil via bitcoin-dev
>>> >> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>>>
>>> Actually this does nothing to provide justification for this
>>> consensus rule change. It is just an attempt to deflect criticism
>>> from the fact that it is such a change.
>>>
>>> e
>>>
>>> > On Nov 15, 2016, at 9:45 AM, Btc Drak >> <mailto:btcd...@gmail.com>> wrote:
>>> >
>>> > I think this is already covered in the BIP text:-
>>> >
>>> > "As of November 2016, the most recent of these changes (BIP 65,
>>> > enforced since December 2015) has nearly 50,000 blocks built on
>>> top of
>>> > it. The occurrence of such a reorg that would cause the activating
>>>     > block to be disconnected would raise fundamental concerns about the
>>> > security assumptions of Bitcoin, a far bigger issue than any
>>> > non-backwards compatible change.
>>> >
>>> > So while this proposal could theoretically result in a consensus
>>> > split, it is extremely unlikely, and in particular any such
>>> > circumstances would be sufficiently damaging to the Bitcoin
>

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

> This means that all future transactions will have different txids...
rules do guarantee it.

No, it means that the chance is small, there is a difference.

If there is an address collision, someone may lose some money. If there
is a tx hash collision, and implementations handle this differently, it
will produce a chain split. As such this is not something that a node
can just dismiss. If they do they are implementing a hard fork.

e

On 11/16/2016 04:31 PM, Tier Nolan via bitcoin-dev wrote:
> 
> 
> On Thu, Nov 17, 2016 at 12:10 AM, Eric Voskuil via bitcoin-dev
>  <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
> 
> Both of these cases resulted from exact duplicate txs, which BIP34 now
> precludes. However nothing precludes different txs from having the same
> hash.
> 
> 
> The only way to have two transactions have the same txid is if their
> parents are identical, since the txids of the parents are included in a
> transaction.
> 
> Coinbases have no parents, so it used to be possible for two of them to
> be identical.
> 
> Duplicate outputs weren't possible in the database, so the later
> coinbase transaction effectively overwrote the earlier one.
> 
> The happened for two coinbases.  That is what the exceptions are for.
> 
> Neither of the those coinbases were spent before the overwrite
> happened.  I don't even think those coinbases were spent at all.
> 
> This means that every activate coinbase transaction has a unique hash
> and all new coinbases will be unique.
> 
> This means that all future transactions will have different txids.
> 
> There might not be an explicit rule that says that txids have to be
> unique, but barring a break of the hash function, they rules do
> guarantee it.



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

Also, it's important to take note of the motivation behind not banning
duplicate tx hashes outright. Doing so would require that spent tx
hashes are retained forever. A pruning node will have no way of knowing
whether a new tx duplicates the hash of a preceding tx. Any
implementation that does retain such hashes and dismisses new txs on
that basis would fork against pruning nodes.

e

On 11/16/2016 04:43 PM, Eric Voskuil wrote:
>> This means that all future transactions will have different txids...
> rules do guarantee it.
> 
> No, it means that the chance is small, there is a difference.
> 
> If there is an address collision, someone may lose some money. If there
> is a tx hash collision, and implementations handle this differently, it
> will produce a chain split. As such this is not something that a node
> can just dismiss. If they do they are implementing a hard fork.
> 
> e
> 
> On 11/16/2016 04:31 PM, Tier Nolan via bitcoin-dev wrote:
>>
>>
>> On Thu, Nov 17, 2016 at 12:10 AM, Eric Voskuil via bitcoin-dev
>> > <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>>
>> Both of these cases resulted from exact duplicate txs, which BIP34 now
>> precludes. However nothing precludes different txs from having the same
>> hash.
>>
>>
>> The only way to have two transactions have the same txid is if their
>> parents are identical, since the txids of the parents are included in a
>> transaction.
>>
>> Coinbases have no parents, so it used to be possible for two of them to
>> be identical.
>>
>> Duplicate outputs weren't possible in the database, so the later
>> coinbase transaction effectively overwrote the earlier one.
>>
>> The happened for two coinbases.  That is what the exceptions are for.
>>
>> Neither of the those coinbases were spent before the overwrite
>> happened.  I don't even think those coinbases were spent at all.
>>
>> This means that every activate coinbase transaction has a unique hash
>> and all new coinbases will be unique.
>>
>> This means that all future transactions will have different txids.
>>
>> There might not be an explicit rule that says that txids have to be
>> unique, but barring a break of the hash function, they rules do
>> guarantee it.
> 



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

On 11/16/2016 05:24 PM, Alex Morcos wrote:
> huh?
> can you give an example of how a duplicate transaction hash (in the same
> chain) can happen given BIP34?

"The pigeonhole principle arises in computer science. For example,
collisions are inevitable in a hash table because the number of possible
keys exceeds the number of indices in the array. A hashing algorithm, no
matter how clever, cannot avoid these collisions."

https://en.wikipedia.org/wiki/Pigeonhole_principle

e

> On Wed, Nov 16, 2016 at 7:00 PM, Eric Voskuil via bitcoin-dev wrote:
> 
> On 11/16/2016 03:58 PM, Jorge Timón via bitcoin-dev wrote:
> > On Wed, Nov 16, 2016 at 3:18 PM, Thomas Kerin via bitcoin-dev
> >  <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
> >> BIP30 actually was given similar treatment after a reasonable amount 
> of time
> >> had passed.
> >> https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L2392
> <https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L2392>
> >
> > This is not really the same. BIP30 is not validated after BIP34 is
> > active because blocks complying with BIP34 will always necessarily
> > comply with BIP30 (ie coinbases cannot be duplicated after they
> > include the block height).
> 
> This is a misinterpretation of BIP30. Duplicate transaction hashes can
> and will happen and are perfectly valid in Bitcoin. BIP34 does not
> prevent this.
> 
> e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

On 11/16/2016 05:50 PM, Pieter Wuille wrote:

> If you were trying to point out that buried softforks are similar to
> checkpoints in this regard, I agree.

Yes, that was my point.

> So are checkpoints good now?
> I believe we should get rid of checkpoints because they seem to be
misunderstood as a security feature rather than as an optimization.

Or maybe because they place control of the "true chain" in the hands of
those selecting the checkpoints? It's not a great leap for the parties
distributing the checkpoints to become the central authority.

I recommend users of our node validate the full chain without
checkpoints and from that chain select their own checkpoints and place
them into config. From that point forward they can apply the
optimization. Checkpoints should never be hardcoded into the source.

> I don't think buried softforks have that problem.

I find "buried softfork" a curious name as you are using it. You seem to
be implying that this type of change is itself a softfork as opposed to
a hardfork that changes the activation of a softfork. It was my
understanding that the term referred to the 3 softforks that were being
"buried", or the proposal, but not the burial itself.

Nevertheless, this proposal shouldn't have "that problem" because it is
clearly neither a security feature nor an optimization. That is the
first issue that needs to be addressed.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

On 11/17/2016 12:44 AM, Peter Todd wrote:
> On Wed, Nov 16, 2016 at 04:43:08PM -0800, Eric Voskuil via bitcoin-dev wrote:
>>> This means that all future transactions will have different txids...
>> rules do guarantee it.
>>
>> No, it means that the chance is small, there is a difference.
>>
>> If there is an address collision, someone may lose some money. If there
>> is a tx hash collision, and implementations handle this differently, it
>> will produce a chain split. As such this is not something that a node
>> can just dismiss. If they do they are implementing a hard fork.
> 
> If there is a tx hash collision it is almost certainly going to be because
> SHA256 has become weak..

Almost certainly is not certainly. Hash collisions happen because of chance.

BIP30 is quite explicit:

> "Fully-spent transactions are allowed to be duplicated in order not to
hinder pruning at some point in the future. Not allowing any transaction
to be duplicated would require evidence to be kept for each transaction
ever made."

BIP34 motivations:

> "2. Enforce block and transaction uniqueness, and assist unconnected
block validation."

But it only specifies making collisions harder, not impossible (i.e.
explicitly rejected by consensus).

Are you proposing that we draft a new BIP that allows us all to not have
to code for this? If we do so it will be impossible to guard against a
chain split due to pruning, but you seem to think that's unimportant.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [BIP Proposal] Buried Deployments

[Eric Voskuil via bitcoin-dev]

On 11/16/2016 06:47 PM, Pieter Wuille wrote:
> On Wed, Nov 16, 2016 at 6:16 PM, Eric Voskuil  > wrote:
> 
> On 11/16/2016 05:50 PM, Pieter Wuille wrote:
> 
> > So are checkpoints good now?
> > I believe we should get rid of checkpoints because they seem to be
> misunderstood as a security feature rather than as an optimization.
> 
> Or maybe because they place control of the "true chain" in the hands of
> those selecting the checkpoints? It's not a great leap for the parties
> distributing the checkpoints to become the central authority.
> 
> Yes, they can be used to control the "true chain", and this has happened
> with various forks. But developers inevitably have this possibility, if
> you ignore review. If review is good enough to catch unintended
> consensus changes, it is certainly enough to catch the introduction of
> an invalid checkpoint. The risk you point out is real, but the way to
> deal with it is good review and release practices.
> 
> I wish we had never used checkpoints the way we did, but here we are.
> Because of this, I want to get rid of them. However, It's not because I
> think they offer an excessive power to developers - but because they're
> often perceived this way (partially as a result of how they've been
> abused in other systems).
>  
> I recommend users of our node validate the full chain without
> checkpoints and from that chain select their own checkpoints and place
> them into config. From that point forward they can apply the
> optimization. Checkpoints should never be hardcoded into the source.
> 
> Having users with the discipline you suggest would be wonderful to have.
> I don't think it's very realistic, though, and I fear that the result
> would be that everyone copies their config from one or a few websites
> "because that's what everyone uses".

Certainly, but embedding them in the code makes that a practical
certainty. People cannot be prevented from doing dumb things, but let's
not make it hard for them to be smart.

> > I don't think buried softforks have that problem.
> 
> I find "buried softfork" a curious name as you are using it. You seem to
> be implying that this type of change is itself a softfork as opposed to
> a hardfork that changes the activation of a softfork. It was my
> understanding that the term referred to the 3 softforks that were being
> "buried", or the proposal, but not the burial itself.


> I do not consider the practice of "buried softforks" to be a fork at
> all. It is a change that modifies the validity of a theoretically
> construable chain from invalid to valid.

I was out at a Bitcoin meetup when I read this and I think beer actually
came out of my nose.

> However, a reorganization to
> that theoretical chain itself is likely already impossible due to the
> vast number of blocks to rewind, and economic damage that is far greater
> than chain divergence itself.

It's either possible or it is not. If it is not there is no reason for a
proposal - just make the change and don't bother to tell anyone. The
reason we are having this discussion is because it is not impossible.

> Nevertheless, this proposal shouldn't have "that problem" because it is
> clearly neither a security feature nor an optimization. That is the
> first issue that needs to be addressed.
> 
> It is clearly not a security feature, agreed. But how would you propose
> to avoid the ISM checks for BIP34 and BIP66 all the time?

I'll call straw man on the question. It is not important to avoid the
activation checks. The question is whether there is a material
performance optimization in eliminating them. This would have to be
significant enough to rise to the level of a change to the protocol.
Having said that there are a few options:

1. The naive approach to activation is, for each new block, to query the
store for the previous 1000 block headers (to the extent there are that
many), and just do so forever, summing up after the query. This is the
most straightforward but also the most costly approach.

2. A slightly less costly approach is, for each new block, to reverse
iterate over the store until all decisions can be made. This would be an
improvement below activation in that it would take it takes as little as
251 vs. 1000 queries to make the determinations.

3. A further improvement is available by caching the height of full
activation of all three soft forks. Unless there is a subsequent reorg
with a fork point prior that height, there is never a need to make
another query. Once fully activated the activation height is cached to
the store (otherwise just query the last 1000 versions at startup to
determine the state), eliminating any ongoing material cost.

4. We may also be interested in optimizing initial block download. A
cache of the last 1000 block versions can be maintained by adding each
to a circular buffer as they are committed. This eliminates *all*
que

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

On 11/17/2016 02:22 AM, Tier Nolan via bitcoin-dev wrote:
> On Thu, Nov 17, 2016 at 12:43 AM, Eric Voskuil  > wrote:
> 
> > This means that all future transactions will have different txids...
> rules do guarantee it.
> 
> No, it means that the chance is small, there is a difference.
> 
> I think we are mostly in agreement then?  It is just terminology.

Sure, if you accept that mostly is not fully - just as unlikely is not
impossible.

> In terms of discussing the BIP, barring a hash collision, it does make
> duplicate txids impossible.

That's like saying, as long as we exclude car accidents from
consideration, car accidents are impossible.

> Given that a hash collision is so unlikely, the qualifier should be
> added to those making claims that require hash collisions rather than
> those who assume that they aren't possible.
> 
> You could have said "However nothing precludes different txs from having
> the same hash, but it requires a hash collision".

I generally try to avoid speaking in tautologies :)

> Thinking about it, a re-org to before the enforcement height could allow
> it.  The checkpoints protect against that though.
>  
> As such this is not something that a node
> can just dismiss. 
> 
> The security of many parts of the system is based on hash collisions not
> being possible.

This is not the case.

Block hash duplicates within the same chain are invalid as a matter of
consensus, which is the opposite of assuming impossibility.

Tx hash collisions are explicitly allowed in the case that preceding tx
with the same hash is unspent. This is also not a reliance on the
impossibility of hash collision. Core certainly implements this distinction:

https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L2419-L2426

Address hashes and script hashes can collide without harming the
security of Bitcoin (although address owner(s) may experience harm).
Rare in this case is sufficient because of this distinction.

Compact blocks contemplates hash collisions:

https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki#Random_collision_probabilty

Checkpoints aren't part of Bitcoin security, so even the remote
possibility of two different potential blocks, with the same hash, at
the same height in the same chain, does not indicate a problem.

There is no case where the security of Bitcoin assumes that hashes never
collide. Consensus rules have specific handling for both block hash
collisions and tx hash collisions.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

On 11/17/2016 03:38 AM, Alex Morcos wrote:
> I think this conversation has gone off the rails and is no longer really
> appropriate for the list.

If this discussion is not appropriate for the Bitcoin Protocol
Discussion list then the list is pointless.

> But just to be clear to any readers.  Bitcoin Core absolutely does rely
> on the impossibility of a hash collision for maintaining consensus. 
> This happens in multiple places in the code but in particular we don't
> check BIP30 any more since the only way it could get violated is by a
> hash collision.

So the protocol change that I suggested to Peter an hour or so ago was
actually implemented, a year ago, by you:

https://github.com/bitcoin/bitcoin/commit/06d81ad516f1d136da9f03ca2ae823211c0f6988

Given that hash collisions are unquestionably possible, this is a clear
break with BIP30 (irrespective of BIP34) and constitutes a hard fork. Is
there going to be a retroactive BIP for this one at some point as well?

I'm aware that the block hash check is performed against the full chain,
as opposed to the candidate block fork height, and as a result is
insufficient to guard against a block hash collision causing a chain
split (though until now I assumed this was a bug).

Would you care to share the other consensus critical reliances on the
impossibility of hash collision that you are implying?

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

On 11/17/2016 07:40 AM, Johnson Lau wrote:
>
>> On 17 Nov 2016, at 20:22, Eric Voskuil via bitcoin-dev
 wrote:
>>
>> Given that hash collisions are unquestionably possible,
>
> Everything you said after this point is irrelevant.

So... you think hash collisions are not possible, or that it's moot
because Core has broken its ability to handle them.


> Having hash collision is **by definition** a consensus failure,

I suppose if you take fairly recent un-BIPped consensus changes in Core
to be the definition of consensus, you would be right about that.


> or a hardfork.

And those changes could definitely result in a chain split. So right
about that too.


> You could replace the already-on-chain tx with the collision and
create 2 different versions of UTXOs (if the colliding tx is valid), or
make some nodes to accept a fork with less PoW (if the colliding tx is
invalid, or making the block invalid, such as being to big).


Not in accordance with BIP30 and not according to the implementation of
it that existed in Core until Nov 2015. A tx was only valid as a
"replacement" if it did not collide with the hash of an existing tx with
unspent outputs. The collision would have been rejected. And an invalid
colliding tx would not be accepted in any case (since nodes presumably
validate blocks and don't rely on checkpoints as a security measure).

A transaction duplicating the hash of another and taking its place in a
block would not only have to collide the hash, but it would have to be
fully valid in the context of the block you are suggesting it is
substituted into. In that case it's simply a fully valid block. This is
not just the case of a hash collision, this is the case of a hash
collision where both transactions are fully valid in the context of the
same block parent. Even if that unlikely event did occur, it's not a
hard fork, it's a reorg. The chain that builds on this block will be
valid to all nodes but necessarily deviates from the other block's valid
chain. This is true whether the magical block is assembled via compact
blocks or otherwise.

Transaction "replacement" is an implementation detail of Core. Once Core
accepted a replacement of a previously spent transaction it would be
unable to provide the previous block/spent-tx, but that would be a
wallet failure and an inability to provide valid historical blocks, not
a consensus/validation failure. The previously spent outputs no longer
contribute to validation, unless there is a reorg back to before the
original tx's block, and at that point it would be moot, since neither
transaction is on the chain.

You are referring to the *current* behavior ("replacement" without
concern for collision). That was an unpublished hard fork, and is the
very source of the problems you are describing.

> To put it simply, the Bitcoin protocol is broken. So with no doubt,
Bitcoin Core and any implementation of the Bitcoin protocol should
assume SHA256 collision is unquestionably **impossible**.

I'm not disagreeing with you that it is broken. I'm pointing out that it
was broken by code that was merged recently - an undocumented hard fork
that reverted the documented BIP30 behavior that was previously
implemented correctly, based on the assumption that hash collisions
cannot occur, for the modest performance boost of not having to check
for unspent duplicates (sounds sort of familiar).

> If some refuse to make such assumption, they should have introduced an
alternative hash algorithm and somehow run it in parallel with SHA256 to
prevent the consensus failure.

No hash algorithm can prevent hash collisions, including one that is
just two running in parallel. A better resolution would be to fix the
problem.

There is no need to replace the BIP30 rule. That resolves the TX hash
collision problem from a consensus standpoint. In order to serve up
whole blocks in the circumstance requires a more robust store than I
believe is exists in Core, but that has nothing to do with validity.

The block hash check and signature validation caching splits caused by
collision can easily be avoided, and doing so doesn't break with
consensus. I'm not aware of any other aspects of consensus that are
effected by an implementation assumption of non-colliding hashes. But in
any case I'm pretty sure there aren't any that are necessary to consensus.

e




signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

Actually both possibilities were specifically covered in my description. Sorry 
if it wasn't clear.

If you create a new valid block out of an old one it's has potential to cause a 
reorg. The blocks that previously built on the original are still able to do so 
but presumably cannot build forever on the *new* block as it has a different 
tx. But other new blocks can. There is no chain split due to a different 
interpretation of valid, there are simply two valid competing chains.

Note that this scenario requires not only block and tx validity with a tx hash 
collision, but also that the tx be valid within the block. Pretty far to reach 
to not even get a chain split, but it could produce a deep reorg with a very 
low chance of success. As I keep telling people, deep reorgs can happen, they 
are just unlikely, as is this scenario.

If you create a new invalid block it is discarded by everyone. That does not 
invalidate the hash of that block. Permanent blocking as you describe it would 
be a p2p protocol design choice, having nothing to do with consensus. 
Libbitcoin for example does not ban invalidated hashes at all. It just discards 
the block and drops the peer.

e

> On Nov 17, 2016, at 9:22 AM, Johnson Lau  wrote:
> 
> I’m not sure if you really understand what you and I am talking. It has 
> nothing to do with BIP30, 34, nor any other BIPs.
> 
> Say tx1 is confirmed 3 years ago in block X. An attacker finds a valid tx2 
> which (tx1 != tx2) and (SHA256(tx1) == SHA256(tx2)). Now he could replace tx1 
> with tx2 in block X and the block is still perfectly valid. Anyone trying to 
> download the blockchain from the beginning may end up with a different 
> ledger. The consensus is irrevocably broken as soon as tx1 or tx2 is spent.
> 
> Or, alternatively, an attacker finds an invalid tx3 which (tx1 != tx3) and 
> (SHA256(tx1) == SHA256(tx3)). Now he could replace tx1 with tx3 in block X. 
> Anyone trying to download the blockchain from the beginning will permanently 
> reject the hash of block X. They will instead accept a fork built on top of 
> block X-1. The chain will be permanently forked.
> 
> jl2012
> 
> 
>> On 18 Nov 2016, at 01:01, Eric Voskuil  wrote:
>> 
>> On 11/17/2016 07:40 AM, Johnson Lau wrote:
>>> 
>>>> On 17 Nov 2016, at 20:22, Eric Voskuil via bitcoin-dev
>>  wrote:
>>>> 
>>>> Given that hash collisions are unquestionably possible,
>>> 
>>> Everything you said after this point is irrelevant.
>> 
>> So... you think hash collisions are not possible, or that it's moot
>> because Core has broken its ability to handle them.
>> 
>> 
>>> Having hash collision is **by definition** a consensus failure,
>> 
>> I suppose if you take fairly recent un-BIPped consensus changes in Core
>> to be the definition of consensus, you would be right about that.
>> 
>> 
>>> or a hardfork.
>> 
>> And those changes could definitely result in a chain split. So right
>> about that too.
>> 
>> 
>>> You could replace the already-on-chain tx with the collision and
>> create 2 different versions of UTXOs (if the colliding tx is valid), or
>> make some nodes to accept a fork with less PoW (if the colliding tx is
>> invalid, or making the block invalid, such as being to big).
>> 
>> 
>> Not in accordance with BIP30 and not according to the implementation of
>> it that existed in Core until Nov 2015. A tx was only valid as a
>> "replacement" if it did not collide with the hash of an existing tx with
>> unspent outputs. The collision would have been rejected. And an invalid
>> colliding tx would not be accepted in any case (since nodes presumably
>> validate blocks and don't rely on checkpoints as a security measure).
>> 
>> A transaction duplicating the hash of another and taking its place in a
>> block would not only have to collide the hash, but it would have to be
>> fully valid in the context of the block you are suggesting it is
>> substituted into. In that case it's simply a fully valid block. This is
>> not just the case of a hash collision, this is the case of a hash
>> collision where both transactions are fully valid in the context of the
>> same block parent. Even if that unlikely event did occur, it's not a
>> hard fork, it's a reorg. The chain that builds on this block will be
>> valid to all nodes but necessarily deviates from the other block's valid
>> chain. This is true whether the magical block is assembled via compact
>> blocks or otherwise.
>> 
>> Transaction "replacement" is an implementation detail of Core. Once Core
>> 

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

You are suggesting that, since a node implements a denial of service policy 
that actually denies itself otherwise valid blocks, those blocks are 
conditionally invalid. And that, since the validity condition is based on order 
of arrival and therefore independently unverifiable, Bitcoin consensus is 
broken in the face of a hash collision.

I am aware of two other hash collision scenarios that cause Core to declare 
blocks invalid based on ordering. The block hash duplicate check (it's not 
fork-point relative) and signature verification caching. Like the "block 
banning" issue above, the latter is related to an internal optimization. I 
would categorize the former as a simple oversight that presumably goes way back.

What then is the consequence of validity that is unverifiable? You believe this 
means that Bitcoin consensus is broken. This is incorrect. First understand 
that it is not possible for consensus rules to invalidate blocks based on order 
of arrival. As such any *implementation* that invalidates blocks based on order 
of arrival is broken. It is an error to claim that these behaviors are part of 
consensus, despite being implemented in the satoshi node(s).

Validity must be verifiable independent of the state of other nodes. Consensus 
is a function of block history and time alone. Time is presumed to be 
universally consistent. To be a consensus rule all nodes must be able to 
independently reach the same validity conclusion, given the same set of blocks, 
independent of order. If this is not the case the behavior is not a consensus 
rule, it is simply a bug. 

Deviating from such bugs is not a break with consensus, since such non-rules 
cannot be part of consensus. One node implementation can behave 
deterministically while others are behaving non-deterministically, with the two 
nodes remaining consistent from a consensus standpoint (deterministic produces 
a subset of non-deterministic results). But, unlike arbitrary nodes, 
deterministic nodes will not cause disruption on the network.

You imply that these determinism bugs are necessary, that there is no fix. This 
is also incorrect.

The block banning hash collision bug is avoided by not using non-chain/clock 
state to determine validity. Doing otherwise is clearly a bug. The hash of a 
block is not the block itself, a logically-correct ban would be to compare the 
wire serialization of the block as opposed to the hash, or not maintain the 
feature at all.

The signature verification caching hash collision bug is the same problem, an 
optimization based on an invalid assumption. A full serialization comparison 
(true identity), or elimination of the feature resolves the  bug.

The block hash check collision bug is trivially resolved by checking at the 
fork point as opposed to the tip. This prevents arbitrary (and irrational) 
invalidity based on conflict with irrelevant blocks that may or may not exist 
above the fork point.

Libbitcoin is deterministic in all three cases (although the third issue is not 
made consistent until v3). I am not aware of any other non-determinism in Core, 
but I don't spend a lot of time there. There is no need to study other 
implementations to ensure determinism, as that can be verified independently.

Any situation in which a node cannot provide deterministic validation of 
unordered blocks constitutes a non-consensus bug, as the behavior is not 
consistently verifiable by others under any conditions. Fixing/preventing these 
bugs is responsible development behavior, and does not require forks or BIPs, 
since Bitcoin doesn't inherently contain any such bugs. They are the 
consequence of incorrect implementation, and in two of the three cases above 
have resulted from supposed optimizations. But any code that creates 
non-determinism in exchange for speed, etc. is not an optimization, it's a bug. 
A node must implement its optimizations in a manner that does not alter 
consensus.

The BIP30 regression hard fork is not a case of non-determinism. This will 
produce deterministic results (apart from the impact of unrelated bugs). 
However the results are both a clear break from previous (and documented) 
consensus but also produce a very undesirable outcome - destruction of all 
unspent outputs in the "replaced" transaction for starters. So this is a 
distinct category, not a determinism bug but a hard fork that produces 
undesired consequences.

The BIP30 regression hard fork actually enables the various pathological 
scenarios that you were describing, where no such issues existed in Bitcoin 
consensus previously. It is now possible to produce a block that mutates 
another arbitrarily deep block, and forces a reorg all the way back to the 
mutated block. This was done to save microseconds per block. Despite the 
improbability of hash collisions, I find this deplorable and the lack of public 
discussion on the decision concerning.

With respect to the original post, the point at issue is the introduction o

Re: [bitcoin-dev] BIP30 and BIP34 interaction (was Re: [BIP Proposal] Buried Deployments)

[Eric Voskuil via bitcoin-dev]

What is the difference between downloading a hash and comparing it to a hash vs 
downloading a hash and then a block and comparing it to a block?

You are talking about breaking a system in order to make it run faster. Using 
the hash is an non-optimization trade against correctness.

There is no "first seen" rule, there is only valid and invalid. Even the name 
exposes the error of this thinking as "first" requires order.

Caching invalidity for DOS protection is fine. It should be quite obvious that 
the blockchain is nothing more than a coach of validity. If it's faster in some 
cases to store both validity and all invalidity that you are aware of it is 
fine, you are trading space for time.

But caching information that is neither validity nor invalidity, and using it 
to validate blocks is a break.

I cannot emphasize this point enough. A technique that provides varied results 
based on communication history, such as this "rule", is an attack vector. It 
allows the attacker to place information into your cache and read it back later 
from another connection. Even optimizing correct results based on communication 
history exposes the node in this manner. These sort of attacks have been shown 
to be very effective at deanonymizing hidden nodes.

The p2p protocol actually makes this sort of attack a matter of communication 
standard via the sharing of address information, but this can be disabled 
without impacting correctness. Due to such non-optimizations as the first seen 
"rule" however, a node becomes a candy store of fingerprinting attack vectors.

Bitcoin provides the mechanism to reject cheaply-produced invalid blocks 
quickly. This is after all the fundamental principle of hash cash - force the 
attacker to pay to spam attack. By obtaining headers first a node can obtain 
proof of work and perform correct and fast validation before ever obtaining the 
block's transactions. This technique is probably no more time-costly than the 
incorrect technique of checking a cache of hashes (ironically, a "hash cache" 
is an incorrect "hash cash"), and avoids the extra space of a secondary cache 
(the blockchain is the primary cache). It also avoids the varied time response 
that a secondary cache creates.

So once again, premature optimization erupts from the underlying design flaw, 
and creates more problems than proper design. The p2p network standard didn't 
have headers first at one point, making correct checks more costly. That is no 
longer the case. But nevertheless, one cannot trade correctness for time.

The tx pool, like the orphan pool, as I mentioned previously, is an 
optimization. It is not a part of consensus, so it isn't relevant to a 
discussion about forks. It is also a design flaw that nodes are expected to 
hold invalid transactions. It exposes nodes to both DOS and fingerprinting 
attacks. Proper tx handling implies that a tx connect to a valid block. There 
is no "header" for a transaction so correctness requires that the tx be 
downloaded before it can be validated.

e

> On Nov 18, 2016, at 8:43 AM, Johnson Lau  wrote:
> 
> In this case I don’t understand how your implementation won’t be DoS-ed. An 
> attacker could keep sending you inv for the same block / transaction. Since 
> you don’t assume the hash is unique, each time you have to download the 
> block/tx again before you could tell if that is the same one you have already 
> known. Otherwise, you are implementing the “first seen” rule.
> 
> Also, you can’t ban a peer just because you get an invalid tx from him, 
> because he might be referring to a hash-colliding UTXO that you don’t know. 
> In that case you need to request for the parent tx to verify. I wonder if you 
> are really doing that.
> 
>> On 18 Nov 2016, at 11:20, Eric Voskuil  wrote:
>> 
>> You are suggesting that, since a node implements a denial of service policy 
>> that actually denies itself otherwise valid blocks, those blocks are 
>> conditionally invalid. And that, since the validity condition is based on 
>> order of arrival and therefore independently unverifiable, Bitcoin consensus 
>> is broken in the face of a hash collision.
>> 
>> I am aware of two other hash collision scenarios that cause Core to declare 
>> blocks invalid based on ordering. The block hash duplicate check (it's not 
>> fork-point relative) and signature verification caching. Like the "block 
>> banning" issue above, the latter is related to an internal optimization. I 
>> would categorize the former as a simple oversight that presumably goes way 
>> back.
>> 
>> What then is the consequence of validity that is unverifiable? You believe 
>> this means that Bitcoin consensus is broken. This is incorrect. First 
>> understand that it is not possible for consensus rules to invalidate blocks 
>> based on order of arrival. As such any *implementation* that invalidates 
>> blocks based on order of arrival is broken. It is an error to claim that 
>> these behaviors are part of consensus, de

Re: [bitcoin-dev] Managing block size the same way we do difficulty (aka Block75)

[Eric Voskuil via bitcoin-dev]

The presumption of the mining aspect of the Bitcoin security model is that the 
mining majority is a broadly distributed set of independent people, not one 
person who controls a majority of the hash power. 

You seem to have overlooked a qualifier in your Satoshi quote: "...by nodes 
that are not cooperating to attack the network". A single miner with majority 
hash power is of course cooperating with himself. At that point the question of 
whether he is attacking the network is moot, it's his network.

I believe that Pieter's point is that a system optimized for orphan rate may in 
effect be optimized for a single entity providing all double spend protection. 
That works directly against the central principle of Bitcoin security. The 
security of the money is a function of the number of independent miners and 
sellers.

e

> On Dec 10, 2016, at 7:17 PM, Daniele Pinna via bitcoin-dev 
>  wrote:
> 
> How is the adverse scenario you describe different from a plain old 51% 
> attack? Each proposed protocol change  where 51% or more  of the network can 
> potentially game the rules and break the system should be considered just as 
> acceptable/unacceptable as another. 
> 
> There comes a point where some form of basic honesty must be assumed on 
> behalf of participants benefiting from the system working properly and 
> reliably. 
> 
> Afterall, what magic line of code prohibits all miners from simultaneously 
> turning all their equipment off...  just because? 
> 
> Maybe this 'one':
> 
> "As long as a majority of CPU power is controlled by nodes that are not 
> cooperating to attack the network, they'll generate the longest chain and 
> outpace attackers. The network itself requires minimal structure."
> 
> Is there such a thing as an unrecognizable 51% attack?  One where the 
> remaining 49% get dragged in against their will? 
> 
> Daniele 
> 
>> On Dec 10, 2016 6:39 PM, "Pieter Wuille"  wrote:
>>> On Sat, Dec 10, 2016 at 4:23 AM, Daniele Pinna via bitcoin-dev 
>>>  wrote:
>>> We have models for estimating the probability that a block is orphaned 
>>> given average network bandwidth and block size. 
>>> 
>>> The question is, do we have objective measures of these two quantities? 
>>> Couldn't we target an orphan_rate < max_rate? 
>> 
>> Models can predict orphan rate given block size and network/hashrate 
>> topology, but you can't control the topology (and things like FIBRE hide the 
>> effect of block size on this as well). The result is that if you're purely 
>> optimizing for minimal orphan rate, you can end up with a single 
>> (conglomerate of) pools producing all the blocks. Such a setup has no 
>> propagation delay at all, and as a result can always achieve 0 orphans.
>> 
>> Cheers,
>> 
>> -- 
>> Pieter
>> 
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[Eric Voskuil via bitcoin-dev]

Credit card reversals involve an escrow agent with control over the entire 
network and with a strong interest in preserving the network. A better analogy 
would be blind acceptance of any slip of paper under the assumption that it is 
sufficient currency. It may or may not be so, but you are on your own in either 
case.

e

> On Jan 3, 2017, at 4:36 PM, Aaron Voisine via bitcoin-dev 
>  wrote:
> 
> Knowing that a transaction is property formatted and that it has been 
> broadcast to the gossip network is useful in many situations. You're only 
> thinking about whether you can know a transaction is valid and/or settled. 
> This is not the only possible useful information in actual real world use. 
> Any situation where credit card transactions are accepted today for instance, 
> it is useful to know that a transaction has been initiated, even though it 
> can be reversed at any time up to 60 days later.
> 
> Aaron Voisine
> co-founder and CEO
> breadwallet
> 
>> On Tue, Jan 3, 2017 at 4:10 PM,  wrote:
>> Unfortunately a non validating SPV wallet has absolutely no idea if
>> the information about an unconfirmed transaction they are seeing is
>> anything but properly formatted. They are connecting to an easily
>> manipulated, sybil attacked, and untrusted network and then asking
>> them for financial information. Seeing an unconfirmed transaction in a
>> wallet that's not also fully validating is at best meaningless.
>> 
>> 
>>> On 2017-01-03 15:46, Aaron Voisine wrote:
>>> If the sender doesn't control the receiver's network connection, then
>>> the information the receiver gains by watching the mempool is if the
>>> transaction has propagated across the bitcoin network. This is useful
>>> to know in all kinds of situations.
>>> 
>>> Aaron Voisine
>>> co-founder and CEO
>>> breadwallet [2]
>>> 
>>> On Tue, Jan 3, 2017 at 3:06 PM, adiabat  wrote:
>>> 
 Mempool transactions have their place, but "unconfirmed" and "SPV"
 don't belong together.  Only a full node can tell if a transaction
 may get confirmed, or is nonsense.  Unfortunately all the light /
 SPV wallets I know of show mempool transactions, which makes it hard
 to go back... (e.g. "why doesn't your software show 0-conf! your
 wallet is broken!", somewhat akin to people complaining about RBF)
 
 So, this is easy, just don't worry about mempool filtering.  Why are
 light clients looking at the mempool anyway?  Maybe if there were
 some way to provide SPV proofs of all inputs, but that's a bit of a
 mess for full nodes to do.
 
 Without mempool filtering, I think the committed bloom filters would
 be a great improvement over the current bloom filter setup,
 especially for lightning network use cases (with lightning, not
 finding out about a transaction can make you lose money).  I want to
 work on it and may be able to at some point as it's somewhat related
 to lightning.
 
 Also, if you're running a light client, and storing the filters the
 way you store block headers, there's really no reason to go all the
 way back to height 0.  You can start grabbing headers at some point
 a while ago, before your set of keys was generated.  I think it'd be
 very worth it even with GB-scale disk usage.
 
 -Tadge
 
 On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev
  wrote:
 
 Unconfirmed transactions are incredibly important for real world
 use. Merchants for instance are willing to accept credit card
 payments of thousands of dollars and ship the goods despite the fact
 that the transaction can be reversed up to 60 days later. There is a
 very large cost to losing the ability to have instant transactions
 in many or even most situations. This cost is typically well above
 the fraud risk.
 
 It's important to recognize that bitcoin serves a wide variety of
 use cases with different profiles for time sensitivity and fraud
 risk.
 
 Aaron
 
 On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev
  wrote:
 The concept combined with the weak blocks system where miners commit
 
 to potential transaction inclusion with fractional difficulty blocks
 
 is possible. I'm not personally convinced that unconfirmed
 transaction
 
 display in a wallet is worth the privacy trade-off. The user has
 very
 
 little to gain from this knowledge until the txn is in a block.
 
 On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:
 
> Hi
 
>> We introduce several concepts that rework the lightweight Bitcoin
 
>> client model in a manner which is secure, efficient and privacy
 
>> compatible.
 
>> 
 
>> The BFD can be used verbatim in replacement of BIP37, where the
 filter
 
>> can be cached between clients without needing to be recomputed.
 It can
 
>> also be used 

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[Eric Voskuil via bitcoin-dev]

On 01/04/2017 11:06 PM, Chris Priest via bitcoin-dev wrote:
> On 1/3/17, Jonas Schnelli via bitcoin-dev
>  wrote:
>>
>> There are plenty, more sane options. If you can't run your own full-node
>> as a merchant (trivial), maybe co-use a wallet-service with centralized
>> verification (maybe use two of them), I guess Copay would be one of
>> those wallets (as an example). Use them in watch-only mode.
> 
> The best way is to connect to the mempool of each miner and check to
> see if they have your txid in their mempool.
> 
> https://www.antpool.com/api/is_in_mempool?txid=334847bb...
> https://www.f2pool.com/api/is_in_mempool?txid=334847bb...
> https://bw.com/api/is_in_mempool?txid=334847bb...
> https://bitfury.com/api/is_in_mempool?txid=334847bb...
> https://btcc.com/api/is_in_mempool?txid=334847bb...
> 
> If each of these services return "True", and you know those services
> so not engage in RBF, then you can assume with great confidence that
> your transaction will be in the next block, or in a block very soon.
> If any one of those services return "False", then you must assume that
> it is possible that there is a double spend floating around, and that
> you should wait to see if that tx gets confirmed. The problem is that
> not every pool runs such a service to check the contents of their
> mempool...
> 
> This is an example of mining centralization increasing the security of
> zero confirm.

A world connected up to a few web services to determine payment validity
is an example of a bitcoin security catastrophe.

e



signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[Eric Voskuil via bitcoin-dev]

It is a useful aspect of discussion at this level as it helps higher lever 
developers understand the actual tradeoffs. Clearly some do not. The market 
will eventually sort them out, but the discussion both gives developers the 
necessary information.

It also helps core development prioritize resources. I personally would not 
prioritize core work to facilitate zero conf. I would even spend time to 
discourage it, as others have done.

I think the cautions in this thread about doing privacy and system security 
damaging things (like checking mining pools for zero conf transactions) will 
prevent some wasted time, which benefits everyone.

e

> On Jan 6, 2017, at 1:35 PM, James MacWhyte via bitcoin-dev 
>  wrote:
> 
> It's my opinion that the purpose of this list and bitcoin protocol 
> development in general is to build the base functionality that other 
> companies and individuals require to provide usability to the end-user. The 
> 0-conf debate is a UX issue. If end users shouldn't rely on 0-conf, it is up 
> to wallet developers to hide 0-conf transactions or mark them appropriately. 
> Instead of using this list to debate what wallet designers should or 
> shouldn't do, we should just provide the tools and "let the market sort it 
> out". If wallet developers start getting inundated with complaints that 
> 0-conf transactions are causing confusion and loss, they will find a 
> solution. If the tools they require for the solution don't exist, they will 
> come to this list to request action.
> 
> Am I wrong?
> 
> On Fri, Jan 6, 2017 at 12:16 PM Chris Priest via bitcoin-dev 
>  wrote:
>> Its a method for determining the probability that a valid tx will be
>> mined in a block before that tx actually gets mined, which is useful
>> when accepting payments in situations when you can't wait for the full
>> confirmation. No one is saying all tx validation should be performed
>> by querying miners mempools, that's ridiculous. Obviously once the tx
>> gets it's first confirmation, you go back to determining validity the
>> way you always have. There is no "security catastrophe".
>> 
>> Even if you're running a full node, you can't know for certain that
>> any given tx will make it into a future block. You can't be certain
>> the future miner who finally does mine that tx will mine your TXID or
>> another TXID that spends the same inputs to another address (a double
>> spend). The only way to actually know for certain is to query every
>> single large hashpower mempool.
>> 
>> On 1/4/17, Eric Voskuil  wrote:
>> > On 01/04/2017 11:06 PM, Chris Priest via bitcoin-dev wrote:
>> >> On 1/3/17, Jonas Schnelli via bitcoin-dev
>> >>  wrote:
>> >>>
>> >>> There are plenty, more sane options. If you can't run your own full-node
>> >>> as a merchant (trivial), maybe co-use a wallet-service with centralized
>> >>> verification (maybe use two of them), I guess Copay would be one of
>> >>> those wallets (as an example). Use them in watch-only mode.
>> >>
>> >> The best way is to connect to the mempool of each miner and check to
>> >> see if they have your txid in their mempool.
>> >>
>> >> https://www.antpool.com/api/is_in_mempool?txid=334847bb...
>> >> https://www.f2pool.com/api/is_in_mempool?txid=334847bb...
>> >> https://bw.com/api/is_in_mempool?txid=334847bb...
>> >> https://bitfury.com/api/is_in_mempool?txid=334847bb...
>> >> https://btcc.com/api/is_in_mempool?txid=334847bb...
>> >>
>> >> If each of these services return "True", and you know those services
>> >> so not engage in RBF, then you can assume with great confidence that
>> >> your transaction will be in the next block, or in a block very soon.
>> >> If any one of those services return "False", then you must assume that
>> >> it is possible that there is a double spend floating around, and that
>> >> you should wait to see if that tx gets confirmed. The problem is that
>> >> not every pool runs such a service to check the contents of their
>> >> mempool...
>> >>
>> >> This is an example of mining centralization increasing the security of
>> >> zero confirm.
>> >
>> > A world connected up to a few web services to determine payment validity
>> > is an example of a bitcoin security catastrophe.
>> >
>> > e
>> >
>> >
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev