Re: [bitcoin-dev] Reminder on the Purpose of BIPs

[John Newbery via bitcoin-dev]

ACK. These seem like very reasonable next steps.

On Mon, Apr 26, 2021 at 8:43 PM David A. Harding via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Sun, Apr 25, 2021 at 05:31:50PM -0400, Matt Corallo via bitcoin-dev
> wrote:
> > In general, I think its time we all agree the BIP process has simply
> failed
> > and move on. Luckily its not really all that critical and proposed
> protocol
> > documents can be placed nearly anywhere with the same effect.
>
> I recommend:
>
> 1. We add additional BIP editors, starting with Kalle Alm (if there are
>no continuing significant objections).
>
> 2. We seek Luke Dashjr's resignation as BIPs editor.
>
> 3. We begin treating protocol documents outside the BIPs repository as
>first-class BIP documentation.
>
> The first recommendation permits continued maintenance of existing BIPs
> plus gives the additional maintainers an opportunity to rebuild the
> credibility of the repository.
>
> The second recommendation addresses the dissatisfaction of many BIP
> authors and potential authors with the current editor, which I think
> will discourage many of them from making additional significant
> contributions to the repository.  It also seems to me to be a better use
> of Luke's talents and interests for him to focus on protocol research
> and review rather than procedurally checking whether a bunch of
> documents are well formed.
>
> The third recommendation provides an escape hatch for anyone, such as
> Matt, who currently thinks the process has failed, or for anyone who
> comes to that same conclusion in the future under a different editing
> team.  My specific recommendations there are:
>
> a. Anyone writing protocol documentation in the spirit of the BIP
>process can post their idea to this mailing list like we've always
>done and, when they've finished collecting initial feedback, they can
>assign themselves a unique decentralized identifier starting with
>"bip-".  They may also define a shorter alias that they encourage
>people to use in cases where the correct document can be inferred
>from context.  E.g.,
>
>   bip-wuille-taproot (bip-taproot)
>   bip-towns-versionbits-min-activation-height (bip-vbmah)
>   bip-todd-harding-opt-in-replace-by-fee (bip-opt-in-rbf)
>
> b. The author then publishes the document to any place they'd like,
> although
>they are strongly encouraged to make any document source available
>under an open license to ensure others can create their own
>modifications.
>
> c. Implementations of BIPs, whether original repository BIPs or
>decentralized BIPs, link to the BIPs they implement to ensure
>researchers and developers can find the relevant protocol
>documentation.  E.g.,
>
> https://github.com/bitcoin/bitcoin/blob/fe5e495c31de47b0ec732b943db11fe345d874af/doc/bips.md
>
>  (It may also be advisable for implementations to mirror copies of
>  the BIPs they implement so later modifications to the document
>  don't confuse anyone.  For this reason, extremely liberal
>  licensing of BIP documents is encouraged.)
>
> d. To help maintain quality and consistency between documentation, the
>BIP editors provide a BIP document template, guidelines similar to
>the existing BIP2, and an easy-to-run format linter.
>
> I think this decentralized BIPs alternative also helps address some
> longstanding problems with the BIPs system: that many casual Bitcoin
> users and developers think of documents in the BIPs repo as
> authoritative and that there are some development teams (such as for LN)
> that have already abandoned the BIPs process because, in part, they want
> complete control over their own documentation.
>
> The recommendations above were developed based on conversations I had
> with a few stakeholders in the BIPs process, but I did not attempt a
> comprehensive survey and I certainly don't claim to speak for anyone
> else.  I hope the recommendations are satisfactory and I look forward to
> your feedback.
>
> Thanks,
>
> -Dave
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Proposed BIP editor: Kalle Alm

[John Newbery via bitcoin-dev]

ACK adding Kalle.

On Fri, Apr 23, 2021 at 4:36 AM Jeremy via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> ACK adding Kalle.
>
> Kalle is a qualified reviewer / editor and well suited for this role.
> --
> @JeremyRubin 
> 
>
>
> On Thu, Apr 22, 2021 at 7:09 PM Luke Dashjr via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Unless there are objections, I intend to add Kalle Alm as a BIP editor to
>> assist in merging PRs into the bips git repo.
>>
>> Since there is no explicit process to adding BIP editors, IMO it should
>> be
>> fine to use BIP 2's Process BIP progression:
>>
>> > A process BIP may change status from Draft to Active when it achieves
>> > rough consensus on the mailing list. Such a proposal is said to have
>> > rough consensus if it has been open to discussion on the development
>> > mailing list for at least one month, and no person maintains any
>> > unaddressed substantiated objections to it.
>>
>> A Process BIP could be opened for each new editor, but IMO that is
>> unnecessary. If anyone feels there is a need for a new Process BIP, we
>> can go
>> that route, but there is prior precedent for BIP editors appointing new
>> BIP
>> editors, so I think this should be fine.
>>
>> Please speak up soon if you disagree.
>>
>> Luke
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Proposal for new "disabletx" p2p message

[John Newbery via bitcoin-dev]

Antoine,

Nothing in my proposal below precludes introducing a more comprehensive
feature negotiation mechanism at some later date. The only changes I'm
proposing are to Bitcoin Core's policy for how it treats its peer
connections.

> If we don't want to introduce a new message and
> corresponding code changes, it would be wise at least to extract VERSION's
> `fRelay` and how Core handles it in its own BIP.

I believe this is what BIP 60 does, or did you have something else in
mind?

> Explicit addr-relay negotiation will offer more
> flexibility

I agree!

> (and more hygienic code paths rather than triggering data
> structures initialization in few different locations).

Not sure what you mean by hygienic here. This seems like a code style
preference.

> Given inbound connections might be attacker-controlled and tx-relay
opt-out
> signaling is also attacker-controlled, wouldn't this give a bias toward an
> attacker in occupying our inbound slots ? Compared to honest inbound
peers,
> which in average are going to be full-relay.

Sorry - I meant that Bitcoin Core should allow a certain number of
inbound peers that do not relay txs. This would be in addition to the
full-relay inbound peers.

John

On Mon, Mar 1, 2021 at 11:11 PM Antoine Riard 
wrote:

> Hi John,
>
> > I think a good counter-argument against simply using `fRelay` for this
> > purpose is that we shouldn't reuse a protocol feature designed for one
> > function to achieve a totally different aim. However, we know that nodes
> > on the network have been using `fRelay` to disable transaction relay
> > since Bitcoin Core version 0.12 (when `-blocksonly` was added), and that
> > usage was expanded to _all_ nodes running Bitcoin Core version 0.19 or
> > later (when block-relay-only connections were introduced), so using
> > `fRelay` to disable transaction relay is now de facto part of the p2p
> > protocol.
>
>
> I don't think this is good practice ecosystem-wise. To understand tx-relay
> opt-out from peers correctly, a _non_ Bitcoin Core client has to implement
> the `fRelay` subset of BIP37, but ignore the wider part around FILTER*
> messages. Or implement those messages, only to disconnect peers sending
> them, thus following BIP111 requirements.
>
> Thus, future developers of bitcoin software have the choice between
> implementing a standard in a non-compliant way or implementing p2p messages
> for a light client protocol in a way of deprecation ? Even further, an
> interpretation of BIP 37 ("Being able to opt-out of _inv_ messages until
> the filter is set prevents a client being flooded with traffic in the brief
> window of time") would make it okay to send TX messages to your inbound
> block-relay-only peers. And that your client shouldn't be disconnected for
> such behavior.
>
> On the long-term, IMHO, better to have a well-defined standard with a
> clean negotiation mechanism rather than relying on code specifics of a
> given Bitcoin client. If we don't want to introduce a new message and
> corresponding code changes, it would be wise at least to extract VERSION's
> `fRelay` and how Core handles it in its own BIP.
>
> > I think a better approach would be for Bitcoin Core to only relay addr
> > records to an inbound peer if it has previously received an `addr` or
> > `addrv2` message from that peer, since that indicates definitively that
> > the peer actively gossips `addr` records. This approach was first
> > suggested by AJ in the original block-relay-only PR[15].
>
> If a node is willingly to opt-out from addr-relay from one of its inbound
> peers, how is it supposed to do ? Of course, you can drop such messages on
> the floor, your peer is just going to waste bandwidth for nothing. IIRC
> from past irc p2p meetings, we're really unclear about what a
> good-propagation-and-privacy-preserving addr-relay strategy should look
> like. Note, that distrusting your inbound peers with your addr-relay might
> be a sane direction. Explicit addr-relay negotiation will offer more
> flexibility (and more hygienic code paths rather than triggering data
> structures initialization in few different locations).
>
> > - update the inbound eviction logic to protect more inbound peers which
> > do not have transaction relay data structures.
>
> Given inbound connections might be attacker-controlled and tx-relay
> opt-out signaling is also attacker-controlled, wouldn't this give a bias
> toward an attacker in occupying our inbound slots ? Compared to honest
> inbound peers, which in average are going to be full-relay.
>
> Cheers,
> Antoine
>
>
>
> Le lun. 1 mars 2021 à 16:07, John Newbery via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> a écrit :
>
>> 

Re: [bitcoin-dev] Proposal for new "disabletx" p2p message

[John Newbery via bitcoin-dev]

Hi Suhas,

Thank you for this proposal. I agree with your aims, but I think a new
P2P message isn't necessary to achieve them.

# Motivation

There are two distinct (but interacting) motivations:

1. Allow a node to accept more incoming connections which will only be
   used for block propagation (no transaction relay or addr gossip),
   while minimizing resource requirements.

2. Prevent `addr` gossip messages from being sent to peers which will
   'black hole' those addrs (i.e. not relay them further).

These motivations interact because if we simply increase the number of
block-relay-only connections that nodes make without making any
allowance for the fact those connections won't gossip addr records, then
we'll increase the number of addr black holes and worsen addr gossip.

# Using fRelay=false to signal no transaction relay.

`fRelay` is an optional field in the `version` message. There are three
BIPs concerned with `fRelay`:

- BIP 37[1] introduced the `fRelay` field to indicate to the recipient
  that they must not relay transactions over the connection until a
  `filteradd` message has been received.

- BIP 60[2] aimed to make the `fRelay` field mandatory. It is not clear
  how widely this BIP has been adopted by implementations.

- BIP 111[3] introduced a `NODE_BLOOM` service bit to indicate that
  bloom filters are served by this node. According to this BIP, "If a
  node does not support bloom filters but receives a "filterload",
  "filteradd", or "filterclear" message from a peer the node should
  disconnect that peer immediately."

Within Bitcoin Core:

- PR 1795[4] (merged in January 2013) added support for BIP 37 Bloom
  filters.

- Since PR 2763[5] (merged in June 2013), Bitcoin Core will _always_
  include the `fRelay` flag in `version` messages that it sends. Bitcoin
  Core will tolerate the `fRelay` field being present or absent in any
  `version` message that it receives[6].

- PR 6579[7] (merged in August 2015) implemented BIP 111. From that
  point on, a Bitcoin Core node would disconnect peers that sent it
  `filter*` messages if it hadn't enabled `NODE_BLOOM`, provided the
  peer's version was >= 70011. In PR 7708[8] (merged in March 2016) this
  was extended to disconnect any peer that sends a `filter*` message,
  regardless of its version (in general, a 'polite disconnect' for any
  peer that requests an unsupported service is probably the best
  behaviour). In PR 16152[9] (merged in July 2019), serving Bloom
  filters was disabled by default, due to potential denial-of-service
  attacks being possible against nodes which serve bloom filters on
  public connections.

- PR 6993[10] (merged in November 2015) started reusing the `fRelay`
  field for the new `-blocksonly` mode. If Bitcoin Core is started with
  `-blocksonly` configured, then it includes `fRelay=false` in all of
  the `version` messages it sends. In PR 15759[11] (merged  in September
  2019), this usage of `fRelay` to permanently disable tx relay was
  extended for use by the new block-relay only connection type.

The net effect is that `fRelay` is already being used to indicate that
transactions should not be relayed over a connection. In the motivation
for your BIP, you write:

> The low-bandwidth / minimal-resource nature of these connections is
> currently known only by the initiator of the connection; this is
> because the transaction relay field in the version message is not a
> permanent setting for the lifetime of the connection.  Consequently, a
> node receiving an inbound connection with transaction relay disabled
> cannot distinguish between a peer that will never enable transaction
> relay (as described in BIP 37) and one that will...

However, as AJ points out in his response [12], the Bitcoin Core node
_does_ know whether transaction relay can be supported as soon as the
`version` message is received:

> [...] you either set m_tx_relay->fRelayTxes to true via the VERSION
> message (either explicitly or by not setting fRelay), or you enable it
> later with FILTERLOAD or FILTERCLEAR, both of which will cause a
> disconnect if bloom filters aren't supported. Bloom filter support is
> (optionally?) indicated via a service bit (BIP 111), so you could
> assume you know whether they're supported as soon as you receive the
> VERSION line.

i.e. if Bitcoin Core node is running under normal configuration with
bloom filters disabled for public connections (which is both the default
setting and highly recommended due to DoS concerns), then as soon as it
receives a `version` message with `fRelay=false`, it can be sure that
there will never be any transaction relay with that peer. If the peer
later tries to enable transaction relay by sending a `filterload`
message, then the node will disconnect that peer immediately.

In summary, we can continue using the `fRelay` field to indicate that
no transaction relay can happen for the entire lifetime of the
connection.  Bitcoin Core can postpone allocating resources for

[bitcoin-dev] BIP 155 (addrv2) update - spec and Bitcoin Core v0.21 implementation

[John Newbery via bitcoin-dev]

Hi folks,

BIP 155 was proposed[1] in Feb 2019 by Wladimir van der Laan as a way of
gossipping longer node addresses over the Bitcoin P2P network, primarily
to support torv3 and other networks.

In the time since that initial mailing list post, several changes have
been made to the proposal. Discussion has been held on the BIPs repo[2],
and (for implementation issues) the Bitcoin Core repo[3].

This email summarizes the changes. Readers should refer to BIP 155[4]
for the full specification.

### Specification changes

1. The `time` field in the `addrv2` message is now stored as a fixed
   length uint32 instead of a variable-length compact size.

2. The `addr` field may be up to a maximum of 512 bytes (4096 bits)
   instead of 32 bytes (256 bits) for compatibility with future address
   formats.

3. Clients now SHOULD gossip addresses for known networks (even if they
   can't connect to those networks). They SHOULD NOT gossip addresses
   for unknown networks. They SHOULD also ignore addresses for known
   networks that are different from the address length for that network
   specified in BIP 155.

4. New network IDs MUST be reserved in a BIP document.

5. Support for `addrv2` is not dependent on a p2p protocol version.
   A new message type `sendaddrv2` is introduced to signal support
   for addrv2. To signal support for addrv2, this message MUST be sent
   after the initial version message is sent and before the verack
   message is sent.

### Implementation detail

During testing of the Bitcoin Core implementation, it was found that
another Bitcoin implementation would disconnect from peers on receipt of
an unknown message[5]. I believe that to be an incorrect interpretation
of the Bitcoin p2p protocol. The original v0.1 Satoshi client (and all
Bitcoin Core versions derived from it) have always explicitly ignored
unknown message types as a mechanism to extend the p2p protocol[6]. This
property allows p2p implementions to permissionlessly deploy opt-in
extensions to the protocol.

As a pragmatic step to prevent those implementations from being
disconnected from v0.21 Bitcoin Core nodes, this initial version will
_only_ send sendaddrv2 messages to peers on p2p protocol version 70016
and higher. This behaviour may be reverted in future, at which point
Bitcoin Core would send sendaddrv2 messages to all peers during the
version/verack handshake.

Thanks to everyone who has contributed to the addrv2
spec/implementation.

John

[1] 
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-February/016687.html
[2] https://github.com/bitcoin/bips/search?q=addrv2+is%3Apr=Issues
[3] https://github.com/bitcoin/bitcoin/pull/20564
[4] https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki
[5] https://github.com/btcsuite/btcd/issues/1661
[6] 
https://github.com/benjyz/bitcoinArchive/blob/master/bitcoin0.1/src/main.cpp#L2035-L2039
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Revisiting squaredness tiebreaker for R point in BIP340

[John Newbery via bitcoin-dev]

Pieter,

Thanks for the illuminating write-up. There seem to be two questions here,
one technical and one process:

1. Is changing to even tie-breaker for R the correct choice technically? I
can't comment on the performance characteristics of using a square/even
tie-breaker and I'll assume the numbers you give are correct. An enormous
benefit that you don't mention (but Nadav and Lloyd do) is that
standardizing to a single tie-breaker for R points and public keys is much
simpler to explain and much easier for implementers and developers to
understand. I've explained the taproot proposals to many people through the
optech workshops and bitdevs meetups, and people are invariably confused by
which type of tie-breaker to use where. Absent a large performance benefit
for having different tiebreakers, I think this alone is good reason to
standardize to one tie-breaker.

2. Is it too late in the process to change? No. We're building things to
last years, hopefully decades. We should measure a hundred times and cut
once. A benefit of the long lead time of taproot is that as we get more
information, we can improve the proposal. Let's do that here. Nadav and
Lloyd have both written alternative implementations of taproot and are
happy to make this change. Presumably if this was going to cause serious
pain for any other implementer/developer they would have raised objections
by now.

Summary: We should change the proposal and implementation to use even
tie-breakers everywhere.

John #notoquadraticresiduetiebreakers Newbery

On Wed, Aug 12, 2020 at 7:49 PM Pieter Wuille via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hello all,
>
> The current BIP340 draft[1] uses two different tiebreakers for conveying
> the Y coordinate of points: for the R point inside signatures squaredness
> is used, while for public keys evenness is used. Originally both used
> squaredness, but it was changed[2] for public keys after observing this
> results in additional complexity for compatibility with existing systems.
>
> The reason for choosing squaredness as tiebreaker was performance: in
> non-batch signature validation, the recomputed R point must be verified to
> have the correct sign, to guarantee consistency with batch validation.
> Whether the Y coordinate is square can be computed directly in Jacobian
> coordinates, while determining evenness requires a conversion to affine
> coordinates first.
>
> This argument of course relies on the assumption that determining whether
> the Y coordinate is square can be done more efficiently than a conversion
> to affine coordinates. It appears now that this assumption is incorrect,
> and the justification for picking the squaredness tiebreaking doesn't
> really exist. As it comes with other trade-offs (it slows down signing, and
> is a less conventional choice), it would seem that we should reconsider the
> option of having the R point use the evenness tiebreaker (like public keys).
>
> It is late in the process, but I feel I owe this explanation so that at
> least the possibility of changing can be discussed with all information. On
> the upside, this was discovered in the context of looking into a cool
> improvement to libsecp256k1[5], which makes things faster in general, but
> specifically benefits the evenness variant.
>
>
> # 1. What happened?
>
> Computing squaredness is done through the Jacobi symbol (same inventor,
> but unrelated to Jacobian coordinates). Computing evenness requires
> converting points to affine coordinates first, and that needs a modular
> inverse. The assumption that Jacobi symbols are faster to compute than
> inverses was based on:
>
> * A (possibly) mistaken belief about the theory: fast algorithms for both
> Jacobi symbols and inverses are internally based on variants of the same
> extended GCD algorithm[3]. Since an inverse needs to extract a full big
> integer out of the transition steps made in the extgcd algorithm, while the
> Jacobi symbol just extracts a single bit, it had seemed that any advances
> applicable to one would be applicable to the other, but inverses would
> always need additional work on top. It appears however that a class of
> extgcd algorithms exists (LSB based ones) that cannot be used for Jacobi
> calculations without losing efficiency. Recent developments[4] and a
> proposed implementation in libsecp256k1[5] by Peter Dettman show that using
> this, inverses in some cases can in fact be faster than Jacobi symbols.
>
> * A broken benchmark. This belief was incorrectly confirmed by a broken
> benchmark[6] in libsecp256k1 for the libgmp-based Jacobi symbol calculation
> and modular inverse. The benchmark was repeatedly testing the same constant
> input, which apparently was around 2.5x faster than the average speed. It
> is a variable-time algorithm, so a good variation of inputs matters. This
> mistake had me (and probably others) convinced for years that Jacobi
> symbols were amazingly fast, while in reality they were 

Re: [bitcoin-dev] [Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients

[John Newbery via bitcoin-dev]

There doesn't seem to be anything in the original email that's specific to
BIP 157. It's a restatement of the arguments against light clients:

- light clients are a burden on the full nodes that serve them
- if light clients become more popular, there won't be enough full nodes to
serve them
- people might build products that depend on altruistic nodes serving data,
which is unsustainable
- maybe at some point in the future, light clients will need to pay for
services

The choice isn't between people using light clients or not. People already
use light clients. The choice between whether we offer them a light client
technology that is better or worse for privacy and scalability.

The arguments for why BIP 157 is better than the existing light client
technologies are available elsewhere, but to summarize:

- they're unique for a block, which means they can easily be cached.
Serving a filter requires no computation, just i/o (or memory access for
cached filter/header data) and bandwidth. There are plenty of other
services that a full node offers that use i/o and bandwidth, such as
serving blocks.
- unique-for-block means clients can download from multiple sources
- the linked-headers/filters model allows hybrid approaches, where headers
checkpoints can be fetched from trusted/signed nodes, with intermediate
headers and filters fetched from untrusted sources
- less possibilities to DoS/waste resources on the serving node
- better for privacy

> The intention, as I understood it, of putting BIP157 directly into
bitcoind was to essentially force all `bitcoind` users to possibly service
BIP157 clients

Please. No-one is forcing anyone to do anything. To serve filters, a node
user needs to download the latest version, set `-blockfilterindex=basic` to
build the compact filters index, and set `-peercfilters` to serve them over
P2P. This is an optional, off-by-default feature.

Regards,
John


On Tue, May 5, 2020 at 9:50 AM ZmnSCPxj via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Good morning ariard and luke-jr
>
>
> > > Trust-minimization of Bitcoin security model has always relied first
> and
> > > above on running a full-node. This current paradigm may be shifted by
> LN
> > > where fast, affordable, confidential, censorship-resistant payment
> services
> > > may attract a lot of adoption without users running a full-node.
> >
> > No, it cannot be shifted. This would compromise Bitcoin itself, which for
> > security depends on the assumption that a supermajority of the economy is
> > verifying their incoming transactions using their own full node.
> >
> > The past few years has seen severe regressions in this area, to the point
> > where Bitcoin's future seems quite bleak. Without serious improvements
> to the
> > full node ratio, Bitcoin is likely to fail.
> >
> > Therefore, all efforts to improve the "full node-less" experience are
> harmful,
> > and should be actively avoided. BIP 157 improves privacy of fn-less
> usage,
> > while providing no real benefits to full node users (compared to more
> > efficient protocols like Stratum/Electrum).
> >
> > For this reason, myself and a few others oppose merging support for BIP
> 157 in
> > Core.
>
> BIP 157 can be implemented as a separate daemon that processes the blocks
> downloaded by an attached `bitcoind`, i.e. what Wasabi does.
>
> The intention, as I understood it, of putting BIP157 directly into
> bitcoind was to essentially force all `bitcoind` users to possibly service
> BIP157 clients, in the hope that a BIP157 client can contact any arbitrary
> fullnode to get BIP157 service.
> This is supposed to improve to the situation relative to e.g. Electrum,
> where there are far fewer Electrum servers than fullnodes.
>
> Of course, as ariard computes, deploying BIP157 could lead to an effective
> DDoS on the fullnode network if a large number of BIP157 clients arise.
> Though maybe this will not occur very fast?  We hope?
>
> It seems to me that the thing that *could* be done would be to have
> watchtowers provide light-client services, since that seems to be the major
> business model of watchtowers, as suggested by ariard as well.
> This is still less than ideal, but maybe is better than nothing.
>
> Regards,
> ZmnSCPxj
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Removal of reject network messages from Bitcoin Core (BIP61)

[John Newbery via bitcoin-dev]

> Is there a NODE_* bit we can use to pick peers that support this (useful!)
feature?

No. BIP 61 has no mechanism for advertising that a node will send REJECT
messages.

On Wed, Oct 16, 2019 at 12:43 PM John Newbery  wrote:

> Following discussion on this mailing list, support for BIP 61 REJECT
> messages was not removed from Bitcoin Core in V0.19. The behaviour in that
> upcoming release is that REJECT messages are disabled by default and can be
> enabled using the `-enablebip61` command line option.
>
> Support for REJECT messages will be removed entirely in Bitcoin Core
> V0.20, expected for release in mid 2020. The PR to remove support was
> merged into Bitcoin Core's master branch this week.
>
> Adoption of new Bitcoin Core versions across reachable nodes generally
> takes several months. https://bitnodes.earn.com/dashboard/?days=365 shows
> that although v0.18 was released in May 2019, there are still several
> hundred reachable nodes on V0.17, V0.16, V0.15 and earlier software.
> Software that currently use REJECT messages from public nodes for
> troubleshooting issues therefore have plenty of time to transition to one
> of the methods listed by Marco in the email above.
>
> John
>
> On Tue, Mar 5, 2019 at 10:28 PM Marco Falke via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Bitcoin Core may send "reject" messages as response to "tx", "block" or
>> "version" messages from a network peer when the message could not be
>> accepted.
>>
>> This feature is toggled by the `-enablebip61` command line option and has
>> been
>> disabled by default since Bitcoin Core version 0.18.0 (not yet released
>> as of
>> time of writing). Nodes on the network can not generally be trusted to
>> send
>> valid ("reject") messages, so this should only ever be used when
>> connected to a
>> trusted node. At this time, I am not aware of any software that requires
>> this
>> feature, and I would like to remove if from Bitcoin Core to make the
>> codebase
>> slimmer, easier to understand and maintain. Let us know if your
>> application
>> relies on this feature and you can not use any of the recommended
>> alternatives:
>>
>> * Testing or debugging of implementations of the Bitcoin P2P network
>> protocol
>>   should be done by inspecting the log messages that are produced by a
>> recent
>>   version of Bitcoin Core. Bitcoin Core logs debug messages
>>   (`-debug=`) to a stream (`-printtoconsole`) or to a file
>>   (`-debuglogfile=`).
>>
>> * Testing the validity of a block can be achieved by specific RPCs:
>>   - `submitblock`
>>   - `getblocktemplate` with `'mode'` set to `'proposal'` for blocks with
>> potentially invalid POW
>>
>> * Testing the validity of a transaction can be achieved by specific RPCs:
>>   - `sendrawtransaction`
>>   - `testmempoolaccept`
>>
>> * Wallets should not use the absence of "reject" messages to indicate a
>>   transaction has propagated the network, nor should wallets use "reject"
>>   messages to set transaction fees. Wallets should rather use fee
>> estimation
>>   to determine transaction fees and set replace-by-fee if desired. Thus,
>> they
>>   could wait until the transaction has confirmed (taking into account the
>> fee
>>   target they set (compare the RPC `estimatesmartfee`)) or listen for the
>>   transaction announcement by other network peers to check for
>> propagation.
>>
>> I propose to remove "reject" messages from Bitcoin Core 0.19.0 unless
>> there are
>> valid concerns about its removal.
>>
>> Marco
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Removal of reject network messages from Bitcoin Core (BIP61)

[John Newbery via bitcoin-dev]

Following discussion on this mailing list, support for BIP 61 REJECT
messages was not removed from Bitcoin Core in V0.19. The behaviour in that
upcoming release is that REJECT messages are disabled by default and can be
enabled using the `-enablebip61` command line option.

Support for REJECT messages will be removed entirely in Bitcoin Core V0.20,
expected for release in mid 2020. The PR to remove support was merged into
Bitcoin Core's master branch this week.

Adoption of new Bitcoin Core versions across reachable nodes generally
takes several months. https://bitnodes.earn.com/dashboard/?days=365 shows
that although v0.18 was released in May 2019, there are still several
hundred reachable nodes on V0.17, V0.16, V0.15 and earlier software.
Software that currently use REJECT messages from public nodes for
troubleshooting issues therefore have plenty of time to transition to one
of the methods listed by Marco in the email above.

John

On Tue, Mar 5, 2019 at 10:28 PM Marco Falke via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Bitcoin Core may send "reject" messages as response to "tx", "block" or
> "version" messages from a network peer when the message could not be
> accepted.
>
> This feature is toggled by the `-enablebip61` command line option and has
> been
> disabled by default since Bitcoin Core version 0.18.0 (not yet released as
> of
> time of writing). Nodes on the network can not generally be trusted to send
> valid ("reject") messages, so this should only ever be used when connected
> to a
> trusted node. At this time, I am not aware of any software that requires
> this
> feature, and I would like to remove if from Bitcoin Core to make the
> codebase
> slimmer, easier to understand and maintain. Let us know if your application
> relies on this feature and you can not use any of the recommended
> alternatives:
>
> * Testing or debugging of implementations of the Bitcoin P2P network
> protocol
>   should be done by inspecting the log messages that are produced by a
> recent
>   version of Bitcoin Core. Bitcoin Core logs debug messages
>   (`-debug=`) to a stream (`-printtoconsole`) or to a file
>   (`-debuglogfile=`).
>
> * Testing the validity of a block can be achieved by specific RPCs:
>   - `submitblock`
>   - `getblocktemplate` with `'mode'` set to `'proposal'` for blocks with
> potentially invalid POW
>
> * Testing the validity of a transaction can be achieved by specific RPCs:
>   - `sendrawtransaction`
>   - `testmempoolaccept`
>
> * Wallets should not use the absence of "reject" messages to indicate a
>   transaction has propagated the network, nor should wallets use "reject"
>   messages to set transaction fees. Wallets should rather use fee
> estimation
>   to determine transaction fees and set replace-by-fee if desired. Thus,
> they
>   could wait until the transaction has confirmed (taking into account the
> fee
>   target they set (compare the RPC `estimatesmartfee`)) or listen for the
>   transaction announcement by other network peers to check for propagation.
>
> I propose to remove "reject" messages from Bitcoin Core 0.19.0 unless
> there are
> valid concerns about its removal.
>
> Marco
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Burying CSV and segwit soft fork activations

[John Newbery via bitcoin-dev]

Once a consensus change has been activated and buried by sufficient work,
we consider the height of that change to be historic fact. The exact
activation method is no longer of practical interest. In some cases the
cause of activation is not even decidable. For example, we know that segwit
activated at height 481,824 but it's debatable whether that was due to BIP
9 version bits signaling, BIP 148 UASF, or a combination of the two.

In such cases, we can significantly simplify the implementation by
hard-coding the activation height. This was done for the 3 ISM soft forks
(BIPs 34, 66 and 65) in BIP 90 [1] [2]. P2SH and segwit script enforcement
were backdated to the genesis block (with the exception of for one block)
for similar code simplification reasons [3] [4].

'Burying' deployments in this way provides a number of benefits:

1. consensus code is simplified and implementers can avoid writing and
testing code paths that are no longer relevant.
2. a hard-coded activation height is far easier to review and re-implement
than complex deployment activation logic.
3. using a non-contextual check (in this case a hard-coded constant) can
provide performance and code structure benefits (eg reducing lock
contention on blockchain data).

Bitcoin Core PR 16060 [5] was recently merged, which buries the CSV and
segwit activation heights to 419328 and 481824 respectively.

It is technically possible for this to be a non-backwards compatible
change. In the event of a re-org below the BIP9 segwit LOCKED_IN height,
this change _could_ cause a chainsplit between pre-0.19 nodes and 0.19
nodes. Such a re-org would require re-doing over 93% of the total work ever
committed to Bitcoin mining (chainwork is 0x7eb6a652531c5ad6a4b8e9 at
height 481824 compared to 0x07d75b9d25fb6602be2b51c6 at height 590393). To
quote from BIP90:

> The occurrence of such a reorg that would cause the activating block to
be disconnected would raise fundamental concerns about the security
assumptions of Bitcoin, a far bigger issue than any non-backwards
compatible change.

> So while this proposal could theoretically result in a consensus split,
it is extremely unlikely, and in particular any such circumstances would be
sufficiently damaging to the Bitcoin network to dwarf any concerns about
the effects of this proposed change.

(See the 'Considerations' section of BIP 90 for more details).

Cheers,
John

[1] https://github.com/bitcoin/bips/blob/master/bip-0090.mediawiki
[2] https://github.com/bitcoin/bitcoin/pull/8391
[3]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-January/015588.html
[4] https://github.com/bitcoin/bitcoin/pull/11739
[5] https://github.com/bitcoin/bitcoin/pull/16060
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Taproot proposal

[John Newbery via bitcoin-dev]

Hi,

> A Taproot output is a SegWit output [...]  with
> version number 1, and a 33-byte witness program whose first byte is 0 or
1.

Given a secret key k and public key P=(x,y), a signer with the knowledge of
k
can sign for -P=(x,p-y) since -k is the secret key for that point. Encoding
the
y value of the public key therefore adds no security. As an alternative to
providing the y value of the taproot output key Q when constructing the
taproot
output, the signer can provide it when signing. We can also restrict the y
value
of the internal key P to be even (or high, or a quadratic residue). That
gives
us 4 options for how to set the y signs for P and Q.

1. Q sign is explictly set in the witness program, P sign is explicitly set
in the control block
=> witness program is 33 bytes, 32 possible leaf versions (one for each
pair of 0xc0..0xff)
2. Q sign is explictly set in the witness program, P sign is implicitly even
=> witness program is 33 bytes, 64 possible leaf versions (one for each
0xc0..0xff)
3. Q sign is explictly set in the control block, P sign is explicitly set
in the control block
=> witness program is 32 bytes, 16 possible leaf versions (one for each
4-tuple of 0xc0..0xff)
4. Q sign is explictly set in the control block, P sign is implicitly even
=> witness program is 32 bytes, 32 possible leaf versions (one for pair
of 0xc0..0xff)

The current proposal uses (1). Using (3) or (4) would reduce the size of a
taproot output by one byte to be the same size as a P2WSH output. That means
that it's not more expensive for senders compared to sending to P2WSH.

(Credit to James Chiang for suggesting omitting the y sign from the public
key and
to sipa for pointing out the 4 options above)

> (native or P2SH-nested, see BIP141)

I'd prefer to not support P2SH-nested TR. P2SH wrapping was useful for
segwit
v0 for compatibility reasons. Most wallets/exchanges/services now support
sending
to native segwit addresses (https://en.bitcoin.it/wiki/Bech32_adoption) and
that
will be even more true if Schnorr/Taproot activate in 12+ months time.

On Mon, May 6, 2019 at 2:36 PM Pieter Wuille via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hello everyone,
>
> Here are two BIP drafts that specify a proposal for a Taproot
> softfork. A number of ideas are included:
>
> * Taproot to make all outputs and cooperative spends indistinguishable
> from eachother.
> * Merkle branches to hide the unexecuted branches in scripts.
> * Schnorr signatures enable wallet software to use key
> aggregation/thresholds within one input.
> * Improvements to the signature hashing algorithm (including signing
> all input amounts).
> * Replacing OP_CHECKMULTISIG(VERIFY) with OP_CHECKSIGADD, to support
> batch validation.
> * Tagged hashing for domain separation (avoiding issues like
> CVE-2012-2459 in Merkle trees).
> * Extensibility through leaf versions, OP_SUCCESS opcodes, and
> upgradable pubkey types.
>
> The BIP drafts can be found here:
> * https://github.com/sipa/bips/blob/bip-schnorr/bip-taproot.mediawiki
> specifies the transaction input spending rules.
> * https://github.com/sipa/bips/blob/bip-schnorr/bip-tapscript.mediawiki
> specifies the changes to Script inside such spends.
> * https://github.com/sipa/bips/blob/bip-schnorr/bip-schnorr.mediawiki
> is the Schnorr signature proposal that was discussed earlier on this
> list (See
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-July/016203.html
> )
>
> An initial reference implementation of the consensus changes, plus
> preliminary construction/signing tests in the Python framework can be
> found on https://github.com/sipa/bitcoin/commits/taproot. All
> together, excluding the Schnorr signature module in libsecp256k1, the
> consensus changes are around 520 LoC.
>
> While many other ideas exist, not everything is incorporated. This
> includes several ideas that can be implemented separately without loss
> of effectiveness. One such idea is a way to integrate SIGHASH_NOINPUT,
> which we're working on as an independent proposal.
>
> The document explains basic wallet operations, such as constructing
> outputs and signing. However, a wide variety of more complex
> constructions exist. Standardizing these is useful, but out of scope
> for now. It is likely also desirable to define extensions to PSBT
> (BIP174) for interacting with Taproot. That too is not included here.
>
> Cheers,
>
> --
> Pieter
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP16 enforcement change in V0.16

[John Newbery via bitcoin-dev]

The upcoming v0.16 release contains a slight change to the way that BIP16
is enforced, by basing activation on block height instead of block time.
This brings BIP 16 enforcement in line with BIP 34, BIP 66 and BIP 65.

This has no impact on consensus since BIP 16 was activated before the last
checkpoint and is buried under >300,000 blocks.

I've written up the changes in the BIP style below, although I don't think
this necessarily requires a full BIP. BIP 16 enforcement will likely change
again with https://github.com/bitcoin/bitcoin/pull/11739 in the next
bitcoin core release, so a formal proposal for this as a BIP will quickly
be superseded.


  BIP: ??
  Layer: Consensus
  Title: Buried Deployments (P2SH)
  Author: John Newbery 
  Comments-Summary: No comments yet
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-
  Status: Draft
  Type: Informational
  Created: 2018-01-22
  License: CC-0



==Abstract==

Enforce BIP 16 consensus rules based on block height rather than block time.

==Background==

BIP 16 was deployed via a hardcoded flag day consensus rule change. Prior
to the date of the consensus rule change being fixed, the miners signaled
readiness for the change by placing the string "/P2SH/" in the scriptSig of
the coinbase transaction txIn. The rule change was originally intended to
come into effect on 15 Feb 2012, but due to lack of miner signaling, the
activation date was pushed back to April 1st 2012. See [
https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki The BIP 16
specification] for full details on the deployment method. The final
activation method was via a hardcoded block time of 1333238400 (April 1st
2012).

Now that the chain has long since passed the block at which the P2SH
consensus rule was activated, we can (as a simplification) replace the
trigger mechanism by caching the block height at which those consensus
rules became enforced.

==Motivation==

Activating the BIP 16 consensus change based on block time has several
disadvantages:

* The consensus change can be activated and later deactivated in the same
chain (since block time is not necessarily monotonically increasing).
* It is less flexible for constructing test chains for testing P2SH and
other soft fork activation

The flag day activation mechanism for code deployments was deprecated in
favor of [https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki
IsSuperMajority] deployments, and later by [
https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki version
bits] deployments.

BIP 34, BIP 65, and BIP 66 deployments were later 'buried' by [
https://github.com/bitcoin/bips/blob/master/bip-0090.mediawiki BIP 90] with
simple height checks. This simplification of the consensus rules reduced
the technical debt associated with deployment of those consensus changes.

This BIP changes the BIP 16 activation method to also be a 'buried'
deployment, activating at the block height that BIP 16 actually activated.
For the mainnet chain, this is block height 173805.

==Considerations==

Just as for the buried BIP 34, BIP 65 and BIP 66 deployments, it is
technically possible for this to be a non-backwards compatible change. For
example, if an alternate chain were created in which block height 173805
was reached after April 1st 2012, and a block with height <173805 but block
time after April 1st 2012 included an invalid P2SH spend, older software
would see that chain as invalid, but newer software implementing this
change would not.

See [https://github.com/bitcoin/bips/blob/master/bip-0090.mediawiki BIP 90]
for justification of why this class of change is acceptable. As of January
2018, BIP 16 is buried by over 300,000 blocks. BIP 16 activation is also
protected by checkpoints (the most recent of which is at height 295000).

==Specification==

The BIP 16 activation height is set to 173805.

To determine whether to enforce BIP 16 on a given block, we just compare
the height of the block being validated with the stored activation height:

// Start enforcing P2SH (BIP16)
if (pindex->nHeight >= consensusparams.BIP16Height) {
flags |= SCRIPT_VERIFY_P2SH;
}

See the implementation for additional details.

==Implementation==

https://github.com/bitcoin/bitcoin/commit/18e071841e83044b47aa45c3e98c0796a407d445

==Acknowledgements==

Thanks to Russ Yanofsky, Marco Falke and Suhas Daftuar for suggestions and
feedback.

==Copyright==

This document is placed in the public domain.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev