Re: [bitcoin-dev] On mempool policy consistency
On Sun, Oct 30, 2022 at 11:02:43AM +1000, Anthony Towns via bitcoin-dev wrote: > > Some napkin math: there are about 250,000 transactions a day; if > > we round that up to 100 million a year and assume we only want one > > transaction per year to fail to initially propagate on a network where > > 30% of nodes have adopted a more permissive policy, lightweight clients > > will need to connect to over 50 randomly selected nodes.[1] > A target failure probability of 1-in-1e8 means: Oh, based on the "receive version message" log entries of a node that only does outbound connections, over the last ~3 weeks I see about 3000 outbound connections (mostly feelers/block-relay-only ones), of which a bunch identify as non-taproot supporting: 10 /Satoshi:0.16.0/: 13 /Satoshi:0.17.0/: 13 /Satoshi:0.17.0.1/: 28 /Satoshi:0.16.3/: 29 /Satoshi:0.19.0.1/: 36 /Satoshi:0.18.1/: 37 /Satoshi:0.19.1/: 39 /Satoshi:0.17.1/: 50 /Satoshi:0.20.0/: 94 /Satoshi:0.21.0/: 95 /Satoshi:0.18.0/: 244 /Satoshi:0.20.1/: Those add up to 688+ of 3065 total; if that's representative, it presumably means a random node connecting to 8 random listening peers has a 6.44-in-1-million chance of only connecting to peers that don't support taproot, ie failing your suggested threshold by a factor of about 644. Cheers, aj ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On Thu, Oct 27, 2022 at 09:29:47PM +0100, Antoine Riard via bitcoin-dev wrote:
> Let's take the contra.
(I don't think I know that phrase? Is it like "play devil's advocate"?)
> I would say the current post describes the state of Bitcoin Core and
> beyond policy
> rules with a high-degree of exhaustivity and completeness, though itt what
> is, mostly a description. While I think it ends with a set of
> recommendations
It was only intended as a description, not a recommendation for anything.
At this point, the only thing I think I could honestly recommend
that doesn't seem like it comes with massive downsides, is for core to
recommend and implement a particular mempool policy, and only have options
that either make it feasible to scale that policy to different hardware
limitations, and provide options that users can activate en-masse if it
turns out people are doing crazy things in the mempool (eg, a new policy
turns out to be ill-conceived, and it's better to revert to a previous
policy; or a potential spam vector gets exploited at scale).
> What should be actually the design goals and
> principles of Core's transaction-relay propagation rules
> of which mempool accepts ones is a subset?
I think the goals of mempool/relay policy are _really_ simple; namely:
* relay transactions from users to all potential miners, so that
non-mining nodes don't have to directly contact miners to announce
their tx, both for efficiency (your tx can appear in the next block
anyone mines, rather than just the next block you mine) and privacy
(so that miners don't know who a transaction belongs to, so that
users don't have to know who miners are, and so there's no/minimal
correlation between who proposed a tx and who mined the block it
appears in)
* having most of the data that makes up the next block pre-validated
and pre-distributed throughout the network, so that block validation
and relay is much more efficient
> By such design goals, I'm
> thinking either, a qualitative framework, like attacks game for a concrete
> application ("Can we prevent pinning against multi-party Coinjoin ?").
I don't think that even makes sense as a question at that level: you can
only ask questions like that if you already have known mempool policies
across the majority of nodes and miners. If you don't, you have to allow
for the possibility that 99% of hashrate is receiving private blacklists
from OFAC and that one of your coinjoin counterparties is on that list,
eg, and at that point, I don't think pinning is even conceivably solvable.
> I believe we would come up with a
> second-order observation. That we might not be able to satisfy every
> use-case with the standard set of policy rules. E.g, a contracting protocol
> could look for package size beyond the upper bound anti-Dos limit.
One reason that limit is in place is that it the larger the tx is
compared to the block limit, the more likely you are to hit corner cases
where greedily filling a block with the highest fee ratex txs first
is significantly suboptimal. That might mean, eg, that there's 410kvB
of higher fee rate txs than your 600kvB large package, and that your
stuff gets delayed, because the miner isn't clever enough to realise
dropping the 10kvB is worthwhile. Or it might mean that your tx gets
delayed because the complicated analysis takes a minute to run and a
block was mined using the simpler algorithm first. Or it might mean that
some mining startup with clever proprietary software that can calculate
this stuff quickly make substantially more profit than everyone else,
so they start dominating block generation, despite the fact that they're
censoring transactions due to OFAC rules.
> Or even the
> global ressources offered by the network of full-nodes are not high enough
> to handle some application event.
Blocks are limited on average to 4MB-per-10-minutes (6.7kB/second),
and applications certainly shouldn't be being designed to only work if
they can monopolise the entirety of the next few blocks. I don't think
it makes any sense to imagine application events in Bitcoin that exceed
the capacity of a random full node. And equally, even if you're talking
about some other blockchain with higher capacity; I don't really think
it makes sense to call it a "full" node if it can't actually cope with
the demands placed on it by any application that works on that network.
> E.g a Lightning Service Provider doing a
> liquidity maintenance round of all its counterparties, and as such
> force-closing and broadcasting more transactions than can be handled at the
> transaction-relay layer due to default MAX_PEER_TX_ANNOUNCEMENTS value.
MAX_PEER_TX_ANNOUNCEMENTS is 5000 txs, and it's per-peer. If you're an
LSP that's doing that much work, it seems likely that you'd at least
be running a long-lived listening node, so likely have 100+ peers, and
could conceivably simultaneously announce 500k txs distributed across
them, which at 130vB each
Re: [bitcoin-dev] On mempool policy consistency
On Fri, Oct 28, 2022 at 09:45:09PM -1000, David A. Harding via bitcoin-dev wrote: > I think this might be understating the problem. A 95% chance of having > an outbound peer accept your tx conversely implies 1 in 20 payments will > fail to propagate on their initial broadcast. Whether that's terrible or not depends on how easy it is to retry (and how likely the retry is to succeed) after a failure -- if a TCP packet fails, it just gets automatically resent, and if that succeeds, there's a little lag, but your connection is still usable. I think it's *conceivable* that a 5% failure rate could be detectable and automatically rectified. Not that I have a good idea how you'd actually do that, in a way that's efficient/private/decentralised... > Some napkin math: there are about 250,000 transactions a day; if > we round that up to 100 million a year and assume we only want one > transaction per year to fail to initially propagate on a network where > 30% of nodes have adopted a more permissive policy, lightweight clients > will need to connect to over 50 randomly selected nodes.[1] A target failure probability of 1-in-1e8 means: * with 8 connections, you need 90% of the network to support your txs * with 12 connections, you need ~79% * with 24 connections (eg everyone running a long-lived node is listening, so long lived nodes make 12 outbound and receive about ~12 inbound; shortlived nodes just do 24 outbound), you need ~54% So with that success target, and no preferential peering, you need a majority of listening nodes to support your tx's features in most reasonable scenarios, I think. > For a more > permissive policy only adopted by 10% of nodes, the lightweight client > needs to connect to almost 150 nodes. I get 175 connections needed for that scenario; or 153 with a target failure rate of 1-in-10-million. Cheers, aj ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On 2022-10-26 13:52, Anthony Towns via bitcoin-dev wrote: The cutoff for that is probably something like "do 30% of listening nodes have a compatible policy"? If they do, then you'll have about a 95% chance of having at least one of your outbound peers accept your tx, just by random chance. I think this might be understating the problem. A 95% chance of having an outbound peer accept your tx conversely implies 1 in 20 payments will fail to propagate on their initial broadcast. That seems to me like an unacceptably high failure rate both for the UX of regular payments and for the safety of time-sensitive transactions like onchain HTLC resolutions. Additionally, the less reliable propagation is, the more reliably spy nodes can assume the first IP address they received a transaction from is the creator of that transaction. I think those two problems combine in an especially unfortunate way for lightweight clients. Lightweight clients wanting to find a peer who supports a more permissive policy than most of the network and whose client authors want to provide a good UX (or safety in the case of time sensitive contract protocols like LN) will need to open large numbers of connections, increasing their chance of connecting to a spy node which will associate their IP address with their transaction, especially since lightweight clients can't pretend to be relaying transactions for other users. Some napkin math: there are about 250,000 transactions a day; if we round that up to 100 million a year and assume we only want one transaction per year to fail to initially propagate on a network where 30% of nodes have adopted a more permissive policy, lightweight clients will need to connect to over 50 randomly selected nodes.[1] For a more permissive policy only adopted by 10% of nodes, the lightweight client needs to connect to almost 150 nodes. This also implies that nodes adopting a more restrictive policy degrades UX, safety, and privacy for users of transactions violating that policy. For example, if 30% of nodes used Knots's -spkreuse configuration option and about 50% of transactions reuse scriptPubKeys, then about 9 transactions a day wouldn't initially propagate (assuming 8 randomly selected peers[2]) and lightweight clients who wanted 1-in-100-million safety would need to connect to about 15 random nodes. Towns's post to which I'm replying describes several alternative approaches which mitigate the above problems, but he also documents that they're not without tradeoffs. -Dave [1] (1-0.3)**50 * 100_000_000 =~ 1.8 [2] That assumes every transaction is sent to a different randomly-selected set of peers, which isn't really the case. However, one day $GIANT_EXCHANGE could suddenly be unable to broadcast hundreds or thousands of withdrawal transactions because all of its peers implement a restrictive policy. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
