Re: [Lightning-dev] Fee Ratecards (your gateway to negativity)

[lisa neigut]

Some interesting points here. Will try to respond to some of them.

> pathfinding algorithms which depend on unscalable data collection

Failed payment attempts are indistinguishable from data collection probing.
I don't think it's realistic to think that payments are going to stop
failing in
an environment where we continue to prioritize privacy of channel balances.

I think I pretty nicely illustrated the benefits of using payment bands in
my original
post wrt to a balance between "probe-ability" and some obfuscation of
balance, along
with a net reduction in the bandwidth consumption required to get this
basically
required information for payment success.

Rene Pickhardt has a less private, less costly, more granular, smaller scope
proposal for sharing channel balances w/ directly connected peers; I think
this is worth investigating independently of this proposal.

> depend on the centralized entities performing the data collection

I like to think that the introduction of negative fees make channel
balance data a competitive advantage and will actually cause node
operators to more closely guard their balances / the balance data
they've collected about peers, which should hopefully reduce the current
trend of sharing this information with centralized parties.

Note that with the present protocol design, the network incentives are such
 that centralized efforts to collect exact balance data already exist. So
moving to this design has the potential to reduce the incentive to
participate
in the data collection, at the very least it does not make it worse than
current.

>  this costs Alice nothing extra but reduces network capacity by consuming
>  HTLC slots for her, Bob, and every other forwarding channel.

the network already limits the size of htlcs that are allowable with the
htlc_maximum_msat, which I understand is typically set at a rate below 1/4
of channel capacity (citation needed). Which is to say that the large
consumption of a channel in a single HTLC is currently relatively
prohibited.
It's likely this proposed change would actually encourage operators to set
their htlc_maximum_msat higher, now that there's a direct financial cost
tied to larger channel bandwidth consumption.

Great questions! Hope that gives a better picture of the current landscape.

Also h/t to ZmnSCPxj for the excellent "four parallel channels" analogy,
this is
a really elegant way to think about the proposal.

On Thu, Sep 22, 2022 at 11:39 PM David A. Harding  wrote:

> On 2022-09-22 16:08, ZmnSCPxj wrote:
> > Basically, you can model a rate card as four separate channels between
> > the same two nodes, with different costs each.
> > If the path at the lowest cost fails, you just try at another route
> > that may have more hops but lower effective cost, or else try the same
> > channel at a higher cost.
>
> That's a very easy to understand explanation of how to use the system,
> thanks!
>
> > If your concern is valid, one wonders why it would not already exist
> > now in the current network where try-and-try-again is the standard
> > overall algorithm for payments.
>
> My concern is about pathfinding algorithms which depend on unscalable
> data collection (e.g. frequent whole network probing).  If such an
> algorithm performs much better than those algorithms which depend on
> scalable data collection (e.g. receiving gossip), then the network may
> grow to depend on the centralized entities performing the data
> collection to the detriment of its robustness and its participants'
> independence.
>
> Try-and-try-another-path is somewhat problematic in this regard since,
> even with no additional action like probing, entities which send many
> payments will likely perform significantly better than entities which
> send few payments owing to the high-frequency spenders gaining more
> knowledge about which channels and amounts recently worked or did not.
> It seems to me that a more idealized system would only rarely have
> forwarding failures so that high frequency spenders wouldn't receive
> much more information than low frequency spenders.  To that regard, fee
> ratecards feels like a small step in the wrong direction because
> modeling one channel as four separate channels further normalizes
> failure and so further moves the system towards centralized dependency.
> That said, as you mentioned in a previous post[1], I agree ratecards is
> better than frequently issuing new channel updates with modified fees.
>
> Thanks,
>
> -Dave
>
> [1]
>
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-June/003598.html
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

[Lightning-dev] CLN Release: v0.12.1 - Web 8 init (dot one)

[lisa neigut]

Hi all,

We'd like to announce the 0.12.1 release[1] of core-lightning, named by
@adi2011.

This is a point release with a few bug fixes and build improvements: it's a
recommended upgrade from 0.12.0, with no new features.

For a list of differences, please see [v0.12.0..v0.12.1](
https://github.com/ElementsProject/lightning/compare/v0.12.0...v0.12.1).

## Highlights for Users

- Now with fewer bugs!
- Topology plugin could crash when it sees duplicate private channel
announcements.
- `getinfo` no longer displays unannounced addresses; shows correct port
for discovered IP addresses.
- Newly created channels sometimes were inexplicably disconnected, this has
been fixed!

## Highlights for the Network

- Bad gossip store reads and gossip propagation fixed.

## Highlights for Developers

- Reliance on `mako` and `mrkd` python libraries has been removed.

Since v0.12.0 we've had 26 commits from 5 different contributors over 29
days.

~ @niftynei, Christian, and Rusty

[1] https://github.com/ElementsProject/lightning/releases/tag/v0.12.1
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

[Lightning-dev] Fee Ratecards (your gateway to negativity)

[lisa neigut]

Hi all,

I've been talking about them for a bit[^1], now[^2], but here's a more
formal proposal for fee ratecards.

Spec PR + implementation to come.

## What is a fee ratecard?

A ratecard is a set of four values, positive or negative, that price different
bands of available liquidity for a channel.

A ratecard replaces the current `fee_proportional_millionths` and
`fee_base_msat` currently used to calculate the owed fees for
forwarding a payment on the
lightning network.

Instead, with ratecards, a channel operator may specify four different
rates for a channel's liquidity, that will automatically be updated
depending on the current channel capacity. The four capacity bands are fixed
at 0-25%, 26-50%, 51-75%, and 76-100%.

Currently, all payments are priced equally. Fee ratecards update this such that
payments can be priced differently, depending on the current available
capacity in a channel. This allows node operators to dynamically price
their liquidity, without needing to continually issue gossip updates as
the balance changes (which may leak payment flow timing).

## Spec Changes + Implementation Notes

To the `channel_update` gossip message, we add a TLV with one defined
type, `ratecard`. The ratecard contains 4 16-bit signed integers, one
for each quarter of channel capacity. (0-25, 26-50, 51-75, 76-100%).

```
1. `tlv_stream`: `channel_update_tlvs`
2. types:
1. type: 1 (`ratecard`)
2. data:
* [`s16`:`fee_basis_0_25`]
* [`s16`:`fee_basis_26_50`]
* [`s16`:`fee_basis_51_75`]
* [`s16`:`fee_basis_76_100`]
```

If a node is advertising a `ratecard` in their `channel_update` for
that channel, a routing node SHOULD pick the rate to pay based on
their current guess of the channel's capacity as well as priority for
the payment.

If a payment is high priority, it is recommended to pay at the highest feeband,
to ensure delivery (if possible).

On payment failure, the response is the same (you return a copy of the
`channel_update` in the onion). You MAY give an additional hint as to
what the current
acceptable feerate is for a payment.

If the feerate card entry is set, the rates there take precedence over the
`fee_base_msat` and the `fee_proportional_millionths`.

The `feerate` card effectively removes the ability to set a `fee_base_msat`.

To minimize payment routing failure, it is recommended to set the
`fee_proportional_millionths` to the same rate as the highest `ratecard`
rate. (Most likely the `ratecard`.`fee_basis_0_25`).

Note that `ratecards` unit is in basis points, not millionths. As basis points
are 1/10,000, they're 100x less than the same value expressed as millionths.

- A `basis_point` value of 5 is a fee of 5msat on a 10k msat payment.
- A `fee_proportional_millionths` of 500 is a fee of 5msat on a 10k
msat payment.

The current expected price for sats to be pushed to the other side of the
channel is to be priced by the lowest ratecard capacity band that that payment
will push the balance into.

For example, if a payment moves 50k through a 100k channel which is currently
at a total available capacity of 75k sats (which means it'll move the capacity
from 75k to 25k), that payment will be expected to pay a rate equal to the
0-25% band, as it'll push the available capacity into the 0-25% range.

Using this rule, HTLCs consuming significant portions of a channel's
total capacity are more likely to pay higher fees.

### Motivation and Rationale
At the last in-person spec meeting, it was noted that channel capacity is
often of variable value depending on the amount of capacity currently
available in that channel.

It was also noted that we'd like to allow node operators to experiment
with negative fees.

You can't easily allow for negative feerates with our current gossip messages,
as it would give gossip an economic value which may have an adverse impact
on the ability of nodes to stay up to date with the current lightning channel
topology; it also stands to greatly increase the volume of gossip
updates issued by any one node, which should scale with payment volume,
as nodes update their gossip to change their current fees on any one channel.

Note that some nodes already automatically update their channel fees based on
the available capacity in a channel; fee ratecards should be a more succinct
way for node operators to express more fine grained prices of their existing
capacity, with a much reduced bandwidth burden.

### Feerate Cards, Frequently Asked Questions

 Why four evenly spaced buckets? Why not X function?

1. Expressibility
Fees are calculated by the node creating the route for a payment, using
parameters set by the channel node operator. These paramters must be
communicated via gossip to every node, and then used as inputs to the
routing function.

Parameters need to be uniform and well understood such that
routing algorithm designers can make efficient use of them.

Four 

[Lightning-dev] Core-Lightning Release v0.12.0 [Web 8-init]

[lisa neigut]

We're pleased to announce the 0.12.0 release of core-lightning, named by
@adi2011.

**Developers**: please note the Great Msat Migration in the APIs!

## Highlights for Users

- *NEW* Built-in `bookkeeper` plugin! This plugin tracks all movements of
msats for your node, gives you a better idea of your costs and revenues,
prints out CSVs that are uploadable to Koinly and CoinTracker, lets you
inspect the on-chain footprint of a channel (useful when it goes to chain).
Check out the new `bkpr-` prefixed commands.
- *NEW* Built-in `commando` plugin! This lets you create runes to allow
access to your node from a commando client, which will let you send and
receive RPC commands over the lightning network.
- *NEW* Emergency channel backup ("static backup")! Keep track of what
peers you have channels with, and in case of node failure ask those peers
to close the channel.
- *NEW* zeroconf channels are possible for whitelisted peers.
- `hsmtool` has a new command, `checkhsm`, which will let you check a BIP30
passphrase against the `hsm_secret`.
- Multiple `log-file` options will open multiple files for logging.
- Various crashes and issues fixed in `connectd` including crash on peer
reconnect and large memory usage when many concurrent peers.
- PSBT: fixes signature encoding to comply with BIP-0174.
- We added dynamically detected public IP addresses to `getinfo`.
- Due to dependency issues on some platforms, a tarball of pre-generated
manual pages is included with this release.


## Highlights for the Network

- We prefer IPv6 connections when available.
- We now accept spam gossip and use it for routing, but don't relay it.
- We no longer create gossip messages with zlib encoding (but still
understand them).
- We treat LND "internal error" as warnings, not force close events
(reverts to v0.10.0 behavior).


## Highlights for Developers

- `_msat` fields are added wherever they were missing in the API: they're
still currently an "msat"-suffixed string, but will soon bean integer
value. Test with deprecated_apis=false.
- The `channel_state_changed` notification now fires when a channel moves
into state `CHANNELD_AWAITING_LOCKIN`.
- `htlc_accepted_hook` will now expose the `short_channel_id` and the
per-channel HTLC `id`.
- `pyln-testing` now includes utilities to read and parse the gossip\_store.
- `startup_regtest.sh` script now includes a `fund_ln` method.
- Rust binaries such as `cln-grpc` now included in our reproducible builds.
- Updated the bolts implementation for pyln-spec.
- Plugins no longer hang indefinitely if `lightningd` closes their
connection.
- M1 architecture support.
- Upgrade docker base image from Debian buster to bullseye, works with
glibc 2.29+.
- Docker images now built with rust plugin `cln-grpc`.


Since v0.11.1 we've had 508 commits from 31 different contributors over 80
days.

A special thanks goes to 9 first time contributors:

- Aditya Sharma
- Alex Myers
- Igor Bubelov
- Justin Moon
- Peter Neuroth
- Swapnil
- Jose A.P.
- Brian Barto
- AutonomoousOrganization


~ @niftynei, Christian, and Rusty
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Achieving Zero Downtime Splicing in Practice via Chain Signals

[lisa neigut]

Had another thought: if you've seen a chain close but also have a gossip
message that
indicates this is a splice, you SHOULD propagate that gossip more
urgently/widely than
any other gossip you've got. Adding an urgency metric to gossip is fuzzy to
enforce... *handwaves*.

You *do* get the onchain signal, we just change the behavior of the
secondary information system
instead of embedding the info into the chain..

"Spamming" gossip with splices expensive -- there's a real-world cost
(onchain fees) to
closing a channel (the signal to promote/prioritize a gossip msg) which
cuts down on the ability to send out these 'urgent' messages with any
frequency.

~nifty

On Wed, Jun 29, 2022 at 7:43 PM lisa neigut  wrote:

> Adding a noticeable on-chain signal runs counter to the goal of the move
> to taproot / gossip v2, which is to make lightning's onchain footprint
> indistinguishable from
> any other onchain usage.
>
> I'm admittedly a bit confused as to why onchain signals are even being
> seriously
>  proposed. Aside from "infallibility", is there another reason for
> suggesting
> we add an onchain detectable signal for this? Seems heavy handed imo,
> given
> that the severity of a comms failure is pretty minimal (*potential* for
> lost routing fees).
>
> > So it appears you don't agree that the "wait N blocks before you close
> your
> channels" isn't a fool proof solution? Why 12 blocks, why not 15? Or 144?
>
> fwiw I seem to remember seeing that it takes  ~an hour for gossip to
> propagate
> (no link sorry). Given that, 2x an hour or 12 blocks is a reasonable first
> estimate.
> I trust we'll have time to tune this after we've had some real-world
> experience with them.
>
> Further, we can always add more robust signaling later, if lost routing
> fees turns
> out to be a huge issue.
>
> Finally, worth noting that Alex Myer's minisketch project may well
> help/improve gossip
> reconciliation efficiency to the point where gossip reliability is less
> of an issue.
>
> ~nifty
>
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Achieving Zero Downtime Splicing in Practice via Chain Signals

[lisa neigut]

Adding a noticeable on-chain signal runs counter to the goal of the move
to taproot / gossip v2, which is to make lightning's onchain footprint
indistinguishable from
any other onchain usage.

I'm admittedly a bit confused as to why onchain signals are even being
seriously
 proposed. Aside from "infallibility", is there another reason for
suggesting
we add an onchain detectable signal for this? Seems heavy handed imo, given
that the severity of a comms failure is pretty minimal (*potential* for
lost routing fees).

> So it appears you don't agree that the "wait N blocks before you close
your
channels" isn't a fool proof solution? Why 12 blocks, why not 15? Or 144?

fwiw I seem to remember seeing that it takes  ~an hour for gossip to
propagate
(no link sorry). Given that, 2x an hour or 12 blocks is a reasonable first
estimate.
I trust we'll have time to tune this after we've had some real-world
experience with them.

Further, we can always add more robust signaling later, if lost routing
fees turns
out to be a huge issue.

Finally, worth noting that Alex Myer's minisketch project may well
help/improve gossip
reconciliation efficiency to the point where gossip reliability is less
of an issue.

~nifty
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

[Lightning-dev] c-lightning discord community

[lisa neigut]

FYI c-lightning now has a discord server for general questions, dev-chats,
and community support.

You can join it here: https://discord.gg/WW56GGHavu

We're also still on Telegram at https://t.me/lightningd and IRC at
#c-lightning on the libera.chat node.

~nifty
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Removing lnd's source code from the Lightning specs repository

[lisa neigut]

Love the idea of moving the specs etc to github.com/lightning, thanks so
much for generously offering to donate this Laolu. Strong ACK from me.

Given how difficult the existing org is wrt ownership etc, moving to a new
one makes a lot of sense to me.

Thanks Fabrice for bringing this up so we could discuss it and get a better
understanding of the difficulties with the existing situation.

- nifty

On Wed, Oct 13, 2021 at 00:48 Damian Mee  wrote:

> While I don't partake in the conversations too often, I just want to say I
> strongly support Olaoluwa suggestion.  AFAIAA there's a lot of automations,
> dependencies, dockerfiles, or direct links to files that rely on the
> location of lnd github repo, and I'm sure not all of it would be able to
> handle Github redirect gracefully.  While accessing spec is mostly a
> human/browser activity, where dealing with redirects is much easier.
>
> Plus, github.com/lightning/spec, github.com/lightning/bolts, and/or
> github.com/lightning/rfc would all be trivial to remember, and quick to
> type.
>
> On Tue, Oct 12, 2021 at 2:57 PM Olaoluwa Osuntokun 
> wrote:
>
>> Hi Fabrice,
>>
>> > I believe that was a mistake: a few days ago, Arcane Research published
>> a
>> > fairly detailed report on the state of the Lightning Network:
>> > https://twitter.com/ArcaneResearch/status/1445442967582302213.  They
>> > obviously did some real work there, and seem to imply that their report
>> > was vetted by Open Node and Lightning Labs.
>>
>> Appreciate the hard work from Arcane on putting together this report. That
>> said, our role wasn't to review the entire report, but instead to provide
>> feedback on questions they had. Had we reviewed the section in question,
>> we
>> would have spotted those errors and told the authors to fix them. Mistakes
>> happen, and we're glad it got corrected.
>>
>> Also note that lnd has _never_ referred to itself as the "reference"
>> implementation.  A few years ago some other implementations adopted that
>> title themselves, but have since adopted softer language.
>>
>> > So I'm proposing that lnd's source code be removed from
>> > https://github.com/lightningnetwork/ (and moved to
>> > https://github.com/lightninglabs for example, with the rest of their
>> > Lightning tools, but it's up to Lightning Labs).
>>
>> I think it's worth briefly revisiting a bit of history here w.r.t the
>> github
>> org in question. In the beginning, the lightningnetwork github org was
>> created by Joseph, and the lightningnetwork/paper repo was added, the
>> manuscript that kicked off this entire thing. Later lightningnetwork/lnd
>> was
>> created where we started to work on an initial implementation (before the
>> BOLTs in their current form existed), and we were added as owners.
>> Eventually we (devs of current impls) all met up in Milan and decided to
>> converge on a single specification, thus we added the BOLTs to the same
>> repo, despite it being used for lnd and knowingly so.
>>
>> We purposefully made a _new_ lightninglabs github org as we wanted to keep
>> lnd, the implementation distinct from any of our future commercial
>> products/services. To this day, we've architected all our paid products to
>> be built _on top_ of lnd, rather than within it. As a result, users always
>> opt into these services.
>>
>> As it seems the primary grievance here is collocating an implementation of
>> Lightning along with the _specification_ of the protocol, and given that
>> the
>> spec was added last, how about we move the spec to an independent repo
>> owned
>> by the community? I currently have github.com/lightning, and would be
>> happy
>> to donate it to the community, or we could create a new org like
>> "lightning-specs" or something similar. We could then move the spec (the
>> BOLTs and also potentially the bLIPs since some devs want it to be within
>> its own repo) there, and have it be the home for any other
>> community-backed/owned projects.  I think the creation of a new github
>> organization would also be a good opportunity to further formalize the set
>> of stakeholders and the general process related to the evolution of
>> Lightning the protocol.
>>
>> Thoughts?
>>
>> -- Laolu
>>
>> On Fri, Oct 8, 2021 at 5:25 PM Fabrice Drouin 
>> wrote:
>>
>>> Hello,
>>>
>>> When you navigate to https://github.com/lightningnetwork/ you find
>>> - the Lightning Network white paper
>>> - the Lightning Network specifications
>>> - and ... the source code for lnd!
>>>
>>> This has been an anomaly for years, which has created some confusion
>>> between Lightning the open-source protocol and Lightning Labs, one of
>>> the companies specifying and implementing this protocol, but we didn't
>>> do anything about it.
>>>
>>> I believe that was a mistake: a few days ago, Arcane Research
>>> published a fairly detailed report on the state of the Lightning
>>> Network: https://twitter.com/ArcaneResearch/status/1445442967582302213.
>>> They obviously did some real work there, and seem 

Re: [Lightning-dev] Full Disclosure: CVE-2021-41591/ CVE-2021-41592 / CVE-2021-41593 "Dust HTLC Exposure Considered Harmful"

[lisa neigut]

FYI the next version of c-lightning will contain the proposed
`max_dust_htlc_exposure_msat` as outlined in #919
; the
given expected vulnerabilities patch table should have reflected this.

> The vulnerabilities are expected to be patched in:
> * Eclair: v0.6.2+ (CVE-2021-41591)
> * LND: v0.13.3+ (CVE-2021-41592)
> * LDK: v0.0.102 (not released as production software yet)

* C-lightning v0.10.2 (CVE-2021-41593)


Lisa

On Mon, Oct 4, 2021 at 10:09 AM Antoine Riard 
wrote:

> Hi,
>
> I'm writing a report to disclose specification-level vulnerabilities
> affecting the Lightning implementations.
>
> The vulnerabilities are expected to be patched in:
> * Eclair: v0.6.2+ (CVE-2021-41591)
> * LND: v0.13.3+ (CVE-2021-41592)
> * LDK: v0.0.102 (not released as production software yet)
>
> The vulnerabilities are also affecting c-lightning (CVE-2021-41593).
>
> Those vulnerabilities can be exploited in a wide range of attacks, going
> from fee blackmailing of node operators, burning liquidity of your
> competing LSPs or even stealing your counterparty channel balance if you
> avail mining capabilities. Exercise of the vulnerability revealed that a
> majority of the balance funds can be at loss.
>
> Credit to Eugene Siegel (Crypt-iQ) for reporting the trimmed-to-dust
> exploitation and multiple insights about attacks.
>
> Thanks to Bastien Teinturier and Matt Corallo for numerous contributions
> about mitigations development.
>
> # Problem
>
> The current BOLT specification only requires Alice's `dust_limit_satoshis`
> (applied on Alice's commitment) to be under Alice's
> `channel_reserve_satoshis` (applied on Bob). As those 2 parameters are
> selectable by Alice, she can inflate the dust limit until reaching the
> implementation-defined max value (e.g LND: 20% of chan capacity, LDK: 100%
> of chan capacity).
>
> Any in-flight incoming HTLC under Alice's dust limit will be converted as
> miner fees on Alice's commitment. This HTLC is deducted from Bob's balance
> and as such they're still owned by Bob, until resolution (i.e a RAA
> removing the HTLC from Alice's commitment). This limitation only applies
> per-HTLC. No implementation enforces a limit on the sum of in-flight HTLCs
> burned as fees. Therefore, Alice is free to inflict a substantial loss to
> Bob funds by publishing her commitment on-chain.
>
> In-flight outgoing HTLC are also committed as fees on Bob's commitment if
> they're under Bob's threshold. Alice can also exploit from this angle by
> circular routing HTLCs until reaching Bob's
> `max_htlc_value_in_flight_msat`. Alice withholds HTLCs resolution until Bob
> goes on-chain to timeout an offered HTLC or claim an accepted HTLC.
>
> Dust HTLC processing can be also exploited at `update_fee` reception.
>
> As the BOLT3's fees computation encompasses the negotiated feerate from
> `update_fee` for the 2nd-stage HTLC fees to decide if the HTLC must be
> trimmed, the amount of balance at risk is a function of current mempool
> feerates.
>
> The maximum of funds at risk on a counterparty commitment is:
>
> counterparty's `max_accepted_htlcs` * (`htlc_success_tx_kw` * opener's
> `feerate_per_kw` + counterparty's `dust_limit_satoshis`) + holder's
> `max_accepted_htlcs` * (`htlc_timeout_tx_kw` * opener's `feerate_per_kw` +
> counterparty's `dust_limit_satoshis`)
>
> If the opener is also the attacker, the negotiated feerate can be
> manipulated beyond the "honest" mempool feerates only upper bounded
> implementation-defined value (before fixes, LDK: 2 * high-feerate of our
> fee-estimator). If the opener is the victim, the negotiated feerate is
> still a safety concern in case of spontaneous mempool spikes.
>
> Note, `anchors_zero_htlc_fee` channels are not affected by the feerate
> inflation as the trimmed-to-dust fee computation mechanism for 2nd-stage
> HTLC is removed. They're still at risk of the sum of the HTLCs under the
> dust limit being maliciously burned.
>
> # Solution
>
> A first mitigation is to verify the counterparty's announced
> `dust_limit_satoshis` at channel opening (`open_channel`/`accept_channel`)
> reception and reject if it's estimated too large (see #894)
>
> For LDK, we choose the value of 660 satoshis as it's beyond the highest
> dust threshold enforced by Bitcoin Core (p2pkh: 546) with a margin of
> safety. Propagation of Lightning time-sensitive transactions shouldn't be
> affected.
>
> A second mitigation is to define a new configurable limit
> `max_dust_htlc_exposure` and apply this one at incoming and outgoing of
> HTLC.
>
> For LDK, we choose the value of 5 000 000 milli-satoshis as we gauged this
> value as a substantial loss for our class of users. Setting this too low
> may prevent the sending or receipt of low-value HTLCs on high-traffic
> nodes. A node operator should fine-tune this value in function of what
> qualifies as an acceptable loss.
>
> We would like to ensure that the node isn't suddenly exposed 

Re: [Lightning-dev] #zerobasefee

[lisa neigut]

The field of economics has done much work over the past few decades
demonstrating that “Free” is problematic in practice because humans will go
out of their way to externalize costs elsewhere (e.g. time, in the case of
lightning), given the promise of freedom. In other words, actors often act
irrationally to get a free deal.

As protocol designers, it would be remiss to ignore this (repeatedly
demonstrated) truth.

To avoid this, we’ve been suggesting setting a min_htlc_value requirement.
The problem with zbf + a min_htlc size requirement is that it makes tiny
payments impossible over lightning, which was one of the original design
goals of the system, and is an important feature to keep/support as
lightning grows into poorer economic bases and the bitcoin market price
continues to rise.

My suggestion would be that, as a compromise, we set a network wide minimum
fee at the protocol level of 1msat. Naively, this seems it should be easy
to add to calculations using single-dimension optimization (or trivial
enough to ignore entirely), it removes the “free lunch” irrationality
honeypot zbf opens, and it provides a way forward for the continued use of
micropayments.

The result is that micropayments have a different payment regime than
“non-micropayments”, (which may still incentive almost irrational behavior)
but at least there’s no *loss* felt by node operators for
handling/supporting low value payments. 10k micropayments is worth 10sats.

It’s also simple to implement and seems rather obvious in retrospect.

The only confounding future change that I can see us making would be the
introduction of negative fees, which are useful as a way to induce payments
to rebalance channels passively. This seems like something we can revisit
once a proposal for negative fees is being seriously considered, however.

On Sun, Aug 15, 2021 at 05:59 ZmnSCPxj via Lightning-dev <
lightning-dev@lists.linuxfoundation.org> wrote:

> Good morning aj, et al.
>
> > Hey *,
> >
> > There's been discussions on twitter and elsewhere advocating for
> > setting the BOLT#7 fee_base_msat value [0] to zero. I'm just writing
> > this to summarise my understanding in a place that's able to easily be
> > referenced later.
> >
> > Setting the base fee to zero has a couple of benefits:
> >
> > -   it means you only have one value to optimise when trying to collect
> > the most fees, and one-dimensional optimisation problems are
> > obviously easier to write code for than two-dimensional optimisation
> > problems
>
> Indeed, this is a good point regarding this.
>
>
> > -   when finding a route, if all the fees on all the channels are
> > proportional only, you'll never have to worry about paying more fees
> > just as a result of splitting a payment; that makes routing easier
> > (see [1])
>
> If we neglect roundoff errors.
>
> On the other hand, roundoff errors involved are <1msat per split, so it
> probably will not matter to most people.
>
> > So what's the cost? The cost is that there's no longer a fixed
> minimum
> > fee -- so if you try sending a 1sat payment you'll pay 0.1% of the
> fee
> > to send a 1000sat payment, and there may be fixed costs that you have
> > in routing payments that you'd like to be compensated for (eg, the
> > computational work to update channel state, the bandwith to forward
> the
> > tx, or the opportunity cost for not being able to accept another
> htlc if
> > you've hit your max htlcs per channel limit).
> >
> > But there's no need to explicitly separate those costs the way we do
> > now; instead of charging 1sat base fee and 0.02% proportional fee,
> > you can instead just set the 0.02% proportional fee and have a
> minimum
> > payment size of 5000 sats (htlc_minimum_msat=5e6, ~$2), since 0.02%
> > of that is 1sat. Nobody will be asking you to route without offering
> a
> > fee of at least 1sat, but all the optimisation steps are easier.
>
> Should this minimum a node will be willing to forward be part of gossip,
> and how does this affect routing algorithms?
>
> > You could go a step further, and have the node side accept smaller
> > payments despite the htlc minimum setting: eg, accept a 3000 sat
> payment
> > provided it pays the same fee that a 5000 sat payment would have.
> That is,
> > treat the setting as minimum_fee=1sat, rather than
> minimum_amount=5000sat;
> > so the advertised value is just calculated from the real settings,
> > and that nodes that want to send very small values despite having to
> > pay high rates can just invert the calculation.
>
> I like this idea, as I think it matches more what the incentives are.
> But it requires a change in gossip and in routing algorithms, and more
> importantly it requires routing algorithms to support two different fee
> schemes (base + proportional vs min + proportional).
>
> On the other hand, this still is a two-dimensional optimization algorithm,
> 

[Lightning-dev] v0.10.1: "eltoo: Ethereum Layer Too"

[lisa neigut]

We're pleased to announce the 0.10.1 release of c-lightning
, named
by @nalinbhardwaj.

This is a *recommended* upgrade: payment secrets in invoices are now
compulsory, and offers and dual funding drafts have been updated, so these
(experimental) features are incompatible with previous releases!

*NOTE*: Users of the rebalance or drain [plugins](
https://github.com/lightningd/plugins) MUST update, as payment secret is
now compulsory.

## Highlights for Users

- `experimental-dual-fund` allows advertizement of funding rates which we
will contribute to channels automatically, on a 1-month lease.
- `withdraw` and `close` (if peer supports) now supports Taproot (and other
future) addresses.
- `listchannels` can now be queried by destination, as well as source.
- `plugin rescan` now automatically reloads plugins which have changed.
- We try to restart automatically if we notice subdaemons have been
upgraded underneath us.
- `fundpsbt` will no longer include uneconomic UTXOs (unless `all`).
- `close` will return a stream of notifications if there is a delay in
closing.
- Unilateral close feerates were reduced from bitcoind's "2 CONSERVATIVE"
to "6 ECONOMICAL".
- Tor v2 is deprecated: please upgrade to v3!
- Fixed disconnection bug when an HTLC failed.
- Fixed bug in rapid feerate changes, and make code space them out (LND
compat).
- Fixed bug where Tor on different ports could be advertized incorrectly.
- Fixed various bugs to make `pay` more robust.

## Highlights for the Network

- payment secrets in invoices are now compulsory, finally closing a
potential probing (or, with amountless invoices, stealing) attack.
- `option_shutdown_anysegwit` allows peers to close channels to any future
segwit version address (taproot!).
- `keysend` now understands routehints, for routing to unpublished nodes,
and sets the final CLTV to 22, for rust-lightning nodes.
- `invoice` now allows creation of wumbo invoices (> 0.0429 BTC).
- We will now discuss old channels with peers who reconnect, even if we
consider them closed.


## Highlights for Developers

- Manual pages now document *exactly* the JSON you can expect from each
command (and it's tested!)
- Plugins can now publish notifications for other plugins to listen to.
- `force-feerates` allows complete feerate override (mainly for regtest),
and a bug fixed where we could send 0 update_fee on regtest.
- The HSM daemon is now separated into libhsmd, which also provides Python
bindings.
- `createonion` can now make variable-sized onions, and `sendonion` no
longer requires `direction` and `channel` for `firsthop`.
- `dev-sendcustommsg` is now simply `sendcustommsg`.
- Many offers API improvements and updates, including unsigned offers
(smaller QR codes!).

More details can be found in the [changelog](
https://github.com/ElementsProject/lightning/blob/v0.10.1/CHANGELOG.md).

Thanks to everyone for their contributions and bug reports; please keep
them coming.

Since 0.10.0, we've had 526 commits from 15 different authors over 114 days.

A special thanks goes to the 6 first time contributors:

- Nalin Bhardwa
- Nathanael
- LightningHelper
- OpenOms
- Urza
- Valentine Wallace

Cheers,
Lisa, Christian,  ZmnSCPxj, Rusty
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

[Lightning-dev] Interactive tx construction and UTXO privacy, some thoughts

[lisa neigut]

Hey all,

The dual-funding proposal has been up for a while now, and while we've had
some really great reviews from a few people (@ariard + @rustyrussell thanks
for
your comments in particular).

As a refresher, the PR is available here:
https://github.com/lightningnetwork/lightning-rfc/pull/851.

I wanted to make a quick summary of some of the biggest objections to
the protocol. Most of these were made during the course of the 2018 spec
meeting in Adelaide and while we've done a decent job of architecting
around them,
I wanted to make them explicit.

If there's more to add, please do.

## Point One
One: dual-funding (really the collaborative transaction protocol) requires
you to share your UTXO set with the channel peer.

This is true, in a 'limited' sense, where 'UTXO set' is actually constricted
to 'UTXOs you're using for funding transactions'.

On the face of it, this seems to be quite concerning to folks.  Here's a
few things
that I think make this less concerning than it might appear at the outset.

First, let's consider the current opening case. You (or your channel peer)
constructs a transaction and shares the TXID and outpoint that the funding
output can be found at. This transaction is published and mined,
after the transaction is at sufficient depth a gossip announcement
is broadcast to the entire network, explaining where the transaction can be
found.

Under the current protocol, it's reasonable\* to conclude that
every UTXO in the funding transaction belongs to a single party, the opener.

Assuming that all the UTXOs in your wallet will, at some point, end up
in a lightning channel, all of your UTXOs will be *publicly* associated with
your node at some point (in the current model). Your UTXO set is only
private in the present, given a long enough time period (and channel opens)
they will all be tied to your node.

For this reason, it's probably not a great idea to fund wallets using a
separate wallet that hasn't been coinjoined or otherwise obfuscated. In
fact, I think you can make a fairly decent argument (under the existing UTXO
model) for not mixing lightning funds with any funds you wish to remain
truly
private/unassociated to an 'entity'.

Given that you're currently telling everyone what UTXOs your node owns
(or at least has a 50/50 chance of owning..) let's consider the interactive
transaction protocol.
Assuming a 2-party dual-funded open, as an outside observer it's much more
difficult to tell which
UTXOs belong to who (though you can probably guess given input amounts and
change outputs; maybe a good heuristic for opens is to use whole UTXOs).

In this on-chain respect, the dual-funding protocol is no worse, and in the
best case much
better for on-chain coin-analysis heuristics than the current protocol.

\* There are a few clear exceptions where other protocols (such as Pool,
iiuc)
have been developed that make this heuristic bunk as they *also* build a
multi-party
transaction, however afaik all of these other protocols (joinmmarket, pool)
still require sharing
your UTXOs with another (trusted) party, difference is you know in theory
where "the Pool
auctioneer" lives, whereas you don't really know much about "any random
node that wants to open
a channel". Seems like there might be some room in this space for better
information
about prospective channel peers...

## Point Two
Two: an active prober can use the interactive transaction protocol to
discover your wallet's entire UTXO set.

Given the assumption that all of your node funds will, at some point,
be used to open a lightning channel, a single attacker discovering a subset
of your UTXOs is simply a 'fast-forward' of what will eventually become
public information.

That being said, much smarter minds (e.g. JoinMarket, which suffers from
a very similar issue in what I would argue an even more privacy-conscious
context) than mine have employed the PoDLE protocol to make the discovery
of every UTXO more costly for a potential attacker. This is something
 we can add quite trivially to the protocol, at the cost of grossly
increasing
the gossip traffic.

Worth noting that even without PoDLE you can much slow this type of
attack by reusing the same UTXO across every open attempt, being choosy
about which peers you do offer liquidity up to (do they have other
established
channels? what's their uptime?) etc.

A dedicated attacker could probably figure out your UTXO set, but that's not
much different from the current system; the only difference is the span of
time
it takes them to figure it out.

## Things We've Done to Counter This:
I had the pleasure of finally meeting Nadav of SuredBits and DLC fame in
Miami
a few weeks ago. The DLC team has adopted a version of the interactive
transaction protocol for their own purposes. Nadav pointed out that the
protocol we landed on for lightning interactive construction transactions
is *quite* interactive; the DLC version modified it to use batching to
transmit the input/output sets (the 

[Lightning-dev] [c-lightning] v0.8.2 Scaling the Ethereum Blockchain

[lisa neigut]

We're pleased to announce the 0.8.2 release of c-lightning, named by
@arowser .

This is a minor release and includes a few new and experimental features,
as well as bug-fixes and performance wins.
Highlights for Users

   - New config option --large-channels (also known as 'wumbo') which
   enables opening channels of any size. (Note that your peer must also
   support large channels.)
   - This release includes a keysend plugin, which will enable receiving
   'keysend' payments, as first introduced by Lightning Labs
   . Note that the
   included keysend plugin is receive only for this release. Nodes which do
   not want the hassle of spontaneous unrequested payments should add
   'disable-plugin=keysend' to their config!
   - We'll now announce multiple connection endpoints for a single 'type',
   e.g. multiple IPv4 addresses.
   - A new FAQ
   !
   - Big performance improvment in the pay command (~1s speedup on average)

Highlights for the Network

   - c-lightning nodes can now participate in creating larger channels
   (with the --large-channel config option).
   - We now wait until the first payment through a channel before updating
   the feerate; this should help with some spurious closures at channel open
   or re-connect that were occurring against older versions of other
   implementations.

Highlights for Developers

   - A new command getsharedsecret for getting the BOLT-compliant shared
   secret finding for a node and a point.
   - The API for the 'Bitcoin backend' plugin infrastructure has been
   documented!
   - Facilities for building rendez-vous compatible onions has been added
   to the onion devtool
   - Plugin options will now respect the type they were given in the
   manifest.
   - Fixes with plugin cleanups and hangs.
   - Python2 has been removed as a dependence.

More details can be found at
https://github.com/ElementsProject/lightning/blob/v0.8.2/CHANGELOG.md

Thanks to everyone for their contributions and bug reports; please keep
them coming.

Since 0.8.1, we've had 236 commits from 10 different authors, with 2 first
time contributors:

   - Dr. Maxim Orlovsky @dr-orlovsky 
   - Dave Scotese @dscotese 

Cheers,

Lisa, Christian, Rusty, and ZmnSCPxj
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

> With PoDLE this would not be possible I think, as you would not be able
to open the PoDLE commitment with the other node as the target (if we go
with the modified PoDLE which also commits to which node an opening is for,
to prevent the pouncing venus flytrap attack).

Good question. It should be possible to do multi-channel open even with the
PoDLE signature committing to a node_id.

- An initiator can use the same utxo (h2) as their proof for multiple
peers; the signatures passed to each peer will have to commit to that
specific peer's node_id, however.
- The revised PoDLE signature commitment requires every initiator to
include at least one of their own inputs in the tx. Attempting to initiate
an additional open etc using someone else's utxo's won't work (this is the
pouncing venus flytrap attack which we're preventing). The initiator
including at least one input is expected behavior, at least in the open
case, since the opener has to cover the fees for the funding output.
- Ideally, a node would remove the PoDLE TLV data from any 'forwarded'
`tx_add_inputs` that isn't the input they're proving for, to prevent
leaking information about which inputs belong to other parties. I say
ideally here because even if you fail to do this, the peer can iterate
through all the provided commitment proofs until one of them
matches/verifies with the upfront provided PoDLE.



On Thu, Feb 13, 2020 at 12:18 AM ZmnSCPxj  wrote:

> Good morning Rusty, niftynei, and list,
>
> > > > -   Serial ids should be chosen at random
> > > >
> > > > -   For multiparty constructions, the initiator MUST flip the bottom
> bit of any received inputs before relaying them to a peer.
> > > >
> > > > -   Collisions of serial ids between peers is a protocol error
> > > >
> > >
> > > I suppose we should define collision to mean "equal in all bits except
> the lowest bit".
> >
> > No, literally equal. i.e. you can only make this error by clashing with
> > yourself.
>
> hmm, I thought the entire point of having the low bit was that you could
> multifund in such a way that the initiator creates multiple channels
> simultaneously with multiple nodes?
> So you would have to take the UTXOs of one peer and give it to the other
> peer claiming it as your own.
> Or something.
>
> With PoDLE this would not be possible I think, as you would not be able to
> open the PoDLE commitment with the other node as the target (if we go with
> the modified PoDLE which also commits to which node an opening is for, to
> prevent the pouncing venus flytrap attack).
>
> Regards,
> ZmnSCPxj
>
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

> Probably so that address reuse is not dinged, i.e. I have two UTXOs with
the same address and want to make two different channels with different
peers.

Having 2 utxos locked to the same pubkey will map to a single H2 value
though, which is what is used to flag utxo reuse. With a PoDLE you're
proving that you have a *key* for a utxo; the verifier checks that the key
you say you know does in fact map to controlling the utxo that you say it's
attached to. Whether or not you added the utxo to the signature commitment
doesn't add anything to the security of the verification.

At worse, it might leak what other utxo that the initiator controls, if
they accidentally commit to the wrong utxo and the peer decided to try
grinding utxo outpoints on the offchance that one matched.



On Tue, Feb 11, 2020 at 10:04 PM ZmnSCPxj  wrote:

> Good morning niftynei, and waxwing, and list,
>
> > > s = k + H(kG || kJ || P || P2 || utxo || receiving-node) x
> >
> > > and as before transfer opening: (P, P2, u, s, e) with receiving-node
> implicitly reconstructed to do the verification of the Schnorr sig. It's
> basically a message in a signature.
> >
> > Oh that's *much* nicer than calculating a second commitment.
> Verification by any node that's not the intended recipient will fail, as
> they'll use the wrong node_id (their own).
> >
> > It seems unnecessary to me to commit to the utxo, since the pubkey pair
> effectively does that. What's the motivation for including it?
>
> Probably so that address reuse is not dinged, i.e. I have two UTXOs with
> the same address and want to make two different channels with different
> peers.
>
> While address reuse Is Bad, you might not have much control over some wog
> who is supposed to pay you and decides to give you your money in two
> separate UTXOs to the same address.
>
> Regards,
> ZmnSCPxj
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

I'd like to propose that we add a second commitment requirement to the
PoDLE that JoinMarket uses, to limit the use of a commitment's validity to
be only between an initiator and a single peer. Otherwise you can enable
something I'll call the "pouncing venus-flytrap attack"[1].  Venus-flytrap
because they sit in wait for victims; pouncing because the venus-flytrap
then attacks other nodes using the provided fly/utxo bait.

## The Attack
A malicious node sits and waits until another, honest, node initiates an
open with them. They wait until the honest initiator has sent them the
commitment and utxo proof. They then use the provided, non-blacklisted utxo
and commitment proof to attempt to open a channel with as many other nodes
as possible, simultaneously. They may either fail to respond or not fail
the original channel open. They fail every other open attempt,
simultaneously. Each of the nodes they've griefed will blacklist the
provided UTXO; the honest initiator has now had their utxo blacklisted.

## Mitigation
Have each initiator provide two commitments: one to the shared/global J
point and one to a point that is found from the hash of the non-initiating
node's node_id.[2]

The global-point commitment is the one that is blacklisted; the node_id's
commitment prevents the other party from being able to re-use a commitment
in another channel, as they'll be unable to produce a valid commitment to
the point derived from the node_id of their victim (so the victim will know
the commitment has been re-used).

This has implications for 'multi-channel opens', in that any node
initiating an open MUST provide at least one utxo of their own. This seems
like an acceptable limitation, imo.

The protocol adjustments required for this are :

- Add a second commitment to the TLV of `open_channel2`; two H2's, one for
the 'global J' and one for the 'nodes J', with the global point commitment
always appearing first.
- The TLV type for the `tx_add_input` for the 'committed utxo' will now
include an array of two `proof of dle's`, in commitment hash order.

1. tlvs: `add_input_tlvs`
2. types:
1. type: 1 (`proof_of_dle_array`)
2. data:
*[`2*proof_of_dle`]

1. subtype (`proof_of_dle`)
*[`64*byte`:`s||e`]
*[`33*byte`:pubkey`]
*[`33*byte`:pubkey2`]

A node that provides a valid global point but an invalid 'local point'
commitment should be immediately errored on and potentially blacklisted.
A failure of this type should *not* result in a blacklist of the global
commitment.

[1] There's probably a better analogy here, but it's escaping me at the
moment.
[2] We reuse JoinMarket's NUMs point generation idea of appending a counter
to a hash until a valid positive-public key point is found, but without the
index.

On Mon, Feb 10, 2020 at 5:11 PM lisa neigut  wrote:

> Here's some thoughts I had on PoDLE's and lightning. An enormous
> tip-of-the-hat is due to ZmnSCPxj for surfacing the work that JoinMarket
> has done here already.
>
> - The initiating message (in the case of open channel, this would be
> `open_channel2`) is extended to include an 'H2' field in its TLV, a 32-byte
> hash commitment to the P2 key.
> - Only one H2 commitment is required.
> - The `tx_add_input` message, as specified previously, is extended to an
> include a TLV type. This must be present on the input addition that
> corresponds with the UTXO used for the originally transmitted commitment
> - The non-initiator SHOULD wait to send any `tx_add_input` messages of
> their own until after receiving a `tx_add_input` message with a valid PoDLE
> TLV extension.
>
> 1. tlvs: `add_input_tlvs`
> 2. types:
> 1. type: 1 (`proof_of_dle`)
> 2. data:
> *[`64*byte`:`s||e`]
> *[`33*byte`:pubkey`]
> *[`33*byte`:pubkey2`]
>
> - If the proof is incorrect, the non-initiator MAY fail the transaction
> collaboration or respond with `tx_complete`. There is no need for them to
> publish the PoDLE.
> - If the proof is correct, the non-initiator verifies that the commitment
> (hash of pubkey2) has not been communicated to them via gossip.
> - If the proof is not in their gossip store, the transaction collaboration
> continues. It is considered 'safe' for the non-initiator to send
> `tx_add_input` to their peer.
> - If the proof IS in their gossip store, the transaction collaboration
> SHOULD reply with `tx_complete`. It is considered 'unsafe' for the
> non-initiator to send `tx_add_input`. (This allows errored/erroring
> initiators to use blacklisted utxos, however it prevents them from privy to
> any other nodes' UTXO set.)
> - The initiator MUST NOT remove the committed to UTXO from the
> collaboration set.
>
> - If the transaction collaboration fails/is errored by the initiator,
> - the non-initiator SHOULD broadcast the original PoDLE commitment to
> the gossip network.

Re: [Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

Here's some thoughts I had on PoDLE's and lightning. An enormous
tip-of-the-hat is due to ZmnSCPxj for surfacing the work that JoinMarket
has done here already.

- The initiating message (in the case of open channel, this would be
`open_channel2`) is extended to include an 'H2' field in its TLV, a 32-byte
hash commitment to the P2 key.
- Only one H2 commitment is required.
- The `tx_add_input` message, as specified previously, is extended to an
include a TLV type. This must be present on the input addition that
corresponds with the UTXO used for the originally transmitted commitment
- The non-initiator SHOULD wait to send any `tx_add_input` messages of
their own until after receiving a `tx_add_input` message with a valid PoDLE
TLV extension.

1. tlvs: `add_input_tlvs`
2. types:
1. type: 1 (`proof_of_dle`)
2. data:
*[`64*byte`:`s||e`]
*[`33*byte`:pubkey`]
*[`33*byte`:pubkey2`]

- If the proof is incorrect, the non-initiator MAY fail the transaction
collaboration or respond with `tx_complete`. There is no need for them to
publish the PoDLE.
- If the proof is correct, the non-initiator verifies that the commitment
(hash of pubkey2) has not been communicated to them via gossip.
- If the proof is not in their gossip store, the transaction collaboration
continues. It is considered 'safe' for the non-initiator to send
`tx_add_input` to their peer.
- If the proof IS in their gossip store, the transaction collaboration
SHOULD reply with `tx_complete`. It is considered 'unsafe' for the
non-initiator to send `tx_add_input`. (This allows errored/erroring
initiators to use blacklisted utxos, however it prevents them from privy to
any other nodes' UTXO set.)
- The initiator MUST NOT remove the committed to UTXO from the
collaboration set.

- If the transaction collaboration fails/is errored by the initiator,
- the non-initiator SHOULD broadcast the original PoDLE commitment to
the gossip network.
- the non-initiator MAY delay broadcast to allow the initiating node to
re-attempt the open.

The gossip message for a PoDLE blacklist entry is as follows:

1. type: 259 (`podle_blacklist`)
2. data:
*[`signature`:`signature`]
*[`32*byte`:`H2`]
*[`point`:`node_id`]
*[`u32`:`timestamp`]

Note that the `node_id` is the id of the node that signs (and broadcasts)
the blacklisted PoDLE. h/t to ZmnSCPxj for the gossip construction.
The timestamp is added as a convenience for peers to trim/discard blacklist
participants as they wish depending on time/staleness.

## Some Notes:
- The JoinMarket protocol allows nodes to use any of a range of secondary
points for J. Since the lightning version of this allows blacklisted UTXOs
to still open channels, albeit without participation from the peer, it
seems unnecessary to allow for more than one valid J point. I'd propose
fixing the J the same zero-index point used by JoinMarket. This reduces the
number of valid H2's that are available for any given utxo set, while also
keeping blacklisted H2's compatible with the blacklist set generated by
JoinMarket implementations.
- The blacklist originates from the 'non-initiating' peer, and does not
reveal the offending node's id.
- Assuming that every node honestly participates in the blacklist, only
verified H2's will be submitted to the blacklist
- A malicious non-initiator can only prevent an honest initiator from using
the committed UTXO for collaborative transactions; they won't prevent them
from successfully initiating a one-sided transaction with honest peers.
- Only nodes that have at least one public channel will be able to
contribute to the public PoDLE blacklist. This means it's possible for a
malicious initiator to grief non-public nodes without much consequence,
however this requires the ability to send inbound messages to private
nodes, i.e. more likely for a close or splice interaction.
- As ZmnSCPxj has pointed out elsewhere, a malicious peer could broadcast
junk H2's; it is acceptable to rate-limit the number of PoDLE blacklists
generated by a peer.peer
- It is possible for a malicious peer to fail to relay their `H2` entries
in the blacklisted gossip set.
- Duplicate H2 gossip should replace older timestamped versions.
- Elsewhere we've had a discussion/concern over floods of PoDLE blacklist
messages. It's possible for gossip message floods to originate from a
malicious peer; they also might signal an ongoing probe attempt. Given a
timestamp and a rough measure of the number of utxos' currently outstanding
in the mempool, however, it should be possible to distinguish the two.

## Open Questions:
- Should PoDLE be required for every collaborative transaction (opens,
splices + closes), or only for opens? It seems reasonable to limit them
just to opens, as for all others you'll already have a shared UTXO with the
peer.
- Is fixing the generator point too restrictive? JoinMarket allows for a
range of acceptable NUMS (J) points (up to 256). The smaller the pool of
eligible J's, the 

Re: [Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

> But if you impose the blockheight - 6 in the Lightning protocol level,
and Lightning succeeds (meaning a substantial fraction of blockchain
transactions are Lightning opens)...
>  --- then transactions with `nLockTime` equal to the block they are
included in minus 5 will be more common than others, and would be a
reliable indicator that the transaction is a Lightning channel funding
attempt.

Ah good point. This can be mitigated by setting the acceptable range up to
100 then, matching the behavior of bitcoind.






On Thu, Feb 6, 2020 at 6:23 PM ZmnSCPxj  wrote:

> Good morning lisa,
>
> > > I am unsure what is the purpose of this minus 6.
> >
> > The original motivation was to keep the funding transaction from being
> rejected from the mempool in the case of a re-org, but as you pointed out,
> the 'next block' is always at -par or ahead of the current chain tip, so
> I'm not sure this accomplishes this goal.  I'm not sure how bitcoind
> handles the mempool in the case of the 'best block' moving to another tip,
> the goal of setting it to -6 is to avoid the funding transaction being
> evicted.
>
> My understanding is that it rewinds the abandoned tip, putting the
> transactions in those blocks back into its local mempool (which may lead to
> evictions if the mempool gets full), all the way to the branch-off point,
> then it re-adds blocks back to the new tip (which can lead to removals from
> the mempool, if transactions in the block spend the same UTXOs (or *are*
> the same transactions) as transactions in the mempool).
> The main effect is that there could be suddenly higher fee pressure for
> the transactions in the reorged-away blocks (because of possible mempool
> congestion if the longer chainsplit has fewer transactions per block), but
> that is why the dual-funding protocol has RBF built-in right?
>
> Setting blockheight - 6 also increases the incentive of potential
> deliberate reorgers to actually perform a reorg attack, because the
> transaction you just added is valid for earlier blocks that the reorger
> wants to rewrite.
> This is a bad thing, because you want your funding txout to be confirmed,
> not have parts of global hashpower contemplating reorgs and delaying your
> confirmations even more.
>
>
> >
> > In practice, setting the locktime back a few blocks makes the funding
> transaction eligible for inclusion in any of the previous six blocks, so in
> case of a reorg there's a higher probability it will have been included in
> the reorganization. In other words, it enables fee-sniping for up to 6
> blocks in the hopes that any 'eligible' re-org includes the funding
> transaction (the short channel id will change, but otherwise the channel
> open will be the same).
> >
> > On second thought, this doesn't seem like something that we should
> include at the protocol level; if a peer wanted to 'allow fee-sniping for
> up to X blocks', then they'd simply relay the "blocktip" that they're using
> for the nLocktime to be at the depth they'd desire. Though it might be
> worth imposing a limit as to how far back in the past a peer can allow
> fee-sniping for... no more than 6 blocks from our current tip seems
> reasonable. (This would then limit the 'acceptable range' for an offset of
> an initiator to 5, as your peer may be off from your tip by one.)
> >
> > On that note, I believe bitcoind fuzzes the nLocktime value to obfuscate
> exactly what blockheight the outgoing transaction was composed / broadcast
> at, which is probably something we should encourage in lightning
> implementations as well.
>
> But if you impose the blockheight - 6 in the Lightning protocol level, and
> Lightning succeeds (meaning a substantial fraction of blockchain
> transactions are Lightning opens) --- then transactions with `nLockTime`
> equal to the block they are included in minus 5 will be more common than
> others, and would be a reliable indicator that the transaction is a
> Lightning channel funding attempt.
> The fuzzing may not be big enough to cover that, as there is a 10% chance
> to fuzz and about 1% subsequent chance (total 0.1% chance) that Bitcoin ore
> will put a transaction at blockheight - 6 (as opposed to the 99 other
> possibilities: blockheight - 0 to blockheight - 99 inclusive).
> So once more than 0.1% of onchain transactions are Lightning
> dual-fundings, an analyst has > 50% chance of correctly betting that a
> blockheight - 5 transaction (yes, - 5, because a transaction can typically
> be added only on the next block) is a Lightning funding.
>
>
> You are better off with blockheight, possibly with SPV-header-chain-proofs
> if one side or the other thinks the blockheight has changed since one side
> or the other proposed it.
>
>
> Regards,
> ZmnSCPxj
>
> >
> > On Wed, Feb 5, 2020 at 8:25 PM ZmnSCPxj  wrote:
> >
> > > Good morning niftynei,
> > >
> > > > Rusty had some suggestions about how to improve the 

Re: [Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

> I am unsure what is the purpose of this minus 6.

The original motivation was to keep the funding transaction from being
rejected from the mempool in the case of a re-org, but as you pointed out,
the 'next block' is always at -par or ahead of the current chain tip, so
I'm not sure this accomplishes this goal.  I'm not sure how bitcoind
handles the mempool in the case of the 'best block' moving to another tip,
the goal of setting it to -6 is to avoid the funding transaction being
evicted.

In practice, setting the locktime back a few blocks makes the funding
transaction eligible for inclusion in any of the previous six blocks, so in
case of a reorg there's a higher probability it will have been included in
the reorganization. In other words, it enables fee-sniping for up to 6
blocks in the hopes that any 'eligible' re-org includes the funding
transaction (the short channel id will change, but otherwise the channel
open will be the same).

On second thought, this doesn't seem like something that we should include
at the protocol level; if a peer wanted to 'allow fee-sniping for up to X
blocks', then they'd simply relay the "blocktip" that they're using for the
nLocktime to be at the depth they'd desire. Though it might be worth
imposing a limit as to how far back in the past a peer can allow
fee-sniping for... no more than 6 blocks from our current tip seems
reasonable. (This would then limit the 'acceptable range' for an offset of
an initiator to 5, as your peer may be off from your tip by one.)

On that note, I believe bitcoind fuzzes the nLocktime value to obfuscate
exactly what blockheight the outgoing transaction was composed / broadcast
at, which is probably something we should encourage in lightning
implementations as well.



On Wed, Feb 5, 2020 at 8:25 PM ZmnSCPxj  wrote:

> Good morning niftynei,
>
>
> > Rusty had some suggestions about how to improve the protocol messages
> for this, namely adding a serial_id to the inputs and outputs, which can
> then be reused for deletions.
> >
> > The serial id can then also be used as the ordering heuristic for
> transaction inputs during construction (replacing current usage of BIP69).
> Inputs can be shared amongst peers by flipping the bottom bit of the
> serial_id before relaying them to another peer (as your own).
>
> What happens if the initiator deliberately provides serial IDs 0x1, 0x3,
>  while the acceptor naively provides serial IDs from `/dev/urandom`?
>
> Then the balance of probability is that the initiator inputs and outputs
> are sorted before the acceptor.
> Now, this is probably not an issue, since the initiator and acceptor both
> know which inputs and outputs are theirs and not theirs, so they could just
> reveal this information to anyone, so an actor providing such lousy serial
> IDs is just hurting its own privacy relative to blockchain analysts, so
> probably will not happen in practice.
>
> My initial reaction was to propose adding a secret-sharing round where the
> resulting key is XORed to each serial ID before sorting by the XORed serial
> ID, but this might be too overweight, and again the initiator is only
> hurting its own privacy, and the two participants already know whose money
> is whose anyway
>
> >
> > See below for details.
> >
> > > 1. type:   440 `tx_add_input`
> > >
> > > 2. data:
> > >
> > > * [`32*byte`:`channel_identifier`]
> >
> > * [`32*byte`:``serial_id`]
> >
> > Add a serial id.
> >
> > Each input addition must have a unique serial id.
> >
> > No addition may have a repeated id number.
> >
> > The initiator's serial id's must be odd. The non-initiator's serial id's
> must be even.
> >
> > Serial ids are used as sorting heuristic for input ordering in final
> transaction, replaces BIP69
> >
> >
> > > * [`u64`:`sats`]
> > >
> > > * [`sha256`:`prevtx_txid`]
> > >
> > > * [`u32`:`prevtx_vout`]
> > >
> > > * [`u16`:`prevtx_scriptpubkey_len`]
> > >
> > > * [`prevtx_scriptpubkey_len*byte`:`prevtx_scriptpubkey`]
> > >
> > > * [`u16`:`max_witness_len`]
> > >
> > > * [`u16`:`scriptlen`]
> > >
> > > * [`scriptlen*byte`:`script`]
> >
> > Removes the signal_rbf; everything will be flagged as RBF eligible.
> (This makes verifying RBF eligibility during a RBF round simpler.)
>
> Yes. Ish.
> RBF and privacy do not work well together unfortunately.
> This is still initiator-pays, right?
>
>
> > > 1. subtype: `witness_element`
> > >
> > > 2. data:
> > >
> > > * [`u16`:`len`]
> > >
> > > * [`len*byte`:`witness`]
> > >
> > > ## General Notes
> > >
> > > - All output scripts must be standard
> > >
> > > - nLocktime is always set to 0x
> >
> > - If a blockheight to be used as nLocktime is communicated in the
> initiation step, is set to blockheight-6; otherwise set to zero-
>
> I am unsure what is the purpose of this minus 6.
>
> If you fear blockheight disagreements, this is probably a good time to
> introduce block headers.
> So for example if the acceptor 

Re: [Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

Rusty had some suggestions about how to improve the protocol messages for
this, namely adding a serial_id to the inputs and outputs, which can then
be reused for deletions.

The serial id can then also be used as the ordering heuristic for
transaction inputs during construction (replacing current usage of BIP69).
Inputs can be shared amongst peers by flipping the bottom bit of the
serial_id before relaying them to another peer (as your own).

See below for details.


1. type:   440 `tx_add_input`
>
> 2. data:
>
> * [`32*byte`:`channel_identifier`]
>
* [`32*byte`:``serial_id`]

Add a serial id.

Each input addition must have a unique serial id.

No addition may have a repeated id number.

The initiator's serial id's must be odd. The non-initiator's serial id's
must be even.
Serial ids are used as sorting heuristic for input ordering in final
transaction, replaces BIP69


* [`u64`:`sats`]
>
> * [`sha256`:`prevtx_txid`]
>
> * [`u32`:`prevtx_vout`]
>
> * [`u16`:`prevtx_scriptpubkey_len`]
>
> * [`prevtx_scriptpubkey_len*byte`:`prevtx_scriptpubkey`]
>
> * [`u16`:`max_witness_len`]
>
> * [`u16`:`scriptlen`]
>
> * [`scriptlen*byte`:`script`]
>
> Removes the signal_rbf; everything will be flagged as RBF eligible. (This
makes verifying RBF eligibility during a RBF round simpler.)


> 1. type: 442 `tx_add_output`
>
> 2. data:
>
> * [`32*byte`:`channel_identifier`]
>
* [`16*byte`:`serial_id`]

Add a serial id. Same rules as for inputs, but a distinct counter set is
used.
Used for ordering the transactions’ outputs, replacing BIP69



> * [`u64`:`sats`]
>
> * [`u16`:`scriptlen`]
>
> * [`scriptlen*byte`:`script`]
>
> 1. type: 444 `tx_remove_input`
>
> 2. data:
>
> * [`32*byte`:`channel_identifier`]
>
* [`16*byte`:`serial_id`]


Input to remove identified by the serial id, not txid and index.



>
> 1. type: 446 `tx_remove_output`
>
> 2. data:
>
> * [`32*byte`:`channel_identifier`]
>
* [`16*byte`:`serial_id]

Output to remove identified by the serial id, not output script and amount.



> 1. type: 448 `tx_complete`
>
> 2. data:
>
> * [`32*byte`:`channel_identifier`]
>

Total counts removed from tx_complete. The txid exchanged in the `tx_sigs`
will serves as a checksum for the transaction.


> 1. type:  448 `tx_sigs`
>
> 2. data:
>
> * [`channel_id`:`channel_identifier`]
>
> * [`u16`:`num_witnesses`]
>
> * [`num_witnesses*witness_stack`:`witness_stack`]
>
> 1. subtype: `witness_stack`
>
> 2. data:
>
* [`u16`:`num_input_witness`]
>
> * [`num_input_witness*witness_element`:`witness_element`]
>

prev_out and prev_txid are removed; witnesses ordered implicitly by
serial_id.


> 1. subtype: `witness_element`
>
> 2. data:
>
> * [`u16`:`len`]
>
> * [`len*byte`:`witness`]
>
>
>
> ## General Notes
>
> - All output scripts must be standard
>
> - nLocktime is always set to 0x
>
- If a blockheight to be used as nLocktime is communicated in the
initiation step, is set to blockheight-6; otherwise set to zero-
- Serial ids should be chosen at random
- For multiparty constructions, the initiator MUST flip the bottom bit of
any received inputs before relaying them to a peer.
- Collisions of serial ids between peers is a protocol error
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

hi max — great question. PSBT is a great protocol for wallet interop but a
bit overweight for tx collaboration between two peers

On Wed, Jan 29, 2020 at 17:29 Max Dignan  wrote:

> Hey Antoine,
>
> Would PSBT (BIP 174 -
> https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki) be a good
> solution to this?
>
> -Max
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

[Lightning-dev] DRAFT: interactive tx construction protocol

[lisa neigut]

Some of the feedback I received from the check-in for the dual-funding
proposal this past Monday was along the lines that we look at simplifying
for breaking it into smaller, more manageable chunks.

The biggest piece of the dual-funding protocol update is definitely the
move from a single peer constructing a transaction to two participants.
We're also going to likely want to reuse this portion of the protocol for
batched closings and splicing. To that extent, it seemed useful to
highlight it in a separate email.

This is a change from the existing proposal in the dual-funding PR #524
 -- it allows
for the removal of inputs and outputs.

The set of messages are as follows.


Note that the 'initiation' of this protocol will be different depending on
the case of the transaction (open, close or splice):

1. type:   440 `tx_add_input`

2. data:

* [`32*byte`:`channel_identifier`]

* [`u64`:`sats`]

* [`sha256`:`prevtx_txid`]

* [`u32`:`prevtx_vout`]

* [`u16`:`prevtx_scriptpubkey_len`]

* [`prevtx_scriptpubkey_len*byte`:`prevtx_scriptpubkey`]

* [`u16`:`max_witness_len`]

* [`u16`:`scriptlen`]

* [`scriptlen*byte`:`script`]

* [`byte`:`signal_rbf`]

1. type: 442 `tx_add_output`

2. data:

* [`32*byte`:`channel_identifier`]

* [`u64`:`sats`]

* [`u16`:`scriptlen`]

* [`scriptlen*byte`:`script`]

1. type: 444 `tx_remove_input`

2. data:

* [`32*byte`:`channel_identifier`]

* [`sha256`:`prevtx_txid`]

* [`u32`:`prevtx_vout`]

1. type: 446 `tx_remove_output`

2. data:

* [`32*byte`:`channel_identifier`]

* [`u64`:`sats`]

* [`u16`:`scriptlen`]

* [`scriptlen*byte`:`script`]

1. type: 448 `tx_complete`

2. data:

* [`32*byte`:`channel_identifier`]

* [`u16`:`num_inputs`]

* [`u16`:`num_outputs`]

1. type:  448 `tx_sigs`

2. data:

* [`channel_id`:`channel_identifier`]

* [`u16`:`num_witnesses`]

* [`num_witnesses*witness_stack`:`witness_stack`]

1. subtype: `witness_stack`

2. data:

* [`sha256`:`prevtx_txid`]

* [`u32`:`prevtx_vout`]

* [`u16`:`num_input_witness`]

* [`num_input_witness*witness_element`:`witness_element`]

1. subtype: `witness_element`

2. data:

* [`u16`:`len`]

* [`len*byte`:`witness`]



## General Notes

- Validity of inputs/outputs is not checked until both peers have sent
consecutive `tx_complete`  messages.

- Duplicate inputs or outputs is a protocol error.

- Feerate is set by the initiator, or in the case of a closing transaction,
negotiated before the transaction construction is initiated.

- Every peer pays fees for the inputs + outputs they contribute, plus
enough to cover the maximum estimate of their witnesses. Overpayment of
fees is permissible.

- Initiator is responsible for contributing the output/input in question,
i.e. the

  funding output in the case of an opening, or the funding input in the
case of a close.

  (This means that the opener will pay for the opening output). In the case
of a splice,

  the initiator of the splice pays for the funding tx's inclusion as an
input and the

  new 'funding tx' output.

- Any contributor may signal that their input is RBF'able. The nSequence
for this input should be set to 0xFEFF , 0x otherwise.

- The initiating peer is understood to be paying the fee for the shared
transaction fields (nVersion [4], segwit marker + flag [2], input + output
counts [2-18], witness count [1-9], nLocktime [4]; total [13-40bytes])

- Inputs MUST be segwit compatible (PW* or P2SH-PW*)

- All output scripts must be standard

- nLocktime is always set to 0x.

- The `num_inputs` and `num_outputs` in `tx_complete` is a count of that
peer’s final input and output contributions, net any removals.

- Either peer may add or remove inputs and outputs until both peers have
successfully

  exchanged a `tx_complete` message in succession.

- Either peer may only add or remove their own input or output.

- In the case that a `tx_complete` agreement cannot be reached, either peer
may

  fail the channel or open protocol (whatever is reasonable for the
particular case)

  - In the case of a splice, this would be a soft error (channel returns to
normal operation until

otherwise failed or closed.)

  - In the case of an open, this would be a failure to open the channel.

  - In the case of a close, a failed collaborative close would result in an
error and a unilateral close.

### Considering the Simple Open case (2 parties)

- Both peers signal `opt_dual_fund`

- Opener initiates a channel open with `open_channel2` message, indicating
the feerate for the opening transaction

- Accepter signals acceptance of channel open as proposed, including
proposed feerate, via `accept_channel2`

- Opener sends `tx_add_output`, with the funding output for the sum of both
peer’s funding_amount

- Opener sends `tx_add_input` for each input the wish to add to the funding

Re: [Lightning-dev] A proposal for up-front payments.

[lisa neigut]

> Imagine the following setup: a network of nodes that trust each other

The goal of this pre-payment proposal is to remove the need for trusted
parties.

On Thu, Nov 7, 2019 at 07:38 Joost Jager  wrote:

> > Isn't spam something that can also be addressed by using rate limits for
>> > failures? If all relevant nodes on the network employ rate limits, they
>> can
>> > isolate the spammer and diminish their disruptive abilities.
>>
>> Sure, once the spammer has jammed up the network, he'll be stopped.  So
>> will everyone else.  Conner had a proposal like this which didn't work,
>> IIRC.
>>
>
> Do you have ref to this proposal?
>
> Imagine the following setup: a network of nodes that trust each other (as
> far as spam is concerned) applies a 100 htlc/sec rate limit to the channels
> between themselves. Channels to untrusted nodes get a rate of only 1
> htlc/sec. Assuming the spammer isn't a trusted node, they can only spam at
> 1 htlc/s and won't jam up the network?
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

[Lightning-dev] c-lightning v0.7.3 "Bitcoin's Proof of Stake" Released

[lisa neigut]

We're pleased to announce 0.7.3, named by @trueptolemy

Note: new dependency gettext is now required when building from source.


*Highlights for Users*- lightningd now supports different SQL backends.
This release includes a PostgresSQL driver, in addition to the default
sqlite3 driver.
- Ability to supply a Bitcoin address to close a channel to. Note that if
you've set an upfront-shutdown script, it'll need to be the same script.
- New plugin notifications: sendpay_success and sendpay_failure
- Encryption of the BIP32 master seed is now available.
- UTXO selection is provided for withdraw and txprepare, which allows you
to finely control which lightningd wallet UTXOs are spent or used.
- Our Bolt11 tools now parse feature bits
- Adds the ability to exclude nodes from route consideration in getroute
- Please note that the deprecated RPC call listpayments has been removed;
you want listpays.


*Highlights for the Network*- Elements support. lightningd can now be made
to support L-BTC.
- c-lightning nodes now announce features in node_announcement broadcasts.
- Further on the feature bits front, we've consolidated them -- all bits
are now advertised in both global + local feature fields
- Support for gossip_queries_ex, for finer grained gossip control
- Tighter gossip bandwidth usage. We now take advantage of gossip queries
and peer rotation to narrow the number of peers that we gossip with and the
amount of gossip we're requesting.
- In further gossip news, we now no longer ask for initial_routing_sync.

More details can be found in
https://github.com/ElementsProject/lightning/blob/v0.7.3/CHANGELOG.md

*Contributions*
Thanks to everyone for their contributions and bug reports; please keep
them coming!

Since 0.7.2.1 we've had 522 commits from 18 different authors, with 5
first-time contributors!

Luca Ambrosini
Yash Bhutwala
@willcl-ark
Jacob Rapoport
@fiatjaf

Release details and binaries can be found at
https://github.com/ElementsProject/lightning/releases/tag/v0.7.3

Cheers,
Lisa, Rusty, Christian, and ZmnSCPxj.
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Dual Funding Proposal

[lisa neigut]

>
>
> >++  ++
> >   where node A is the ‘initiator’ and node B is the ‘dual-funder’
>
> We currently use the terms funder and fundee, which are now
> inaccurate ofc.  Perhaps 'opener' and 'accepter' are not great english,
> but they map to the messages well?
>

'opener' and 'accepter' do map to the messages. I've adopted it for the
rest of this response, to see how it fits in context.

"Another subtle point is the feerate_per_kw field; in the old scheme it
applied to the first commitment tx, but here it applies to both the
first commitment tx and the funding tx itself"

You're right. Initially I didn't think the `accepter` would care since
they're not paying them,
but you need it to be able to construct the funding transaction. I'll add a
second field, it seems
important to keep them separated esp since the timing consideration for the
fees is different (now vs the future).

> The sending node:
> >-
> >MAY begin channel establishment using `open_channel2`
>
>  - MUST NOT send `open_channel`.
>
> > Otherwise the receiving node:
> >-
> >MUST return an error.
>
> This is a requirement for receiving `open_channel`  IIUC?
>
> ie.
>
> The receiving node MUST fail the channel if:
>...
>- `option_dual_fund` has been negotiated.
>

Does v2 of channel open necessarily deprecate the original between two
upgraded nodes?

This seems more sane than having both as an option...will update.


>
> > `funding_puts2`
>
> We can probably drop the 2 here and call it, um.. `funding_compose`?
> (Thanks thesaurus.com).  I get where you're going with 'puts, but it
> took me a while :)


Initially only the duplicated messages had the 2-suffix, but ended up
adding it to all of them to denote that they belonged to the v2 of channel
open... I can see how that's confusing though.

+1 for `funding_compose`, it's definitely more easily understood. :-D

...

> >MUST NOT send a number of `input_data` and/or `output_data` which
> >exceeds the `put_limit`
>
> Side note: I wonder if we should relax this limit when we talk about
> `option_will_fund_for_food`?
>

Yes! Thanks for pointing this out.


>
> >-
> >MAY send an empty message
>
> Be explicit? MAY offer zero `num_inputs` and `num_outputs`.  That's not
> quite an empty message...
>

I defined it a few lines above, but that's not super easy to see from this.
Will fix.


>
> > The receiving node:
> >
> >   If is the initiator (A):
> >
> >-
> >MUST fail the channel if the `num_inputs` plus `num_outputs` is
> greater
> >than the `put_limit`
>
> How about MAY?  It's a protection thing, but less to change when we
> option_will_fund_for_food.  Unless we set the `put_limit` to min (4) or
> something in that case?
>

+1 for MAY, considering that the opener will be paying the fees.
The limit for what's reasonable to pay is fairly subjective, i.e. perhaps
the opener doesn't
 care how many inputs/outputs the acceptor adds.


>
> Oh, it needs to check max_extra_witness_len is reasonable too, since
> that will affect the fees.  Each signature adds 74, and pubkey adds 34,
> so I think MUST BE less than 500 is perfectly reasonable (for both
> reader and writer).
>

Ack


> > ___`funding_locked2`
> >
> > // same as v1
> >
> > Requirements:
> >
> > A dual-funding node (B):
> >
> >-
> >
> >SHOULD broadcast their funding transaction if it does not see the
> >transaction broadcast after a reasonable timeout.
>
> Let's just reuse `funding_locked` maybe?
>
> Not sure why this should wait for broadcast?
>

I was overthinking this*. Can't think of a reason for both sides not to
broadcast; will amend.

* confused it with conflicting transaction broadcast behavior


> > == RBF for Channel Establishment v2
> >
> > _`init_rbf`
> >
> > This message is sent by the initiator, after the funding transaction has
> > been broadcast but before the `funding_locked2` has been exchanged.
> >
> > [32: `channel_id`]
> > [8: funding_satoshis]
> > [8:dust_limit_satoshis]
> > [8:channel_reserve_satoshis]
> > [4: feerate_per_kw]
> > [`2`:`num_inputs`]
> > [`num_inputs*input_info`]
> > [`2`:`num_outputs`]
> > [`num_outputs`*ouput_info`]
>
> Typo again :)
>
> > Requirements
> >
> > The sending node:
> >- MUST be the initiator (A)
> >- MAY update the amount, fee rate, dust limit, or channel reserve for
> the
> >channel
>
>  - MAY send init_rbf if it considers the most recent funding tx unlikely
>to be confirmed in reasonable time.
>  - MUST set `feerate_per_kw` larger than the most recent funding tx.
>

Another good reason to break out `funding_txn_feerate_per_kw` from
`commitment_txn_feerate_per_kw` in `open_channel2`


>
> Do we really want to negotiate everything again?  It seems like the
> funder should be able to maybe add *new* inputs and outputs (though TBH
> I think that's going to be unusual enough that we could omit it), but
> doing a wholesale replacement means we have to be careful that the all
> 

Re: [Lightning-dev] Dual Funding Proposal

[lisa neigut]

On Tue, Nov 27, 2018 at 11:26 PM ZmnSCPxj  wrote:

> Good morning Lisa,
>
> Minor comments only, have not studied in detail:
>
>
>
>  `accept_channel2`:
>
> [32:temporary_channel_id]
>
> …  // unchanged
>
> [33:first_per_commitment_point]
>
> [?: options_tlv]
>
> options_tlv:
>
>1.
>
>Type: 1 `option_upfront_shutdown_script`
>
>[2:len]
>
>   Value: `shutdown_scriptpubkey`
>
>
> I believe an even type is more appropriate, since the other side MUST
> enforce that sthudown only goes to the specified script?
>
>
Good point. This definitely should be an even type number as you suggest.
My initial thought was that being inside an `options_tlv` would preclude
the need to maintain the even/oddness designation, but I think this is a
mistake.

`funding_puts2`
>
> This message exchanges the input and output information necessary to
> compose the funding transaction.
>
> [32:temporary_channel_id]
>
> [`2`:`num_inputs`]
>
> [`num_inputs*input_info`]
>
> [`2`:`num_outputs`]
>
> [`num_outputs`*ouput_info`]
>
> 1. subtype: `input_info`
>
> 2. data:
>
>  * [`8`:`satoshis`]
>
>  * [`32`:`prevtxid`]
>
>  * [`4`:`prevtxoutnum`]
>
>  * [`2`:`scriptlen`]
>
>  * [`scriptlen`:`script`]
>
>  * [`2`:`max_extra_witness_len`]
>
>  * [`2`:`wscriptlen`]
>
>  * [`wscriptlen`:`wscript`]
>
>
> `script` here is the `scriptPubKey`?  This is needed for `hashPrevouts` in
> BIP143 I believe.
>
> What is the `wscript`?  Is this the `scriptCode` in BIP143?
>
> Are non-SegWit inputs disallowed?
>

This borrows heavily from Rusty's splicing proposal; whatever was specified
there should be assumed to also be specified here for inputs.  Rationale
being there should be no difference between the input requirements for a
splice versus a dual fund.


>
>
> 1. subtype: `output_info`
>
> 2. data:
>
>  * [`8`:`satoshis`]
>
>  * [`2`:`scriptlen`]
>
>  * [`scriptlen`:`script`]
>
> Requirements:
>
> The sending node:
>
>-
>
>MUST ensure each `input_info` refers to an existing UTXO
>-
>
>MUST ensure the `output_info`.`script` is a standard script
>-
>
>MUST NOT spend any UTXOs specified in funding_puts2 until/unless the
>channel establishment has failed
>
>
> If a violation of this is detected, what MUST we do?
>

I'm not sure what you mean here, it's not exactly enforceable other than
from an implementation perspective. If the funding transaction is invalid
because of an input (ie a double spend), the funding transaction won't be
confirmed. This should be discovered when the node attempts to broadcast
the funding transaction, and can be handled (ie treated as a channel
failure) there.


> Regards,
> ZmnSCPxj
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Splicing Proposal: Now with RBF

[lisa neigut]

Hello Rusty. Exciting stuff!  A few observations:

On Fri, Nov 16, 2018 at 12:18 AM Rusty Russell 
wrote:

> ### Confirming a splice: `splice_confirm`
>
> 1. type: 43 (`splice_confirm`) (`option_splice`)
> 2. data:
>* [`32`:`channel_id`]
>* [`64`:`signature`]
>* [`2`:`num_witnesses`]
>* [`num_witnesses*witness_stack`]
>
>
I don't believe that you need the `signature` field here; if I'm
understanding correctly the sigs for the inputs should be the witness stack
that you're sending.


> The sender:
>
...

>   - MUST ensure it will have sufficient funds post-splice above its
> reserve to pay for the splice transaction at the new feerate.
>

If fees outstrip the value of the updated splice transaction, what then?
It's not really possible to abandon a splice, practically you'd end up
closing the channel. This feels like an obvious observation, but worth
noting that splicing is 'risky' in that regard i.e. channel closure due to
extenuating circumstances (fee spike).


> Message Changes During Splicing
> ---
> Once you've sent `splice_confirm` each commitment transaction is needs
> to be duplicated for every splice transaction (thanks to RBF, there can
> be multiple at once).  These are in rbf-received order (increasing fee
> order, if initiator is spec compliant):
>
> Are HTLC's to be duplicated as well? CPFP seems like a neater construction
than RBF in this case, as it avoids fee rate negotiation and ballooning
HTLC/commitment txn management. It also makes the single-payer for fees
(initiator) less burdensome which is nice for skewed benefit updates. We
can reuse the scheme we came up with for commitment txns (either party can
spend, I believe).

Was there an argument against using CPFP on funding txns that I'm not
remembering?


> NOTES:
>
> 1. I suggest that the option_data_loss_protect fields MUST BE set here if
>option_splice (there's no reason not to AFAICT).  Or do we want to try
> TLV
>here?
>

+1 for moving to TLV, in the spirit of moving towards the new spec
guidelines.


> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Proposal for Advertising Channel Liquidity

[lisa neigut]

On Wed, Nov 7, 2018 at 11:02 PM Olaoluwa Osuntokun 
wrote:

> > A node, via their node_announcement,
>
> Most implementations today will ignore node announcements from nodes that
> don't have any channels, in order to maintain the smallest routing set
> possible (no zombies, etc). It seems for this to work, we would need to
> undo
> this at a global scale to ensure these announcements propagate?
>

Right on. I'm not too worried about this tbh; a new node on the network
could easily fix this by taking liquidity from another node that's already
offering it, to create a few balanced channels to itself. This would a) put
it on the map and b) make the liquidity that it's looking to offer more
valuable, as the other channels it's opened make it more likely to be
routed through.


>
> Aside from the incentives for leaches to arise that accept the fee then
> insta close (they just drain the network and then no one uses this), I
> think
> this is a dope idea in general! In the past, I've mulled over similar
> constructions under a general umbrella of "Channel Liquidity Markets"
> (CLM),
> though via extra-protocol negotiation.
>
> -- Laolu
>
>
> On Wed, Nov 7, 2018 at 2:38 PM lisa neigut  wrote:
>
>> Problem
>> 
>> Currently it’s difficult to reliably source inbound capacity for your
>> node. This is incredibly problematic for vendors and nodes hoping to setup
>> shop as a route facilitator. Most solutions at the moment require an
>> element of out of band negotiation in order to find other nodes that can
>> help with your capacity needs.
>>
>> While splicing and dual funding mechanisms will give some relief by
>> allowing for the initial negotiation to give the other node an opportunity
>> to put funds in either at channel open or after the fact, the problem of
>> finding channel liquidity is still left as an offline problem.
>>
>> Proposal
>> =
>> To solve the liquidity discovery problem, I'd like to propose allowing
>> nodes to advertise initial liquidity matching. The goal of this proposal
>> would be to allow nodes to independently source inbound capacity from a
>> 'market' of advertised liquidity rates, as set by other nodes.
>>
>> A node, via their node_announcement, can advertise that they will match
>> liquidity and a fee rate that they will provide to any incoming
>> open_channel request that indicates requests it.
>>
>> `node_announcement`:
>> new feature flag: option_liquidity_provider
>> data:
>>  [4 liquidity_fee_proportional_millionths] (option_liquidity_provider)
>> fee charged per satoshi of liquidity added at channel open
>>  [4 liquidity_fee_base_msat] (option_liquidity_provider) base fee charged
>> for providing liquidity at channel open
>>
>> `open_channel`:
>> new feature flag (channel_flags): option_liquidity_buy [2nd least
>> significant bit]
>> push_msat: set to fee payment for requested liquidity
>> [8 liquidity_msat_request]: (option_liquidity_buy) amount of dual funding
>> requested at channel open
>>
>> `accept_channel`:
>> tbd. hinges on a dual funding proposal for how second node would send
>> information about their funding input.
>>
>> If a node cannot provide the liquidity requested in `open_channel`, it
>> must return an error.
>> If the amount listed in `push_msat` does not cover the amount of
>> liquidity provided, the liquidity provider node must return an error.
>>
>> Errata
>> ==
>> It's an open question as to whether or not a liquidity advertising node
>> should also include a maximum amount of liquidity that they will
>> match/provide. As currently proposed, the only way to discover if a node
>> can meet your liquidity requirement is by sending an open channel request.
>>
>> This proposal depends on dual funding being possible.
>>
>> Should a node be able to request more liquidity than they put into the
>> channel on their half? In the case of a vendor who wants inbound capacity,
>> capping the liquidity request allowed seems unnecessary.
>>
>> Conclusion
>> ===
>> Allowing nodes to advertise liquidity paves the way for automated node
>> re-balancing. Advertised liquidity creates a market of inbound capacity
>> that any node can take advantage of, reducing the amount of out-of-band
>> negotiation needed to get the inbound capacity that you need.
>>
>>
>> Credit to Casey Rodamor for the initial idea.
>> ___
>> Lightning-dev mailing list
>> Lightning-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Proposal for Advertising Channel Liquidity

[lisa neigut]

On Wed, Nov 7, 2018 at 10:17 PM Anthony Towns  wrote:

> On Wed, Nov 07, 2018 at 06:40:13PM -0800, Jim Posen wrote:
> > can simply close the channel. So if I'm charging for liquidity, I'd
> actually
> > want to charge for the amount (in mSAT/BTC) times time.
>
> So perhaps you could make a market here by establishing a channel saying
> that
>
>   "I'll pay 32 msat per 500 satoshi per hour for the first 3 days"
>
> When you open the channel with 500,000 satoshi donated by the other guy,
> you're then obliged to transfer 32 satoshi every hour to the other guy
> for three days (so a total of 14c or so).
>
> If the channel fails beforehand, they don't get paid; if you stop
> paying you can still theoretically do a mutual close.
>

I think that this can also be gamed by a second, cooperating node that
sends payments through the channel to meet the rate and capture the fees
for the first. You can make this less likely by charging higher
transmission fees that make such an attack infeasible, and it's less
'damaging' than an immediate close in that there's still open capacity
available for some time, at least until the 'bogus' payments have drained
the capacity that you solicited in the first place.


> Maybe a complicated addition to the protocol though?
>
> Cheers,
> aj
>
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Proposal for Advertising Channel Liquidity

[lisa neigut]

Hello ZmnSCPxj,

You bring up some good points.

On Wed, Nov 7, 2018, 21:19 ZmnSCPxj  Good morning Lisa,
>
> On Wednesday, November 7, 2018 2:17 PM, ZmnSCPxj via Lightning-dev <
> lightning-dev@lists.linuxfoundation.org> wrote:
>
> Good morning Lisa,
>
> >Should a node be able to request more liquidity than they put into the
> channel on their half? In the case of a vendor who wants inbound capacity,
> capping the liquidity request
> >allowed seems unnecessary.
>
> My initial thought is that it would be dangerous to allow the initiator of
> the request to request for arbitrary capacity.
>
> For instance, suppose that, via my legion of captive zombie computers
> (which are entirely fictional and exist only in this example, since I am an
> ordinary human person) I have analyzed the blockchain and discovered that
> you have 1.0 BTC you have reserved for liquidity requests under this
> protocol.  I could then have one of those computers spin up a temporary
> Lightning Node, request for 1.0BTC incoming capacity with only some nominal
> fee, then shut down the node permanently, leaving your funds in an
> unuseable channel, unable to earn routing fees or such.  This loses you
> potential earnings from this 1.0 BTC.
>
> If instead I were obligated to have at least greater capacity tied into
> this channel, then I would also be tying up at least 1.0 BTC into this
> channel as well, making this attack more expensive for me, as it also loses
> me any potential earnings from the 1.0 BTC of my own that I have locked up.
>
> As you point out below, at the very least a liquidity providing node would
get paid. Another thing worth considering is that the spec, as written, is
merely a mechanism for advertising and receiving offers for dual funding.
There are no rules about what offers you, as a liquidity advertising node,
have to accept. A node operator has the flexibility to reject any offer
above or below their stated fee rate, or if they don't relish the idea of
funding a badly skewed channel. If you're worried about capital being tied
up unnecessarily, you can reject offers without a sizeable input of their
own.

There are, however, scenarios where requests for badly skewed channels make
sense. Imagine that you're a large vendor, such as Amazon. You're likely
not going to ever need much outbound capacity, but you will be perpetually
in the market for more inbound capacity.

In fact, as a liquidity provider, I think that you'll probably be delighted
to have an open channel with Amazon, as there's a good chance that channel
will be highly utilized, which means more fee traffic for you, and a high
probability that they'll be requesting more liquidity from you in the
future, as the existing channel gets unbalanced.


> A counterpoint to this argument, however, is that if the fee for the
> liquidity is high enough, then it does not matter to you whether I use the
> 1.0 BTC or not: you have already been paid for it.
>
> This however brings up the other potential attack:
>
> 1.  I advertise that I have 1.0 BTC available for liquidity requests.
> 2.  You answer this advertisement, and pay me a good fee for this 1.0 BTC
> being locked into a channel.
> 3.  After the channel is opened, I immediately close it, having earned the
> fee for liquidity, without in fact delivering that liquidity.
>
> Perhaps we can modify channel commitment transactions for channels opened
> via liquidity requests, so that they have an `nSequence` that prevents them
> from being claimed for a month or so.  What do you think?
>

At what point should a liquidity providing node (maker) be able to close
the channel? Immediately is not very beneficial to either of you -- you
both tied up your money for the time required to push through bitcoin txns
through, plus you lose closing + opening fees. Stipulating a length of time
isn't necessarily beneficial either -- if you've connected to a high volume
payment channel, the liquidity you've provided will be used up rather
quickly, rendering the channel itself pretty useless.

I think there's definitely some clever things we can do to provide stronger
guarantees around a 'minimum service offer', and they can be investigated
independently of the advertisement mechanism that I've proposed here.
Independent of what guarantees the protocol offers, there's a bunch of
strategies that individual nodes can additionally take to limit potential
losses: starting with by soliciting small liquidity offers, shopping around
for the best rates, blacklisting IP addresses/node id's of unreliable
nodes, using a ratcheting mechanism (start with a small liquidity request
that you close/rebalance upward as the incoming capacity is drained).

Considering incentives, keeping a high-traffic channel open should be worth
more in routing fees than the liquidity that you've provided. If the
liquidity market acts rationally, it should price itself to reflect this
reality and the risk of being laolu'd should remain fairly insignificant.


> 

Re: [Lightning-dev] Splicing Proposal: Feedback please!

[lisa neigut]

To add some context to this, if you start accepting HTLC's for the new
balance after the parallel commitment is made, but before the re-anchor is
buried, there's the potential for a race condition between a unilateral
close (or any revoked commitment transaction) and the re-anchoring
commitment transaction, that spends the 'pre-committed' UTXO of splicing in
funds and the original funding transaction.

You can get around this by waiting until both the pre-commitment UTXO and
the re-anchor have cleared a minimum depth before accepting HTLC's for the
new balance totals, but that's twice as long of a wait as the first,
synchronized re-commitment scheme that Rusty originally proposed.

It also makes leaving the original funding transaction 'exposed' (ie Rene's
version of parallel splice) untenable, as there's always the risk of an old
state being published to consume that input. This foobars your current HTLC
commitments.

On Tue, Oct 16, 2018 at 3:31 PM Rusty Russell  wrote:

> Rusty Russell  writes:
> > If we're going to do side splice-in like this, I would use a very
> > different protocol: the reason for this protocol was to treat splice-in
> > and splice-out the same, and inline splice-in requires wait time.  Since
> > splice-out doesn't, we don't need this at all.
> >
> > It would look much more like:
> >
> > 1. Prepare any output with script of specific form. eg:
> > OP_DEPTH 3 OP_EQUAL OP_IF
> >   OP_CHECKMULTISIG
> > OP_ELSE
> >  OP_CHECKLOCKTIMEVERIFY OP_DROP
> >  OP_CHECKSIG
> > OP_ENDIF
> >
> > 1. type: 40 (`splice_in`) (`option_splice`)
> > 2. data:
> >* [`32`:`channel_id`]
> >* [`8`: `satoshis`]
> >* [`32`: `txid`]
> >* [`4`: `txoutnum`]
> >* [`4`: `blockheight`]
> >* [`33`: `myrescue_pubkey`]
> >
> > 1. type: 137 (`update_splice_in_accept`) (`option_splice`)
> >data:
> >* [`32`:`channel_id`]
> >* [`32`: `txid`]
> >* [`4`: `txoutnum`]
> >
> > 1. type: 138 (`update_splice_in_reject`) (`option_splice`)
> >data:
> >* [`32`:`channel_id`]
> >* [`32`: `txid`]
> >* [`2`:`len`]
> >* [`len`:`errorstr`]
> >
> > The recipient of `splice_in` checks that it's happy with the
> > `blockheight` (far enough in future).  Once it sees the tx referred to
> > buried to its own `minimum_depth`, it checks output is what they
> > claimed, then sends `update_splice_in_accept`; it's followed up
> > `commitment_signed` like normal, but from this point onwards, all
> > commitment txs signatures have one extra sig.
>
> Lisa started asking pointed questions, and so I noticed that parallel
> splice doesn't work with Poon-Dryja channels.
>
> The counterparty can spend the old funding txout with a revoked spend.
> Sure, I can take all the money from that, but what about the spliced
> input?
>
> I came up with increasingly elaborate workarounds, but nothing stuck.
>
> Back to Plan A...
> Rusty.
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev