Hi Matt,
Appreciate your responses. Hope you'll bear with me as I'm a bit new to this.
> Instead of trying to make sure everyone’s gossip acceptance matches exactly,
> which as you point it seems like a quagmire, why not (a) do a sync on startup
> and (b) do syncs of the *new* things.
I'm not opposed to this technique, and maybe it ends up as a better solution.
The rationale for not going full Erlay approach was that it's far less overhead
to maintain a single sketch than to maintain a per-peer sketch and associated
state for every gossip peer. In this way there's very little cost to adding
additional gossip peers, which further encourages propagation and convergence
of the gossip network.
IIUC Erlay's design was concerned for privacy of originating nodes. Lightning
gossip is public by nature, so I'm not sure we should constrain ourselves to
the same design route without trying the alternative first.
> if we're gonna add a minisketch-based sync anyway, please lets also use it
> for initial sync after restart
This was out of the scope of what I had in mind, but I will give this some
thought. I could see how a block_height reference coupled with set
reconciliation could provide some better options here. This may not be all that
difficult to shoe-horn in.
Regardless of single sketch or per-peer set reconciliation, it should be easier
to implement with tighter rules on rate-limiting. (Keep in mind, the node's
graph can presumably be updated independently of the gossip it rebroadcasts if
desired.) As a thought experiment, if we consider a CLN-LDK set reconciliation,
and that each node is gossiping with 5 other peers in an evenly spaced
frequency, we would currently see 42.8 commonly accepted channel_updates over
an average 60s window along with 11 more updates which LDK accepts and CLN
rejects (spam.)[1] Assuming the other 5 peers have shared 5/6ths of this gossip
before the CLN/LDK set reconciliation, we're left with CLN seeing 7 updates to
reconcile, while LDK sees 18. Already we've lost 60% efficiency due to lack of
a common rate-limit heuristic.
I understand gossip traffic is manageable now, but I'm not sure it will be that
long before it becomes an issue. Furthermore, any particular set reconciliation
technique would benefit from a simple common rate-limit heuristic, not to
mention originating nodes, who may not currently realize their channel updates
are being rejected by a portion of the network due to differing criteria across
implementations.
Thanks,
Alex
[1] https://github.com/endothermicdev/lnspammityspam/blob/main/sampleoutput.txt
------- Original Message -------
On Thursday, April 21st, 2022 at 3:47 PM, Matt Corallo [email protected]
wrote:
> On 4/21/22 1:31 PM, Alex Myers wrote:
>
>> Hello Bastien,
>>
>> Thank you for your feedback. I hope you don't mind I let it percolate for a
>> while.
>>
>> Eclair doesn't do any rate-limiting. We wanted to "feel the pain" before
>> adding
>> anything, and to be honest we haven't really felt it yet.
>>
>> I understand the “feel the pain first” approach, but attempting set
>> reconciliation has forced me to
>> confront the issue a bit early.
>>
>> My thoughts on sync were that set-reconciliation would only be used once a
>> node had fully synced
>> gossip through traditional means (initial_routing_sync / gossip_queries.)
>> There should be many
>> levers to pull in order to help maintain sync after this. I'm going to have
>> to experiment with them
>> a bit before I can claim they are sufficient, but I'm optimistic.
>
> Please, no. initial_routing_sync was removed from most implementations (it
> sucks) and gossip queries
> is broken in at least five ways. May we can recover it by adding yet more
> extensions but if we're
> gonna add a minisketch-based sync anyway, please lets also use it for initial
> sync after restart
> (unless you have no channels at all, in which case lets maybe revive
> initial_routing_sync...)
>
> Matt
_______________________________________________
Lightning-dev mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
