Re: [Sks-devel] Dumps/importing & de-peering (WAS: Re: SKS apocalypse mitigation)

[Andrew Gallagher]

On 05/05/18 17:28, brent s. wrote:
> 
>>> but the problem for
>>> (b) is the "standard place" - SKS/recon/HKP/peering is, by nature,
>>> unfederated/decentralized. sure, there's the SKS pool, but that
>>> certainly isn't required for peering (even with keyservers that ARE in
>>> the pool) nor running sks. how does one decide the "canonical" dump to
>>> be downloaded in (b)?
>>
>> There can be no canonical dump of course. Each peer can provide its own dump 
>> at a well known local URL. This is even more important if and when we allow 
>> divergent policy. 
> 
> hrm. i suppose, but i'm under the impression not many keyserver admins
> run their own dumps? (which i don't fault them for; the current dump i
> have in its uncompressed form is 11 GB (5054255 keys). granted, you
> don't see new keyserver turnups often, but still -- that can be a
> lengthy download, plus the fairly sizeable chunk of time it takes for
> the initial import.)

Right.

I've thought about this a bit more, and the bootstrapping issue can be
solved without requiring every keyserver to produce a unique dump. We
just need one more database [table]...!

Let us call it Limbo. It contains the hashes of objects that the local
server does not have and has never seen (so has never had the chance to
test against policy), but knows must exist because they were in another
server's blacklist.

When bootstrapping, all that the new server needs to know is a
reasonably complete list of hashes. If it knows the real data as well,
all the better. But for recon to get started, given that we can perform
fake recon, the hashes are sufficient.

When performing a dump, a reference server also dumps its local
blacklist. When loading that dump, the blacklist of the reference is
used to populate the fresh server's Limbo. Now, the fresh server can
generate a low-delta fake recon immediately, by merging the DB, Local-BL
(initially empty) and Limbo hash lists. Recon then proceeds as discussed
before, and so long as the peer graph is well-connected, new peers can
be added without having to reference their dumps.

Limbo entries will return 404, just like missing entries (and unlike
blacklist entries). But the server will request a proportion of the
Limbo entries from its peers during each catchup. This would happen at a
much higher rate than the blacklist cache refresh, but still low enough
that its peers shouldn't suffer from the extra load.

Let's say that at each recon, the number of missing keys is found to be
N. The local server will then request these N keys from its peer. If at
the same time it were to also request (M=a*N) limbo entries thus:

(SELECT hash from limbo where hash NOT IN (SELECT hash from
peer_bl_cache where peer = $PEER) LIMIT $M)`

the extra load on the peer should not be excessive, and Limbo should be
drained at a rate roughly proportional to the parameter `a` and the rate
of new keys.

(This would also be a good place to perform the peer_bl_cache refresh).

When calculating key deltas for pool membership purposes, the fresh
server should not include its Limbo database in the count. This will
ensure that servers do not get added to the pool until their Limbo is
well drained. Alternatively, we could make an explicitly drained Limbo a
condition for pool membership.

This still leaves the issue of eventual consistency as an open problem,
but it can be addressed manually by encouraging good graph connectivity.

-- 
Andrew Gallagher



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Dumps/importing & de-peering (WAS: Re: SKS apocalypse mitigation)

[brent s.]

On 05/05/2018 10:22 AM, Andrew Gallagher wrote:
> 
>> On 5 May 2018, at 15:00, brent s.  wrote:
>>
>> (a) is taken care of by recon already (in a way),
> 
> According to a list message from earlier today it is not. If the delta is 
> small, recon proceeds. If it is large, it breaks catastrophically. There is 
> no (current) way to test nicely.
> 

sorry, should have clarified- i mean the "generating deltas" part of (a).

>> but the problem for
>> (b) is the "standard place" - SKS/recon/HKP/peering is, by nature,
>> unfederated/decentralized. sure, there's the SKS pool, but that
>> certainly isn't required for peering (even with keyservers that ARE in
>> the pool) nor running sks. how does one decide the "canonical" dump to
>> be downloaded in (b)?
> 
> There can be no canonical dump of course. Each peer can provide its own dump 
> at a well known local URL. This is even more important if and when we allow 
> divergent policy. 

hrm. i suppose, but i'm under the impression not many keyserver admins
run their own dumps? (which i don't fault them for; the current dump i
have in its uncompressed form is 11 GB (5054255 keys). granted, you
don't see new keyserver turnups often, but still -- that can be a
lengthy download, plus the fairly sizeable chunk of time it takes for
the initial import.)


-- 
brent saner
https://square-r00t.net/
GPG info: https://square-r00t.net/gpg-info



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Dumps/importing & de-peering (WAS: Re: SKS apocalypse mitigation)

[brent s.]

On 05/05/2018 08:30 AM, Andrew Gallagher wrote:
> 
>> On 5 May 2018, at 11:31, brent s.  wrote:
>>
>> it is SO IMPORTANT for both ends of the peering to have a relatively
>> recent keyset. i don't see how we can "fix" this without entirely
>> restructuring how HKP recon behaves, 
> 
> Yes. Perhaps it would be a good idea to systematise the dump/restore process 
> so that instead of a human being following written instructions, a new peer 
> of server A will attempt to a) probe server A to find the key difference b) 
> if the difference is large, download a dump from some standard place c) 
> reinitialise itself before trying again. 
> 
> Removing human error from such processes is A Good Thing in any case...
> 
> A
> 
> 

(a) is taken care of by recon already (in a way), but the problem for
(b) is the "standard place" - SKS/recon/HKP/peering is, by nature,
unfederated/decentralized. sure, there's the SKS pool, but that
certainly isn't required for peering (even with keyservers that ARE in
the pool) nor running sks. how does one decide the "canonical" dump to
be downloaded in (b)?

i WOULD say that removing human error is good, and normally i'd totally
agree - but i think this should instead be solved in documentation, as
implementing it in the software itself seems like a lot of work that
even breaks part of SKS/peering philosophy (to me, at least) with low
payoff. i can't speak to it, but i'd be curious if anyone could
anecdotally recall how often peering requests are made to this list
without them first importing a dump.


i instead propose that:

- in the default membership file, a note should be added to the comments
at the beginning about importing a dump first for peering with
"public(?) peers" should be done (and link to one or both of [0])

- in the man page for sks, under "FILES..membership", a note be added
saying the same/similar

- in /README.md, under "Setup and Configuration..### Membership
file", the same note be added


This way, there is *no possible way* a new keyserver administrator will
even know HOW to peer WITHOUT first knowing that they should use a
keydump import beforehand.


Adding in an optional refusal threshold directive (max_key_delta or
something?) for a keycount delta of more than /n/ to sks.conf
(optionally perhaps with the ability to override that value per-peer in
membership?), however, would absolutely hold value, I think.



[0] https://bitbucket.org/skskeyserver/sks-keyserver/wiki/Peering
https://bitbucket.org/skskeyserver/sks-keyserver/wiki/KeydumpSources

-- 
brent saner
https://square-r00t.net/
GPG info: https://square-r00t.net/gpg-info



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Dumps/importing & de-peering (WAS: Re: SKS apocalypse mitigation)

[Andrew Gallagher]

> On 5 May 2018, at 11:31, brent s.  wrote:
> 
> it is SO IMPORTANT for both ends of the peering to have a relatively
> recent keyset. i don't see how we can "fix" this without entirely
> restructuring how HKP recon behaves, 

Yes. Perhaps it would be a good idea to systematise the dump/restore process so 
that instead of a human being following written instructions, a new peer of 
server A will attempt to a) probe server A to find the key difference b) if the 
difference is large, download a dump from some standard place c) reinitialise 
itself before trying again. 

Removing human error from such processes is A Good Thing in any case...

A



___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

[Sks-devel] Dumps/importing & de-peering (WAS: Re: SKS apocalypse mitigation)

[brent s.]

On 05/05/2018 03:48 AM, Phil Pennock wrote:
> On 2018-05-04 at 17:13 +0100, Andrew Gallagher wrote:
>> AFAICT, the limitation that SKS servers should only recon with known
>> peers was introduced as a measure against abuse. But it's a pretty
>> flimsy anti-abuse system considering that anyone can submit or search
>> for anything over the HKP interface without restriction.
>>
>> I think all SKS servers should attempt to recon with as many other
>> servers as they can find.
> 
> The SKS reconciliation algorithm scales with the count of the
> differences in key-counts.  If you peer with someone with no keys
> loaded, it will render your server nearly inoperable.
> 
> We've seen this failure mode before.  Repeatedly.  It's part of why I
> wrote the initial Peering wiki document.  It's why I walked people
> through showing how many keys they have loaded, and is why peering is so
> much easier these days: most people who post to sks-devel follow the
> guidance and take the hints, and get things sorted out before they post.

indeed; i'm with phil on this. the importing process is integral to the
turnup, which is why i offer keydumps[0] myself (available via both http
and rsync, compressed - maybe FTP someday as well), and offer
instructions in that section. and why i wrote this query tool[1]. and
this dumping script[2]. and packaged this[3].

(thanks, phil, by the way for those instructions. i found them super
helpful when i first turned up. and thanks to whomever it was on IRC(?)
that gave me the brilliant idea of running a modified second SKS
instance locally for no-downtime dumps!)

one of the key (no pun intended) criteria i have for peering is their
delta for # of keys off from mine. (i should add in a delta/comparison
function to [1] at some point. hrmmm...)

it is SO IMPORTANT for both ends of the peering to have a relatively
recent keyset. i don't see how we can "fix" this without entirely
restructuring how HKP recon behaves, which is no easy task from my
understanding (should it be even necessary first - i don't believe it
requires "fixing", personally).

> 
> This is why we only peer with people we whitelist, and why most people
> look for as much demonstration of Clue as they can get before peering,
> and it's a large part of why we do see de-peering when actions
> demonstrate a lack of trustworthiness.

relevant to this point, i'm still relatively new to keyserver
administration and this list - is there a sort of established procedure
or policy for "announcing" a peer that individuals should de-peer with
(should they be peering with said peer)? what incident response policy
should one follow? what criteria/actions would lead to suggested de-peering?


i diverted the thread because i feel we're crossing into off-topic with
those questions i had and i don't want to hijack the original topic,
since it seems to still be under consideration.



[0] http://mirror.square-r00t.net/#dumps
[1] https://git.square-r00t.net/OpTools/tree/gpg/keystats.py
[2] https://git.square-r00t.net/OpTools/tree/gpg/sksdump.py
[3] https://aur.archlinux.org/packages/sks-local/

-- 
brent "i said 'peer(ing|ed|)' too many times in this email" saner
https://square-r00t.net/
GPG info: https://square-r00t.net/gpg-info



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel