Re: An evil idea :-)

[Marcel Waldvogel]

Gabor,

so, please call me Mr. Evil ;-)

A few weeks ago, I set up a simple Nginx load balancer (two lines with
https-portal[1]) statically seeded with the nodes that were in the pool
at that time for test purposes. It randomly returns the status page of
one of the backend servers, though, but that could be easily changed.

I wasn't as evil-minded to start faking a pool that would gradually
fake an increasing delta of keys against the "real" keys. Kudos for
that! ;-)

(The idea was triggered by the general unreliability of pool members. I
think we need to work on that. And also spam, trust, GDPR compliance,
and RTBF; but these are topics for a different thread.)

Greetings,
-Marcel

PS: Even if you are just load-balancing your own servers, you might
include the following line into your Nginx load balancer config
("non_idempotent" is fine, as even the POST requests that modify
anything, notably /pks/add, are in fact idempotent):

proxy_next_upstream error timeout http_500 http_502 http_503 http_504
http_404 http_429 non_idempotent;

[1] https://github.com/SteveLTN/https-portal

On Mon, 2021-03-22 at 21:08 +0100, Kiss Gabor (Bitman) wrote:
> One can decide to setup a proxy server without any own backend
> but redirecting queries to some of the existing servers.
> No one would recognize the cheating. :-)
> 
> Gabor
> -- 
> "Virgil Brigman back on the air" (Abyss)
> 

Re: Lying about Hockeypuck being SKS?

[Marcel Waldvogel]

pgpkeys.eu is in the pool, however, the "Software" table cell is red.
Further investigation showed that they spell "sks" in lower case,
unlike the real SKS.

Verifying against the source code [1], the server selection code is a
blacklist: "SKS" < 1.1.6, "GnuKS", and "hockeypuck" are blacklisted
(case-sensitive).

How about instead of claiming to be "SKS", Hockeypuck servers just
claim to be "Hockeypuck" (title case instead of lower case)? That would
be kind of a "white(list) lie". They will have a red cell, but that
will not preclude their

I have changed my Hockeypucks to return this version and will check in
30 minutes whether the theory can be corroborated.

Happy keyserving,
-Marcel

[1] 
https://git.sumptuouscapital.com/?p=sks-keyservers-pool.git;a=blob;f=sks-keyservers.net/status-srv/sks-status.inc.php;h=972bb5b56412ae54b8aade234ea02bb8c9545d45;hb=HEAD#l309

On Mon, 2021-03-22 at 21:13 +0100, Andreas Puls wrote:
> 
> 
> Am 22.03.2021 um 20:41 schrieb Marcel Waldvogel:
> > On Sun, 2021-03-21 at 22:56 +0100, Andreas Puls wrote:
> > > 
> > > I've created now a patch that just replaces in the json export
> > > contact
> > > with server_contact and Total with numkeys.
> > > https://github.com/apuls/hockeypuck/commit/34fbdfcf73b60e6001f3770b86d8750d1c8b5385
> > 
> > Great, thanks! I just merged this. Now my Hockeypuck server appears
> > in
> > the statistics.
> > 
> You're welcome!
> 
> > > In my hockeypuck configuration i've set Version to 1.1.6+ and
> > > Software
> > > to SKS
> > 
> Yeah, i've done it too. :)
> 
> > Hockeypuck is blacklisted in the sks-keyservers.net code, because
> > it
> > was not good enough to be incorporated into the pool when Kristian
> > wrote the code. Now, it seems to be in the same ballpark as SKS.
> > 
> > Asking Kristian to remove the Hockeypuck ban resulted in him
> > explaining
> > that he does not plan to change the code or accept changes;
> > instead, we
> > should set up our own fork of his code.
> > 
> > I think this leaves us with the following ways to progress:
> > 
> > a) We leave it as is, Hockeypuck is fine, but just not in the pool.
> > b) We create a second pool, where Hockeypuck is acceptable (and
> > probably SKS as well).
> > c) We agree that Hockeypuck lying to be SKS is accepted in the
> > pool,
> > and maybe even recommended.
> > 
> > I would favor (c), plus keeping the version number in the 2.x
> > range, so
> > that experts still can tell the difference.
> > 
> b would be great but i think this is a hell of work.
> 
> Since we haven't heard for a while from Kristian and the pool is
> working
> - ok more or less - i would go with option c too. Also with the
> Version
> string 2.x .
> > Opinions?
> We need to fix the peers field which will be reported via options=mr
> to
> meet the requirements from the pool skript.
> 
> > -Marcel
> Br
>    Andreas
> 

Re: Lying about Hockeypuck being SKS?

[Andrew Gallagher]

On 23/03/2021 09:19, Marcel Waldvogel wrote:
How about instead of claiming to be "SKS", Hockeypuck servers just claim 
to be "Hockeypuck" (title case instead of lower case)? That would be 
kind of a "white(list) lie". They will have a red cell, but that will 
not preclude their


Sounds good to me, will change that on pgpkeys.eu now.

--
Andrew Gallagher



OpenPGP_signature
Description: OpenPGP digital signature

Re: Pool dried up

[Andrew Gallagher]

Hi, Todd.

On 23/03/2021 03:37, Todd Fleisher wrote:
On Mar 22, 2021, at 13:28, Andrew Gallagher > wrote:


I happened to check the pool just now, and there are only three nodes 
in it:


1pgpkeys.uk [@]
2sks.pod01.fleetstreetops.com [@]
3sks.pod02.fleetstreetops.com [@]


BTW it has just happened again:

1 pgpkeys.eu[@] 
2 pgpkeys.uk[@] 
3 sks.pod02.fleetstreetops.com[@]   

Looking at the cached metadata it appears that when the spider ran, 
pod02.fleetstreetops nodes was unavailable, as was pgpkeys.co.uk 
 (the domain registration has expired).


I can’t speak for pgpkeys.co.uk , but I have not 
seen any issues with sks.pod02.fleetstreetops.com 
 (nor hkps.pool.sks-keyservers.net 
, which it powers) today.


Apologies, I didn't mean to cast doubt on the reliability of your node, 
but rather on that of the spider. It does not maintain much of a 
historical record, and so depends on a single measurement per node each 
hour for its operation. This makes it very vulnerable to transient 
issues, such as connection timeouts. One dropped connection, and 90% of 
the pool disappears for an hour, even if all the nodes stay up.


There are a few interlocking design issues at work here IMO, none of 
which are the responsibility of individual operators.


--
Andrew Gallagher



OpenPGP_signature
Description: OpenPGP digital signature

Re: keyserver.dobrev.eu is back running Hockeypuck

[Andrew Gallagher]

On 21/03/2021 21:48, Martin Dobrev wrote:
I had to play with mod_rewrite and force a redirect from 
//pks/lookup?op=stats&options=mr/ to //pks/lookup?op=stats /to let the 
script parse HTML. I don't have a proper explanation why peers and recon 
port are not picked from the produced JSON but left out (line 286/287).


I can confirm this works, and it has the unexpected side effect that 
pgpkeys.eu is now recognised as SKS, even though it is still declaring 
itself Hockeypuck.


--
Andrew Gallagher



OpenPGP_signature
Description: OpenPGP digital signature

Re: keyserver.dobrev.eu is back running Hockeypuck

[Martin Dobrev]

On 23/03/2021 12:58, Andrew Gallagher wrote:
> On 21/03/2021 21:48, Martin Dobrev wrote:
>> I had to play with mod_rewrite and force a redirect from
>> //pks/lookup?op=stats&options=mr/ to //pks/lookup?op=stats /to let
>> the script parse HTML. I don't have a proper explanation why peers
>> and recon port are not picked from the produced JSON but left out
>> (line 286/287).
>
> I can confirm this works, and it has the unexpected side effect that
> pgpkeys.eu is now recognised as SKS, even though it is still declaring
> itself Hockeypuck.
>
This only because Status page sets 'Software' to SKS by default
.
Only way to "overwrite" it is to output Software

in the stats page.

Re: Pool dried up

[Todd Fleisher]

> On Mar 23, 2021, at 02:38, Andrew Gallagher  wrote:
> 
> Hi, Todd.
> 
> On 23/03/2021 03:37, Todd Fleisher wrote:
>>> On Mar 22, 2021, at 13:28, Andrew Gallagher >> > wrote:
>>> 
>>> I happened to check the pool just now, and there are only three nodes in it:
>>> 
>>> 1pgpkeys.uk [@]
>>> 2sks.pod01.fleetstreetops.com [@]
>>> 3sks.pod02.fleetstreetops.com [@]
> 
> BTW it has just happened again:
> 
> 1 pgpkeys.eu[@]
> 2 pgpkeys.uk[@]
> 3 sks.pod02.fleetstreetops.com[@]
> 
>>> Looking at the cached metadata it appears that when the spider ran, 
>>> pod02.fleetstreetops nodes was unavailable, as was pgpkeys.co.uk 
>>>  (the domain registration has expired).
>> I can’t speak for pgpkeys.co.uk , but I have not seen 
>> any issues with sks.pod02.fleetstreetops.com 
>>  (nor hkps.pool.sks-keyservers.net 
>> , which it powers) today.
> 
> Apologies, I didn't mean to cast doubt on the reliability of your node, but 
> rather on that of the spider. It does not maintain much of a historical 
> record, and so depends on a single measurement per node each hour for its 
> operation. This makes it very vulnerable to transient issues, such as 
> connection timeouts. One dropped connection, and 90% of the pool disappears 
> for an hour, even if all the nodes stay up.

No worries. I was neither offended nor trying to dispute  that at times the 
pools run thin (if not fully bottom out). I was just trying to keep the things 
in perspective. My nodes also suffer from issues periodically and as a result I 
have seen issues where they are non-responsive at times. This usually isn’t 
visible to the public on account of the load balanced setup.

> There are a few interlocking design issues at work here IMO, none of which 
> are the responsibility of individual operators.

Indeed.

-T

> --
> Andrew Gallagher
> 



signature.asc
Description: Message signed with OpenPGP

Re: Lying about Hockeypuck being SKS?

[Martin Dobrev]

I did this earlier for keyserver.dobrev.eu.

On 23/03/2021 09:37, Andrew Gallagher wrote:
> On 23/03/2021 09:19, Marcel Waldvogel wrote:
>> How about instead of claiming to be "SKS", Hockeypuck servers just
>> claim to be "Hockeypuck" (title case instead of lower case)? That
>> would be kind of a "white(list) lie". They will have a red cell, but
>> that will not preclude their
>
> Sounds good to me, will change that on pgpkeys.eu now.
>

Re: keyserver.dobrev.eu is back running Hockeypuck

[Andrew Gallagher]

On 23/03/2021 13:32, Martin Dobrev wrote:


Only way to "overwrite" it is to output Software 
 
in the stats page.


Got it. Applied now in pgpkeys.eu for consistency. Thanks!

(And I've also fixed its ipv6...)

--
Andrew Gallagher



OpenPGP_signature
Description: OpenPGP digital signature

Re: Lying about Hockeypuck being SKS?

[Andrew Gallagher]

On 22/03/2021 22:02, Ryan Hunt wrote:


However I’d like to see some efforts made towards:
  - Rolling our SKS hacks back upstream with HP, initially this seems 
stupid but HP has already put in efforts to maintain compatibility with 
SKS peers.. I think a transitional SKS emulation mode that is easy to 
implement and maintain upstream is worthwhile, especially if we can come 
up with a plan to deprecate this nonsense down the road and its just to 
get us through the near future.


There is still a lot of traffic coming in to the pool, and it's become a 
running theme in the PGP space to have archaic software still in 
production use. However even when there were only three nodes in the 
pool earlier today, pgpkeys.eu didn't break a sweat serving its share.


There is a strong argument that it is better to break things entirely so 
that people still using old software don't have a false sense of 
security. But I really dislike the idea of breaking things without 
building the replacement first.


I’m thinking like a dedicated machine readable status/health 
API endpoint that the server can sign with its own key and the pool 
provider can verify its the server it claims to be, and make 
accommodations for blacklisted/removed keys/max key sizes/etc accounting 
for variations in key counts.


+1

TBH I think creating our own pool is likely our best option going 
forward, yeah it’ll take some time (ie years) before Distros and the 
various PGP clients come back.. but most of em that I used personally 
that came out the box w/the SKS pool no longer do so I think the damage 
has long been done.


I think the idea of a self-organising pool has some fundamental flaws. 
As was pointed out in another thread, it would be relatively easy for a 
malefactor to set up a few new pool nodes and then artificially inflate 
their key count in order to exclude honest operators. Even as an 
otherwise well-behaved member, it's possible for an operator to gather a 
log of IP addresses vs keys searched, in order to build a contacts graph 
(for the record, pgpkeys.eu permanently deletes its logs on a short 
cycle, currently <=48h and likely to decrease in the near future).


Tor is one way to address this, but it is not suitable for everyone. So 
running a keyserver comes with a requirement of trust and reputation, 
because privacy and security are not the same thing, and must sometimes 
be balanced.


My own view of the medium-term future is that there will be a diversity 
of keyserver operators that will not agree on everything (e.g. policy 
blacklists, data retention) and we have to find a way to coexist. The 
SKS pool structure is based on the assumption that there is one notional 
"complete" (if time-dependent) dataset, and all keyservers will tend to 
converge towards it. That was always a fragile assumption, and we should 
be working to make it unnecessary, so that individual operators can 
choose to what extent they wish to stay in sync with others.


Currently, we have a number of disconnected "big" keyserver operators 
(keys.openpgp.org, mailvelope, keybase, ubuntu...), but I don't believe 
it is user-friendly to expect non-experts to worry about which 
keyservers to publish to or fetch from. I hope to see them all back in 
communion with each other, even if only to distribute the bare minimum 
such as key revocations. This will by necessity have to be an open (and 
preferably well-defined) protocol, and it would be nice to have two or 
three independent implementations, but one step at a time.


So I suspect we'll end up with something more akin to DNS, where there 
are a smaller number of "root" servers, and then individual 
organisations can set up their own instances that field internal 
requests (thereby minimising behaviour leakage) but also sync with the 
roots. Individual users can choose to talk to whichever keyserver has 
the best reputation, rather than relying on a random pool member to be 
an honest one.


--
Andrew Gallagher



OpenPGP_signature
Description: OpenPGP digital signature