Re: [Sks-devel] Analyzing key server data

[dirk astrath]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

> Hockeypuck uses MongoDB or PostgreSQL as key storage backends. I
> don't know about MongoDB, but I know that while the tables are kept
> quite minimal in PostgreSQL, you could easily create a new table
> which inherits from the original one, adding more columns you can
> filter for. The size of the whole database in PostgreSQL is 21 GB.

I run two Hockeypuck-Instances (started with the same set of keys) on
two different machines (one machine at home, one notebook):

hockeypuck.fidocon.de (using nginx ... ;-) )

semi(sksclient2) runs on a postgres-database
vdr-sg  (skslcient3) runs on a mongodb-database

There is one issue I detected using hockeypuck:

The "total number" is updated, but the daily/hourly statistics doesn't
seem to be stored after doing a restart of the hockeypuck service.

Both hockeypuck-servers are not 24/7 currently (due to limited memory
resources on both machines) ... ;-(

About the database-sizes:

berkley (sks)   Around 13 GB
mongodb (hockeypuck)Around 20 GB
postgres (hockeypuck)   Around 25 GB

If somebody needs dumps of these databases feel free to contact me ... ;-)

Kind regards ...

PS: If you try to access the servers using IPv4 you'll reach one of my
sks-keyservers ... ;-)
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlW/cGgACgkQVuf/iihAxwjIjgCfYFg0HxUNbqNCvyA1ZGbRf/yH
DgEAn10bBU1Gs8pFSIviveQcJr8vg5+7
=Sfpv
-END PGP SIGNATURE-

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Analyzing key server data

[ma...@wk3.org]

Hi,

Hockeypuck uses MongoDB or PostgreSQL as key storage backends. I don't know 
about MongoDB, but I know that while the tables are kept quite minimal in 
PostgreSQL, you could easily create a new table which inherits from the 
original one, adding more columns you can filter for.

The size of the whole database in PostgreSQL is 21 GB.

It's quite fast.


Sincerely,

Malte

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Analyzing key server data

[Daniel Roesler]

Here's what I got with openpgp-python[1]. Seems to parse fine, though
I haven't added signature validation yet.

https://paste.ee/p/2NDMF

Daniel

[1]: python openpgp.py -m ~/Desktop/3A6CED98.asc | python -c "import
json, sys; print json.dumps(json.loads(sys.stdin.read()), indent=4,
sort_keys=True)" > ~/Desktop/3A6CED98.asc.pretty.json

On Mon, Jul 6, 2015 at 1:40 PM, Tobias Mueller  wrote:
> Hi!
>
> On Mon, Jul 06, 2015 at 08:22:48AM -0700, Daniel Roesler wrote:
>> Ok, just confirmed that openpgp-python can still parse the pool.
> Cool! Thanks for that.
>
>> Do you have a public key that throws an exception?
> I tried a couple of years back:
> http://lists.nongnu.org/archive/html/sks-devel/2012-06/msg00025.html
>
> Cheers,
>   Tobi

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Analyzing key server data

[Tobias Mueller]

Hi!

On Mon, Jul 06, 2015 at 08:22:48AM -0700, Daniel Roesler wrote:
> Ok, just confirmed that openpgp-python can still parse the pool.
Cool! Thanks for that.

> Do you have a public key that throws an exception?
I tried a couple of years back:
http://lists.nongnu.org/archive/html/sks-devel/2012-06/msg00025.html

Cheers,
  Tobi

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Analyzing key server data

[Daniel Roesler]

Ok, just confirmed that openpgp-python can still parse the pool. Do
you have a public key that throws an exception?

https://research.daylightpirates.org/sks-dumps/2015-07-06/json/

NOTE: Each file contains 10,000 rows of json objects (1 for each public key).

Daniel

On Sun, Jul 5, 2015 at 7:16 PM, Daniel Roesler  wrote:
> Hmmm, I maintain openpgp-python and was able convert a full dump to
> json about two months ago. Perhaps one of the recent updates broke
> things? Do you have a particular public key that raises an Exception?
>
> Daniel
>
> On Fri, Jul 3, 2015 at 2:23 AM, Tobias Mueller  wrote:
>> Hi Hanno!
>>
>> On So, 2015-03-22 at 12:58 +0100, Hanno Böck wrote:
>>> Code:
>>> https://github.com/hannob/pgpecosystem
>>>
>> This is great work, thanks.
>>
>> I tried to parse SKS dumps in the past, but I failed miserably, using
>> python-openpgp.
>> I'm looking forward to seeing your implementation.
>>
>> Have you seen the Analysing the Web of Trust paper?
>> http://link.springer.com/chapter/10.1007/978-3-642-23822-2_27
>> http://dl.acm.org/citation.cfm?id=844108
>>
>> There is also .
>>
>> Cheers,
>>   Tobi
>>
>>
>> ___
>> Sks-devel mailing list
>> Sks-devel@nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/sks-devel

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Analyzing key server data

[Daniel Roesler]

Hmmm, I maintain openpgp-python and was able convert a full dump to
json about two months ago. Perhaps one of the recent updates broke
things? Do you have a particular public key that raises an Exception?

Daniel

On Fri, Jul 3, 2015 at 2:23 AM, Tobias Mueller  wrote:
> Hi Hanno!
>
> On So, 2015-03-22 at 12:58 +0100, Hanno Böck wrote:
>> Code:
>> https://github.com/hannob/pgpecosystem
>>
> This is great work, thanks.
>
> I tried to parse SKS dumps in the past, but I failed miserably, using
> python-openpgp.
> I'm looking forward to seeing your implementation.
>
> Have you seen the Analysing the Web of Trust paper?
> http://link.springer.com/chapter/10.1007/978-3-642-23822-2_27
> http://dl.acm.org/citation.cfm?id=844108
>
> There is also .
>
> Cheers,
>   Tobi
>
>
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Analyzing key server data

[Tobias Mueller]

Hi Hanno!

On So, 2015-03-22 at 12:58 +0100, Hanno Böck wrote:
> Code:
> https://github.com/hannob/pgpecosystem
> 
This is great work, thanks.

I tried to parse SKS dumps in the past, but I failed miserably, using
python-openpgp.
I'm looking forward to seeing your implementation.

Have you seen the Analysing the Web of Trust paper?
http://link.springer.com/chapter/10.1007/978-3-642-23822-2_27
http://dl.acm.org/citation.cfm?id=844108

There is also .

Cheers,
  Tobi


___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Analyzing key server data

[Daniel Kahn Gillmor]

On Sun 2015-03-22 10:33:01 -0500, Daniel Roesler wrote:
> I was under the impression that SKS verified signature packets both
> during upload and during gossip.

SKS does no cryptographic verification. :(  Even if it were to start
doing verification, it's not clear how that would work with
certifications from keys it doesn't know about.  And the introduction of
cryptographic verification would segment the SKS keyserver network into
those that do verification and those that do not; it's like applying a
filter -- it either needs to be done on all SKS instances or none of
them :/

--dkg

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Analyzing key server data

[Daniel Roesler]

Great paper! Thanks!

From the paper:
> However when trying to calculate the private keys it turns out most
> of these results aren't real signatures.

I was under the impression that SKS verified signature packets both
during upload and during gossip. If so, how did invalid or corrupt
signature packets make it into the database? Do you have a count of
the total number of invalid signature packets?

Daniel

On Sun, Mar 22, 2015 at 4:58 AM, Hanno Böck  wrote:
> Hi,
>
> I think this could be interesting for a couple of people:
>
> I had a project running in private for quite a while, I now published
> the details: I wrote a script that analyzes the dumps from key servers
> and puts the crypto values into a mysql database.
>
> This can be used to search for vulnerable keys or signatures on large
> scale. I did this for two potential threats: DSA signatures with
> duplicate k values and RSA keys with shared factors.
>
> The overall result is a good one: It seems OpenPGP implementations with
> completely broken random number generators exist, but they are a rare
> thing.
>
> Code:
> https://github.com/hannob/pgpecosystem
>
> Background paper:
> http://eprint.iacr.org/2015/262
>
> cu,
> --
> Hanno Böck
> http://hboeck.de/
>
> mail/jabber: ha...@hboeck.de
> GPG: BBB51E42
>
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel
>

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel