Re: Bug#835465: [Reproducible-builds] Bug#835465: python-apt: FTBFS: AptKeyError: recv from 'hkp://localhost:19191' failed for '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
On Tue 2016-08-30 10:31:50 -0400, Julian Andres Klode wrote: > On Tue, Aug 30, 2016 at 10:18:33AM -0400, Daniel Kahn Gillmor wrote: >> Can you point me to a report about that? I'm not seeing it in my scan >> of the bts at https://bugs.debian.org/src:software-properties > > There is none. And it does not seem to be that annoying. While it now only > shows key ids and no names, you can at least still remove them. So it > basically > works, but without it showing names, it's kind of useless for the target > audience. ok, i'm not going to worry about marking software-properties with a Breaks: unless i hear otherwise from you. thanks for giving me a heads-up, though. It does look like there's code in there that should probably be pulled out just have it use python-apt directly. > APT uses gpgv and then parses the output of [GNUPG:] VALIDSIG to > determine the key id used to sign (and forgets to include the > subkey field). > > If a list of allowed keys has been specified for the source via a > Signed-By field in the sources.list or the Release file, it then > checks that the key it parsed from GPG is in that list. > > If nobody specifies a Signed-By field (which is the default > right now), that of course does not affect anyone. I do want > to roll out the Release-file Signed-By field at some point > though, as it (very) slightly improves security. Huh, i was unaware of the Signed-By field, actually, but i'm happy to see it. I'm not sure what i think about a primary key being listed there by fingerprint while the Release file being signed by a subkey. I could see a (fairly weak) argument that rejecting that signature is "working as it should", but i think you're probably right that this might be better considered as a bug. at least it's a bug that fails closed rather than failing open though :) --dkg signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#835465: [Reproducible-builds] Bug#835465: python-apt: FTBFS: AptKeyError: recv from 'hkp://localhost:19191' failed for '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
On Tue, Aug 30, 2016 at 10:18:33AM -0400, Daniel Kahn Gillmor wrote: > On Tue 2016-08-30 09:47:34 -0400, Julian Andres Klode wrote: > > On Tue, Aug 30, 2016 at 09:32:15AM -0400, Daniel Kahn Gillmor wrote: > >> On Tue 2016-08-30 08:49:20 -0400, Julian Andres Klode wrote: > >> >> apt/auth.py appears to want to force gnupg to store its secret key > >> >> material in secring.gpg. This isn't a best practice, and modern > >> >> versions of gpg do not do so by default. I'd recommend dropping > >> >> tmp_secret_keyring entirely. > >> > > >> > Hmm, there should not even be any secret key material, as apt only > >> > deals with public keys. > >> > >> agreed, all the more reason to strip out those extra directives ;) > > > > I assume that might change behavior if used with gpg1, so I'd rather > > keep it in if it does no harm. > > There should be no behavior that changes based on secret keys if secret > keys are never used. I'd rather not have spare directives floating > around that we don't need -- i've been doing triage on things that try > to manipulate gnupg programmatically, and the simpler we can make them > the easier it will be to fix any problems that come up in GnuPG itself, > i think. There's a lot of cruft, and it would help my sanity to > minimize the cruft. Mmh, okay. > > > I don't really care. More important are probably Breaks for > > software-properties, > > it's Authentication tab is fairly broken now. I think that's also where > > apt.auth > > was split off from, but I'm not entirely sure... > > Can you point me to a report about that? I'm not seeing it in my scan > of the bts at https://bugs.debian.org/src:software-properties There is none. And it does not seem to be that annoying. While it now only shows key ids and no names, you can at least still remove them. So it basically works, but without it showing names, it's kind of useless for the target audience. > > >> > Maybe there's also an option to display fingerprints instead of keyids > >> > in --with-colons --list-keys? > >> > >> sure! > >> > >> gpg --fixed-list-mode --with-fingerprint --with-fingerprint > >> --with-colons --list-keys > >> > >> will produce lines of the form: > >> > >> fpr:0EE5BE979282D80B9F7540F1CCD2ED94D21739E9: > > > > How awful. There should be a way to integrate this into the pub output > > To be clear, the output has pub:, then fpr:, then uid: lines in a row. > it's pretty straightforward to track as you read the lines linearly. > for any uid line, it is associated with the most recent fpr line, which > itself is associated with the most recent pub line. Yes. > > the uid line is split out because you can have multiple uids associated > with each pub+fpr. > > For the fields we're interested in, this is the same output across all > versions. > > > (and with all the other breaks, it should have just gone fingerprint > > by default everywhere). > > i'm working on that, but there are internal data structure > considerations that make it more costly to display the fingerprint > (unfortunately, the datastructures in the keyring are 64-bit keyids, not > full fingerprints). > > > APT's subkey handling is fairly broken anyway (it's gpg verification > > does not consider subkeys at all, so if you specify a list signed-by > > of master key ids, APT would fail to validate a repo signed with a > > subkey, unless the subkey is in the list itself...) > > interesting -- how is this done? i thought apt was using gpgv to verify > the signatures, and if there are subkeys in the keyrings gpgv knows > about gpgv will be willing to accept those subkeys. Are you saying apt > itself parses the output of gpgv and the fingprints in it to some > internal list of acceptable fingerprints? If you could point to the > right spot in the source, i'd be happy to look into it further. APT uses gpgv and then parses the output of [GNUPG:] VALIDSIG to determine the key id used to sign (and forgets to include the subkey field). If a list of allowed keys has been specified for the source via a Signed-By field in the sources.list or the Release file, it then checks that the key it parsed from GPG is in that list. If nobody specifies a Signed-By field (which is the default right now), that of course does not affect anyone. I do want to roll out the Release-file Signed-By field at some point though, as it (very) slightly improves security. -- Debian Developer - deb.li/jak | jak-linux.org - free software dev When replying, only quote what is necessary, and write each reply directly below the part(s) it pertains to (`inline'). Thank you. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#835465: [Reproducible-builds] Bug#835465: python-apt: FTBFS: AptKeyError: recv from 'hkp://localhost:19191' failed for '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
On Tue 2016-08-30 09:47:34 -0400, Julian Andres Klode wrote: > On Tue, Aug 30, 2016 at 09:32:15AM -0400, Daniel Kahn Gillmor wrote: >> On Tue 2016-08-30 08:49:20 -0400, Julian Andres Klode wrote: >> >> apt/auth.py appears to want to force gnupg to store its secret key >> >> material in secring.gpg. This isn't a best practice, and modern >> >> versions of gpg do not do so by default. I'd recommend dropping >> >> tmp_secret_keyring entirely. >> > >> > Hmm, there should not even be any secret key material, as apt only >> > deals with public keys. >> >> agreed, all the more reason to strip out those extra directives ;) > > I assume that might change behavior if used with gpg1, so I'd rather > keep it in if it does no harm. There should be no behavior that changes based on secret keys if secret keys are never used. I'd rather not have spare directives floating around that we don't need -- i've been doing triage on things that try to manipulate gnupg programmatically, and the simpler we can make them the easier it will be to fix any problems that come up in GnuPG itself, i think. There's a lot of cruft, and it would help my sanity to minimize the cruft. > I don't really care. More important are probably Breaks for > software-properties, > it's Authentication tab is fairly broken now. I think that's also where > apt.auth > was split off from, but I'm not entirely sure... Can you point me to a report about that? I'm not seeing it in my scan of the bts at https://bugs.debian.org/src:software-properties >> > Maybe there's also an option to display fingerprints instead of keyids >> > in --with-colons --list-keys? >> >> sure! >> >> gpg --fixed-list-mode --with-fingerprint --with-fingerprint --with-colons >> --list-keys >> >> will produce lines of the form: >> >> fpr:0EE5BE979282D80B9F7540F1CCD2ED94D21739E9: > > How awful. There should be a way to integrate this into the pub output To be clear, the output has pub:, then fpr:, then uid: lines in a row. it's pretty straightforward to track as you read the lines linearly. for any uid line, it is associated with the most recent fpr line, which itself is associated with the most recent pub line. the uid line is split out because you can have multiple uids associated with each pub+fpr. For the fields we're interested in, this is the same output across all versions. > (and with all the other breaks, it should have just gone fingerprint > by default everywhere). i'm working on that, but there are internal data structure considerations that make it more costly to display the fingerprint (unfortunately, the datastructures in the keyring are 64-bit keyids, not full fingerprints). > APT's subkey handling is fairly broken anyway (it's gpg verification > does not consider subkeys at all, so if you specify a list signed-by > of master key ids, APT would fail to validate a repo signed with a > subkey, unless the subkey is in the list itself...) interesting -- how is this done? i thought apt was using gpgv to verify the signatures, and if there are subkeys in the keyrings gpgv knows about gpgv will be willing to accept those subkeys. Are you saying apt itself parses the output of gpgv and the fingprints in it to some internal list of acceptable fingerprints? If you could point to the right spot in the source, i'd be happy to look into it further. --dkg signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#835465: [Reproducible-builds] Bug#835465: python-apt: FTBFS: AptKeyError: recv from 'hkp://localhost:19191' failed for '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
On Tue, Aug 30, 2016 at 09:32:15AM -0400, Daniel Kahn Gillmor wrote:
> On Tue 2016-08-30 08:49:20 -0400, Julian Andres Klode wrote:
> >> apt/auth.py appears to want to force gnupg to store its secret key
> >> material in secring.gpg. This isn't a best practice, and modern
> >> versions of gpg do not do so by default. I'd recommend dropping
> >> tmp_secret_keyring entirely.
> >
> > Hmm, there should not even be any secret key material, as apt only
> > deals with public keys.
>
> agreed, all the more reason to strip out those extra directives ;)
I assume that might change behavior if used with gpg1, so I'd rather
keep it in if it does no harm.
>
> >> I'll be releasing a new version of gnupg shortly that will explicitly
> >> declare that it Breaks: python-apt (<= 1.1.0~beta4).
> >
> > I think that's a bit overkill. While this part of python-apt is broken
> > with the new gnupg, the rest works fine; and nobody uses the apt.auth
> > module. Not to mention that I'm deprecating it, as we deprecated the gpg
> > stuff in apt-key.
>
> If you want me to remove the Breaks: i can do so -- my goal was to
> address the concerns raised in https://bugs.debian.org/835349.
>
> If you'd rather that i not provide a Breaks: or a Conflicts: for
> python-apt, i can avoid it -- speak up though, i'm hoping to release the
> next version of gnupg2 to unstable shortly :)
I don't really care. More important are probably Breaks for software-properties,
it's Authentication tab is fairly broken now. I think that's also where apt.auth
was split off from, but I'm not entirely sure...
>
> >> Ideally, the next version of python-apt can have these bugs fixed and it
> >> will work cleanly with the modern version of gnupg.
> >
> > Sure. But we should really support both old and new gpg versions, otherwise
> > it gets a bit annoying.
> >
> > Maybe there's also an option to display fingerprints instead of keyids
> > in --with-colons --list-keys?
>
> sure!
>
> gpg --fixed-list-mode --with-fingerprint --with-fingerprint --with-colons
> --list-keys
>
> will produce lines of the form:
>
> fpr:0EE5BE979282D80B9F7540F1CCD2ED94D21739E9:
How awful. There should be a way to integrate this into the pub output (and with
all the other breaks, it should have just gone fingerprint by default
everywhere).
But oh well, it's better than nothing.
>
> The hex string shows up in $10 for "awk -F:", fields[9] in python after
> fields = line.split(":").
>
> providingn --with-fingerprint twice ensures that you get fingerprints
> for both primary keys and subkeys -- if that's what you want.
Hmm, I don't know. APT's subkey handling is fairly broken anyway (it's
gpg verification does not consider subkeys at all, so if you specify a list
signed-by of master key ids, APT would fail to validate a repo signed with
a subkey, unless the subkey is in the list itself...)
>
> >> However, if your next upload of python-apt can't be built or run against
> >> modern versions of GnuPG
> >
> > That would be silly :)
>
> i'm glad it will be straightforward to sort it out ;)
>
> Thanks for your work on this,
>
> --dkg
--
Debian Developer - deb.li/jak | jak-linux.org - free software dev
When replying, only quote what is necessary, and write each reply
directly below the part(s) it pertains to (`inline'). Thank you.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#835465: [Reproducible-builds] Bug#835465: python-apt: FTBFS: AptKeyError: recv from 'hkp://localhost:19191' failed for '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
On Tue 2016-08-30 08:49:20 -0400, Julian Andres Klode wrote:
>> apt/auth.py appears to want to force gnupg to store its secret key
>> material in secring.gpg. This isn't a best practice, and modern
>> versions of gpg do not do so by default. I'd recommend dropping
>> tmp_secret_keyring entirely.
>
> Hmm, there should not even be any secret key material, as apt only
> deals with public keys.
agreed, all the more reason to strip out those extra directives ;)
>> I'll be releasing a new version of gnupg shortly that will explicitly
>> declare that it Breaks: python-apt (<= 1.1.0~beta4).
>
> I think that's a bit overkill. While this part of python-apt is broken
> with the new gnupg, the rest works fine; and nobody uses the apt.auth
> module. Not to mention that I'm deprecating it, as we deprecated the gpg
> stuff in apt-key.
If you want me to remove the Breaks: i can do so -- my goal was to
address the concerns raised in https://bugs.debian.org/835349.
If you'd rather that i not provide a Breaks: or a Conflicts: for
python-apt, i can avoid it -- speak up though, i'm hoping to release the
next version of gnupg2 to unstable shortly :)
>> Ideally, the next version of python-apt can have these bugs fixed and it
>> will work cleanly with the modern version of gnupg.
>
> Sure. But we should really support both old and new gpg versions, otherwise
> it gets a bit annoying.
>
> Maybe there's also an option to display fingerprints instead of keyids
> in --with-colons --list-keys?
sure!
gpg --fixed-list-mode --with-fingerprint --with-fingerprint --with-colons
--list-keys
will produce lines of the form:
fpr:0EE5BE979282D80B9F7540F1CCD2ED94D21739E9:
The hex string shows up in $10 for "awk -F:", fields[9] in python after
fields = line.split(":").
providingn --with-fingerprint twice ensures that you get fingerprints
for both primary keys and subkeys -- if that's what you want.
>> However, if your next upload of python-apt can't be built or run against
>> modern versions of GnuPG
>
> That would be silly :)
i'm glad it will be straightforward to sort it out ;)
Thanks for your work on this,
--dkg
signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#835465: [Reproducible-builds] Bug#835465: python-apt: FTBFS: AptKeyError: recv from 'hkp://localhost:19191' failed for '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
On Tue, Aug 30, 2016 at 06:35:07AM -0400, Daniel Kahn Gillmor wrote:
> Control: affects 835465 + gnupg2
>
> Hi python-apt folks--
>
> On Thu 2016-08-25 20:55:27 -0400, Chris Lamb wrote:
> > Source: python-apt
> > Version: 1.1.0~beta4
> > Severity: serious
> > Justification: fails to build from source
> > User: reproducible-builds@lists.alioth.debian.org
> > Usertags: ftbfs
> > X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org
> >
> > Dear Maintainer,
> >
> > python-apt fails to build from source in unstable/amd64:
> >
> > set -e; for python in python2.7 python3.5 ; do \
> > $python tests/test_all.py -q || [ "linux" = "hurd" ]; \
> > done;
> > Warning: apt-key output should not be parsed (stdout is not a terminal)
> > Warning: apt-key output should not be parsed (stdout is not a terminal)
> > Warning: apt-key output should not be parsed (stdout is not a terminal)
> > Warning: apt-key output should not be parsed (stdout is not a terminal)
> > Warning: apt-key output should not be parsed (stdout is not a terminal)
> > Warning: apt-key output should not be parsed (stdout is not a terminal)
>
> this warning is advice about one of the things that might be going wrong
> in some places in the test suite ;) I see that in upstream
> 7a9a292fd604bc164eed3d3fee1dc9167141d88c you're deprecating this, but i
> strongly caution against this.
>
> > [tests] Running on 2.7.12+ (default, Aug 4 2016, 20:04:34) [GCC 6.1.1
> > 20160724]
> > Using library_dir:
> > '/home/lamby/temp/cdt.20160826014142.YuAeJNcC8b.db.python-apt/python-apt-1.1.0~beta4/build/lib.linux-x86_64-2.7'WARNING:
> > Failed to read mirror file
> > WARNING: Failed to read mirror file
> > WARNING: Failed to read mirror file
> > WARNING: Failed to read mirror file
> > WARNING: Failed to read mirror file
> > WARNING: Failed to read mirror file
> > WARNING: Failed to read mirror file
> > WARNING: Failed to read mirror file
> > ==
> > ERROR: testAddKeyFromServer (test_auth.TestAuthKeys)
> > Install a GnuPG key from a remote server.
> > --
> > Traceback (most recent call last):
> > File
> > "/home/lamby/temp/cdt.20160826014142.YuAeJNcC8b.db.python-apt/python-apt-1.1.0~beta4/tests/test_auth.py",
> > line 220, in testAddKeyFromServer
> > "hkp://localhost:%d" % self.keyserver_port)
> > File
> > "/home/lamby/temp/cdt.20160826014142.YuAeJNcC8b.db.python-apt/python-apt-1.1.0~beta4/build/lib.linux-x86_64-2.7/apt/auth.py",
> > line 128, in add_key_from_keyserver
> > _add_key_from_keyserver(keyid, keyserver, tmp_keyring_dir)
> > File
> > "/home/lamby/temp/cdt.20160826014142.YuAeJNcC8b.db.python-apt/python-apt-1.1.0~beta4/build/lib.linux-x86_64-2.7/apt/auth.py",
> > line 156, in _add_key_from_keyserver
> > keyserver, keyid))
> > AptKeyError: recv from 'hkp://localhost:19191' failed for
> > '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
>
> apt/auth.py appears to want to force gnupg to store its secret key
> material in secring.gpg. This isn't a best practice, and modern
> versions of gpg do not do so by default. I'd recommend dropping
> tmp_secret_keyring entirely.
Hmm, there should not even be any secret key material, as apt only
deals with public keys.
>
> furthermore, recent versions of gnupg (>= 2.1) do not fetch things from
> keyservers directly -- if you want modern gpg to talk to the network,
> you'll need to ensure that dirmngr is installed.
>
> If you prefer to keep this test intact, you might want to build-depend
> on dirmngr.
>
> > ==
> > FAIL: testAddAndExportKey (test_auth.TestAuthKeys)
> > Add an example key.
> > --
> > Traceback (most recent call last):
> > File
> > "/home/lamby/temp/cdt.20160826014142.YuAeJNcC8b.db.python-apt/python-apt-1.1.0~beta4/tests/test_auth.py",
> > line 157, in testAddAndExportKey
> > WHEEZY_KEY.split("\n")[2:])
> > AssertionError: Lists differ: ['mQINBE+a7rUBEADQiEKtLOgqiq8Y... != ['',
> > 'mQINBE+a7rUBEADQiEKtLOgq...
> >
> > First differing element 0:
> > 'mQINBE+a7rUBEADQiEKtLOgqiq8YY/p7IFODMqGPR+o1vtXaksie8iTOh3Vxab38'
> > ''
> >
> > Second list contains 1 additional elements.
> > First extra element 81:
> > '-END PGP PUBLIC KEY BLOCK-'
> >
> > Diff is 5698 characters long. Set self.maxDiff to None to see it.
>
> This change is due to the fact that upstream has stopped emitting the
> Version: pseudoheader at all as of 2.1.14. Perhaps your build-dependency on
> gnupg should be (>= 2.1.14) and you should strike the Version:
> pseudoheader in WHEEZY_KEY in tests/test_auth.py
>
> > ==
> > FAIL: testAddAndListKey (test_auth.TestAuthKe
