Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
On Tue 2019-03-05 10:57:03 -0800, Felix Lechner wrote: > With source format 3.0 (git) that logic even found a way into the packaging > system. Let's flip it around for a moment: Why not validate upstream > signatures when the package is built? sorry, i think i'm still not following. I *do* validate uptsream signatures when i build the package. I *also* want other people to be able to validate those upstream signatures when *they* are inspecting the source, not just to validate an assertion from me as the debian maintainer. It's hard enough to be able to validate direct signatures, and from years of working in this space, i'm pretty confident in saying that even the most sophisticated users don't really understand what a "trust path" is, or how to map it back to a meaningful understanding of the provenance of a system. That's why i'm thinking here explicitly about exporting a verifiable upstream signature. fwiw, the debian developer themselves already distributes a signature with each new upstream release, which covers the upstream tarball. For example: https://tracker.debian.org/news/1028687/accepted-knot-276-1-source-into-unstable/ contains a cryptographic signature from me, over a simple text document that describes the cryptographic digest of the orig.tar.gz. So we have an assertion from the debian developer of what they think the upstream tarball is. So I think i don't understand what additional benefits we'd get from the mechanism you are describing. Sorry for the confusion! > To pick up on Chris's comment, the validation happens when our tools can be > reasonably expected to have network access (or the maintainer has a git > repo nearby). I think Chris's comment points to the tooling wanting to guarantee that things work *without* network access -- just with access to the debian archive, which i think is a laudable goal. The network can change or break over time, and we really do want a more reliable archive. > As an aside, maintainer involvement is always required when repacking for > DFSG-compliance. I agree that repacked upstream tarballs *cannot* ship with the most commonly-distributed forms of verifiable upstream signatures. I think that's a different problem, though, and probably needs to be solved in a different way. --dkg signature.asc Description: PGP signature
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
Hi Daniel, On Fri, Mar 1, 2019 at 4:12 AM Daniel Kahn Gillmor wrote: > > > I think what you're describing takes aim a different problem, so i don't > think it addresses the underlying concern here. I believe it solves your problem, just not the way you expect. As you point out, the current approach of validating different upstream signature formats is the point of this bug. You list two, but as you note there could be more: > At issue in #920763 is our attempt to capture verified *upstream* > cryptographic signatures. There are (at least) two common practices for > such signatures by upstreams across the free software ecosystem: > > a) detached signatures over tarballs > b) signed git tags With source format 3.0 (git) that logic even found a way into the packaging system. Let's flip it around for a moment: Why not validate upstream signatures when the package is built? My shipping manifest offers an "origin statement" to include git or tarball signatures (even multiple) that tie the original source (in any format) to the list of file hashes. Signed by a Debian maintainer, the manifest provides a trust path. To pick up on Chris's comment, the validation happens when our tools can be reasonably expected to have network access (or the maintainer has a git repo nearby). > aiui, your tool is > designed to something operated by the debian developer/maintainer. > That cannot be an obstacle. You just suggested it earlier in this thread: I guess if we wanted some version of lintian to be able to check on the > git tag, we need to have some sort of export (git shallow > something-or-other?) that could be included in debian/ to recreate a git > repo that would be sufficient to verify the contents of the files and > confirm the git signature. > Yours were the very words that made me believe I could help. :) A signed manifest is less complex, more universal, and would likely take up less space. As an aside, maintainer involvement is always required when repacking for DFSG-compliance. A manifest with an origin statement would handle that case with ease. > Today, we have pretty decent tooling to handle (a). we even distribute> upstream tarball signatures directly when we have them: > (e.g. https://mirrors.edge.kernel.org/debian/pool/main/k/knot/knot_2.6.8.orig.tar.xz > can be verified against the upstream signer's key by fetching > https://mirrors.edge.kernel.org/debian/pool/main/k/knot/knot_2.6.8.orig.tar.xz.asc ) > > So for (a), we're effectively assembling an archive of all of the > upstream signatures that we know about, which could be used later for > verifying provenance of the source code used. > > What we don't have is tooling to handle or aggregate such a verifiable > archive for those upstream signatures that fall under (b). Do you think > the tool you're describing would help with that? I believe my tool covers a) and b) as well as any other past, current, or future authentication method for upstream sources. As a side note, the signed manifest could even be the sole item uploaded for a new version if it also includes a list of URLs where the sources may be found. Right now, I call it the Light Upload Format. Kind regards, Felix
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
Hi Felix-- On Wed 2019-02-27 13:07:20 -0800, Felix Lechner wrote: > I wrote a Debian tool to create a shipping manifest with file-based > hashes. Would it help to include that at the time of packaging? If the > manifest is signed, we could do away with tarball signatures. I think what you're describing takes aim a different problem, so i don't think it addresses the underlying concern here. aiui, your tool is designed to something operated by the debian developer/maintainer. if i'm misunderstanding, i'd be happy to learn more. At issue in #920763 is our attempt to capture verified *upstream* cryptographic signatures. There are (at least) two common practices for such signatures by upstreams across the free software ecosystem: a) detached signatures over tarballs b) signed git tags Today, we have pretty decent tooling to handle (a). we even distribute upstream tarball signatures directly when we have them: (e.g. https://mirrors.edge.kernel.org/debian/pool/main/k/knot/knot_2.6.8.orig.tar.xz can be verified against the upstream signer's key by fetching https://mirrors.edge.kernel.org/debian/pool/main/k/knot/knot_2.6.8.orig.tar.xz.asc) So for (a), we're effectively assembling an archive of all of the upstream signatures that we know about, which could be used later for verifying provenance of the source code used. What we don't have is tooling to handle or aggregate such a verifiable archive for those upstream signatures that fall under (b). Do you think the tool you're describing would help with that? --dkg signature.asc Description: PGP signature
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
Hi Daniel, On Wed, Feb 27, 2019 at 12:03 PM Daniel Kahn Gillmor wrote: > > I guess if we wanted some version of lintian to be able to check on the > git tag, we need to have some sort of export (git shallow > something-or-other?) that could be included in debian/ to recreate a git > repo that would be sufficient to verify the contents of the files and > confirm the git signature. I wrote a Debian tool to create a shipping manifest with file-based hashes. Would it help to include that at the time of packaging? If the manifest is signed, we could do away with tarball signatures. Felix
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
On Tue 2019-02-26 16:36:05 -0500, Chris Lamb wrote: > I'm afraid it would, and would not be visible on lintian.d.o, and > would also give different results in different environments. Whilst > there is no strict written policy about this anywhere, this just > feels "kinda" wrong, alas. gotcha, thanks for the explanation and scope of what is in-bounds for lintian. I guess if we wanted some version of lintian to be able to check on the git tag, we need to have some sort of export (git shallow something-or-other?) that could be included in debian/ to recreate a git repo that would be sufficient to verify the contents of the files and confirm the git signature. I don't know how to do that yet though :/ --dkg
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
Hi dkg, > > Lintian "cannot" talk to external sources, so this is out alas… > > How about talking to the local git repository, if it's operating in one? > does that count as an "external" source? I'm afraid it would, and would not be visible on lintian.d.o, and would also give different results in different environments. Whilst there is no strict written policy about this anywhere, this just feels "kinda" wrong, alas. Best wishes, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org chris-lamb.co.uk `-
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
On Tue 2019-02-26 14:24:11 +, Chris Lamb wrote: > [ dkg wrote: ] >> Ideally, lintian would verify that there exists a signed tag in the git >> repo found at Vcs-Git: (from d/control) […] > > Lintian "cannot" talk to external sources, so this is out alas… How about talking to the local git repository, if it's operating in one? does that count as an "external" source? if not, perhaps this should be a separate lintian warning: If mode=git,pgpmode=gittag, and the local working copy is itself a git repository: * check if the tag appears * check that the tag matches the orig tarball * check that the tag is cryptographically signed by debian/upstream/signing-key.asc If any of these three checks fail, maybe that's worth a warning? --dkg
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
Hi dkg, > Ideally, lintian would verify that there exists a signed tag in the git > repo found at Vcs-Git: (from d/control) […] Lintian "cannot" talk to external sources, so this is out alas… > If going that far is too fancy for lintian for now, then a simple grep > of d/watch would do for starters … indeed. Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org chris-lamb.co.uk `-
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
Control: tags 920763 - moreinfo Hi Chris-- On Tue 2019-01-29 09:29:50 +0100, Chris Lamb wrote: > Probably a silly question for this time in the morning but what is > stopping you extracting the associated signature and calling it > $origname.asc? the signature matches the git commit, but not the tarball. If we have a $origname.asc i think it's expected to be verifiable via: gpgv $origname.asc $origname but that would pretty clearly fail. > (If this is not possible/sensible/whatever, if Lintian essentially > grepped debian/watch, would that be good enough?) Ideally, lintian would verify that there exists a signed tag in the git repo found at Vcs-Git: (from d/control), which matches the name of upstream-tag (from d/gbp.conf), and whose contents corresponds to the expected contents of the orig.tar.gz (presumably with a standardized prefix). One approach would be to: * identify the tag by its expected name * cryptogrpahically verify it * extract the expected archive from the git repo via sth. like git archive --format=tar --prefix=$pkgname-$origversion/ piped through the expected buildpackage.compression value (from d/gbp.conf) * compare it bytewise with $origname I suspect that will work in most cases, though i don't know whether git has explicitly committed to a stable output for git archive --format=tar. If going that far is too fancy for lintian for now, then a simple grep of d/watch would do for starters, and we could just convert this bug report to a suggestion for future lintian enhancement. Regards, --dkg
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
tags 920763 + moreinfo thanks Hi dkg, > So we don't use an "upstream tarball" other than the git repo, which > has a signed tag in it. But lintian complains: > >W: orig-tarball-missing-upstream-signature Probably a silly question for this time in the morning but what is stopping you extracting the associated signature and calling it $origname.asc? (If this is not possible/sensible/whatever, if Lintian essentially grepped debian/watch, would that be good enough?) Best wishes, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org chris-lamb.co.uk `-
Bug#920763: lintian: orig-tarball-missing-upstream-signature interacts poorly with mode=git,pgpmode=gittag
Package: lintian Version: 2.5.124 Severity: normal Control: affects -1 src:wireguard In wireguard's debian/watch, we have: opts=mode=git,pgpmode=gittag So we don't use an "upstream tarball" other than the git repo, which has a signed tag in it. But lintian complains: W: orig-tarball-missing-upstream-signature It would be nice if lintian could identify that this is using signed git tags, and avoid the warning -- or, if it is using signed git tags, at least produce a suggestion of what to do with them. --dkg -- System Information: Debian Release: buster/sid APT prefers testing-debug APT policy: (500, 'testing-debug'), (500, 'testing'), (200, 'unstable-debug'), (200, 'unstable'), (1, 'experimental-debug'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.19.0-1-amd64 (SMP w/4 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages lintian depends on: ii binutils 2.31.1-11 ii bzip2 1.0.6-9 ii diffstat 1.62-1 ii dpkg 1.19.2 ii dpkg-dev 1.19.2 ii file 1:5.35-2 ii gettext0.19.8.1-9 ii gpg2.2.12-1 ii intltool-debian0.35.0+20060710.5 ii libapt-pkg-perl0.1.34+b1 ii libarchive-zip-perl1.64-1 ii libcgi-pm-perl 4.40-1 ii libclass-accessor-perl 0.51-1 ii libclone-perl 0.41-1+b1 pn libdigest-sha-perl ii libdpkg-perl 1.19.2 ii libemail-valid-perl1.202-1 ii libfile-basedir-perl 0.08-1 ii libio-async-perl 0.72-1 ii libipc-run-perl20180523.0-1 ii liblist-moreutils-perl 0.416-1+b4 ii libparse-debianchangelog-perl 1.2.0-13 ii libtext-levenshtein-perl 0.13-1 ii libtimedate-perl 2.3000-2 ii liburi-perl1.76-1 ii libxml-simple-perl 2.25-1 ii libyaml-libyaml-perl 0.76+repack-1 ii man-db 2.8.5-1 ii patchutils 0.3.4-2 ii perl 5.28.1-3 ii t1utils1.41-3 ii xz-utils 5.2.2-1.3 Versions of packages lintian recommends: pn libperlio-gzip-perl Versions of packages lintian suggests: pn binutils-multiarch ii libhtml-parser-perl3.72-3+b3 ii libtext-template-perl 1.54-1 -- no debconf information
