Bug#888046: devscripts: Support signatures against uncompressed tarballs
On 2018-01-23, Osamu Aoki wrote: > I am in a good mood to do my user support duty :-) So let me show. Thanks! > On Mon, Jan 22, 2018 at 01:24:20PM -0800, Vagrant Cascadian wrote: >> There are a number of projects hosted at kernel.org that use the >> kup-client utility to handle uploads. While it may upload a signature to >> verify the uploaded tarballs, those signatures are against the >> uncompressed tarball, rather than the compressed tarballs. >> >> For example, for dtc version 1.4.6, there is: >> >> https://www.kernel.org/pub/software/utils/dtc/ >> >> dtc-1.4.6.tar.gz >> dtc-1.4.6.tar.sign >> dtc-1.4.6.tar.xz >> >> I can download either .tar.gz or .tar.xz, decompress them, and then use >> the .tar.sign to verify it, but I don't see any obvious way to do this >> From debian/watch. > The obvious way is to read the manpage of uscan. ... many ways but > something along I've read the uscan manpage quite a number of times, but even after using uscan for well over a decade and reading the manpage many times over the years, nothing really comes across as obvious. So there's a difference between reading the fine manual and comprehending it. Fortunately, It's one of those things I get working once for a package and infrequently need to update it, so that's good. And yet... > version=4 > opts="pgpmode=mangle, pgpsigurlmangle=s%tar\..z$%tar\.sign%" \ > https://www.kernel.org/pub/software/utils/dtc/ \ > @PACKAGE@@ANY_VERSION@@ARCHIVE_EXT@ \ > debian uupdate Thanks for the suggestion... with debian/watch: version=4 opts="pgpmode=mangle, pgpsigurlmangle=s%tar\..z$%tar\.sign%" \ https://www.kernel.org/pub/software/utils/dtc/ \ dtc-@ANY_VERSION@@ARCHIVE_EXT@ \ debian uupdate Using @PACKAGE@ didn't work because of upstream is named differently (device-tree-compiler vs. dtc). But even with that fixed/worked around: uscan: Newest version of device-tree-compiler on remote site is 1.4.6, local version is 1.4.5 uscan:=> Newer package available from https://www.kernel.org/pub/software/utils/dtc/dtc-1.4.6.tar.xz gpgv: Signature made Tue Jan 2 22:12:20 2018 PST gpgv:using RSA key 75F46586AE61A66CC44E87DC6C38CACA20D9B392 gpgv: BAD signature from "David Gibson" uscan die: OpenPGP signature did not verify. If I manually take the files that uscan downloaded and verify them like so: $ xz -d dtc-1.4.6.tar.xz $ gpg --verify $ gpg --verify dtc-1.4.6.tar.xz.n dtc-1.4.6.tar gpg: Signature made Tue Jan 2 22:12:20 2018 PST gpg:using RSA key 75F46586AE61A66CC44E87DC6C38CACA20D9B392 gpg: Good signature from "David Gibson " [unknown] gpg: aka "David Gibson (kernel.org) " [unknown] gpg: aka "David Gibson (Red Hat) " [unknown] gpg: aka "David Gibson (ozlabs.org) " [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 75F4 6586 AE61 A66C C44E 87DC 6C38 CACA 20D9 B392 I am fairly certain this is because the signature is not against the .tar.xz, but against the uncompressed tarball. Does uscan (attempt to) decompress the tarball before verifying the signature? If not, I don't see how this could possibly work; in fact, if it did, it would be a serious security bug, as the signature is against the uncompressed tarball. >> I'm also not sure the Debian archive supports uploading a signature file >> against a file that isn't included in the distribution, so maybe this >> isn't really an issue worth handling in uscan... > > That is not a uscan bug. I as the primary uscan committer want to hear > your experience. Did you try? If you find out the answer, please let > me know what shall be done. I haven't tried because I haven't yet figured out a way to automate the verification of the signature (short of writing something entirely outside of uscan). live well, vagrant signature.asc Description: PGP signature ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
[devscripts] 01/01: python3 migration: pylint -> pylint3
This is an automated email from the git hooks/post-receive script. osamu pushed a commit to branch master in repository devscripts. commit b17add3d88bbcd46796b6d842a9c661271607786 Author: Osamu AokiDate: Tue Jan 23 23:50:29 2018 +0900 python3 migration: pylint -> pylint3 Signed-off-by: Osamu Aoki --- debian/control | 2 +- scripts/devscripts/test/test_pylint.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/debian/control b/debian/control index 843479d..d62134b 100644 --- a/debian/control +++ b/debian/control @@ -27,7 +27,7 @@ Build-Depends: bash-completion, perl:any, pkg-config, po4a, - pylint , + pylint3 , python3-all:any, python3-apt , python3-debian , diff --git a/scripts/devscripts/test/test_pylint.py b/scripts/devscripts/test/test_pylint.py index 3671725..33b0b9c 100644 --- a/scripts/devscripts/test/test_pylint.py +++ b/scripts/devscripts/test/test_pylint.py @@ -40,7 +40,7 @@ class PylintTestCase(unittest.TestCase): if 'python' in f.readline(): files.append(script) f.close() -cmd = ['pylint', '--rcfile=devscripts/test/pylint.conf', '-E', +cmd = ['pylint3', '--rcfile=devscripts/test/pylint.conf', '-E', '--include-ids=y', '--'] + files process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) -- Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/collab-maint/devscripts.git ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
[devscripts] branch master updated (240cae9 -> b17add3)
This is an automated email from the git hooks/post-receive script. osamu pushed a change to branch master in repository devscripts. from 240cae9 uscan: test scripts migrate to python3 new b17add3 python3 migration: pylint -> pylint3 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: debian/control | 2 +- scripts/devscripts/test/test_pylint.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/collab-maint/devscripts.git ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
Re: Good but ... mk-origtargz Allow more files to be deleted than can fit inside argv (`getconf ARG_MAX`)
Osamu Aoki: > Hi, > > [..] > > This looks a reasonable patch. But I don't know why you chose > 16384=0x400 as the max figure. > > Following your comment, I tried on my local machine > $ getconf ARG_MAX > 2097152 > > This is bigger than 16384. > > If this is different on different system, why not dynamically check and > set it with some safety margin like `getconf ARG_MAX` - 16 etc.? > Hi Osamu, ARG_MAX is measured in bytes, not number of arguments. I calculated 16384 as 2M / 128, 128 being a generously-high estimated value for "average path length" in debian packages. I agree it would be clearer to replace 16384 with `getconf ARG_MAX / AV_PATH_LENGTH_EST` and define AV_PATH_LENGTH_EST = 128 near the top, but my perl knowledge is very basic and I wasn't sure of the best way to shell out to another program. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
Re: Good but ... mk-origtargz Allow more files to be deleted than can fit inside argv (`getconf ARG_MAX`)
On Tue, Jan 23, 2018 at 10:58:53PM +0900, Osamu Aoki wrote: > This looks a reasonable patch. But I don't know why you chose > 16384=0x400 as the max figure. > > Following your comment, I tried on my local machine > $ getconf ARG_MAX > 2097152 > > This is bigger than 16384. > > If this is different on different system, why not dynamically check and > set it with some safety margin like `getconf ARG_MAX` - 16 etc.? Related lines from IRC: Jan 13 21:39:31infinity0: why a magic number in the mk-origtargz patch? Jan 13 21:43:10(instead of running a getconf and getting the value and use it, I suppose) Jan 13 21:45:38(and then doing some computation to turn number in something usable) Jan 13 21:50:07 jamessan[m]: it's `getconf ARG_MAX` / 128 which i felt was a reasonable path length for source filenames Jan 13 21:50:33 and worked in practise for me (larger values like 3 broke) Jan 13 21:56:52infinity0: yes, but why not detecting it at runtime (and a comment explaining why 128 is better than NAME_MAX=255) Jan 13 21:57:09 because it would involve more perl code So I agree. Please don't merge that as it is until somebody does what's written here ↑. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
Bug#888046: marked as done (devscripts: Support signatures against uncompressed tarballs)
On Tue, Jan 23, 2018, Osamu Aoki wrote: > > I'm also not sure the Debian archive supports uploading a signature file > > against a file that isn't included in the distribution, so maybe this > > isn't really an issue worth handling in uscan... > > That is not a uscan bug. I as the primary uscan committer want to hear > your experience. Did you try? If you find out the answer, please let > me know what shall be done. I have the answer for you: the Debian archive doesn't even check that the uploaded .asc is an actual signature. IIRC it only does a check on the filename (to assure that you are uploading something that is related to an already known file), but nothing else. That also means that it doesn't actually performe a signature check either. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
[devscripts] 01/01: uscan: test scripts migrate to python3
This is an automated email from the git hooks/post-receive script. osamu pushed a commit to branch master in repository devscripts. commit 240cae9a19c5532ab418bc63fcaca5231de1a470 Author: Osamu AokiDate: Tue Jan 23 23:37:48 2018 +0900 uscan: test scripts migrate to python3 Signed-off-by: Osamu Aoki --- test/test_uscan | 2 +- test/test_uscan_mangle| 2 +- test/uscan/server-head.py | 14 +++--- test/uscan/server.py | 8 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/test/test_uscan b/test/test_uscan index 7fafd12..944c014 100755 --- a/test/test_uscan +++ b/test/test_uscan @@ -55,7 +55,7 @@ spawnHttpServer(){ ( mkdir -p $TMPDIR/repo cd $TMPDIR/repo - python "$test_dir/uscan/server.py" & + python3 "$test_dir/uscan/server.py" & echo $! > pid while ! [ -s port ]; do : diff --git a/test/test_uscan_mangle b/test/test_uscan_mangle index 05f1dfc..9afdabb 100755 --- a/test/test_uscan_mangle +++ b/test/test_uscan_mangle @@ -97,7 +97,7 @@ spawnHttpServer(){ USCAN_HTTP_SERVER=${USCAN_HTTP_SERVER:-server.py} mkdir -p $TMPDIR/$REPOPATH cd $TMPDIR/$REPOPATH - python "$test_dir/uscan/$USCAN_HTTP_SERVER" 2>log & + python3 "$test_dir/uscan/$USCAN_HTTP_SERVER" 2>log & echo $! > pid while ! [ -s port ]; do : diff --git a/test/uscan/server-head.py b/test/uscan/server-head.py index 3d1c796..13daf4d 100644 --- a/test/uscan/server-head.py +++ b/test/uscan/server-head.py @@ -1,16 +1,16 @@ -#!/usr/bin/python -import BaseHTTPServer -import SimpleHTTPServer +#!/usr/bin/python3 +import http.server +from http.server import SimpleHTTPRequestHandler import logging -class GetHandler(SimpleHTTPServer.SimpleHTTPRequestHandler): +class GetHandler(SimpleHTTPRequestHandler): def do_GET(self): - logging.error(self.headers) -SimpleHTTPServer.SimpleHTTPRequestHandler.do_GET(self) +logging.error(self.headers) +SimpleHTTPRequestHandler.do_GET(self) def test(): Handler = GetHandler -httpd = BaseHTTPServer.HTTPServer(('', 0), Handler) +httpd = http.server.HTTPServer(('', 0), Handler) sa = httpd.socket.getsockname() with open('port', 'w') as f: diff --git a/test/uscan/server.py b/test/uscan/server.py index 7309288..b9cf832 100644 --- a/test/uscan/server.py +++ b/test/uscan/server.py @@ -1,10 +1,10 @@ -#!/usr/bin/python -import BaseHTTPServer -from SimpleHTTPServer import SimpleHTTPRequestHandler +#!/usr/bin/python3 +import http.server +from http.server import SimpleHTTPRequestHandler def test(): SimpleHTTPRequestHandler.protocol_version='HTTP/1.0' -httpd = BaseHTTPServer.HTTPServer(('', 0), SimpleHTTPRequestHandler) +httpd = http.server.HTTPServer(('', 0), SimpleHTTPRequestHandler) sa = httpd.socket.getsockname() with open('port', 'w') as f: -- Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/collab-maint/devscripts.git ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
[devscripts] branch master updated (14834f2 -> 240cae9)
This is an automated email from the git hooks/post-receive script. osamu pushed a change to branch master in repository devscripts. from 14834f2 uscan: changelog update new 240cae9 uscan: test scripts migrate to python3 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: test/test_uscan | 2 +- test/test_uscan_mangle| 2 +- test/uscan/server-head.py | 14 +++--- test/uscan/server.py | 8 4 files changed, 13 insertions(+), 13 deletions(-) -- Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/collab-maint/devscripts.git ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
Good but ... mk-origtargz Allow more files to be deleted than can fit inside argv (`getconf ARG_MAX`)
Hi, On Sat, Jan 13, 2018 at 04:23:34PM +, Ximin Luo wrote: > This is an automated email from the git hooks/post-receive script. > > infinity0 pushed a commit to branch pu/mk-origtargz-argmax > in repository devscripts. > > commit 3fa9a36bb4f352c07a3537cb597aaf734ce936f1 > Author: Ximin Luo> Date: Sat Jan 13 17:22:02 2018 +0100 > > Allow more files to be deleted than can fit inside argv (`getconf > ARG_MAX`) > --- > scripts/mk-origtargz.pl | 12 +++- > 1 file changed, 7 insertions(+), 5 deletions(-) > > diff --git a/scripts/mk-origtargz.pl b/scripts/mk-origtargz.pl > index 4299b93..4533899 100644 > --- a/scripts/mk-origtargz.pl > +++ b/scripts/mk-origtargz.pl > @@ -571,11 +571,13 @@ if ($do_repack || $deletecount) { > # We have to use piping because --delete is broken otherwise, as > documented > # at https://www.gnu.org/software/tar/manual/html_node/delete.html > if (@to_delete) { > - spawn(exec => ['tar', '--delete', @to_delete ], > - from_file => $destfiletar, > - to_file => $destfiletar . ".tmp", > - wait_child => 1) if scalar(@to_delete) > 0; > - move ($destfiletar . ".tmp", $destfiletar); > + while (my @next_n = splice @to_delete, 0, 16384) { > + spawn(exec => ['tar', '--delete', @next_n ], > + from_file => $destfiletar, > + to_file => $destfiletar . ".tmp", > + wait_child => 1) if scalar(@next_n) > 0; > + move ($destfiletar . ".tmp", $destfiletar); > + } > } > compress_archive($destfiletar, $destfile, $compression); This looks a reasonable patch. But I don't know why you chose 16384=0x400 as the max figure. Following your comment, I tried on my local machine $ getconf ARG_MAX 2097152 This is bigger than 16384. If this is different on different system, why not dynamically check and set it with some safety margin like `getconf ARG_MAX` - 16 etc.? Osamu ___ devscripts-devel mailing list devscripts-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/devscripts-devel
Bug#888046: marked as done (devscripts: Support signatures against uncompressed tarballs)
Your message dated Tue, 23 Jan 2018 22:14:19 +0900 with message-id <20180123131418.ga5...@goofy.tc4.so-net.ne.jp> and subject line Re: Bug#888046: devscripts: Support signatures against uncompressed tarballs has caused the Debian Bug report #888046, regarding devscripts: Support signatures against uncompressed tarballs to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 888046: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=888046 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems --- Begin Message --- Package: devscripts Version: 2.17.12~bpo9+1 Severity: wishlist File: /usr/bin/uscan There are a number of projects hosted at kernel.org that use the kup-client utility to handle uploads. While it may upload a signature to verify the uploaded tarballs, those signatures are against the uncompressed tarball, rather than the compressed tarballs. For example, for dtc version 1.4.6, there is: https://www.kernel.org/pub/software/utils/dtc/ dtc-1.4.6.tar.gz dtc-1.4.6.tar.sign dtc-1.4.6.tar.xz I can download either .tar.gz or .tar.xz, decompress them, and then use the .tar.sign to verify it, but I don't see any obvious way to do this From debian/watch. I'm also not sure the Debian archive supports uploading a signature file against a file that isn't included in the distribution, so maybe this isn't really an issue worth handling in uscan... live well, vagrant -- Package-specific info: --- /etc/devscripts.conf --- --- ~/.devscripts --- Not present -- System Information: Debian Release: 9.3 APT prefers stable APT policy: (500, 'stable'), (210, 'proposed-updates'), (120, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: armhf, arm64 Kernel: Linux 4.9.0-5-amd64 (SMP w/4 CPU cores) Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE=C.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages devscripts depends on: ii dpkg-dev 1.18.24 ii libc6 2.24-11+deb9u1 ii libfile-homedir-perl 1.00-1 ii perl 5.24.1-3+deb9u2 ii python3 3.5.3-1 ii sensible-utils0.0.9+deb9u1 Versions of packages devscripts recommends: ii apt 1.4.8 ii at 3.1.20-3 ii curl7.52.1-5+deb9u3 ii dctrl-tools 2.24-2+b1 ii debian-keyring 2017.11.24 ii dput-ng [dput] 1.13 ii equivs 2.0.9+nmu1 ii fakeroot1.21-3.1 ii file1:5.30-1+deb9u1 ii gnupg 2.1.18-8~deb9u1 ii gnupg2 2.1.18-8~deb9u1 ii libdistro-info-perl 0.14 ii libdpkg-perl1.18.24 ii libencode-locale-perl 1.05-1 ii libgit-wrapper-perl 0.047-1 ii liblist-compare-perl0.53-1 ii liblwp-protocol-https-perl 6.06-2 ii libsoap-lite-perl 1.20-1 ii liburi-perl 1.71-1 ii libwww-perl 6.15-1 ii licensecheck3.0.29-1 ii lintian 2.5.67~bpo9+1 ii man-db 2.7.6.1-2 ii patch 2.7.5-1+b2 ii patchutils 0.3.4-2 ii python3-apt 1.4.0~beta3 ii python3-debian 0.1.30 ii python3-magic 1:5.30-1+deb9u1 ii python3-requests2.12.4-1 pn python3-unidiff ii python3-xdg 0.25-4 ii strace 4.15-2 ii unzip 6.0-21 ii wdiff 1.2.2-2 ii wget1.18-5+deb9u1 ii xz-utils5.2.2-1.2+b1 Versions of packages devscripts suggests: pn adequate ii autopkgtest 4.4 pn bls-standalone ii bsd-mailx [mailx]8.1.2-0.20160123cvs-4 ii build-essential 12.3 pn check-all-the-things pn cvs-buildpackage pn devscripts-el pn diffoscope pn disorderfs pn dose-extra pn duck pn faketime pn gnuplot ii gpgv 2.1.18-8~deb9u1 pn how-can-i-help ii libauthen-sasl-perl 2.1600-1 ii libfile-desktopentry-perl0.22-1 pn libnet-smtps-perl pn libterm-size-perl ii libtimedate-perl 2.3000-2 pn libyaml-syck-perl pn