Re: Bug#855282: debsign: support .buildinfo files
Hi! On Thu, 2017-02-16 at 17:23:00 +, Ximin Luo wrote: > Control: tags + patch > I've done an initial implementation here: > > https://anonscm.debian.org/cgit/collab-maint/devscripts.git/log/?h=pu/debsign-buildinfo > > Please review! I think something like the attached patch on top of your branch HEAD is also needed. Thanks, Guillem From 8ccb601c3e3f1cb16db923b8464e8a73b4cf03df Mon Sep 17 00:00:00 2001 From: Guillem Jover Date: Fri, 17 Feb 2017 06:04:27 +0100 Subject: [PATCH] Improve .buildinfo support --- scripts/debsign.sh | 19 +-- scripts/dscverify.pl | 7 --- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/scripts/debsign.sh b/scripts/debsign.sh index ede3aa79..4b2b22cb 100755 --- a/scripts/debsign.sh +++ b/scripts/debsign.sh @@ -418,7 +418,7 @@ fixup_control() { '" \$file=\"$child\"; \$md5=\"$md5\"; "' '" \$sha1=\"$sha1\"; \$sha256=\"$sha256\"; "' $size=(-s $file); ($base=$file) =~ s|.*/||; -$infiles=0; $insha1=0; $insha256=0; $format=""; +$infiles=0; $inmd5=0; $insha1=0; $insha256=0; $format=""; } if(/^Format:\s+(.*)/) { $format=$1; @@ -429,19 +429,26 @@ fixup_control() { die "Unsupported .$parenttype format: $format\n" if('"$filter_out"'); } -/^Files:/i && ($infiles=1,$insha1=0,$insha256=0); -if(/^Checksums-Sha1:/i) {$insha1=1;$infiles=0;$insha256=0;} +/^Files:/i && ($infiles=1,$inmd5=0,$insha1=0,$insha256=0); +if(/^Checksums-Sha1:/i) {$insha1=1;$infiles=0;$inmd5=0;$insha256=0;} elsif(/^Checksums-Sha256:/i) { - $insha256=1;$infiles=0;$insha1=0; + $insha256=1;$infiles=0;$inmd5=0;$insha1=0; +} elsif(/^Checksums-Md5:/i) { + $inmd5=1;$infiles=0;$insha1=0;$insha256=0; } elsif(/^Checksums-.*?:/i) { die "Unknown checksum format: $_\n"; } -/^\s*$/ && ($infiles=0,$insha1=0,$insha256=0); +/^\s*$/ && ($infiles=0,$inmd5=0,$insha1=0,$insha256=0); if ($infiles && /^ (\S+) (\d+) (\S+) (\S+) \Q$base\E\s*$/) { $_ = " $md5 $size $3 $4 $base\n"; $infiles=0; } +if ($inmd5 && + /^ (\S+) (\d+) \Q$base\E\s*$/) { +$_ = " $md5 $size $base\n"; +$inmd5=0; +} if ($insha1 && /^ (\S+) (\d+) \Q$base\E\s*$/) { $_ = " $sha1 $size $base\n"; @@ -455,7 +462,7 @@ fixup_control() { } fixup_buildinfo() { -fixup_control '$major < 1 and $minor < 2' dsc buildinfo "$@" +fixup_control '$major != 0 or $minor > 2' dsc buildinfo "$@" } fixup_changes() { diff --git a/scripts/dscverify.pl b/scripts/dscverify.pl index 8bc0857c..381ebff5 100755 --- a/scripts/dscverify.pl +++ b/scripts/dscverify.pl @@ -198,7 +198,7 @@ sub process_file { } } -if ($file =~ /\.changes$/ and $out =~ /^Format:\s*(.*)$/mi) { +if ($file =~ /\.(changes|buildinfo)$/ and $out =~ /^Format:\s*(.*)$/mi) { my $format = $1; unless ($format =~ /^(\d+)\.(\d+)$/) { xwarn "$file has an unrecognised format: $format\n"; @@ -207,7 +207,8 @@ sub process_file { my ($major, $minor) = split /\./, $format; $major += 0; $minor += 0; - unless ($major == 1 and $minor <= 8) { + if ($file =~ /\.changes$/ and ($major != 1 or $minor > 8) or + $file =~ /\.buildinfo$/ and ($major != 0 or $minor > 2)) { xwarn "$file is an unsupported format: $format\n"; return; } @@ -220,7 +221,7 @@ sub process_file { } my @checksums = map { split /\n/ } $out =~ /^Checksums-(\S+):\s*\n/mgi; -@checksums = grep {!/^Sha(1|256)$/i} @checksums; +@checksums = grep {!/^(Md5|Sha(1|256))$/i} @checksums; if (@checksums) { xwarn "$file contains unsupported checksums:\n" . join (", ", @checksums) . "\n"; -- 2.11.0.483.g087da7b7c ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
diffoscope is marked for autoremoval from testing
diffoscope 67 is marked for autoremoval from testing on 2017-03-19 It is affected by these RC bugs: 854723: diffoscope: CVE-2017-0359: writes to arbitrary locations on disk based on the contents of an untrusted archive ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
reprotest is marked for autoremoval from testing
reprotest 0.6 is marked for autoremoval from testing on 2017-03-19 It (build-)depends on packages with these RC bugs: 854723: diffoscope: CVE-2017-0359: writes to arbitrary locations on disk based on the contents of an untrusted archive ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: New armhf node (Pine64+)
On Thu, Feb 16, 2017 at 12:25:15PM -0800, Vagrant Cascadian wrote: > Just purged the un-unsed kernel packages which didn't have support for > these boards. ah! > I guess you must have installed something that triggered > an "update-initramfs" call on the older kernel versions... I wonder what this was… > Removed your workaround, and re-ran update-initramfs. Should be working > now. thanks. (I had to add two workarounds…) > >> This one is interesting in that it's running an arm64 kernel with armhf > >> userland (like the i386 builders that run amd64 kernels). > > nice! is this the same for p64c too? > Yup. very nice! > Thanks for getting themn into production! my pleasure! :) -- cheers, Holger signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: New armhf node (Pine64+)
On 2017-02-16, Holger Levsen wrote: > On Mon, Feb 06, 2017 at 01:39:47PM -0800, Vagrant Cascadian wrote: > linux-image-4.10.0-rc6-arm64-unsigned (4.10~rc6-1~exp1) wird eingerichtet ... > /etc/kernel/postinst.d/initramfs-tools: > update-initramfs: Generating /boot/initrd.img-4.10.0-rc6-arm64 > DTB: sun50i-a64-pine64-plus.dtb > Couldn't find ... > Can you fix this up ("somehow" on the host…), please?! Just purged the un-unsed kernel packages which didn't have support for these boards. I guess you must have installed something that triggered an "update-initramfs" call on the older kernel versions... Removed your workaround, and re-ran update-initramfs. Should be working now. >> This one is interesting in that it's running an arm64 kernel with armhf >> userland (like the i386 builders that run amd64 kernels). > > nice! is this the same for p64c too? Yup. Thanks for getting themn into production! live well, vagrant signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#855282: debsign: support .buildinfo files
Control: tags + patch Hi all, I've done an initial implementation here: https://anonscm.debian.org/cgit/collab-maint/devscripts.git/log/?h=pu/debsign-buildinfo Please review! I haven't yet updated debrsign but I think that program is a bit pointless anyway, and have documented this in debsign(1): "note that it is probably safer to have your trusted signing machine use \fBdebsign\fR to connect to the untrusted non-signing machine, rather than using \fBdebrsign\fR to make the connection in the reverse direction." X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: New armhf node (Pine64+)
On Thu, Feb 16, 2017 at 01:31:55PM +, Holger Levsen wrote: > E: Sub-process /usr/bin/dpkg returned an error code (1) > Can you fix this up ("somehow" on the host…), please?! "fixed" this for now by adding "exit 0" at the beginning of /etc/initramfs/post-update.d//flash-kernel … -- cheers, Holger signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Another new armhf node (Pine64+)
On Tue, Feb 14, 2017 at 03:52:06PM -0800, Vagrant Cascadian wrote: > Yet Another arm board ready to be configured for the build farm! set up as well. (I've only configured maintenance and setup jobs so far, but no builder jobs as p64b aint setup fully yet due to the linux-image install problem…) -- cheers, Holger signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: New armhf node (Pine64+)
Hi Vagrant, sorry for the delay in getting these boards used…! On Mon, Feb 06, 2017 at 01:39:47PM -0800, Vagrant Cascadian wrote: > Another arm board ready to be configured for the build farm! this is now basically setup, however this is quite annoying (as we expect "apt install" to exit 0…) on p64b-armhf-rb.debian.net: linux-image-4.10.0-rc6-arm64-unsigned (4.10~rc6-1~exp1) wird eingerichtet ... /etc/kernel/postinst.d/initramfs-tools: update-initramfs: Generating /boot/initrd.img-4.10.0-rc6-arm64 DTB: sun50i-a64-pine64-plus.dtb Couldn't find run-parts: /etc/initramfs/post-update.d//flash-kernel exited with return code 1 run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1 dpkg: Fehler beim Bearbeiten des Paketes linux-image-4.10.0-rc6-arm64-unsigned (--configure): Unterprozess installiertes post-installation-Skript gab den Fehlerwert 1 zurück Fehler traten auf beim Bearbeiten von: linux-image-4.10.0-rc6-arm64-unsigned E: Sub-process /usr/bin/dpkg returned an error code (1) Can you fix this up ("somehow" on the host…), please?! > Running a non-Debian kernel, but built from the linux-next tree, so > should be possible to switch to experimental and/or stretch-backports > when the time comes. cool! > This one is interesting in that it's running an arm64 kernel with armhf > userland (like the i386 builders that run amd64 kernels). nice! is this the same for p64c too? > We may not > have enough of these to do this systematically yet unless we divert some > of the other arm64 builders, though I'll likely get a few more in this > configuration set up "soon" regardless. cool! > Space is getting a little tight, so if this one > performs well, I'll probably want to decomission one of the slower > boards. I've got another Pine64+ that should be ready soon, and *maybe* > an odroid-c2 as well, and likely some additional board donations > coming... maybe I should get a bigger UPS and another network switch to > support another 8 boards... sounds like it! > I think it is only configured with ssh keys for holger, but if someone > else is able to configure it and has the time I can add them as well. mattia's keys should be there as well now… -- cheers, Holger signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#855282: debsign: support .buildinfo files
user reproducible-builds@lists.alioth.debian.org usertag 855282 toolchain thanks -- cheers, Holger signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#855282: debsign: support .buildinfo files
Package: devscripts Version: 2.17.1 Severity: wishlist Dear Maintainer, dpkg since version 1.18.19 has been signing buildinfo files by default. debsign at the moment will ignore these and leave them unsigned. It would be good to support them. Ximin -- Package-specific info: --- /etc/devscripts.conf --- --- ~/.devscripts --- Not present -- System Information: Debian Release: 9.0 APT prefers testing APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (300, 'unstable'), (200, 'experimental'), (1, 'experimental-debug') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.8.0-2-amd64 (SMP w/4 CPU cores) Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages devscripts depends on: ii dpkg-dev 1.18.22 ii libc62.24-9 ii perl 5.24.1-1 pn python3:any Versions of packages devscripts recommends: ii apt 1.4~rc1 ii at 3.1.20-3 ii curl7.52.1-2 ii dctrl-tools 2.24-2 ii debian-keyring 2017.01.20 ii dput0.12.0 ii equivs 2.0.9+nmu1 ii fakeroot1.21-3.1 ii file1:5.29-3 ii gnupg 2.1.18-3 ii gnupg2 2.1.18-3 ii libdistro-info-perl 0.14 ii libdpkg-perl1.18.22 ii libencode-locale-perl 1.05-1 ii libgit-wrapper-perl 0.047-1 ii liblist-compare-perl0.53-1 ii liblwp-protocol-https-perl 6.06-2 ii libsoap-lite-perl 1.20-1 ii liburi-perl 1.71-1 ii libwww-perl 6.15-1 ii licensecheck3.0.29-1 ii lintian 2.5.50.1 ii man-db 2.7.6.1-2 ii patch 2.7.5-1 ii patchutils 0.3.4-2 ii python3-debian 0.1.30 ii python3-magic 1:5.29-3 ii sensible-utils 0.0.9 ii strace 4.15-2 ii unzip 6.0-21 ii wdiff 1.2.2-2 ii wget1.18-4 ii xz-utils5.2.2-1.2 Versions of packages devscripts suggests: ii adequate 0.15.1 ii autopkgtest 4.3 pn bls-standalone ii bsd-mailx [mailx]8.1.2-0.20160123cvs-3 ii build-essential 12.3 pn check-all-the-things pn cvs-buildpackage pn devscripts-el ii diffoscope 67 ii disorderfs 0.5.1-1 pn dose-extra pn duck ii faketime 0.9.6-7 ii gnuplot 5.0.5+dfsg1-5 ii gpgv 2.1.18-3 pn how-can-i-help ii libauthen-sasl-perl 2.1600-1 ii libfile-desktopentry-perl0.22-1 pn libnet-smtps-perl pn libterm-size-perl ii libtimedate-perl 2.3000-2 ii libyaml-syck-perl1.29-1+b2 pn mozilla-devscripts ii mutt 1.7.2-1 ii openssh-client [ssh-client] 1:7.4p1-6 ii piuparts 0.75 pn ratt ii reprotest0.6 pn svn-buildpackage pn w3m -- no debconf information ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#855273: diffoscope: still fails to clean up after SIGTERM
On Thu, Feb 16, 2017 at 09:58:00AM +, Ximin Luo wrote: > > |Thu Feb 16 03:30:35 UTC 2017 E: otb failed to build reproducibly in > > experimental on i386. > Where did you do this? On the page for otb on tests.r-b.org in the rbuild.log > I see: > > Mon Jan 23 06:11:14 UTC 2017 E: otb failed to build reproducibly in unstable > on amd64. You are lookig on unstable/amd64, I look at experimental/i386 (read the messagges). other packages that might have similar messagges: 2.4M./girara_unstable_armhf_tmp-OdOGN 15M ./madness_testing_arm64_tmp-6cC4d 16M ./lazarus_unstable_arm64_tmp-NRSHQ 25M ./opal_unstable_arm64_tmp-fvWhN 32M ./libgtkada_experimental_i386_tmp-D9a8y 90M ./libgtkada_experimental_armhf_tmp-ZK0R3 106M./itksnap_testing_amd64_tmp-r0qPs 233M./gcc-avr_unstable_arm64_tmp-GiDvh 282M./connectome-workbench_experimental_i386_tmp-qz15D 339M./scribus-ng_experimental_i386_tmp-eO3ve 346M./octave_experimental_i386_tmp-iTHf3 573M./mariadb-10.0_unstable_arm64_tmp-ZX1Jo 626M./asl_unstable_arm64_tmp-kK0ay 627M./gmsh_testing_amd64_tmp-U0i82 670M./gcc-mingw-w64_unstable_arm64_tmp-RySny 1.1G./magics++_testing_amd64_tmp-bSrFG 2.9G./libreoffice_unstable_arm64_tmp-SleAn 3.1G./blender_unstable_arm64_tmp-YEcAH 4.5G./espresso_testing_arm64_tmp-5Jpk0 4.8G./mongodb_testing_amd64_tmp-IETFA 5.5G./icedove_unstable_i386_tmp-UKZGF 13G ./espresso_testing_amd64_tmp-hZlQq 14G ./polymake_unstable_arm64_tmp-gUfUC 16G ./espresso_unstable_arm64_tmp-oYmFV 20G ./metview_testing_amd64_tmp-MQ7e1 24G ./otb_experimental_i386_tmp-C27b2 110G. > This possibly means diffoscope got a second SIGTERM whilst it was trying to > clean up the first one. But this isn't present in your example above. We only do timout 120m nice schroot diffoscope so it should really receive only one SIGTERM. > I tried to download this URL but it looks like it's been deleted already. :( Yes, I deleted everything, as it was filling the disk… (and disabled the saving of artifacts, again). Besides, the temporary directory is 700, so it would need a chmod to be visible on the web. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#855273: diffoscope: still fails to clean up after SIGTERM
Mattia Rizzolo: > Package: diffoscope > Version: 77 > Severity: important > > So, yesterday we tried to re-enable artifacts saving on jenkins, and the > disc filled again because of GBs of temporary files left around. > > In a log the only message I see is: > > |Wed Feb 15 23:28:21 UTC 2017 I: diffoscope 77 will be used to compare the > two builds: > |E: Caught signal ‘Terminated’ > |Thu Feb 16 03:30:35 UTC 2017 E: otb failed to build reproducibly in > experimental on i386. > > I have yet to try to reproduce it this time (and weird, because in when > I tried before reenabling the saving artifacts it did clean up for me). > > [..] > Where did you do this? On the page for otb on tests.r-b.org in the rbuild.log I see: Mon Jan 23 04:11:07 UTC 2017 I: diffoscope 69 will be used to compare the two builds: E: Caught signal ‘Terminated’: terminating immediately E: Caught signal ‘Terminated’ Mon Jan 23 06:11:14 UTC 2017 E: otb failed to build reproducibly in unstable on amd64. This possibly means diffoscope got a second SIGTERM whilst it was trying to clean up the first one. But this isn't present in your example above. WARNING: You shouldn't trust packages downloaded from this host, they can contain malware or the worst of your fears, packaged nicely in debian format. If you are aware of this and just want to use these artifacts to investigate why diffoscope 69 had issues, you can download the artifacts from the following location: https://tests.reproducible-builds.org/debian/artifacts/r00t-me/otb_unstable_amd64_tmp-LLmJ5/ I tried to download this URL but it looks like it's been deleted already. :( X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#855273: diffoscope: still fails to clean up after SIGTERM
Package: diffoscope Version: 77 Severity: important So, yesterday we tried to re-enable artifacts saving on jenkins, and the disc filled again because of GBs of temporary files left around. In a log the only message I see is: |Wed Feb 15 23:28:21 UTC 2017 I: diffoscope 77 will be used to compare the two builds: |E: Caught signal ‘Terminated’ |Thu Feb 16 03:30:35 UTC 2017 E: otb failed to build reproducibly in experimental on i386. I have yet to try to reproduce it this time (and weird, because in when I tried before reenabling the saving artifacts it did clean up for me). -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds