Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
* Cyril Brulebois [2014-03-03 01:40]: > > There must be a small packaging bug somewhere… > > The timestamp issue was indeed a nice clue. gzip has -n to avoid storing > such information, improving build reproducibility: > I shall note pigz needs has -n and -T: ... > Tagging with patch as the solution has been identified; I haven't > committed a proper patch yet though. Sounds like this bug should be closed: commit c86563eeeb78b4d6b5841a012b21abac55aec56d Author: Cyril Brulebois Date: Thu Nov 26 01:57:36 2015 +0100 build/config/x86.cfg: Also pass -n to gzip. commit d7a975094883477cc708f382e430a280fc8c7488 Author: Cyril Brulebois Date: Thu Nov 26 01:46:33 2015 +0100 Rename GZIP into gzip, and pass an extra -T to pigz. It needs both -n and -T to behave as gzip's -n. -- Martin Michlmayr http://www.cyrius.com/
Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
Package: mirrors Hi, I don't really know where to file that; please reassign if I'm wrong. I recently downloaded the debian-installer netboot archive at http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/netboot.tar.gz which is the GTK version of the netboot installer (this bug doesn't affect the text version). I noticed, after unpacking it and checking for the sums, hoping that they will be the same as if I downloaded all the files individually, that they all validate except one: the initrd.gz for this installer. (I checked the sum of the .tar.gz itself, and it's OK). The sums are, e.g., here: http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/SHA256SUMS When getting directly the initrd.gz from http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/debian-installer/amd64/initrd.gz I correctly get f8971317915ed2ce8358b24ca88ea95c75ebe97a1e0a95f60c7977da368b3352 But when extracting it from the archive, I get c2103b9533baa88814e770b493e5ba93f3043f3d73ce6e018addcd7c84b22cd4 By looking closer, the uncompressed initrd from both files is the same. Only the date (from the gzip header) differs by a couple of seconds. And this only happens for the GTK installer, not the text one, once again… Furthermore, I also realized that Ubuntu is affected, too (!); the sum for the GTK installer of Precise Pangolin has the same problem. There must be a small packaging bug somewhere… Regards, -- Benjamin Cama -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
Control: tag -1 patch Benjamin Cama (2013-03-28): > I don't really know where to file that; please reassign if I'm wrong. No worries; it finally landed in the debian-installer territory, where it belongs. > I noticed, after unpacking it and checking for the sums, hoping that > they will be the same as if I downloaded all the files individually, > that they all validate except one: the initrd.gz for this installer. (I > checked the sum of the .tar.gz itself, and it's OK). > > The sums are, e.g., here: > http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/SHA256SUMS > > When getting directly the initrd.gz from > http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/debian-installer/amd64/initrd.gz > I correctly get > f8971317915ed2ce8358b24ca88ea95c75ebe97a1e0a95f60c7977da368b3352 > But when extracting it from the archive, I get > c2103b9533baa88814e770b493e5ba93f3043f3d73ce6e018addcd7c84b22cd4 > > By looking closer, the uncompressed initrd from both files is the same. > Only the date (from the gzip header) differs by a couple of seconds. And > this only happens for the GTK installer, not the text one, once again… > > Furthermore, I also realized that Ubuntu is affected, too (!); the sum > for the GTK installer of Precise Pangolin has the same problem. > > There must be a small packaging bug somewhere… The timestamp issue was indeed a nice clue. gzip has -n to avoid storing such information, improving build reproducibility: |-n --no-name | When compressing, do not save the original file name and time | stamp by default. (The original name is always saved if the name | had to be truncated.) When decompressing, do not restore the | original file name if present (remove only the gzip suffix from | the compressed file name) and do not restore the original time | stamp if present (copy it from the compressed file). This option | is the default when decompressing. I shall note pigz needs has -n and -T: |-n --no-name | Do not store or restore file name in/from header. |-T --no-time | Do not store or restore mod time in/from header. But why do we need that, you ask? Let's mention the md5sum of each generated initrd.gz during a full build: | MD5SUM: c47a2e5b24eb72737e566c0fee4bc2fe ./tmp/cdrom_gtk/initrd.gz | MD5SUM: 3767e38d0790c2a89f5ccacc7f4845af ./tmp/cdrom_isolinux/initrd.gz | MD5SUM: 99646ced5c6045386c4ebbac9334bee7 ./tmp/cdrom_gtk/initrd.gz | MD5SUM: 73902a6e23b5c281172182a19633cf4d ./tmp/cdrom_gtk/initrd.gz | MD5SUM: bb6a60f95815569a5cc41fe9466e9d97 ./tmp/netboot/initrd.gz | MD5SUM: 4707ee1a9a643e4513723029129e07dd ./tmp/netboot-gtk/initrd.gz [1] | MD5SUM: 6614937379a7f83a95f8741a63f00394 ./tmp/netboot-gtk/initrd.gz [2] | MD5SUM: 586b60ef353423481f828ee03927a682 ./tmp/hd-media_gtk/initrd.gz | MD5SUM: 0e3bfa9304fbcf9a1081e6746866bb7f ./tmp/hd-media/initrd.gz | MD5SUM: 5a5f192055293b6042463650dfa1a0b4 ./tmp/hd-media_gtk/initrd.gz Let's look at the file extracted from netboot.tar.gz: | 4707ee1a9a643e4513723029129e07dd ./foo/debian-installer/amd64/initrd.gz Let's look at the MD5SUMS file: | 6614937379a7f83a95f8741a63f00394 ./netboot/gtk/debian-installer/amd64/initrd.gz So netboot.tar.gz contains the initrd.gz that was generated on a first pass, while the MD5SUMS (generated at the end of the build process, through “make release”) contains the checksum of the one generated on a second pass. Adding -n/-T to gzip/pigz makes both initrd.gz identical so this issue goes away. Tagging with patch as the solution has been identified; I haven't committed a proper patch yet though. Mraw, KiBi. signature.asc Description: Digital signature
Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
Control: tag -1 +confirmed , severity -1 minor , reassign -1 cdimage.debian.org
Hi,
On Thu, Mar 28, 2013 at 07:28:54PM +0100, Benjamin Cama wrote:
> I don't really know where to file that; please reassign if I'm wrong.
If this is specific to one mirror, it's a mirroring bug, else it's a bug
against cdimage.debian.org
http://www.debian.org/Bugs/pseudo-packages
> I recently downloaded the debian-installer netboot archive at
> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/netboot.tar.gz
> which is the GTK version of the netboot installer (this bug doesn't
> affect the text version).
>
> I noticed, after unpacking it and checking for the sums, hoping that
> they will be the same as if I downloaded all the files individually,
> that they all validate except one: the initrd.gz for this installer. (I
> checked the sum of the .tar.gz itself, and it's OK).
>
> The sums are, e.g., here:
> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/SHA256SUMS
>
> When getting directly the initrd.gz from
> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/debian-installer/amd64/initrd.gz
> I correctly get
> f8971317915ed2ce8358b24ca88ea95c75ebe97a1e0a95f60c7977da368b3352
> But when extracting it from the archive, I get
> c2103b9533baa88814e770b493e5ba93f3043f3d73ce6e018addcd7c84b22cd4
>
> By looking closer, the uncompressed initrd from both files is the same.
> Only the date (from the gzip header) differs by a couple of seconds. And
> this only happens for the GTK installer, not the text one, once again…
>
> Furthermore, I also realized that Ubuntu is affected, too (!); the sum
> for the GTK installer of Precise Pangolin has the same problem.
>
> There must be a small packaging bug somewhere…
In order to reproduce this:
cp -a gtk gtk.check
cd gtk.check; rm -r debian-installer pxelinux*
tar xf netboot.tar.gz
vimdiff <(cd gtk/ && find -type f -exec sha256sum '{}' \;) <(cd gtk.check &&
find -type f -exec sha256sum '{}' \;)
--> we can confirm ./debian-installer/amd64/initrd.gz has a different sum once
extracted.
--
Simon Paillard
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
Control: reassign -1 debian-installer
Hmmm. This seems like an odd bug. But it's definitely not a CD issue -
this belongs to the d-i team.
On Wed, Jun 05, 2013 at 10:57:05PM +0200, Simon Paillard wrote:
>Control: tag -1 +confirmed , severity -1 minor , reassign -1 cdimage.debian.org
>
>Hi,
>
>On Thu, Mar 28, 2013 at 07:28:54PM +0100, Benjamin Cama wrote:
>> I don't really know where to file that; please reassign if I'm wrong.
>
>If this is specific to one mirror, it's a mirroring bug, else it's a bug
>against cdimage.debian.org
>http://www.debian.org/Bugs/pseudo-packages
>
>> I recently downloaded the debian-installer netboot archive at
>> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/netboot.tar.gz
>> which is the GTK version of the netboot installer (this bug doesn't
>> affect the text version).
>>
>> I noticed, after unpacking it and checking for the sums, hoping that
>> they will be the same as if I downloaded all the files individually,
>> that they all validate except one: the initrd.gz for this installer. (I
>> checked the sum of the .tar.gz itself, and it's OK).
>>
>> The sums are, e.g., here:
>> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/SHA256SUMS
>>
>> When getting directly the initrd.gz from
>> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/debian-installer/amd64/initrd.gz
>> I correctly get
>> f8971317915ed2ce8358b24ca88ea95c75ebe97a1e0a95f60c7977da368b3352
>> But when extracting it from the archive, I get
>> c2103b9533baa88814e770b493e5ba93f3043f3d73ce6e018addcd7c84b22cd4
>>
>> By looking closer, the uncompressed initrd from both files is the same.
>> Only the date (from the gzip header) differs by a couple of seconds. And
>> this only happens for the GTK installer, not the text one, once again…
>>
>> Furthermore, I also realized that Ubuntu is affected, too (!); the sum
>> for the GTK installer of Precise Pangolin has the same problem.
>>
>> There must be a small packaging bug somewhere…
>
>In order to reproduce this:
>
>cp -a gtk gtk.check
>cd gtk.check; rm -r debian-installer pxelinux*
>tar xf netboot.tar.gz
>vimdiff <(cd gtk/ && find -type f -exec sha256sum '{}' \;) <(cd gtk.check &&
>find -type f -exec sha256sum '{}' \;)
>
>--> we can confirm ./debian-installer/amd64/initrd.gz has a different sum once
>extracted.
>
>
>--
>Simon Paillard
>
>
>--
>To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
>with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
>Archive: http://lists.debian.org/20130605205705.ga1...@glenfiddich.mraw.org
>
>
--
Steve McIntyre, Cambridge, UK.st...@einval.com
Who needs computer imagery when you've got Brian Blessed?
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
