Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
* Cyril Brulebois [2014-03-03 01:40]: > > There must be a small packaging bug somewhere… > > The timestamp issue was indeed a nice clue. gzip has -n to avoid storing > such information, improving build reproducibility: > I shall note pigz needs has -n and -T: ... > Tagging with patch as the solution has been identified; I haven't > committed a proper patch yet though. Sounds like this bug should be closed: commit c86563eeeb78b4d6b5841a012b21abac55aec56d Author: Cyril Brulebois Date: Thu Nov 26 01:57:36 2015 +0100 build/config/x86.cfg: Also pass -n to gzip. commit d7a975094883477cc708f382e430a280fc8c7488 Author: Cyril Brulebois Date: Thu Nov 26 01:46:33 2015 +0100 Rename GZIP into gzip, and pass an extra -T to pigz. It needs both -n and -T to behave as gzip's -n. -- Martin Michlmayr http://www.cyrius.com/
Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
Package: mirrors Hi, I don't really know where to file that; please reassign if I'm wrong. I recently downloaded the debian-installer netboot archive at http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/netboot.tar.gz which is the GTK version of the netboot installer (this bug doesn't affect the text version). I noticed, after unpacking it and checking for the sums, hoping that they will be the same as if I downloaded all the files individually, that they all validate except one: the initrd.gz for this installer. (I checked the sum of the .tar.gz itself, and it's OK). The sums are, e.g., here: http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/SHA256SUMS When getting directly the initrd.gz from http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/debian-installer/amd64/initrd.gz I correctly get f8971317915ed2ce8358b24ca88ea95c75ebe97a1e0a95f60c7977da368b3352 But when extracting it from the archive, I get c2103b9533baa88814e770b493e5ba93f3043f3d73ce6e018addcd7c84b22cd4 By looking closer, the uncompressed initrd from both files is the same. Only the date (from the gzip header) differs by a couple of seconds. And this only happens for the GTK installer, not the text one, once again… Furthermore, I also realized that Ubuntu is affected, too (!); the sum for the GTK installer of Precise Pangolin has the same problem. There must be a small packaging bug somewhere… Regards, -- Benjamin Cama -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
Control: tag -1 patch Benjamin Cama (2013-03-28): > I don't really know where to file that; please reassign if I'm wrong. No worries; it finally landed in the debian-installer territory, where it belongs. > I noticed, after unpacking it and checking for the sums, hoping that > they will be the same as if I downloaded all the files individually, > that they all validate except one: the initrd.gz for this installer. (I > checked the sum of the .tar.gz itself, and it's OK). > > The sums are, e.g., here: > http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/SHA256SUMS > > When getting directly the initrd.gz from > http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/debian-installer/amd64/initrd.gz > I correctly get > f8971317915ed2ce8358b24ca88ea95c75ebe97a1e0a95f60c7977da368b3352 > But when extracting it from the archive, I get > c2103b9533baa88814e770b493e5ba93f3043f3d73ce6e018addcd7c84b22cd4 > > By looking closer, the uncompressed initrd from both files is the same. > Only the date (from the gzip header) differs by a couple of seconds. And > this only happens for the GTK installer, not the text one, once again… > > Furthermore, I also realized that Ubuntu is affected, too (!); the sum > for the GTK installer of Precise Pangolin has the same problem. > > There must be a small packaging bug somewhere… The timestamp issue was indeed a nice clue. gzip has -n to avoid storing such information, improving build reproducibility: |-n --no-name | When compressing, do not save the original file name and time | stamp by default. (The original name is always saved if the name | had to be truncated.) When decompressing, do not restore the | original file name if present (remove only the gzip suffix from | the compressed file name) and do not restore the original time | stamp if present (copy it from the compressed file). This option | is the default when decompressing. I shall note pigz needs has -n and -T: |-n --no-name | Do not store or restore file name in/from header. |-T --no-time | Do not store or restore mod time in/from header. But why do we need that, you ask? Let's mention the md5sum of each generated initrd.gz during a full build: | MD5SUM: c47a2e5b24eb72737e566c0fee4bc2fe ./tmp/cdrom_gtk/initrd.gz | MD5SUM: 3767e38d0790c2a89f5ccacc7f4845af ./tmp/cdrom_isolinux/initrd.gz | MD5SUM: 99646ced5c6045386c4ebbac9334bee7 ./tmp/cdrom_gtk/initrd.gz | MD5SUM: 73902a6e23b5c281172182a19633cf4d ./tmp/cdrom_gtk/initrd.gz | MD5SUM: bb6a60f95815569a5cc41fe9466e9d97 ./tmp/netboot/initrd.gz | MD5SUM: 4707ee1a9a643e4513723029129e07dd ./tmp/netboot-gtk/initrd.gz [1] | MD5SUM: 6614937379a7f83a95f8741a63f00394 ./tmp/netboot-gtk/initrd.gz [2] | MD5SUM: 586b60ef353423481f828ee03927a682 ./tmp/hd-media_gtk/initrd.gz | MD5SUM: 0e3bfa9304fbcf9a1081e6746866bb7f ./tmp/hd-media/initrd.gz | MD5SUM: 5a5f192055293b6042463650dfa1a0b4 ./tmp/hd-media_gtk/initrd.gz Let's look at the file extracted from netboot.tar.gz: | 4707ee1a9a643e4513723029129e07dd ./foo/debian-installer/amd64/initrd.gz Let's look at the MD5SUMS file: | 6614937379a7f83a95f8741a63f00394 ./netboot/gtk/debian-installer/amd64/initrd.gz So netboot.tar.gz contains the initrd.gz that was generated on a first pass, while the MD5SUMS (generated at the end of the build process, through “make release”) contains the checksum of the one generated on a second pass. Adding -n/-T to gzip/pigz makes both initrd.gz identical so this issue goes away. Tagging with patch as the solution has been identified; I haven't committed a proper patch yet though. Mraw, KiBi. signature.asc Description: Digital signature
Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
Control: tag -1 +confirmed , severity -1 minor , reassign -1 cdimage.debian.org Hi, On Thu, Mar 28, 2013 at 07:28:54PM +0100, Benjamin Cama wrote: > I don't really know where to file that; please reassign if I'm wrong. If this is specific to one mirror, it's a mirroring bug, else it's a bug against cdimage.debian.org http://www.debian.org/Bugs/pseudo-packages > I recently downloaded the debian-installer netboot archive at > http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/netboot.tar.gz > which is the GTK version of the netboot installer (this bug doesn't > affect the text version). > > I noticed, after unpacking it and checking for the sums, hoping that > they will be the same as if I downloaded all the files individually, > that they all validate except one: the initrd.gz for this installer. (I > checked the sum of the .tar.gz itself, and it's OK). > > The sums are, e.g., here: > http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/SHA256SUMS > > When getting directly the initrd.gz from > http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/debian-installer/amd64/initrd.gz > I correctly get > f8971317915ed2ce8358b24ca88ea95c75ebe97a1e0a95f60c7977da368b3352 > But when extracting it from the archive, I get > c2103b9533baa88814e770b493e5ba93f3043f3d73ce6e018addcd7c84b22cd4 > > By looking closer, the uncompressed initrd from both files is the same. > Only the date (from the gzip header) differs by a couple of seconds. And > this only happens for the GTK installer, not the text one, once again… > > Furthermore, I also realized that Ubuntu is affected, too (!); the sum > for the GTK installer of Precise Pangolin has the same problem. > > There must be a small packaging bug somewhere… In order to reproduce this: cp -a gtk gtk.check cd gtk.check; rm -r debian-installer pxelinux* tar xf netboot.tar.gz vimdiff <(cd gtk/ && find -type f -exec sha256sum '{}' \;) <(cd gtk.check && find -type f -exec sha256sum '{}' \;) --> we can confirm ./debian-installer/amd64/initrd.gz has a different sum once extracted. -- Simon Paillard -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#704162: Wrong sum for GTK installer initrd.gz inside netboot.tar.gz
Control: reassign -1 debian-installer Hmmm. This seems like an odd bug. But it's definitely not a CD issue - this belongs to the d-i team. On Wed, Jun 05, 2013 at 10:57:05PM +0200, Simon Paillard wrote: >Control: tag -1 +confirmed , severity -1 minor , reassign -1 cdimage.debian.org > >Hi, > >On Thu, Mar 28, 2013 at 07:28:54PM +0100, Benjamin Cama wrote: >> I don't really know where to file that; please reassign if I'm wrong. > >If this is specific to one mirror, it's a mirroring bug, else it's a bug >against cdimage.debian.org >http://www.debian.org/Bugs/pseudo-packages > >> I recently downloaded the debian-installer netboot archive at >> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/netboot.tar.gz >> which is the GTK version of the netboot installer (this bug doesn't >> affect the text version). >> >> I noticed, after unpacking it and checking for the sums, hoping that >> they will be the same as if I downloaded all the files individually, >> that they all validate except one: the initrd.gz for this installer. (I >> checked the sum of the .tar.gz itself, and it's OK). >> >> The sums are, e.g., here: >> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/SHA256SUMS >> >> When getting directly the initrd.gz from >> http://debian.univ-nantes.fr/debian/dists/wheezy/main/installer-amd64/current/images/netboot/gtk/debian-installer/amd64/initrd.gz >> I correctly get >> f8971317915ed2ce8358b24ca88ea95c75ebe97a1e0a95f60c7977da368b3352 >> But when extracting it from the archive, I get >> c2103b9533baa88814e770b493e5ba93f3043f3d73ce6e018addcd7c84b22cd4 >> >> By looking closer, the uncompressed initrd from both files is the same. >> Only the date (from the gzip header) differs by a couple of seconds. And >> this only happens for the GTK installer, not the text one, once again… >> >> Furthermore, I also realized that Ubuntu is affected, too (!); the sum >> for the GTK installer of Precise Pangolin has the same problem. >> >> There must be a small packaging bug somewhere… > >In order to reproduce this: > >cp -a gtk gtk.check >cd gtk.check; rm -r debian-installer pxelinux* >tar xf netboot.tar.gz >vimdiff <(cd gtk/ && find -type f -exec sha256sum '{}' \;) <(cd gtk.check && >find -type f -exec sha256sum '{}' \;) > >--> we can confirm ./debian-installer/amd64/initrd.gz has a different sum once >extracted. > > >-- >Simon Paillard > > >-- >To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org >with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org >Archive: http://lists.debian.org/20130605205705.ga1...@glenfiddich.mraw.org > > -- Steve McIntyre, Cambridge, UK.st...@einval.com Who needs computer imagery when you've got Brian Blessed? -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org