Re: [Reproducible-builds] Bug#805321: Bug#805321: debian-installer: builds unreproducible netboot images
On Sat, Nov 28, 2015 at 12:08:44AM +, Steven Chamberlain wrote: > > FWIW, I'm not exactly entirely convinced by the exporting of the > > SOURCE_DATE_EPOCH variable from debian/rules; all other variables have > > been passed without exporting so I'm wondering if we shouldn't adapt > > this to behave like other variables, reducing possible surprise for > > users. > > Just to explain that -- if it's defined in the environment, it requires > no special handling and doesn't need to be (re-)exported. I think this > is maybe the case now for dpkg-buildpackage in sid? it's not, dpkg hasn't merged that patch (tbh I'm not even sure we forwarded that). Though debhelper exports SOURCE_DATE_EPOCH when using the dh sequencer. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: http://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#805321: Bug#805321: debian-installer: builds unreproducible netboot images
Hi, Cyril Brulebois wrote: > I've cherry-picked 3 patches from there onto master locally and I'm > currently running diffoscope to see how that goes (and it's taking > ages…): I'm guessing the initrd would differ if the Linux tool to generate it stores timestamps. If its compressed size varies much due to those differences, the .iso block numbers may vary as a result. > FWIW, I'm not exactly entirely convinced by the exporting of the > SOURCE_DATE_EPOCH variable from debian/rules; all other variables have > been passed without exporting so I'm wondering if we shouldn't adapt > this to behave like other variables, reducing possible surprise for > users. Just to explain that -- if it's defined in the environment, it requires no special handling and doesn't need to be (re-)exported. I think this is maybe the case now for dpkg-buildpackage in sid? If the dpkg-buildpackage environment doesn't have SOURCE_DATE_EPOCH (e.g. jessie), debian/rules sets it to the correct value, and so must export that. Or (for jessie or sid), in case build/Makefile is used directly, outside of a package build, we set SOURCE_DATE_EPOCH to a dummy value ("now") if undefined (since ../debian/changelog may not exist), which we need when calling makefs from within that Makefile. We export it for use by gen-tarball to avoid duplication there. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#805321: Bug#805321: debian-installer: builds unreproducible netboot images
Hi, Steven Chamberlain(2015-11-22): > I rewrote the patches according to KiBi's feedback and they are > now uploaded to our jessie-kfreebsd suite, and this Git branch: > https://anonscm.debian.org/cgit/d-i/debian-installer.git/log/?h=jessie-kfreebsd I've cherry-picked 3 patches from there onto master locally and I'm currently running diffoscope to see how that goes (and it's taking ages…): c182491b05fec16497f2bf1290cac16773d175f9 5d59fd1813e794d0821c00757dd56fd9ca25ed16 d126622567cfbe10d7f8a207a292eaab622ef73e > In my own testing on ZFS, file ordering was still an issue for the > makefs tool that builds the initrd. But if I were to try again > on UFS, I hope to be able to reproduce the entire > netboot-installer-images tarball as built by the buildds. > > This tarball includes bits that are bundled onto the official release > images by debian-cd tools. Making this reproducible is a prerequisite > for someday having reproducibly-built official release images. > > I could merge these patches into sid if they seem okay? The only > commit that should not be merged is this one, which is specific to > jessie-kfreebsd and must be slightly changed for sid: > kfreebsd: use makefs -T to clamp timestamps I suppose your time is better spent actually working on kfreebsd so that's why I decided to cherry-pick the patches myself. FWIW, I'm not exactly entirely convinced by the exporting of the SOURCE_DATE_EPOCH variable from debian/rules; all other variables have been passed without exporting so I'm wondering if we shouldn't adapt this to behave like other variables, reducing possible surprise for users. I don't think that's a showstopper for a push to master though; just thinking out loud. > I expect that Linux d-i builds will have some reproducibility issues > in whatever generates the initrd or ISOs, but I may look into that > after the jessie-kfreebsd release is done. Sure thing, thanks again! Mraw, KiBi. signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#805321: Bug#805321: debian-installer: builds unreproducible netboot images
Cyril Brulebois(2015-11-26): > I've cherry-picked 3 patches from there onto master locally and I'm > currently running diffoscope to see how that goes (and it's taking > ages…): > c182491b05fec16497f2bf1290cac16773d175f9 > 5d59fd1813e794d0821c00757dd56fd9ca25ed16 > d126622567cfbe10d7f8a207a292eaab622ef73e The following files are different after two builds with debuild -b once the 3 commits are applied: ./installer-amd64/20151024/images/cdrom/gtk/debian-cd_info.tar.gz ./installer-amd64/20151024/images/cdrom/gtk/initrd.gz ./installer-amd64/20151024/images/cdrom/initrd.gz ./installer-amd64/20151024/images/hd-media/boot.img.gz ./installer-amd64/20151024/images/hd-media/gtk/initrd.gz ./installer-amd64/20151024/images/hd-media/initrd.gz ./installer-amd64/20151024/images/netboot/debian-installer/amd64/initrd.gz ./installer-amd64/20151024/images/netboot/gtk/debian-installer/amd64/initrd.gz ./installer-amd64/20151024/images/netboot/gtk/netboot.tar.gz ./installer-amd64/20151024/images/netboot/netboot.tar.gz I see at least timestamp issues in initramfses; this might be due to the fact that pigz (installed in my development chroots) seems to need both -n and -T to behave like gzip's -n. I might poke a bit more around this before pushing. Mraw, KiBi. signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds