[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
Disabling fragments, by passing the -no-fragments option to mksquashfs, retains performance while providing reproducibility: A) 12 core Xeon E5-2609 v3 with 16GB of ram Without the frag deflator removal patch and with the -no-fragments option: real 0.90-0.97(0.928+/-0.023) user 9.47-9.62(9.531+/-0.046) sys 0.09-0.18(0.12+/-0.022) B) Raspberry Pi 2 (4 core, 1GB of ram) Without the frag deflator removal patch and with the -no-fragments option: real 14.48-14.61(14.547+/-0.033) user 53.25-53.66(53.467+/-0.13) sys 2.91-3.24(3.056+/-0.094) We actually see a small performance improvement. The downside here is that the resulting squashfs image isn't as small. This particular nmap snap increases from 5.4M to 5.8M in size. However, I suspect that the small increase in size is negated entirely by mksquashfs producing consistent snaps that can be accurately binary diffed for delta upgrades. More investigation is needed to make that claim but that's my current hunch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
I've done some performance testing of the patch that removes the frag deflator. The numbers below are in "min-max(avg+/-dev)" formatting and they were gathered with 10 consecutive runs of the following command: $ ./mksquashfs nmap_unpack nmap_$i.snap -noappend -comp xz -all-root -no-xattrs -quiet A) 12 core Xeon E5-2609 v3 with 16GB of ram Without the frag deflator removal patch: real 0.91-1.01(0.965+/-0.032) user 9.76-10.06(9.937+/-0.076) sys 0.15-0.37(0.235+/-0.073) With the frag deflator removal patch: real 2.93-3.00(2.96+/-0.024) user 9.46-9.59(9.53+/-0.045) sys 0.10-0.14(0.116+/-0.012) B) Raspberry Pi 2 (4 core, 1GB of ram) Without the frag deflator removal patch: real 14.96-15.32(15.116+/-0.096) user 55.10-55.73(55.365+/-0.16) sys 3.53-4.17(3.764+/-0.18) With the frag deflator removal patch: real 20.98-21.36(21.107+/-0.13) user 51.51-52.10(51.694+/-0.19) sys 1.41-1.77(1.572+/-0.086) There's a considerable increase in mksquashfs execution time when the frag deflator threads are removed. I would assume that this nmap snap is relatively small compared to other snaps so it might be a factor to keep in mind. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
@jdstrand yes, that's unfortunately correct -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
IIUC, this would mean that snaps would have to be created with a mksquashfs with a patched/removed frag deflator to make reproducible snaps. Is that correct? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
This bug has to do with the frag deflator thread in mksquashfs. I haven't fully determined the problem but it is likely due to parallel processing (NOTE: the '-processors 1' option does not solve the bug). There's an out-of-tree patch that disables the thread by removing its working queue: https://github.com/intrigeri/squashfs- tools/commit/c2c4d15580adee66b68c29eb9d2ee660225449bc I've verified that the patch results in consistent, reproducible squashfs images: $ md5sum *.snap 0161f2c324cb2753429b8c51818132c5 nmap_1.snap 0161f2c324cb2753429b8c51818132c5 nmap_2.snap 0161f2c324cb2753429b8c51818132c5 nmap_3.snap 0161f2c324cb2753429b8c51818132c5 nmap_4.snap 0161f2c324cb2753429b8c51818132c5 nmap_5.snap 5fccb0276e2cd329021591129855eb83 nmap_7.12SVN-0.4_armhf.snap Note that nmap_7.12SVN-0.4_armhf.snap is the snap that was originally attached to this bug which was created with a mksquashfs that had the frag deflator thread enabled, so its snap differs from the ones created with the patched mksquashfs. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
I'm having trouble with this one because I can't reproduce it in the same way that's described in the bug description. The bug description shows that two of the snap files share one sha512 and two other snap files share another sha512. What I'm seeing is that no snap files share the same sha512 (md5sum shown below for easier reading): $ for i in 1 2 3 4 5; do sudo sh -c "unsquashfs -f -d nmap_unpack_$i nmap_7.12SVN-0.4_armhf.snap; mksquashfs nmap_unpack_$i nmap_$i.snap -fstime 1461597924 -noappend -comp xz -all-root -no-xattrs"; done ... $ md5sum *.snap 50c3f6ca2c64a85d89be90086c6025a6 nmap_1.snap b14a8b87581b73e6c68ae7f1e07d1282 nmap_2.snap 91da8708b2ee90e1398215671f5aa4d5 nmap_3.snap 823f9f332a1cdd898904d02886a28f19 nmap_4.snap 1c5ec183fed50dbba8c166f2467ad43a nmap_5.snap 5fccb0276e2cd329021591129855eb83 nmap_7.12SVN-0.4_armhf.snap This test run was on a raspi2 device inside of an ubuntu:16.04 lxd container. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
Hello - the bug title/description makes it sound like this bug is specific to armhf (armhf's squashfs-tools deb package and the armhf nmap snap) but I'd like to confirm that's what you meant to imply. If I use amd64's squashfs-tools deb package and the armhf nmap snap, I can still reproduce this bug. Have you tested if that's the case when using the amd64 squashfs-tools and an amd64 nmap snap? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
** Changed in: click-reviewers-tools (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
root@localhost:~# dpkg -l | grep squashfs ii squashfs-tools 1:4.3-3ubuntu2 armhf Tool to create and append to squashfs filesystems -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1576763] Re: pi2 images generate different checksums on repeated runs
Here's the snap I used for the tests. ** Attachment added: "nmap_7.12SVN-0.4_armhf.snap" https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+attachment/4651490/+files/nmap_7.12SVN-0.4_armhf.snap -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1576763 Title: pi2 images generate different checksums on repeated runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/click-reviewers-tools/+bug/1576763/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
