[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
This bug was fixed in the package livecd-rootfs - 2.525.30 --- livecd-rootfs (2.525.30) bionic; urgency=medium [ Robert C Jennings ] * ubuntu-cpc: Only produce explicitly specified artifacts (LP: #1837254) [ Ethan Hsieh ] * Add support for the Nitrogen6x board (LP: #1838064). -- Łukasz 'sil2100' Zemczak Tue, 10 Sep 2019 16:56:26 +0200 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
This bug was fixed in the package livecd-rootfs - 2.578.8 --- livecd-rootfs (2.578.8) disco; urgency=medium * ubuntu-cpc: Only produce explicitly specified artifacts (LP: #1837254) -- Robert C Jennings Mon, 26 Aug 2019 16:25:24 -0500 ** Changed in: livecd-rootfs (Ubuntu Disco) Status: Fix Committed => Fix Released ** Changed in: livecd-rootfs (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
This bug was fixed in the package livecd-rootfs - 2.408.54 --- livecd-rootfs (2.408.54) xenial; urgency=medium * Revert exclusion of makedev from buildd chroots, as it turned out not to be the problem. Instead, fix up /dev/ptmx to be a character device node rather than a symlink to /dev/pts/ptmx, in line with the discussion in https://bugs.debian.org/817236; I think this is safer than cherry-picking the fix to debootstrap at this point in a stable release cycle (LP: #1844504). livecd-rootfs (2.408.53) xenial; urgency=medium * Fix exclusion of makedev from buildd chroots; debootstrap doesn't respect --exclude for "Priority: required" packages, so we have to purge makedev later (LP: #1844504). livecd-rootfs (2.408.52) xenial; urgency=medium * Exclude makedev from buildd chroots, since it was historically excluded and apparently breaks some builds (e.g. mir; LP: #1844504). livecd-rootfs (2.408.51) xenial; urgency=medium * ubuntu-cpc: Only produce explicitly specified artifacts (LP: #1837254) -- Colin Watson Thu, 26 Sep 2019 10:53:23 +0100 ** Changed in: livecd-rootfs (Ubuntu Xenial) Status: Fix Committed => Fix Released ** Bug watch added: Debian Bug tracker #817236 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=817236 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
Test builds completed for ubuntu-cpc for xenial, bionic, and disco and I'm marking this verification done and ready for promotion. ** Tags removed: verification-needed verification-needed-bionic verification-needed-disco verification-needed-xenial ** Tags added: verification-done verification-done-bionic verification-done-disco verification-done-xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
The autopkgtest failure was a transient network failure during the build. After log analysis the test was re-run and is passing. I will perform verification tests and update the bug with results in the next few days. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
Hello Robert, or anyone else affected, Accepted livecd-rootfs into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd- rootfs/2.578.8 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: livecd-rootfs (Ubuntu Disco) Status: New => Fix Committed ** Tags added: verification-needed verification-needed-disco ** Changed in: livecd-rootfs (Ubuntu Bionic) Status: New => Fix Committed ** Tags added: verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
Yes, please. I would like them sponsored. Thank you! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
I have reviewed and merged all the D/B/X branches for livecd-rootfs. Would you want me to sponsor those? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
I've completed image build testing across xenial, bionic, and disco for the SRU MPs and they're ready for review and upload. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
** Also affects: livecd-rootfs (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: livecd-rootfs (Ubuntu Disco) Importance: Undecided Status: New ** Also affects: livecd-rootfs (Ubuntu Bionic) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
** Description changed: + [Impact] + In parallel builds where a list of image targets are provided, the build may produce binaries that are not part of the named set of targets but are created by series dependencies. These implicitly created binaries may be generated by multiple builds but are unused as our convention for the ubuntu-cpc project is to only consume binaries from the explicitly named image targets; this avoids overwriting the same object by multiple parallel builds. The problem is that as we build out parallel jobs for a number of image targets (multiplied by each supported architecture) we're wasting a lot of resources. + + [Test Case] + + * Specify an image target within the ubuntu-cpc project for a build + + * Observe artifacts in addition to the ones for the named image target + are produced and downloaded + + + [Regression Potential] + + * An artifact from an explicitly named artifact is missing and builds + will begin to remove desired output. Testing is being done to address + this regression. + + [Other Info] + + * This has been running in production in Eoan for a few weeks and we're satisfied with the results. + + [Original Description] + + In parallel builds where a list of image targets are provided, the build + may produce binaries that are not part of the named set of targets but + are created by series dependencies. These implicitly created binaries + may be generated by multiple builds but are unused as our convention for + the ubuntu-cpc project is to only consume binaries from the explicitly + named image targets; this avoids overwriting the same object by multiple + parallel builds. The problem is that as we build out parallel jobs for + a number of image targets (multiplied by each supported architecture) + we're wasting a lot of resources. ** Description changed: [Impact] In parallel builds where a list of image targets are provided, the build may produce binaries that are not part of the named set of targets but are created by series dependencies. These implicitly created binaries may be generated by multiple builds but are unused as our convention for the ubuntu-cpc project is to only consume binaries from the explicitly named image targets; this avoids overwriting the same object by multiple parallel builds. The problem is that as we build out parallel jobs for a number of image targets (multiplied by each supported architecture) we're wasting a lot of resources. [Test Case] - * Specify an image target within the ubuntu-cpc project for a build + * Specify an image target within the ubuntu-cpc project for a build - * Observe artifacts in addition to the ones for the named image target + * Observe artifacts in addition to the ones for the named image target are produced and downloaded + [Regression Potential] - [Regression Potential] - - * An artifact from an explicitly named artifact is missing and builds + * An artifact from an explicitly named artifact is missing and builds will begin to remove desired output. Testing is being done to address this regression. [Other Info] - - * This has been running in production in Eoan for a few weeks and we're satisfied with the results. + + * This has been running in production in Eoan for a few weeks and we're + satisfied with the results. We have seen overall time reduced by 20-50% + on some builds. [Original Description] In parallel builds where a list of image targets are provided, the build may produce binaries that are not part of the named set of targets but are created by series dependencies. These implicitly created binaries may be generated by multiple builds but are unused as our convention for the ubuntu-cpc project is to only consume binaries from the explicitly named image targets; this avoids overwriting the same object by multiple parallel builds. The problem is that as we build out parallel jobs for a number of image targets (multiplied by each supported architecture) we're wasting a lot of resources. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
This bug was fixed in the package livecd-rootfs - 2.603 --- livecd-rootfs (2.603) eoan; urgency=medium [ Tobias Koch ] * Add retry logic to snap-tool to make downloads more resilient [ Robert C Jennings ] * ubuntu-cpc: Only produce explicitly specified artifacts (LP: #1837254) -- Tobias Koch Mon, 15 Jul 2019 17:59:06 +0200 ** Changed in: livecd-rootfs (Ubuntu) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
** Tags added: id-5d31721934a49770eea1ff43 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
** Description changed: - In parallel builds where a list of image targets are provided the build + In parallel builds where a list of image targets are provided, the build may produce binaries that are not part of the named set of targets but are created by series dependencies. These implicitly created binaries may be generated by multiple builds but are unused as our convention for the ubuntu-cpc project is to only consume binaries from the explicitly named image targets; this avoid overwriting the same object by multiple parallel builds. ** Description changed: In parallel builds where a list of image targets are provided, the build may produce binaries that are not part of the named set of targets but are created by series dependencies. These implicitly created binaries may be generated by multiple builds but are unused as our convention for the ubuntu-cpc project is to only consume binaries from the explicitly - named image targets; this avoid overwriting the same object by multiple - parallel builds. + named image targets; this avoids overwriting the same object by multiple + parallel builds. The problem is that as we build out parallel jobs for + a number of image targets (multiplied by each supported architecture) + we're wasting a lot of resources. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
I have attached an MP. In testing with the ubuntu-cpc project on eoan, when building the amd64 qcow2 image target we eliminate 1.38GiB in unnecessary transfer and storage. This saves a lot of time, storage, and transit for each parallel build on each architecture (so it adds up fast). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837254] Re: ubuntu-cpc parallel builds produce unused files
** Merge proposal linked: https://code.launchpad.net/~rcj/livecd-rootfs/+git/livecd-rootfs/+merge/370361 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1837254 Title: ubuntu-cpc parallel builds produce unused files To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1837254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
