On Mon, 09 Mar 2020 at 10:09:46 +0100, Guillem Jover wrote: > We'd like to standardize on a new set of artifact build pathnames > for our deb toolchain. These would have the following form: > > - debian/.build/upstream* > > These could be used for out-of-tree builds replacing current > build-tree/ and similar directories.
Presumably the '*' means it's considered OK for packages that do separate production/debug/udeb builds to use multiple directories matching that glob? For example, a simple debhelper build might eventually (in a future compat level) default to "dh $@ --builddirectory=debian/.build/upstream", but because src:glib2.0 does separate deb and udeb builds (with non-essential features disabled in the udeb), it would use "--builddirectory=debian/.build/upstream-deb" and "--builddirectory=debian/.build/upstream-udeb" instead of its current "--builddirectory=debian/build/deb" and "--builddirectory=debian/build/udeb"? > - We could just (say) .gitignore a single pathname. > - Cleaning the Debian artifacts would imply doing just > «rm -rf debian/.build/» (even though this could be abstracted via a > new dpkg-<something>clean<something> command). > - Namespacing would avoid any collisions. > - Would make parallelization way easier and faster. > - dpkg-dev tools could start defaulting to use package specfific > directories, instead of requiring the user/tool to specify them. This all seems good to have. The Salsa-CI pipeline (which runs in a git clone of a package, and currently uses debian/output/ for intermediate build stuff) could maybe also move its scratch directory to debian/.build/salsa-ci. Have you seen <https://bugs.launchpad.net/launchpad/+bug/1845159> and related discussions in the context of sbuild and pbuilder? It seems like a related topic. Briefly, there's a desire to be able to have "artifacts" that are not part of the main build result, are produced by both successful and failing builds, are kept for a while by the buildd infrastructure, and are probably not uploaded to the Debian archive, as a generalization of the build log. For example, Autotools packages have ${builddir}/config.log, which is not a useful thing to publish in a .deb (not at all reproducible!), but can be useful to diff between successful and failing builds to debug why a build failed. At the moment there's a special case in debhelper to cat config.log if configure fails, but it is forgotten about after the build if configure succeeds. Meson has ${builddir}/meson-logs/meson-log.txt, which is analogous. Similarly, Autotools packages normally output a test summary on success or a more detailed log on failure (${builddir}/**/test-suite.log describes all tests that failed, were partially or entirely skipped, or were otherwise not 100% successful, and there are also detailed logs that capture diagnostics from even the completely successful tests), and Meson ${builddir}/meson-logs/testlog.txt is analogous. At the moment the only artifact, other than the .changes file and the files it references when the build is successful, is the build log itself, which means the build process must either cat these other artifacts to get them into the build log (resulting in large build logs where it's difficult to find what you're looking for), or lose them forever. Some packages have machine-readable test output that is used to track regressions and intermittent failures between builds, for which cat'ing the test output into the build log is less than ideal: to compare test results between runs it's necessary to screen-scrape the test output from among the unstructured/human-readable messages that make up most of the build log. A few packages work around this by packaging their test results, which usually makes them non-reproducible. autopkgtest's ${AUTOPKGTEST_ARTIFACTS} (a directory into which tests are encouraged to write or copy more detailed results than the ones shown on stdout) and Gitlab-CI's artifacts:paths: parameter (a list of globs matching paths in the built directory) are conceptually similar ideas. For example, dpkg-buildpackage could perhaps read one glob per line from debian/artifacts and hardlink matched files (if any) into debian/.build/artifacts for collection by a "larger" framework like sbuild or pbuilder, or individual packages could copy/link files into there as they go, or debhelper build-system classes like Autotools and Meson could know the names of common log files from their build system, or some combination of those. smcv