commit: 333ca73072b5f3628a2913c136c5c5b52a371bf5
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Sat Nov 29 00:54:25 2025 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Mon Dec 29 13:41:42 2025 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=333ca730
Implement FEATURES=packdebug
Create a tarball of debug information and source files for use with
binary packages and debuginfod.
The tarball is owned by the package, but we add it to UNINSTALL_IGNORE
so it is kept around on upgrades etc., and we add it to PKG_INSTALL_MASK
so that it is not part of binpkgs themselves (should be fetched via debuginfod).
That is, for FEATURES=packdebug, we:
* Create a tarball in
/usr/lib/debug/.tarball/${CATEGORY}/${PN}/${PF}-${BUILD_ID}-debug.tar.xz
with debug information from /usr/lib/debug/*, /usr/src/debug/*. This
is installed but not part of the binpkg.
* Set UNINSTALL_IGNORE="/usr/lib/debug/.tarball" to keep old
debug information tarballs around to serve.
* Set COLLISION_IGNORE="/usr/lib/debug/.tarball" to ignore collisions from
such orphaned debug information tarballs left around by UNINSTALL_IGNORE.
* Set PKG_INSTALL_MASK="/usr/lib/debug/* /usr/src/debug/*" so binpkgs
do not contain debug information (they should fetch it via debuginfod
instead).
We also prune empty directories left under /usr/lib and /usr/src (from /debug,
as nothing may be left after moving things out for packdebug). This is OK as
it's undefined behaviour in the specification.
The model is as follows:
* binhost
Builds packages with -g* in *FLAGS and has FEATURES=splitdebug
(and possibly FEATURES=installsources) to get debuginfo. Must also
have build IDs enabled in the compiler.
Uses FEATURES=packdebug to create tarballs containing that info.
Runs `debuginfod ... -Z tar.xz \
/usr/lib/debug/.tarball \
-Z.gpkg.tar='(bsdtar -xf- -O '*/image.tar.*' | bsdtar --strip-components
1 -cf- @-)<' \
/var/cache/binpkgs`
to serve the tarballs generated by packdebug, as well as binpkgs themselves
for clients to request full binaries corresponding to coredumps etc.
Users may want to set INSTALL_MASK in make.conf on the binhost side to
avoid duplicate copies of debug information on the live filesystem
(/usr/lib/debug, /usr/src/debug) and in the packdebug tarballs:
```
INSTALL_MASK="${INSTALL_MASK} /usr/lib/debug/ /usr/src/debug/
-/usr/lib/debug/.tarball"
```
(That is not done by default because one may wish to have non-debuginfod-
aware software work, and it also feels like it's a bit surprising for us
to do automatically.)
* binpkg client
Sets the `DEBUGINFOD_URLS` environment variable to the httpd that
`debuginfod` runs.
Thanks to Arsen for the original bashrc-based implementation which
the estrip impl. is based upon, motivation for getting this done, and
the idea. See also https://wiki.gentoo.org/wiki/User:Arsen/Deguginfod. Thanks
also to Eli for the discussions.
This is also really useful for not just serving binpkgs to clients and
letting them get debuginfo (of course the main usecase), but also getting
debuginfo for local binaries you don't even publish anywhere after an upgrade
when an old process is still running. For this part to work well, you need
the binaries themselves too which the above `debuginfod` invocation covers.
Note that xz is used for the debug tarball because it supports [0][1] random
access.
[0]
https://blog.osandov.com/2024/07/25/making-debuginfod-viable-for-the-linux-kernel.html
[1]
https://developers.redhat.com/articles/2025/01/14/debuginfod-project-update-2024#addressing_kernel_vdso_extraction_bottlenecks
Bug: https://sourceware.org/PR33693
Bug: https://bugs.gentoo.org/639376
Bug: https://bugs.gentoo.org/728818
Bug: https://bugs.gentoo.org/953869
Co-authored-by: Arsen Arsenović <arsen <AT> gentoo.org>
Thanks-to: Eli Schwartz <eschwartz <AT> gentoo.org>
Signed-off-by: Sam James <sam <AT> gentoo.org>
NEWS | 7 +++++++
bin/misc-functions.sh | 49 ++++++++++++++++++++++++++++++++++++++++++++
lib/_emerge/EbuildPhase.py | 1 +
lib/_emerge/PackagePhase.py | 11 ++++++++--
lib/portage/const.py | 1 +
lib/portage/dbapi/vartree.py | 18 +++++++++++++++-
man/make.conf.5 | 28 +++++++++++++++++++++++++
7 files changed, 112 insertions(+), 3 deletions(-)
diff --git a/NEWS b/NEWS
index 5e082d0540..5d20300d17 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,13 @@ Features:
Support is still experimental and its semantics may change.
+* Introduce support for seperate debuginfo tarballs via FEATURES="packdebug"
+ (bug #728818).
+
+ This produces tarballs under /usr/lib/debug/.tarball which debuginfod
+ (from dev-libs/elfutils) can index and serve to clients. Requires
+ build IDs to be enabled in the compiler.
+
* profiles: Support 'profile-license' extension for package.license files.
* Monitor free space in PORTAGE_TMPDIR (bug #934382).
diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index 22531d98b1..cf287933da 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -502,6 +502,37 @@ preinst_selinux_labels() {
fi
}
+# Generate a separate tarball with debug information (and sources) for
+# use with debuginfod.
+__generate_packdebug() {
+ local debugpath="${T}"/.tarball
+ if ! [[ -d "${ED}"/usr/src/debug || -d "${ED}"/usr/lib/debug ]]; then
+ return
+ fi
+
+ install -d "${debugpath}"/"${CATEGORY}"{,/"${PN}"} \
+ || die "Failed to generate target debug directory"
+
+ (
+ unset IFS
+ local
tarfile="${debugpath}/${CATEGORY}/${PN}/${PF}-${BUILD_ID}-debug.tar.xz"
+ cd "${ED}" || die
+ tar -cJf - > "${tarfile}" \
+ $([[ -d ./usr/src/debug ]] && echo ./usr/src/debug) \
+ $([[ -d ./usr/lib/debug ]] && echo ./usr/lib/debug) \
+ || die "Failed to pack up debug info for FEATURES=packdebug"
+ )
+
+ # The package may not have any splitdebug info available, but still
+ # have sources.
+ mkdir -p
"${PORTAGE_TMPDIR}"/portage/${CATEGORY}/${PF}/image/${EPREFIX}/usr/lib/debug ||
die
+ # We don't use ${D} here because __generate_packdebug is called from
+ # __dyn_package where ${D} points to a pretend ${D}. We want these files
+ # in the real image but not in the binpkg. Unfortunately, we can't
+ # easily leverage PKG_INSTALL_MASK because of when it runs.
+ mv "${debugpath}"
"${PORTAGE_BUILDDIR}/image/${EPREFIX}/usr/lib/debug/". || die
+}
+
__dyn_package() {
if ! ___eapi_has_prefix_variables; then
local EPREFIX=
@@ -524,6 +555,24 @@ __dyn_package() {
if [[ ! -z "${BUILD_ID}" ]]; then
echo -n "${BUILD_ID}" >
"${PORTAGE_BUILDDIR}"/build-info/BUILD_ID
+
+ # We generate the packdebug tarball at this point as we need
+ # the BUILD_ID, but it's not installed as part of the binpkg
+ # by design. We install it later when merging.
+ if contains_word packdebug "${FEATURES}" ; then
+ __generate_packdebug
+
+ # The injected PKG_INSTALL_MASK combined with us
+ # having splitdebug/installsources on may mean we
+ # have an empty /usr/src or /usr/lib left. Prune those.
+ #
+ # XXX: We use ${D}/${EPREFIX} here because we don't set
+ # ${ED} to the fake ${D}.
+ (cd "${D}/${EPREFIX}"/usr && find \
+ ./lib \
+ ./src \
+ -type d -empty -exec rmdir -p {} 2>/dev/null)
+ fi
fi
if [[ "${BINPKG_FORMAT}" == "xpak" ]]; then
diff --git a/lib/_emerge/EbuildPhase.py b/lib/_emerge/EbuildPhase.py
index b0e13698d2..ae43b91d32 100644
--- a/lib/_emerge/EbuildPhase.py
+++ b/lib/_emerge/EbuildPhase.py
@@ -128,6 +128,7 @@ class EbuildPhase(CompositeTask):
"network-sandbox",
"network-sandbox-proxy",
"nostrip",
+ "packdebug",
"preserve-libs",
"sandbox",
"selinux",
diff --git a/lib/_emerge/PackagePhase.py b/lib/_emerge/PackagePhase.py
index b87ae5dc00..6a5c6d42a5 100644
--- a/lib/_emerge/PackagePhase.py
+++ b/lib/_emerge/PackagePhase.py
@@ -32,6 +32,12 @@ class PackagePhase(CompositeTask):
_shell_binary = portage.const.BASH_BINARY
def _start(self):
+ packdebug_maskstr = ""
+ if "packdebug" in self.settings.features:
+ # We don't want to include debug information in binpkgs themselves
+ # w/ packdebug as binpkg consumers should fetch them via
debuginfod.
+ packdebug_maskstr += " /usr/src/debug/ /usr/lib/debug/"
+
try:
with open(
_unicode_encode(
@@ -46,9 +52,10 @@ class PackagePhase(CompositeTask):
encoding=_encodings["repo.content"],
errors="replace",
) as f:
- self._pkg_install_mask = InstallMask(f.read())
+ self._pkg_install_mask = InstallMask(f.read() +
packdebug_maskstr)
except OSError:
- self._pkg_install_mask = None
+ self._pkg_install_mask = InstallMask(packdebug_maskstr)
+
if self._pkg_install_mask:
self._proot = os.path.join(self.settings["T"], "packaging")
self._start_task(
diff --git a/lib/portage/const.py b/lib/portage/const.py
index 900c30767c..31de3b99cb 100644
--- a/lib/portage/const.py
+++ b/lib/portage/const.py
@@ -213,6 +213,7 @@ SUPPORTED_FEATURES = frozenset(
"noman",
"nostrip",
"notitles",
+ "packdebug",
"parallel-fetch",
"parallel-install",
"pid-sandbox",
diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
index abf7710acb..2239fba271 100644
--- a/lib/portage/dbapi/vartree.py
+++ b/lib/portage/dbapi/vartree.py
@@ -2750,7 +2750,14 @@ class dblink:
# process symlinks second-to-last, directories last.
mydirs = set()
- uninstall_ignore =
shlex.split(self.settings.get("UNINSTALL_IGNORE", ""))
+ uninstall_ignore = self.settings.get("UNINSTALL_IGNORE", "")
+ if "packdebug" in self.settings.features:
+ # For packdebug, we don't want to clean up old debuginfo
+ # tarballs that we made, as we want to keep serving them
+ # to binhost clients for some time.
+ uninstall_ignore += " /usr/lib/debug/.tarball/*"
+
+ uninstall_ignore = shlex.split(uninstall_ignore)
def unlink(file_name, lstatobj):
if bsd_chflags:
@@ -3940,6 +3947,15 @@ class dblink:
x += "/*"
collision_ignore.append(x)
+ # packdebug tarballs are orphaned for management outside of Portage,
+ # so it's expected that we may collide with an orphaned file. Ignore
it.
+ if "packdebug" in self.settings.features:
+ x = "/usr/lib/debug/.tarball"
+ if os.path.isdir(os.path.join(self._eroot, x.lstrip(os.sep))):
+ x = normalize_path(x)
+ x += "/*"
+ collision_ignore.append(x)
+
# For collisions with preserved libraries, the current package
# will assume ownership and the libraries will be unregistered.
if self.vartree.dbapi._plib_registry is None:
diff --git a/man/make.conf.5 b/man/make.conf.5
index f2d6f0e001..813577ff08 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -670,6 +670,34 @@ Prevents the stripping of binaries that are merged to the
live filesystem.
.B notitles
Disables xterm titlebar updates (which contains status info).
.TP
+.B packdebug
+Create a tarball of debug information and source files for use with
+debuginfod. Debug tarballs are placed at
+\fI/usr/lib/debug/.tarball/${CATEGORY}/${PF}-${BUILD_ID}.tar.xz\fR.
+The debug tarball is installed but is not included in binary packages.
+
+\fIUNINSTALL_IGNORE\fR is automatically modified to leave old debug
+tarballs around to allow debuginfod to serve them after upgrades. See
+\fBsplitdebug\fR for general split debug information (upon which this
+feature depends). Similarly, \fICOLLISION_IGNORE\fR is automatically
+modified to ignore collisions on these files.
+
+Setting \fBpackdebug\fR temporarily on the command-line or per\-package is
+discouraged because the new (unset) value will take effect when the package
+is removed, meaning the automatic adjustments made to accommodate
+\fBpackdebug\fR cannot apply.
+
+Users may wish to avoid double storage of both the live debug information
+in \fI/usr/lib/debug\fR and \fI/usr/src/debug\fR as well as inside of
+the \fIpackdebug\fR tarballs at \fI/usr/lib/debug/.tarball\fR. This can
+be done by setting \fIINSTALL_MASK\fR:
+
+.nf
+# Debug information is inside of packdebug tarballs already, don't install
+# twice.
+INSTALL_MASK="${INSTALL_MASK} /usr/lib/debug/ /usr/src/debug/
-/usr/lib/debug/.tarball"
+.fi
+.TP
.B parallel\-fetch
Fetch in the background while compiling. Run
`tail \-f /var/log/emerge\-fetch.log` in a
