On Sat Jan 24, 2026 at 12:54 PM CST, Kefu Chai wrote: > On Fri Jan 23, 2026 at 9:03 PM CST, Fabian Grünbichler wrote: >> On January 23, 2026 8:56 am, Kefu Chai wrote: >>> The dwz tool tries to deduplicate debug information across binaries, >>> but it has limits on the number of DWARF DIE (Debug Information Entries) >>> it can handle. Large C++ binaries especially those using templates >>> heavilily (like Ceph's crimson components), often exceed these limits. >>> >>> When building packages with DWZ enabled, the debian packging fails with: >>> >>> ``` >>> dh_dwz: error: Aborting due to earlier error >>> ``` >>> >>> So let's make ceph-crimson-osd an exception when running dwz. This >>> change will not backported to tentacle as tentacle does not build >>> crimson by default. >> >> FWIW, dh_dwz will be dropped from the default sequence in forky/DH-14, >> so we could also consider nop-ing it entirely (unless that blows up >> binary sizes too much?). >> > > Thanks for sharing! This information is critical for this change. Will > we release tentacle along with forky as the base system? > > I will compare the package sizes with and without dwz enabled, and get > back to you over the mailing list. if sizes difference is minimal or > smallish, I will export DWZ=false to disable it. >
The following table shows the size comparison of debug packages (dbg) compiled from ceph 20.2.0-pve1 with and without DWZ compression: +---------------------------------+--------------+--------------+--------------+------------+ | Package Name | With DMZ (MB) | Without (MB) | Diff (MB) | Ratio | +=================================+==============+==============+==============+============+ | ceph-base-dbg | 151.0 | 181.7 | +30.7 | 16.89% | | ceph-common-dbg | 949.5 | 1091.1 | +141.7 | 12.99% | | ceph-exporter-dbg | 8.3 | 8.6 | +0.3 | 3.94% | | ceph-fuse-dbg | 14.1 | 15.2 | +1.1 | 7.13% | | ceph-immutable-object-cache-dbg | 4.7 | 5.3 | +0.5 | 9.86% | | ceph-mds-dbg | 79.3 | 88.2 | +8.9 | 10.10% | | ceph-mgr-dbg | 33.8 | 36.8 | +2.9 | 7.99% | | ceph-mon-dbg | 194.2 | 220.4 | +26.2 | 11.91% | | ceph-osd-dbg | 3.7 | 3.9 | +0.2 | 4.53% | | ceph-test-dbg | 1488.9 | 1827.4 | +338.5 | 18.52% | | cephfs-mirror-dbg | 8.0 | 9.3 | +1.3 | 14.19% | | libcephfs-daemon-dbg | 0.1 | 0.1 | +0.0 | 2.51% | | libcephfs-proxy2-dbg | 0.1 | 0.1 | +0.0 | 1.43% | | libcephfs2-dbg | 12.7 | 13.5 | +0.8 | 6.02% | | librados2-dbg | 98.4 | 108.4 | +10.0 | 9.22% | | libradosstriper1-dbg | 7.1 | 7.3 | +0.2 | 2.83% | | librbd1-dbg | 113.7 | 150.9 | +37.3 | 24.70% | | librgw2-dbg | 300.4 | 332.9 | +32.5 | 9.75% | | libsqlite3-mod-ceph-dbg | 2.2 | 2.3 | +0.0 | 1.78% | | python3-cephfs-dbg | 1.3 | 1.3 | -0.0 | -0.03% | | python3-rados-dbg | 1.2 | 1.2 | +0.0 | 0.82% | | python3-rbd-dbg | 2.2 | 2.2 | -0.0 | -0.01% | | python3-rgw-dbg | 0.6 | 0.6 | +0.0 | 0.17% | | radosgw-dbg | 584.8 | 646.6 | +61.8 | 9.56% | | rbd-fuse-dbg | 2.1 | 2.2 | +0.1 | 6.22% | | rbd-mirror-dbg | 117.5 | 153.7 | +36.2 | 23.56% | | rbd-nbd-dbg | 3.4 | 3.6 | +0.2 | 4.28% | +---------------------------------+--------------+--------------+--------------+------------+ | TOTAL | 4183.2 | 4914.8 | +731.5 | 14.88% | +---------------------------------+--------------+--------------+--------------+------------+ While a 14.88% reduction may seem modest, it translates to over 700 MB saved per build. Not sure if we care about this overhead though.. Please let me know what do you think. I will update the patch accordingly. >>> >>> Signed-off-by: Kefu Chai <[email protected]> >>> --- >>> ...lude-ceph-osd-crimson-from-dwz-compr.patch | 52 +++++++++++++++++++ >>> patches/series | 1 + >>> 2 files changed, 53 insertions(+) >>> create mode 100644 >>> patches/0053-debian-rules-exclude-ceph-osd-crimson-from-dwz-compr.patch >>> >>> diff --git >>> a/patches/0053-debian-rules-exclude-ceph-osd-crimson-from-dwz-compr.patch >>> b/patches/0053-debian-rules-exclude-ceph-osd-crimson-from-dwz-compr.patch >>> new file mode 100644 >>> index 00000000000..c5f477adb30 >>> --- /dev/null >>> +++ >>> b/patches/0053-debian-rules-exclude-ceph-osd-crimson-from-dwz-compr.patch >>> @@ -0,0 +1,52 @@ >>> +From 126f3623e3c36bf0bd233b01fd452aeec0415d89 Mon Sep 17 00:00:00 2001 >>> +From: Kefu Chai <[email protected]> >>> +Date: Fri, 23 Jan 2026 15:04:47 +0800 >>> +Subject: [PATCH] debian/rules: exclude ceph-osd-crimson from dwz >>> compression >>> + >>> +When building with DWZ enabled, the debian packaging fails with: >>> +``` >>> + dh_dwz: error: Aborting due to earlier error >>> +``` >>> +Running the dwz command manually reveals the root cause: >>> +``` >>> + $ dwz >>> -mdebian/ceph-osd-crimson/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug >>> \ >>> + -M/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug -- \ >>> + debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson \ >>> + debian/ceph-osd-crimson/usr/bin/crimson-store-nbd >>> + dwz: debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson: Too many DIEs, >>> not optimizing >>> + dwz: Too few files for multifile optimization >>> +``` >>> +The dwz tool has a limit on the number of DWARF DIEs (Debug Information >>> +Entries) it can process. The ceph-osd-crimson binary, being a large C++ >>> +executable with extensive template usage, exceeds this limit, causing >>> +dwz to exit with status 1 and fail the build. >>> + >>> +This change excludes only ceph-osd-crimson from dwz processing using >>> +the -X flag, allowing other binaries in the package to still benefit >>> +from DWARF compression while avoiding the build failure. >>> + >>> +Please note, we always disable DWZ in ceph-build's ceph-dev-pipeline. >>> + >>> +Signed-off-by: Kefu Chai <[email protected]> >>> +--- >>> + debian/rules | 4 ++++ >>> + 1 file changed, 4 insertions(+) >>> + >>> +diff --git a/debian/rules b/debian/rules >>> +index 4d31ffd069f..9082c9c4dd2 100755 >>> +--- a/debian/rules >>> ++++ b/debian/rules >>> +@@ -53,6 +53,10 @@ endif >>> + >>> + ifeq ($(DWZ), false) >>> + override_dh_dwz: >>> ++else >>> ++override_dh_dwz: >>> ++ # Exclude ceph-osd-crimson due to excessive debug info (too many DIEs) >>> ++ dh_dwz -Xceph-osd-crimson >>> + endif >>> + >>> + # for python3-${pkg} packages >>> +-- >>> +2.47.3 >>> + >>> diff --git a/patches/series b/patches/series >>> index 53e0e6b83ca..874bef96aa4 100644 >>> --- a/patches/series >>> +++ b/patches/series >>> @@ -50,3 +50,4 @@ >>> 0050-debian-rules-enable-WITH_CRIMSON-when-pkg.ceph.crims.patch >>> 0051-debian-ceph.spec-fix-ceph-osd-upgrade-conflicts.patch >>> 0052-cmake-build-static-seastar-for-release-builds.patch >>> +0053-debian-rules-exclude-ceph-osd-crimson-from-dwz-compr.patch >>> -- >>> 2.47.3 >>> >>> >>> >>> _______________________________________________ >>> pve-devel mailing list >>> [email protected] >>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >>> >>> >>> >> >> >> _______________________________________________ >> pve-devel mailing list >> [email protected] >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel > > > > _______________________________________________ > pve-devel mailing list > [email protected] > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel _______________________________________________ pve-devel mailing list [email protected] https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
