On January 24, 2026 5:13 am, Kefu Chai wrote: > On Fri Jan 23, 2026 at 9:03 PM CST, Fabian Grünbichler wrote: >> one small question inline, otherwise this one LGTM >> >> On January 23, 2026 8:56 am, Kefu Chai wrote: >>> Previously, cephadm's build process pulled dependencies from PyPI, >>> which failed in environments with restricted network access. This >>> change backports upstream improvements that enable building cephadm >>> using system packages instead, mirroring the existing RPM packaging >>> workflow. >>> >>> And the vendored Arrow pulled xsimd from GitHub, this also failed in >>> building environment without network access. This change backports >>> upstream improvements enabling the build to use libxsimd-dev package >>> when building debian packages. >>> >>> Backported changes: >>> - https://github.com/ceph/ceph/pull/66256 - Use system packages for >>> cephadm bundled dependencies >>> - https://github.com/ceph/ceph/pull/66248 - Add libxsimd-dev build >>> dependency for vendored Arrow >>> - https://github.com/ceph/ceph/pull/65292 - cephadm: fix building >>> rpm-sourced cephadm zippapp on el10 >>> >>> Note: Test commits from these PRs are excluded from this backport. >>> >>> Signed-off-by: Kefu Chai <[email protected]> >>> --- >>> ...-issues-running-existing-cephadm-bui.patch | 66 +++++ >>> ...root_entries-population-in-version-c.patch | 62 +++++ >>> ...cephadm-rpm-based-builds-without-top.patch | 89 ++++++ >>> ...d-Debian-package-support-for-bundled.patch | 261 ++++++++++++++++++ >>> ...m-packages-for-cephadm-bundled-depen.patch | 54 ++++ >>> ...-Use-AUTO-mode-for-xsimd-dependency-.patch | 55 ++++ >>> ...dd-libxsimd-dev-build-dependency-for.patch | 41 +++ >>> patches/series | 8 + >>> 8 files changed, 636 insertions(+) >>> create mode 100644 >>> patches/0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch >>> create mode 100644 >>> patches/0043-cephadm-fix-zip_root_entries-population-in-version-c.patch >>> create mode 100644 >>> patches/0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch >>> create mode 100644 >>> patches/0045-cephadm-build-Add-Debian-package-support-for-bundled.patch >>> create mode 100644 >>> patches/0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch >>> create mode 100644 >>> patches/0047-cmake-BuildArrow-Use-AUTO-mode-for-xsimd-dependency-.patch >>> create mode 100644 >>> patches/0048-debian-control-Add-libxsimd-dev-build-dependency-for.patch >>> >>> diff --git >>> a/patches/0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch >>> b/patches/0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch >>> new file mode 100644 >>> index 00000000000..c6208b0db2a >>> --- /dev/null >>> +++ >>> b/patches/0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch >>> @@ -0,0 +1,66 @@ >>> +From 2685f6c35245d033e73e16ece3e356fcead44cea Mon Sep 17 00:00:00 2001 >>> +From: John Mulligan <[email protected]> >>> +Date: Thu, 21 Aug 2025 12:30:55 -0400 >>> +Subject: [PATCH 42/46] cephadm: fix some issues running existing cephadm >>> build >>> + tests >>> + >>> +As time has marched on and people changed things our tests no longer >>> +match the expected inputs. >>> + >>> +Signed-off-by: John Mulligan <[email protected]> >>> +(cherry picked from commit 31c8010faa417ca53614bd30379a9b9c0c9199de) >>> +--- >>> + src/cephadm/tests/build/test_cephadm_build.py | 18 ++++++++++++------ >>> + 1 file changed, 12 insertions(+), 6 deletions(-) >>> + >>> +diff --git a/src/cephadm/tests/build/test_cephadm_build.py >>> b/src/cephadm/tests/build/test_cephadm_build.py >>> +index c2995a76d4b..9beef67ffe0 100644 >>> +--- a/src/cephadm/tests/build/test_cephadm_build.py >>> ++++ b/src/cephadm/tests/build/test_cephadm_build.py >>> +@@ -34,12 +34,12 @@ CONTAINERS = { >>> + }, >>> + 'ubuntu-20.04': { >>> + 'name': 'cephadm-build-test:ubuntu-20-04-py3', >>> +- 'base_image': 'quay.io/library/ubuntu:20.04', >>> ++ 'base_image': 'docker.io/library/ubuntu:20.04', >>> + 'script': 'apt update && apt install -y python3-venv', >>> + }, >>> + 'ubuntu-22.04': { >>> + 'name': 'cephadm-build-test:ubuntu-22-04-py3', >>> +- 'base_image': 'quay.io/library/ubuntu:22.04', >>> ++ 'base_image': 'docker.io/library/ubuntu:22.04', >>> + 'script': 'apt update && apt install -y python3-venv', >>> + }, >>> + } >>> +@@ -128,8 +128,8 @@ def test_cephadm_build(env, source_dir, tmp_path): >>> + assert all('requirements_entry' in v for v in >>> data['bundled_packages']) >>> + assert 'zip_root_entries' in data >>> + zre = data['zip_root_entries'] >>> +- assert any(e.startswith('Jinja2') for e in zre) >>> +- assert any(e.startswith('MarkupSafe') for e in zre) >>> ++ assert any(_dist_info(e, 'Jinja2') for e in zre) >>> ++ assert any(_dist_info(e, 'MarkupSafe') for e in zre) >>> + assert any(e.startswith('jinja2') for e in zre) >>> + assert any(e.startswith('markupsafe') for e in zre) >>> + assert any(e.startswith('cephadmlib') for e in zre) >>> +@@ -184,9 +184,15 @@ def test_cephadm_build_from_rpms(env, source_dir, >>> tmp_path): >>> + assert all('requirements_entry' in v for v in >>> data['bundled_packages']) >>> + assert 'zip_root_entries' in data >>> + zre = data['zip_root_entries'] >>> +- assert any(e.startswith('Jinja2') for e in zre) >>> +- assert any(e.startswith('MarkupSafe') for e in zre) >>> ++ assert any(_dist_info(e, 'Jinja2') for e in zre) >>> ++ assert any(_dist_info(e, 'MarkupSafe') for e in zre) >>> + assert any(e.startswith('jinja2') for e in zre) >>> + assert any(e.startswith('markupsafe') for e in zre) >>> + assert any(e.startswith('cephadmlib') for e in zre) >>> + assert any(e.startswith('_cephadmmeta') for e in zre) >>> ++ >>> ++ >>> ++def _dist_info(entry, name): >>> ++ return ( >>> ++ entry.startswith(entry) or entry.startswith(entry.lower()) >>> ++ ) and (entry.endswith('.dist-info') or entry.endswith('.egg-info')) >>> +-- >>> +2.47.3 >>> + >>> diff --git >>> a/patches/0043-cephadm-fix-zip_root_entries-population-in-version-c.patch >>> b/patches/0043-cephadm-fix-zip_root_entries-population-in-version-c.patch >>> new file mode 100644 >>> index 00000000000..7a0b1a18d68 >>> --- /dev/null >>> +++ >>> b/patches/0043-cephadm-fix-zip_root_entries-population-in-version-c.patch >>> @@ -0,0 +1,62 @@ >>> +From 2c20ae95f32a2bdf78576252f1c9171f00a2790e Mon Sep 17 00:00:00 2001 >>> +From: Kefu Chai <[email protected]> >>> +Date: Mon, 10 Nov 2025 12:11:08 +0800 >>> +Subject: [PATCH 43/46] cephadm: fix zip_root_entries population in version >>> + command >>> + >>> +The 'cephadm version --verbose' command was returning an empty >>> +zip_root_entries list because it relied on the private '_files' >>> +attribute of zipimport.zipimporter, which is not reliably populated >>> +across Python versions. >>> + >>> +This commit fixes the issue by using the zipfile module to properly >>> +read the archive contents via the loader.archive path. This ensures >>> +that zip_root_entries is correctly populated with the root-level >>> +directories in the zipapp. >>> + >>> +This fix is necessary for the cephadm build tests to properly validate >>> +that all expected packages and modules are included in the built zipapp. >>> + >>> +Signed-off-by: Kefu Chai <[email protected]> >>> +(cherry picked from commit 2c68c1496dbb7cd01bf783e31510940445040a34) >>> +--- >>> + src/cephadm/cephadm.py | 16 ++++++++++++---- >>> + 1 file changed, 12 insertions(+), 4 deletions(-) >>> + >>> +diff --git a/src/cephadm/cephadm.py b/src/cephadm/cephadm.py >>> +index f75aaa86dac..8d08c700868 100755 >>> +--- a/src/cephadm/cephadm.py >>> ++++ b/src/cephadm/cephadm.py >>> +@@ -1688,6 +1688,7 @@ def command_version(ctx): >>> + # type: (CephadmContext) -> int >>> + import importlib >>> + import zipimport >>> ++ import zipfile >>> + import types >>> + >>> + vmod: Optional[types.ModuleType] >>> +@@ -1744,10 +1745,17 @@ def command_version(ctx): >>> + out['bundled_packages'] = deps_info >>> + except OSError: >>> + pass >>> +- files = getattr(loader, '_files', {}) >>> +- out['zip_root_entries'] = sorted( >>> +- {p.split('/')[0] for p in files.keys()} >>> +- ) >>> ++ # Use zipfile module to properly read the archive contents >>> ++ # loader.archive contains the path to the zip file >>> ++ try: >>> ++ with zipfile.ZipFile(loader.archive, 'r') as zf: >>> ++ files = zf.namelist() >>> ++ out['zip_root_entries'] = sorted( >>> ++ {p.split('/')[0] for p in files if p} >>> ++ ) >>> ++ except (OSError, zipfile.BadZipFile): >>> ++ # Fallback to empty list if we can't read the zip >>> ++ out['zip_root_entries'] = [] >>> + >>> + json.dump(out, sys.stdout, indent=2) >>> + print() >>> +-- >>> +2.47.3 >>> + >>> diff --git >>> a/patches/0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch >>> b/patches/0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch >>> new file mode 100644 >>> index 00000000000..6e1b4dd2e0e >>> --- /dev/null >>> +++ >>> b/patches/0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch >>> @@ -0,0 +1,89 @@ >>> +From 5ce72178a782dfe2e80648063b025ba23d96d5d8 Mon Sep 17 00:00:00 2001 >>> +From: John Mulligan <[email protected]> >>> +Date: Thu, 21 Aug 2025 12:44:17 -0400 >>> +Subject: [PATCH 44/46] cephadm: support cephadm rpm based builds without >>> + top_level.txt >>> + >>> +Signed-off-by: John Mulligan <[email protected]> >>> +(cherry picked from commit 26a499a8da339d870af193ea964368afbc84c694) >>> +--- >>> + src/cephadm/build.py | 56 +++++++++++++++++++++++++++++++++----------- >>> + 1 file changed, 42 insertions(+), 14 deletions(-) >>> + >>> +diff --git a/src/cephadm/build.py b/src/cephadm/build.py >>> +index 43bc58a4003..01c91fc1680 100755 >>> +--- a/src/cephadm/build.py >>> ++++ b/src/cephadm/build.py >>> +@@ -434,6 +434,47 @@ def _install_rpm_deps(tempdir, config): >>> + return dinfo >>> + >>> + >>> ++def _gather_rpm_package_dirs(paths): >>> ++ # = The easy way = >>> ++ # the top_level.txt file can be used to determine where the python >>> packages >>> ++ # actually are. We need all of those and the meta-data dir (parent of >>> ++ # top_level.txt) to be included in our zipapp >>> ++ top_level = None >>> ++ for path in paths: >>> ++ if path.endswith('top_level.txt'): >>> ++ top_level = pathlib.Path(path) >>> ++ if top_level: >>> ++ meta_dir = top_level.parent >>> ++ pkg_dirs = [ >>> ++ top_level.parent.parent / p >>> ++ for p in top_level.read_text().splitlines() >>> ++ ] >>> ++ return meta_dir, pkg_dirs >>> ++ # = The hard way = >>> ++ # loop through the directories to find the .dist-info dir (containing >>> the >>> ++ # mandatory METADATA file, according to the spec) and once we know the >>> ++ # location of dist info we find the sibling paths from the rpm listing >>> ++ dist_info = None >>> ++ ppaths = [] >>> ++ for path in paths: >>> ++ ppath = pathlib.Path(path) >>> ++ ppaths.append(ppath) >>> ++ if ppath.name == 'METADATA' and >>> ppath.parent.name.endswith('.dist-info'): >>> ++ dist_info = ppath.parent >>> ++ break >>> ++ if not dist_info: >>> ++ raise ValueError('no .dist-info METADATA found') >>> ++ if not dist_info.parent.name == 'site-packages': >>> ++ raise ValueError( >>> ++ 'unexpected parent directory (not site-packages):' >>> ++ f' {dist_info.parent.name}' >>> ++ ) >>> ++ siblings = [ >>> ++ p for p in ppaths if p.parent == dist_info.parent and p != >>> dist_info >>> ++ ] >>> ++ return dist_info, siblings >>> ++ >>> ++ >>> + def _deps_from_rpm(tempdir, config, dinfo, pkg): >>> + # first, figure out what rpm provides a particular python lib >>> + dist = f'python3.{sys.version_info.minor}dist({pkg})'.lower() >>> +@@ -469,20 +510,7 @@ def _deps_from_rpm(tempdir, config, dinfo, pkg): >>> + ['rpm', '-ql', rpmname], check=True, stdout=subprocess.PIPE >>> + ) >>> + paths = [l.decode('utf8') for l in res.stdout.splitlines()] >>> +- # the top_level.txt file can be used to determine where the python >>> packages >>> +- # actually are. We need all of those and the meta-data dir (parent of >>> +- # top_level.txt) to be included in our zipapp >>> +- top_level = None >>> +- for path in paths: >>> +- if path.endswith('top_level.txt'): >>> +- top_level = pathlib.Path(path) >>> +- if not top_level: >>> +- raise ValueError('top_level not found') >>> +- meta_dir = top_level.parent >>> +- pkg_dirs = [ >>> +- top_level.parent.parent / p >>> +- for p in top_level.read_text().splitlines() >>> +- ] >>> ++ meta_dir, pkg_dirs = _gather_rpm_package_dirs(paths) >>> + meta_dest = tempdir / meta_dir.name >>> + log.info(f"Copying {meta_dir} to {meta_dest}") >>> + # copy the meta data directory >>> +-- >>> +2.47.3 >>> + >>> diff --git >>> a/patches/0045-cephadm-build-Add-Debian-package-support-for-bundled.patch >>> b/patches/0045-cephadm-build-Add-Debian-package-support-for-bundled.patch >>> new file mode 100644 >>> index 00000000000..94241490b64 >>> --- /dev/null >>> +++ >>> b/patches/0045-cephadm-build-Add-Debian-package-support-for-bundled.patch >>> @@ -0,0 +1,261 @@ >>> +From a4744feb1457aa7fb3f47b42f29138682cc7d41c Mon Sep 17 00:00:00 2001 >>> +From: Kefu Chai <[email protected]> >>> +Date: Tue, 14 Oct 2025 21:04:42 +0800 >>> +Subject: [PATCH 45/46] cephadm/build: Add Debian package support for >>> bundled >>> + dependencies >>> + >>> +Extends the cephadm build script to support bundling dependencies from >>> +Debian packages in addition to pip and RPM packages. This allows building >>> +cephadm on Debian-based distributions using system packages. >>> + >>> +Key changes: >>> +- Add 'deb' to DependencyMode enum to enable Debian package mode >>> +- Implement _setup_deb() to configure Debian dependency requirements >>> +- Add _install_deb_deps() to orchestrate Debian package installation >>> +- Add _gather_deb_package_dirs() to parse Debian package file listings >>> + and locate Python package directories (handles both site-packages and >>> + dist-packages directories used by Debian) >>> +- Add _deps_from_deb() to extract Python dependencies from installed >>> + Debian packages using dpkg/apt-cache tools >>> +- Fix variable reference bug in _install_deps() (deps.mode -> >>> config.deps_mode) >>> + >>> +The Debian implementation follows a similar pattern to the existing RPM >>> +support, using dpkg-query and dpkg -L to locate installed packages and >>> +their files, with special handling for Debian naming conventions >>> +(e.g., PyYAML -> python3-yaml). >>> + >>> +Signed-off-by: Kefu Chai <[email protected]> >>> +(cherry picked from commit 3ff9b0c24e33debe95a0a0c6b42da30be788871c) >>> +--- >>> + src/cephadm/build.py | 158 +++++++++++++++++++++++++++++++++++++++---- >>> + 1 file changed, 143 insertions(+), 15 deletions(-) >>> + >>> +diff --git a/src/cephadm/build.py b/src/cephadm/build.py >>> +index 01c91fc1680..53d3983de89 100755 >>> +--- a/src/cephadm/build.py >>> ++++ b/src/cephadm/build.py >>> +@@ -154,6 +154,7 @@ class PipEnv(enum.Enum): >>> + class DependencyMode(enum.Enum): >>> + pip = enum.auto() >>> + rpm = enum.auto() >>> ++ deb = enum.auto() >>> + none = enum.auto() >>> + >>> + >>> +@@ -169,6 +170,8 @@ class Config: >>> + self._setup_pip() >>> + elif self.deps_mode == DependencyMode.rpm: >>> + self._setup_rpm() >>> ++ elif self.deps_mode == DependencyMode.deb: >>> ++ self._setup_deb() >>> + >>> + def _setup_pip(self): >>> + if self._maj_min == (3, 6): >>> +@@ -180,6 +183,9 @@ class Config: >>> + def _setup_rpm(self): >>> + self.requirements = [InstallSpec(**v) for v in PY_REQUIREMENTS] >>> + >>> ++ def _setup_deb(self): >>> ++ self.requirements = [InstallSpec(**v) for v in PY_REQUIREMENTS] >>> ++ >>> + >>> + class DependencyInfo: >>> + """Type for tracking bundled dependencies.""" >>> +@@ -333,7 +339,9 @@ def _install_deps(tempdir, config): >>> + return _install_pip_deps(tempdir, config) >>> + if config.deps_mode == DependencyMode.rpm: >>> + return _install_rpm_deps(tempdir, config) >>> +- raise ValueError(f'unexpected deps mode: {deps.mode}') >>> ++ if config.deps_mode == DependencyMode.deb: >>> ++ return _install_deb_deps(tempdir, config) >>> ++ raise ValueError(f'unexpected deps mode: {config.deps_mode}') >>> + >>> + >>> + def _install_pip_deps(tempdir, config): >>> +@@ -434,7 +442,26 @@ def _install_rpm_deps(tempdir, config): >>> + return dinfo >>> + >>> + >>> +-def _gather_rpm_package_dirs(paths): >>> ++def _install_deb_deps(tempdir, config): >>> ++ log.info("Installing dependencies using Debian packages") >>> ++ dinfo = DependencyInfo(config) >>> ++ for pkg in config.requirements: >>> ++ log.info(f"Looking for debian package for: {pkg.name!r}") >>> ++ _deps_from_deb(tempdir, config, dinfo, pkg.name) >>> ++ return dinfo >>> ++ >>> ++ >>> ++def _gather_package_dirs(paths, expected_parent_dir): >>> ++ """Parse package file listing to find Python package directories. >>> ++ >>> ++ Args: >>> ++ paths: List of file paths from package listing >>> ++ expected_parent_dir: Expected parent directory name (e.g., >>> 'site-packages' for RPM, >>> ++ 'dist-packages' for Debian) >>> ++ >>> ++ Returns: >>> ++ Tuple of (metadata_dir, package_dirs) >>> ++ """ >>> + # = The easy way = >>> + # the top_level.txt file can be used to determine where the python >>> packages >>> + # actually are. We need all of those and the meta-data dir (parent of >>> +@@ -453,7 +480,7 @@ def _gather_rpm_package_dirs(paths): >>> + # = The hard way = >>> + # loop through the directories to find the .dist-info dir (containing >>> the >>> + # mandatory METADATA file, according to the spec) and once we know the >>> +- # location of dist info we find the sibling paths from the rpm listing >>> ++ # location of dist info we find the sibling paths from the package >>> listing >>> + dist_info = None >>> + ppaths = [] >>> + for path in paths: >>> +@@ -464,9 +491,9 @@ def _gather_rpm_package_dirs(paths): >>> + break >>> + if not dist_info: >>> + raise ValueError('no .dist-info METADATA found') >>> +- if not dist_info.parent.name == 'site-packages': >>> ++ if dist_info.parent.name != expected_parent_dir: >>> + raise ValueError( >>> +- 'unexpected parent directory (not site-packages):' >>> ++ f'unexpected parent directory (not {expected_parent_dir}):' >>> + f' {dist_info.parent.name}' >>> + ) >>> + siblings = [ >>> +@@ -475,6 +502,31 @@ def _gather_rpm_package_dirs(paths): >>> + return dist_info, siblings >>> + >>> + >>> ++def _copy_package_files(tempdir, paths, expected_parent_dir): >>> ++ """Copy package files to the build directory. >>> ++ >>> ++ Args: >>> ++ tempdir: Temporary directory to copy files to >>> ++ paths: List of file paths from package listing >>> ++ expected_parent_dir: Expected parent directory name per packaging >>> convention: >>> ++ - 'site-packages' for RPM-based distributions >>> ++ - 'dist-packages' for Debian-based distributions >>> ++ >>> ++ Returns: >>> ++ None >>> ++ """ >>> ++ meta_dir, pkg_dirs = _gather_package_dirs(paths, expected_parent_dir) >>> ++ meta_dest = tempdir / meta_dir.name >>> ++ log.info(f"Copying {meta_dir} to {meta_dest}") >>> ++ # copy the meta data directory >>> ++ shutil.copytree(meta_dir, meta_dest, ignore=_ignore_cephadmlib) >>> ++ # copy all the package directories >>> ++ for pkg_dir in pkg_dirs: >>> ++ pkg_dest = tempdir / pkg_dir.name >>> ++ log.info(f"Copying {pkg_dir} to {pkg_dest}") >>> ++ shutil.copytree(pkg_dir, pkg_dest, ignore=_ignore_cephadmlib) >>> ++ >>> ++ >>> + def _deps_from_rpm(tempdir, config, dinfo, pkg): >>> + # first, figure out what rpm provides a particular python lib >>> + dist = f'python3.{sys.version_info.minor}dist({pkg})'.lower() >>> +@@ -510,16 +562,92 @@ def _deps_from_rpm(tempdir, config, dinfo, pkg): >>> + ['rpm', '-ql', rpmname], check=True, stdout=subprocess.PIPE >>> + ) >>> + paths = [l.decode('utf8') for l in res.stdout.splitlines()] >>> +- meta_dir, pkg_dirs = _gather_rpm_package_dirs(paths) >>> +- meta_dest = tempdir / meta_dir.name >>> +- log.info(f"Copying {meta_dir} to {meta_dest}") >>> +- # copy the meta data directory >>> +- shutil.copytree(meta_dir, meta_dest, ignore=_ignore_cephadmlib) >>> +- # copy all the package directories >>> +- for pkg_dir in pkg_dirs: >>> +- pkg_dest = tempdir / pkg_dir.name >>> +- log.info(f"Copying {pkg_dir} to {pkg_dest}") >>> +- shutil.copytree(pkg_dir, pkg_dest, ignore=_ignore_cephadmlib) >>> ++ # RPM-based distributions use 'site-packages' for Python packages >>> ++ _copy_package_files(tempdir, paths, 'site-packages') >>> ++ >>> ++ >>> ++def _deps_from_deb(tempdir, config, dinfo, pkg): >>> ++ """Extract Python dependencies from Debian packages. >>> ++ >>> ++ Args: >>> ++ tempdir: Temporary directory to copy package files to >>> ++ config: Build configuration >>> ++ dinfo: DependencyInfo instance to track dependencies >>> ++ pkg: Python package name (e.g., 'MarkupSafe', 'Jinja2', 'PyYAML') >>> ++ """ >>> ++ # Convert Python package name to Debian package name >>> ++ # Python packages are typically named python3-<lowercase-name> >>> ++ # Handle special cases: PyYAML -> python3-yaml, MarkupSafe -> >>> python3-markupsafe >>> ++ pkg_lower = pkg.lower() >>> ++ if pkg_lower == 'pyyaml': >>> ++ deb_pkg_name = 'python3-yaml' >>> ++ else: >>> ++ deb_pkg_name = f'python3-{pkg_lower}' >>> ++ >>> ++ # First, try to find the package using apt-cache >>> ++ # This helps verify the package exists before trying to list its files >>> ++ try: >>> ++ res = subprocess.run( >>> ++ ['apt-cache', 'show', deb_pkg_name], >>> ++ check=True, >>> ++ stdout=subprocess.PIPE, >>> ++ stderr=subprocess.PIPE, >>> ++ ) >>> ++ except subprocess.CalledProcessError: >>> ++ # Package not found, try alternative naming >>> ++ log.warning(f"Package {deb_pkg_name} not found via apt-cache, >>> trying dpkg -S") >>> ++ # Try to search for files that might belong to this package >>> ++ # Search for the Python module in site-packages >>> ++ search_pattern = f'/usr/lib/python3*/dist-packages/{pkg.lower()}*' >>> ++ try: >>> ++ res = subprocess.run( >>> ++ ['dpkg', '-S', search_pattern], >>> ++ check=True, >>> ++ stdout=subprocess.PIPE, >>> ++ stderr=subprocess.PIPE, >>> ++ ) >>> ++ # dpkg -S output format: "package: /path/to/file" >>> ++ deb_pkg_name = res.stdout.decode('utf8').split(':')[0].strip() >>> ++ except subprocess.CalledProcessError as err: >>> ++ log.error(f"Could not find Debian package for {pkg}") >>> ++ log.error(f"Tried: {deb_pkg_name} and pattern search") >>> ++ sys.exit(1) >>> ++ >>> ++ # Get version information using dpkg-query >>> ++ try: >>> ++ res = subprocess.run( >>> ++ ['dpkg-query', '-W', '-f=${Version}\\n', deb_pkg_name], >>> ++ check=True, >>> ++ stdout=subprocess.PIPE, >>> ++ ) >>> ++ version = res.stdout.decode('utf8').strip() >>> ++ except subprocess.CalledProcessError as err: >>> ++ log.error(f"Could not query version for package {deb_pkg_name}: >>> {err}") >>> ++ sys.exit(1) >>> ++ >>> ++ log.info(f"Debian Package: {deb_pkg_name} (version: {version})") >>> ++ dinfo.add( >>> ++ pkg, >>> ++ deb_name=deb_pkg_name, >>> ++ version=version, >>> ++ package_source='deb', >>> ++ ) >>> ++ >>> ++ # Get the list of files provided by the Debian package >>> ++ try: >>> ++ res = subprocess.run( >>> ++ ['dpkg', '-L', deb_pkg_name], >>> ++ check=True, >>> ++ stdout=subprocess.PIPE, >>> ++ ) >>> ++ except subprocess.CalledProcessError as err: >>> ++ log.error(f"Could not list files for package {deb_pkg_name}: >>> {err}") >>> ++ sys.exit(1) >>> ++ >>> ++ paths = [l.decode('utf8') for l in res.stdout.splitlines()] >>> ++ # Debian-based distributions use 'dist-packages' for system-managed >>> Python packages >>> ++ # per Debian Python Policy: >>> https://www.debian.org/doc/packaging-manuals/python-policy/ >>> ++ _copy_package_files(tempdir, paths, 'dist-packages') >>> + >>> + >>> + def generate_version_file(versioning_vars, dest): >>> +-- >>> +2.47.3 >>> + >>> diff --git >>> a/patches/0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch >>> b/patches/0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch >>> new file mode 100644 >>> index 00000000000..0b4daa63f79 >>> --- /dev/null >>> +++ >>> b/patches/0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch >>> @@ -0,0 +1,54 @@ >>> +From 779c9251669b65b7ed77a841f281e2d957174db2 Mon Sep 17 00:00:00 2001 >>> +From: Kefu Chai <[email protected]> >>> +Date: Tue, 21 Oct 2025 11:25:00 +0800 >>> +Subject: [PATCH 46/46] debian: Use system packages for cephadm bundled >>> + dependencies >>> + >>> +Configure the Debian build to use CEPHADM_BUNDLED_DEPENDENCIES=deb, >>> +which instructs the cephadm build script to bundle dependencies from >>> +system-installed Debian packages instead of downloading from PyPI. >>> + >>> +This change addresses build failures in restricted network environments >>> +where Debian build tools do not permit internet access. By leveraging >>> +the Debian package support added in commit 9378a2988e1, the build now >>> +uses python3-markupsafe, python3-jinja2, and python3-yaml packages >>> +that are already installed as build dependencies. >>> + >>> +This approach mirrors the existing RPM packaging workflow, ensuring >>> +consistent behavior across different distribution package formats. >>> + >>> +Signed-off-by: Kefu Chai <[email protected]> >>> +(cherry picked from commit 25680021ee2422f8b8b4075b3ab77af39126eecc) >> >> long term, it might be even better to not bundle at all, but add those >> python deps as runtime dependencies? > > I concur with your concerns, as bundling violates Debian's phylosophy of > avoiding "vendored" dependencies. The distro intends to control and > track all packages. And bunlding also implies more disk space usage and > duplications. > > But it's also an upstream design choice: cephadm was explicitly designed > to be standalone. Part of cephadm'svalue is that you can just curl | > python3 it onto a fresh system. If it requires installing 10+ Python > packages first, you've lost that simplicity. > > What do you think?
I understand that reasoning, and cephadm isn't something we are using anyway (atm).. _______________________________________________ pve-devel mailing list [email protected] https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
