one small question inline, otherwise this one LGTM On January 23, 2026 8:56 am, Kefu Chai wrote: > Previously, cephadm's build process pulled dependencies from PyPI, > which failed in environments with restricted network access. This > change backports upstream improvements that enable building cephadm > using system packages instead, mirroring the existing RPM packaging > workflow. > > And the vendored Arrow pulled xsimd from GitHub, this also failed in > building environment without network access. This change backports > upstream improvements enabling the build to use libxsimd-dev package > when building debian packages. > > Backported changes: > - https://github.com/ceph/ceph/pull/66256 - Use system packages for > cephadm bundled dependencies > - https://github.com/ceph/ceph/pull/66248 - Add libxsimd-dev build > dependency for vendored Arrow > - https://github.com/ceph/ceph/pull/65292 - cephadm: fix building > rpm-sourced cephadm zippapp on el10 > > Note: Test commits from these PRs are excluded from this backport. > > Signed-off-by: Kefu Chai <[email protected]> > --- > ...-issues-running-existing-cephadm-bui.patch | 66 +++++ > ...root_entries-population-in-version-c.patch | 62 +++++ > ...cephadm-rpm-based-builds-without-top.patch | 89 ++++++ > ...d-Debian-package-support-for-bundled.patch | 261 ++++++++++++++++++ > ...m-packages-for-cephadm-bundled-depen.patch | 54 ++++ > ...-Use-AUTO-mode-for-xsimd-dependency-.patch | 55 ++++ > ...dd-libxsimd-dev-build-dependency-for.patch | 41 +++ > patches/series | 8 + > 8 files changed, 636 insertions(+) > create mode 100644 > patches/0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch > create mode 100644 > patches/0043-cephadm-fix-zip_root_entries-population-in-version-c.patch > create mode 100644 > patches/0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch > create mode 100644 > patches/0045-cephadm-build-Add-Debian-package-support-for-bundled.patch > create mode 100644 > patches/0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch > create mode 100644 > patches/0047-cmake-BuildArrow-Use-AUTO-mode-for-xsimd-dependency-.patch > create mode 100644 > patches/0048-debian-control-Add-libxsimd-dev-build-dependency-for.patch > > diff --git > a/patches/0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch > b/patches/0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch > new file mode 100644 > index 00000000000..c6208b0db2a > --- /dev/null > +++ b/patches/0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch > @@ -0,0 +1,66 @@ > +From 2685f6c35245d033e73e16ece3e356fcead44cea Mon Sep 17 00:00:00 2001 > +From: John Mulligan <[email protected]> > +Date: Thu, 21 Aug 2025 12:30:55 -0400 > +Subject: [PATCH 42/46] cephadm: fix some issues running existing cephadm > build > + tests > + > +As time has marched on and people changed things our tests no longer > +match the expected inputs. > + > +Signed-off-by: John Mulligan <[email protected]> > +(cherry picked from commit 31c8010faa417ca53614bd30379a9b9c0c9199de) > +--- > + src/cephadm/tests/build/test_cephadm_build.py | 18 ++++++++++++------ > + 1 file changed, 12 insertions(+), 6 deletions(-) > + > +diff --git a/src/cephadm/tests/build/test_cephadm_build.py > b/src/cephadm/tests/build/test_cephadm_build.py > +index c2995a76d4b..9beef67ffe0 100644 > +--- a/src/cephadm/tests/build/test_cephadm_build.py > ++++ b/src/cephadm/tests/build/test_cephadm_build.py > +@@ -34,12 +34,12 @@ CONTAINERS = { > + }, > + 'ubuntu-20.04': { > + 'name': 'cephadm-build-test:ubuntu-20-04-py3', > +- 'base_image': 'quay.io/library/ubuntu:20.04', > ++ 'base_image': 'docker.io/library/ubuntu:20.04', > + 'script': 'apt update && apt install -y python3-venv', > + }, > + 'ubuntu-22.04': { > + 'name': 'cephadm-build-test:ubuntu-22-04-py3', > +- 'base_image': 'quay.io/library/ubuntu:22.04', > ++ 'base_image': 'docker.io/library/ubuntu:22.04', > + 'script': 'apt update && apt install -y python3-venv', > + }, > + } > +@@ -128,8 +128,8 @@ def test_cephadm_build(env, source_dir, tmp_path): > + assert all('requirements_entry' in v for v in data['bundled_packages']) > + assert 'zip_root_entries' in data > + zre = data['zip_root_entries'] > +- assert any(e.startswith('Jinja2') for e in zre) > +- assert any(e.startswith('MarkupSafe') for e in zre) > ++ assert any(_dist_info(e, 'Jinja2') for e in zre) > ++ assert any(_dist_info(e, 'MarkupSafe') for e in zre) > + assert any(e.startswith('jinja2') for e in zre) > + assert any(e.startswith('markupsafe') for e in zre) > + assert any(e.startswith('cephadmlib') for e in zre) > +@@ -184,9 +184,15 @@ def test_cephadm_build_from_rpms(env, source_dir, > tmp_path): > + assert all('requirements_entry' in v for v in data['bundled_packages']) > + assert 'zip_root_entries' in data > + zre = data['zip_root_entries'] > +- assert any(e.startswith('Jinja2') for e in zre) > +- assert any(e.startswith('MarkupSafe') for e in zre) > ++ assert any(_dist_info(e, 'Jinja2') for e in zre) > ++ assert any(_dist_info(e, 'MarkupSafe') for e in zre) > + assert any(e.startswith('jinja2') for e in zre) > + assert any(e.startswith('markupsafe') for e in zre) > + assert any(e.startswith('cephadmlib') for e in zre) > + assert any(e.startswith('_cephadmmeta') for e in zre) > ++ > ++ > ++def _dist_info(entry, name): > ++ return ( > ++ entry.startswith(entry) or entry.startswith(entry.lower()) > ++ ) and (entry.endswith('.dist-info') or entry.endswith('.egg-info')) > +-- > +2.47.3 > + > diff --git > a/patches/0043-cephadm-fix-zip_root_entries-population-in-version-c.patch > b/patches/0043-cephadm-fix-zip_root_entries-population-in-version-c.patch > new file mode 100644 > index 00000000000..7a0b1a18d68 > --- /dev/null > +++ b/patches/0043-cephadm-fix-zip_root_entries-population-in-version-c.patch > @@ -0,0 +1,62 @@ > +From 2c20ae95f32a2bdf78576252f1c9171f00a2790e Mon Sep 17 00:00:00 2001 > +From: Kefu Chai <[email protected]> > +Date: Mon, 10 Nov 2025 12:11:08 +0800 > +Subject: [PATCH 43/46] cephadm: fix zip_root_entries population in version > + command > + > +The 'cephadm version --verbose' command was returning an empty > +zip_root_entries list because it relied on the private '_files' > +attribute of zipimport.zipimporter, which is not reliably populated > +across Python versions. > + > +This commit fixes the issue by using the zipfile module to properly > +read the archive contents via the loader.archive path. This ensures > +that zip_root_entries is correctly populated with the root-level > +directories in the zipapp. > + > +This fix is necessary for the cephadm build tests to properly validate > +that all expected packages and modules are included in the built zipapp. > + > +Signed-off-by: Kefu Chai <[email protected]> > +(cherry picked from commit 2c68c1496dbb7cd01bf783e31510940445040a34) > +--- > + src/cephadm/cephadm.py | 16 ++++++++++++---- > + 1 file changed, 12 insertions(+), 4 deletions(-) > + > +diff --git a/src/cephadm/cephadm.py b/src/cephadm/cephadm.py > +index f75aaa86dac..8d08c700868 100755 > +--- a/src/cephadm/cephadm.py > ++++ b/src/cephadm/cephadm.py > +@@ -1688,6 +1688,7 @@ def command_version(ctx): > + # type: (CephadmContext) -> int > + import importlib > + import zipimport > ++ import zipfile > + import types > + > + vmod: Optional[types.ModuleType] > +@@ -1744,10 +1745,17 @@ def command_version(ctx): > + out['bundled_packages'] = deps_info > + except OSError: > + pass > +- files = getattr(loader, '_files', {}) > +- out['zip_root_entries'] = sorted( > +- {p.split('/')[0] for p in files.keys()} > +- ) > ++ # Use zipfile module to properly read the archive contents > ++ # loader.archive contains the path to the zip file > ++ try: > ++ with zipfile.ZipFile(loader.archive, 'r') as zf: > ++ files = zf.namelist() > ++ out['zip_root_entries'] = sorted( > ++ {p.split('/')[0] for p in files if p} > ++ ) > ++ except (OSError, zipfile.BadZipFile): > ++ # Fallback to empty list if we can't read the zip > ++ out['zip_root_entries'] = [] > + > + json.dump(out, sys.stdout, indent=2) > + print() > +-- > +2.47.3 > + > diff --git > a/patches/0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch > b/patches/0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch > new file mode 100644 > index 00000000000..6e1b4dd2e0e > --- /dev/null > +++ b/patches/0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch > @@ -0,0 +1,89 @@ > +From 5ce72178a782dfe2e80648063b025ba23d96d5d8 Mon Sep 17 00:00:00 2001 > +From: John Mulligan <[email protected]> > +Date: Thu, 21 Aug 2025 12:44:17 -0400 > +Subject: [PATCH 44/46] cephadm: support cephadm rpm based builds without > + top_level.txt > + > +Signed-off-by: John Mulligan <[email protected]> > +(cherry picked from commit 26a499a8da339d870af193ea964368afbc84c694) > +--- > + src/cephadm/build.py | 56 +++++++++++++++++++++++++++++++++----------- > + 1 file changed, 42 insertions(+), 14 deletions(-) > + > +diff --git a/src/cephadm/build.py b/src/cephadm/build.py > +index 43bc58a4003..01c91fc1680 100755 > +--- a/src/cephadm/build.py > ++++ b/src/cephadm/build.py > +@@ -434,6 +434,47 @@ def _install_rpm_deps(tempdir, config): > + return dinfo > + > + > ++def _gather_rpm_package_dirs(paths): > ++ # = The easy way = > ++ # the top_level.txt file can be used to determine where the python > packages > ++ # actually are. We need all of those and the meta-data dir (parent of > ++ # top_level.txt) to be included in our zipapp > ++ top_level = None > ++ for path in paths: > ++ if path.endswith('top_level.txt'): > ++ top_level = pathlib.Path(path) > ++ if top_level: > ++ meta_dir = top_level.parent > ++ pkg_dirs = [ > ++ top_level.parent.parent / p > ++ for p in top_level.read_text().splitlines() > ++ ] > ++ return meta_dir, pkg_dirs > ++ # = The hard way = > ++ # loop through the directories to find the .dist-info dir (containing > the > ++ # mandatory METADATA file, according to the spec) and once we know the > ++ # location of dist info we find the sibling paths from the rpm listing > ++ dist_info = None > ++ ppaths = [] > ++ for path in paths: > ++ ppath = pathlib.Path(path) > ++ ppaths.append(ppath) > ++ if ppath.name == 'METADATA' and > ppath.parent.name.endswith('.dist-info'): > ++ dist_info = ppath.parent > ++ break > ++ if not dist_info: > ++ raise ValueError('no .dist-info METADATA found') > ++ if not dist_info.parent.name == 'site-packages': > ++ raise ValueError( > ++ 'unexpected parent directory (not site-packages):' > ++ f' {dist_info.parent.name}' > ++ ) > ++ siblings = [ > ++ p for p in ppaths if p.parent == dist_info.parent and p != dist_info > ++ ] > ++ return dist_info, siblings > ++ > ++ > + def _deps_from_rpm(tempdir, config, dinfo, pkg): > + # first, figure out what rpm provides a particular python lib > + dist = f'python3.{sys.version_info.minor}dist({pkg})'.lower() > +@@ -469,20 +510,7 @@ def _deps_from_rpm(tempdir, config, dinfo, pkg): > + ['rpm', '-ql', rpmname], check=True, stdout=subprocess.PIPE > + ) > + paths = [l.decode('utf8') for l in res.stdout.splitlines()] > +- # the top_level.txt file can be used to determine where the python > packages > +- # actually are. We need all of those and the meta-data dir (parent of > +- # top_level.txt) to be included in our zipapp > +- top_level = None > +- for path in paths: > +- if path.endswith('top_level.txt'): > +- top_level = pathlib.Path(path) > +- if not top_level: > +- raise ValueError('top_level not found') > +- meta_dir = top_level.parent > +- pkg_dirs = [ > +- top_level.parent.parent / p > +- for p in top_level.read_text().splitlines() > +- ] > ++ meta_dir, pkg_dirs = _gather_rpm_package_dirs(paths) > + meta_dest = tempdir / meta_dir.name > + log.info(f"Copying {meta_dir} to {meta_dest}") > + # copy the meta data directory > +-- > +2.47.3 > + > diff --git > a/patches/0045-cephadm-build-Add-Debian-package-support-for-bundled.patch > b/patches/0045-cephadm-build-Add-Debian-package-support-for-bundled.patch > new file mode 100644 > index 00000000000..94241490b64 > --- /dev/null > +++ b/patches/0045-cephadm-build-Add-Debian-package-support-for-bundled.patch > @@ -0,0 +1,261 @@ > +From a4744feb1457aa7fb3f47b42f29138682cc7d41c Mon Sep 17 00:00:00 2001 > +From: Kefu Chai <[email protected]> > +Date: Tue, 14 Oct 2025 21:04:42 +0800 > +Subject: [PATCH 45/46] cephadm/build: Add Debian package support for bundled > + dependencies > + > +Extends the cephadm build script to support bundling dependencies from > +Debian packages in addition to pip and RPM packages. This allows building > +cephadm on Debian-based distributions using system packages. > + > +Key changes: > +- Add 'deb' to DependencyMode enum to enable Debian package mode > +- Implement _setup_deb() to configure Debian dependency requirements > +- Add _install_deb_deps() to orchestrate Debian package installation > +- Add _gather_deb_package_dirs() to parse Debian package file listings > + and locate Python package directories (handles both site-packages and > + dist-packages directories used by Debian) > +- Add _deps_from_deb() to extract Python dependencies from installed > + Debian packages using dpkg/apt-cache tools > +- Fix variable reference bug in _install_deps() (deps.mode -> > config.deps_mode) > + > +The Debian implementation follows a similar pattern to the existing RPM > +support, using dpkg-query and dpkg -L to locate installed packages and > +their files, with special handling for Debian naming conventions > +(e.g., PyYAML -> python3-yaml). > + > +Signed-off-by: Kefu Chai <[email protected]> > +(cherry picked from commit 3ff9b0c24e33debe95a0a0c6b42da30be788871c) > +--- > + src/cephadm/build.py | 158 +++++++++++++++++++++++++++++++++++++++---- > + 1 file changed, 143 insertions(+), 15 deletions(-) > + > +diff --git a/src/cephadm/build.py b/src/cephadm/build.py > +index 01c91fc1680..53d3983de89 100755 > +--- a/src/cephadm/build.py > ++++ b/src/cephadm/build.py > +@@ -154,6 +154,7 @@ class PipEnv(enum.Enum): > + class DependencyMode(enum.Enum): > + pip = enum.auto() > + rpm = enum.auto() > ++ deb = enum.auto() > + none = enum.auto() > + > + > +@@ -169,6 +170,8 @@ class Config: > + self._setup_pip() > + elif self.deps_mode == DependencyMode.rpm: > + self._setup_rpm() > ++ elif self.deps_mode == DependencyMode.deb: > ++ self._setup_deb() > + > + def _setup_pip(self): > + if self._maj_min == (3, 6): > +@@ -180,6 +183,9 @@ class Config: > + def _setup_rpm(self): > + self.requirements = [InstallSpec(**v) for v in PY_REQUIREMENTS] > + > ++ def _setup_deb(self): > ++ self.requirements = [InstallSpec(**v) for v in PY_REQUIREMENTS] > ++ > + > + class DependencyInfo: > + """Type for tracking bundled dependencies.""" > +@@ -333,7 +339,9 @@ def _install_deps(tempdir, config): > + return _install_pip_deps(tempdir, config) > + if config.deps_mode == DependencyMode.rpm: > + return _install_rpm_deps(tempdir, config) > +- raise ValueError(f'unexpected deps mode: {deps.mode}') > ++ if config.deps_mode == DependencyMode.deb: > ++ return _install_deb_deps(tempdir, config) > ++ raise ValueError(f'unexpected deps mode: {config.deps_mode}') > + > + > + def _install_pip_deps(tempdir, config): > +@@ -434,7 +442,26 @@ def _install_rpm_deps(tempdir, config): > + return dinfo > + > + > +-def _gather_rpm_package_dirs(paths): > ++def _install_deb_deps(tempdir, config): > ++ log.info("Installing dependencies using Debian packages") > ++ dinfo = DependencyInfo(config) > ++ for pkg in config.requirements: > ++ log.info(f"Looking for debian package for: {pkg.name!r}") > ++ _deps_from_deb(tempdir, config, dinfo, pkg.name) > ++ return dinfo > ++ > ++ > ++def _gather_package_dirs(paths, expected_parent_dir): > ++ """Parse package file listing to find Python package directories. > ++ > ++ Args: > ++ paths: List of file paths from package listing > ++ expected_parent_dir: Expected parent directory name (e.g., > 'site-packages' for RPM, > ++ 'dist-packages' for Debian) > ++ > ++ Returns: > ++ Tuple of (metadata_dir, package_dirs) > ++ """ > + # = The easy way = > + # the top_level.txt file can be used to determine where the python > packages > + # actually are. We need all of those and the meta-data dir (parent of > +@@ -453,7 +480,7 @@ def _gather_rpm_package_dirs(paths): > + # = The hard way = > + # loop through the directories to find the .dist-info dir (containing > the > + # mandatory METADATA file, according to the spec) and once we know the > +- # location of dist info we find the sibling paths from the rpm listing > ++ # location of dist info we find the sibling paths from the package > listing > + dist_info = None > + ppaths = [] > + for path in paths: > +@@ -464,9 +491,9 @@ def _gather_rpm_package_dirs(paths): > + break > + if not dist_info: > + raise ValueError('no .dist-info METADATA found') > +- if not dist_info.parent.name == 'site-packages': > ++ if dist_info.parent.name != expected_parent_dir: > + raise ValueError( > +- 'unexpected parent directory (not site-packages):' > ++ f'unexpected parent directory (not {expected_parent_dir}):' > + f' {dist_info.parent.name}' > + ) > + siblings = [ > +@@ -475,6 +502,31 @@ def _gather_rpm_package_dirs(paths): > + return dist_info, siblings > + > + > ++def _copy_package_files(tempdir, paths, expected_parent_dir): > ++ """Copy package files to the build directory. > ++ > ++ Args: > ++ tempdir: Temporary directory to copy files to > ++ paths: List of file paths from package listing > ++ expected_parent_dir: Expected parent directory name per packaging > convention: > ++ - 'site-packages' for RPM-based distributions > ++ - 'dist-packages' for Debian-based distributions > ++ > ++ Returns: > ++ None > ++ """ > ++ meta_dir, pkg_dirs = _gather_package_dirs(paths, expected_parent_dir) > ++ meta_dest = tempdir / meta_dir.name > ++ log.info(f"Copying {meta_dir} to {meta_dest}") > ++ # copy the meta data directory > ++ shutil.copytree(meta_dir, meta_dest, ignore=_ignore_cephadmlib) > ++ # copy all the package directories > ++ for pkg_dir in pkg_dirs: > ++ pkg_dest = tempdir / pkg_dir.name > ++ log.info(f"Copying {pkg_dir} to {pkg_dest}") > ++ shutil.copytree(pkg_dir, pkg_dest, ignore=_ignore_cephadmlib) > ++ > ++ > + def _deps_from_rpm(tempdir, config, dinfo, pkg): > + # first, figure out what rpm provides a particular python lib > + dist = f'python3.{sys.version_info.minor}dist({pkg})'.lower() > +@@ -510,16 +562,92 @@ def _deps_from_rpm(tempdir, config, dinfo, pkg): > + ['rpm', '-ql', rpmname], check=True, stdout=subprocess.PIPE > + ) > + paths = [l.decode('utf8') for l in res.stdout.splitlines()] > +- meta_dir, pkg_dirs = _gather_rpm_package_dirs(paths) > +- meta_dest = tempdir / meta_dir.name > +- log.info(f"Copying {meta_dir} to {meta_dest}") > +- # copy the meta data directory > +- shutil.copytree(meta_dir, meta_dest, ignore=_ignore_cephadmlib) > +- # copy all the package directories > +- for pkg_dir in pkg_dirs: > +- pkg_dest = tempdir / pkg_dir.name > +- log.info(f"Copying {pkg_dir} to {pkg_dest}") > +- shutil.copytree(pkg_dir, pkg_dest, ignore=_ignore_cephadmlib) > ++ # RPM-based distributions use 'site-packages' for Python packages > ++ _copy_package_files(tempdir, paths, 'site-packages') > ++ > ++ > ++def _deps_from_deb(tempdir, config, dinfo, pkg): > ++ """Extract Python dependencies from Debian packages. > ++ > ++ Args: > ++ tempdir: Temporary directory to copy package files to > ++ config: Build configuration > ++ dinfo: DependencyInfo instance to track dependencies > ++ pkg: Python package name (e.g., 'MarkupSafe', 'Jinja2', 'PyYAML') > ++ """ > ++ # Convert Python package name to Debian package name > ++ # Python packages are typically named python3-<lowercase-name> > ++ # Handle special cases: PyYAML -> python3-yaml, MarkupSafe -> > python3-markupsafe > ++ pkg_lower = pkg.lower() > ++ if pkg_lower == 'pyyaml': > ++ deb_pkg_name = 'python3-yaml' > ++ else: > ++ deb_pkg_name = f'python3-{pkg_lower}' > ++ > ++ # First, try to find the package using apt-cache > ++ # This helps verify the package exists before trying to list its files > ++ try: > ++ res = subprocess.run( > ++ ['apt-cache', 'show', deb_pkg_name], > ++ check=True, > ++ stdout=subprocess.PIPE, > ++ stderr=subprocess.PIPE, > ++ ) > ++ except subprocess.CalledProcessError: > ++ # Package not found, try alternative naming > ++ log.warning(f"Package {deb_pkg_name} not found via apt-cache, > trying dpkg -S") > ++ # Try to search for files that might belong to this package > ++ # Search for the Python module in site-packages > ++ search_pattern = f'/usr/lib/python3*/dist-packages/{pkg.lower()}*' > ++ try: > ++ res = subprocess.run( > ++ ['dpkg', '-S', search_pattern], > ++ check=True, > ++ stdout=subprocess.PIPE, > ++ stderr=subprocess.PIPE, > ++ ) > ++ # dpkg -S output format: "package: /path/to/file" > ++ deb_pkg_name = res.stdout.decode('utf8').split(':')[0].strip() > ++ except subprocess.CalledProcessError as err: > ++ log.error(f"Could not find Debian package for {pkg}") > ++ log.error(f"Tried: {deb_pkg_name} and pattern search") > ++ sys.exit(1) > ++ > ++ # Get version information using dpkg-query > ++ try: > ++ res = subprocess.run( > ++ ['dpkg-query', '-W', '-f=${Version}\\n', deb_pkg_name], > ++ check=True, > ++ stdout=subprocess.PIPE, > ++ ) > ++ version = res.stdout.decode('utf8').strip() > ++ except subprocess.CalledProcessError as err: > ++ log.error(f"Could not query version for package {deb_pkg_name}: > {err}") > ++ sys.exit(1) > ++ > ++ log.info(f"Debian Package: {deb_pkg_name} (version: {version})") > ++ dinfo.add( > ++ pkg, > ++ deb_name=deb_pkg_name, > ++ version=version, > ++ package_source='deb', > ++ ) > ++ > ++ # Get the list of files provided by the Debian package > ++ try: > ++ res = subprocess.run( > ++ ['dpkg', '-L', deb_pkg_name], > ++ check=True, > ++ stdout=subprocess.PIPE, > ++ ) > ++ except subprocess.CalledProcessError as err: > ++ log.error(f"Could not list files for package {deb_pkg_name}: {err}") > ++ sys.exit(1) > ++ > ++ paths = [l.decode('utf8') for l in res.stdout.splitlines()] > ++ # Debian-based distributions use 'dist-packages' for system-managed > Python packages > ++ # per Debian Python Policy: > https://www.debian.org/doc/packaging-manuals/python-policy/ > ++ _copy_package_files(tempdir, paths, 'dist-packages') > + > + > + def generate_version_file(versioning_vars, dest): > +-- > +2.47.3 > + > diff --git > a/patches/0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch > b/patches/0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch > new file mode 100644 > index 00000000000..0b4daa63f79 > --- /dev/null > +++ b/patches/0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch > @@ -0,0 +1,54 @@ > +From 779c9251669b65b7ed77a841f281e2d957174db2 Mon Sep 17 00:00:00 2001 > +From: Kefu Chai <[email protected]> > +Date: Tue, 21 Oct 2025 11:25:00 +0800 > +Subject: [PATCH 46/46] debian: Use system packages for cephadm bundled > + dependencies > + > +Configure the Debian build to use CEPHADM_BUNDLED_DEPENDENCIES=deb, > +which instructs the cephadm build script to bundle dependencies from > +system-installed Debian packages instead of downloading from PyPI. > + > +This change addresses build failures in restricted network environments > +where Debian build tools do not permit internet access. By leveraging > +the Debian package support added in commit 9378a2988e1, the build now > +uses python3-markupsafe, python3-jinja2, and python3-yaml packages > +that are already installed as build dependencies. > + > +This approach mirrors the existing RPM packaging workflow, ensuring > +consistent behavior across different distribution package formats. > + > +Signed-off-by: Kefu Chai <[email protected]> > +(cherry picked from commit 25680021ee2422f8b8b4075b3ab77af39126eecc)
long term, it might be even better to not bundle at all, but add those python deps as runtime dependencies? > +--- > + debian/control | 2 ++ > + debian/rules | 1 + > + 2 files changed, 3 insertions(+) > + > +diff --git a/debian/control b/debian/control > +index 35c1dbe5bd2..94997827bd2 100644 > +--- a/debian/control > ++++ b/debian/control > +@@ -106,6 +106,8 @@ Build-Depends: automake, > + python3-requests <pkg.ceph.check>, > + python3-scipy <pkg.ceph.check>, > + python3-onelogin-saml2 <pkg.ceph.check>, > ++ python3-jinja2, > ++ python3-markupsafe, > + python3-setuptools, > + python3-sphinx, > + python3-venv, > +diff --git a/debian/rules b/debian/rules > +index b935efaa8cd..ee4eff2bdc5 100755 > +--- a/debian/rules > ++++ b/debian/rules > +@@ -30,6 +30,7 @@ extraopts += -DWITH_CEPHFS_JAVA=ON > + extraopts += -DWITH_CEPHFS_SHELL=ON > + extraopts += -DWITH_SYSTEMD=ON -DCEPH_SYSTEMD_ENV_DIR=/etc/default > + extraopts += -DWITH_GRAFANA=ON > ++extraopts += -DCEPHADM_BUNDLED_DEPENDENCIES=deb > + ifeq ($(DEB_HOST_ARCH), amd64) > + extraopts += -DWITH_RBD_RWL=ON > + else > +-- > +2.47.3 > + > diff --git > a/patches/0047-cmake-BuildArrow-Use-AUTO-mode-for-xsimd-dependency-.patch > b/patches/0047-cmake-BuildArrow-Use-AUTO-mode-for-xsimd-dependency-.patch > new file mode 100644 > index 00000000000..ab5681bece0 > --- /dev/null > +++ b/patches/0047-cmake-BuildArrow-Use-AUTO-mode-for-xsimd-dependency-.patch > @@ -0,0 +1,55 @@ > +From 46803fcaa3b2b4f14318b1320e8e3beb9cf3eec4 Mon Sep 17 00:00:00 2001 > +From: Kefu Chai <[email protected]> > +Date: Wed, 15 Oct 2025 15:46:22 +0800 > +Subject: [PATCH 47/48] cmake/BuildArrow: Use AUTO mode for xsimd dependency > + detection > +MIME-Version: 1.0 > +Content-Type: text/plain; charset=UTF-8 > +Content-Transfer-Encoding: 8bit > + > +Arrow requires xsimd >= 9.0.1 according to arrow/cpp/thirdparty/versions.txt. > +Previously, we unconditionally set -Dxsimd_SOURCE=BUNDLED, forcing the use > +of Arrow's vendored xsimd regardless of system package availability. > + > +This commit changes to -Dxsimd_SOURCE=AUTO, which allows Arrow's > +resolve_dependency mechanism to automatically: > +1. Try to find system xsimd package > +2. Check if version >= 9.0.1 > +3. Use system version if found and sufficient > +4. Fall back to bundled version otherwise > + > +This reduces build time and dependencies on systems with sufficient xsimd, > +while maintaining compatibility with older distributions. > + > +Distribution availability: > +- Ubuntu Noble (24.04): libxsimd-dev 12.1.1 (✓ will use system) > +- Ubuntu Jammy (22.04): libxsimd-dev 7.6.0 (✗ will use bundled) > +- Debian Trixie (13): libxsimd-dev 13.2.0 (✓ will use system) > +- CentOS Stream 9: xsimd-devel 7.4.9 (✗ will use bundled) > + > +Signed-off-by: Kefu Chai <[email protected]> > +(cherry picked from commit 2defde30e4c61cd9f33196b91ac16505928ce774) > +--- > + cmake/modules/BuildArrow.cmake | 6 +++--- > + 1 file changed, 3 insertions(+), 3 deletions(-) > + > +diff --git a/cmake/modules/BuildArrow.cmake b/cmake/modules/BuildArrow.cmake > +index 0ee1d85b49f..cadc2909fa4 100644 > +--- a/cmake/modules/BuildArrow.cmake > ++++ b/cmake/modules/BuildArrow.cmake > +@@ -69,9 +69,9 @@ function(build_arrow) > + list(APPEND arrow_DEPENDS Boost) > + endif() > + > +- # since Arrow 15.0.0 needs xsimd>=8.1.0 and since Ubuntu Jammy > +- # Jellyfish only provides 7.6.0, we'll have arrow build it as source > +- list(APPEND arrow_CMAKE_ARGS -Dxsimd_SOURCE=BUNDLED) > ++ # Arrow requires xsimd >= 9.0.1 (see arrow/cpp/thirdparty/versions.txt). > ++ # Use AUTO to let Arrow detect system xsimd and fall back to bundled if > needed. > ++ list(APPEND arrow_CMAKE_ARGS -Dxsimd_SOURCE=AUTO) > + > + # cmake doesn't properly handle arguments containing ";", such as > + # CMAKE_PREFIX_PATH, for which reason we'll have to use some other > separator. > +-- > +2.47.3 > + > diff --git > a/patches/0048-debian-control-Add-libxsimd-dev-build-dependency-for.patch > b/patches/0048-debian-control-Add-libxsimd-dev-build-dependency-for.patch > new file mode 100644 > index 00000000000..c431fb6edc1 > --- /dev/null > +++ b/patches/0048-debian-control-Add-libxsimd-dev-build-dependency-for.patch > @@ -0,0 +1,41 @@ > +From 20fb1b8eff9d9b4e683c33e556752085b408c24b Mon Sep 17 00:00:00 2001 > +From: Kefu Chai <[email protected]> > +Date: Wed, 15 Oct 2025 15:46:26 +0800 > +Subject: [PATCH 48/48] debian/control: Add libxsimd-dev build dependency for > + vendored Arrow > + > +In commit e8460cbd, we introduced the "pkg.ceph.arrow" build profile to > +support building with system Arrow packages. However, neither Debian nor > +Ubuntu currently ships Arrow packages. > + > +Since WITH_RADOSGW_SELECT_PARQUET is always enabled in debian/rules, > +Arrow support is required for all builds. When the pkg.ceph.arrow profile > +is not selected, the build uses vendored Arrow. With the recent change to > +use AUTO mode for xsimd detection, Arrow will attempt to find system xsimd > +>= 9.0.1. Adding libxsimd-dev as a build dependency ensures it's available > +for Arrow to detect and use, reducing build time on supported distributions. > + > +On distributions with insufficient xsimd versions (< 9.0.1), Arrow will > +automatically fall back to its bundled version. > + > +Signed-off-by: Kefu Chai <[email protected]> > +(cherry picked from commit 929b47f83880683b376b9dcc0060938447f11c88) > +--- > + debian/control | 1 + > + 1 file changed, 1 insertion(+) > + > +diff --git a/debian/control b/debian/control > +index 94997827bd2..7d30e1faa70 100644 > +--- a/debian/control > ++++ b/debian/control > +@@ -85,6 +85,7 @@ Build-Depends: automake, > + libpmem-dev <pkg.ceph.pmdk>, > + libpmemobj-dev (>= 1.8) <pkg.ceph.pmdk>, > + libprotobuf-dev <pkg.ceph.crimson>, > ++ libxsimd-dev <!pkg.ceph.arrow>, > + ninja-build, > + nlohmann-json3-dev, > + patch, > +-- > +2.47.3 > + > diff --git a/patches/series b/patches/series > index d775a8a068d..f2bfed1b939 100644 > --- a/patches/series > +++ b/patches/series > @@ -39,3 +39,11 @@ > 0039-mgr-fix-module-import-by-making-NOTIFY_TYPES-in-py-m.patch > 0040-pybind-rbd-disable-on_progress-callbacks-to-prevent-.patch > 0041-systemd-services-fix-installing-ceph-volume.patch > +0042-cephadm-fix-some-issues-running-existing-cephadm-bui.patch > +0043-cephadm-fix-zip_root_entries-population-in-version-c.patch > +0044-cephadm-support-cephadm-rpm-based-builds-without-top.patch > +0045-cephadm-build-Add-Debian-package-support-for-bundled.patch > +0046-debian-Use-system-packages-for-cephadm-bundled-depen.patch > +0047-cmake-BuildArrow-Use-AUTO-mode-for-xsimd-dependency-.patch > +0048-debian-control-Add-libxsimd-dev-build-dependency-for.patch > + > -- > 2.47.3 > > > > _______________________________________________ > pve-devel mailing list > [email protected] > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel > _______________________________________________ pve-devel mailing list [email protected] https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
