Hi, I would appreciate feedback on whether this approach would be considered at all. The optimization requires changing the private IPC architecture between dpkg and dpkg-deb, so it is more intrusive than the copy operation itself. Before refining it further, I would like to know whether that tradeoff is acceptable to the dpkg maintainers, or whether a different architectural boundary would be preferred.
The primary motivation is building many OS images from the same set of uncompressed packages. When the package archive and image roots reside on a reflink-capable filesystem, copy_file_range() can let the extracted files share their underlying extents with the data.tar member in the package archive. An image builder repeatedly installing the same packages into many roots can then avoid storing another physical copy of all those files for every image. Depending on the workload, this has the potential to reduce disk usage massively, in addition to avoiding userspace copying and reducing the time spent extracting packages. Realizing the reflink benefit also requires regular-file payloads to be aligned suitably in the uncompressed tar stream. The proposed GNU tar --align=SIZE support provides that archive layout by padding PAX extended headers so file data starts at the requested boundary: https://lists.gnu.org/archive/html/bug-tar/2026-07/msg00003.html The dpkg copy_file_range() support does not depend on that option for correctness, but unaligned archives will generally miss the intended extent sharing optimization. Once the GNU tar patch is merged, dpkg will therefore also need to expose the alignment control when creating packages, so builders can request appropriately aligned, uncompressed data.tar members. To make this possible, the patch adds a private protocol through which dpkg-deb can expose a seekable, bounded range for an uncompressed data.tar member. dpkg tracks that bound while parsing the tar stream and attempts copy_file_range() only when installing a regular file directly. Compressed archives, non-seekable input, unsupported filesystems, and older dpkg-deb implementations continue to use the existing streaming path. In particular, feedback would be welcome on: * whether optimizing the uncompressed-package image-building use case is something dpkg would be willing to support; * whether extending the private dpkg/dpkg-deb IPC in this way is acceptable; and * whether there is a preferred alternative for exposing the raw data.tar member without adding this protocol. The patch includes fallback, compatibility, malformed-member, protocol, and end-to-end tests. Daan De Meyer (1): dpkg: Extract uncompressed tar members with copy_file_range configure.ac | 1 + lib/dpkg/Makefile.am | 1 + lib/dpkg/fdio.c | 85 ++++++++++++++ lib/dpkg/fdio.h | 19 +++ lib/dpkg/libdpkg.map | 1 + lib/dpkg/t/t-fdio.c | 232 +++++++++++++++++++++++++++++++++++++ src/Makefile.am | 2 + src/at/deb-streaming.at | 53 +++++++++ src/at/local.at | 1 + src/common/fsys-tarfile.h | 28 +++++ src/deb/extract.c | 76 +++++++++++- src/main/archives.c | 116 ++++++++++++++++--- src/main/archives.h | 11 +- src/main/unpack.c | 140 +++++++++++++++++++++- tests/t-filtering/Makefile | 60 ++++++++++ 15 files changed, 800 insertions(+), 26 deletions(-) create mode 100644 lib/dpkg/t/t-fdio.c create mode 100644 src/common/fsys-tarfile.h -- 2.55.0
