Package: dpkg
Version: 1.21.9
Severity: wishlist
Tags: upstream d-i

Hi,

please reconsider to provide a `--force-really-unsafe-io` (or similar)
option that skips the calls to `fsync()` & friends in dpkg.

I tried installing a larger package set (in stable), including
texlive-full, KDE, GCC and other packages. It took:

  Default dpkg: ~4h10m = 250m
  --force-unsafe-io: ~2h20m = 140m
  eatmydata: ~22m

So skipping all fsync() calls provides a speedup of 11 compared to
dpkg's defaults and still 6 compared to --force-unsafe-io. This is a
very noticable change.

The dpkg FAQ[1] asks users to use a different filesystem, non-default
mount options, `--force-unsafe-io` or `eatmydata`.

I don't think asking users to use a different filesystem is a good
option (and the installer doesn't ask in many cases); same for
non-default mount options. Doing so also has other downsides.

Options like `--force-unsafe-io` are okay (and d-i can and does use
it), but are suboptimal in many situations where one really does not
need any calls to `fsync`.  `eatmydata` is a bit of a hack and harder
to use (make sure apt-get calls a wrapper dpkg script or everything
calls apt-get/aptitude/* via some wrapper) or not possible (d-i
doesn't use eatmydata).  I think dpkg supporting to skip fsync on its
own is the better solution (compared to implementing eatmydata support
in d-i and other places).

There are various cases where this is useful:

- Initial installation (d-i or initial setup with ansible/puppet/scripts/*).
- Containers.
- CI systems.
- Anything else using throw-away systems (chroots, containers, VMs, ...).

Even dpkg itself uses eatmydata to avoid `fsync()`s (in .gitlab-ci.yml).

I know this was previously proposed and not implemented, but I think
the numer of cases where this is useful has increased over time and
will probably increase more in the future.

Ansgar

  [1]: 
https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Why_is_dpkg_so_slow_when_using_new_filesystems_such_as_btrfs_or_ext4.3F

Reply via email to