Bug#982461: apt: Please provide a configuration option for disabling fsync()-s
Hello, I'd like to support Balint here, since for example on mobile phones and tablets, which using eMMC and UFS storages, with limited lifetime it's using eatmydata almost requirement, but not all users are aware of the option. Would be nice to have simple option which could distributions based on Debian (for example Mobian) easily enable and improve speed of instalation and durability of eMMCs. Thank you David On Wed, 10 Feb 2021 15:58:59 +0100 Balint Reczey wrote: > Hi Julian, > > On Wed, Feb 10, 2021 at 3:23 PM Julian Andres Klode wrote: > > > > > > On Wed, Feb 10, 2021 at 03:10:45PM +0100, Balint Reczey wrote: > > > Source: apt > > > Version: > > > Severity: wishlist > > > > > > Hi, > > > > > > I run eatmydata apt ... frequently and it is used widely by others > > > when there is no need to fsync()-s during package installations or > > > removals, but it is nice to save time and wear off SSDs later. > > > > > > Using fsync() is unlikely to bring any benefit when a system is very > > > unlikely to crash or it is cheap to recreate in case of an > > > installation failure like it is the case for provisioning containers. > > > Also using fsync is fairly pointless when apt-btrfs-snapshot is > > > installed and a snapshot is taken after the apt operation anyway. > > > > > > Ideally dpkg would provide a command line option for skipping fsync, > > > but it is not yet the case: #923423. > > > > > > On APT's side I'd like to propose a config option which when set would > > > prefix dpkg calls with eatmydata until dpkg has a better interface for > > > disabling fsyncs. > > > > dpkg already has --force-unsafe-io to avoid syncs, it's what I use. I > > don't want to have an option for eatmydata inside apt, it also affects a > > lot of other stuff down the line like services outside on sysvinit > > systems and all stuff happening in maintainer scripts; but then it only > > works on the native architecture, not for foreign ones, and scripts > > might be unsetting LD_PRELOAD. Heck APT probably should unset > > LD_PRELOAD like it cleans up PATH. > > > > I considered adding a seccomp filter to apt that would block fsync() and > > friends, which is more persuasive than eatmydata. But force-unsafe-io is > > usually enough. > > > > Lastly, you can also just create a dpkg -> eatmydata symlink somewhere, > > and then specify that as your Dir::Bin::dpkg, and that would work too. > > eatmydata could ship some symlinks in /usr/libexec/eatmydata, similar to > > what ccache does. > > > > I do not believe that adding such a hack to apt is the right approach. > > IMO the practice of using eatmydata with is widespread enough to be > considered safe (checking for systemd as init), but let's not consider > if for now. > > As I see apt does not pass --force-unsafe-io either yet to dpkg and > the proposed option could add it: > > test@debian:~$ sudo strace -ff -ofirefox-install.trace -efsync apt > install --reinstall ./firefox_85.0.1-1_amd64.deb > ... > Processing triggers for gnome-menus (3.36.0-1) ... > test@debian:~$ grep fsync firefox-install.trace* | wc -l Best regards David Heidelberg
Bug#982461: apt: Please provide a configuration option for disabling fsync()-s
Hi Julian, On Wed, Feb 10, 2021 at 3:23 PM Julian Andres Klode wrote: > > > On Wed, Feb 10, 2021 at 03:10:45PM +0100, Balint Reczey wrote: > > Source: apt > > Version: > > Severity: wishlist > > > > Hi, > > > > I run eatmydata apt ... frequently and it is used widely by others > > when there is no need to fsync()-s during package installations or > > removals, but it is nice to save time and wear off SSDs later. > > > > Using fsync() is unlikely to bring any benefit when a system is very > > unlikely to crash or it is cheap to recreate in case of an > > installation failure like it is the case for provisioning containers. > > Also using fsync is fairly pointless when apt-btrfs-snapshot is > > installed and a snapshot is taken after the apt operation anyway. > > > > Ideally dpkg would provide a command line option for skipping fsync, > > but it is not yet the case: #923423. > > > > On APT's side I'd like to propose a config option which when set would > > prefix dpkg calls with eatmydata until dpkg has a better interface for > > disabling fsyncs. > > dpkg already has --force-unsafe-io to avoid syncs, it's what I use. I > don't want to have an option for eatmydata inside apt, it also affects a > lot of other stuff down the line like services outside on sysvinit > systems and all stuff happening in maintainer scripts; but then it only > works on the native architecture, not for foreign ones, and scripts > might be unsetting LD_PRELOAD. Heck APT probably should unset > LD_PRELOAD like it cleans up PATH. > > I considered adding a seccomp filter to apt that would block fsync() and > friends, which is more persuasive than eatmydata. But force-unsafe-io is > usually enough. > > Lastly, you can also just create a dpkg -> eatmydata symlink somewhere, > and then specify that as your Dir::Bin::dpkg, and that would work too. > eatmydata could ship some symlinks in /usr/libexec/eatmydata, similar to > what ccache does. > > I do not believe that adding such a hack to apt is the right approach. IMO the practice of using eatmydata with is widespread enough to be considered safe (checking for systemd as init), but let's not consider if for now. As I see apt does not pass --force-unsafe-io either yet to dpkg and the proposed option could add it: test@debian:~$ sudo strace -ff -ofirefox-install.trace -efsync apt install --reinstall ./firefox_85.0.1-1_amd64.deb ... Processing triggers for gnome-menus (3.36.0-1) ... test@debian:~$ grep fsync firefox-install.trace* | wc -l 123 The other problem with force-unsafe-io is that it disables only part of the fsync()-s, around half of them: $ sudo strace -ofirefox-install.trace -efsync dpkg --force-unsafe-io -i ./firefox_85.0.1-1_amd64.deb (Reading database ... 194419 files and directories currently installed.) ... test@debian:~$ grep fsync firefox-install.trace* | wc -l 64 test@debian:~$ sudo strace -ofirefox-install.trace -efsync dpkg -i ./firefox_85.0.1-1_amd64.deb (Reading database ... 194419 files and directories currently installed.) ... test@debian:~$ grep fsync firefox-install.trace* | wc -l 117 Maybe dpkg would either extend --force-unsafe-io's scope or add --force-all-unsafe-io to skip all fsync()-s. Cheers, Balint > -- > debian developer - deb.li/jak | jak-linux.org - free software dev > ubuntu core developer i speak de, en -- Balint Reczey Ubuntu & Debian Developer
Bug#982461: apt: Please provide a configuration option for disabling fsync()-s
On Wed, Feb 10, 2021 at 03:10:45PM +0100, Balint Reczey wrote: > Source: apt > Version: > Severity: wishlist > > Hi, > > I run eatmydata apt ... frequently and it is used widely by others > when there is no need to fsync()-s during package installations or > removals, but it is nice to save time and wear off SSDs later. > > Using fsync() is unlikely to bring any benefit when a system is very > unlikely to crash or it is cheap to recreate in case of an > installation failure like it is the case for provisioning containers. > Also using fsync is fairly pointless when apt-btrfs-snapshot is > installed and a snapshot is taken after the apt operation anyway. > > Ideally dpkg would provide a command line option for skipping fsync, > but it is not yet the case: #923423. > > On APT's side I'd like to propose a config option which when set would > prefix dpkg calls with eatmydata until dpkg has a better interface for > disabling fsyncs. dpkg already has --force-unsafe-io to avoid syncs, it's what I use. I don't want to have an option for eatmydata inside apt, it also affects a lot of other stuff down the line like services outside on sysvinit systems and all stuff happening in maintainer scripts; but then it only works on the native architecture, not for foreign ones, and scripts might be unsetting LD_PRELOAD. Heck APT probably should unset LD_PRELOAD like it cleans up PATH. I considered adding a seccomp filter to apt that would block fsync() and friends, which is more persuasive than eatmydata. But force-unsafe-io is usually enough. Lastly, you can also just create a dpkg -> eatmydata symlink somewhere, and then specify that as your Dir::Bin::dpkg, and that would work too. eatmydata could ship some symlinks in /usr/libexec/eatmydata, similar to what ccache does. I do not believe that adding such a hack to apt is the right approach. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
Bug#982461: apt: Please provide a configuration option for disabling fsync()-s
Source: apt Version: Severity: wishlist Hi, I run eatmydata apt ... frequently and it is used widely by others when there is no need to fsync()-s during package installations or removals, but it is nice to save time and wear off SSDs later. Using fsync() is unlikely to bring any benefit when a system is very unlikely to crash or it is cheap to recreate in case of an installation failure like it is the case for provisioning containers. Also using fsync is fairly pointless when apt-btrfs-snapshot is installed and a snapshot is taken after the apt operation anyway. Ideally dpkg would provide a command line option for skipping fsync, but it is not yet the case: #923423. On APT's side I'd like to propose a config option which when set would prefix dpkg calls with eatmydata until dpkg has a better interface for disabling fsyncs. apt-btrfs-snapshot and similar tools could ship a config file snippet setting the option, Cheers, Balint -- Balint Reczey Ubuntu & Debian Developer
