Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

[Theodore Ts'o]

On Fri, Oct 14, 2022 at 03:37:29PM +0200, Marco d'Itri wrote:
> On Oct 14, Vincent Lefevre  wrote:
> 
> > > This is obviously convenient on Guillem's part, but we have to optimize 
> > > systems by default for the general case and not just for dpkg -i.
> > This dpkg FAQ says that this is not beneficial for just dpkg,
> > but also "for any application in the system".
> [citation needed]
> 
> I hope that you understand why at this point I cannot trust as is the 
> opinions of the dpkg maintainer.

The dpkg FAQ is just wrong.  It relates to controversy which is over a
dozen years old.  For more information see, see Josef Sipek's blog
post from 2009, "O_PONIES & Other Assorted Wishes"

https://blahg.josefsipek.net/?p=364

The O_PONIES mention references an 2009 April Fool's patch:


https://lore.kernel.org/linux-fsdevel/20090401041843.gn19...@josefsipek.net/

Because buggy applications and clueless application programmers vastly
outnumbers file system maintainers, at the 2009 LSF/MM workshop, a
number of the major file system developers agreed on the following
workaround.  If the application opens a pre-existing file with
O_TRUNC, or renames a newly created file on top of pre-existing file,
we will force the delayed allocation to be automatically resolved when
the file is closed (in the first case) or the rename (in the second
case).  It does *not* force a file system commit.  So if you crash
within 5 seconds of the close(2) or rename(2), you will still suffer
data loss.

HOWEVER, this was always the case for buggy applicatons that refused
to call fsync(2) who were relying on the old ext3 file system
semantics.  It did not guarantee that things would work; it would just
*mostly* work, since *usually* you didn't crash within 5 seconds of
rewriting a file.

So no, we're not going to be making the default change to /etc/fstab.

NACK.

- Ted

Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

[Marco d'Itri]

On Oct 14, Vincent Lefevre  wrote:

> > This is obviously convenient on Guillem's part, but we have to optimize 
> > systems by default for the general case and not just for dpkg -i.
> This dpkg FAQ says that this is not beneficial for just dpkg,
> but also "for any application in the system".
[citation needed]

I hope that you understand why at this point I cannot trust as is the 
opinions of the dpkg maintainer.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

[Vincent Lefevre]

On 2022-10-14 12:44:25 +0200, Marco d'Itri wrote:
> On Oct 14, Vincent Lefevre  wrote:
> > > This is not "the Debian FAQ" but "the DPKG FAQ", which has been known to 
> > > recommend awful things in the past.
> > But it is still considered in the present times by the dpkg developers.
> > Bug 923423 was closed several hours ago based on this dpkg FAQ:
> This is obviously convenient on Guillem's part, but we have to optimize 
> systems by default for the general case and not just for dpkg -i.

This dpkg FAQ says that this is not beneficial for just dpkg,
but also "for any application in the system".

> > (the issue here is that there is a fsync for *every* file, so that for
> And at this point I am wondering if this is a good design, or if it 
> would be better to use a different strategy.

IMHO, this should time based or perhaps the strategy could depend
on the package being installed (critical packages like libc6 could
use fsync for every file, but all these intermediate fsync's are
useless for packages that can simply be reinstalled in case of
crash or hardware related failure).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

[Marco d'Itri]

On Oct 14, Vincent Lefevre  wrote:

> > This is not "the Debian FAQ" but "the DPKG FAQ", which has been known to 
> > recommend awful things in the past.
> But it is still considered in the present times by the dpkg developers.
> Bug 923423 was closed several hours ago based on this dpkg FAQ:
This is obviously convenient on Guillem's part, but we have to optimize 
systems by default for the general case and not just for dpkg -i.

> (the issue here is that there is a fsync for *every* file, so that for
And at this point I am wondering if this is a good design, or if it 
would be better to use a different strategy.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

[Vincent Lefevre]

On 2022-10-14 11:59:09 +0200, Marco d'Itri wrote:
> On Oct 14, Vincent Lefevre  wrote:
> 
> > The /etc/fstab file is created using by default ext4 with just
> > the errors=remount-ro option. However, the Debian FAQ recommends
> > the nodelalloc mount option to avoid performance degradation and
> > preserve data safety:
> This is not "the Debian FAQ" but "the DPKG FAQ", which has been known to 
> recommend awful things in the past.

But it is still considered in the present times by the dpkg developers.
Bug 923423 was closed several hours ago based on this dpkg FAQ:


All these fsync()s you see in rapid succession are used as a
synchronization points, way after the data has been requested to be
synced to disk asynchronously via sync_file_range().

What this is trying to achieve is durability, so that dpkg can know
the data is on the disk, so that it can mark the package as installed.

This is explained on the dpkg FAQ:

  



(the issue here is that there is a fsync for *every* file, so that for
a package with many small files, such as texlive-fonts-extra, which
currently has more than 92000 files, this can be awfully slow).

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

[Marco d'Itri]

On Oct 14, Vincent Lefevre  wrote:

> The /etc/fstab file is created using by default ext4 with just
> the errors=remount-ro option. However, the Debian FAQ recommends
> the nodelalloc mount option to avoid performance degradation and
> preserve data safety:
This is not "the Debian FAQ" but "the DPKG FAQ", which has been known to 
recommend awful things in the past.

There is a reason if delayed allocation is enabled by default and this 
should not be changed without a detailed analysis.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

[nick black]

i'm pretty sure that the corruption issues leading to the
nodelalloc option were considered largely remedied by the
"auto_da_alloc" capability introduced (and enabled by default)
in 2.6.30? how would nodelalloc equal the performance of
delalloc? nodelalloc was all about reliability for programs that
weren't conforming to certain POSIX semantics, as i recall.

Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

[Vincent Lefevre]

Package: general
Severity: normal

The /etc/fstab file is created using by default ext4 with just
the errors=remount-ro option. However, the Debian FAQ recommends
the nodelalloc mount option to avoid performance degradation and
preserve data safety:

https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Why_is_dpkg_so_slow_when_using_new_filesystems_such_as_btrfs_or_ext4.3F

  For ext4, use instead the "nodelalloc" mount option, which should
  fix both the performance degradation and the data safety issues,
  and not for just dpkg, but for any application in the system.

So this option should be used by default for btrfs and ext4.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)