On 2013-10-27 Richard W.M. Jones wrote:
> On Sat, Oct 26, 2013 at 08:06:45PM +0300, Lasse Collin wrote:
> > > - For reasons I don't understand, both regular xzcat and pxzcat
> > > cause the output file to be flushed to disk after the program
> > > exits. This causes any program which consumes the output of the
> > > file to slow down.
> >
> > I have no idea. I see you committed something that seems to be
> > related to this after your email. With a quick reading I don't
> > understand it well, it seems to be working around some issue with
> > ftruncate() with ext4.
>
> That's right. It turned out to be a misfeature in ext4: if you
> truncate a file from a non-zero size down to a zero size, ext4 flags
> the file and flushes it on close. ("Truncate" includes both O_TRUNC
> and ftruncate). This can be disabled with the noauto_da_alloc mount
> option, but of course that is not the default.
If I remember correctly, that "misfeature" was added because there are
too many programs that write config files by overwriting the old files
with O_TRUNC. By flushing quickly ext4 tries to avoid zero-length
config files after a crash or a power failure.
With xz there's nothing I can do to work around the flushes (and I'm not
sure if I wanted do anything for this even if I could). The truncation
is done by the shell when one redirects the output from xz to a file,
so to avoid the flushes one would need to patch the shell.
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
