Russell Coker russ...@coker.com.au writes:
I recently had a situation where I was doing a backup to a USB flash device
and I decided to install some Debian packages. The sync() didn't complete
until the backup completed because the write-back buffers were never empty!
Which is odd because
On Sat, 19 Mar 2011, Goswin von Brederlow goswin-...@web.de wrote:
I recently had a situation where I was doing a backup to a USB flash
device and I decided to install some Debian packages. The sync()
didn't complete until the backup completed because the write-back
buffers were never
Marius Vollmer marius.voll...@nokia.com writes:
ext Chow Loong Jin hyper...@ubuntu.com writes:
Could we somehow avoid using sync()? sync() syncs all mounted filesystems,
which
isn't exactly very friendly when you have a few slow-syncing filesystems like
btrfs (or even NFS) mounted.
Hmm,
Adrian von Bidder avbid...@fortytwo.ch writes:
On Wednesday 02 March 2011 17.02:11 Marius Vollmer wrote:
- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
big sync() right before carefully writing
On Fri, 18 Mar 2011, Goswin von Brederlow goswin-...@web.de wrote:
On a machine with lots of RAM (== disk cache...) and high I/O load, you
don't want to do a (global!) sync(). This can totally kill the machine
for 20min or more and is a big no go.
-- vbi
Then don't use the option.
On Fri, Mar 18, 2011 at 12:11 AM, Russell Coker russ...@coker.com.au wrote:
Then don't use the option. It should definetly be an option:
It's a pity that there is no kernel support for synching one filesystem (or
maybe a few filesystems).
That'd be only a partial work around. Even with a
On Fri, 18 Mar 2011, Russell Coker wrote:
On Fri, 18 Mar 2011, Goswin von Brederlow goswin-...@web.de wrote:
On a machine with lots of RAM (== disk cache...) and high I/O load, you
don't want to do a (global!) sync(). This can totally kill the machine
for 20min or more and is a big no
ext Chow Loong Jin hyper...@ubuntu.com writes:
Could we somehow avoid using sync()? sync() syncs all mounted filesystems,
which
isn't exactly very friendly when you have a few slow-syncing filesystems like
btrfs (or even NFS) mounted.
Hmm, right. We could keep a list of all files that need
Hi Marius,
no need to CC Guillem privately, the dpkg maintainers are reachable at
debian-d...@lists.debian.org. :)
On Wed, 02 Mar 2011, Marius Vollmer wrote:
- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
ext Raphael Hertzog hert...@debian.org writes:
On Wed, 02 Mar 2011, Marius Vollmer wrote:
- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
big sync() right before carefully writing /var/lib/dpkg/status.
The
On Thu, Mar 3, 2011 at 8:33 AM, Marius Vollmer marius.voll...@nokia.com wrote:
And in the big picture, all we need is some guarantee that renames are
comitted in order, and after the content of the file that is being
renamed. I have the impression that all reasonable filesystems give
that
On Thu, 03 Mar 2011, Marius Vollmer wrote:
ext Raphael Hertzog hert...@debian.org writes:
On Wed, 02 Mar 2011, Marius Vollmer wrote:
- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
big sync() right
I have another proposal. It looks like right now dpkg extracts all of
the files in the archive, then for each one, calls fsync() then
rename(). Because this is done serially for each file in the archive,
it forces small, out of order writes that cause extra seeking and queue
plugging. It would
Hi,
On Thu, 03 Mar 2011, Phillip Susi wrote:
I have another proposal. It looks like right now dpkg extracts all of
the files in the archive, then for each one, calls fsync() then
rename(). Because this is done serially for each file in the archive,
it forces small, out of order writes that
On Thu, 2011-03-03 at 18:49:44 +0100, Raphael Hertzog wrote:
On Thu, 03 Mar 2011, Phillip Susi wrote:
It would be much better to use aio to queue up all of the syncs at once,
so that the elevator can coalesce and reorder them for optimal writing.
I'm not convinced it would help. You're
On 3/3/2011 12:49 PM, Raphael Hertzog wrote:
That's wrong. The writeback is initiated before the fsync() so the
filesystem can order the write how it wants.
Don't you mean it MAY be initiated if the cache decides there is enough
memory pressure? I don't know of any other call besides fsync and
On Thu, Mar 3, 2011 at 7:32 PM, Phillip Susi ps...@cfl.rr.com wrote:
And we use some linux specific ioctl to avoid that fragmentation.
Could you be more specific?
sync_file_range(fd.a, 0, 0, SYNC_FILE_RANGE_WRITE);
sync_file_range(fd.a, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE);
Olaf
--
To
On 3/3/2011 1:30 PM, Guillem Jover wrote:
Actually, this was discarded early on, as Linux does not implement
aio_fsync() for any file system. Also the interface is quite cumbersome
as it requires to keep state for each aio operation, and using SA_SIGINFO
(which is not yet available
On 3/3/2011 1:32 PM, Phillip Susi wrote:
Don't you mean it MAY be initiated if the cache decides there is enough
memory pressure? I don't know of any other call besides fsync and
friends to force the writeback so before that is called, it could ( and
likely is if you have plenty of memory )
Hi,
I have recently been looking into where dpkg spends most of its time
when installing very many small packages, and came up with the following
idea to speed it up.
- Most of the time is spent writing files very carefully, a lot of them
in /var/lib/dpkg/updates.
- We can avoid this by
Hi,
On Thursday 03,March,2011 12:02 AM, Marius Vollmer wrote:
[...]
- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
big sync() right before carefully writing /var/lib/dpkg/status.
Could we somehow avoid
Chow Loong Jin wrote the following on 02.03.2011 18:51
Hi,
On Thursday 03,March,2011 12:02 AM, Marius Vollmer wrote:
[...]
- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
big sync() right before carefully
On Thu, Mar 03, 2011 at 01:51:35AM +0800, Chow Loong Jin wrote:
Hi,
On Thursday 03,March,2011 12:02 AM, Marius Vollmer wrote:
[...]
- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
big sync() right
On Wed, Mar 02, 2011 at 08:13:06PM +, Roger Leigh wrote:
On Thu, Mar 03, 2011 at 01:51:35AM +0800, Chow Loong Jin wrote:
Hi,
On Thursday 03,March,2011 12:02 AM, Marius Vollmer wrote:
[...]
- Instead, we move all packages that are to be unpacked into
half-installed /
On Wed, Mar 02, 2011 at 08:13:06PM +, Roger Leigh wrote:
Btrfs is quite simply awful in chroots at present, and it seems
--force-unsafe-io doesn't really seem to help massively either.
It's dog slow--it's quicker to untar a chroot onto ext3 than to
bother with Btrfs.
Because unsafe-io
On Wednesday 02 March 2011 17.02:11 Marius Vollmer wrote:
- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
big sync() right before carefully writing /var/lib/dpkg/status.
You don't want to do this. While
Yodel again!
On Wednesday 02 March 2011 17.02:11 Marius Vollmer wrote:
It shows a speed up between factor six and two in our environment (ext4
on a slowish flash drive) . I am not sure whether messing with the
fundamentals of dpkg is worth a factor of two in performance
To not be all
On Thursday 03,March,2011 02:45 PM, Marius Vollmer wrote:
ext Chow Loong Jin hyper...@ubuntu.com writes:
Could we somehow avoid using sync()? sync() syncs all mounted filesystems,
which
isn't exactly very friendly when you have a few slow-syncing filesystems like
btrfs (or even NFS)
ext Chow Loong Jin hyper...@ubuntu.com writes:
I remember seeing there being some list of files to be fsynced in one of the
older dpkgs. It's probably that which led to the ext4 slowdown [...]
Hmm, performance is the ultimate reason for doing all this, but right
now, I am mostly interested in
29 matches
Mail list logo
