On Sat, 19 Mar 2011, Goswin von Brederlow wrote:
> > I recently had a situation where I was doing a backup to a USB flash
> > device and I decided to install some Debian packages. The sync()
> > didn't complete until the backup completed because the write-back
> > buffers were never empty!
>
>
Russell Coker writes:
> I recently had a situation where I was doing a backup to a USB flash device
> and I decided to install some Debian packages. The sync() didn't complete
> until the backup completed because the write-back buffers were never empty!
Which is odd because I've used sync() w
On Fri, 18 Mar 2011, Russell Coker wrote:
> On Fri, 18 Mar 2011, Goswin von Brederlow wrote:
> > > On a machine with lots of RAM (== disk cache...) and high I/O load, you
> > > don't want to do a (global!) sync(). This can totally kill the machine
> > > for 20min or more and is a big no go.
> >
On Fri, Mar 18, 2011 at 12:11 AM, Russell Coker wrote:
>> Then don't use the option. It should definetly be an option:
>
> It's a pity that there is no kernel support for synching one filesystem (or
> maybe a few filesystems).
That'd be only a partial work around. Even with a single fs one big
sy
On Fri, 18 Mar 2011, Goswin von Brederlow wrote:
> > On a machine with lots of RAM (== disk cache...) and high I/O load, you
> > don't want to do a (global!) sync(). This can totally kill the machine
> > for 20min or more and is a big no go.
> >
> > -- vbi
>
> Then don't use the option. It sh
Adrian von Bidder writes:
> On Wednesday 02 March 2011 17.02:11 Marius Vollmer wrote:
>> - Instead, we move all packages that are to be unpacked into
>> half-installed / reinstreq before touching the first one, and put a
>> big sync() right before carefully writing /var/lib/dpkg/status.
>
> Y
Marius Vollmer writes:
> ext Chow Loong Jin writes:
>
>> Could we somehow avoid using sync()? sync() syncs all mounted filesystems,
>> which
>> isn't exactly very friendly when you have a few slow-syncing filesystems like
>> btrfs (or even NFS) mounted.
>
> Hmm, right. We could keep a list of
On 3/3/2011 1:32 PM, Phillip Susi wrote:
> Don't you mean it MAY be initiated if the cache decides there is enough
> memory pressure? I don't know of any other call besides fsync and
> friends to force the writeback so before that is called, it could ( and
> likely is if you have plenty of memory
On 3/3/2011 1:30 PM, Guillem Jover wrote:
> Actually, this was discarded early on, as Linux does not implement
> aio_fsync() for any file system. Also the interface is quite cumbersome
> as it requires to keep state for each aio operation, and using SA_SIGINFO
> (which is not yet available everywhe
On Thu, Mar 3, 2011 at 7:32 PM, Phillip Susi wrote:
>> And we use some linux specific ioctl to avoid that fragmentation.
>
> Could you be more specific?
sync_file_range(fd.a, 0, 0, SYNC_FILE_RANGE_WRITE);
sync_file_range(fd.a, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE);
Olaf
--
To UNSUBSCRIBE,
On 3/3/2011 12:49 PM, Raphael Hertzog wrote:
> That's wrong. The writeback is initiated before the fsync() so the
> filesystem can order the write how it wants.
Don't you mean it MAY be initiated if the cache decides there is enough
memory pressure? I don't know of any other call besides fsync an
On Thu, 2011-03-03 at 18:49:44 +0100, Raphael Hertzog wrote:
> On Thu, 03 Mar 2011, Phillip Susi wrote:
> > It would be much better to use aio to queue up all of the syncs at once,
> > so that the elevator can coalesce and reorder them for optimal writing.
>
> I'm not convinced it would help. You'
Hi,
On Thu, 03 Mar 2011, Phillip Susi wrote:
> I have another proposal. It looks like right now dpkg extracts all of
> the files in the archive, then for each one, calls fsync() then
> rename(). Because this is done serially for each file in the archive,
> it forces small, out of order writes th
I have another proposal. It looks like right now dpkg extracts all of
the files in the archive, then for each one, calls fsync() then
rename(). Because this is done serially for each file in the archive,
it forces small, out of order writes that cause extra seeking and queue
plugging. It would b
On Thu, 03 Mar 2011, Marius Vollmer wrote:
> ext Raphael Hertzog writes:
>
> > On Wed, 02 Mar 2011, Marius Vollmer wrote:
> >> - Instead, we move all packages that are to be unpacked into
> >> half-installed / reinstreq before touching the first one, and put a
> >> big sync() right before car
On Thu, Mar 3, 2011 at 8:33 AM, Marius Vollmer wrote:
> And in the big picture, all we need is some guarantee that renames are
> comitted in order, and after the content of the file that is being
> renamed. I have the impression that all reasonable filesystems give
> that guarantee now, no?
No,
ext Raphael Hertzog writes:
> On Wed, 02 Mar 2011, Marius Vollmer wrote:
>> - Instead, we move all packages that are to be unpacked into
>> half-installed / reinstreq before touching the first one, and put a
>> big sync() right before carefully writing /var/lib/dpkg/status.
>
> The big sync()
Hi Marius,
no need to CC Guillem privately, the dpkg maintainers are reachable at
debian-d...@lists.debian.org. :)
On Wed, 02 Mar 2011, Marius Vollmer wrote:
> - Instead, we move all packages that are to be unpacked into
> half-installed / reinstreq before touching the first one, and put a
>
ext Chow Loong Jin writes:
> Could we somehow avoid using sync()? sync() syncs all mounted filesystems,
> which
> isn't exactly very friendly when you have a few slow-syncing filesystems like
> btrfs (or even NFS) mounted.
Hmm, right. We could keep a list of all files that need fsyncing, and
t
ext Chow Loong Jin writes:
> I remember seeing there being some list of files to be fsynced in one of the
> older dpkgs. It's probably that which led to the ext4 slowdown [...]
Hmm, performance is the ultimate reason for doing all this, but right
now, I am mostly interested in whether my changes
On Thursday 03,March,2011 02:45 PM, Marius Vollmer wrote:
> ext Chow Loong Jin writes:
>
>> Could we somehow avoid using sync()? sync() syncs all mounted filesystems,
>> which
>> isn't exactly very friendly when you have a few slow-syncing filesystems like
>> btrfs (or even NFS) mounted.
>
> Hm
Yodel again!
On Wednesday 02 March 2011 17.02:11 Marius Vollmer wrote:
> It shows a speed up between factor six and two in our environment (ext4
> on a slowish flash drive) . I am not sure whether messing with the
> fundamentals of dpkg is worth a factor of two in performance
To not be all negati
On Wednesday 02 March 2011 17.02:11 Marius Vollmer wrote:
> - Instead, we move all packages that are to be unpacked into
> half-installed / reinstreq before touching the first one, and put a
> big sync() right before carefully writing /var/lib/dpkg/status.
You don't want to do this. While prod
On Wed, Mar 02, 2011 at 08:13:06PM +, Roger Leigh wrote:
> Btrfs is quite simply awful in chroots at present, and it seems
> --force-unsafe-io doesn't really seem to help massively either.
> It's dog slow--it's quicker to untar a chroot onto ext3 than to
> bother with Btrfs.
Because unsafe-io
On Wed, Mar 02, 2011 at 08:13:06PM +, Roger Leigh wrote:
> On Thu, Mar 03, 2011 at 01:51:35AM +0800, Chow Loong Jin wrote:
> > Hi,
> >
> > On Thursday 03,March,2011 12:02 AM, Marius Vollmer wrote:
> > > [...]
> > > - Instead, we move all packages that are to be unpacked into
> > > half-insta
On Thu, Mar 03, 2011 at 01:51:35AM +0800, Chow Loong Jin wrote:
> Hi,
>
> On Thursday 03,March,2011 12:02 AM, Marius Vollmer wrote:
> > [...]
> > - Instead, we move all packages that are to be unpacked into
> > half-installed / reinstreq before touching the first one, and put a
> > big sync()
Chow Loong Jin wrote the following on 02.03.2011 18:51
> Hi,
>
> On Thursday 03,March,2011 12:02 AM, Marius Vollmer wrote:
>> [...]
>> - Instead, we move all packages that are to be unpacked into
>> half-installed / reinstreq before touching the first one, and put a
>> big sync() right before
Hi,
On Thursday 03,March,2011 12:02 AM, Marius Vollmer wrote:
> [...]
> - Instead, we move all packages that are to be unpacked into
> half-installed / reinstreq before touching the first one, and put a
> big sync() right before carefully writing /var/lib/dpkg/status.
Could we somehow avoid u
28 matches
Mail list logo
