Re: [PATCH] Use FIBMAP to sort .list files before scanning

2009-10-05 Thread Goswin von Brederlow
Guillem Jover guil...@debian.org writes: Hi Morten! On Sat, 2009-09-05 at 01:34:50 +0200, Morten Hustveit wrote: When running dpkg from a cold cache on a system where /var/lib/dpkg/info/ lies on a harddisk, a lot of time is spent waiting for seeks between (typically) thousands of files.

Re: [PATCH] Use FIBMAP to sort .list files before scanning

2009-10-05 Thread Florian Weimer
* Guillem Jover: On kFreeBSD there's FIOGETLBA, but it seems to only be implemented for the cd9660 file system, so not of much use. Also AFAIU from taking a look at Linux, FIBMAP is being phased out by FIEMAP? FIBMAP is the correct system call here because FIEMAP does unnecessary work. It's

Re: [PATCH] Use FIBMAP to sort .list files before scanning

2009-10-05 Thread Florian Weimer
* Goswin von Brederlow: Wouldn't it be better to read the files asynchronous and let the underlying elevator code sort them into physical block order? Linux doesn't support asynchronous I/O which doesn't bypass the cache, so it's pretty much unusable for this purpose. It's also doubtful if

Re: [PATCH] Use FIBMAP to sort .list files before scanning

2009-10-05 Thread Guillem Jover
On Mon, 2009-10-05 at 11:56:25 +, Florian Weimer wrote: * Guillem Jover: On kFreeBSD there's FIOGETLBA, but it seems to only be implemented for the cd9660 file system, so not of much use. Also AFAIU from taking a look at Linux, FIBMAP is being phased out by FIEMAP? FIBMAP is the

Re: [PATCH] Use FIBMAP to sort .list files before scanning

2009-10-05 Thread Tollef Fog Heen
]] Goswin von Brederlow | PS: How much does backup+restore of /var/lib/dpkg/info give you and | how much does the ordering give you after that? I tested a backup + restore on my laptop with ext4, and dpkg -S went from ~30s to ~20s, so it's a significant improvement, but still slow. (Sorry, I

Re: [PATCH] Use FIBMAP to sort .list files before scanning

2009-10-04 Thread Guillem Jover
Hi Morten! On Sat, 2009-09-05 at 01:34:50 +0200, Morten Hustveit wrote: When running dpkg from a cold cache on a system where /var/lib/dpkg/info/ lies on a harddisk, a lot of time is spent waiting for seeks between (typically) thousands of files. This patch ensures that the package iterator

Re: [PATCH] Use FIBMAP to sort .list files before scanning

2009-09-06 Thread Florian Weimer
* Morten Hustveit: I haven't tried this, because it would require much more changes to the dpkg code. The FIBMAP every .list file step takes less than one second when operating from a cold disk cache, probably because inodes aren't as scattered as file contents. Very interesting. It turns

Re: [PATCH] Use FIBMAP to sort .list files before scanning

2009-09-05 Thread Florian Weimer
* Morten Hustveit: When running dpkg from a cold cache on a system where /var/lib/dpkg/info/ lies on a harddisk, a lot of time is spent waiting for seeks between (typically) thousands of files. This patch ensures that the package iterator used by ensure_allinstfiles_available returns the

[PATCH] Use FIBMAP to sort .list files before scanning

2009-09-04 Thread Morten Hustveit
When running dpkg from a cold cache on a system where /var/lib/dpkg/info/ lies on a harddisk, a lot of time is spent waiting for seeks between (typically) thousands of files. This patch ensures that the package iterator used by ensure_allinstfiles_available returns the packages sorted by their