Slow package database

2007-03-30 Thread Christoph Haas
Dear list... what has always annoyed me is now ready to be asked on this list. :) It appears like apt-get/aptitude/dpkg dealing with the list/database of installed packages is terribly slow at times. I have roughly 1800 packages installed and it sometimes takes 20-30 seconds to install a single

Re: Slow package database

2007-04-02 Thread Christoph Haas
On Friday 30 March 2007 22:25, Florian Weimer wrote: > * Christoph Haas: > > What might be the cause? Is there some fragmentation effect? > > It's probably ext3's directory hashing. It tries to access the files > in /var/lib/dpkg/info in hash order, which leads to essentially random > disk I/O. I

Re: Slow package database

2007-04-02 Thread Tim Dijkstra
On Mon, 2 Apr 2007 11:10:55 +0200 Christoph Haas <[EMAIL PROTECTED]> wrote: > Is there anything that I can do about it? Tuning the ext3 partition? Or is > somebody capable of turning the info directory into - say - an sqlite > database or in future Debian releases? There is a thread on debian-d

Re: Slow package database

2007-03-30 Thread Roberto C . Sánchez
On Fri, Mar 30, 2007 at 03:39:29PM +0200, Christoph Haas wrote: > Dear list... > > what has always annoyed me is now ready to be asked on this list. :) > It appears like apt-get/aptitude/dpkg dealing with the list/database of > installed packages is terribly slow at times. I have roughly 1800 pac

Re: Slow package database

2007-03-30 Thread Christoph Haas
On Friday 30 March 2007 15:43, Roberto C. Sánchez wrote: > On Fri, Mar 30, 2007 at 03:39:29PM +0200, Christoph Haas wrote: > > It appears like apt-get/aptitude/dpkg dealing with the list/database > > of installed packages is terribly slow at times. > > Interesting. I have ~1600 packages installed

Re: Slow package database

2007-03-30 Thread Roberto C . Sánchez
On Fri, Mar 30, 2007 at 04:11:46PM +0200, Christoph Haas wrote: > On Friday 30 March 2007 15:43, Roberto C. Sánchez wrote: > > On Fri, Mar 30, 2007 at 03:39:29PM +0200, Christoph Haas wrote: > > > It appears like apt-get/aptitude/dpkg dealing with the list/database > > > of installed packages is te

Re: Slow package database

2007-03-30 Thread Margarita Manterola
On 3/30/07, Christoph Haas <[EMAIL PROTECTED]> wrote: what has always annoyed me is now ready to be asked on this list. :) It appears like apt-get/aptitude/dpkg dealing with the list/database of installed packages is terribly slow at times. I have roughly 1800 packages installed and it sometimes

Re: Slow package database

2007-03-30 Thread nextime
debian sid installed about 9 years ago, actually on amd64 (but with 32bit kernel and 32bit userland ) sempron 3300+ with 1 gigs of ram. The system is on a dm-crypted in aes256 device over software raid1 composed by two slow ide disks, (so, I/O is *VERY* slow), i have something like 2500 packages

Re: Slow package database

2007-03-30 Thread Loïc Minier
On Fri, Mar 30, 2007, Christoph Haas wrote: > It appears like apt-get/aptitude/dpkg dealing with the list/database of > installed packages is terribly slow at times. I think dpkg spends a lot of time reading all /var/lib/dpkg/info/*.list files. These are not big and this is typically in cache

Re: Slow package database

2007-03-30 Thread Joey Hess
Loïc Minier wrote: > I think dpkg spends a lot of time reading all /var/lib/dpkg/info/*.list > files. These are not big and this is typically in cache between dpkg > runs, and I think this is the reason that cold dpkg runs are very very > slow. It can also lead to a kind of "fragmentation", s

Re: Slow package database

2007-03-30 Thread Nico Golde
Hi, * Loïc Minier <[EMAIL PROTECTED]> [2007-03-30 20:29]: > On Fri, Mar 30, 2007, Christoph Haas wrote: > > It appears like apt-get/aptitude/dpkg dealing with the list/database of > > installed packages is terribly slow at times. > > I think dpkg spends a lot of time reading all /var/lib/dpkg/in

Re: Slow package database

2007-03-30 Thread Loïc Minier
On Fri, Mar 30, 2007, Joey Hess wrote: > It can also lead to a kind of "fragmentation", since during an initial > install these files tend to be fairly close together on disk while over > time new files will be scattered about and more seeking needed to read > them all. Indeed it accounts for som

Re: Slow package database

2007-03-30 Thread Adam Borowski
On Fri, Mar 30, 2007 at 08:57:03PM +0200, Loïc Minier wrote: > Indeed it accounts for some part of the problem; after I cloned and > replaced my /var/lib/dpkg/info tree with the copy, the figure dropped > from 22 seconds to 15 seconds. It's not that. It's /var/lib/dpkg/available. Try this: f

Re: Slow package database

2007-03-30 Thread Florian Weimer
* Christoph Haas: > What might be the cause? Is there some fragmentation effect? It's probably ext3's directory hashing. It tries to access the files in /var/lib/dpkg/info in hash order, which leads to essentially random disk I/O. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject o

Re: Slow package database

2007-03-30 Thread Brad Sawatzky
On Fri, 30 Mar 2007, Adam Borowski wrote: > It's not that. It's /var/lib/dpkg/available. > > Try this: > > for x in /var/lib/apt/lists/*Packages;do dpkg --merge-avail "$x";done > (or "dselect update") > Install+purge of an empty package: > 9-ish seconds on one box (my home desktop, years

Re: Slow package database

2007-03-30 Thread Ganesan Rajagopal
> "Roberto" == Roberto C Sánchez <[EMAIL PROTECTED]> writes: > Interesting. I have ~1600 packages installed on my development machine > and I do not experience the slowness you talk about. It was installed > about 3.5 years ago. > How much RAM/CPU does the machine have? How fast are the di

Re: Slow package database

2007-03-31 Thread Sam Hocevar
On Fri, Mar 30, 2007, Adam Borowski wrote: > On Fri, Mar 30, 2007 at 08:57:03PM +0200, Loïc Minier wrote: > > Indeed it accounts for some part of the problem; after I cloned and > > replaced my /var/lib/dpkg/info tree with the copy, the figure dropped > > from 22 seconds to 15 seconds. > > It's

Re: Slow package database

2007-03-31 Thread Florian Weimer
* Adam Borowski: > On Fri, Mar 30, 2007 at 08:57:03PM +0200, Loïc Minier wrote: >> Indeed it accounts for some part of the problem; after I cloned and >> replaced my /var/lib/dpkg/info tree with the copy, the figure dropped >> from 22 seconds to 15 seconds. > > It's not that. It's /var/lib/dpk

Re: Slow package database

2007-03-31 Thread Loïc Minier
On Fri, Mar 30, 2007, Adam Borowski wrote: > > Indeed it accounts for some part of the problem; after I cloned and > > replaced my /var/lib/dpkg/info tree with the copy, the figure dropped > > from 22 seconds to 15 seconds. > It's not that. It's /var/lib/dpkg/available. Err, I timed before an