On Tue, Feb 26, 2008 at 2:07 AM, Tomasz Chmielewski <[EMAIL PROTECTED]> wrote: > But I didn't have IO::Dirent installed, thanks for the hint. Let's hope > the list of directories in "trash" will keep decreasing now. Right now, > I have almost 100 directories there, and it is growing each day a bit. > > Although - with IO::Dirent "wa" is now 100% almost all the time, and the > system feels much slower. Hm. Let's hope it's coincidence, and assume > the system was just committing a big write...
I would have thought that if nightly is running all the time, IO wait time would have been 100% with or without IO::Dirent since you appear to be IO bound 100% of the time. > > I did find it odd that you said after dropping the caches deletes ran > > fairly quickly for a while, then slowed down. This seems to point to > > some sort of kernel bug. > > No, actually, everything is fine here. > > To remove anything, kernel has to read directory structures etc. first. > Then, it can "remove" lots of files really fast without committing > anything to the disk (yet) - everything stays in memory. > Once kernel feels the pressure on memory, it will start committing the > changes (file removals) to the disk. > This is where seek problem leads to poor performance: write commits > compete with reads, so there is lots of seek. Right, and I would expect the kernel to be flushing out those writes at very small intervals (whatever the default is, I think 5 seconds) compared to how often reads are happening so the effect on those writes should be small. But if the writes are going out very frequently, then yes, that would be an issue and really increase the seek load. > I managed to get around this a bit by increasing commit= ext3 mount > option, and also, setting these values (and putting 3 GB of RAM into the > machine): > > echo 50 > /proc/sys/vm/dirty_ratio > echo 50 > /proc/sys/vm/dirty_background_ratio > echo 6000 > /proc/sys/vm/dirty_writeback_centisecs > echo 6000 > /proc/sys/vm/dirty_expire_centisecs Yep, those should help. > I guess enabling write cache would help a lot, as then, the storage > could commit the changes out of order (less seek) - but I don't trust it > much. Where would you be enabling the write cache? It looks like you are using iscsi (I'm not familar with it myself). If it's a battery-backed write cache on the controller, then yes, that would be very helpful and should be safe. Can you give us more details on your disk array? Controller, disks, RAID layout, ext3 fs creation options, etc... I noticed in your linux-kernel thread that you said that large writes are are very slow (1MB/s). That certainly sounds abnormal. I would expect at least 25MB/s and with an array that big, you probably have at least 3 disks so it should be at least 2-3 times faster than that for writing large files. Have you also tried mounting the filesystem in data=writeback mode? Accord to this whitepaper[1] it should significantly improve the performance of creating/deleting small files at the expense of some data integrity in the case of a crash (which should not be a big deal since BackupPC does a good job of verifying the pool during backups). -Dave [1] http://www.redhat.com/support/wpapers/redhat/ext3/tuning.html ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
