On 2015-04-13 16:28:27 -0600, Bob Proulx wrote: > Vincent Lefevre wrote: > > Loïc Grenié wrote: > > > Vincent Lefevre wrote: > > > > The problem is that this operation is (always?) very slow: something > > > > like 100 seconds (1 minute and 40 seconds). It has been reproducible > > > > for several months. The logs show nothing during this operation. > > > > > > > > Any idea? > > Of course immediately after a reboot the file system buffer cache is > completely clean. You can avoid the reboot and recreate this > situation by this: > > # echo 3 > /proc/sys/vm/drop_caches > > See this for the documentation: > > https://www.kernel.org/doc/Documentation/sysctl/vm.txt
Thanks for the reference (also available locally in linux-doc-* packages). > That would certainly be less disruptive to recreate the situation than > rebooting the entire system. > > > > Maybe the directory is very large (even though its empty). Try > > > > > > ls -ld tmp. > > > > > > and see if the file "tmp" is large. > > > > Thanks! I didn't know that (I thought that the file system would > > automatically "optimize" directories when files are removed, so > > I've never looked closely at their size). Indeed: > > > > ypig:~/eftests> ls -ld tmp > > drwxr-xr-x 2 vlefevre vlefevre 29655040 2015-04-13 15:25:55 tmp/ > > What type of file system is it? Does it have dir_index? Both ext3 > and ext4 support it. Others are similar. > > # tune2fs -l /dev/mapper/v1-var | grep --color Filesystem.features > Filesystem features: has_journal ext_attr resize_inode dir_index > filetype needs_recovery sparse_super large_file This is ext3 with dir_index: # tune2fs -l /dev/sda1 | grep Filesystem.features Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file And for the FS creation: Filesystem created: Mon Jan 4 16:26:16 2010 My machine is old, but I've never changed anything concerning the file system. > Without dir_index an ext filesystem with large directories is slow due > to the linear nature of directories. But with dir_index it should be > using a B-tree data structure and should be much faster. So, why is it slow? I also notice slowness with a large maildir directory: drwx------ 2 vlefevre vlefevre 8409088 2015-03-24 14:04:33 Mail/oldarc/cur/ In this one, the files are real (145400 files), but I have a Perl script that basically reads the headers and it takes a lot of time (several dozens of minutes) after a reboot or dropping the caches as you suggested above. With a second run of this script, it just takes 8 seconds. -- Vincent Lefèvre <vinc...@vinc17.net> - Web: <https://www.vinc17.net/> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150414130659.gb25...@ypig.lip.ens-lyon.fr