Anyone heard of the unlink command?On Wed, 15 Feb 2012, Stan Hoeppner wrote:
> On 2/15/2012 12:55 AM, Jochen Spieker wrote: > > Bilal mk: > >> > >> I tried to remove 5GB directory. In that directory around 30000 files and > >> directory. It will take more than 30 min to complete. > >> There is no other cpu intensive process running. After sometime it goes to > >> D state and unbale to kii that process. > > > > Removing that many files is I/O bound. > > This isn't correct. Removing a kernel source tree is fast, and it > contains on the order of 4500 directories and 50K files. > > EXT4 can 'rm -rf' the kernel source in 2-3 seconds. XFS prior to > delaylog could take a minute or two, with delaylog it's 4 seconds. > > So that's 2-4 seconds to remove a directory tree of 50k files. The OP's > system is taking forever then freezing. So if it's EXT4 he's using, > this isn't an IO problem but a bug, or something else, maybe bad hardware. > > > The CPU doesn't play any > > significant role in it. > > CPU, and memory, play a very significant role here if the filesystem is > XFS. Delayed logging takes all of the log journal writes and buffers > them, so duplicate changes to the metadata are rolled up into a single > physical IO. With enough metadata changes it becomes CPU bound. But > we're talking lots of metadata if we have a modern fast CPU. > > I can't speak to EXTx behavior in this regard as I'm not familiar with it. > > > The only way to speed things up is to either get > > faster storage (an SSD with high IOPS value for random writing) or you > > can try another filesystem. IIRC XFS is good at what you are doing. I > > cannot recommend it, though, as I don't have any recent experience with > > it. > > XFS is absolutely *horrible* with this workload prior to kernel 2.6.35, > when delayed logging was introduced. So if this is your workload, and > you want XFS, you need mainline 2.6.35, better still 3.0.0. > > ---------------------------------------------------------------- Jude <jdashiel-at-shellworld-dot-net> <http://www.shellworld.net/~jdashiel/nj.html> -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/alpine.bsf.2.01.1202150237390....@freire1.furyyjbeyq.arg