Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-23 Thread Miles Bader
Bilal mk writes: > I am using xfs filesystem and also did the fsck. DMA is enabled. > Also perfomed xfs defragmentation( xfs_fsr). But still an issue not only rm > -rf but also cp command Traditionally XFS is super slow when deleting lots of little files -- much, _much_, slower than ext3, for ins

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-18 Thread Joe Pfeiffer
Is there any chance the OP might have the filesystem mounted with the 'sync' option? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1bobsw2fc7@pfeifferfamily.net

Re: Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-17 Thread Bob Proulx
Christofer C. Bell wrote: > cbell@circe:~/test$ time find rm -type f -exec rm {} \+ There isn't any need to escape the '+' character. time find rm -type f -exec rm {} + > It doesn't seem possible to run a similar test for unlink as it > appears it only operates on 1 file at a time. So it does

Re: Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-16 Thread Christofer C. Bell
On Wed, Feb 15, 2012 at 4:51 PM, Clive Standbridge wrote: >> But may provide some benefit when removing a large number (3) of >> files (at least empty ones). >> >> cbell@circe:~/test$ time find rm -type f -exec rm {} \; >> >> real              0m48.127s >> user              1m32.926s >> sys  

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-16 Thread Robert Brockway
On Thu, 16 Feb 2012, Bilal mk wrote: I am using xfs filesystem and also did the fsck. DMA is enabled. Also perfomed xfs defragmentation( xfs_fsr). But still an issue not only rm -rf but also cp command Until quite recently XFS was notable for being slow to delete. Others have noted that this

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-15 Thread Bilal mk
On Wed, Feb 15, 2012 at 12:14 PM, Bob Proulx wrote: > Bilal mk wrote: > > I tried to remove 5GB directory. In that directory around 3 files and > > directory. It will take more than 30 min to complete. > > A large number of files consuming a large number of blocks will take a > significant am

Re: Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-15 Thread Clive Standbridge
> But may provide some benefit when removing a large number (3) of > files (at least empty ones). > > cbell@circe:~/test$ time find rm -type f -exec rm {} \; > > real 0m48.127s > user 1m32.926s > sys 0m38.750s First thought - how much of that 48 second

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-15 Thread Bob Proulx
Christofer C. Bell wrote: > unlink is slower than rm removing a 1.5GB file (at least on ext3): > ... > I suspect that removing a large number of non-zero byte files will be > slower with unlink than rm. If it is then it is pointing to a kernel performance issue. Because there is very little diffe

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-15 Thread Christofer C. Bell
On Wed, Feb 15, 2012 at 1:38 AM, Jude DaShiell wrote: > Anyone heard of the unlink command? unlink is slower than rm removing a 1.5GB file (at least on ext3): cbell@circe:~$ time rm test1 real0m0.278s user0m0.000s sys 0m0.264s cbell@circe:~$ time unlink test2 real0m0.375s user

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-15 Thread Chris Davies
Jude DaShiell wrote: > Anyone heard of the unlink command? Yes. And your point is...? Chris -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/dvls09xg3r@news.roai

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-14 Thread Jude DaShiell
Anyone heard of the unlink command?On Wed, 15 Feb 2012, Stan Hoeppner wrote: > On 2/15/2012 12:55 AM, Jochen Spieker wrote: > > Bilal mk: > >> > >> I tried to remove 5GB directory. In that directory around 3 files and > >> directory. It will take more than 30 min to complete. > >> There is no

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-14 Thread Stan Hoeppner
On 2/15/2012 12:55 AM, Jochen Spieker wrote: > Bilal mk: >> >> I tried to remove 5GB directory. In that directory around 3 files and >> directory. It will take more than 30 min to complete. >> There is no other cpu intensive process running. After sometime it goes to >> D state and unbale to ki

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-14 Thread Jochen Spieker
Bilal mk: > > I tried to remove 5GB directory. In that directory around 3 files and > directory. It will take more than 30 min to complete. > There is no other cpu intensive process running. After sometime it goes to > D state and unbale to kii that process. Removing that many files is I/O bo

Re: rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-14 Thread Bob Proulx
Bilal mk wrote: > I tried to remove 5GB directory. In that directory around 3 files and > directory. It will take more than 30 min to complete. A large number of files consuming a large number of blocks will take a significant amount of time to process. That is all there is to it. Some files

rm -rf is too slow on large files and directory structure(Around 30000)

2012-02-14 Thread Bilal mk
Hello, I tried to remove 5GB directory. In that directory around 3 files and directory. It will take more than 30 min to complete. There is no other cpu intensive process running. After sometime it goes to D state and unbale to kii that process. I have also tried find with xargs method to rem