um, this is what xargs -P is for ... -- Dan.
On Thu, Jul 14, 2011 at 07:24:52PM +0400, Jim Klimov wrote: > 2011-07-14 15:48, Frank Van Damme ?????: >> It seems counter-intuitive - you'd say: concurrent disk access makes >> things only slower - , but it turns out to be true. I'm deleting a >> dozen times faster than before. How completely ridiculous. Thank you >> :-) > > Well, look at it this way: it is not only about singular disk accesses > (i.e. unlike other FSes, you do not in-place modify a directory entry), > with ZFS COW it is about rewriting a tree of block pointers, with any > new writes going into free (unreferenced ATM) disk blocks anyway. > > So by hoarding writes you have a chance to reduce mechanical > IOPS required for your tasks. Until you run out of RAM ;) > > Just in case it helps, to quickly fire up removals of the specific > directory > after yet another reboot of the box, and not overwhelm it with hundreds > of thousands queued "rm"processes either, I made this script as /bin/RM: > > === > #!/bin/sh > > SLEEP=10 > [ x"$1" != x ] && SLEEP=$1 > > A=0 > # To rm small files: find ... -size -10 > find /export/OLD/PATH/TO/REMOVE -type f | while read LINE; do > du -hs "$LINE" > rm -f "$LINE" & > A=$(($A+1)) > [ "$A" -ge 100 ] && ( date; while [ `ps -ef | grep -wc rm` -gt 50 ]; do > echo "Sleep $SLEEP..."; ps -ef | grep -wc rm ; sleep $SLEEP; ps -ef > | grep -wc rm; > done > date ) && A="`ps -ef | grep -wc rm`" > done ; date > === > > Essentially, after firing up 100 "rm attempts" it waits for the "rm" > process count to go below 50, then goes on. Sizing may vary > between systems, phase of the moon and computer's attitude. > Sometimes I had 700 processes stacked and processed quickly. > Sometimes it hung on 50... > > HTH, > //Jim > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
pgprXDuV2KRuK.pgp
Description: PGP signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
