Hi, Just like Malcolm suggested, before I implemented Robinhood, I built the list and removed files (that were not accessed in more than 31 days). I found it much faster than all other methods.
/usr/bin/lfs find /scratch/${folder} -A +31 -p -t f | xargs -0 -n 10 -P 8 /bin/rm 10 and 8 for flags n and p worked well for me on our system. Sreedhar. New York University. On Friday, July 10, 2015, Cowe, Malcolm J <malcolm.j.c...@intel.com <javascript:_e(%7B%7D,'cvml','malcolm.j.c...@intel.com');>> wrote: > There’s something in the rm command that makes recursive deletes really > expensive, although I don’t know why. I’ve found in the past that even > running a regular find ... –exec rm {} \; has been quicker. Running lfs > find to build the file list would presumably be quicker still. > > > > Malcolm. > > > > > > *From:* lustre-discuss [mailto:lustre-discuss-boun...@lists.lustre.org] *On > Behalf Of *Andrus, Brian Contractor > *Sent:* Saturday, July 11, 2015 8:05 AM > *To:* lustre-discuss@lists.lustre.org > *Subject:* [lustre-discuss] Removing large directory tree > > > > All, > > > > I understand that doing recursive file operations can be taxing on lustre. > > So, I wonder if there is a preferred performance-minded way to remove an > entire directory tree that is several TB in size. > > The standard rm -rf ./dir seems to spike the cpu usage on my OSSes where > it sits and sometimes causes clients to be evicted. > > > > Brian Andrus > > ITACS/Research Computing > > Naval Postgraduate School > > Monterey, California > > voice: 831-656-6238 > > > > _______________________________________________ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > -- Sreedhar Manchu HPC Systems Administrator eSystems & Research Services New York University, New York 10012 *http://unixoperator.blogspot.com <http://unixoperator.blogspot.com>* *https://wikis.nyu.edu/display/~sm4082 <https://wikis.nyu.edu/display/~sm4082>*
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org