Re: Migrate large amount of small files

2009-10-11 Thread Jin Guojun
ory naming, use Year-Month-week for auto directory naming. --- On Sun, 10/11/09, Mike Tancsa wrote: > From: Mike Tancsa > Subject: Re: Migrate large amount of small files > To: net.h...@m2k.com.tw, "freebsd-performance" > > Date: Sunday, October 11, 2009, 9:07 PM > A

Re: Migrate large amount of small files

2009-10-11 Thread Mike Tancsa
At 11:46 AM 10/10/2009, Ckcheng wrote: Hi all, Currently, I have a directory with over 5M small files (1~32K). Now, I want to transfer this directory to another machine and found that it's extremely slow and painful process. I tried the following method: It might help if you mount all -onoatime

Re: Migrate large amount of small files

2009-10-11 Thread Francisco Reyes
Ckcheng writes: 1. rsync 2. tar .. If this is a migration I find that tar to the local machine, copy over, restore, then rsync are likely the best options. In my experience copying lots of small files is going to take a long time, no matter which method you use. From all the combinations

Migrate large amount of small files

2009-10-10 Thread Ckcheng
Hi all, Currently, I have a directory with over 5M small files (1~32K). Now, I want to transfer this directory to another machine and found that it's extremely slow and painful process. I tried the following method: 1. rsync 2. tar via ssh 3. tar via nc (all take hours and hours to finish) None o