On Wed, Jul 23, 2003 at 03:12:08PM +0100, Kelerion wrote:
> I'm not exactly an expert on this subject (or many others for that
> matter) but is NFS really a suitable solutions to transfering 1.5TB+ of
> data over a network?? I didn't think it was *very* stable...
Previous messages have suggested rsync, and rsync is easy to use
between mounted volumes, but you probably don't want to do that. My
first crack would be to use rsync with a "-e ssh" option, it will be
faster than NFS, more reliable, and encrypted (not a bottleneck on a
modern machine, however, because you will be IO-bound).
Note that rsync tries to be efficient about not copying things that
don't need to be copied, sending only diffs for large files that
already exist but are now changed, etc. This means that rsync spends
a lot of time up front deciding what it is going to do. And the
bigger the transfer, the more RAM it will take to do all that. You
might not want to use rsync for that reason. However, because you are
doing so much stuff it will take a while, things might crash and you
might have to start over, etc. In this case rsync is very nice for
all the subsequent attempts as it will not start over each time.
Another consideration, rsync (or anything that backups so much stuff)
will slow down your machine as it blows away your disk and kernel
caches by constantly reading fresh data. You might want to run it
when there is less activity, possibly at night. Also, look at the
bandwidth limit ("--bwlimit") option, it won't limit the instantaneous
bandwidth for transferring a single file, but it will put pauses
between files to keep the average down.
So maybe start an rsync during a quiet time, let it slow down the
machine as it decides what to do (this will take *some* time in your
case), but use a bandwidth limit to keep the actual copying from
slowing down continued use of the machine. You might stretch it out
to several days if you can, to be sure that current use is not too
badly hurt.
-kb
--
redhat-list mailing list
unsubscribe mailto:[EMAIL PROTECTED]
https://www.redhat.com/mailman/listinfo/redhat-list
