I have an "oppurtunity" to migrate some data from one Unix server to another. I was going to use *nix tools to do it and had plumped for Rsync. I have a lot of data to move (1.5Tb+ from one site and 1Tb+ from another), but time is not an issue, reliability of the data is. Is Rsync the correct animal for the job, or is there something better that I could Use. Concerns have been raised by some in the organisation who had heard that Rsync misses data. Any comments I can use to confirm or deny that?
Never moved a TB, but I very regularly move 30GB chunks over 100Mbps lines. Never missed data yet.
I would suggest first that you ask those "concerned people" to provide some substance to their claim that rsync misses data, so that the burden of proof is on them; I've never heard of it happening yet.
rsync is a wonderful mirroring tool. What I would do in order to make sure it doesn't eat your machine for lunch calculating so much stuff to move is to use the built-in exclude features in rsync so it only transfers part. Then you can remove the excludes one by one and move more stuff, with rsync always making sure that the previously-transferred stuff is kept up to date.
For example, if you are transferring the whole RedHat FTP site, you can exclude (one by one) all the directories in /pub/redhat/linux/ (i.e. 3.0.3, 4.0, 5.0, 5.1, 5.2, 6.0, 6.1, 6.2, 7.0, 7.1, 7.2, 7.3, 8.0) and leave only 9. You can also exclude just the /ja directories such that Japanese language RPMS in any part of the directory tree don't get transferred.
Now rsync will transfer only /pub/redhat/linux/9 (about 8GB) and stop. Then you remove the exclusion from /pub/redhat/linux/8 and you transfer another 8GB... but rsync checks to make sure that the 9 data is up to date and updates any changes. And so on until you have brought up all the versions. Finally you allow the /ja, /fr, and other languages. In this way you can, with one rsync command that you modify each time, do the whole transfer, and not have rsync eat all your RAM calculating 273,000 files to send. :-)
Does this make sense?
-- Rodolfo J. Paiz [EMAIL PROTECTED]
-- redhat-list mailing list unsubscribe mailto:[EMAIL PROTECTED] https://www.redhat.com/mailman/listinfo/redhat-list
