-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 09/01/12 09:00, Les Mikesell wrote: > On Sun, Jan 8, 2012 at 4:48 AM, John Habermann > <jhaberm...@cook.qld.gov.au> wrote: >> >> You can see that the backup of the /opt share takes nearly the >> total time of the incremental taking about 8 and half hours to >> complete while the backup of the /opt rsync share in the full >> backup takes about 3 and half hours. The full backup is slightly >> longer than what it takes if I just do a rsync over ssh copy of >> the file from the client server to the backup server. >> >> I have found that rsync seems to always transfer the whole file >> when copying this file from the client server to the backup >> server: My questions for the list are: 1. Is it reasonable for >> rsync to transfer the whole file when copying a large ntbackup >> file? > > Yes, those files may have little or nothing in common with the > previous copy. If compression or encryption are used they will > ensure that no blocks match and even if they aren't, the common > blocks may be skewed enough that rsync can't match them up.
Definitely, I find that transferring (for example) uncompressed mysqldump files will transfer less bytes than transferring the compressed files. (Obviously because the compressed file will transfer 100%, while the uncompressed will only transfer the changes). >> 2. Why does an incremental backup of this file take so much >> longer than a full backup of it or a plain rsync of this file? > That doesn't make sense to me either. Are you sure that is > consistent and not related to something else that might have been > using the link concurrently? I'm not exactly sure, but I've found the best way to approach these types of files (any dump/export from another application, MS SQL, MySQL, custom applications, etc), is to: 1) Ensure the file is uncompressed if it is in plain text format of some sort, which may mean uncompressing it after the other application creates it, and before backuppc will see it. The size of chunks can depend on transfer speeds available, total file size, etc... 2) Any file over about 100M, I split into chunks (using split) of anything from 20M to 100M chunks. I've found that rsync will transfer these chunks much more quickly than transferring the whole file. Also, if the transfer is interrupted (on a full backup), it will save the backup as a partial, and continue from the last chunk (as opposed to re-starting that 5G file, you only lose a maximum of 100M). Regards, Adam - -- Adam Goryachev Website Managers www.websitemanagers.com.au -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8PqjwACgkQGyoxogrTyiVi6ACdGNld8qhmEeLqvX6HSQMwLhcS amQAmwegtCByLBo4dXrXEbKpMqrHuSJO =tBKs -----END PGP SIGNATURE----- ------------------------------------------------------------------------------ RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/