I asked this at the tail end of an email back in December but no one replied:
It seems to me that under most conditions, something with minimal overhead (such as tar) is best for fulls while rsync is best for incrementals. As far as I know, there's no way in backuppc to do this on the same host. Right? If not, is this an idea worth investigating, Craig? Cheers, Stephen -- Stephen Joyce Systems Administrator P A N I C Physics & Astronomy Department Physics & Astronomy University of North Carolina at Chapel Hill Network Infrastructure voice: (919) 962-7214 and Computing fax: (919) 962-0480 http://www.panic.unc.edu If you want to master emacs, it helps to believe in reincarnation, because there is no way in hell you are going to learn it all in a single lifetime. On Sat, 20 Jan 2007, Nils Breunese (Lemonbit) wrote: > Hello all, > > I think this post by Holger is a pretty good explanation of the differences > between the backup types and transfer methods available in BackupPC. Maybe > this information could be reworked for inclusion in the BackupPC > documentation? > > Nils Breunese. > > Holger Parplies wrote: > >> Hi, >> >> Clemens von Musil wrote on 20.01.2007 at 00:31:12 [[BackupPC-users] >> Avoiding long backup times]: >>> [...] >>> One configured backup host has a very slow network connection - a full >>> backup lasts about two days. Because of the exclusive run of >>> backuppc_nightly in 2.xx, I learned, this full backup stalls all other >>> backups. I want to avoid tihs situation and got stuck in following: >>> >>> What exactly happens during full backup? >> >> that depends on the transfer method. For a slow connection, you want rsync >> (or rsyncd), not tar or smb. For a very slow connection, you *definitely* >> want rsync/rsyncd (which I'll just call "rsync" for simplicity). >> >>> I read, that backuppc stores every identical file only one time. >> >> Basically true, but BackupPC needs to determine that the file is identical >> to >> something and to what. Sparing you the transfer (if possible with >> reasonable >> cost) is rsyncs job. Doing so comes at a cost in terms of CPU usage, so >> you've got the option of using tar if bandwidth is cheaper than CPU power. >> In your case, it obviously is not. >> >>> [...] What happens with an unchanged file in a full backup? >> >> tar and smb will transfer it, rsync will not (presuming you mean >> "unchanged" as in "same name, same content"). >> >> For a changed file, rsync will try to speed up the transfer. If you append >> a >> few bytes to a large file, tar/smb will transfer the whole file (even on an >> incremental), while rsync will (basically) transfer some checksums and the >> few bytes only (on full and incremental backups). >> >>> If the file will not be transferred again - what is the difference >>> between full and incremenal? >> >> This only applies to rsync (as tar/smb will transfer it). rsync will always >> transfer missing files as well as update files that have apparently >> changed. >> The difference between full and incremental backups lies firstly in the >> amount >> of trouble rsync will go to to determine whether a file has changed or not. >> For >> an incremental backup, I believe rsync will only look at size and >> modification >> time, whereas for a full backup, checksums of the files are calculated, >> thus >> consuming much more CPU-time and disk-I/O-bandwidth at least on the client >> (I >> understand the server caches checksums if you tell it to), with the benefit >> of >> detecting files that have been changed and the modification time reset. >> The second difference is that a full backup gives a new reference point for >> following backups while an incremental backup does not necessarily. >> In version 2 (of BackupPC) all incrementals were relative to the last full >> (so >> after one year of only incrementals you'd be transfering everything changed >> in >> that year on each incremental backup), whereas version 3 supports >> multi-level >> incrementals, with each incremental transfering everything changed since >> the >> last incremental of lower level (resp. full backup at level 0). >> >>> And if yes - does it make sense to keep one full for ever and dealing >>> only with following incrementals? >> >> No. With a very slow network connection, you want to avoid transfering >> changes more often than necessary. Configuring multi-level incrementals in >> BackupPC 3 seems to be simple enough that you *could* say "do incrementals >> of >> increasing level each day for the next, say, 10 years", but that will make >> browsing, restoring and even doing the backups increasingly expensive (and >> you'll need to keep all the incrementals), even if you *can* neglect >> modified >> files your backups are missing. A full backup each day *might* be your best >> choice, because it avoids duplicating transfers. With any incrementals in >> between, you'll have at least the fulls re-transfering everything since the >> last full (less or equal to the sum of the intervening incrementals (plus >> the changes since the last incremental, obviously), due to files modified >> more than once or modified and deleted). >> You'll have to find out the best trade-off in your specific situation >> between retransfering file content and calculating checksums. You probably >> need to find out first, how much data you need to transfer each day. Then >> you can estimate, the traffic of how many days you could allow yourself to >> transfer at once. That should be about the maximum interval between full >> backups ... >> >> >> Regards, >> Holger >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share >> your >> opinions on IT & business topics through brief surveys - and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> BackupPC-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/backuppc-users >> http://backuppc.sourceforge.net/ > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ BackupPC-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
