We have a CMS that basically stores user data in a fs structure such as /users/a/b/abraham/. Whenever a user edits one of their own files, the webapp will touch a file in a specific location such as /activeUsers/abraham. We use a predump script that quickly generates a list of recently active user's paths such as: ./users/a/b/abraham ./users/b/r/brien
With tar, the "full" backup is straightforward (and time-consuming). The incrementals, however, are fed the "-T" option for the filelist (very fast). We quickly realized that our incrementals are not "filled", presumably because tar is not creating the empty directory tree for the rest of the filesystem (backupc thinks those directories were deleted). We've really thrown a monkey wrench into the backup semantics since not only are the incrementals based off the last full (--newer), but the list (-T) will only include paths that had updates since the last incremental (which could be hours). SO, now we are installing backuppc 3.0beta for the multilevel incrementals feature (very cool!!). We are hoping that we can use rsync instead of tar so that the incrementals will be 'filled in" and such (not to mention deleted/renamed files are detected). One problem is there is no "RsyncFullArgs" and "RsyncIncArgs" with which we can trick backuppc the same way we trick tar. Are we being nuts here? If this even possible with Rsync? Our "end goal" is to be able to take a Monthly Full, and then incrementals every 2 hours (360 incrementals). By using the hints it is very fast. (~10 mins per backup). Our motivation is the fact that we have 8 million+ files and regular tar/rsync fulls or incrementals literally take 15+ hours (and growing) to complete simply due to the traversal of all these trees and checking stats on these small files (generates unacceptable iowait as well). Here is our tar config that works besides the lack of filled incs. $Conf{TarClientCmd} = '$tarPath -c -v -f - -C /mnt/nfs' . ' --totals'; $Conf{TarClientRestoreCmd} = '$tarPath -x -v -f - -C /mnt/nfs' #change dir to where nfs data is mounted . ' --totals'; $Conf{TarFullArgs} = './users'; #backup entire users folder $Conf{TarIncrArgs} = '--newer=$incrDate -T /var/lib/backuppc-meta/backuplist.txt #backup only files newer than last full, but ALSO only in the backuplist.txt (webapp generated) '; $Conf{TarShareName} = '/'; #this is arbitrary $Conf{DumpPreUserCmd} = '/usr/local/bin/predump.sh'; #generates backuplist.txt $Conf{DumpPostUserCmd} = '/usr/local/bin/postdump.sh $xferOK'; #removes backuplist.txt if the xfer was OK Thanks for any advice!! brien ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/