I've only looked at the code briefly, but I believe this *should* be possible. I don't know if I'll be implementing it, at least not right away, but it shouldn't actually be that hard, so I wanted to throw it out so someone else could run with it if ey wants.
It's an idea I had about rsync resumption: Keep an array of all the things you haven't backed up yet, starting with the inital arguments; let's say we're transferring "/a" and "/b" from the remote machine. Start by putting "a/" and "b/" in the array. Then get the directory listing for a/, and replace "a/" in the array with "a/d", "a/e", ... for all files and directories in there. When each file is transferred, it gets removed. Directories are replaced with their contents. If the transfer breaks, you can resume with that list of things-what-still-need-transferring/recursing-through without having to walk the parts of the tree you've already walked. This should solve the SIGPIPE problem. In fact, it could even deal with incrementals from things like laptops: if you have settings for NumRetries and RetryDelay, you could, say, retry every 60 seconds for a week if you wanted. On top of that, you could use the same retry system to *significantly* limit the memory usage: stop rsyncing every N files (where N is a config value). If you only do, say, 1000 files at a time, the memory usage will be very low indeed. As I said, none of this should be *especially* hard to implement; it's just changes to lib/BackupPC/Xfer/RsyncFileIO.pm and lib/BackupPC/Xfer/Rsync.pm , and doing this would make BackupPC be as robust as a lot of commercial backup packages, on top of all its current benefits. (Note the "-d" option to rsync; you'd want to pass that to the remote rsync to get the directory listings, I think.) -Robin -- They say: "The first AIs will be built by the military as weapons." And I'm thinking: "Does it even occur to you to try for something other than the default outcome?" See http://shrunklink.com/cdiz http://www.digitalkingdom.org/~rlpowell/ *** http://www.lojban.org/ ------------------------------------------------------------------------------ Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
