Les Mikesell <[EMAIL PROTECTED]> wrote on 01/26/2007 03:23:45 PM: > Timothy J. Massey wrote: > > I would love to see this abstracted a little more into a "copy-host" > > feature, that could copy a host to a new host, either within the same > > pool or to a different pool. After reading about how the NewFiles > > file works, it doesn't seem like we would even have to worry about > > preserving hardlinks if NewFiles were configured to write down *all* > > files as new files: the BackupPC_link process would resolve the > > issues for us. > > > > I don't have time right now, but that will be how I think I will > > attempt to move a host from one pool to another: let BackupPC_link > > take care of it. All I'll have to do is walk the tree, copying files > > and adding them to NewFiles. It's hard on BackupPC_link, but I can > > live with that. > > When copying within the pool, all you have to do is preserve the > existing links which cp -a or cp -l would do, since they will already be > to the correct pool files. When copying to a new server, a utility > could be a little more intelligent if it tracked the inodes of the > source files and when it saw duplicates, sent the filename of the 1st > copy instead of the data. I think Craig has done some work on a utility > to copy an entire archive, but I think that requires the entire original > pool to be copied first which might have collisions in the hashing if > you move to a different existing server and is overkill if you only want > to move some of the hosts.
As a start, how about a utility that simply clones one host to another using only the pc/host directory tree, and assumes that none of the source files are in the pool, just like it would during a brand-new rsync backup? Imagine two BackupPC servers, server A and server B. I want to move a single host (HostX) from server A to server B. Could I mount Server A's pc/host directory somewhere on Server B (e.g. __TOPDIR__/pc/AHostX), and then run a utility (e.g. BackupPC_clone AHostX HostX) that would create a new host based on the old host. It would assume that *every* file in the source tree is a new file. If the file does not exist in the pool, it would then copy the file into the pool. In either case, it would then create a hardlink in the destination tree pointing to the proper pool item. You don't have to worry about e.g. pool collisions: you use the existing BackupPC tools to avoid this, just like an initial full rsync backup (which can't use pooling, either). This is simple, and should be easy to do: it should be a straightforward modification of BackupPC_link. The disadvantage is that without the intelligence you describe (caching hash results or inodes between different backups of the same host), it would be forced to read every single file of *every single backup* of the source host, first to put the file into the pool, and every other time to check to see if the file is the same as the item in the pool. But that could be added later. At least we'd have a tool for properly combining one host into a different pool. What about using a modified BackupPC_tarCreate / BackupPC_tarExtract for this? Wouldn't that give us identification of common hardlinks for free? Craig: Is this something valuable to you? I do not know perl, but it's something that's important enough to me to start hacking away at it... It doesn't hurt that 99% of what I need is already in BackupPC_link/tarCreate/tarExtract... :) This could even be used to migrate from a compressed pool to an uncompressed pool and back again, couldn't it? > I guess > that wouldn't help with a remote move where you'd like to avoid the 1st > copy too. Maybe a special version of rsync at both ends could do the > hash computations to avoid copying anything already in the pool. For now, I think I'm going to have to give up on that. Without a BackupPC-specific client that can do a BackupPC-style hash on the host and send only that information to the server, and then do what rsync or tar do for you afterward, I just don't see that it's possible. But who wants to write and maintain (and debug, and port and support and ...) such a client when there's rsync and tar? I think I can get the same effect by intelligently migrating BackupPC hosts from one system to another. In reality, I'm still talking about a custom BackupPC client, but instead of targeting the host, I'm targeting a much more confined target: BackupPC servers. How about BackupPC servers that can replicate their data to each other in remote locations? A local BackupPC server is set up in the default way: incrementals are performed daily, fulls are performed weekly. In addition, this server also replicates all of its backups once a week to a remote BackupPC server, which integrates them into its pool. There are a lot of advantages to this. It can take advantage of pooling *before* an item is sent. It might be able to take advantage of an rsync-style copy: that I can't yet envision without re-writing rsync (which I don't want to do), but I haven't delved into RsyncP yet. I don't care if it takes all *week* to send that data over the slow link (I only have 1 transaction to do per week anyway!), and because it will understand hardlinks, you're better off waiting for a week to do it anyway: less hashing to do for repetitive files. Yet my hosts still get backed up daily--and quickly--via a local server. Is there any interest in this? One of the reasons I rejected Amanda was its complexity. I like the simplicity of BackupPC. I think that this could give BackupPC some of the distributed featuers of something like Amanda, without increasing the complexity of a single-server configuration. Tim Massey ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
