Hi, Adam Goryachev wrote on 2009-06-03 10:29:34 +1000 [Re: [BackupPC-users] Backing up a BackupPC server]: > [...] > Might I (re)-propose the following: > [...] > 2) Extend backuppc-link so that after it links a file to the pool, or > replaces a file in the pc directory with a link to the pool, then it > will also add an entry to a DB (use the perl-DBI interface to make it > generic or similar). Add some code to backuppc_nightly to modify the > database when it modifies the pool as well. > [...] > Next, write a tool that can read the database + pool, and create the > hardlinks under the pc directory. (So after you copy the pool, you can > re-create the hardlinks). > > Thus, for people you want to copy their entire backuppc pool/etc, they > can do it with a database + overhead, and everyone else can continue > as-is.
I don't want to disappoint you, but that is trivial (to implement) even without modifying BackupPC. If I'm allowed to write to a database, I can, with a few lines of Perl code, 1. Create a table "pool" mapping inode numbers to pool files (that's a simple iteration over the pool adding (inode, path) pairs) 2. Create a second table "pc" mapping pc/ path names to inode numbers (that's a simple iteration over pc/ adding (path, inode) pairs) Your pc-directory-re-creation tool now reads something like SELECT pool.path AS src, pc.path AS dst FROM pc, pool WHERE pc.inode = pool.inode ORDER BY pc.path and does "link ($src, $dest);" after creating any directories $dest might need (would ordering by pc.inode work better?). Yes, that statement is going to produce an *enormous* result table. Let's just hope the database is smart enough to handle that. If not, you can always do a SELECT pool.path FROM pool WHERE pool.inode = ? and execute that for each inode number you find in pc/ ... you won't even need the "pc" table then. But still: the "pool" table will be large. The size of this table is what determines when rsync et al. will stop working. Let's hope an index on pool.inode will make the DB work wonders. We're still accessing the information in random inode order, so we're not likely to get (m)any benefits from caching. No, this is probably not the way to go. Obviously, the same is true if this information is added to the database during BackupPC_link operation and maintained by BackupPC_nightly. > I'm sure the above doesn't handle 100% of needed cases, Same for my ideas. If I'm *not* allowed to write to a database, I'll need slightly more brains, but the difficult part is handled by a "sort" invocation. Sorting 1.1 GB took 2.5 minutes - considerably less than traversing the pool and pc directories. Actually, the same approach can be used with a database and one single table, but then the database is really doing nothing except sorting data and supplying it one time in the correct order. I like your idea of separating the pool copy operation from re-establishing the pc/ links. That sort of eliminates all RPC-type problems from my original task. I only need to figure out how to handle non-pooled content, but that's probably only stuff immediately in the pc/$host directories and 0-byte files (which aren't that hard to re-create on the target system ;-). That way, I would actually end up with something like BackupPC_tarAllPCsCopy without the tar part (and without the overhead of calculating all pool hashes). Regards, Holger ------------------------------------------------------------------------------ OpenSolaris 2009.06 is a cutting edge operating system for enterprises looking to deploy the next generation of Solaris that includes the latest innovations from Sun and the OpenSource community. Download a copy and enjoy capabilities such as Networking, Storage and Virtualization. Go to: http://p.sf.net/sfu/opensolaris-get _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/