Hello!
Here are some notes on how to submit an archive job to BackupPC in the
same way that the GUI does. This can be used via cron, etc., yet will
still be logged and visible from the GUI.
In Googling ways to archive via cron, the only items I've found used
individual BackuPC_archiveHost commands. The command you need touse can
be found when you examine the logs, and is pretty straightforward: copy
and paste the command line into a crontab file for your backuppc user,
make a couple of small changes, and you're done.
However, this does not integrate into the GUI. This means that you
either have to mail the cron output to the user, or pipe the output into
a file and get to it another way, or something similar. It does not
allow you to see the history the same way as if they were started from
the GUI.
According to Craig, the way to get the jobs to show up in the GUI is to
submit them the same way that the GUI does. How that is done is left as
an exercise to the reader... :) So, here's what I've found.
If you only want the bottom line, jump to the Summary section at the
bottom. If you would like to help me to answer 3 questions I found
while doing this, jump to the
How the GUI Submits an Archive
==============================
The code responsible for providing the archive GUI is found in
__INSTALLDIR__/lib/BackupPC/CGI/Archive.pm . As you go through the
library, you see the code that builds the pages that make up the archive
process. The part that actually generates the request that launches the
archive job is at the very end:
my $openPath = "$TopDir/pc/$archivehost/$reqFileName";
if ( open(REQ, ">", $openPath) ) {
binmode(REQ);
print(REQ $archive->Dump);
close(REQ);
} else {
ErrorExit(eval("qq{$Lang->{Can_t_open_create__openPath}}"));
}
$reply = $bpc->ServerMesg("archive $User $archivehost
$reqFileName");
"$archive" is an array of values that are provided by the user in the
previous screens: which hosts to back up, which backup jobs for each
host, type of compression, parity, etc. This is written to a file:
"$TopDir/pc/$archivehost/$reqFileName".
If you make a request and then do an ls on the archive host's directory,
you will see a file named something like "archiveReq.####.#" where ####
are digits. Here's the code that generated that file name.
for ( my $i = 0 ; ; $i++ ) {
$reqFileName = "archiveReq.$$.$i";
last if ( !-f "$TopDir/pc/$archivehost/$reqFileName" );
}
The last digit is generated by loop to the first digit that doesn't
already exist in a file (in case you have stacked up more than one
archive job). Seeing as I don't know perl, I can't figure out what the
"$$" does, and trying to google for "perl $$" doesn't help...
No matter: the file name is not very important at all: from what I can
tell, the whole file name could be completely random. So, onward we go.
Here is the contents of a sample file made by selecting localhost, and
accepting the defaults (at least, as I have them configured):
%ArchiveReq = (
'archiveloc' => '/var/lib/BackupPC/removable',
'reqTime' => '1170042557',
'BackupList' => [
'12'
],
'host' => 'archive',
'parfile' => '5',
'archtype' => '0',
'compression' => '/bin/cat',
'compext' => '.raw',
'HostList' => [
'localhost'
],
'user' => 'backupuser',
'splitsize' => '0000000'
);
Once the file is created, there is one more step: submit a message to
the server telling it about this request file. BackupPC will now open
the archive request file, parse the parameters, and run the archive job
you've requested. It then creates a log file in the archive directory
that can be parsed by the GUI.
How Your Code Can Submit an Archive
===================================
To perform your own archive, all you have to do is have a
properly-formatted file that contains the details for your archive in
the archive directory, just like the file created by the GUI, and tell
BackupPC to start the archive. A quick and dirty way to get such a file
would be to use the GUI and copy the archiveReq file it generates for
you. You will then have a perfectly-formatted file for your use.
There are two problems with this. The first is that the specific backup
number to be archvied (BackupList) is hard-coded. You probably want the
latest file, not the specific one that was the latest when you captured
the file. Fortunately, that's an easy fix. If, instead of an actual
backup number, you use "-1", BackupPC will get the latest version for
you automatically. The only thing I find annoying about that is that
BackupPC will still use "-1" for the filename. This makes it ugly to
try to have multiple days' worth of archives in the same directory. You
either have to have code somewhere that renames the jobs for you (such
as a pre- or post-dump command), or change the destination of the
archive every time to a unique directory. It would be nicer, though, if
BackupPC just substituted the right number for you. But that's how it is.
The second problem is the fact that the time the job was requested
(reqTime) is also hard-coded. That means that every job will have the
exact same request time. However, this may not be that bad. When you
look at a log entry for an archive, you will see that two times are
recorded: the request time and the start time. Only the request time
will be incorrect: the start time will show when the job was started by
BackupPC. So, you may be able to ignore it. Or, you can certainly come
up with code that updates the reqTime field with the current time; that
way, the logs will show the correct submission time as well.
Once you have created the request file in the proper directory, you
simply tell BackupPC to process it. That can be done easily from the
command line:
__INSTALLDIR__/bin/BackupPC_serverMesg archive <user> <archiveclient>
<reqFileName>
This will run your job just as if you had submitted it from the GUI. It
can be scripted from wherever you would like, such as from a crontab,
just as easily as using BackupPC_archiveHost.
Notes about Archive Parameters
==============================
I have not delved extensively into the code to find out exactly how each
parameter is parsed, and what (if any) error-checking is done. However,
I was curious about a couple of parameters.
** First, why is there a user parameter on the command line, when the
request file also contains this information? How are they used?
From what I can tell, the user that is submitted on the command line is
used for exactly one thing: it is set as the $user variable that is
passed via XferArgs to the pre- and post-dump commands. The user that
is contained in the request file is used for a different, but similar
purpose: it is passed via XferArgs to $xfer. $xfer is the procedure
that actually does the archive. I do not know where it goes or what it
does with it: I haven't been able to delve that deeply into BackupPC's
internals yet.
Why are there two parameters for assigning the user name? And why is
one used in one place and the other somewhere else? Only Craig knows...
:) Would you mind telling us?
** Second, how is reqTime used?
AFAICT, reqTime is only used by the GUI: it is not actually used during
an archive. At least, BackupPC_archive does not use it, and it is not
passed to xfer$, so I don't think anyone else could use it either. My
guess is the only time that this parameter is parsed is when rendering
the archive job's log details in the GUI. In fact, I altered the date
by putting a z before the numbers. When you do this, the job runs fine.
However, in the GUI, the request time is always the current time, not
the request time.
** Third: how does BackupPC_archive know which number is the latest
backup? Is there an easy way to do this outside of BackupPC, preferably
without having to duplicate code?
Here is the relevant code (with unrelated code removed) for how the GUI
determines the latest backup:
foreach my $host ( sort(keys(%Status)) ) {
my @Backups = $bpc->BackupInfoRead($host);
for ( my $i = 0 ; $i < @Backups ; $i++ ) {
$backupnumber = $Backups[$i]{num};
}
}
The GUI code that determines the latest backup uses
$bpc->BackupInfoRead($host) to get that information. That's how the
latest backup is determined by the GUI.
I would like to know what the code that parses the archive request
file's backup number parameter does, but it's parsed by $xfer, and I'm
not sure where that code is. I think that this is merely passed to
BackupPC_archiveHost, but I am not sure.
I also looked at BackupPC_archiveHost: it too can handle the "-1"
parameter. It passes the "-1" to BackupPC_tarCreate.
BackupPC_tarCreate reads the files and generates a tar datastream for
archiveHost. It also converts the "-1" into the latest backup for that
host. (NOTE: That may present a problem for getting archiveHost to
create files with the proper backup number, mentioned in my previous
e-mail: how can I get the data from tarCreate back to archiveHost?
Might it be possible to do the "-1" to backup number conversion in a
library function available to both of them, as well as the GUI? I will
have to do some more digging.) Here is the code it uses (with some
unrelated lines removed) (NOTE: $Num is the backup number, such as "-1"):
my @Backups = $bpc->BackupInfoRead($Host);
$Num = [EMAIL PROTECTED] + $Num]{num} if ( [EMAIL PROTECTED] <= $Num && $Num <
0 );
for ( $i = 0 ; $i < @Backups ; $i++ ) {
last if ( $Backups[$i]{num} == $Num );
}
It is similar to the GUI code. In both cases, it loops through the data
returned by BackupInfoRead($Host) and finds either the latest backup
only (in the case of the GUI) or the requested backup (in the case of
BackupPC_tarCreate).
So, what does BackupInfoRead() do? The code is in
__INSTALLDIR__/lib/BackupPC/Storage/text.pm:
sub BackupInfoRead
{
my($s, $host) = @_;
local(*BK_INFO, *LOCK);
my(@Backups);
flock(LOCK, LOCK_EX) if open(LOCK, "$s->{TopDir}/pc/$host/LOCK");
if ( open(BK_INFO, "$s->{TopDir}/pc/$host/backups") ) {
binmode(BK_INFO);
while ( <BK_INFO> ) {
s/[\n\r]+//;
next if ( !/^(\d+\t(incr|full|partial).*)/ );
$_ = $1;
@[EMAIL PROTECTED]@{$s->{BackupFields}}} = split(/\t/);
}
close(BK_INFO);
}
close(LOCK);
return @Backups;
}
It parses __TOPDIR__/pc/$host/backups and gets the info from there.
Others have suggested parsing the filesystem to get this information,
but I am not confident of its robustness. It *should* work reasonably
well, but obviously the above way is safer.
However, it does not seem like there is an easy way to get this
information from outside of BackupPC. Other than the fact that the
names of the files are "-1", and the fact that you can't create a new
archive (with parity, anyway) on top of an existing archive without an
error, there isn't much of a reason for getting this information anyway.
So, I would like to figure out a way to do one of these two things:
either get the latest backup number or save archive files with the
proper number in the filename even when using "-1" as the backup number.
Summary
=======
To submit an archive job like the GUI, you must:
1) Create a properly-formatted archive request file in the archive
host's directory.
This can be done by using the GUI to create the file, saving it
somewhere else and copying it into place when needed, or by generating
the file from scratch using some other method. If you use a
pre-generated file, you may need to make a couple of small changes:
change BackupList to the proper value (likely "-1"), and update reqTime.
2) use the BackupPC_serverMesg command to notify BackupPC of the pending
archive. it uses the following syntax:
__INSTALLDIR__/bin/BackupPC_serverMesg archive <user> <archiveclient>
<reqFileName>
where <user> is the user passed to the pre- and post-dump commands,
<archiveclient> is the host name for your archive host, and
<reqFileName> is the file you created in step 1.
I hope this information will be useful to others: hopefully it can save
you a few hours. If you have any thoughts, I would appreciate any info
you might have particularly regarding suggestions for updating
BackupPC_archiveHost to save files with the proper backup number in the
filename.
Thank you,
Timothy J. Massey
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
BackupPC-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
