On Wed, Mar 24, 2004 at 02:46:08PM +0800, Mathias Koerber wrote:
>
> d) I was looking into pre-post dump actions (eg to stop a database or email
> delivery)
> I found the tips to write a specialized tar (amgtar) script which does
> this.
> My question however is whether the protocol between server and client
> could
> handle this, or at least handle whether a pre-post function is to be
> called and
> its results etc.
>
> I understand that security is an issue, which means that which scripts
> are allowed to
> run on each client should be controlled on that client itself, which
> speaks for a
> local amgtar.
>
> Does anyone already have such a script which is flexible to support
> several pre/post
> dump (and pre/post estimate?) scripts as well as handles errors in such
> scripts?
Quite a while ago JRJ posted the attached readme and wrapper script.
I've not used them, just still have them available.
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road (609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Subject: Re: Synchronizing Amanda with Applications
>I looked at the archives. I found some references to such wrappers,
>but I've failed to find a good example of one, and what needs to be done
>on which machine (client/server).
OK, you asked for it :-).
Some notes on the following over-engineered, under-tested, script:
* It has **NOT** been run in a full Amanda setup, other than what
little I did to make sure syntax was right and the general flow
worked.
* The script uses **EXAMPLE** execution point functions (see the code).
You **WILL** have to change it to do what you want (and remove what
you don't).
* There are definitions of the Amanda debug directory (e.g. /tmp/amanda)
and path to GNU tar (on the client) in the first few lines that may
need to be tweaked for your environment.
* It is written in ksh (although now that I think about it, I probably
could have used Perl). Hopefully the examples and your own shell
coding experience will be sufficient to fit in what you need, but
if you want some help, ask me offline.
The idea is to install this on the client and build Amanda there
using --with-gnutar=<path-to-script>. Then set up the disks you want
to use this wrapper with GNU tar as the dump program in disklist and
add execution point functions (see the code) to the script to do what
you need.
Note that the script doesn't have to actually use GNU tar -- that's just
the trick to get Amanda to call it with a set of arguments that are easy
to decipher. You can use a run_* function to use whatever dump program
you want, as long as it generates an estimate to stdout that Amanda can
parse and sends the actual dump image to stdout.
Note also that compression is a separate step Amanda will tack on to the
stdout of this script (or not). If you want to do your own compression,
add it to the script and tell Amanda "compress none".
Then test it at least once :-). If you set the DEBUG environment variable
to "print" (or "echo" -- same difference), the script will run but not
execute anything (although you'll have to set up your own functions to
follow the examples in the code). Here is a sample test line I used
(using ksh as a login shell):
DEBUG=print gtar-wrapper.ksh \
--directory /home/HomeAC \
--file - \
.
Amanda will pass whatever you set in disklist as the --directory argument.
To test estimates, change "-" to "/dev/null" for the --file argument.
Amanda passes a mess of other parameters, but only --directory and
--file are of interest to the script. If you do any significant argument
processing, make sure you don't loose anything (see function do_home in
the script).
I think. :-)
John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
** Additional notes -- 12-Sep-01 **
Fixed some truly silly bugs. Rewrote enough of the code to be able to
run under bash as well as ksh.
#!/bin/ksh
###
# This script is called by an Amanda client to run GNU tar. We look
# through the arguments for what is being processed and (optionally)
# run extra things before and/or after GNU tar, or alter the actual
# command run (e.g. different GNU tar flags and options or a completely
# different program).
#
# Remember that this script runs as root under the Amanda runtar program.
# Exercise appropriate caution.
#
# To test, run with the DEBUG environment variable set to "echo".
# The command line need not look exactly like what Amanda issues, but
# must include the --directory <dir-to-back-up> and --file <outfile>
# flags as well as the trailing ".". To test an estimate, set the
# output file to "/dev/null":
#
# env DEBUG=echo ./gtar-wrapper.ksh --directory /whatever --file /dev/null .
#
# and to test the real dump, set it to "-":
#
# env DEBUG=echo ./gtar-wrapper.ksh --directory /whatever --file - .
#
# John R. Jackson ([EMAIL PROTECTED])
# Purdue University Computing Center
#
# 24-Mar-00: Initial version.
#
# 12-Sep-01: Fixed several bugs (sigh) and converted the code so it would
# work under bash as well as ksh.
###
PN=${0##*/}
if [[ -z $DEBUG ]]
then
log=/tmp/amanda/$PN.$$ # <<< change as needed
rm -f $log
else
log=/dev/tty
fi
GTAR=${GTAR:-/opt/tar/bin/gtar} # <<< change as needed
###
# Define functions to be called at various execution points. The name
# determines when the function is called:
#
# pre_estimate_* called before a GNU tar estimate
# pre_real_* called before a GNU tar real run
# run_estimate_* called to run a GNU tar estimate
# run_real_* called to run a GNU tar real run
# post_estimate_* called after a GNU tar estimate
# post_real_* called after a GNU tar real run
#
# The rest of the function name comes from the directory being processed
# by converting everything other than alphanumerics and the underscore
# to an underscore, e.g. /home/abc becomes _home_abc.
#
# If a function does not exist for a particular execution point, nothing
# special is done.
###
###
# The following set of functions (*_some_db) show how you might set
# up the execution points to shut down and restart a database around
# the Amanda steps.
###
###
# This function is called before we get the estimate of /some/db.
###
function pre_estimate__some_db
{
echo "$PN: shutting down the database" >> $log
}
###
# This function is called after we get the estimate of /some/db.
###
function post_estimate__some_db
{
echo "$PN: starting the database" >> $log
}
###
# This function is called before we dump /some/db.
###
function pre_real__some_db
{
echo "$PN: shutting down the database" >> $log
}
###
# This function is called after we dump /some/db.
###
function post_real__some_db
{
echo "$PN: starting the database" >> $log
}
###
# The following set of function are called to process groups of home
# directories. The Amanda disklist entries are "faked" to indicate
# a range of directories to be processed instead of a single item.
# For instance, /home/HomeAC (which should not actually exist) would be
# converted into all the directories in /home that start with a, b or c.
#
# This might be used to get around the current limit on the number of
# "disks" Amanda can handle on a given client or to make it easier to
# set up groups of things to back up without exclusion lists.
###
function run_estimate__home_HomeAC
{
echo "$PN: preparing to estimate homes a* through c*" >> $log
do_home a c "$@"
return $?
}
function run_real__home_HomeAC
{
echo "$PN: preparing to dump homes a* through c*" >> $log
do_home a c "$@"
return $?
}
###
# The first and second args to do_home are the start and end names.
# The remaining args are passed to GNU tar pretty much as is, except
# --directory which is shortened to the home directory base.
###
function do_home
{
###
# Copy all the args except the last one (which should be ".") and
# tweak the --directory value.
###
start=$1
shift
end=$1
shift
typeset -i do_directory=0
new_args=
typeset -i first_arg=1
for arg
do
if ((first_arg == 0))
then
a=$last_arg
if ((do_directory))
then
base_directory=${last_arg%/*}
a=$base_directory
do_directory=0
elif [[ X"$last_arg" = X"--directory" ]]
then
do_directory=1 # --directory dir-to-back-up
fi
new_args="$new_args $a"
fi
first_arg=0
last_arg=$arg
done
if [[ X"$base_directory" = X"" ]]
then
echo "$PN: cannot parse args: --directory not found" >> $log
echo "$PN: cannot parse args: --directory not found" 1>&2
return 1
elif [[ X"$last_arg" != X"." ]]
then
echo "$PN: cannot parse args: . not at end" >> $log
echo "$PN: cannot parse args: . not at end" 1>&2
return 1
fi
\cd $base_directory || return 1
typeset -i s e
s=36#$start
e=36#$end
((s = s - 10)) # 'a' -> 0
((e = e - 10)) # 'z' -> 25
alpha[0]=a alpha[1]=b alpha[2]=c alpha[3]=d alpha[4]=e alpha[5]=f
alpha[6]=g alpha[7]=h alpha[8]=i alpha[9]=j alpha[10]=k alpha[11]=l
alpha[12]=m alpha[13]=n alpha[14]=o alpha[15]=p alpha[16]=q alpha[17]=r
alpha[18]=s alpha[19]=t alpha[20]=u alpha[21]=v alpha[22]=w alpha[23]=x
alpha[24]=y alpha[25]=z
typeset -i found_homes=0
while (($s <= $e))
do
for dir in $(ls -1d ${alpha[$s]}* 2>/dev/null)
do
if [[ -d $dir ]]
then
new_args="$new_args $dir"
found_homes=1
fi
done
((s = s + 1))
done
\cd - > /dev/null
if ((found_homes == 0))
then
echo "$PN: no homes to process in range $start..$end" >> $log
echo "$PN: no homes to process in range $start..$end" 1>&2
return 1
fi
set -- $new_args
echo "$PN: running $GTAR" "$@" >> $log
$DEBUG $GTAR "$@"
}
###
# Utility functions.
###
function IsFunction
{
f=$1
echo "$PN: looking for $f" >> $log
if [[ -n "$BASH_VERSION" ]]
then
t=$(type -t "$f" 2> /dev/null)
r=$?
else
t=$(whence -v "$f" 2> /dev/null)
r=$?
fi
if [[ $r -eq 0 ]]
then
if [[ "$t" != *function* ]]
then
r=1
fi
fi
return $r
}
###
# Start of main code.
###
###
# Set up a log file in /tmp/amanda.
###
echo "$PN: start: $(date)" >> $log
echo "$PN: args:" "$@" >> $log
###
# Find the directory Amanda is asking us to back up. Also figure out
# if we are doing an estimate or the real thing.
###
typeset -i get_directory=0
typeset -i get_file=0
directory_arg=
file_arg=
for arg
do
if ((get_directory))
then
directory_arg=$arg
get_directory=0
elif ((get_file))
then
file_arg=$arg
get_file=0
elif [[ X"$arg" = X"--directory" ]]
then
get_directory=1 # --directory dir-to-back-up
elif [[ X"$arg" = X"--file" ]]
then
get_file=1 # --file file-to-write-to
fi
done
echo "$PN: directory: $directory_arg" >> $log
echo "$PN: file: $file_arg" >> $log
if [[ X"$file_arg" = X"-" ]]
then
type=real # real dump if output is stdout
elif [[ X"$file_arg" = X"/dev/null" ]]
then
type=estimate # estimate if output is /dev/null
else
type=unknown # no idea what is going on
fi
###
# Make the directory name into something we can use as part of a function
# name by converting the slashes to underscores. Hopefully there is
# not anything else annoying in the name.
###
d=$(echo $directory_arg | /bin/sed 's/[^a-zA-Z0-9_]/_/g')
###
# See if there is something to be done before we call GNU tar.
###
if IsFunction pre_${type}_$d
then
echo "$PN: running pre_${type}_$d" >> $log
pre_${type}_$d
else
echo "$PN: pre_${type}_$d not found so nothing special run" >> $log
fi
###
# Run GNU tar or a private function.
###
if IsFunction run_${type}_$d
then
echo "$PN: running run_${type}_$d" >> $log
run_${type}_$d "$@"
exit_code=$?
else
echo "$PN: running $GTAR" >> $log
$DEBUG $GTAR "$@"
exit_code=$?
fi
###
# See if there is something to be done after we ran GNU tar.
###
if IsFunction post_${type}_$d
then
echo "$PN: running post_${type}_$d" >> $log
post_${type}_$d
else
echo "$PN: post_${type}_$d not found so nothing special run" >> $log
fi
###
# Exit with the GNU tar exit code.
###
echo "$PN: end: $(date)" >> $log
exit $exit_code
