Jeff Janes <jeff.ja...@gmail.com> wrote: > Kevin Grittner <kevin.gritt...@wicourts.gov> wrote: >> We pipe the file into pg_clearxlogtail | gzip and pipe it out to >> the archive directory (with a ".gz" suffix), rather than using cp >> and processing it later. Well, actually, we pipe it to a >> directory on the same mount point as the archive directory and mv >> it into place, as part of our scheme to avoid problems with >> partial files. > > Do you have an example of that which you could share? > > I've run into two problems I'm trying to overcome. One is that > pg_clearxlogtail fails on file formats it doesn't recognize but is > asked to archive anyway, such as > '000000010000000200000065.00000020.backup', for example. Perhaps > it could just issue a warning and then pass the unrecognized file > through unchanged, instead of bailing out with a fatal error. > > The other is that a pipeline in bash reports success even if an > interior member of it failed. I think I know how to fix that > under an actual shell script (as opposed to a pipeline stuffed > into "archive_command"), but would like to see how other people > have dealt with it.
I make no claims that this couldn't be improved upon, but it has been humming along without any problems without modification for about five years now on 80-some machines. Our archive_command is just a call to a bash script that looks like this: walPathname=$1 walFilename=$2 tempfile=/var/pgsql/data/wal-files-temp/$walFilename.gz targetfile=/var/pgsql/data/wal-files/$walFilename.gz errfile=/home/ccsa/archive.err emailfile=/home/ccsa/$walFilename.rpt cat <<END > runtime walPathname=$walPathname walFilename=$walFilename tempfile=$tempfile targetfile=$targetfile errfile=$errfile emailfile=$emailfile END if [ -f $targetfile ] ; then echo "Cannot copy $walPathName to $targetfile \ because it already exists" 2> $errfile archivereturnval=3 else if echo $walFilename | grep -Eq ^[0-9A-F]{24}$ - ; then pg_clearxlogtail < $walPathname 2> $errfile \ | gzip > $tempfile 2>> $errfile \ && mv $tempfile $targetfile 2>> $errfile archivereturnval=$? else gzip < $walPathname > $tempfile 2> $errfile \ && mv $tempfile $targetfile 2>> $errfile archivereturnval=$? fi fi if [ "$archivereturnval" -eq "0" ] ; then # Successful archive. # Delete any email file from a possible prior failure. rm -f $emailfile $errfile exit 0 fi # Failed to archive. if [ ! -f $emailfile ] ; then # No existing email files -- make one and send it. echo "date: $(date +%F_%H%M)" > $emailfile echo "to: dba.gr...@wicourts.gov" >> $emailfile echo "subject: Problem with archiving $walFilename ">> $emailfile echo " " >> $emailfile echo "$archivereturnval return code from archive" >> $emailfile echo " " >> $emailfile cat $errfile >> $emailfile echo " " >> $emailfile cat runtime >> $emailfile /usr/sbin/sendmail -t < $emailfile fi cat $errfile > /dev/stderr exit $archivereturnval -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general