Re: Running amanda client only
[EMAIL PROTECTED] wrote: The server is reporting that client down, so I checked and noticed that the FBSD port for the amanda-client did not install amindexd or amidxtaped, although it did install amandad. Are all 3 needed for a client? No, for client only amandad is needed. -- Toomas Aas
"selfcheck request timed out" WAS Re: Running amanda client only
On Wed, 13 Sep 2006, Toomas Aas wrote: > [EMAIL PROTECTED] wrote: > > > The server is reporting that client down, so I checked and noticed that > > the FBSD port for the amanda-client did not install amindexd or > > amidxtaped, although it did install amandad. Are all 3 needed for a > > client? > > No, for client only amandad is needed. Then I cannot figure out why I'm getting "selfcheck request timed out" from that client. The path in inetd.conf is correct, as is the user (amanda) and /tmp/amanda is owned by amanda and has debug files there (just config info). .amandaclients has localhost.fqdn as well as hostname.fqdn. That client IS running alot of IP addresses on it, but I've done that before with no trouble. Here is amcheck -c output: su-2.05b$ amcheck -c weekly Amanda Backup Client Hosts Check ERROR: old.amanda.client: [host someIP.comcastbiz.net: hostname lookup failed] WARNING: new.amanda.client: selfcheck request timed out. Host down? Client check: 2 hosts checked in 30.153 seconds, 2 problems found I understand the first error from the old client...there is no forward DNS on that IP (BTW, is there a way around that? Just have it look at IP address?). I can't figure out the cause of the second one...I went through everything on the FAQ-O-Matic about it... James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
Re: Directory too large for single tape.
Ian Turner wrote:
On Monday 11 September 2006 01:48, Stephen Carville wrote:
I've been using Amanda for about five years now. The only problems I've
ever had are because a single dumps has to fit on a single tape.
If you upgrade to Amanda 2.5.x, then you can instruct Amanda to split dumps
across tapes; so you can make your partitions as large as you like, and just
use as many tapes as are required.
AFAICT, this doesn't really split the tar but tries to create separate
tars of subdirectories. Unfortunately RMAN puts it all in one directory
(There may be a way to split it -- I'm checking into that too).
Is it possible to get amanda to follow symlinks? Every file name ends
in a digit is it is trivial to do something like:
rm /empty/GIS1/*
find /NFS/euphrates_backup/GIS -name '*[1,3,5,7,9]' -exec ln -sf {}
/empty/GIS1/ \;
rm /empty/GIS2/*
find /NFS/euphrates_backup/GIS -name '*[0,2,4,6,8]' -exec ln -sf {}
/empty/GIS2/ \;
Then backup GIS1 and GIS2 as separate volumes.
Would it be better to try this trick with hard links? RMAN needs to be
able to delete uneeded files after each run.
The tricky part of this path is that if you have a single large filesystem
(and a few smaller ones), the single large one will dominate your tape usage,
resulting in an unbalanced backup window.
Cheers,
--Ian
--
Stephen Carville -- polluting the ranks of skeptics since 1995.
---
As democracy is perfected, the office of president represents, more and
more closely, the inner soul of the people. On some great and glorious
day the plain folks of the land will reach their heart's desire at last
and the White House will be adorned by a downright moron.
-- H. L. Mencken
Re: Directory too large for single tape.
Jon LaBadie wrote:
On Sun, Sep 10, 2006 at 10:48:08PM -0700, Stephen Carville wrote:
I've been using Amanda for about five years now. The only problems I've
ever had are because a single dumps has to fit on a single tape. I
usually solved this by using tar, client side compression, and breaking
up the partitions into smaller chunks. However, I seem to have reached
a limit.
One of my Oracle RMAN backups now stands at 109G. Even with compression
this requires 44G to 45G of space and will not fit on an AIT-2 tape.
The backups are dumped into a single directory and I don't know of any
way to split a directory with no subdirectories into two dumps. If
there is, I'd appreciate knowing how. Any other sugesstions are welcome.
Stephen,
what are we talking about in terms of files in a single directory
that total nearly 110GB? Is it one or two huge files? A hundred,
maybe a thousand, or more? What are the naming schemes? Always
the same or always predictable? Are there name extensions for
file types? If the numbers are reasonable you could create a
set of DLEs using include statements and a catch-all using
exclude statements.
For example, (assumptions include reasonable number of files,
say < 1000 but I'm not certain what is reasonable, most with names
beginning with lowercase letters but a few with digits or uppercase
and an ascii charset :)
host Ora-a2g /path/to/RMAN/backup/dir {
your_current_dumptype
include file ./[a-g]*
}
host Ora-h2s /path/to/RMAN/backup/dir {
your_current_dumptype
include file ./[h-s]*
}
host Ora-t2z /path/to/RMAN/backup/dir {
your_current_dumptype
include file ./[t-z]*
}
host Ora-rest /path/to/RMAN/backup/dir {
your_current_dumptype
exclude file ./[a-z]*
}
All of the filenames end with a digit so something like this might work:
host Ora1 /path/to/RMAN/backup/dir {
your_current_dumptype
include file ./*[0,2,4,6,8]
}
host Ora2 /path/to/RMAN/backup/dir {
your_current_dumptype
include file ./*[1,3,5,7,9]
}
Do the hostnames have to be different or was that just for illutration?
--
Stephen Carville -- polluting the ranks of skeptics since 1995.
---
As democracy is perfected, the office of president represents, more and
more closely, the inner soul of the people. On some great and glorious
day the plain folks of the land will reach their heart's desire at last
and the White House will be adorned by a downright moron.
-- H. L. Mencken
amanda 2.5.1 on Solaris 9
Hi, I'm having trouble getting the amanda 2.5.1 server installed on Solaris 9. It always gets hung during "make install" when it gets to restore-src. The client-only install works fine. I tried the server install on a couple of different Solaris 9 boxes with the same error. 2.5.0p2 installed just fine on the same box before this. Here's what I have: ./configure \ --with-user=amanda \ --with-group=backups \ --with-index-server=amanda-server \ --with-tape-server=amanda-server \ --with-tape-device=/dev/rmt/0cbn \ --with-changer-device=/dev/scsi/changer/c2t0d0 \ --with-gnutar=/usr/local/bin/tar \ --with-gnutar-listdir=/usr/local/var/amanda/gnutar-lists \ --with-assertions=yes \ --with-debugging=/var/log/amanda/debug \ --with-fqdn=yes \ --with-pid-debug-files=yes \ --without-built-manpages \ --with-ssh-security ...and here's the error during the make install: Making install in restore-src /usr/local/tars/amanda-2.5.1/restore-src make[1]: Entering directory `/usr/local/tars/amanda-2.5.1/restore-src' make[2]: Entering directory `/usr/local/tars/amanda-2.5.1/restore-src' test -z "/usr/local/lib" || /bin/bash ../config/mkinstalldirs "/usr/local/lib" /bin/bash ../libtool --mode=install /usr/ucb/install 'librestore.la' '/usr/local/lib/librestore.la' libtool: install: warning: relinking `librestore.la' (cd /usr/local/tars/amanda-2.5.1/restore-src; /bin/bash ../libtool --tag=CC --mode=relink gcc -Wall -Wextra -Wparentheses -Wdeclaration-after-statement -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wformat -Wsign-compare -Wfloat-equal -Wold-style-definition -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -o librestore.la -rpath /usr/local/lib -release 2.5.1 restore.lo ../common-src/libamanda.la ../tape-src/libamtape.la ../server-src/libamserver.la -lgen -lm -lreadline -ltermcap -lsocket -lnsl -lresolv -lintl ) gcc -shared -Wl,-h -Wl,librestore-2.5.1.so -o .libs/librestore-2.5.1.so .libs/restore.o -R/usr/local/lib -L/usr/local/lib -lamanda -lamtape -lamserver -lgen -lm -lreadline -ltermcap -lsocket -lnsl -lresolv -lintl -lc Text relocation remains referenced against symbol offset in file 0xa24 /usr/local/lib/libtermcap.a(tparam.o) 0xa28 /usr/local/lib/libtermcap.a(tparam.o) 0xa2c /usr/local/lib/libtermcap.a(tparam.o) . . . etc. . . . ld: fatal: relocations remain against allocatable but non-writable sections collect2: ld returned 1 exit status libtool: install: error: relink `librestore.la' with the above command before in stalling it make[2]: *** [install-libLTLIBRARIES] Error 1 make[2]: Leaving directory `/usr/local/tars/amanda-2.5.1/restore-src' make[1]: *** [install-am] Error 2 make[1]: Leaving directory `/usr/local/tars/amanda-2.5.1/restore-src' make: *** [install-recursive] Error 1 I've tried GNU make and GNU ld as well...same error. Anyone else have the same problem? Thanks, Ray This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Re: Directory too large for single tape.
On Wed, 13 Sep 2006 at 6:59am, Stephen Carville wrote Ian Turner wrote: If you upgrade to Amanda 2.5.x, then you can instruct Amanda to split dumps across tapes; so you can make your partitions as large as you like, and just use as many tapes as are required. AFAICT, this doesn't really split the tar but tries to create separate tars of subdirectories. Unfortunately RMAN puts it all in one directory (There may be a way to split it -- I'm checking into that too). You're incorrect -- as of 2.5.x amanda can span a single DLE across multiple tapes. Is it possible to get amanda to follow symlinks? Every file name ends in a digit is it is trivial to do something like: As other folks have mentioned (I believe), you can split a single directory up with include and/or exclude directives in the DLEs. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: Directory too large for single tape.
On 2006-09-13 15:59, Stephen Carville wrote:
Ian Turner wrote:
On Monday 11 September 2006 01:48, Stephen Carville wrote:
I've been using Amanda for about five years now. The only problems I've
ever had are because a single dumps has to fit on a single tape.
If you upgrade to Amanda 2.5.x, then you can instruct Amanda to split
dumps across tapes; so you can make your partitions as large as you
like, and just use as many tapes as are required.
AFAICT, this doesn't really split the tar but tries to create separate
tars of subdirectories. Unfortunately RMAN puts it all in one directory
(There may be a way to split it -- I'm checking into that too).
AFAICT, (and I can tell it far enough, because I've done it, instead
of going about rumours!) no no, it really splits the bytestream on
tape, nothing to do with separate tar archives of subdirectories.
Even one big 400 Gbyte single file can be put on a a tape medium less
than 400 Gbyte capacity (but you need a changer).
http://wiki.zmanda.com/index.php/Splitting_dumps_across_tapes
Is it possible to get amanda to follow symlinks? Every file name ends
in a digit is it is trivial to do something like:
rm /empty/GIS1/*
find /NFS/euphrates_backup/GIS -name '*[1,3,5,7,9]' -exec ln -sf {}
/empty/GIS1/ \;
rm /empty/GIS2/*
find /NFS/euphrates_backup/GIS -name '*[0,2,4,6,8]' -exec ln -sf {}
/empty/GIS2/ \;
Then backup GIS1 and GIS2 as separate volumes.
Too complicated.
(And no, gnutar invoked by Amanda will not follow symlinks, too
dangerous: what if "ln -s ../.. up" or similar exists?)
Disklist entries like this will work just fine:
host.example.com GIS_UNEVEN /NFS/euphrates_backup/GIS {
comp-user-tar
include "./*[13579]"
}
host.example.com GIS_EVEN /NFS/euphrates_backup/GIS {
comp-user-tar
include "./*[02468]"
}
host.example.com GIS_REST /NFS/euphrates_backup/GIS {
comp-user-tar
exclude "./*[0-9]"
}
Note that I've added a "REST" archive for those files
not ending in a number. Always use such catch-all at the
end.
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... "Are you sure?" ... YES ... Phew ... I'm out *
***
RE: amanda 2.5.1 on Solaris 9
P.S., I did a "make uninstall" and "make distclean" on the 2.5.0p2 version before trying to install 2.5.1. Thanks, Ray -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Lee, Raymond Sent: Wednesday, September 13, 2006 10:15 AM To: amanda-users@amanda.org Subject: amanda 2.5.1 on Solaris 9 Hi, I'm having trouble getting the amanda 2.5.1 server installed on Solaris 9. It always gets hung during "make install" when it gets to restore-src. The client-only install works fine. I tried the server install on a couple of different Solaris 9 boxes with the same error. 2.5.0p2 installed just fine on the same box before this. Here's what I have: ./configure \ --with-user=amanda \ --with-group=backups \ --with-index-server=amanda-server \ --with-tape-server=amanda-server \ --with-tape-device=/dev/rmt/0cbn \ --with-changer-device=/dev/scsi/changer/c2t0d0 \ --with-gnutar=/usr/local/bin/tar \ --with-gnutar-listdir=/usr/local/var/amanda/gnutar-lists \ --with-assertions=yes \ --with-debugging=/var/log/amanda/debug \ --with-fqdn=yes \ --with-pid-debug-files=yes \ --without-built-manpages \ --with-ssh-security ...and here's the error during the make install: Making install in restore-src /usr/local/tars/amanda-2.5.1/restore-src make[1]: Entering directory `/usr/local/tars/amanda-2.5.1/restore-src' make[2]: Entering directory `/usr/local/tars/amanda-2.5.1/restore-src' test -z "/usr/local/lib" || /bin/bash ../config/mkinstalldirs "/usr/local/lib" /bin/bash ../libtool --mode=install /usr/ucb/install 'librestore.la' '/usr/local/lib/librestore.la' libtool: install: warning: relinking `librestore.la' (cd /usr/local/tars/amanda-2.5.1/restore-src; /bin/bash ../libtool --tag=CC --mode=relink gcc -Wall -Wextra -Wparentheses -Wdeclaration-after-statement -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wformat -Wsign-compare -Wfloat-equal -Wold-style-definition -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -o librestore.la -rpath /usr/local/lib -release 2.5.1 restore.lo ../common-src/libamanda.la ../tape-src/libamtape.la ../server-src/libamserver.la -lgen -lm -lreadline -ltermcap -lsocket -lnsl -lresolv -lintl ) gcc -shared -Wl,-h -Wl,librestore-2.5.1.so -o .libs/librestore-2.5.1.so .libs/restore.o -R/usr/local/lib -L/usr/local/lib -lamanda -lamtape -lamserver -lgen -lm -lreadline -ltermcap -lsocket -lnsl -lresolv -lintl -lc Text relocation remains referenced against symbol offset in file 0xa24 /usr/local/lib/libtermcap.a(tparam.o) 0xa28 /usr/local/lib/libtermcap.a(tparam.o) 0xa2c /usr/local/lib/libtermcap.a(tparam.o) . . . etc. . . . ld: fatal: relocations remain against allocatable but non-writable sections collect2: ld returned 1 exit status libtool: install: error: relink `librestore.la' with the above command before in stalling it make[2]: *** [install-libLTLIBRARIES] Error 1 make[2]: Leaving directory `/usr/local/tars/amanda-2.5.1/restore-src' make[1]: *** [install-am] Error 2 make[1]: Leaving directory `/usr/local/tars/amanda-2.5.1/restore-src' make: *** [install-recursive] Error 1 I've tried GNU make and GNU ld as well...same error. Anyone else have the same problem? Thanks, Ray This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Re: amanda 2.5.1 on Solaris 9
On Wed, Sep 13, 2006 at 09:29:38AM -0500, Lee, Raymond wrote: > P.S., I did a "make uninstall" and "make distclean" on the 2.5.0p2 > version before trying to install 2.5.1. > > Thanks, > Ray > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Lee, Raymond > Sent: Wednesday, September 13, 2006 10:15 AM > To: amanda-users@amanda.org > Subject: amanda 2.5.1 on Solaris 9 > > > Hi, > > I'm having trouble getting the amanda 2.5.1 server installed on Solaris > 9. It always gets hung during "make install" when it gets to > restore-src. The client-only install works fine. I tried the server > install on a couple of different Solaris 9 boxes with the same error. > 2.5.0p2 installed just fine on the same box before this. Here's what I > have: > > ./configure \ > --with-user=amanda \ > --with-group=backups \ > --with-index-server=amanda-server \ > --with-tape-server=amanda-server \ > --with-tape-device=/dev/rmt/0cbn \ > --with-changer-device=/dev/scsi/changer/c2t0d0 \ > --with-gnutar=/usr/local/bin/tar \ > --with-gnutar-listdir=/usr/local/var/amanda/gnutar-lists \ > --with-assertions=yes \ > --with-debugging=/var/log/amanda/debug \ > --with-fqdn=yes \ > --with-pid-debug-files=yes \ > --without-built-manpages \ > --with-ssh-security > > > gcc -shared -Wl,-h -Wl,librestore-2.5.1.so -o .libs/librestore-2.5.1.so > .libs/restore.o -R/usr/local/lib -L/usr/local/lib -lamanda -lamtape > -lamserver -lgen -lm -lreadline -ltermcap -lsocket -lnsl -lresolv -lintl > -lc > Text relocation remains referenced > against symbol offset in file >0xa24 > /usr/local/lib/libtermcap.a(tparam.o) > . Libtermcap has always seems to be a problem with Solaris. I eliminate it and use ncurses instead. Try adding these to your configure line: --without-termcap \ --without-ltermcap \ --without-LIBTERMCAP \ --with-ncurses=yes \ --with-lncurses=yes \ --with-LIBNCURSES=yes \ This assumes you have a libncurses available. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Why didn't my backup work (the way I thought it should)?
I'm just completing the first bare-metal recovery I've experienced in my
six years as a system administrator. A backplane seemed to fail and
wiped out two disks of a four-disk RAID-5 array. After replacing all the
suspect parts, I followed this process to restore the host:
--Installed an IDE hard drive and configured to boot before the SCSI
RAID drives
--Installed Debian 3.1r3 on the IDE drive.
--Mounted the SCSI array and partitioned as the original was
--Manually examined my backup tapes (tar) and assembled a list of the
tape file to restore, in the proper order.
--Restored the data to the proper partitions on the SCSI array.
--Powered down the host, removed the IDE drive, and booted using Debian
CD and 'rescue root=/dev/sda2'. This worked correctly.
--Ran lilo to restore the MBR. After removing CD, 'shutdown -r' seems to
work correctly.
I thought that after all this, I'd be done. However, I'm discovering
pockets of files and directories that seem to have never made it to the
backup tapes in the first place. Some examples:
--/service/tinydns, dnscache, qmail-send, ... All these were soft links
to the files, so from the recent discussion, they wouldn't have been
backed up by tar. Is there a recommended procedure for dealing with
this? Would using 'dump' have prevented this?
--/var/qmail/* Directories below this level, such as control and bin,
exist, but they're empty
--/var/lib/dpkg/* (This is frustrating my efforts to restore this Debian
system with 'aptitude' or 'apt-get.')
--/var/spool/cron/crontabs Empty
--/var/amanda/* (All my amanda database information is missing.)
One common thread is that all of these directories are in /var/.
However, other files in this branch, such as /var/lib/apt/lists/* seem
to be restored just fine.
I don't understand why these files and directories are missing. My
disklist includes:
[EMAIL PROTECTED]:/etc/amanda/DBackup$ grep -i center disklist
centernet sda2 tar #/
centernet sda14 tar #/boot
centernet sda3 tar #/usr
centernet sda5 tar #/opt/analog/logdata
centernet sda6 tar #/var/www/centernet/htdocs
centernet sda7 tar #/var/lib/mysql
centernet sda9 tar #/var/www/centernet/logs
[EMAIL PROTECTED]:/etc/amanda/DBackup$
This matches the /etc/fstab on this host:
[EMAIL PROTECTED]:/etc/amanda/DBackup$ cat /etc/fstab
# /etc/fstab: static file system information.
#
#
/dev/sda2 / ext2
errors=remount-ro 0 1
/dev/sda14 /boot ext2defaults
0 2
/dev/sda3 /usrext2defaults
0 2
/dev/sda5 /opt/analog/logdata ext2defaults
0 2
/dev/sda6 /var/www/centernet/htdocs ext2defaults
0 2
/dev/sda7 /var/lib/mysql ext2defaults
0 2
/dev/sda9 /var/www/centernet/logs ext2defaults
0 2
/dev/sda8 /dumps/amanda ext2defaults
0 2
/dev/sda10 /dumps/amanda2 ext2defaults
0 2
[EMAIL PROTECTED]:/etc/amanda/DBackup$
Here's what I think are the relevant portions of amanda.conf:
define dumptype global {
comment "Global definitions"
}
define dumptype tar {
global
program "GNUTAR"
}
I don't understand why these files and directories didn't end up on the
root (sda2) tape, like I expected.
Can anyone show me an error that I made, or did something go wrong that
wasn't supposed to occur?
To make this work more smoothly the next time, six years from now, when
I have to do another bare-metal backup, what should I change?
Thanks for your advice and suggestions.
-Kevin
Kevin Zembower
Internet Services Group manager
Center for Communication Programs
Bloomberg School of Public Health
Johns Hopkins University
111 Market Place, Suite 310
Baltimore, Maryland 21202
410-659-6139
Re: "selfcheck request timed out" WAS Re: Running amanda client only
Replying to my own post (sorry, but I'm getting desperate here), I took the advice of the FAQ-O-Matic and installed lsof and ran this command: su-2.05b# lsof -uamanda There was no output. Should there have been? Does this mean that amandad is not running? This is what I have in my /etc/inetd.conf: amanda dgram udp wait amanda /usr/local/libexec/amandad amandad inetd.conf has beek killed and restarted, the client even rebooted...anything? On Wed, 13 Sep 2006 [EMAIL PROTECTED] wrote: > On Wed, 13 Sep 2006, Toomas Aas wrote: > > > [EMAIL PROTECTED] wrote: > > > > > The server is reporting that client down, so I checked and noticed that > > > the FBSD port for the amanda-client did not install amindexd or > > > amidxtaped, although it did install amandad. Are all 3 needed for a > > > client? > > > > No, for client only amandad is needed. > > Then I cannot figure out why I'm getting "selfcheck request timed out" > from that client. The path in inetd.conf is correct, as is the user > (amanda) and /tmp/amanda is owned by amanda and has debug files there > (just config info). .amandaclients has localhost.fqdn as well as > hostname.fqdn. That client IS running alot of IP addresses on it, but > I've done that before with no trouble. > > Here is amcheck -c output: > > su-2.05b$ amcheck -c weekly > > Amanda Backup Client Hosts Check > > ERROR: old.amanda.client: [host someIP.comcastbiz.net: hostname lookup > failed] > WARNING: new.amanda.client: selfcheck request timed out. Host down? > Client check: 2 hosts checked in 30.153 seconds, 2 problems found > > I understand the first error from the old client...there is no forward DNS > on that IP (BTW, is there a way around that? Just have it look at IP > address?). I can't figure out the cause of the second one...I went > through everything on the FAQ-O-Matic about it... > > James Smallacombe PlantageNet, Inc. CEO and Janitor > [EMAIL PROTECTED] > http://3.am > = > > James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
Re: Why didn't my backup work (the way I thought it should)?
Zembower, Kevin wrote: I thought that after all this, I'd be done. However, I'm discovering pockets of files and directories that seem to have never made it to the backup tapes in the first place. Some examples: --/service/tinydns, dnscache, qmail-send, ... All these were soft links to the files, so from the recent discussion, they wouldn't have been backed up by tar. Is there a recommended procedure for dealing with this? Would using 'dump' have prevented this? I must have missed something. Symbolic links are supposed to be backed up by gnutar (as links obviously, not the files they point to), independently of where they point or, for that matter, whether their target exists or not. If that's not what you observe, or what the consensus on the list indicates, I think there's something wrong. Alex -- Alexander Jolk / BUF Compagnie tel +33-1 42 68 18 28 / fax +33-1 42 68 18 29
Re: "selfcheck request timed out" WAS Re: Running amanda client only
[EMAIL PROTECTED] wrote: > Replying to my own post (sorry, but I'm getting desperate here), I took > the advice of the FAQ-O-Matic and installed lsof and ran this command: > > su-2.05b# lsof -uamanda > > There was no output. Should there have been? Does this mean that amandad > is not running? No, amandad should not be running except during a backup. When the server connects to the amanda port on the client inetd starts amandad. > This is what I have in my /etc/inetd.conf: > > amanda dgram udp wait amanda /usr/local/libexec/amandad amandad > > inetd.conf has beek killed and restarted, the client even > rebooted...anything? Have you tried running /usr/local/libexec/amandad from the command line on the client (it should just sit there and eventually time out and return your prompt, or immediately exit if you hit a key)? Perhaps you're missing a library (or need to run ldconfig or whatever is needed to update the dynamic library cache on your platform). Does the system log inetd even running amandad? Does amandad run and create anything in /tmp/amanda? Frank > > On Wed, 13 Sep 2006 [EMAIL PROTECTED] wrote: > >> On Wed, 13 Sep 2006, Toomas Aas wrote: >> >>> [EMAIL PROTECTED] wrote: >>> The server is reporting that client down, so I checked and noticed that the FBSD port for the amanda-client did not install amindexd or amidxtaped, although it did install amandad. Are all 3 needed for a client? >>> No, for client only amandad is needed. >> Then I cannot figure out why I'm getting "selfcheck request timed out" >> from that client. The path in inetd.conf is correct, as is the user >> (amanda) and /tmp/amanda is owned by amanda and has debug files there >> (just config info). .amandaclients has localhost.fqdn as well as >> hostname.fqdn. That client IS running alot of IP addresses on it, but >> I've done that before with no trouble. >> >> Here is amcheck -c output: >> >> su-2.05b$ amcheck -c weekly >> >> Amanda Backup Client Hosts Check >> >> ERROR: old.amanda.client: [host someIP.comcastbiz.net: hostname lookup >> failed] >> WARNING: new.amanda.client: selfcheck request timed out. Host down? >> Client check: 2 hosts checked in 30.153 seconds, 2 problems found >> >> I understand the first error from the old client...there is no forward DNS >> on that IP (BTW, is there a way around that? Just have it look at IP >> address?). I can't figure out the cause of the second one...I went >> through everything on the FAQ-O-Matic about it... >> >> James Smallacombe PlantageNet, Inc. CEO and Janitor >> [EMAIL PROTECTED] >> http://3.am >> = >> >> > > James Smallacombe PlantageNet, Inc. CEO and Janitor > [EMAIL PROTECTED] > http://3.am > = > -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Continued ACK Problems with 2.5.1
ambiance still refuses to back up: ambiance.med.utah.edu sdc1 lev 0 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu sdc1 lev 0 FAILED [too many dumper retry: "[request failed: timeout waiting for ACK]"] ambiance.med.utah.edu sdc1 lev 0 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu sdb5 lev 0 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu / lev 1 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu sdb5 lev 0 FAILED [too many dumper retry: "[request failed: timeout waiting for ACK]"] ambiance.med.utah.edu sdb5 lev 0 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu / lev 1 FAILED [too many dumper retry: "[request failed: timeout waiting for ACK]"] ambiance.med.utah.edu / lev 1 FAILED [cannot read header: got 0 instead of 32768] In common-src/protocol.c I have: #define ACK_WAIT 10 /* time (secs) to wait for ACK - keep short */ #define ACK_TRIES 10 /* num retries after ACK_WAIT timeout */ the maxdumps setting worked great though, I logged in last night and the load average on ambiance was around 5 with 4 gtars running. Does anyone have any ideas? Steve -- Steven J. BackusComputer Specialist University of Utah E-Mail: [EMAIL PROTECTED] Biomedical Informatics Alternate: [EMAIL PROTECTED] 391 Chipeta Way -- Suite D150 Office: 801.587.9308 Salt Lake City, UT 84108-1266 http://www.math.utah.edu/~backus
RE: "selfcheck request timed out" WAS Re: Running amanda client only
The command you should run is: lsof -I :amanda Which should show the inetd/xinetd process listening on the amanda port (UDP port 10080). The output should look something like this: ritcdx> lsof -i :amanda COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME inetd 1285913 root 28u IPv4 0x5fe888c0 0t0 UDP *:amanda Unless you are very lucky during your testing, this is probably all you will ever see on the port except when Amanda is actually performing a backup. Best wishes, Donald L. (Don) Ritchey Information Technology, Exelon Corporation -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, September 13, 2006 11:41 AM To: amanda-users@amanda.org Subject: Re: "selfcheck request timed out" WAS Re: Running amanda client only Replying to my own post (sorry, but I'm getting desperate here), I took the advice of the FAQ-O-Matic and installed lsof and ran this command: su-2.05b# lsof -uamanda There was no output. Should there have been? Does this mean that amandad is not running? This is what I have in my /etc/inetd.conf: amanda dgram udp wait amanda /usr/local/libexec/amandad amandad inetd.conf has beek killed and restarted, the client even rebooted...anything? On Wed, 13 Sep 2006 [EMAIL PROTECTED] wrote: > On Wed, 13 Sep 2006, Toomas Aas wrote: > > > [EMAIL PROTECTED] wrote: > > > > > The server is reporting that client down, so I checked and noticed that > > > the FBSD port for the amanda-client did not install amindexd or > > > amidxtaped, although it did install amandad. Are all 3 needed for a > > > client? > > > > No, for client only amandad is needed. > > Then I cannot figure out why I'm getting "selfcheck request timed out" > from that client. The path in inetd.conf is correct, as is the user > (amanda) and /tmp/amanda is owned by amanda and has debug files there > (just config info). .amandaclients has localhost.fqdn as well as > hostname.fqdn. That client IS running alot of IP addresses on it, but > I've done that before with no trouble. > > Here is amcheck -c output: > > su-2.05b$ amcheck -c weekly > > Amanda Backup Client Hosts Check > > ERROR: old.amanda.client: [host someIP.comcastbiz.net: hostname lookup > failed] > WARNING: new.amanda.client: selfcheck request timed out. Host down? > Client check: 2 hosts checked in 30.153 seconds, 2 problems found > > I understand the first error from the old client...there is no forward DNS > on that IP (BTW, is there a way around that? Just have it look at IP > address?). I can't figure out the cause of the second one...I went > through everything on the FAQ-O-Matic about it... > > James Smallacombe PlantageNet, Inc. CEO and Janitor > [EMAIL PROTECTED] http://3.am > = > > James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am = - ** This e-mail and any of its attachments may contain Exelon Corporation proprietary information, which is privileged, confidential, or subject to copyright belonging to the Exelon Corporation family of Companies. This e-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately and permanently delete the original and any copy of this e-mail and any printout. Thank You. **
Re: "selfcheck request timed out" WAS Re: Running amanda client only
On Wed, 13 Sep 2006, Frank Smith wrote: > [EMAIL PROTECTED] wrote: > > Replying to my own post (sorry, but I'm getting desperate here), I took > > the advice of the FAQ-O-Matic and installed lsof and ran this command: > > > > su-2.05b# lsof -uamanda > > > > There was no output. Should there have been? Does this mean that amandad > > is not running? > > No, amandad should not be running except during a backup. When the server > connects to the amanda port on the client inetd starts amandad. > > > This is what I have in my /etc/inetd.conf: > > > > amanda dgram udp wait amanda /usr/local/libexec/amandad amandad > > > > inetd.conf has beek killed and restarted, the client even > > rebooted...anything? > > Have you tried running /usr/local/libexec/amandad from the command line > on the client (it should just sit there and eventually time out and > return your prompt, or immediately exit if you hit a key)? Perhaps > you're missing a library (or need to run ldconfig or whatever is > needed to update the dynamic library cache on your platform). > Does the system log inetd even running amandad? > Does amandad run and create anything in /tmp/amanda? Thanks for your reply. I did run it from the command line and it eventually times out, although it does not exit immediately if I type something. debug files are created in /tmp/amanda, but not with any useful info...just how it was compiled. There is absolutely nothing in /var/log/messages about amanda(d). I thought ldconfig was run upon boot...in any case, if I'm missing some kind of lib, wouldn't amanda complain about it when trying to build it? Thanks again for any help... James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
Re: Why didn't my backup work (the way I thought it should)?
Zembower, Kevin wrote:
> I'm just completing the first bare-metal recovery I've experienced in my
> six years as a system administrator. A backplane seemed to fail and
> wiped out two disks of a four-disk RAID-5 array. After replacing all the
> suspect parts, I followed this process to restore the host:
> --Installed an IDE hard drive and configured to boot before the SCSI
> RAID drives
> --Installed Debian 3.1r3 on the IDE drive.
> --Mounted the SCSI array and partitioned as the original was
> --Manually examined my backup tapes (tar) and assembled a list of the
> tape file to restore, in the proper order.
> --Restored the data to the proper partitions on the SCSI array.
> --Powered down the host, removed the IDE drive, and booted using Debian
> CD and 'rescue root=/dev/sda2'. This worked correctly.
> --Ran lilo to restore the MBR. After removing CD, 'shutdown -r' seems to
> work correctly.
>
> I thought that after all this, I'd be done. However, I'm discovering
> pockets of files and directories that seem to have never made it to the
> backup tapes in the first place. Some examples:
> --/service/tinydns, dnscache, qmail-send, ... All these were soft links
> to the files, so from the recent discussion, they wouldn't have been
> backed up by tar. Is there a recommended procedure for dealing with
> this? Would using 'dump' have prevented this?
> --/var/qmail/* Directories below this level, such as control and bin,
> exist, but they're empty
> --/var/lib/dpkg/* (This is frustrating my efforts to restore this Debian
> system with 'aptitude' or 'apt-get.')
> --/var/spool/cron/crontabs Empty
> --/var/amanda/* (All my amanda database information is missing.)
>
> One common thread is that all of these directories are in /var/.
> However, other files in this branch, such as /var/lib/apt/lists/* seem
> to be restored just fine.
Did you have an exclude list?
Were any of those directories symlinks to other filesystems or NFS
mounts? A symlink would have still been backed up, just not what it
was pointing to, and a mount point would just show up as an empty
directory.
Did the system have problems before the last backup? Perhaps these
files were corrupted/gone before/during the last backup. Since
Amanda restores the state at a point in time, you may have most of
the data on a previous level that the latest level removed (related
to recommendations often given here to restore to a tmp directory
and move files into place since otherwise existing new data in the
target directory would be deleted by the restore). You might want
to try recovering from tapes from a day or two before and see if
the data is there, the dpkg data at least probably hadn't changed
much from earlier backups.
>
> I don't understand why these files and directories are missing. My
> disklist includes:
> [EMAIL PROTECTED]:/etc/amanda/DBackup$ grep -i center disklist
> centernet sda2 tar #/
> centernet sda14 tar #/boot
> centernet sda3 tar #/usr
> centernet sda5 tar #/opt/analog/logdata
> centernet sda6 tar #/var/www/centernet/htdocs
> centernet sda7 tar #/var/lib/mysql
> centernet sda9 tar #/var/www/centernet/logs
> [EMAIL PROTECTED]:/etc/amanda/DBackup$
>
> This matches the /etc/fstab on this host:
> [EMAIL PROTECTED]:/etc/amanda/DBackup$ cat /etc/fstab
> # /etc/fstab: static file system information.
> #
> #
>
> /dev/sda2 / ext2
> errors=remount-ro 0 1
>
> /dev/sda14 /boot ext2defaults
> 0 2
> /dev/sda3 /usrext2defaults
> 0 2
> /dev/sda5 /opt/analog/logdata ext2defaults
> 0 2
> /dev/sda6 /var/www/centernet/htdocs ext2defaults
> 0 2
> /dev/sda7 /var/lib/mysql ext2defaults
> 0 2
> /dev/sda9 /var/www/centernet/logs ext2defaults
> 0 2
> /dev/sda8 /dumps/amanda ext2defaults
> 0 2
> /dev/sda10 /dumps/amanda2 ext2defaults
> 0 2
> [EMAIL PROTECTED]:/etc/amanda/DBackup$
>
> Here's what I think are the relevant portions of amanda.conf:
> define dumptype global {
> comment "Global definitions"
> }
>
> define dumptype tar {
> global
> program "GNUTAR"
> }
>
> I don't understand why these files and directories didn't end up on the
> root (sda2) tape, like I expected.
>
> Can anyone show me an error that I made, or did something go wrong that
> wasn't supposed to occur?
>
> To make this work more smoothly the next time, six years from now, when
> I have to do another bare-metal backup, what should I change?
Test restores more than once per 6 years to verify you are backing up
what you think you are, since data seems to get shifted around and
the excludes you need today eventually end up excluding things you need.
Also, backup media (tape and disk) can fail in ways that g
Re: "selfcheck request timed out" WAS Re: Running amanda client only
[EMAIL PROTECTED] wrote: > On Wed, 13 Sep 2006, Frank Smith wrote: > >> [EMAIL PROTECTED] wrote: >>> Replying to my own post (sorry, but I'm getting desperate here), I took >>> the advice of the FAQ-O-Matic and installed lsof and ran this command: >>> >>> su-2.05b# lsof -uamanda >>> >>> There was no output. Should there have been? Does this mean that amandad >>> is not running? >> No, amandad should not be running except during a backup. When the server >> connects to the amanda port on the client inetd starts amandad. >> >>> This is what I have in my /etc/inetd.conf: >>> >>> amanda dgram udp wait amanda /usr/local/libexec/amandad amandad >>> >>> inetd.conf has beek killed and restarted, the client even >>> rebooted...anything? >> Have you tried running /usr/local/libexec/amandad from the command line >> on the client (it should just sit there and eventually time out and >> return your prompt, or immediately exit if you hit a key)? Perhaps >> you're missing a library (or need to run ldconfig or whatever is >> needed to update the dynamic library cache on your platform). >> Does the system log inetd even running amandad? >> Does amandad run and create anything in /tmp/amanda? > > Thanks for your reply. I did run it from the command line and it > eventually times out, although it does not exit immediately if I type > something. 2.4.5 will exit if you hit enter, 2.5 doesn't. debug files are created in /tmp/amanda, but not with any > useful info...just how it was compiled. There is absolutely nothing in > /var/log/messages about amanda(d). Inetd doesn't normally log unless you start it with a debug option. In your case, since a debug file is created then inetd is working so you may not need to check further on that. Post the debug file on the client and maybe we can see at what stage of the connection it stops. Are you running a firewall on or between the client? > I thought ldconfig was run upon > boot...in any case, if I'm missing some kind of lib, wouldn't amanda > complain about it when trying to build it? The ldconfig suggestion is only relevant if running amandad from the command line returns a missing library error, sorry I wasn't clear about that. And yes it would complain on build, I was thinking more of cases where amanda lives on a shared filesystem or are copied to multiple machines. Frank > > Thanks again for any help... > > James Smallacombe PlantageNet, Inc. CEO and Janitor > [EMAIL PROTECTED] > http://3.am > = > -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
RE: amanda 2.5.1 on Solaris 9
Thanks, Jon. That did the trick. I already had ncurses installed, but running configure --without-termcap, etc. still tried to use termcap. So I uninstalled termcap and ran configure again. The make install ran clean this time. amrecover is working fine, so I guess it's all good. Thanks, Ray -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jon LaBadie Sent: Wednesday, September 13, 2006 11:24 AM To: amanda-users@amanda.org Subject: Re: amanda 2.5.1 on Solaris 9 On Wed, Sep 13, 2006 at 09:29:38AM -0500, Lee, Raymond wrote: > P.S., I did a "make uninstall" and "make distclean" on the 2.5.0p2 > version before trying to install 2.5.1. > > Thanks, > Ray > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Lee, Raymond > Sent: Wednesday, September 13, 2006 10:15 AM > To: amanda-users@amanda.org > Subject: amanda 2.5.1 on Solaris 9 > > > Hi, > > I'm having trouble getting the amanda 2.5.1 server installed on Solaris > 9. It always gets hung during "make install" when it gets to > restore-src. The client-only install works fine. I tried the server > install on a couple of different Solaris 9 boxes with the same error. > 2.5.0p2 installed just fine on the same box before this. Here's what I > have: > > ./configure \ > --with-user=amanda \ > --with-group=backups \ > --with-index-server=amanda-server \ > --with-tape-server=amanda-server \ > --with-tape-device=/dev/rmt/0cbn \ > --with-changer-device=/dev/scsi/changer/c2t0d0 \ > --with-gnutar=/usr/local/bin/tar \ > --with-gnutar-listdir=/usr/local/var/amanda/gnutar-lists \ > --with-assertions=yes \ > --with-debugging=/var/log/amanda/debug \ > --with-fqdn=yes \ > --with-pid-debug-files=yes \ > --without-built-manpages \ > --with-ssh-security > > > gcc -shared -Wl,-h -Wl,librestore-2.5.1.so -o .libs/librestore-2.5.1.so > .libs/restore.o -R/usr/local/lib -L/usr/local/lib -lamanda -lamtape > -lamserver -lgen -lm -lreadline -ltermcap -lsocket -lnsl -lresolv -lintl > -lc > Text relocation remains referenced > against symbol offset in file >0xa24 > /usr/local/lib/libtermcap.a(tparam.o) > . Libtermcap has always seems to be a problem with Solaris. I eliminate it and use ncurses instead. Try adding these to your configure line: --without-termcap \ --without-ltermcap \ --without-LIBTERMCAP \ --with-ncurses=yes \ --with-lncurses=yes \ --with-LIBNCURSES=yes \ This assumes you have a libncurses available. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax) This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Re: "selfcheck request timed out" WAS Re: Running amanda client only
On Wed, 13 Sep 2006, Frank Smith wrote: > [EMAIL PROTECTED] wrote: > > On Wed, 13 Sep 2006, Frank Smith wrote: > > > >> [EMAIL PROTECTED] wrote: > >>> Replying to my own post (sorry, but I'm getting desperate here), I took > >>> the advice of the FAQ-O-Matic and installed lsof and ran this command: > >>> > >>> su-2.05b# lsof -uamanda > >>> > >>> There was no output. Should there have been? Does this mean that amandad > >>> is not running? > >> No, amandad should not be running except during a backup. When the server > >> connects to the amanda port on the client inetd starts amandad. > >> > >>> This is what I have in my /etc/inetd.conf: > >>> > >>> amanda dgram udp wait amanda /usr/local/libexec/amandad amandad > >>> > >>> inetd.conf has beek killed and restarted, the client even > >>> rebooted...anything? > >> Have you tried running /usr/local/libexec/amandad from the command line > >> on the client (it should just sit there and eventually time out and > >> return your prompt, or immediately exit if you hit a key)? Perhaps > >> you're missing a library (or need to run ldconfig or whatever is > >> needed to update the dynamic library cache on your platform). > >> Does the system log inetd even running amandad? > >> Does amandad run and create anything in /tmp/amanda? > > > > Thanks for your reply. I did run it from the command line and it > > eventually times out, although it does not exit immediately if I type > > something. > > 2.4.5 will exit if you hit enter, 2.5 doesn't. The client is 2.5.0p2, the server is 2.4.5. Could that be an issue? If so, which one do you recommend? > debug files are created in /tmp/amanda, but not with any > > useful info...just how it was compiled. There is absolutely nothing in > > /var/log/messages about amanda(d). > > Inetd doesn't normally log unless you start it with a debug option. > In your case, since a debug file is created then inetd is working > so you may not need to check further on that. > Post the debug file on the client and maybe we can see at what stage > of the connection it stops. Are you running a firewall on or between > the client? No firewall that I've configured...I did just hook up via comcast business though...nmap from the client back to the server looks ok...I can ssh back, as well. the debug files had what looked like nothing but a config.log, until I just looked now and noticed this huge (48MB) debug file with 670 thousand lines of this: amandad: dgram_recv: recvfrom() failed: Socket operation on non-socket amandad: dgram_recv: recvfrom() failed: Socket operation on non-socket amandad: dgram_recv: recvfrom() failed: Socket operation on non-socket James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
RE: Why didn't my backup work (the way I thought it should)?
Frank and Alex, thanks for replying. Alex, you're right, I believe, GNUtar behaves differently than tar with respect to symbolic links, and I'm using GNUtar: [EMAIL PROTECTED]:/var/lib$ tar --version tar (GNU tar) 1.14 Frank, all my entries look like this one for sda2 (root): [EMAIL PROTECTED]:/var/lib$ amadmin DBackup disklist centernet line 48: host centernet: interface default disk sda2: program "GNUTAR" priority 1 dumpcycle 28 maxdumps 1 maxpromoteday 1 strategy STANDARD compress CLIENT FAST comprate 0.50 0.50 auth BSD kencrypt NO holdingdisk YES record YES index YES skip-incr NO skip-full NO >From this, it seems there's no exclude lists in the disklist. It also seems that there isn't one on the host, which is both client and server (tapehost, amandahost): [EMAIL PROTECTED]:/var/lib$ locate exclude /home/kevinz/perl-5.8.6/perl-5.8.6/plan9/exclude [EMAIL PROTECTED]:/var/lib$ None were NSF filesystems or links to other filesystems. Your suggestion that the last tapes might be corrupt is an excellent one, and one that I'll begin exploring immediately. Your suggestion to check the restore procedures more than once every six years is also good, and one that I'll also put into practice just as soon as this fire is out. You also caused me to question if I performed the restore correctly. I manually went through the first record on each tape file and copied the filesystem and level. It's possible that I omitted a level 0 and only restored a level 1. Just about all the files missing are very static ones that change infrequently. I'll have to check into this, too. Thanks, again. -Kevin -Original Message- From: Frank Smith [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 13, 2006 1:20 PM To: Zembower, Kevin Cc: amanda-users@amanda.org Subject: Re: Why didn't my backup work (the way I thought it should)? Did you have an exclude list? Were any of those directories symlinks to other filesystems or NFS mounts? A symlink would have still been backed up, just not what it was pointing to, and a mount point would just show up as an empty directory. Did the system have problems before the last backup? Perhaps these files were corrupted/gone before/during the last backup. Since Amanda restores the state at a point in time, you may have most of the data on a previous level that the latest level removed (related to recommendations often given here to restore to a tmp directory and move files into place since otherwise existing new data in the target directory would be deleted by the restore). You might want to try recovering from tapes from a day or two before and see if the data is there, the dpkg data at least probably hadn't changed much from earlier backups. Test restores more than once per 6 years to verify you are backing up what you think you are, since data seems to get shifted around and the excludes you need today eventually end up excluding things you need. Also, backup media (tape and disk) can fail in ways that give no errors during backups but make a restore impossible, and the sooner you are aware of it the better. Frank > > Thanks for your advice and suggestions. > > -Kevin > > Kevin Zembower > Internet Services Group manager > Center for Communication Programs > Bloomberg School of Public Health > Johns Hopkins University > 111 Market Place, Suite 310 > Baltimore, Maryland 21202 > 410-659-6139 > -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Fixed amcheck, one more question (or two)
Sorry for all the whining...it turned out the "host down" was the same reverse DNS problem that the other amanda client was returning, but since the error message was totally different, it baffled. I hooked the server back up to the old network and it worked fine. 2 more minor issues: I'm getting this: ERROR: label FULL0 match labelstr but it not listed in the tapelist file. (expecting a new tape) I thought that amanda would simply create a new tapelist...I deleted the old one because it was years old. Do I need to manually create a new one? If so, which dates do I use...I simply created an empty file figuring amanda would take care of the rest. Lastly (I hope), is there a way to get amanda to not do host lookups and simply use IP addresses in .amandahosts? For security, I think IP addresses would be better. Thanks again! James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
Re: Why didn't my backup work (the way I thought it should)?
On Wed, Sep 13, 2006 at 02:47:26PM -0400, Zembower, Kevin wrote: > Frank and Alex, thanks for replying. Alex, you're right, I believe, > GNUtar behaves differently than tar with respect to symbolic links, and > I'm using GNUtar: > [EMAIL PROTECTED]:/var/lib$ tar --version > tar (GNU tar) 1.14 > Likely unrelated to your current difficulties, but plain 1.14 gnutar has be reported to have difficulties with amanda. I haven't followed the discussion closely, but I think 1.14.1 and 1.14.2 are both considered ok. A complicating factor is the report that some linux vendors, I think RH was mentioned, supply a 1.14 with the later bug fix patches applied. These work ok also but because they are based on the 1.14 code, that is the version reported. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Directory too large for single tape.
On Wed, Sep 13, 2006 at 07:07:52AM -0700, Stephen Carville wrote:
> Jon LaBadie wrote:
> >On Sun, Sep 10, 2006 at 10:48:08PM -0700, Stephen Carville wrote:
> >
> >>I've been using Amanda for about five years now. The only problems I've
> >>ever had are because a single dumps has to fit on a single tape. I
> >>usually solved this by using tar, client side compression, and breaking
> >>up the partitions into smaller chunks. However, I seem to have reached
> >>a limit.
> >>
> >>One of my Oracle RMAN backups now stands at 109G. Even with compression
> >>this requires 44G to 45G of space and will not fit on an AIT-2 tape.
> >>The backups are dumped into a single directory and I don't know of any
> >>way to split a directory with no subdirectories into two dumps. If
> >>there is, I'd appreciate knowing how. Any other sugesstions are welcome.
> >
> >
> >Stephen,
> >what are we talking about in terms of files in a single directory
> >that total nearly 110GB? Is it one or two huge files? A hundred,
> >maybe a thousand, or more? What are the naming schemes? Always
> >the same or always predictable? Are there name extensions for
> >file types? If the numbers are reasonable you could create a
> >set of DLEs using include statements and a catch-all using
> >exclude statements.
> >
> >For example, (assumptions include reasonable number of files,
> >say < 1000 but I'm not certain what is reasonable, most with names
> >beginning with lowercase letters but a few with digits or uppercase
> >and an ascii charset :)
> >
> > host Ora-a2g /path/to/RMAN/backup/dir {
> > your_current_dumptype
> > include file ./[a-g]*
> > }
> > host Ora-h2s /path/to/RMAN/backup/dir {
> > your_current_dumptype
> > include file ./[h-s]*
> > }
> > host Ora-t2z /path/to/RMAN/backup/dir {
> > your_current_dumptype
> > include file ./[t-z]*
> > }
> > host Ora-rest /path/to/RMAN/backup/dir {
> > your_current_dumptype
> > exclude file ./[a-z]*
> > }
> >
>
> All of the filenames end with a digit so something like this might work:
>
> host Ora1 /path/to/RMAN/backup/dir {
> your_current_dumptype
> include file ./*[0,2,4,6,8]
> }
>
> host Ora2 /path/to/RMAN/backup/dir {
> your_current_dumptype
> include file ./*[1,3,5,7,9]
> }
>
Well, that first one says you want to backup files whose
names end in even digits OR comma. Probably not what you
mean. Drop the commas.
And be sure to also include a third, catchall DLE that
does an exclude file ./*[0-9].
OH, I made the error of forgetting double quotes around
each of the include/exclude file patterns "./*[02468]"
> Do the hostnames have to be different or was that just for illutration?
>
I didn't have different hostnames. Each one simply said "host".
What you probably are refering to is the DLE_Name.
The start of a disklist entry can have two forms:
HostName DLE_Name
HostName DLE_Name PathToStartingDirectory
The PathTo... may also be a device name for mounted filesystems
listed in /etc/{v}fstab.
If the first form is used, the DLE_Name must take the form of a
pathname because it is also the PathTo...
With either form, an amanda requirement is that the combination
Hostname:DLE_Name must be unique in your disklist. Because of
this requirement, if you have more than one DLE starting at the
same directory, you must use the second form and have different
DLE_Names for each DLE with the same starting directory.
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Why didn't my backup work (the way I thought it should)?
On Wednesday 13 September 2006 14:47, Zembower, Kevin wrote: >Frank and Alex, thanks for replying. Alex, you're right, I believe, >GNUtar behaves differently than tar with respect to symbolic links, and >I'm using GNUtar: >[EMAIL PROTECTED]:/var/lib$ tar --version >tar (GNU tar) 1.14 > 1.14 is busted. Use 1.13-19, 1.13-25, or 1.15-1. >Frank, all my entries look like this one for sda2 (root): >[EMAIL PROTECTED]:/var/lib$ amadmin DBackup disklist centernet >line 48: >host centernet: >interface default >disk sda2: >program "GNUTAR" >priority 1 >dumpcycle 28 >maxdumps 1 >maxpromoteday 1 >strategy STANDARD >compress CLIENT FAST >comprate 0.50 0.50 >auth BSD >kencrypt NO >holdingdisk YES >record YES >index YES >skip-incr NO >skip-full NO > >>From this, it seems there's no exclude lists in the disklist. It also >seems that there isn't one on the host, which is both client and server >(tapehost, amandahost): >[EMAIL PROTECTED]:/var/lib$ locate exclude >/home/kevinz/perl-5.8.6/perl-5.8.6/plan9/exclude >[EMAIL PROTECTED]:/var/lib$ > >None were NSF filesystems or links to other filesystems. > >Your suggestion that the last tapes might be corrupt is an excellent >one, and one that I'll begin exploring immediately. Your suggestion to >check the restore procedures more than once every six years is also >good, and one that I'll also put into practice just as soon as this fire >is out. > >You also caused me to question if I performed the restore correctly. I >manually went through the first record on each tape file and copied the >filesystem and level. It's possible that I omitted a level 0 and only >restored a level 1. Just about all the files missing are very static >ones that change infrequently. I'll have to check into this, too. > >Thanks, again. > >-Kevin > > >-Original Message- >From: Frank Smith [mailto:[EMAIL PROTECTED] >Sent: Wednesday, September 13, 2006 1:20 PM >To: Zembower, Kevin >Cc: amanda-users@amanda.org >Subject: Re: Why didn't my backup work (the way I thought it should)? > >Did you have an exclude list? >Were any of those directories symlinks to other filesystems or NFS >mounts? A symlink would have still been backed up, just not what it >was pointing to, and a mount point would just show up as an empty >directory. >Did the system have problems before the last backup? Perhaps these >files were corrupted/gone before/during the last backup. Since >Amanda restores the state at a point in time, you may have most of >the data on a previous level that the latest level removed (related >to recommendations often given here to restore to a tmp directory >and move files into place since otherwise existing new data in the >target directory would be deleted by the restore). You might want >to try recovering from tapes from a day or two before and see if >the data is there, the dpkg data at least probably hadn't changed >much from earlier backups. > >Test restores more than once per 6 years to verify you are backing up >what you think you are, since data seems to get shifted around and >the excludes you need today eventually end up excluding things you need. >Also, backup media (tape and disk) can fail in ways that give no errors >during backups but make a restore impossible, and the sooner you are >aware of it the better. > >Frank > >> Thanks for your advice and suggestions. >> >> -Kevin >> >> Kevin Zembower >> Internet Services Group manager >> Center for Communication Programs >> Bloomberg School of Public Health >> Johns Hopkins University >> 111 Market Place, Suite 310 >> Baltimore, Maryland 21202 >> 410-659-6139 -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: Why didn't my backup work (the way I thought it should)?
On Wednesday 13 September 2006 15:02, Jon LaBadie wrote: >On Wed, Sep 13, 2006 at 02:47:26PM -0400, Zembower, Kevin wrote: >> Frank and Alex, thanks for replying. Alex, you're right, I believe, >> GNUtar behaves differently than tar with respect to symbolic links, and >> I'm using GNUtar: >> [EMAIL PROTECTED]:/var/lib$ tar --version >> tar (GNU tar) 1.14 > >Likely unrelated to your current difficulties, >but plain 1.14 gnutar has be reported to have >difficulties with amanda. I haven't followed >the discussion closely, but I think 1.14.1 and >1.14.2 are both considered ok. A complicating >factor is the report that some linux vendors, >I think RH was mentioned, supply a 1.14 with >the later bug fix patches applied. These work >ok also but because they are based on the 1.14 >code, that is the version reported. The 1.14-* rpms supplied for FC2 did not work for me, Jon. I backed up to 1.13-25 since the rpm was still on-site, and eventually built 1.15-1 from the tarball, which also works fine. I used 1.13-19 for quite a while early on. -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: Fixed amcheck, one more question (or two)
On Wednesday 13 September 2006 14:54, [EMAIL PROTECTED] wrote: >Sorry for all the whining...it turned out the "host down" was the same >reverse DNS problem that the other amanda client was returning, but since >the error message was totally different, it baffled. I hooked the server >back up to the old network and it worked fine. > >2 more minor issues: > >I'm getting this: > >ERROR: label FULL0 match labelstr but it not listed in the tapelist file. > (expecting a new tape) > >I thought that amanda would simply create a new tapelist...I deleted the >old one because it was years old. Do I need to manually create a new > one? If so, which dates do I use...I simply created an empty file > figuring amanda would take care of the rest. tapelist is created by amlabel. It is edited by amdump to shuffle the order as the tapes are used, and may be edited by amcheck as it finds the next tape to use. If you deleted it, then I would have dd dump the first 32k of every labeled tape you have, and add that tapes data in turn to the tapelist with an editor. A somewhat lengthy process if you have many tapes. You want to wind up with something that resembles this: 20060913001001 Dailys-19 reuse 2147483647 Dailys-17 reuse 2147483647 Dailys-16 reuse 20060912001001 Dailys-18 reuse 20060911001001 Dailys-15 reuse 20060910001001 Dailys-14 reuse 20060909001001 Dailys-13 reuse 20060908001001 Dailys-12 reuse 20060907001002 Dailys-11 reuse 20060906001002 Dailys-10 reuse 20060905001001 Dailys-9 reuse 20060904001001 Dailys-8 reuse 20060903001001 Dailys-7 reuse 20060902001502 Dailys-6 reuse 20060901001501 Dailys-5 reuse 20060831001502 Dailys-4 reuse 20060830001502 Dailys-3 reuse 20060829001500 Dailys-2 reuse 20060828001501 Dailys-1 reuse 20060827 Dailys-21 reuse 20060827 Dailys-20 reuse Obviously I have a few left yet that do not have the new timestamps. >Lastly (I hope), is there a way to get amanda to not do host lookups and >simply use IP addresses in .amandahosts? For security, I think IP >addresses would be better. Sure, but I believe they'll need to be compiled in by putting them in your configure driver script. But while I did that in earlier years, I do all my local resolving now with host files, so I've been using FQDN's for quite a while now. As to the differences in security, not much in real life IMO since all my addresses are local 192.168.x.x addresses anyway. >Thanks again! > >James SmallacombePlantageNet, Inc. CEO and Janitor >[EMAIL PROTECTED] http://3.am >= -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: Fixed amcheck, one more question (or two)
On Wed, Sep 13, 2006 at 02:54:13PM -0400, [EMAIL PROTECTED] wrote: > > ERROR: label FULL0 match labelstr but it not listed in the tapelist file. >(expecting a new tape) > > I thought that amanda would simply create a new tapelist...I deleted the > old one because it was years old. Do I need to manually create a new one? > If so, which dates do I use...I simply created an empty file figuring > amanda would take care of the rest. > amlabel creates the tapelist, not amdump and friends. If you want to recreate it by hand do entries of the form: 0 vtape64 reuse 0 vtape63 reuse Reverse order, the first one you want used at the bottom, last at the top. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Searching for a particular file....
Using amanda 2.4.5 with gnu-tar.This has probably been already answered somewhere but I can't find an answer in the list or the wikiIs there a way to list the content of an amanda tape (without having to load and read the tape) ? Someone has lost a file, and he does not know on which host and on which disk it was. Is there a way to list the content of each tape (index?) and grep for some file pattern ?ex: find all files ending with ".dmp" and list the tape name and the DLE where I could find it so I can then use amrecover and know where to begin. Thanks
Re: Amanda error "Unexpected field value..."
I have run into the exact same problem on a client that is running Debian (Etch) linux using tar version 1.15.91 (and amanda 2.5.0p2-1). I have now noticed, given your message, that things worked fine with the first Debian tar 1.15.91 package but, when a revised tar 1.15.91 package was released end of July, 2 of the 3 partitions on the client began failing during the size estimation stage with the same tar error message, "Unexpected field value". I'm going to force a full level backup of all partitions using the new tar release yet, if this or the following level 1 fails (given what Olivier said), I will try downgrading to the previous package similarly. Paul ->>In response to your message<<- --received from McGraw, Robert P.-- > > It seems that tar 1.15.91 does not work with my version of amanda. I have > dropped back to an older version of tar until I can upgrade my amanda. > > Thanks > > Robert > > > _ > Robert P. McGraw, Jr. > Manager, Computer System EMAIL: [EMAIL PROTECTED] > Purdue University ROOM: MATH-807 > Department of MathematicsPHONE: (765) 494-6055 > 150 N. University Street FAX: (419) 821-0540 > West Lafayette, IN 47907-2067 > > > > -Original Message- > > From: Olivier Nicole [mailto:[EMAIL PROTECTED] > > Sent: Friday, September 08, 2006 10:37 AM > > To: McGraw, Robert P. > > Cc: amanda-users@amanda.org > > Subject: Re: Amanda error "Unexpected field value..." > > > > > > > The difference is that the good run was a level 0 and the run with the > > error > > > was a level 1. > > > What is the "Unexpected field value"? > > > > Well that's the problem with cygwin... We have little reference. > > > > Apparently running level 0 gtar creates a sort of index file, to use > > in level 1 and compare files that have changed. > > > > And it seems that in between the run 0 and 1 gtar cannot read the file > > it created. > > > > If you look at the sendzie debug you should see a command very close > > to this one > > > > gtar --create --file - --directory /local > > --one-file-system --listed-incremental /local/Amanda/amanda-2.4 > > .5p1/var/amanda/gnutar-lists/coriolis_local_1.new --sparse > > --ignore-failed-read --totals . > > > > but that will say "gtar --create --file /dev/null ..." > > > > Did that one completed without error? > > > > Try to run the long gtar command by hand, with --file - and --file > > /dev/null > > > > Tryto extract the comamnd for level 0 and run it... > > > > Check your version of gtar. > > > > Bests, > > > > Olivier -- Paul Yeatman (858) 534-9896[EMAIL PROTECTED] == ==Proudly brought to you by Mutt== ==
Re: Searching for a particular file....
On Wed, Sep 13, 2006 at 04:25:56PM -0400, Guy Dallaire wrote: > Using amanda 2.4.5 with gnu-tar. > > This has probably been already answered somewhere but I can't find an answer > in the list or the wiki > > Is there a way to list the content of an amanda tape (without having to load > and read the tape) ? > > Someone has lost a file, and he does not know on which host and on which > disk it was. > > Is there a way to list the content of each tape (index?) and grep for some > file pattern ? > > ex: find all files ending with ".dmp" and list the tape name and the DLE > where I could find it so I can then use amrecover and know where to begin. > cd $(amgetconf indexdir) you'll find a set of hostname dirs, under that, DLEname dirs In the DLEname dirs are the indexes by date. The indexes have one filename per line. They are gzipped, so if you have the "gzgrep" command they are easy to look at. Just a little more work otherwise. So after the above cd, you could do a gzgrep '\.dmp$' */*/* And sit back. Of course, if you have a lot of hosts/DLEs/saved dumps, you might exceed the maximum command line length. So it might take a few greps :) -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Searching for a particular file....
Thanks !2006/9/13, Jon LaBadie <[EMAIL PROTECTED]>: On Wed, Sep 13, 2006 at 04:25:56PM -0400, Guy Dallaire wrote:> Using amanda 2.4.5 with gnu-tar.>> This has probably been already answered somewhere but I can't find an answer> in the list or the wiki >> Is there a way to list the content of an amanda tape (without having to load> and read the tape) ?>> Someone has lost a file, and he does not know on which host and on which> disk it was. >> Is there a way to list the content of each tape (index?) and grep for some> file pattern ?>> ex: find all files ending with ".dmp" and list the tape name and the DLE> where I could find it so I can then use amrecover and know where to begin. >cd $(amgetconf indexdir)you'll find a set of hostname dirs,under that, DLEname dirsIn the DLEname dirs are the indexes by date.The indexes have one filename per line. They are gzipped, so if you have the "gzgrep"command they are easy to look at. Just a littlemore work otherwise.So after the above cd, you could do agzgrep '\.dmp$' */*/* And sit back. Of course, if you have a lot ofhosts/DLEs/saved dumps, you might exceed themaximum command line length. So it might takea few greps :)--Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
