Hi all,
In order of what I am hoping to accomplish:
1) get amanda to back up the troubled client again
2) kill the wedged dump processes
The nitty gritty:
I have approximately 15 solaris clients running amanda. I recently set
up a Linux box as an amanda client. It ran correctly for the first
two runs, doing a level 0 first, and then a level 1 (as expected).
I use a separate FullSet tapeset to capture all fullsets on the
weekend, and that failed:
tetris /dev/md2 lev 0 FAILED [Request to tetris timed out.]
When I resumed the regular DailySet on Monday, I got:
tetris /dev/md2 lev 0 FAILED [tetris NAK: amandad busy]
There appeared to be amanda processes lurking from the FullSet
failure, and I manually killed them. Since then, amanda has failed on
every run with the "amandad busy" message.
Twice, "dump" was started, and the amanda processes were left running
on the machine. "dump" is in an unkillable state:
14280 amanda 9 0 616 616 488 D 0.0 0.0 0:00 dump
12581 amanda 9 0 616 616 488 D 0.0 0.0 0:00 dump
Hints on how I might kill these without rebooting would be appreciated.
tetris is running
kernel: 2.4.3-12smp
glibc: glibc-2.2.2-10
tar: tar-1.13.19-4
(Having seen some notes in the archives about potential tar core dumps
with the Kernel/glibc combo I have, I verified the version of tar.)
Here are my most recent *.debug out of /tmp/amanda on the machine:
sendsize: debug 1 pid 12579 ruid 650 euid 650 start time Mon Oct 15 23:59:14 2001
/usr/local/amanda-2.4.2p2/libexec/sendsize: version 2.4.2p2
calculating for amname '/dev/md2', dirname '/home'
sendsize: getting size via dump for /dev/md2 level 0
sendsize: running "/sbin/dump 0Ssf 1048576 - /dev/md2"
running /usr/local/amanda-2.4.2p2/libexec/killpgrp
killpgrp: debug 1 pid 12580 ruid 650 euid 0 start time Mon Oct 15 23:59:14 2001
/usr/local/amanda-2.4.2p2/libexec/killpgrp: version 2.4.2p2
sending SIGTERM to process group 12580
it won't die with SIGTERM, but SIGKILL should do
do't expect any further output, this will be suicide
amandad: debug 1 pid 12578 ruid 650 euid 650 start time Mon Oct 15 23:59:14 2001
amandad: version 2.4.2p2
amandad: build: VERSION="Amanda-2.4.2p2"
amandad: BUILT_DATE="Tue Oct 9 13:59:33 CDT 2001"
amandad: BUILT_MACH="Linux tetris.software.umn.edu 2.4.3-12smp #1 SMP Fri Jun 8
14:38:50 EDT 2001 i686 unknown"
amandad: CC="gcc"
amandad: paths: bindir="/usr/local/amanda-2.4.2p2/bin"
amandad: sbindir="/usr/local/amanda-2.4.2p2/sbin"
amandad: libexecdir="/usr/local/amanda-2.4.2p2/libexec"
amandad: mandir="/usr/local/amanda-2.4.2p2/man"
amandad: AMANDA_TMPDIR="/tmp/amanda" AMANDA_DBGDIR="/tmp/amanda"
amandad: CONFIG_DIR="/usr/local/amanda-2.4.2p2/etc/amanda"
amandad: DEV_PREFIX="/dev/" RDEV_PREFIX="/dev/" DUMP="/sbin/dump"
amandad: RESTORE="/sbin/restore" GNUTAR="/bin/gtar"
amandad: COMPRESS_PATH="/bin/gzip" UNCOMPRESS_PATH="/bin/gzip"
amandad: MAILER="/usr/bin/Mail"
amandad: listed_incr_dir="/usr/local/amanda-2.4.2p2/var/amanda/gnutar-lists"
amandad: defs: DEFAULT_SERVER="moby" DEFAULT_CONFIG="DailySet1"
amandad: DEFAULT_TAPE_SERVER="moby"
amandad: DEFAULT_TAPE_DEVICE="/dev/nrst28" HAVE_MMAP HAVE_SYSVSHM
amandad: LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE
amandad: AMANDA_DEBUG_DAYS=4 BSD_SECURITY USE_AMANDAHOSTS
amandad: CLIENT_LOGIN="amanda" FORCE_USERID HAVE_GZIP
amandad: COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
amandad: COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"
got packet:
- --------
Amanda 2.4 REQ HANDLE 00D-0008D618 SEQ 1003208113
SECURITY USER amanda
SERVICE sendsize
OPTIONS maxdumps=1;hostname=tetris;
DUMP /dev/md2 0 1970:1:1:0:0:0 -1
DUMP /dev/md2 1 2001:10:10:3:36:15 -1
DUMP /dev/md2 2 2001:10:12:3:54:24 -1
- --------
sending ack:
- ----
Amanda 2.4 ACK HANDLE 00D-0008D618 SEQ 1003208113
- ----
bsd security: remote host moby.jaws.umn.edu user amanda local user amanda
amandahosts security check passed
amandad: running service "/usr/local/amanda-2.4.2p2/libexec/sendsize"
amandad: got packet:
- ----
Amanda 2.4 REQ HANDLE 00D-00090DD0 SEQ 1003287366
SECURITY USER root
SERVICE selfcheck
OPTIONS ;
DUMP /dev/md2 0 OPTIONS |;bsd-auth;compress-fast;index;
- ----
amandad: received other packet, NAKing it
addr: peer 134.84.132.41 dup 134.84.132.41, port: peer 936 dup 782
sending nack:
- ----
Amanda 2.4 NAK HANDLE 00D-00090DD0 SEQ 1003287366
ERROR amandad busy
- ----
amandad: got packet:
- ----
Amanda 2.4 REQ HANDLE 00D-0008BDA8 SEQ 1003287913
SECURITY USER amanda
SERVICE sendsize
OPTIONS maxdumps=1;hostname=tetris;
DUMP /dev/md2 0 1970:1:1:0:0:0 -1
DUMP /dev/md2 1 2001:10:10:3:36:15 -1
DUMP /dev/md2 2 2001:10:12:3:54:24 -1
- ----
amandad: received other packet, NAKing it
addr: peer 134.84.132.41 dup 134.84.132.41, port: peer 936 dup 788
sending nack:
- ----
Amanda 2.4 NAK HANDLE 00D-0008BDA8 SEQ 1003287913
ERROR amandad busy
- ----
amandad: got packet:
- ----
Amanda 2.4 REQ HANDLE 00D-0008BDA8 SEQ 1003374014
SECURITY USER amanda
SERVICE sendsize
OPTIONS maxdumps=1;hostname=tetris;
DUMP /dev/md2 0 1970:1:1:0:0:0 -1
DUMP /dev/md2 1 2001:10:10:3:36:15 -1
DUMP /dev/md2 2 2001:10:12:3:54:24 -1
- ----
amandad: received other packet, NAKing it
addr: peer 134.84.132.41 dup 134.84.132.41, port: peer 936 dup 688
sending nack:
- ----
Amanda 2.4 NAK HANDLE 00D-0008BDA8 SEQ 1003374014
ERROR amandad busy
- ----
amandad: got packet:
- ----
Amanda 2.4 REQ HANDLE 00D-0008D290 SEQ 1003456814
SECURITY USER amanda
SERVICE sendsize
OPTIONS maxdumps=1;hostname=tetris;
DUMP /dev/md2 0 1970:1:1:0:0:0 -1
DUMP /dev/md2 1 2001:10:10:3:36:15 -1
DUMP /dev/md2 2 2001:10:12:3:54:24 -1
- ----
amandad: received other packet, NAKing it
addr: peer 134.84.132.41 dup 134.84.132.41, port: peer 936 dup 697
sending nack:
- ----
Amanda 2.4 NAK HANDLE 00D-0008D290 SEQ 1003456814
ERROR amandad busy
- ----
amandad: got packet:
- ----
Amanda 2.4 REQ HANDLE 00D-0008D3A0 SEQ 1003543213
SECURITY USER amanda
SERVICE sendsize
OPTIONS maxdumps=1;hostname=tetris;
DUMP /dev/md2 0 1970:1:1:0:0:0 -1
DUMP /dev/md2 1 2001:10:10:3:36:15 -1
DUMP /dev/md2 2 2001:10:12:3:54:24 -1
- ----
amandad: received other packet, NAKing it
addr: peer 134.84.132.41 dup 134.84.132.41, port: peer 936 dup 611
sending nack:
- ----
Amanda 2.4 NAK HANDLE 00D-0008D3A0 SEQ 1003543213
ERROR amandad busy
- ----
amandad: got packet:
- ----
Amanda 2.4 REQ HANDLE 008-00089348 SEQ 1003590009
SECURITY USER amanda
SERVICE sendsize
OPTIONS maxdumps=1;hostname=tetris;
DUMP /dev/md2 0 1970:1:1:0:0:0 -1
- ----
amandad: received other packet, NAKing it
addr: peer 134.84.132.41 dup 134.84.132.41, port: peer 936 dup 1020
sending nack:
- ----
Amanda 2.4 NAK HANDLE 008-00089348 SEQ 1003590009
ERROR amandad busy
- ----
Thanks for any assistance.
Lynette Bellini
Systems Administrator
University of Minnesota
