I recently added a new machine to my backups, and seem to be having a
problem with a couple of file systems with "index tee cannot write [Broken
pipe]" errors.
Client is Solaris 7
Server is Solaris 7
amanda version 2.4.2p2
gnutar 1.13.19
firewall between the two servers, but no security restrictions (as in all
other backups work fine). Server is NAT'ed at firewall
I've run some tests just backing one of those filesystems up, and no matter
how I fiddle with the dumptype (thinking compression in particular might be
part of the problem) I cannot get this error to go away. In test mode I'm
trying to dump only to holding disk which has 11.5 GB free. The file system
in question is only ~1.5 GBs. Larger filesystems on the same host can be
backed up with the same dumptype parameters with no problems.
When I was watching the backup being performed with "amstatus" it showed
data being transferred, and the correct file was being created and updated
on the server. I think when it stopped updating, the size sent reported by
amstatus was a few KB (32?) over the estimated backup size (didnt write the
data down like a fool) - I realize this may or may not mean a thing, but
just throwing it out there.
Here are some (hopefully!) appropriate log files (they dont tell me too
much):
=====================
log.20020225.0 from server
=====================
START planner date 20020225
START driver date 20020225
ERROR taper no-tape [no tape online]
FINISH planner date 20020225
STATS driver startup time 2.472
FAIL dumper rnbuyer /export/home 0 [mesg read: Connection timed out]
sendbackup: start [rnbuyer:/export/home level 0]
sendbackup: info BACKUP=/usr/local/bin/tar
sendbackup: info RECOVER_CMD=/usr/local/bin/tar -f... -
sendbackup: info end
FINISH driver date 20020225 time 7744.120
=====================
amdump.1 from server
=====================
amdump: start at Mon Feb 25 10:38:00 EST 2002
planner: pid 5123 executable /usr/local/libexec/planner version 2.4.2p2
planner: build: VERSION="Amanda-2.4.2p2"
planner: BUILT_DATE="Mon Jul 30 12:22:45 EDT 2001"
planner: BUILT_MACH="SunOS utl-atl-05 5.7 Generic_106541-15 sun4u
sparc SUNW,Ultra-4"
planner: CC="gcc"
planner: paths: bindir="/usr/local/bin" sbindir="/usr/local/sbin"
planner: libexecdir="/usr/local/libexec" mandir="/usr/local/man"
planner: AMANDA_TMPDIR="/tmp/amanda" AMANDA_DBGDIR="/tmp/amanda"
planner: CONFIG_DIR="/usr/local/etc/amanda" DEV_PREFIX="/dev/dsk/"
planner: RDEV_PREFIX="/dev/rdsk/" DUMP="/usr/sbin/ufsdump"
planner: RESTORE="/usr/sbin/ufsrestore"
planner: SAMBA_CLIENT="/usr/local/samba/bin/smbclient"
planner: GNUTAR="/usr/local/bin/tar"
planner: COMPRESS_PATH="/usr/local/bin/gzip"
planner: UNCOMPRESS_PATH="/usr/local/bin/gzip"
planner: MAILER="/usr/bin/mailx"
planner: listed_incr_dir="/usr/local/var/amanda/gnutar-lists"
planner: defs: DEFAULT_SERVER="utl-atl-06" DEFAULT_CONFIG="DailySet1"
planner: DEFAULT_TAPE_SERVER="utl-atl-06"
planner: DEFAULT_TAPE_DEVICE="/dev/null" HAVE_MMAP HAVE_SYSVSHM
planner: LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE
planner: AMANDA_DEBUG_DAYS=4 BSD_SECURITY USE_AMANDAHOSTS
planner: CLIENT_LOGIN="siteops" FORCE_USERID HAVE_GZIP
planner: COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
planner: COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"
planner: dgram_bind: socket bound to 0.0.0.0.603
READING CONF FILES...
startup took 0.010 secs
SETTING UP FOR ESTIMATES...
setting up estimates for rnbuyer:/export/home
rnbuyer:/export/home overdue 11744 days for level 0
setup_estimate: rnbuyer:/export/home: command 4, options:
last_level -1 next_level0 -11744 level_days 0
getting estimates 0 (0) -1 (-1) -1 (-1)
setting up estimates took 0.000 secs
GETTING ESTIMATES...
driver: pid 5122 executable /usr/local/libexec/driver version 2.4.2p2
driver: send-cmd time 0.005 to taper: START-TAPER 20020225
taper: pid 5124 executable taper version 2.4.2p2
driver: started dumper0 pid 5126
dumper: dgram_bind: socket bound to 0.0.0.0.606
dumper: pid 5126 executable dumper version 2.4.2p2, using port 606
dumper: dgram_bind: socket bound to 0.0.0.0.607
dumper: pid 5127 executable dumper version 2.4.2p2, using port 607
driver: started dumper1 pid 5127
driver: started dumper2 pid 5128
driver: started dumper3 pid 5129
driver: started dumper4 pid 5130
driver: started dumper5 pid 5131
dumper: dgram_bind: socket bound to 0.0.0.0.608
dumper: pid 5128 executable dumper version 2.4.2p2, using port 608
dumper: dgram_bind: socket bound to 0.0.0.0.610
dumper: pid 5130 executable dumper version 2.4.2p2, using port 610
dumper: dgram_bind: socket bound to 0.0.0.0.612
dumper: pid 5132 executable dumper version 2.4.2p2, using port 612
dumper: dgram_bind: socket bound to 0.0.0.0.611
dumper: pid 5131 executable dumper version 2.4.2p2, using port 611
driver: started dumper6 pid 5132
dumper: dgram_bind: socket bound to 0.0.0.0.609
dumper: pid 5129 executable dumper version 2.4.2p2, using port 609
dumper: dgram_bind: socket bound to 0.0.0.0.613
dumper: pid 5133 executable dumper version 2.4.2p2, using port 613
driver: started dumper7 pid 5133
got result for host rnbuyer disk /export/home: 0 -> 1546080K, -1 -> -1K, -1
-> -1K
getting estimates took 2.524 secs
FAILED QUEUE: empty
DONE QUEUE:
0: rnbuyer /export/home
ANALYZING ESTIMATES...
pondering rnbuyer:/export/home... next_level0 -11744 last_level -1 (due for
level 0) (new disk, can't switch to degraded mode)
curr level 0 size 1546080 total size 1552176 total_lev0 1546080
balanced-lev0size 220868
INITIAL SCHEDULE (size 1552176):
rnbuyer /export/home pri 11745 lev 0 size 1546080
DELAYING DUMPS IF NEEDED, total_size 1552176, tape length 51200000 mark 2000
delay: Total size now 1552176.
PROMOTING DUMPS IF NEEDED, total_lev0 1546080, balanced_size 220868...
analysis took 0.000 secs
GENERATING SCHEDULE:
--------
rnbuyer /export/home 11745 0 1970:1:1:0:0:0 1546080 51536
--------
driver: adding holding disk 0 dir /backup/hd3/test size 11837581
reserving 5918790 out of 11837581 for degraded-mode dumps
driver: start time 2.472 inparallel 8 bandwidth 89000 diskspace 11837581 dir
OBSOLETE datestamp 20020225 driver: drain-ends
tapeq LFFO big-dumpers 5
driver: result time 2.472 from taper: TAPE-ERROR [no tape online]
dump of driver schedule before start degraded mode:
--------
rnbuyer /export/home lv 0 t 51536 s 1546112 p 11745
--------
dump of driver schedule after start degraded mode:
--------
rnbuyer /export/home lv 0 t 51536 s 1546112 p 11745
--------
driver: send-cmd time 2.473 to dumper0: FILE-DUMP 00-00001
/backup/hd3/test/20020225/rnbuyer._export_home.0 rnbuyer /export/
home 0 1970:1:1:0:0:0 1048576 GNUTAR 1546176
|;bsd-auth;index;exclude-list=/usr/local/lib/amanda/exclude.gtar;
driver: state time 2.473 free kps: 88970 space: 10291405 taper: DOWN
idle-dumpers: 7 qlen tapeq: 0 runq: 0 roomq: 0 wakeup:
86400 driver-idle: not-idle
driver: interface-state time 2.473 if : free 7970 if QFE0: free 1000 if
HME0: free 40000 if LOCAL: free 40000
driver: hdisk-state time 2.473 hdisk 0: free 10291405 dumpers 1
dumper: stream_client: connected to 172.16.0.51.39486
dumper: stream_client: our side is 0.0.0.0.49950
dumper: stream_client: connected to 172.16.0.51.39487
dumper: stream_client: our side is 0.0.0.0.49951
dumper: stream_client: connected to 172.16.0.51.39488
dumper: stream_client: our side is 0.0.0.0.49952
dumper: kill index command
driver: result time 7728.230 from dumper0: FAILED 00-00001 [mesg read:
Connection timed out]
driver: state time 7744.109 free kps: 89000 space: 11837581 taper: DOWN
idle-dumpers: 8 qlen tapeq: 0 runq: 0 roomq: 0 wakeu
p: 86400 driver-idle: not-idle
driver: interface-state time 7744.109 if : free 8000 if QFE0: free 1000 if
HME0: free 40000 if LOCAL: free 40000
driver: hdisk-state time 7744.109 hdisk 0: free 11837581 dumpers 0
driver: QUITTING time 7744.109 telling children to quit
driver: send-cmd time 7744.109 to dumper0: QUIT
driver: send-cmd time 7744.109 to dumper1: QUIT
driver: send-cmd time 7744.109 to dumper2: QUIT
driver: send-cmd time 7744.110 to dumper3: QUIT
driver: send-cmd time 7744.111 to dumper4: QUIT
driver: send-cmd time 7744.113 to dumper5: QUIT
driver: send-cmd time 7744.115 to dumper6: QUIT
driver: send-cmd time 7744.115 to dumper7: QUIT
driver: send-cmd time 7744.116 to taper: QUIT
taper: DONE [idle wait: 7744.090 secs]
driver: FINISHED time 7744.120
amdump: end at Mon Feb 25 12:47:04 EST 2002
=====================
sendbackup.20020225103738.debug from client side
=====================
sendbackup: debug 1 pid 24045 ruid 505 euid 505 start time Mon Feb 25
10:37:38 2002
/usr/local/libexec/sendbackup: version 2.4.2p2
sendbackup: got input request: GNUTAR /export/home 0 1970:1:1:0:0:0 OPTIONS
|;bsd-auth;index;exclude-list=/usr/local/lib/ama
nda/exclude.gtar;
parsed request as: program `GNUTAR'
disk `/export/home'
lev 0
since 1970:1:1:0:0:0
opt
`|;bsd-auth;index;exclude-list=/usr/local/lib/amanda/exclude.gtar;'
sendbackup: exclude list file "/usr/local/lib/amanda/exclude.gtar" does not
exist, ignoring
sendbackup: try_socksize: send buffer size is 65536
sendbackup: stream_server: waiting for connection: 0.0.0.0.39486
sendbackup: stream_server: waiting for connection: 0.0.0.0.39487
sendbackup: stream_server: waiting for connection: 0.0.0.0.39488
waiting for connect on 39486, then 39487, then 39488
sendbackup: stream_accept: connection from 63.112.211.70.49950
sendbackup: stream_accept: connection from 63.112.211.70.49951
sendbackup: stream_accept: connection from 63.112.211.70.49952
got all connections
sendbackup-gnutar: doing level 0 dump as listed-incremental to
/usr/local/var/amanda/gnutar-lists/rnbuyer_export_home_0.new
sendbackup-gnutar: doing level 0 dump from date: 1970-01-01 0:00:00 GMT
sendbackup: spawning /usr/local/libexec/runtar in pipeline
sendbackup: argument list: gtar --create --file - --directory /export/home
--one-file-system --listed-incremental /usr/local
/var/amanda/gnutar-lists/rnbuyer_export_home_0.new --sparse
--ignore-failed-read --totals .
sendbackup-gnutar: /usr/local/libexec/runtar: pid 24047
sendbackup: started index creator: "/usr/local/bin/tar -tf - 2>/dev/null |
sed -e 's/^\.//'"
index tee cannot write [Broken pipe]
sendbackup: pid 24046 finish time Mon Feb 25 12:46:23 2002
Thanks for any help!
Matt
+-------------------------
Matthew Galer
Senior Systems Engineer
[EMAIL PROTECTED]
770-453-9001 x127
