Re: seemingly random pattern of dump failures

[Chris Hoogendyk]

So,

On the client, in /tmp/amanda/amandad/, I see the debug files for the estimates and for each of the 
DLEs except for the one that failed. No entry for it from 5/8 or 5/9 (after midnight). Did a more on 
`ls *0508* *0509* | sort` and scanned through that with find.


On the server, in /tmp/amanda/server/daily/, I also don't see any references to the DLE  "/data"  in 
any of the dumper debug files from 5/8 or 5/9. I do see some sections where the "timeout waiting for 
ACK" appear, but I don't know what to make of them. They don't seem to offer me much in the way of 
diagnostics or the assurance that they are talking about the "/data" DLE.


Here is one of those pieces with the successful items before and after:

dumper: Building type 4 (FILE) header of size 32768 using:
dumper: Contents of *(dumpfile_t *)28894:
dumper: type = 4 (FILE)
dumper: datestamp= '20130508224500'
dumper: dumplevel= 1
dumper: compressed   = 1
dumper: encrypted= 0
dumper: comp_suffix  = '.gz'
dumper: encrypt_suffix   = 'N'
dumper: name = 'metzi1.physics.umass.edu'
dumper: disk = '/usr/local'
dumper: program  = '/bin/tar'
dumper: srvcompprog  = ''
dumper: clntcompprog = ''
dumper: srv_encrypt  = ''
dumper: clnt_encrypt = ''
dumper: recover_cmd  = '/bin/gzip -dc |/bin/tar -f - ...'
dumper: uncompress_cmd   = ''
dumper: encrypt_cmd  = ''
dumper: decrypt_cmd  = ''
dumper: srv_decrypt_opt  = ''
dumper: clnt_decrypt_opt = ''
dumper: cont_filename= ''
dumper: is_partial   = 0
dumper: partnum  = 0
dumper: totalparts   = 0
dumper: blocksize= 32768
security_stream_seterr(57cd0, EOF)
security_stream_close(57cd0)
security_stream_seterr(47c50, EOF)
security_stream_close(47c50)
security_stream_seterr(4fc90, EOF)
security_stream_close(4fc90)
dumper: connect_port: Try  port 1025: Available   -
dumper: connected to 127.0.0.1.36800
dumper: our side is 0.0.0.0.1025
dumper: try_socksize: send buffer size is 65536
security_getdriver(name=ssh) returns ff316638
security_handleinit(handle=335a0, driver=ff316638 (SSH))
security_streaminit(stream=3fc10, driver=ff316638 (SSH))
security_seterror(handle=335a0, driver=ff316638 (SSH) error=timeout waiting for 
ACK)
security_close(handle=335a0, driver=ff316638 (SSH))
security_stream_close(3fc10)
dumper: connect_port: Try  port 1025: Available   -
dumper: connected to 127.0.0.1.36806
dumper: our side is 0.0.0.0.1025
dumper: try_socksize: send buffer size is 65536
security_getdriver(name=ssh) returns ff316638
security_handleinit(handle=335a0, driver=ff316638 (SSH))
security_streaminit(stream=3fc10, driver=ff316638 (SSH))
security_seterror(handle=335a0, driver=ff316638 (SSH) error=timeout waiting for 
ACK)
security_close(handle=335a0, driver=ff316638 (SSH))
security_stream_close(3fc10)
dumper: connect_port: Try  port 1025: Available   -
dumper: connected to 127.0.0.1.36810
dumper: our side is 0.0.0.0.1025
dumper: try_socksize: send buffer size is 65536
security_getdriver(name=ssh) returns ff316638
security_handleinit(handle=33398, driver=ff316638 (SSH))
security_streaminit(stream=3fc10, driver=ff316638 (SSH))
security_streaminit(stream=47c50, driver=ff316638 (SSH))
security_streaminit(stream=4fc90, driver=ff316638 (SSH))
security_streaminit(stream=57cd0, driver=ff316638 (SSH))
security_close(handle=33398, driver=ff316638 (SSH))
security_stream_close(3fc10)
dumper: Building type 4 (FILE) header of size 32768 using:
dumper: Contents of *(dumpfile_t *)28894:
dumper: type = 4 (FILE)
dumper: datestamp= '20130508224500'
dumper: dumplevel= 1
dumper: compressed   = 1
dumper: encrypted= 0
dumper: comp_suffix  = '.gz'
dumper: encrypt_suffix   = 'N'
dumper: name = 'rassilon.crc.isb.nsm'
dumper: disk = '/var/radmind'
dumper: program  = '/bin/tar'
dumper: srvcompprog  = ''
dumper: clntcompprog = ''
dumper: srv_encrypt  = ''
dumper: clnt_encrypt = ''
dumper: recover_cmd  = '/bin/gzip -dc |/bin/tar -f - ...'
dumper: uncompress_cmd   = ''
dumper: encrypt_cmd  = ''
dumper: decrypt_cmd  = ''
dumper: srv_decrypt_opt  = ''
dumper: clnt_decrypt_opt = ''
dumper: cont_filename= ''
dumper: is_partial   = 0
dumper: partnum  = 0
dumper: totalparts   = 0
dumper: blocksize= 32768




On 5/9/13 3:24 PM, Jean-Louis Martineau wrote:

Chris,

look in the amandad debug files on the client and the dumper debug file on the 
server.

Jean-Louis

On 05/09/2013 03:06 PM, Chris Hoogendyk wrote:
On one of my older setups that is running Amanda 2.5.1p3, I'm getting patterns of failures that I 
can't make sense of. Some days everything works just fine. Other days I get everything from a 
couple of DLE

Re: seemingly random pattern of dump failures

[Debra S Baddorf]

That's odd -- I'm running v3.3.3 and I just started having the same sort of 
failures.  There is no amandad  debug file.
There is one odd line in the  server/client/chunker  debug file:

 chunker: putresult: 10 FAILED

and a few  0 length files sitting around on my spool disk.

Did something OUTSIDE of amanda change recently?   Like,  an update to   gtar   
or something else that was automatically applied
to us?
Deb

On May 9, 2013, at 2:06 PM, Chris Hoogendyk wrote:

> On one of my older setups that is running Amanda 2.5.1p3, I'm getting 
> patterns of failures that I can't make sense of. Some days everything works 
> just fine. Other days I get everything from a couple of DLEs failing to a 
> half dozen. They are typically the same three servers, which are in another 
> building. However, there is a server that has the largest DLEs in that same 
> building that does not exhibit failures. There are also several servers in 
> the same building as the Amanda backup server that don't show any failures. I 
> even made a spreadsheet that shows a 0, 1 or 2 for backup levels of successes 
> and a red x for failures. Can't see any pattern.
> 
> I've looked at a bunch of things and pored over the log files to no avail.
> 
> The errors show up in the Amanda reports as:
> 
>  anise.nsm.umass.edu/home lev 1  FAILED [cannot read header: got 
> 0 instead of 32768]
>  metzi1.physics.umass.edu   /data lev 1  FAILED [cannot read header: got 
> 0 instead of 32768]
>  anise.nsm.umass.edu/home lev 1  FAILED [cannot read header: got 
> 0 instead of 32768]
>  anise.nsm.umass.edu/home lev 1  FAILED [too many dumper retry: 
> "[request failed: timeout waiting for ACK]"]
>  metzi1.physics.umass.edu   /data lev 1  FAILED [too many dumper retry: 
> "[request failed: timeout waiting for ACK]"]
>  metzi1.physics.umass.edu   /data lev 1  FAILED [cannot read header: got 
> 0 instead of 32768]
> 
> The interesting thing is that if I go to metzi1, into 
> /tmp/amanda/client/daily/ and do an `ls *0508*` I can see the debug logs from 
> last night. If I do a `grep '/data' *0508*` I can see any entries that 
> mention the DLE /data. I see no instances of sendbackup. I see only those 
> runtar debug files that correspond to the size estimates for 0, 1 and 2 level 
> backups. There is no runtar that would correspond to an actual dump.
> 
> If I do the same thing with `grep '/home' *0508*` (the DLE /home was 
> successfully backed up), then I see all the runtar debug files for the 
> estimates as well as a runtar debug file for the actual backup. I also see 
> several lines in the sendbackup debug file for /home.
> 
> I've also looked through /var/log/syslog, /var/log/auth.log, etc. on the 
> client (which is Ubuntu 12.04 LTS), and I've looked through Amanda debug 
> logs, /var/adm/messages, /var/adm/authlog, etc. on the server (which is 
> Solaris 10). I don't see any logged errors for dropped connections or 
> failures of any sort. The Amanda logs just don't mention /data on metzi1. A 
> couple of the other servers that are being backed up are Ubuntu 12.04 and 
> several are Ubuntu 10.04. None of them have been tweeked for sshd_config. All 
> have tcpkeepalive turned on.
> 
> I tried bumping up the timeouts in amanda.conf (by a factor of 5). That seems 
> a bit much, and it didn't seem to make any difference.
> 
> What should I be looking for? Where would Amanda log what is going on? (Or, 
> why would it not be logging it?)
> 
> 
> Thank you,
> 
> 
> -- 
> ---
> 
> Chris Hoogendyk
> 
> -
>   O__   Systems Administrator
>  c/ /'_ --- Biology & Geology Departments
> (*) \(*) -- 140 Morrill Science Center
> ~~ - University of Massachusetts, Amherst
> 
> 
> 
> ---
> 
> Erdös 4
> 

Re: seemingly random pattern of dump failures

[Jean-Louis Martineau]

Chris,

look in the amandad debug files on the client and the dumper debug file 
on the server.


Jean-Louis

On 05/09/2013 03:06 PM, Chris Hoogendyk wrote:
On one of my older setups that is running Amanda 2.5.1p3, I'm getting 
patterns of failures that I can't make sense of. Some days everything 
works just fine. Other days I get everything from a couple of DLEs 
failing to a half dozen. They are typically the same three servers, 
which are in another building. However, there is a server that has the 
largest DLEs in that same building that does not exhibit failures. 
There are also several servers in the same building as the Amanda 
backup server that don't show any failures. I even made a spreadsheet 
that shows a 0, 1 or 2 for backup levels of successes and a red x for 
failures. Can't see any pattern.


I've looked at a bunch of things and pored over the log files to no 
avail.


The errors show up in the Amanda reports as:

  anise.nsm.umass.edu/home lev 1  FAILED [cannot read 
header: got 0 instead of 32768]
  metzi1.physics.umass.edu   /data lev 1  FAILED [cannot read 
header: got 0 instead of 32768]
  anise.nsm.umass.edu/home lev 1  FAILED [cannot read 
header: got 0 instead of 32768]
  anise.nsm.umass.edu/home lev 1  FAILED [too many dumper 
retry: "[request failed: timeout waiting for ACK]"]
  metzi1.physics.umass.edu   /data lev 1  FAILED [too many dumper 
retry: "[request failed: timeout waiting for ACK]"]
  metzi1.physics.umass.edu   /data lev 1  FAILED [cannot read 
header: got 0 instead of 32768]


The interesting thing is that if I go to metzi1, into 
/tmp/amanda/client/daily/ and do an `ls *0508*` I can see the debug 
logs from last night. If I do a `grep '/data' *0508*` I can see any 
entries that mention the DLE /data. I see no instances of sendbackup. 
I see only those runtar debug files that correspond to the size 
estimates for 0, 1 and 2 level backups. There is no runtar that would 
correspond to an actual dump.


If I do the same thing with `grep '/home' *0508*` (the DLE /home was 
successfully backed up), then I see all the runtar debug files for the 
estimates as well as a runtar debug file for the actual backup. I also 
see several lines in the sendbackup debug file for /home.


I've also looked through /var/log/syslog, /var/log/auth.log, etc. on 
the client (which is Ubuntu 12.04 LTS), and I've looked through Amanda 
debug logs, /var/adm/messages, /var/adm/authlog, etc. on the server 
(which is Solaris 10). I don't see any logged errors for dropped 
connections or failures of any sort. The Amanda logs just don't 
mention /data on metzi1. A couple of the other servers that are being 
backed up are Ubuntu 12.04 and several are Ubuntu 10.04. None of them 
have been tweeked for sshd_config. All have tcpkeepalive turned on.


I tried bumping up the timeouts in amanda.conf (by a factor of 5). 
That seems a bit much, and it didn't seem to make any difference.


What should I be looking for? Where would Amanda log what is going on? 
(Or, why would it not be logging it?)



Thank you,

seemingly random pattern of dump failures

[Chris Hoogendyk]

On one of my older setups that is running Amanda 2.5.1p3, I'm getting patterns of failures that I 
can't make sense of. Some days everything works just fine. Other days I get everything from a couple 
of DLEs failing to a half dozen. They are typically the same three servers, which are in another 
building. However, there is a server that has the largest DLEs in that same building that does not 
exhibit failures. There are also several servers in the same building as the Amanda backup server 
that don't show any failures. I even made a spreadsheet that shows a 0, 1 or 2 for backup levels of 
successes and a red x for failures. Can't see any pattern.


I've looked at a bunch of things and pored over the log files to no avail.

The errors show up in the Amanda reports as:

  anise.nsm.umass.edu/home lev 1  FAILED [cannot read header: got 0 
instead of 32768]
  metzi1.physics.umass.edu   /data lev 1  FAILED [cannot read header: got 0 
instead of 32768]
  anise.nsm.umass.edu/home lev 1  FAILED [cannot read header: got 0 
instead of 32768]
  anise.nsm.umass.edu/home lev 1  FAILED [too many dumper retry: 
"[request failed: timeout waiting for ACK]"]
  metzi1.physics.umass.edu   /data lev 1  FAILED [too many dumper retry: 
"[request failed: timeout waiting for ACK]"]
  metzi1.physics.umass.edu   /data lev 1  FAILED [cannot read header: got 0 
instead of 32768]

The interesting thing is that if I go to metzi1, into /tmp/amanda/client/daily/ and do an `ls 
*0508*` I can see the debug logs from last night. If I do a `grep '/data' *0508*` I can see any 
entries that mention the DLE /data. I see no instances of sendbackup. I see only those runtar debug 
files that correspond to the size estimates for 0, 1 and 2 level backups. There is no runtar that 
would correspond to an actual dump.


If I do the same thing with `grep '/home' *0508*` (the DLE /home was successfully backed up), then I 
see all the runtar debug files for the estimates as well as a runtar debug file for the actual 
backup. I also see several lines in the sendbackup debug file for /home.


I've also looked through /var/log/syslog, /var/log/auth.log, etc. on the client (which is Ubuntu 
12.04 LTS), and I've looked through Amanda debug logs, /var/adm/messages, /var/adm/authlog, etc. on 
the server (which is Solaris 10). I don't see any logged errors for dropped connections or failures 
of any sort. The Amanda logs just don't mention /data on metzi1. A couple of the other servers that 
are being backed up are Ubuntu 12.04 and several are Ubuntu 10.04. None of them have been tweeked 
for sshd_config. All have tcpkeepalive turned on.


I tried bumping up the timeouts in amanda.conf (by a factor of 5). That seems a bit much, and it 
didn't seem to make any difference.


What should I be looking for? Where would Amanda log what is going on? (Or, why would it not be 
logging it?)



Thank you,


--
---

Chris Hoogendyk

-
   O__   Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~ - University of Massachusetts, Amherst



---

Erdös 4

Re: Dump failures

[Toomas Aas]

Mike R wrote:



NOTE: skipping tape-writable test
Tape daily-010 label ok
Server check took 0.410 seconds

The flush worked but I think I screwed up the labeling of the vtapes 
in the process




So should I be concerned about this? Are the tapes alright based on the 
results of the relabeling and is incorrect labeling something I should 
be concerned about? Any suggestions on my /var/www dump that reports 
problems due to files changing?




I've lost the beginning of this thread, but considering that flushing to 
daily-010 worked it should be fine. Maybe the labels do not look as nice as 
you'd want, but as long as they match the labelstr parameter in amanda.conf 
you're technically OK. If you want, you can always re-label the tapes 
manually later (just before the tape is about to be reused). I'd recommend 
doing this one tape at a time and avoiding fancy shell scripting stuff that 
might break in your particular environment.


--
Toomas Aas

Re: Dump failures

[Mike R]

Mike R wrote:

Jean-Louis Martineau wrote:

What are in those slots?
use 'amlabel' to label them.

Jean-Louis

slot 10: not an amanda tape (Read 0 bytes)
slot 11: not an amanda tape (Read 0 bytes)
slot 12: not an amanda tape (Read 0 bytes)
slot 13: not an amanda tape (Read 0 bytes)
slot 14: not an amanda tape (Read 0 bytes)
slot 15: not an amanda tape (Read 0 bytes)
slot 16: not an amanda tape (Read 0 bytes)
slot 17: not an amanda tape (Read 0 bytes)
slot 18: not an amanda tape (Read 0 bytes)
slot 19: not an amanda tape (Read 0 bytes)
slot 20: not an amanda tape (Read 0 bytes)
slot 21: not an amanda tape (Read 0 bytes)
slot 22: not an amanda tape (Read 0 bytes)
slot 23: not an amanda tape (Read 0 bytes)
slot 24: not an amanda tape (Read 0 bytes)
slot 25: not an amanda tape (Read 0 bytes)






Hmm, I'm noticing the command that I was instructed to use to label the 
tapes are as follows from http://www.zmanda.com/quick-backup-setup.html


7.Just as with physical tapes, the virtual tapes now need to be 
labeled. (Please note that the output below has been truncated.)


bash-3.00$ for ((i=1; $i<=9;i++)); do amlabel DailySet1 DailySet1-0$i 
slot $i; done

changer: got exit: 0 str: 1 file://space/vtapes/DailySet1/slots
labeling tape in slot 1 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-01, checking label, done.
...
changer: got exit: 0 str: 9 file://space/vtapes/DailySet1/slots
labeling tape in slot 9 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-09, checking label, done.

-bash-3.00$ for ((i=10; $i<=25;i++)); do amlabel DailySet1 DailySet1-$i 
slot $i; done

changer: got exit: 0 str: 10 file://space/vtapes/DailySet1/slots
labeling tape in slot 10 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
 rewinding, writing label DailySet1-10, checking label, done.
...
changer: got exit: 0 str: 25 file://space/vtapes/DailySet1/slots
labeling tape in slot 25 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-25, checking label, done.


/SNIP

SO I guess that was what was culpable of the initial problem but I think 
I might have screwed it up even worse. I noticed that the relabeling 
command that was provided in the how-to only had disk 1-9 set for the 
labeling and obviously, like a lemming, i simply copy and pasted it from 
the manual. Thinking I could fix the problem with the following command:


for ((i=10; $i<=25;i++)); do /usr/sbin/amlabel daily daily-0$i slot $i; 
done


Now my amcheck is showing the following

Holding disk /opt/dumps: 1246830 MB disk space available, using 1246730 MB
slot 25: read label `daily-025', date `20091118'
slot 1: read label `daily-01', date `20091109'
slot 2: read label `daily-02', date `20091109'
slot 3: read label `daily-03', date `20091109'
slot 4: read label `daily-04', date `20091109'
slot 5: read label `daily-05', date `20091110'
slot 6: read label `daily-06', date `2009'
slot 7: read label `daily-07', date `20091112'
slot 8: read label `daily-08', date `20091113'
slot 9: read label `daily-09', date `20091116'
slot 10: read label `daily-010', date `X'

NOTE: skipping tape-writable test
Tape daily-010 label ok
Server check took 0.410 seconds

The flush worked but I think I screwed up the labeling of the vtapes in 
the process




So should I be concerned about this? Are the tapes alright based on the 
results of the relabeling and is incorrect labeling something I should 
be concerned about? Any suggestions on my /var/www dump that reports 
problems due to files changing?

Re: Dump failures

[Mike R]

Jean-Louis Martineau wrote:

What are in those slots?
use 'amlabel' to label them.

Jean-Louis

slot 10: not an amanda tape (Read 0 bytes)
slot 11: not an amanda tape (Read 0 bytes)
slot 12: not an amanda tape (Read 0 bytes)
slot 13: not an amanda tape (Read 0 bytes)
slot 14: not an amanda tape (Read 0 bytes)
slot 15: not an amanda tape (Read 0 bytes)
slot 16: not an amanda tape (Read 0 bytes)
slot 17: not an amanda tape (Read 0 bytes)
slot 18: not an amanda tape (Read 0 bytes)
slot 19: not an amanda tape (Read 0 bytes)
slot 20: not an amanda tape (Read 0 bytes)
slot 21: not an amanda tape (Read 0 bytes)
slot 22: not an amanda tape (Read 0 bytes)
slot 23: not an amanda tape (Read 0 bytes)
slot 24: not an amanda tape (Read 0 bytes)
slot 25: not an amanda tape (Read 0 bytes)






Hmm, I'm noticing the command that I was instructed to use to label the 
tapes are as follows from http://www.zmanda.com/quick-backup-setup.html


7.Just as with physical tapes, the virtual tapes now need to be 
labeled. (Please note that the output below has been truncated.)


bash-3.00$ for ((i=1; $i<=9;i++)); do amlabel DailySet1 DailySet1-0$i 
slot $i; done

changer: got exit: 0 str: 1 file://space/vtapes/DailySet1/slots
labeling tape in slot 1 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-01, checking label, done.
...
changer: got exit: 0 str: 9 file://space/vtapes/DailySet1/slots
labeling tape in slot 9 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-09, checking label, done.

-bash-3.00$ for ((i=10; $i<=25;i++)); do amlabel DailySet1 DailySet1-$i 
slot $i; done

changer: got exit: 0 str: 10 file://space/vtapes/DailySet1/slots
labeling tape in slot 10 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
 rewinding, writing label DailySet1-10, checking label, done.
...
changer: got exit: 0 str: 25 file://space/vtapes/DailySet1/slots
labeling tape in slot 25 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-25, checking label, done.


/SNIP

SO I guess that was what was culpable of the initial problem but I think 
I might have screwed it up even worse. I noticed that the relabeling 
command that was provided in the how-to only had disk 1-9 set for the 
labeling and obviously, like a lemming, i simply copy and pasted it from 
the manual. Thinking I could fix the problem with the following command:


for ((i=10; $i<=25;i++)); do /usr/sbin/amlabel daily daily-0$i slot $i; done

Now my amcheck is showing the following

Holding disk /opt/dumps: 1246830 MB disk space available, using 1246730 MB
slot 25: read label `daily-025', date `20091118'
slot 1: read label `daily-01', date `20091109'
slot 2: read label `daily-02', date `20091109'
slot 3: read label `daily-03', date `20091109'
slot 4: read label `daily-04', date `20091109'
slot 5: read label `daily-05', date `20091110'
slot 6: read label `daily-06', date `2009'
slot 7: read label `daily-07', date `20091112'
slot 8: read label `daily-08', date `20091113'
slot 9: read label `daily-09', date `20091116'
slot 10: read label `daily-010', date `X'

NOTE: skipping tape-writable test
Tape daily-010 label ok
Server check took 0.410 seconds

The flush worked but I think I screwed up the labeling of the vtapes in 
the process

Re: Dump failures

[Jean-Louis Martineau]

What are in those slots?
use 'amlabel' to label them.

Jean-Louis

slot 10: not an amanda tape (Read 0 bytes)
slot 11: not an amanda tape (Read 0 bytes)
slot 12: not an amanda tape (Read 0 bytes)
slot 13: not an amanda tape (Read 0 bytes)
slot 14: not an amanda tape (Read 0 bytes)
slot 15: not an amanda tape (Read 0 bytes)
slot 16: not an amanda tape (Read 0 bytes)
slot 17: not an amanda tape (Read 0 bytes)
slot 18: not an amanda tape (Read 0 bytes)
slot 19: not an amanda tape (Read 0 bytes)
slot 20: not an amanda tape (Read 0 bytes)
slot 21: not an amanda tape (Read 0 bytes)
slot 22: not an amanda tape (Read 0 bytes)
slot 23: not an amanda tape (Read 0 bytes)
slot 24: not an amanda tape (Read 0 bytes)
slot 25: not an amanda tape (Read 0 bytes)

Re: Dump failures

[Mike R]

Jean-Louis Martineau wrote:

What's the complete output of: amcheck -s daily


Mike R wrote:

Jean-Louis Martineau wrote:

What is your tapecycle?
How many vtapes do you have?
What is the output of amcheck?

Jean-Louis

Mike R wrote:
Good day all. I'm a fairly new amanda user, I used the 15 minute 
backup guide for my setup. During the first week my backups seemed 
to have gone off without a hitch as indicated by the backup reports. 
Now beginning on Monday I've been getting failed backup reports, as 
follows:


*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: a new tape.

FAILURE AND STRANGE DUMP SUMMARY:
epsilon.bleh.com /var/www lev 1 STRANGE


STATISTICS:
Total Full Incr.
  
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:02
Dump Time (hrs:min) 0:01 0:00 0:01
Output Size (meg) 79.9 0.0 79.9
Original Size (meg) 919.2 0.0 919.2
Avg Compressed Size (%) 8.7 -- 8.7 (level:#disks ...)
Filesystems Dumped 3 0 3 (1:3)
Avg Dump Rate (k/s) 1110.8 -- 1110.8

Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 0 0 0

Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --


FAILED AND STRANGE DUMP DETAILS:

/-- epsilon.bleh.com /var/www lev 1 STRANGE
sendbackup: start [epsilon.bleh.com:/var/www level 1]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
? gtar: ./cacti/log/cacti.log: file changed as we read it
| Total bytes written: 946503680 (903MiB, 14MiB/s)
sendbackup: size 924320
sendbackup: end
\


DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SS KB/s MMM:SS KB/s
-- - 
-

epsilon.bleh /etc/nagios 1 0 0 10.0 0:01 1.3 N/A N/A
epsilon.bleh /var/www 1 903 75 8.3 1:08 1132.5 N/A N/A
gamma.bleh.com /websites 1 17 5 28.1 0:05 979.1 N/A N/A

(brought to you by Amanda version 2.5.0p2)


I'm starting to think that I might have ran too many testing backups 
after the implementation and might have thrown the vtape usage off 
course if it's technically possible. I ran amflush daily as amanda 
and there were no results from it, the report I got said the same 
thing. I'm also concerned with the strange dump failure for cacti, 
due to its nature and the fact that it is configured to spread the 
load out evenly it is probably constantly writing data to the rra 
files, so I need to make sure I'm getting reliable dumps. Any 
additional insight in my paltry problems would help as I will be 
spending the day researching it on my end. Thanks!





Thank you for your response.

dumpcycle 4 weeks
runspercycle 20
tapecycle 25 tapes

there are 25 "slots" in my vtape directory (I believe a slot = a tape 
correct?)


[r...@sigma daily]# su - amanda
-bash-3.2$ /usr/sbin/amflush daily
Scanning /opt/dumps...
  20091117010001: found Amanda directory.
  20091118010001: found Amanda directory.

Multiple Amanda directories, please pick one by letter:
  A. 20091117010001
  B. 20091118010001
Select directories to flush [A..B]: [ALL]

Today is: 20091118
Flushing dumps in 20091117010001, 20091118010001 using tape changer 
"chg-disk".

Expecting a new tape.  (The last dumps were to tape daily-09)
Are you sure you want to do this [yN]? y
Running in background, you can log off now.
You'll get mail when amflush is finished.

RESULT

*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush again to flush them to tape.
The next tape Amanda expects to use is: a new tape.


Also I have posted my config here: http://pastebin.ca/1676675

Also, when setting up the system I followed these directions:

http://www.zmanda.com/quick-backup-setup.html

Thank you for your help!






[r...@sigma daily]# su - amanda
-bash-3.2$ /usr/sbin/amcheck -s daily
Amanda Tape Server Host Check
-
Holding disk /opt/dumps: 1246831 MB disk space available, using 1246731 MB
slot 5: read label `daily-05', date `20091110'
slot 6: read label `daily-06', date `2009'
slot 7: read label `daily-07', date `20091112'
slot 8: read label `daily-08', date `20091113'
slot 9: read label `daily-09', date `20091116'
slot 10: not an amanda tape (Read 0 bytes)
slot 11: not an amanda tape (Read 0 bytes)
slot 12: not an amanda tape (Read 0 bytes)
slot 13: not an amanda tape (Read 0 bytes)
slot 14: not an amanda tape (Read 0 bytes)
slot 15: not an amanda tape (Read 0 bytes)
slot 16: not an amanda tape (Read 0 bytes)
slot 17: not an amanda tape (Read 0 bytes)
slot 18: not an amanda tape (Read 0 bytes)
slot 19: not an amanda tape (Read 0 bytes)
slot 20: not an amanda tape (Read 0 bytes)
slot 21: not an amanda tape (Read 0 bytes)
slot 22: not an amanda tape (Read 0 bytes)
slot 23: not an am

Re: Dump failures

[Jean-Louis Martineau]

What's the complete output of: amcheck -s daily


Mike R wrote:

Jean-Louis Martineau wrote:

What is your tapecycle?
How many vtapes do you have?
What is the output of amcheck?

Jean-Louis

Mike R wrote:
Good day all. I'm a fairly new amanda user, I used the 15 minute 
backup guide for my setup. During the first week my backups seemed 
to have gone off without a hitch as indicated by the backup reports. 
Now beginning on Monday I've been getting failed backup reports, as 
follows:


*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: a new tape.

FAILURE AND STRANGE DUMP SUMMARY:
epsilon.bleh.com /var/www lev 1 STRANGE


STATISTICS:
Total Full Incr.
  
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:02
Dump Time (hrs:min) 0:01 0:00 0:01
Output Size (meg) 79.9 0.0 79.9
Original Size (meg) 919.2 0.0 919.2
Avg Compressed Size (%) 8.7 -- 8.7 (level:#disks ...)
Filesystems Dumped 3 0 3 (1:3)
Avg Dump Rate (k/s) 1110.8 -- 1110.8

Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 0 0 0

Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --


FAILED AND STRANGE DUMP DETAILS:

/-- epsilon.bleh.com /var/www lev 1 STRANGE
sendbackup: start [epsilon.bleh.com:/var/www level 1]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
? gtar: ./cacti/log/cacti.log: file changed as we read it
| Total bytes written: 946503680 (903MiB, 14MiB/s)
sendbackup: size 924320
sendbackup: end
\


DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SS KB/s MMM:SS KB/s
-- - 
-

epsilon.bleh /etc/nagios 1 0 0 10.0 0:01 1.3 N/A N/A
epsilon.bleh /var/www 1 903 75 8.3 1:08 1132.5 N/A N/A
gamma.bleh.com /websites 1 17 5 28.1 0:05 979.1 N/A N/A

(brought to you by Amanda version 2.5.0p2)


I'm starting to think that I might have ran too many testing backups 
after the implementation and might have thrown the vtape usage off 
course if it's technically possible. I ran amflush daily as amanda 
and there were no results from it, the report I got said the same 
thing. I'm also concerned with the strange dump failure for cacti, 
due to its nature and the fact that it is configured to spread the 
load out evenly it is probably constantly writing data to the rra 
files, so I need to make sure I'm getting reliable dumps. Any 
additional insight in my paltry problems would help as I will be 
spending the day researching it on my end. Thanks!





Thank you for your response.

dumpcycle 4 weeks
runspercycle 20
tapecycle 25 tapes

there are 25 "slots" in my vtape directory (I believe a slot = a tape 
correct?)


[r...@sigma daily]# su - amanda
-bash-3.2$ /usr/sbin/amflush daily
Scanning /opt/dumps...
  20091117010001: found Amanda directory.
  20091118010001: found Amanda directory.

Multiple Amanda directories, please pick one by letter:
  A. 20091117010001
  B. 20091118010001
Select directories to flush [A..B]: [ALL]

Today is: 20091118
Flushing dumps in 20091117010001, 20091118010001 using tape changer 
"chg-disk".

Expecting a new tape.  (The last dumps were to tape daily-09)
Are you sure you want to do this [yN]? y
Running in background, you can log off now.
You'll get mail when amflush is finished.

RESULT

*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush again to flush them to tape.
The next tape Amanda expects to use is: a new tape.


Also I have posted my config here: http://pastebin.ca/1676675

Also, when setting up the system I followed these directions:

http://www.zmanda.com/quick-backup-setup.html

Thank you for your help!

Re: Dump failures

[Mike R]

Jean-Louis Martineau wrote:

What is your tapecycle?
How many vtapes do you have?
What is the output of amcheck?

Jean-Louis

Mike R wrote:
Good day all. I'm a fairly new amanda user, I used the 15 minute 
backup guide for my setup. During the first week my backups seemed to 
have gone off without a hitch as indicated by the backup reports. Now 
beginning on Monday I've been getting failed backup reports, as follows:


*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: a new tape.

FAILURE AND STRANGE DUMP SUMMARY:
epsilon.bleh.com /var/www lev 1 STRANGE


STATISTICS:
Total Full Incr.
  
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:02
Dump Time (hrs:min) 0:01 0:00 0:01
Output Size (meg) 79.9 0.0 79.9
Original Size (meg) 919.2 0.0 919.2
Avg Compressed Size (%) 8.7 -- 8.7 (level:#disks ...)
Filesystems Dumped 3 0 3 (1:3)
Avg Dump Rate (k/s) 1110.8 -- 1110.8

Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 0 0 0

Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --


FAILED AND STRANGE DUMP DETAILS:

/-- epsilon.bleh.com /var/www lev 1 STRANGE
sendbackup: start [epsilon.bleh.com:/var/www level 1]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
? gtar: ./cacti/log/cacti.log: file changed as we read it
| Total bytes written: 946503680 (903MiB, 14MiB/s)
sendbackup: size 924320
sendbackup: end
\


DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SS KB/s MMM:SS KB/s
-- - 
-

epsilon.bleh /etc/nagios 1 0 0 10.0 0:01 1.3 N/A N/A
epsilon.bleh /var/www 1 903 75 8.3 1:08 1132.5 N/A N/A
gamma.bleh.com /websites 1 17 5 28.1 0:05 979.1 N/A N/A

(brought to you by Amanda version 2.5.0p2)


I'm starting to think that I might have ran too many testing backups 
after the implementation and might have thrown the vtape usage off 
course if it's technically possible. I ran amflush daily as amanda and 
there were no results from it, the report I got said the same thing. 
I'm also concerned with the strange dump failure for cacti, due to its 
nature and the fact that it is configured to spread the load out 
evenly it is probably constantly writing data to the rra files, so I 
need to make sure I'm getting reliable dumps. Any additional insight 
in my paltry problems would help as I will be spending the day 
researching it on my end. Thanks!





Thank you for your response.

dumpcycle 4 weeks
runspercycle 20
tapecycle 25 tapes

there are 25 "slots" in my vtape directory (I believe a slot = a tape 
correct?)


[r...@sigma daily]# su - amanda
-bash-3.2$ /usr/sbin/amflush daily
Scanning /opt/dumps...
  20091117010001: found Amanda directory.
  20091118010001: found Amanda directory.

Multiple Amanda directories, please pick one by letter:
  A. 20091117010001
  B. 20091118010001
Select directories to flush [A..B]: [ALL]

Today is: 20091118
Flushing dumps in 20091117010001, 20091118010001 using tape changer 
"chg-disk".

Expecting a new tape.  (The last dumps were to tape daily-09)
Are you sure you want to do this [yN]? y
Running in background, you can log off now.
You'll get mail when amflush is finished.

RESULT

*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush again to flush them to tape.
The next tape Amanda expects to use is: a new tape.


Also I have posted my config here: http://pastebin.ca/1676675

Also, when setting up the system I followed these directions:

http://www.zmanda.com/quick-backup-setup.html

Thank you for your help!

Re: Dump failures

[Jean-Louis Martineau]

What is your tapecycle?
How many vtapes do you have?
What is the output of amcheck?

Jean-Louis

Mike R wrote:
Good day all. I'm a fairly new amanda user, I used the 15 minute 
backup guide for my setup. During the first week my backups seemed to 
have gone off without a hitch as indicated by the backup reports. Now 
beginning on Monday I've been getting failed backup reports, as follows:


*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: a new tape.

FAILURE AND STRANGE DUMP SUMMARY:
epsilon.bleh.com /var/www lev 1 STRANGE


STATISTICS:
Total Full Incr.
  
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:02
Dump Time (hrs:min) 0:01 0:00 0:01
Output Size (meg) 79.9 0.0 79.9
Original Size (meg) 919.2 0.0 919.2
Avg Compressed Size (%) 8.7 -- 8.7 (level:#disks ...)
Filesystems Dumped 3 0 3 (1:3)
Avg Dump Rate (k/s) 1110.8 -- 1110.8

Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 0 0 0

Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --


FAILED AND STRANGE DUMP DETAILS:

/-- epsilon.bleh.com /var/www lev 1 STRANGE
sendbackup: start [epsilon.bleh.com:/var/www level 1]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
? gtar: ./cacti/log/cacti.log: file changed as we read it
| Total bytes written: 946503680 (903MiB, 14MiB/s)
sendbackup: size 924320
sendbackup: end
\


DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SS KB/s MMM:SS KB/s
-- - 
-

epsilon.bleh /etc/nagios 1 0 0 10.0 0:01 1.3 N/A N/A
epsilon.bleh /var/www 1 903 75 8.3 1:08 1132.5 N/A N/A
gamma.bleh.com /websites 1 17 5 28.1 0:05 979.1 N/A N/A

(brought to you by Amanda version 2.5.0p2)


I'm starting to think that I might have ran too many testing backups 
after the implementation and might have thrown the vtape usage off 
course if it's technically possible. I ran amflush daily as amanda and 
there were no results from it, the report I got said the same thing. 
I'm also concerned with the strange dump failure for cacti, due to its 
nature and the fact that it is configured to spread the load out 
evenly it is probably constantly writing data to the rra files, so I 
need to make sure I'm getting reliable dumps. Any additional insight 
in my paltry problems would help as I will be spending the day 
researching it on my end. Thanks!

Dump failures

[Mike R]

Good day all. I'm a fairly new amanda user, I used the 15 minute backup 
guide for my setup. During the first week my backups seemed to have gone 
off without a hitch as indicated by the backup reports. Now beginning on 
Monday I've been getting failed backup reports, as follows:


*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: a new tape.

FAILURE AND STRANGE DUMP SUMMARY:
epsilon.bleh.com /var/www lev 1 STRANGE


STATISTICS:
Total Full Incr.
  
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:02
Dump Time (hrs:min) 0:01 0:00 0:01
Output Size (meg) 79.9 0.0 79.9
Original Size (meg) 919.2 0.0 919.2
Avg Compressed Size (%) 8.7 -- 8.7 (level:#disks ...)
Filesystems Dumped 3 0 3 (1:3)
Avg Dump Rate (k/s) 1110.8 -- 1110.8

Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 0 0 0

Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --


FAILED AND STRANGE DUMP DETAILS:

/-- epsilon.bleh.com /var/www lev 1 STRANGE
sendbackup: start [epsilon.bleh.com:/var/www level 1]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
? gtar: ./cacti/log/cacti.log: file changed as we read it
| Total bytes written: 946503680 (903MiB, 14MiB/s)
sendbackup: size 924320
sendbackup: end
\


DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SS KB/s MMM:SS KB/s
-- - 
-

epsilon.bleh /etc/nagios 1 0 0 10.0 0:01 1.3 N/A N/A
epsilon.bleh /var/www 1 903 75 8.3 1:08 1132.5 N/A N/A
gamma.bleh.com /websites 1 17 5 28.1 0:05 979.1 N/A N/A

(brought to you by Amanda version 2.5.0p2)


I'm starting to think that I might have ran too many testing backups 
after the implementation and might have thrown the vtape usage off 
course if it's technically possible. I ran amflush daily as amanda and 
there were no results from it, the report I got said the same thing. I'm 
also concerned with the strange dump failure for cacti, due to its 
nature and the fact that it is configured to spread the load out evenly 
it is probably constantly writing data to the rra files, so I need to 
make sure I'm getting reliable dumps. Any additional insight in my 
paltry problems would help as I will be spending the day researching it 
on my end. Thanks!

Re: Dump failures

[Gene Heskett]

On Wednesday 01 July 2009, Gene Heskett wrote:
>Greetings;
>
>I have been getting a failure from amcheckdump since the 28th of June,
>complaining according to a re-run, of the /usr/movies DLE.  du -h . says its
>11 GB, chunk size is set to 2GB, and the tapesize at 40GB.  Seems to me like
>it would fit, but:
>
>Validating image coyote:/usr/movies datestamp 20090701001504 level 0 part 1
> on tape Dailys-9 file #2
>/bin/tar: Unexpected EOF in archive
>/bin/tar: Error is not recoverable: exiting now
>Dump was not successfully validated: Validation process returned 2 (full
>status 512)
>  using '/usr/local/libexec/amanda/application/amgtar validate > /dev/null
> && cat > /dev/null'.
>
>I sic'ed a session of dd piped to tar after it, and it has read a lot more
>than amcheckdump does before it exits, or I used the wrong syntax, possible
>since its been ages since I last did that.
>
>This is 2.6.2, built from the 20090622 snapshot.  kernel 2.6.30, fedora 10
>install.
>
>Ideas anybody?
>
>Thanks.

I know, its considered very poor form to reply to ones own posts, but I have 
another head scratcher to relate.  I have been mounting this partition as a 
plain, no frills ext4 filesystem per recommendations from the kernel folks, 
and I don't believe that I couldn't remount it as ext3 since it was originally 
formatted as ext3.

Consider that this is the virtual tape as written last night:

[r...@coyote data]# pwd
/amandatapes/Dailys/data

[r...@coyote data]# du -h .
329M./Loris-wedding
172K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding017
92K ./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding005
152K./Nikis-Wedding/nicciswedding-qdvd/Nikkis-Wedding
132K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding008
156K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding009
140K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding014
140K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding019
164K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding010
124K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding004
148K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding012
164K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding016
116K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding003
132K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding007
156K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding013
60K ./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding018
12K ./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding001
184K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding002
120K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding006
140K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding015
148K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding011
3.0M./Nikis-Wedding/nicciswedding-qdvd
12K ./Nikis-Wedding/nikis-Wedding2
768M./Nikis-Wedding/tmp/VIDEO_TS
4.0K./Nikis-Wedding/tmp/AUDIO_TS
768M./Nikis-Wedding/tmp
6.8G./Nikis-Wedding
2.3G./emc-at-work
46G .

46G?  in data/* ?

Somebodies adding machine is broken.  Bug in du -h?  Bug in ext4?

Hu, I just unmounted it and remounted it as ext3, and the du -h . results 
are unchanged.  Still 46G.  But the src of that level 0 backup is 11G.

[r...@coyote data]# cd /usr/movies
[r...@coyote movies]# du -h .
329M./Loris-wedding
2.3G./emc-at-work
72K ./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding018
168K./Nikis-Wedding/nicciswedding-qdvd/Nikkis-Wedding
200K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding002
104K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding005
152K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding014
132K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding006
184K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding017
128K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding003
176K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding016
160K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding011
152K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding015
168K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding013
144K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding008
144K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding007
176K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding010
152K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding019
160K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding012
24K ./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding001
136K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding004
168K./Nikis-Wedding/nicciswedding-qdvd/nicci-wedding009
3.3M./Nikis-Wedding/nicciswedding-qdvd
20K ./Nikis-Wedding/nikis-Wedding2
8.0K./Nikis-Wedding/tmp/AUDIO_TS
768M./Nikis-Wedding/tmp/VIDEO_TS
768M./Nikis-Wedding/tmp
6.8G./Nikis-Wedding
11G .

Not 46G.

FWIW, e2fsck 1.41.4 says the device filesystem is clean.

Call me puzzled, but looking for clues.  If anyone has a spare clue I could 
sure use it.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed 

Dump failures

[Gene Heskett]

Greetings;

I have been getting a failure from amcheckdump since the 28th of June, 
complaining according to a re-run, of the /usr/movies DLE.  du -h . says its 
11 GB, chunk size is set to 2GB, and the tapesize at 40GB.  Seems to me like 
it would fit, but:

Validating image coyote:/usr/movies datestamp 20090701001504 level 0 part 1 on 
tape Dailys-9 file #2
/bin/tar: Unexpected EOF in archive
/bin/tar: Error is not recoverable: exiting now
Dump was not successfully validated: Validation process returned 2 (full 
status 512)
  using '/usr/local/libexec/amanda/application/amgtar validate > /dev/null && 
cat > /dev/null'.

I sicced a session of dd piped to tar after it, and it has read a lot more 
than amcheckdump does before it exits, or I used the wrong syntax, possible 
since its been ages since I last did that.

This is 2.6.2, built from the 20090622 snapshot.  kernel 2.6.30, fedora 10 
install.

Ideas anybody?

Thanks.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
The NRA is offering FREE Associate memberships to anyone who wants them.


There is an innocence in admiration; it is found in those to whom it
has not yet occurred that they, too, might be admired some day.
-- Friedrich Nietzsche

Dump failures

[Frank Bertrand]

If anyone can give me a clue as to what is going on, it would be greatly appreciated.

Help 1:


I have a Linux box running RedHat 7.1;

Linux sxgate 2.4.2-2smp #1 SMP Sun Apr 8 20:21:34 EDT 2001 i686 unknown

FilesystemSize  Used Avail Use% Mounted on
/dev/sda5  13G  3.2G  9.5G  25% /
/dev/sda1 517M  7.8M  483M   2% /boot
/dev/md0   33G  3.5G   28G  11% /home (Software striped (Raid5) across 3 
drives)

When I start my dumps with all 3 filesystems, only one will start, the other two will 
fail.
(Not always the same one starts)

amanda.debug:
=

Amanda 2.4 REQ HANDLE 000-10010148 SEQ 999202365
SECURITY USER root
SERVICE sendbackup
OPTIONS hostname=sxgate;
DUMP /boot 0 1970:1:1:0:0:0 OPTIONS |;bsd-auth;


sending ack:

Amanda 2.4 ACK HANDLE 000-10010148 SEQ 999202365


bsd security: remote host sumire.hstc.necsyl.com user root local user amanda
amandahosts security check passed
amandad: running service "/usr/lib/amanda/sendbackup"
amandad: got packet:

Amanda 2.4 REQ HANDLE 000-10010398 SEQ 999202365
SECURITY USER root
SERVICE sendbackup
OPTIONS hostname=sxgate;
DUMP / 0 1970:1:1:0:0:0 OPTIONS |;bsd-auth;


amandad: received other packet, NAKing it
  addr: peer 198.170.2.24 dup 198.170.2.24, port: peer 567 dup 519
sending nack:

Amanda 2.4 NAK HANDLE 000-10010398 SEQ 999202365
ERROR amandad busy


amandad: got packet:

Amanda 2.4 REQ HANDLE 000-10010148 SEQ 999202365
SECURITY USER root
SERVICE sendbackup
OPTIONS hostname=sxgate;
DUMP /home 0 1970:1:1:0:0:0 OPTIONS |;bsd-auth;


amandad: received other packet, NAKing it
  addr: peer 198.170.2.24 dup 198.170.2.24, port: peer 567 dup 574
sending nack:

Amanda 2.4 NAK HANDLE 000-10010148 SEQ 999202365
ERROR amandad busy


amdump:
===
amdump: start at Thu Aug 30 15:12:45 CDT 2001
got result for host sxgate disk /home: 0 -> 3621366K, -1 -> -1K, -1 -> -1K
got result for host sxgate disk /boot: 0 -> 7954K, -1 -> -1K, -1 -> -1K
got result for host sxgate disk /: 0 -> 3251847K, -1 -> -1K, -1 -> -1K
getting estimates took 15.440 secs

GENERATING SCHEDULE:

sxgate /home 9 0 1970:1:1:0:0:0 3621366 2150
sxgate / 7 0 1970:1:1:0:0:0 3251847 108394
sxgate /boot 7 0 1970:1:1:0:0:0 7954 1

driver: send-cmd time 15.488 to dumper0: FILE-DUMP 00-1 
/amanda/118/20010830/sxgate._boot.0 sxgate /boot 0 1970:1:1:0:0:0 DUMP |;bsd-auth;
driver: send-cmd time 15.488 to dumper1: FILE-DUMP 01-2 
/amanda/118/20010830/sxgate._home.0 sxgate /home 0 1970:1:1:0:0:0 DUMP |;bsd-auth;
driver: send-cmd time 15.488 to dumper2: FILE-DUMP 02-3 
/amanda/118/20010830/sxgate._.0 sxgate / 0 1970:1:1:0:0:0 DUMP |;bsd-auth;
driver: result time 15.502 from dumper2: FAILED 02-3 amandad busy
driver: result time 15.504 from dumper1: FAILED 01-2 amandad busy


Help 2:

When I run only the dump for the filesystem that is striped, the dump does dump across 
the network to the
holding disk, but will stop after a while, wait for the timeout period, then fail.

Any ideas, is it because of the striping?

Again, any suggestion will be appreciated.

Thanks.

-
***
Frank Bertrand | Phone:  281/465-1503
NEC Systems, Inc.  | Fax:281/465-1599
4200 Research Forest Drive | E-mail: [EMAIL PROTECTED]
The Woodlands, TX 77381| URL: http://www.necservers.com
***