Comperession/Tape Size Issues
I am currently experiencing problems with Amanda backup runs completing
successfully, supposedly due to running out of tape. I am using
Amanda 2.4.3 on a FreeBSD 4.8-STABLE machine, backing up to 35/70GB
DLT 7000 drive. Compression is enabled on the drive, and we are also
using compression within Amanda. We are getting nowhere near the
expected 70GB compressed capacity of the tapes.
I would guess that something is misconfigured somehow, and I am simply
not catching it. If somebody else has suggestions or information on how
to deal with these problems, it would be greatly appreciated. Thanks in
advance!!
--Joshua D. Bello
Relevant info:
## mt status output ##
Mode Density Blocksize bpi Compression
Current: 0x1b:DLTapeIV(35GB)variable 85937IDRC
-available modes-
0:0x1b:DLTapeIV(35GB)variable 85937IDRC
1:0x1b:DLTapeIV(35GB)variable 85937IDRC
2:0x1b:DLTapeIV(35GB)variable 85937IDRC
3:0x1b:DLTapeIV(35GB)variable 85937IDRC
-
Current Driver State: at rest.
-
File Number: 0 Record Number: 0Residual Count 0
## Relevant lines of amanda.conf ##
define tapetype DLT7000 {
comment Quantum DLT7000
length 35000 mbytes # 35Gb tapes
filemark 8 kbytes # as per all examples
look like this
speed 10 mbytes # not sure about this just yet
#speed 2500 kbytes
}
## An example of the lines in disklist ##
ra / comp-root
ra /varcomp-high
ra /usr/local comp-user
ra /local0 comp-high
Re: Comperession/Tape Size Issues
On Mon, 7 Jul 2003 at 10:26am, Joshua D. Bello wrote I am currently experiencing problems with Amanda backup runs completing successfully, supposedly due to running out of tape. I am using Amanda 2.4.3 on a FreeBSD 4.8-STABLE machine, backing up to 35/70GB DLT 7000 drive. Compression is enabled on the drive, and we are also using compression within Amanda. We are getting nowhere near the expected 70GB compressed capacity of the tapes. Bad, bad, bad. In trying to hardware compress already software-compressed data, it's going to expand. Either: Use software compression only, which allows amanda to better estimate tape usage (amanda will keep track of the compressed size of the FSs). In this case, use the actual size of the tapes in your tapetype. Or use hardware compression. In this case, lie to amanda about how big your tapes are (you'll likely not get anywhere near 2X compression). You save CPU cycles, but your tapelength will just be an estimate. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: Comperession/Tape Size Issues
On Monday 07 July 2003 13:26, Joshua D. Bello wrote:
I am currently experiencing problems with Amanda backup runs
completing successfully, supposedly due to running out of tape. I
am using Amanda 2.4.3 on a FreeBSD 4.8-STABLE machine, backing up
to 35/70GB DLT 7000 drive. Compression is enabled on the drive,
and we are also using compression within Amanda. We are getting
nowhere near the expected 70GB compressed capacity of the tapes.
I would guess that something is misconfigured somehow, and I am
simply not catching it. If somebody else has suggestions or
information on how to deal with these problems, it would be greatly
appreciated. Thanks in advance!!
--Joshua D. Bello
As Joshua Baker-LePain says, bad, bad, bad.
You cannot use both compressions. In doing so, its quite likely that
the data, already being compressed, will expand enough that what
amanda thinks is 35Gb of data, measured after compression, will be
enough over 35Gb to cause the tape to hit EOT, its full.
That 70Gb rating is what you'll get under ideal conditions using
hardware only compression on 'sparse' files.
Feed that HW chip a tar.bz2, and some are smart enough to step aside,
but if it doesn't, the file will grow.
You have 2 choices. You can shut down the software compression and
let the drive do its thing. The disadvantage to that is that you'll
have to lie to amanda and set the tapetypes capacity down 15 to 20%
from the 70Gb the makers use as sales propaganda. This will be
required because amanda has no way to tell how many bytes have
actually been written to the tape when the HW chip is in use.
Or you can shut down the hardware compression. That will probably
take a conversion routine applied to each tape to get rid of the
hey, I'm compressed flags that will turn the compression back on
even when the dipswitch has been set to off, this so that the drive
can read a formerly compressed tape. Unforch, it seems to carry over
into the next write, maintaining the status quo.
Anyway, once thats done, then amanda can compress, probably better
than the hardware can, and it keep track of the files compressed
size, so it can know to within 1% of the tapes raw capacity just how
much has been written to the media. That would be 35Gb in your case.
Generally speaking, this group tends to recommend against useing the
hardware compressor, particularly if the tape drive is a bit small
for the job because the software can do a better job of squeezing,
averageing around 40% of the input size for the output size.
Those of us who are drive challenged (lets face it, nothing compares
in cost per megabyte to an old DDS2 drive) will often break our
disklists up into smallish pieces just so we can compress that which
can be compressed, and skip that which doesn't compress.
But those are the sorts of things you learn after running amanda for a
while, paying attention to the emails amanda sends you.
Relevant info:
## mt status output ##
Mode Density Blocksize bpi Compression
Current: 0x1b:DLTapeIV(35GB)variable 85937IDRC
-available modes-
0:0x1b:DLTapeIV(35GB)variable 85937IDRC
1:0x1b:DLTapeIV(35GB)variable 85937IDRC
2:0x1b:DLTapeIV(35GB)variable 85937IDRC
3:0x1b:DLTapeIV(35GB)variable 85937IDRC
-
Current Driver State: at rest.
-
File Number: 0 Record Number: 0Residual Count 0
## Relevant lines of amanda.conf ##
define tapetype DLT7000 {
comment Quantum DLT7000
length 35000 mbytes # 35Gb tapes
filemark 8 kbytes # as per all examples
look like this
speed 10 mbytes # not sure about this just yet
#speed 2500 kbytes
}
## An example of the lines in disklist ##
ra / comp-root
ra /varcomp-high
ra /usr/local comp-user
ra /local0 comp-high
Break this up into much smaller pieces, staying at no more than 5% of
the drives 35Gb capacity. This will allow amanda to fine tune the
schedule, eventually arriving at a very consistent amount of tape
useage per run, but this will be over a few dumpcycles before its
well settled. When its all in one swell foop like '/' above, amanda
doesn't have anything to use for 'wiggling room', which is also
contributing to your EOT problems
I am doing 4 disks, in 2 machines here, with 44 entries in my
disklist. A total of over 60Gb on about 210 Gb of disks, not all of
which is in the disklist. After compression, I'm writing around 3.4
Gb per run on DDS2 tapes with a tapecycle of 28, dumpcycle and
runspercycle of 7.
Tonight I'm going to see if I can squeeze my firewalls /usr/local into
a 45th entry. It should compress fairly well, like its /usr/src
does. Both of those will often come back out at less
Re: Comperession/Tape Size Issues
Thank you to everybody for their help and suggestions. I went ahead and disabled hardware compression on my drive. Unfortunately, I am running into further problems in my attempts to clear out the previously spooled dumps with amflush. amflush fails after a short time without writing anything at all to tape. Here is what happens... I rum amflush and get the following output: Scanning /local0/ambackup/nextrials... 20030703: found Amanda directory. 20030704: found Amanda directory. Multiple Amanda directories, please pick one by letter: A. 20030703 B. 20030704 Select directories to flush [A..B]: [ALL] I get the same results, whether I pick A, B, or ALL. The amflush report looks like: STATISTICS: big fat 0's for every reported time and size value Avg Compressed Size, Avg Dump Rate and Avg Tp Write Rate all report -- NOTES: driver: WARNING: /local0/ambackup: 83886080 KB requested, but only 61325902 KB available taper: tape NEXTRIALS026 kb 0 fm 0 [OK] DUMP SUMMARY: DUMPER STATSTAPER STATS HOSTNAME DISKL ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s -- - --- anubis / NO FILE TO FLUSH --- anubis /local0 NO FILE TO FLUSH --- anubis /usr/localNO FILE TO FLUSH --- anubis /var NO FILE TO FLUSH --- athene.web / NO FILE TO FLUSH --- athene.web /var NO FILE TO FLUSH --- etc, etc, etc... While amflush is running, amstatus reports that everything is waiting to flush, as in: flush : 47 7288612k 0k, 0.00% values for everything else wait to flush : 47 7288612kk 7288612k (100.00%) ( 0.00%) 4 dumpers idle, 0 dumpers busy... The amanda log file reports: DISK amflush hades / DISK amflush hades /usr etc, etc... DISK amflush bali.web /var START amflush date 20030707 START driver date 20030707 WARNING driver WARNING: /local0/ambackup/nextrials: 104857600 KB requested, but only 61325652 KB available. STATS driver startup time 0.071 START taper datestamp 20030707 label NEXTRIALS026 tape 0 The amflush log file reports: amflush: datestamp 20030707 driver: pid 58387 executable driver version 2.4.3b2 driver: send-cmd time 0.004 to taper: START-TAPER 20030707 FLUSH ra /local0 20030703 0 /local0/ambackup/nextrials/20030703/ra._local0.0 FLUSH ra /usr/local 20030703 1 /local0/ambackup/nextrials/20030703/ra._usr_local .1 etc, etc... ENDFLUSH driver: adding holding disk 0 dir /local0/ambackup/nextrials size 61325652 reserving 61325652 out of 61325652 for degraded-mode dumps driver: start time 0.071 inparallel 4 bandwidth 1410666408 diskspace 61325652 di r OBSOLETE datestamp 20030707 driver: drain-ends tapeq LFFO big-dumpers sssS taper: pid 58388 executable taper version 2.4.3b2 taper: page size is 4096 taper: buffer size is 32768 taper: buffer[00] at 0x30048000 taper: buffer[01] at 0x3005 taper: buffer[02] at 0x30058000 taper: buffer[03] at 0x3006 taper: buffer[04] at 0x30068000 taper: buffer[05] at 0x3007 taper: buffer[06] at 0x30078000 taper: buffer[07] at 0x3008 taper: buffer[08] at 0x30088000 taper: buffer[09] at 0x3009 taper: buffer[10] at 0x30098000 taper: buffer[11] at 0x300a taper: buffer[12] at 0x300a8000 taper: buffer[13] at 0x300b taper: buffer[14] at 0x300b8000 taper: buffer[15] at 0x300c taper: buffer[16] at 0x300c8000 taper: buffer[17] at 0x300d taper: buffer[18] at 0x300d8000 taper: buffer[19] at 0x300e taper: buffer structures at 0x300e8000 for 240 bytes taper: read label `NEXTRIALS026' date `20030707' taper: wrote label `NEXTRIALS026' date `20030707' driver: result time 8.932 from taper: TAPER-OK driver: send-cmd time 8.932 to taper: FILE-WRITE 00-1 /local0/ambackup/nextr ials/20030703/ra._local0.0 ra /local0 0 20030703 driver: state time 8.932 free kps: 1410666408 space: 61325652 taper: writing idl e-dumpers: 4 qlen tapeq: 46 runq: 0 roomq: 0 wakeup: 86400 driver-idle: not-idle driver: interface-state time 8.932 if : free 60 if FXP0: free 1410065408 if LOCAL: free 1000 driver: hdisk-state time 8.932 hdisk 0: free 61325652 dumpers 0 This is frustrating, I need to get the spool disk flushed or else backups will fail completely tonight due to lack of space. Amanda has been chugging along just fine for over a year, this problem has just now crept up as my ammount of data storage has grown. Thanks again, you folks are super helpful! --Joshua D. Bello On Mon, Jul 07, 2003 at 10:26:09AM -0700, Joshua D. Bello wrote: I am currently experiencing problems with Amanda backup runs completing successfully, supposedly due to running out of tape. I am using Amanda 2.4.3 on a FreeBSD 4.8-STABLE machine, backing up to 35/70GB DLT 7000 drive. Compression is
Re: Comperession/Tape Size Issues
On Mon, Jul 07, 2003 at 03:16:33PM -0700, Joshua D. Bello wrote: I get the same results, whether I pick A, B, or ALL. The amflush report looks like: STATISTICS: big fat 0's for every reported time and size value Avg Compressed Size, Avg Dump Rate and Avg Tp Write Rate all report -- It would be useful if you could post the *full* versions of: - the amflush mail report - log file - amflush file A few things we could learn from those, that we can't learn from your exerpts: - how long amflush ran before it failed - did amflush complain about tape errors? - are you sure the DLEs *all* say NO FILE TO FLUSH, or does one somewhere in the middle of the list look different? - whatever else might turn out to be useful, but we don't yet have enough info to even know what that is :-) As well, the log and amflush exerpts seem to be for a still-running amflush, not for one that's already failed. Neither file seems to contain anything past the first few seconds into the run. Or is it really the case that amflush simply died some time after t=8.932 seconds, without writing anything more to either log file? Hard to believe, because of the existence of a mail report -- unless you generated the report manually with amcleanup or amreport. Did you? -- | | /\ |-_|/ Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / When I came back around from the dark side, there in front of me would be the landing area where the crew was, and the Earth, all in the view of my window. I couldn't help but think that there in front of me was all of humanity, except me. - Michael Collins, Apollo 11 Command Module Pilot
Re: Comperession/Tape Size Issues
On Monday 07 July 2003 18:16, Joshua D. Bello wrote: Thank you to everybody for their help and suggestions. I went ahead and disabled hardware compression on my drive. Unfortunately, I am running into further problems in my attempts to clear out the previously spooled dumps with amflush. amflush fails after a short time without writing anything at all to tape. Here is what happens... I rum amflush and get the following output: Scanning /local0/ambackup/nextrials... 20030703: found Amanda directory. 20030704: found Amanda directory. Multiple Amanda directories, please pick one by letter: A. 20030703 B. 20030704 Select directories to flush [A..B]: [ALL] I get the same results, whether I pick A, B, or ALL. The amflush report looks like: STATISTICS: big fat 0's for every reported time and size value Avg Compressed Size, Avg Dump Rate and Avg Tp Write Rate all report -- NOTES: driver: WARNING: /local0/ambackup: 83886080 KB requested, but only 61325902 KB available taper: tape NEXTRIALS026 kb 0 fm 0 [OK] DUMP SUMMARY: DUMPER STATSTAPER STATS HOSTNAME DISKL ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s -- - --- anubis / NO FILE TO FLUSH --- anubis /local0 NO FILE TO FLUSH --- anubis /usr/localNO FILE TO FLUSH --- anubis /var NO FILE TO FLUSH --- athene.web / NO FILE TO FLUSH --- athene.web /var NO FILE TO FLUSH --- etc, etc, etc... While amflush is running, amstatus reports that everything is waiting to flush, as in: flush : 47 7288612k 0k, 0.00% values for everything else wait to flush : 47 7288612kk 7288612k (100.00%) ( 0.00%) 4 dumpers idle, 0 dumpers busy... The amanda log file reports: DISK amflush hades / DISK amflush hades /usr etc, etc... DISK amflush bali.web /var START amflush date 20030707 START driver date 20030707 WARNING driver WARNING: /local0/ambackup/nextrials: 104857600 KB requested, but only 61325652 KB available. STATS driver startup time 0.071 START taper datestamp 20030707 label NEXTRIALS026 tape 0 The amflush log file reports: amflush: datestamp 20030707 driver: pid 58387 executable driver version 2.4.3b2 driver: send-cmd time 0.004 to taper: START-TAPER 20030707 FLUSH ra /local0 20030703 0 /local0/ambackup/nextrials/20030703/ra._local0.0 FLUSH ra /usr/local 20030703 1 /local0/ambackup/nextrials/20030703/ra._usr_local .1 etc, etc... ENDFLUSH driver: adding holding disk 0 dir /local0/ambackup/nextrials size 61325652 reserving 61325652 out of 61325652 for degraded-mode dumps driver: start time 0.071 inparallel 4 bandwidth 1410666408 diskspace 61325652 di r OBSOLETE datestamp 20030707 driver: drain-ends tapeq LFFO big-dumpers sssS taper: pid 58388 executable taper version 2.4.3b2 taper: page size is 4096 taper: buffer size is 32768 taper: buffer[00] at 0x30048000 taper: buffer[01] at 0x3005 taper: buffer[02] at 0x30058000 taper: buffer[03] at 0x3006 taper: buffer[04] at 0x30068000 taper: buffer[05] at 0x3007 taper: buffer[06] at 0x30078000 taper: buffer[07] at 0x3008 taper: buffer[08] at 0x30088000 taper: buffer[09] at 0x3009 taper: buffer[10] at 0x30098000 taper: buffer[11] at 0x300a taper: buffer[12] at 0x300a8000 taper: buffer[13] at 0x300b taper: buffer[14] at 0x300b8000 taper: buffer[15] at 0x300c taper: buffer[16] at 0x300c8000 taper: buffer[17] at 0x300d taper: buffer[18] at 0x300d8000 taper: buffer[19] at 0x300e taper: buffer structures at 0x300e8000 for 240 bytes taper: read label `NEXTRIALS026' date `20030707' taper: wrote label `NEXTRIALS026' date `20030707' driver: result time 8.932 from taper: TAPER-OK driver: send-cmd time 8.932 to taper: FILE-WRITE 00-1 /local0/ambackup/nextr ials/20030703/ra._local0.0 ra /local0 0 20030703 driver: state time 8.932 free kps: 1410666408 space: 61325652 taper: writing idl e-dumpers: 4 qlen tapeq: 46 runq: 0 roomq: 0 wakeup: 86400 driver-idle: not-idle driver: interface-state time 8.932 if : free 60 if FXP0: free 1410065408 if LOCAL: free 1000 driver: hdisk-state time 8.932 hdisk 0: free 61325652 dumpers 0 This is frustrating, I need to get the spool disk flushed or else backups will fail completely tonight due to lack of space. Amanda has been chugging along just fine for over a year, this problem has just now crept up as my ammount of data storage has grown. From the looks of it, you have more than a tapefull, so its not going to fit, no way, no how. This goes back to the relatively huge entries in the disklist. I'd bite the bullet, delete the missed backups from the holding disk, break the disklist up into manageable sized pieces and only expose about
