Re: Testing tapes before use / bad tape
Hi! Am Mo, 2003-11-24 um 13.54 schrieb Gene Heskett: On Monday 24 November 2003 03:46, Martin Oehler wrote: [...] Doing this to a tape also should tell you whether or not the drives internal compression is in use, which for amanda, should be turned off as that hides the true size of the tape from amanda. I need to work with hardware compression because I have 6 hours at night to execute my backup before people are starting to change files. With software compression, backup doesn't finish in time because the compression itself takes to long (I think an amanda mode that first moves the data completely to the backup server and then starting to compress, freeing the backed up boxes, would be a good solution. The online compresssion eats a lot of time). Amanda counts bytes sent to the drive after any gzip is applied, and if the drives compressor is on, the data normally will grow slightly and amanda may hit EOT thinking it still has 10-15% of the tape left. The Quantum drive hardware compression is completely manageable via the mt command. I'm now turning the compression off before starting the tape test and turn it back on for backup. When I'm finished with tuning the amtapetype test I want to use this inside my backup script, so each tape is tested before amflush starts (I'm using amanda with holding disks). Humm, do I recall that name from a decade or so back when I was running an amiga? Hehe, sorry, I still own an Amiga 500 but I wasn't very skilled a decade ago (at least not that skilled to become popular). Regards, Martin Öhler
Re: Testing tapes before use / bad tape
Hi! Am Mo, 2003-11-24 um 23.30 schrieb Eric Siegerman: On Mon, Nov 24, 2003 at 09:46:31AM +0100, Martin Oehler wrote: Hmm, the only option that sounds like it could speed up the [amtapetype] process is blocksize. Does anyone know a good value for this? The same value as amdump will be using! With some tape technologies, the tape's capacity depends very much on the block size. In such a case, using a different block size for the test would give misleading results. Ok. On Sun, Nov 23, 2003 at 10:28:38AM +0100, Martin Oehler wrote: My second problem is how to handle the short write? I have to send in the tape, but the are 3-4 GB of data on this tape. Without this data, my backup is inconsistent. The only possibility I see (at the moment) is doing a full backup of the partitions having some data on this tape. That's one possibility. You can use amadmin force, staging the full backups over a few runs if necessary to fit them in. Another possibility would be to wait a tapecycle (or at the very least a dumpcycle) for the backups to expire on their own. (Don't forget to erase the tape before sending it back, if it contains anything confidential.) I plan to use an new tape that's tested and dump the data from the damaged tape to the new one. This way I still have the incremental backups and don't lose a tape in my tape order. I have to admit that my paranoia urged me to do full backups on the affected partitions. ;) Thanks for your advice, Martin Öhler
Re: Testing tapes before use / bad tape
On Tue, Nov 25, 2003 at 10:46:10AM +0100, Martin Oehler wrote: Hi! Am Mo, 2003-11-24 um 13.54 schrieb Gene Heskett: On Monday 24 November 2003 03:46, Martin Oehler wrote: [...] Doing this to a tape also should tell you whether or not the drives internal compression is in use, which for amanda, should be turned off as that hides the true size of the tape from amanda. I need to work with hardware compression because I have 6 hours at night to execute my backup before people are starting to change files. With software compression, backup doesn't finish in time because the compression itself takes to long (I think an amanda mode that first moves the data completely to the backup server and then starting to compress, freeing the backed up boxes, would be a good solution. The online compresssion eats a lot of time). Amanda counts bytes sent to the drive after any gzip is applied, and if the drives compressor is on, the data normally will grow slightly and amanda may hit EOT thinking it still has 10-15% of the tape left. The Quantum drive hardware compression is completely manageable via the mt command. I'm now turning the compression off before starting the tape test and turn it back on for backup. When I'm finished with tuning the amtapetype test I make this comment based on your using hardware compression for your amdumps/amflushes. If you are using amtapetype for its original purpose, determining the capacity of the tape, the effort of running amtapetype is not really needed. The vendor can give you the native capacity of the drive/format and amtaptype generally comes within a few percent of this. The vendor will also give a capacity with compression, typically 2X to 2.6X the native capacity. The actual compression you realize during tape writes depends on the nature of your data. So the actual capacity, for your data, is an unknown and has to be guessed when creating a tapetype definition in amanda.conf. You do know the upper and lower bounds however, native capacity and advertised capacity with compression. Neither of these values needs amtapetype to determine. So an amtapetype run is unneeded to create your tapetype definition. I want to use this inside my backup script, so each tape is tested before amflush starts (I'm using amanda with holding disks). What does that mean? That you run amdump with no tape in the drive, forcing the entire dump to the holding disk? Then at some later time you do an amflush? I ask as I too am using a holding disk (big enough for about 2 wks of dumps) but I seldom run amflush. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Testing tapes before use / bad tape
Hi! Am So, 2003-11-23 um 14.16 schrieb Gene Heskett: There is amtapetype, which will destructively write the tape till it hits EOT, and will tell you the size it found. See the man page for running options to help speed it up as its quite slow, doing 2 passes. SYNOPSIS amtapetype [-h] [-c] [-b blocksize] [-e estsize] [-f tapedev] [-t typename] Hmm, the only option that sounds like it could speed up the process is blocksize. Does anyone know a good value for this? Is the test faster the bigger I choose this value? (so 1 would take about 1 GB) Thanks in advance, Martin Öhler
Re: Testing tapes before use / bad tape
Martin Oehler wrote: Hi! Am So, 2003-11-23 um 14.16 schrieb Gene Heskett: There is amtapetype, which will destructively write the tape till it hits EOT, and will tell you the size it found. See the man page for running options to help speed it up as its quite slow, doing 2 passes. SYNOPSIS amtapetype [-h] [-c] [-b blocksize] [-e estsize] [-f tapedev] [-t typename] Hmm, the only option that sounds like it could speed up the process is blocksize. Does anyone know a good value for this? Read on a few lines further in the man page. The most important parameter for speed is the estimated size. It's the stop/start when writing a filemark that slows down the most. The default estimated size is 1 Gbyte. Amtapetype will write 100+200 files to the tape only if the estimated size is more or less ok. If you insert a a tape with 200 Gbyte real capacity, amtapetype will actually write 2+6 files to the tape. If start/stop takes about a second, then there will be 8 seconds lost in writing filemarks only, instead of the expected 300 seconds. Is the test faster the bigger I choose this value? (so 1 would take about 1 GB) 32 Kbyte (the amanda default) is just fine. -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Testing tapes before use / bad tape
On Monday 24 November 2003 03:46, Martin Oehler wrote: Hi! Am So, 2003-11-23 um 14.16 schrieb Gene Heskett: There is amtapetype, which will destructively write the tape till it hits EOT, and will tell you the size it found. See the man page for running options to help speed it up as its quite slow, doing 2 passes. SYNOPSIS amtapetype [-h] [-c] [-b blocksize] [-e estsize] [-f tapedev] [-t typename] Hmm, the only option that sounds like it could speed up the process is blocksize. Does anyone know a good value for this? I would use that only if you are using a non-std blocksize, which I am, rather than the default 512 byte, I'm using 32768 for reasons other than any percieved speed advantage. There isn't any as long as the drive is streaming. But changing this is an mt command I think, done ahead of time. I do mine in /etc/rc.d/rc.local at boot time. However, there must be a tape in the drive or that fails later for amanda. Is the test faster the bigger I choose this value? (so 1 would take about 1 GB) The -e size option allows it to march right along on the first pass, so if you have an 80G tape (see the options defines for estsize syntax) you can pass this to it. We're told that speeds up the first pass considerably. Doing this to a tape also should tell you whether or not the drives internal compression is in use, which for amanda, should be turned off as that hides the true size of the tape from amanda. Amanda counts bytes sent to the drive after any gzip is applied, and if the drives compressor is on, the data normally will grow slightly and amanda may hit EOT thinking it still has 10-15% of the tape left. Turning off the drives internal compressor can be a problem child too, and must be done external to amanda. However I've worked out a script that saves the tapes label info, turns the compressor off, and re-writes the label. The programming switches on the drive must also be set to the off position before doing this, and on some drives, you will have to powercycle the drive to get it to re-read the switches state. Since the switches aren't that accessable normally, the machine powerdown done to remove and access the drive will take care of that. Thanks in advance, Martin Öhler Humm, do I recall that name from a decade or so back when I was running an amiga? -- Cheers, Gene AMD [EMAIL PROTECTED] 320M [EMAIL PROTECTED] 512M 99.27% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are: Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: Testing tapes before use / bad tape
On Mon, Nov 24, 2003 at 09:46:31AM +0100, Martin Oehler wrote: Hi! Am So, 2003-11-23 um 14.16 schrieb Gene Heskett: There is amtapetype, which will destructively write the tape till it hits EOT, and will tell you the size it found. See the man page for running options to help speed it up as its quite slow, doing 2 passes. Check your tapes capacity and rated writing speed. Be sure they are the native values, not the values for HW compression on. Then divide one by the other. You come out with a time value that I'm betting will be about 3 hours. This is the time for a single pass under the vendor's optimum conditions. You will probably not reach their rated speed. As amtapetype makes two complete passes, double that time value. This is the minimum time an amtapetype run could take. It is not amtapetype that is slow, it is the medium. As Paul B. notes, make sure you give amtapetype a reasonable estimate of the native size of the tape. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
amtapetype idea (Was: Testing tapes before use / bad tape)
On Mon, Nov 24, 2003 at 10:23:12AM +0100, Paul Bijnens wrote:
Read on a few lines further in the man page. The most important
parameter for speed is the estimated size. It's the stop/start when
writing a filemark that slows down the most. The default estimated
size is 1 Gbyte. Amtapetype will write 100+200 files to the tape only
if the estimated size is more or less ok. If you insert a a tape
with 200 Gbyte real capacity, amtapetype will actually write 2+6
files to the tape. If start/stop takes about a second, then there
will be 8 seconds lost in writing filemarks only, instead of
the expected 300 seconds.
Might it be a good idea to have amtapetype note too many files
being written during the first phase and taking some action?
Maybe with an option to override the checking.
Possible actions:
- abort with an error message to rerun with a different estimate
when the number of files written exceeded some value, say 250
- print an error message recommending the operator interupt the
and rerun with a corrected estimate, eg.
amtapetype: pass 1: estimated tape capacity (X GB) error: expected
to write 100 files, now writing file Y00: restart recommended
In your example above (200GB tape with default estimate) I
think the error message would come out about every minute or
two. For a dds3 tape, about every 20 min.
Aside from the new option processing to set an override flag)
I think all it would take is an if statement like:
if (files_written % 100 files_written 100 ! overried_flag)
{
print/log errors
}
jl
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amtapetype idea (Was: Testing tapes before use / bad tape)
Jon LaBadie wrote: Might it be a good idea to have amtapetype note too many files being written during the first phase and taking some action? Maybe with an option to override the checking. My idea was to write only one large file in the first pass, just until it hits end of tape. Then rewind and write tapesize/100 files to measure the size of a filemark. That way you don't need to give an estimated tapesize at all. Never got enough time to implement it though: Wife, children, house,... the full catastrophy -- Zorba The Greek. You ony have to look out for overflow while doing the math for blocksize, speed etc. (200 Gbyte 2^32 bytes) -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amtapetype idea (Was: Testing tapes before use / bad tape)
On Mon, Nov 24, 2003 at 07:14:56PM +0100, Paul Bijnens wrote: Jon LaBadie wrote: Might it be a good idea to have amtapetype note too many files being written during the first phase and taking some action? Maybe with an option to override the checking. My idea was to write only one large file in the first pass, just until it hits end of tape. Then rewind and write tapesize/100 files to measure the size of a filemark. That way you don't need to give an estimated tapesize at all. Never got enough time to implement it though: Just revert to an earlier version :) Way back tapetype wrote pass 1 with no files. Then in pass 2 it wrote lots of small files (one 32K block) comes to mind. That of course was rediculous when V large tape sizes came along. I submitted a change that did a similar calculation to that you suggest, but wrote 1000 files. After that was in place for a while, someone noted a problem. What it was I don't recall, but someone, I think JRJ, changed it to the 100/200 file scheme. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Testing tapes before use / bad tape
On Mon, Nov 24, 2003 at 09:46:31AM +0100, Martin Oehler wrote: Hmm, the only option that sounds like it could speed up the [amtapetype] process is blocksize. Does anyone know a good value for this? The same value as amdump will be using! With some tape technologies, the tape's capacity depends very much on the block size. In such a case, using a different block size for the test would give misleading results. On Sun, Nov 23, 2003 at 10:28:38AM +0100, Martin Oehler wrote: My second problem is how to handle the short write? I have to send in the tape, but the are 3-4 GB of data on this tape. Without this data, my backup is inconsistent. The only possibility I see (at the moment) is doing a full backup of the partitions having some data on this tape. That's one possibility. You can use amadmin force, staging the full backups over a few runs if necessary to fit them in. Another possibility would be to wait a tapecycle (or at the very least a dumpcycle) for the backups to expire on their own. (Don't forget to erase the tape before sending it back, if it contains anything confidential.) -- | | /\ |-_|/ Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Re: amtapetype idea (Was: Testing tapes before use / bad tape)
On Mon, Nov 24, 2003 at 07:14:56PM +0100, Paul Bijnens wrote: My idea was to write only one large file in the first pass, just until [amtapetype] hits end of tape. One problem with that is that the drive's internal buffering might distort the results, by letting amtapetype think it has successfully written blocks that in fact won't make it to tape. (That's a problem anyway, of course, but sticking in a filemark every once in a while puts a known upper bound on the error.) Perhaps amtapetype could have a test-tape flag, that would basically tell it to suppress the second pass. Or the second pass could become a verification pass (just re-seed the random-number generator to the value from the beginning of the write pass). Or provide both options. Of course that would make amtapetype a rather misleading name. amtape would be a great choice for a new name; too bad it's taken :-/ -- | | /\ |-_|/ Eric Siegerman, Toronto, Ont.[EMAIL PROTECTED] | | / It must be said that they would have sounded better if the singer wouldn't throw his fellow band members to the ground and toss the drum kit around during songs. - Patrick Lenneau
Testing tapes before use / bad tape
Hi! I'm using amanda 2.4.4p1 in combination with a Quantum SDLT 320 tape drive. This week I received some replacement tapes for broken ones. Using the first tape I got some kind of short write while using amflush: [...] *** A TAPE ERROR OCCURRED: [[writing file: Input/output error]]. [...] xxx /home lev 6 FAILED [out of tape] [...] taper: tape yyy3 kb 5972448 fm 14 writing file: Input/output error The system wrote about 3 GB on a 160/320 GB tape. The amflush succeeded with one of the older tapes (I flushed the data to the next tape yyy4). Before having this problems again, how can I test a tape before use? I thought about a second config doing full-backups and using amverify to check the tape after the backup. But that's not using the complete tape, so there could be an error near the end of the tape an I wouldn't recognize. Is there a test tool available running under linux/unix? Would be nice if one could have a kind of report to print (tape error at position xxx GB or something) for warranty reasons. My second problem is how to handle the short write? I have to send in the tape, but the are 3-4 GB of data on this tape. Without this data, my backup is inconsistent. The only possibility I see (at the moment) is doing a full backup of the partitions having some data on this tape. Regards, Martin Öhler
Re: Testing tapes before use / bad tape
On Sunday 23 November 2003 04:28, Martin Oehler wrote: Hi! I'm using amanda 2.4.4p1 in combination with a Quantum SDLT 320 tape drive. This week I received some replacement tapes for broken ones. Using the first tape I got some kind of short write while using amflush: [...] *** A TAPE ERROR OCCURRED: [[writing file: Input/output error]]. [...] xxx /home lev 6 FAILED [out of tape] [...] taper: tape yyy3 kb 5972448 fm 14 writing file: Input/output error The system wrote about 3 GB on a 160/320 GB tape. The amflush succeeded with one of the older tapes (I flushed the data to the next tape yyy4). Before having this problems again, how can I test a tape before use? I thought about a second config doing full-backups and using amverify to check the tape after the backup. But that's not using the complete tape, so there could be an error near the end of the tape an I wouldn't recognize. Is there a test tool available running under linux/unix? Would be nice if one could have a kind of report to print (tape error at position xxx GB or something) for warranty reasons. My second problem is how to handle the short write? I have to send in the tape, but the are 3-4 GB of data on this tape. Without this data, my backup is inconsistent. The only possibility I see (at the moment) is doing a full backup of the partitions having some data on this tape. Regards, Martin Öhler There is amtapetype, which will destructively write the tape till it hits EOT, and will tell you the size it found. See the man page for running options to help speed it up as its quite slow, doing 2 passes. -- Cheers, Gene AMD [EMAIL PROTECTED] 320M [EMAIL PROTECTED] 512M 99.27% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are: Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: Testing tapes
I think I'm getting closer to the problem. First off, it seems that ide-tape doesn't work properly with the Colorado drives. I haven't yet had time to test, but I'll install ide-scsi when I next get a chance. Does anyone know of a reference as to the differences between ide-tape/ide and st/ide-scsi/ide? Regards, Brian Jonnes -- Init Systems - Linux consulting 031 767-0139082 769-2320[EMAIL PROTECTED]
RE: Testing tapes
Just make a backup of a few large files, recover them into a temporary directory, and compare the recoverds with the originals using cmp -l. Michael Martinez System Administrator Information Systems and Technology Management CSREES - United States Department of Agriculture (202) 720-6223 -Original Message- From: Dietmar Goldbeck [mailto:[EMAIL PROTECTED]] Sent: Saturday, August 31, 2002 10:20 AM To: Brian Jonnes Cc: [EMAIL PROTECTED] Subject: Re: Testing tapes On Sat, Aug 31, 2002 at 01:45:44PM +0200, Brian Jonnes wrote: Hi, I think I have a faulty tape or two. What is the best recommended way of testing this? Generate a file from /dev/urandom, write it to the tape and md5sum it? If you use software compression with Amanda, one easy way would be amrecover -c /dev/tape gzip -tv *.? This checks the internal gzip CRC on all files. You could also run Amanda without tape, compute the md5sum of all files on the holding disk, flush them and then recover and compare md5. I personally would not just write one large file, because the tape might have some special problem ocurring with Amanda (writing lots of file marks, writing 32k blocks etc.) I always used Amanda itself and checked _all_ gzip CRC. When testing a new Amanda server/new tape drive i also do some real restores on to some spare disk and compare m5sums of the original tree and the restored tree. Also, what is the expected lifetime for Travan 4 tapes (each used once a week)? When should I retire them? I don't have experience with travan. My experience with several DDS and DLT tape drives (and hundreds of tapes) suggests that problems with more than _only_ _one_ tape are probably problems of the drive :-(( -- Alles Gute / best wishes Dietmar GoldbeckE-Mail: [EMAIL PROTECTED] Reporter (to Mahatma Gandhi): Mr Gandhi, what do you think of Western Civilization? Gandhi: I think it would be a good idea.
Re: Testing tapes
On Sun 01 Sep 02 20:51, Gene Heskett wrote: Thats just a wee bit more than my tapetype says it is here, I got 3780mb as the capacity. Odd. What version of amanda? There was a period of a couple of weeks where the internal rewind function was a bit foobar. The tape might not have been rewound, but then again, thats wrong cause it had to rewind it to verify it was the right tape, which it apparently did. Anyway, the current snapshot is 2.4.3b4-20020829 and all *known* bugs have been squished. Hrm. Running Debian Woody's version (2.4.2p2). Also, what scsi card? Adaptec 1542's are getting a bad rep in this group. They do not do tape happily. It be an IDE drive. I've been getting mine on ebay, and allthough the price has risen somewhat, you should be able to get a couple of 10 packs for $35-$40 a 10-pack. Plus the inevitable shipping of course. Dunno if the shipping is gonna kill it for me, being in SA ;) Regards, Brian Jonnes -- Init Systems - Linux consulting 031 767-0139082 769-2320[EMAIL PROTECTED]
Re: Testing tapes
On Monday 02 September 2002 02:24, Brian Jonnes wrote: On Sun 01 Sep 02 20:51, Gene Heskett wrote: Thats just a wee bit more than my tapetype says it is here, I got 3780mb as the capacity. Odd. No, not really. It just demonstrates that the marketing weenies tend to round things off to the next higher gigabyte in their propaganda releases. I'd use whatever 'tapetype' spits out for your drive and tape. What version of amanda? There was a period of a couple of weeks where the internal rewind function was a bit foobar. The tape might not have been rewound, but then again, thats wrong cause it had to rewind it to verify it was the right tape, which it apparently did. Anyway, the current snapshot is 2.4.3b4-20020829 and all *known* bugs have been squished. Hrm. Running Debian Woody's version (2.4.2p2). Quite a bit has been fixed in 2.4.3b4. Use the same configuration as for 2.4.2p2 and it should drop right in on top of the old install, I do that here as soon as the next one is out. That might indeed fix this problem. Its clean code, even builds with gcc-3.2 :-) [...] It be an IDE drive. Humm, that I've no experience with at all, but I'd assume you have to use the 'ide-scsi' interface in order to talk to it since the ide interface was never intended to be used for sequential block devices. Atapi extends this. I use ide-scsi here, but to talk to my cdrom/cdrw drive. I've been getting mine on ebay, and allthough the price has risen somewhat, you should be able to get a couple of 10 packs for $35-$40 a 10-pack. Plus the inevitable shipping of course. Dunno if the shipping is gonna kill it for me, being in SA ;) Ah, yeah there is that. But still, even if it was another 40 USD to get it there, thats still cheaper by quite a bit than TR-4's are in this neck of the woods. However I've heard some stories about merchandise being held for ransom by the local agents too in some areas of your hemisphere. Is that a problem in your area? -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.13% setiathome rank, not too shabby for a WV hillbilly
Re: Testing tapes
On Sat 31 Aug 02 16:11, Gene Heskett wrote:
On Saturday 31 August 2002 07:45, Brian Jonnes wrote:
Hi,
I think I have a faulty tape or two. What is the best recommended
way of testing this? Generate a file from /dev/urandom, write it
to the tape and md5sum it?
Most drives do the equ of that themselves. its when they can't
recover the error that you become aware that its doing re-read
after re-read and reporting a failure.
Okay, here's the deal. I'll use my DAILY_04 tape as an example. Dump on 13/08:
---
These dumps were to tape DAILY_04.
The next tape Amanda expects to use is: DAILY_05.
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:01
Run Time (hrs:min) 2:46
Dump Time (hrs:min)1:46 1:20 0:25
Output Size (meg)3940.3 2940.1 1000.2
Original Size (meg) 6798.4 5010.1 1788.3
Avg Compressed Size (%)58.0 58.7 55.9 (level:#disks ...)
Filesystems Dumped 27 13 14 (1:8 2:4 3:2)
Avg Dump Rate (k/s) 637.2 626.3 671.6
---
As you can see, the tape was used to the full. Now, on 31/08 I had a
failure:
---
These dumps were to tape DAILY_04.
*** A TAPE ERROR OCCURRED: [[writing filemark: Input/output error]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: DAILY_05.
FAILURE AND STRANGE DUMP SUMMARY:
valhalla /export/mail/pmail.usr lev 0 FAILED [disk /export/mail/pmail.usr
offline on valhalla?]
valhalla /export/mail/subs lev 0 FAILED [out of tape]
---
Now from this I'd assume it was trying to write too much data. But looking at
the log files, it seems that it only wrote a couple of kb (unfortunately I
can't access the server right at the mo').
Where am I going wrong? Are the logs straightforward to interpret (i.e. I add
up the kb for the DUMPER lines up until the point of failure)? Can tapes be
intermittent?
failed, one of them ripping the tapes in two violently, in less
than a year. So generally speaking, TR4's aren't any more, or less
dependable than a DDS2, but the DDSx format is slowly pulling ahead
in my personal tally sheets. Plus the DDS2 tapes are beaucoup
cheaper...
I have to get a new (and bigger) drive in the near future, so I'll definitely
look into these.
Regards,
Brian Jonnes
--
Init Systems - Linux consulting
031 767-0139082 769-2320[EMAIL PROTECTED]
Re: Testing tapes
On Sunday 01 September 2002 10:09, Brian Jonnes wrote:
On Sat 31 Aug 02 16:11, Gene Heskett wrote:
On Saturday 31 August 2002 07:45, Brian Jonnes wrote:
Hi,
I think I have a faulty tape or two. What is the best
recommended way of testing this? Generate a file from
/dev/urandom, write it to the tape and md5sum it?
Most drives do the equ of that themselves. its when they can't
recover the error that you become aware that its doing re-read
after re-read and reporting a failure.
Okay, here's the deal. I'll use my DAILY_04 tape as an example.
Dump on 13/08:
---
These dumps were to tape DAILY_04.
The next tape Amanda expects to use is: DAILY_05.
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:01
Run Time (hrs:min) 2:46
Dump Time (hrs:min)1:46 1:20 0:25
Output Size (meg)3940.3 2940.1 1000.2
Original Size (meg) 6798.4 5010.1 1788.3
Avg Compressed Size (%)58.0 58.7 55.9
(level:#disks ...) Filesystems Dumped 27 13
14 (1:8 2:4 3:2) Avg Dump Rate (k/s) 637.2 626.3
671.6
---
Thats just a wee bit more than my tapetype says it is here, I got
3780mb as the capacity.
As you can see, the tape was used to the full. Now, on 31/08 I had
a failure:
---
These dumps were to tape DAILY_04.
*** A TAPE ERROR OCCURRED: [[writing filemark: Input/output
error]]. Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: DAILY_05.
FAILURE AND STRANGE DUMP SUMMARY:
valhalla /export/mail/pmail.usr lev 0 FAILED [disk
/export/mail/pmail.usr offline on valhalla?]
valhalla /export/mail/subs lev 0 FAILED [out of tape]
---
Now from this I'd assume it was trying to write too much data. But
looking at the log files, it seems that it only wrote a couple of
kb (unfortunately I can't access the server right at the mo').
Where am I going wrong? Are the logs straightforward to interpret
(i.e. I add up the kb for the DUMPER lines up until the point of
failure)? Can tapes be intermittent?
What version of amanda? There was a period of a couple of weeks
where the internal rewind function was a bit foobar. The tape
might not have been rewound, but then again, thats wrong cause it
had to rewind it to verify it was the right tape, which it
apparently did. Anyway, the current snapshot is 2.4.3b4-20020829
and all *known* bugs have been squished.
Also, what scsi card? Adaptec 1542's are getting a bad rep in this
group. They do not do tape happily.
failed, one of them ripping the tapes in two violently, in less
than a year. So generally speaking, TR4's aren't any more, or
less dependable than a DDS2, but the DDSx format is slowly
pulling ahead in my personal tally sheets. Plus the DDS2 tapes
are beaucoup cheaper...
I've been getting mine on ebay, and allthough the price has risen
somewhat, you should be able to get a couple of 10 packs for
$35-$40 a 10-pack. Plus the inevitable shipping of course.
In any event $4 for a DDS2 sure beats nearly $50 for a pair of
TR4's.
I have to get a new (and bigger) drive in the near future, so I'll
definitely look into these.
Regards,
Brian Jonnes
--
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz 512M
99.13% setiathome rank, not too shabby for a WV hillbilly
Testing tapes
Hi, I think I have a faulty tape or two. What is the best recommended way of testing this? Generate a file from /dev/urandom, write it to the tape and md5sum it? Also, what is the expected lifetime for Travan 4 tapes (each used once a week)? When should I retire them? ..Brian -- Init Systems - Linux consulting 031 767-0139082 769-2320[EMAIL PROTECTED]
Re: Testing tapes
On Saturday 31 August 2002 07:45, Brian Jonnes wrote: Hi, I think I have a faulty tape or two. What is the best recommended way of testing this? Generate a file from /dev/urandom, write it to the tape and md5sum it? Most drives do the equ of that themselves. its when they can't recover the error that you become aware that its doing re-read after re-read and reporting a failure. Also, what is the expected lifetime for Travan 4 tapes (each used once a week)? When should I retire them? My experience with TR4's is that they will usually outlast the drive. Thats not saying much, but we did have one in an NT system that probably accumulated in excess of 1500 passes (never changed the tape) when we retired the box it was in. 2 other TR4 drives failed, one of them ripping the tapes in two violently, in less than a year. So generally speaking, TR4's aren't any more, or less dependable than a DDS2, but the DDSx format is slowly pulling ahead in my personal tally sheets. Plus the DDS2 tapes are beaucoup cheaper... -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.13% setiathome rank, not too shabby for a WV hillbilly
Re: Testing tapes
On Sat, Aug 31, 2002 at 01:45:44PM +0200, Brian Jonnes wrote: Hi, I think I have a faulty tape or two. What is the best recommended way of testing this? Generate a file from /dev/urandom, write it to the tape and md5sum it? If you use software compression with Amanda, one easy way would be amrecover -c /dev/tape gzip -tv *.? This checks the internal gzip CRC on all files. You could also run Amanda without tape, compute the md5sum of all files on the holding disk, flush them and then recover and compare md5. I personally would not just write one large file, because the tape might have some special problem ocurring with Amanda (writing lots of file marks, writing 32k blocks etc.) I always used Amanda itself and checked _all_ gzip CRC. When testing a new Amanda server/new tape drive i also do some real restores on to some spare disk and compare m5sums of the original tree and the restored tree. Also, what is the expected lifetime for Travan 4 tapes (each used once a week)? When should I retire them? I don't have experience with travan. My experience with several DDS and DLT tape drives (and hundreds of tapes) suggests that problems with more than _only_ _one_ tape are probably problems of the drive :-(( -- Alles Gute / best wishes Dietmar GoldbeckE-Mail: [EMAIL PROTECTED] Reporter (to Mahatma Gandhi): Mr Gandhi, what do you think of Western Civilization? Gandhi: I think it would be a good idea.
