If estimated disk size tape size
will amanda tape out the data that cant fit onto that 40gig tape and then onto the next tape OR will it start over again from the beginning ( as the docs currently say ) and try to fit something that will never fit onto that tape?. If docs are correct, why does amanda bother bother attempting to write any partition that has more data than the tape than the tape can hold? should'nt there be a sorry can't do that error? /gat
Re: [amanda@gorn.americom.com: AmeriCom AMANDA MAIL REPORT FOR April 9, 2002]
i guess your next step would be the more detailed logs. How about reviewing the 'amdump.[0-9]' files in the 'logdir' directory defined in amanda.conf ? [EMAIL PROTECTED] wrote: Thanks for your reply... However, the Ecrix V17 tapesize is set to 35000 mbytes and... Filesystem Size Used Avail Use% Mounted on solo.americom.com:/ is: /dev/sda3 7.9G 6.9G 632M 92% / gorn.americom.com:/ is: /dev/sda4 5.1G 4.3G 547M 89% / The other MISSING filesystems are also easily under 35G. Is amanda
Re: Linux DUMP
does this mean that there was a definitive conclusion? Christoph Scheeder wrote: Please not again this discussion...
Re: Linux DUMP
Sorry, thats a general conclusion to most things in life. Is there a situation(s) where DUMP can fail. If yes, why are there no warning labels ( ie the probability of failure is 1 in 1billion ). If NO, than can I see the proof that absolutely refutes Mr. Torvolds statement. /gat Its interesting that I was unaware of this dilema ( the possible failure of DUMP ) until it was posted on this list. Maybe others, as they post DUMP v. TAR inquiries should also be made aware of this possible scenario. I'm also reasonably quite sure that most parents would not contemplate placing small children ( as well as small adults ) in front seats with air-bags - now-adays, even though testing proved that air-bags are safe, and a proven safety feature. Joshua Baker-LePain wrote: does this mean that there was a definitive conclusion? Yup -- use what you are comfortable with and what your testing proves works.
Re: Any automagic exclusion of filesystems?
Jens Rohde wrote: Hi I suppose that the disk/fs that will/might be restored to, that is to be recognized by SUN, will be done on a machine that can create the (unofficial) sun partition correctly? I'm changing the dump-method on some of my filesystems from ufsdump to gtar (so I can restore these filesystems on non-Sun hardware/OS).
Re: Linux DUMP
Ya, but didnt someone post that DUMP on linux can fail - if the conditions are right? I think is was suggested that SMP systems can demonstrate the failure sooner. I think that Mr. Torvolds ( sorry is i mis-spelled) made that comment or conclusion. Are there some caveats that need to be added here ? /gat Bernhard R. Erdmann wrote: Which backup program is best? dump, says some people. Elizabeth D. Zwicky torture tested lots of backup programs. The clear choice for preserving all your data and all the peculiarities of Unix filesystems is dump, she stated. Elizabeth created filesystems containing a large variety of unusual conditions (and some not so unusual ones) and tested each program by do a backup and restore of that filesystems. The peculiarities included: files with holes, files with holes and a block of nulls, files with funny characters in their names, unreadable and unwriteable files, devices, files that change size during the backup, files that are created/deleted during the backup and more. She presented the results at LISA V in Oct. 1991. This article is archived here: http://berdmann.dyndns.org/doc/dump/zwicky/testdump.doc.html
Re: Estimates - 7 hour 50mins
I presume u are using tar?! From My experience, 17gigs of a (few) large files does not take a long time. On the other hand, backing up a large amount of files in a (single) directory takes a long time. The orig tar that came with this (linux) sys readily reached 80% cpu usage. The later tar fixed that, so now it still takes a long time ( but mainly it just waits for all the disk/directory activity ?! ) So whats the culprit? Since there are 2 phases ( size estimate, and then data transfer ) i suspect that each part takes about 4 hours.To find out, you can do a time tar cf - .. just like what the sendsize program would do on your system. just to See how long that takes. Then how fast is your tape drive? a 5.0mb/sec tape drive would take 56 min just to back up 17gig. This presumes that the tape is screaming ( i ment streaming ) . a df -i might help in determing how many files might be out on the system. David Flood wrote: We kick off our backup at 10pm at the moment we are only backing up 4 seperate directorys i.e. 4 seperate disklist entries. These are estimated by amanda to be 17.4GB. The estimates are taking just short of 8 hours to complete which is unacceptable. This is after making every dump a full dump in an attempt to cut down time with doing estimates for level 1 2. To force it to do full dumps I have did the following in amanda.conf: dumpcycle 0 strategey noinc skip-incr yes I'm running: amanda 2.4.3b3 Solaris 7 Using tar I'm restore onto a 35/70 DLT with software compression. The 4 directorys mentioned above are all on the tapeserver and those are the only thing the tapeserver is backing up. Does anyone have any ideas, because the estimates are taking so long the backups are still running when the users come in - in the morning. This means the backups I'm getting if I get them are 'dirty' not to mention it slows the system down as backups and users are trying to access the same data. Thanks in advance David Flood Systems Administrator
Re: dos partion
u probably want to back it up with tar - i suppose. There is not enough info, but i could guess ur using dump. /gat Tom Beer wrote: Hi, I want to back a dos partition on a dual boot machine. I mount this parition /mnt/dos and the disklist entry is vaio.system ad0s1 comp-user sendsize.debug states bad sblock number entiry dump terminated. Any pointers? Thanks Tom
data timeout on sendbackup phase
with a dtimeout of 1800 ( seconds, or 30 min ) the 'sendbackup' of
/mnt/hdg3 fails with a data timeout.
The Client side gets a:
=
ckup: argument list: gtar --create --file - --directory /mnt/hdg3
--one-file-system --listed-incremental
/usr/local/var/amanda/gnutar-lists/lx_mnt_hdg3_0.new --sparse
--ignore-failed-read --totals --exclude-from
/usr/local/etc/amanda/gtar/exclude.gtar .
sendbackup-gnutar: /usr/local/libexec/runtar: pid 1006
sendbackup: started index creator: /bin/gtar -tf - 2/dev/null | sed -e
's/^\.//'
index tee cannot write [Broken pipe]
sendbackup: pid 1005 finish time Mon Apr 15 13:21:25 2002
==
The server gets a :
==
driver: result time 9669.653 from taper: PORT 1032
driver: send-cmd time 9669.653 to dumper0: PORT-DUMP 01-4 1032 lx
/mnt/hdg3 0 1970:1:1:0:0:0 GNUTAR
|;bsd-auth;index;exclude-list=/usr/local/etc/amanda/gtar/exclude.gtar;
driver: state time 9669.653 free kps: 2239 space: 0 taper: writing
idle-dumpers: 3 qlen tapeq: 0 runq: 0 roomq: 0 wakeup: 86400
driver-idle: not-idle
driver: interface-state time 9669.653 if : free 839 if LE0: free 400 if
LOCAL: free 1000
driver: hdisk-state time 9669.653 hdisk 0: free 0 dumpers 0 hdisk 1:
free 0 dumpers 0
dumper: stream_client: connected to 127.0.0.1.1032
dumper: stream_client: our side is 0.0.0.0.1033
dumper: try_socksize: send buffer size is 65536
taper: stream_accept: connection from 127.0.0.1.1033
taper: try_socksize: receive buffer size is 32768
dumper: stream_client: connected to 192.168.0.10.32772
dumper: stream_client: our side is 0.0.0.0.1034
dumper: stream_client: connected to 192.168.0.10.32773
dumper: stream_client: our side is 0.0.0.0.1035
dumper: stream_client: connected to 192.168.0.10.32774
dumper: stream_client: our side is 0.0.0.0.1036
driver: result time 11470.452 from dumper0: FAILED 01-4 [data
timeout]
dumper: kill index command
taper: reader-side: got label AmandaFullBackup22 filenum 2
driver: result time 11470.454 from taper: DONE 00-3
AmandaFullBackup22 2 [sec 1800.798 kb 32 kps 0.0 {wr: writers 1 rdwait
1798.614 wrwait 0.000 filemark 2.183}]
On the side, can you tell me who, or what is fed into this script?. If
its the tar'd data, then there is a potential problem. Some of the files
are in the gigabyte range. Although at 100mbps ethernet can take some
min, but taping it out at 1.5MBps may take a lot longer that 30min to
get another 'index' name, or even a buffer of names. Is this where the
'data timeout' is happening ?
/gat
Re: [amanda@gorn.americom.com: AmeriCom AMANDA MAIL REPORT FOR April 9, 2002]
i guess your next step would be the more detailed logs. How about reviewing the 'amdump.[0-9]' files in the 'logdir' directory defined in amanda.conf ? [EMAIL PROTECTED] wrote: Thanks for your reply... However, the Ecrix V17 tapesize is set to 35000 mbytes and... Filesystem Size Used Avail Use% Mounted on solo.americom.com:/ is: /dev/sda3 7.9G 6.9G 632M 92% / gorn.americom.com:/ is: /dev/sda4 5.1G 4.3G 547M 89% / The other MISSING filesystems are also easily under 35G. Is amanda
Re: Only 3 tapes in 7 tape changer gets scaned twice
It appears that the offline_before_unload switch/feature does not work in this script! Am i :-/ /gat John R. Jackson wrote: I think the following patch fixes that version. Also, there is a new copy of the whole thing at: ftp://gandalf.cc.purdue.edu/pub/amanda/chg-zd-mtx.sh.in-243 Also, could you send your chg-zd-mtx config file? I think I understand
Re: Only 3 tapes in 7 tape changer gets scaned twice
This is far short of a million, even if its binary, questions! actually plugged it in, and linked it to ...sh.in rebuilt it ( make, not configure ) , but no install. Just cp'd the chg-zd-mtx to /usr/local/libexec. I know its there bec the old version is ~17k, and newer one is ~37k it does not work bec it does not (apparently) do a mt rewoffl before changing to the next tape. I can manually do a mt rewoffl, and then amtape confname slot 2 will work. firstslot=1 lastslot=7 cleanslot=9 havereader=0 offlinestatus=1 AUTOCLEAN=0 autocleancount=0 offline_before_unload=1 John R. Jackson wrote: It appears that the offline_before_unload switch/feature does not work in this script! ... Insufficient details, so you get a million questions in response: Did you drop this file in place of your existing chg-zd-mtx.sh.in file (saving the original in case there is a problem)? Did you rerun ./configure, then make then make install? What did you put in your changer config file? What is in the changer log file? Why do you think it's not working? /gat John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
Re: Only 3 tapes in 7 tape changer gets scaned twice
Interesting, the log file shows that offline_before_unload is set to 0
:-{
Storage Element 5:Full
Storage Element 6:Empty
Storage Element 7:Full
12:18:14 SLOTLIST - firstslot set to 1
12:18:14 SLOTLIST - lastslot set to 7
12:18:14 SLOTLIST - 1 2 3 4 5 6 7
12:18:15 Config info:
firstslot = 1
lastslot = 7
cleanslot = -1
cleancycle = -1
offline_before_unload = 0
unloadpause = 0
autoclean = 0
autocleancount = 99
havereader = 0
driveslot = 0
poll_drive_ready = 3
max_drive_wait = 120
John R. Jackson wrote:
It appears that the offline_before_unload switch/feature does not work
in this script! ...
What is in the changer log file?
Re: Only 3 tapes in 7 tape changer gets scaned twice
There are still some bug-a-boo's. non-amanda tape in slot 5, and loaded in tape drive. no tape in slot 6. There is an amanda tape in slot 7 issue an amcheck confname get -- [Amanda@kodak Amanda]$ amtape confname slot 5 amtape: changed to slot 5 on /dev/nst0 [Amanda@kodak Amanda]$ amcheck confname Amanda Tape Server Host Check - ERROR: holding disk /mnt/hdd4/amanda: statfs: No such file or directory ERROR: holding disk /mnt/sda1/amanda: statfs: No such file or directory amcheck-server: slot 5: not an amanda tape amcheck-server: fatal slot 6: source Element Address 8 is Empty ERROR: label AmandaFullBackup22 or new tape not found in rack (expecting tape AmandaFullBackup22 or a new tape) NOTE: skipping tape-writable test Server check took 31.128 seconds Amanda Backup Client Hosts Check WARNING: lx: selfcheck request timed out. Host down? Client check: 1 host checked in 30.002 seconds, 1 problem found (brought to you by Amanda 2.4.2p2) [Amanda@kodak Amanda]$ -- It does not check tape 7, nor tape 1 or tape 2. If tape 7 is loaded ( AmandaFullBackup22 ), then amcheck is happy. I suppose one can mix match, but i suppose in a BIG tape changer there may be a few empty slots, and a few full ( and even some with the tapes you want to use as the backup media ). ALSO openening the changer door resets the changers idea of whats loaded in each slot. The first time i run amcheck, i get a tape error. The changer does not move. But his test would ruin one nights backup attempt! A second amcheck gets it going. [Amanda@kodak Amanda]$ amcheck confname Amanda Tape Server Host Check - ERROR: holding disk /mnt/hdd4/amanda: statfs: No such file or directory ERROR: holding disk /mnt/sda1/amanda: statfs: No such file or directory amcheck-server: could not get changer info: no slots available John R. Jackson wrote: This is far short of a million, even if its binary, questions!
Re: Unloading tapes when task done ?
Its sorta like when you go to your kar mechanic. The job is not complete till you put all of your tools away, and cleaned your work area for the next job. That would be the mechanics responsibility, and not the cleaning staff that follows in the evening. But at this moment, some 18hours after amanda started, it is still going, and only on the third ( of an estimated 6 tape backup ). So the unload/eject is the least of my current concerns with regard to getting a backup to happen. Right now the most efficient system i have would be to run tar cMf remoteHost:/dev/st0 / directly on the remote system. after each EOT i will run a shell script to change to the next tape. AFTER I DO THIS CORRECTLY, only then will I get a TRUE feel of the time needed to do this phase of the task. /gat John R. Jackson wrote: Maybe they can OPT-OUT of the feature ... It's just as easy for someone to opt-in and do their own tape operations when Amanda is done. Amanda will currently support both camps -- unload it when done vs. leave it alone. Why add more complexity? We already have too many options (and lots of other things on the TODO list). btw, what do u mean each operation. ... I meant amdump and amflush. John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
Re: sendbackup with dumplevel 0 takes a while to startup :-{
Problem with this is obtaining a definitive PROOF. you do the amanda process, and you notice that the tape is not spinning, even though you are in the sendbackup phase. You notice that the ethernet switch is not blinking. You can also notice the task(s) that are running. You also notice that the disk controller light is light. You can make some general conclusions, like gee the reason i up'ed the destimate value from 30min to 6hours is due to what is occuring now. I dont know why, or all of the facts, but i do notice thats it's idle. and i do notice transfer timeouts, although i dont know why exactly either - yet. /gat John R. Jackson wrote: according to the tar docs, the --listed-incremental will check on the files listed in the file if the incremental file is not empty. I suppose the listed-incremental file got filled during the sendsize proceedure?! No. It got filled by the previous sendbackup run (e.g. yesterday). And, FYI, for a level 0 (full) dump, the listed incremental file comes from /dev/null, i.e. it is used, but empty. During the sendbackup step, tar is apparently going through and checking the list, as no data is being transmitted while tar is doing this check. I'm not sure exactly how tar does this. You'd have to look at their code or ask them. I'd be surprised if it completely lost it for 30 minutes, though. Is this how its suppose to work for a level 0 dump? ( 4 hours (wall-time) sendsize, ~4hours (wall-time) incremental check, and finally the transfer of the data ( i suppose longer that 4 hours )) ... Probably true. Is this how it will work with other level dumps ( with the exception of the transfer of data step) ? Yes. Except Amanda may also request a third estimate one level higher than the previous one if it might be time to bump (you can control this with the bump* parameters). You might want to consider using an alternate size calculation tool called calcsize. It's not well supported, and does not handle exclusion lists, but does all the estimates at the same time so you might gain quite a bit of time that way. If you want to go this route, let me know and I'll find the postings from a while back about turning it back on. /gat John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
Re: Unloading tapes when task done ?
DLT's are not flying head technology. They are like 9trk, QIC, i think travan, colorado. The heads do not spin ( i had to think about it ), as the heads move up down to change tracks. But the DLT 8000, now, also place the heads at an angle to tape direction, as well as going up and down. I dont even think there is an idler pully, just a tach. I'd like to know if there is 'continual' tension on the tape while it is loaded ( on a dlt )( Like that of a DEC Tape, or 9trk ) but I do not know. But for whatever technology reasons the schemes that tape mechanisms have evolved, they all rely on knowing what 'state' that they are in. When you have a power outage, turn off the drive, lightning, whatever, you may find that the tape left inside the mechanism to be of little use to you. Quantum says dont do that, AND i'd bet that the legal staff of the other drive manufacturers will never certify that you will always recover a tape left inside the mechanism. Gene Heskett wrote: I'd love to see the tapes stored and used at or slightly below 50 degrees F, and 50% relative humidity as the tape is many times less abrasive then. Some TV stations have even gone so far as to store their tapes in a small room adjacent to the control room which is maintained in the 40 degree and 40% range. Everyting lasts longer, a lot longer. But i suppose that if you needed a tape right away, you'd have to wait for the temp rise, otherwise ud get condensation on the kolder tapes. The tape makers themselves recommend it too, and have data to
Re: DLT
technically yes. A DLT-IV tape is the tape of choice on a quantum 7000, and quantum 8000. I do not know, however, if the tape that was written with a 7000 series, and later re-used on an 8000 if it would be reliable . I suspect that it should work. But for a definitive answer, give www.quantum.com a call. David Flood wrote: Sorry if this subject is off topic but can you use a 40/80 DLT tape in a 35/70 DLT tape drive. Although we'd only be using it as a 35/70 tape. - David Flood Systems Administrator [EMAIL PROTECTED] Tel: +44 (0)1224 262721 The Robert Gordon University School of Computing St. Andrews Street Aberdeen -
Unloading tapes when task done ?
It seems like when the whole procedure ( whether its labeling tape(s), amchecking tape, or amdumping ) is complete, the tape is left in the tape drive in an on-line state. Is there some reason why the tape is not at least rewind-offlined, or for tape changer folks rewoffl/unload'ed back to the carousel slot when the job/task is finished? Just my 2 cents
Re: Unloading tapes when task done ?
I dunno either, will it work on a 6 drive auto changer/tape library? /gat Dunno, my crontab sez: 0 18 * * 1-5 /volume/amanda/sbin/amdump lto; mt -f /dev/rmt/3h rewoffl works like a charm.
Re: Unloading tapes when task done ?
Guess I'm from old school. Older 9trk drives when left on, continually have the vacuum on, and under constant tension. Its not healthy for the tape. Does that happen with a DLT tape, how about the DDS tapes. I would NOT like a live tape to be left in the drive for long periods of time. Who knows when a power failure will hit, who knows when the drive will go awol. An unloaded tape, immediately after backup, is safe from other folks, as well as the server, writing onto it. Gene Heskett wrote: I'd druther it didn't. By leaving the tape online, the display
Re: Unloading tapes when task done ?
Maybe they can OPT-OUT of the feature, but on a busy site, allowing the customers to access the tape drives, would appear to make good business sense. Particularily when the drives are doing just nothing. /gat btw, what do u mean each operation. Each run? If funny things happen, maybe they should be addressed. After all, you got all these logs, and status files hanging around. John R. Jackson wrote: Is there some reason why the tape is not at least rewind-offlined ... Because other users specifically asked for it to be left alone (we had to remove code that used to rewind after each operation). I assume they run something else after amdump (for instance) to tack on a little more data to the tape. And in the case of a changer, offlining the drive may cause other things to happen, such as dropping the stack and loading the next tape, which might not be what you want. Better that we leave things alone and if a particular setup needs something different, it's easy for them to do. John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
Re: Unloading tapes when task done ?
Jay Lessert wrote: On Wed, Apr 03, 2002 at 06:19:19AM -0500, Uncle George wrote: It seems like when the whole procedure ( whether its labeling tape(s), amchecking tape, or amdumping ) is complete, the tape is left in the tape drive in an on-line state. Is there some reason why the tape is not at least rewind-offlined, or for tape changer folks rewoffl/unload'ed back to the carousel slot when the job/task is finished? 1) You would piss off people that are already doing: amdump DAILY; amverify DAILY ...from cron on a non-changer drive. Maybe a visit to the toilet facility would do wonders to your disposition. Its exceptionally arrogant of you to speak for us all. Since you dont know how it will be implemented, i can only wonder how you or god know that this will be true. 2) Since it's already so easy to do any of: amdump DAILY; amtape DAILY eject amdump DAILY; amtape DAILY slot next amdump DAILY; amtape DAILY slot advance ...or anything else that makes sense for your particular installation, why hardwire some particular behavior into amdump (which would then inevitably be exactly the wrong behavior for some other poor schmoe's particular installation)? Amazingly arrogant. I might think that on a power failure, as Quantum points out, they will not be responsible for the tape left inside the drive. AND i agree with their, slightly over stated, belief. The purpose of a backup is not just to do a backup, but be able to restore that data - generally when Kaos strikes. A trashed tape, even to the poor schlep who believes in you, is of no use to anyone. When you are finished, take the tape out, and put it in a safe place. BTW who said it was, or has to be hardwired. it need not be any more 'hardwired' than dumpcycle 0, or compress none, or simply a EjectWhenDone yes . BTW#2 you should be doing an amdump BACKUP; amtape BACKUP physically_set_write_protect_tab; amverify BACKUP
Re: Unloading tapes when task done ?
I Beleive you are in error. 4mm dds simply have the head stop spinning, the tape is still wrapped around in the mechanism. The DLT I have does not unload at all, unless specifically directed to. Maybe you can show me where u got your information from ? Maybe i can find the quantum docs that say 'dont do that'. Jay Lessert wrote: On Wed, Apr 03, 2002 at 08:58:29AM -0500, Uncle George wrote: Guess I'm from old school. Older 9trk drives when left on, continually have the vacuum on, and under constant tension. Its not healthy for the tape. Does that happen with a DLT tape, how about the DDS tapes. 8mm, AIT*, DDS*, DLT*, LTO* drives all unload the tape (unwrap the tape from around the head and power down the servos) after some hardwired time period. I would NOT like a live tape to be left in the drive for long periods of time. Then don't. Is there something you need to do that amtape doesn't know how to do? -- Jay Lessert [EMAIL PROTECTED] Accelerant Networks Inc. (voice)1.503.439.3461 Beaverton OR, USA(fax)1.503.466-9472
sendbackup with dumplevel 0 takes a while to startup :-{
according to the tar docs, the --listed-incremental will check on the files listed in the file if the incremental file is not empty. I suppose the listed-incremental file got filled during the sendsize proceedure?! During the sendbackup step, tar is apparently going through and checking the list, as no data is being transmitted while tar is doing this check. with a dtimeout of 30 minutes, i would get this unexpected timeout. Is this how its suppose to work for a level 0 dump? ( 4 hours (wall-time) sendsize, ~4hours (wall-time) incremental check, and finally the transfer of the data ( i suppose longer that 4 hours )) Is this how it will work with other level dumps ( with the exception of the transfer of data step) ? /gat 10690 ?S 0:00 /usr/local/libexec/sendbackup 10691 ?S 0:00 /usr/local/libexec/sendbackup 10692 ?D 1:37 gtar --create --file - --directory /mnt/hdg3 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar 10693 ?S 0:00 sh -c /bin/gtar -tf - 2/dev/null | sed -e 's/^\.//' 10694 ?S 0:01 /bin/gtar -tf - PWD=/tmp/amanda HOSTNAME=LX MACHTYPE=alpha-redhat-linux-gnu SHLVL=1 SHELL=/bin/bash HOSTTYPE=alp 10695 ?S 0:00 sed -e s/^\.// PWD=/tmp/amanda HOSTNAME=LX MACHTYPE=alpha-redhat-linux-gnu SHLVL=1 SHELL=/bin/bash HOSTTYPE=alph
Only 3 tapes in 7 tape changer gets scaned twice
I put 3 tapes in ( from a previous run ) with wrong labels ( ie they look like they are correct but they are not ) What is interesting is that the autochanger goes through tape 1, 2, 3, 1, 2, 3, 1 looking for a label. Slots 1, 2, 3 have tapes. Slots 4, 5 , 6 , 7 do not have any tapes. changer is chg-zd-mtx Whats also interesting is the quick scan of the 0 problems found made me believe everything was A-Ok. But No. No summary info for all of the parts? /gat BTW 1)the patch for the 'spindle' problem appear(s) to be fixed. 2) The patch for calculating what can fit on ONE tape vs. partition size appears to work now ( i get an error now ) [Amanda@kodak Amanda]$ amcheck confname Amanda Tape Server Host Check - Holding disk /mnt/hdd4/amanda: 5806570 KB disk space available, using 5601770 KBHolding disk /mnt/sda1/amanda: 1975122 KB disk space available, using 1678162 KBamcheck-server: slot 1: date 20020326 label GatWorksSet103 (no match) amcheck-server: slot 2: date Xlabel GatWorksSet107 (no match) amcheck-server: slot 3: date Xlabel GatWorksSet106 (no match) amcheck-server: slot 1: date 20020326 label GatWorksSet103 (no match) amcheck-server: slot 2: date Xlabel GatWorksSet107 (no match) amcheck-server: slot 3: date Xlabel GatWorksSet106 (no match) amcheck-server: slot 1: date 20020326 label GatWorksSet103 (no match) ERROR: label AmandaFullBackup3 or new tape not found in rack (expecting tape AmandaFullBackup3 or a new tape) NOTE: skipping tape-writable test Server check took 483.323 seconds Amanda Backup Client Hosts Check Client check: 1 host checked in 0.047 seconds, 0 problems found (brought to you by Amanda 2.4.2p2)
If estimated disk size tape size
will amanda tape out the data that cant fit onto that 40gig tape and then onto the next tape OR will it start over again from the beginning ( as the docs currently say ) and try to fit something that will never fit onto that tape?. If docs are correct, why does amanda bother bother attempting to write any partition that has more data than the tape than the tape can hold? should'nt there be a sorry can't do that error? /gat
Re: If estimated disk size tape size
Hummm, DLT tape set to 3mb ( 2 + .5 compression ) partition on /mnt/hde4 is 69578056k ( via df ) /mnt/hde4 'sendsizes' to 70727004160 ( 66GB, 4.6MB/s ) at 5am it tried to fit the (/mnt/hde4) partition onto the tail end of the tape EOT'd. At 8:30 it tried again onto the next tape. At 9am i stopped it and posed this inquiry. John R. Jackson wrote: will amanda tape out the data that cant fit onto that 40gig tape and then onto the next tape OR will it start over again from the beginning ( as the docs currently say ) ... If Amanda hits any tape error (including EOT) it will start the current image over, from the beginning, on the next tape. If it hits EOT, then didnt sendsize friends miscalculat? ... If docs are correct, why does amanda bother bother attempting to write any partition that has more data than the tape than the tape can hold? It doesn't. should'nt there be a sorry can't do that error? There is. If the estimate is larger than the tapetype length field you'll get one of these messages (depending on other settings): dump larger than tape, but cannot incremental dump skip-incr disk dump larger than tape, but cannot incremental dump new disk Dump larger than tape: full dump of XXX delayed. dump larger than tape, skipping incremental i guess i should ask why I didnt get any of these errors? And is there any consideration on fitting a large partition onto one or more tapes?
Re: If estimated disk size tape size
runtapes set to 7 INITIAL SCHEDULE (size 94289500): lx /mnt/hde4 pri 1 lev 0 size 69069340 lx /mnt/hdg3 pri 1 lev 0 size 6060890 lx /mnt/hdg2 pri 1 lev 0 size 5286010 lx /mnt/hdg4 pri 1 lev 0 size 5186500 lx /mnt/sdb2 pri 1 lev 0 size 4156550 lx / pri 1 lev 0 size 1962280 lx /mnt/sdb4 pri 1 lev 0 size 1810290 lx /mnt/sda2 pri 1 lev 0 size 737320 DELAYING DUMPS IF NEEDED, total_size 94289500, tape length 21504 mark 2000 delay: Total size now 94289500. John R. Jackson wrote: Hummm, DLT tape set to 3mb ( 2 + .5 compression ) partition on /mnt/hde4 is 69578056k ( via df ) /mnt/hde4 'sendsizes' to 70727004160 ( 66GB, 4.6MB/s ) ... i guess i should ask why I didnt get any of these errors? Don't know without more information. Take a look at the amdump.NN file for the got result for host lines to make sure what planner was working with. Then look for the DELAYING DUMPS IF NEEDED line, e.g.: DELAYING DUMPS IF NEEDED, total_size 33616007, tape length 55296000 mark 2000 This says the total estimated size was 33616007 KBytes, the tape length (including the runtapes multiplier) is 55296000 KBytes and a tape mark (space between each image) is 2000 KBytes. What is your runtapes set to? If it hits EOT, then didnt sendsize friends miscalculat? Not necessarily. If someone touch's a 30 GByte file that the estimate skipped over because it looked idle at that time, the dump will have to do it and that would throw everything out of whack. And is there any consideration on fitting a large partition onto one or more tapes? Definitely. All it takes is a few weeks of dedicated programming time by someone and a whole lot of testing. John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
i asked it to change label of tape in slot 6, but it erased slot 1 instead! :-{
There is no tape in slot 6, so I suppose ANY other tape would do . I suppose this is a bug. [Amanda@kodak sbin]$ for i in 6; do ./amlabel -f confname GatWorksSet16$i slot $i; done labeling tape in slot 1 (/dev/nst0): rewinding, reading label GatWorksSet161, tape is active rewinding, writing label GatWorksSet166, checking label, done. [Amanda@kodak sbin]$
i think that there is a (small) error in amanda(8) docs
shouldn';t the comment DLT4000 tape drives with Compact-IV tapes be
placed after the DLT4000-III tapetype ?
define tapetype DLT4000-III {
comment DLT4000 tape drives with Compact-III tapes
length 12500 mbytes # 10 Gig tapes with some
compression
filemark 2000 kbytes
speed 1536 kps
}
define tapetype DLT4000-IV {
comment DLT4000 tape drives with Compact-IV tapes
DLT4000-III
length 25000 mbytes # 20 Gig tapes with some
compression
sendsize appears to rerun disklist twice?
I waited 4 hours for /mnt/hde4 to complete the first run of sendsize. When it completes it appears to run the sendsize again on /mnt/hde4. This would take another 4 hours, and i'm wondering why? tar is 1.13.19 btw the lx_mnt_hde4_0.new does exist, but not just plain lx_mnt_hde4_0 sendsize: getting size via gnutar for /mnt/hde4 level 1 sendsize-gnutar: error opening /usr/local/var/amanda/gnutar-lists/lx_mnt_hde4_0: No such file or directory sendsize: spawning /usr/local/libexec/runtar in pipeline __ sendsize: debug 1 pid 1588 ruid 501 euid 501 start time Fri Mar 29 05:04:12 2002 /usr/local/libexec/sendsize: version 2.4.2p2 calculating for amname '/mnt/hde4', dirname '/mnt/hde4' calculating for amname '/mnt/hdg3', dirname '/mnt/hdg3' sendsize: getting size via gnutar for /mnt/hde4 level 0 sendsize: getting size via gnutar for /mnt/hdg3 level 0 calculating for amname '/mnt/hdg2', dirname '/mnt/hdg2' sendsize: spawning /usr/local/libexec/runtar in pipeline sendsize: argument list: /bin/gtar --create --file /dev/null --directory /mnt/hde4 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar-lists/lx_mnt_hde4_0.new --sparse --ignore-failed-read --totals --exclude-from /usr/local/etc/amanda/gtar/exclude.gtar . sendsize: spawning /usr/local/libexec/runtar in pipeline sendsize: argument list: /bin/gtar --create --file /dev/null --directory /mnt/hdg3 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg3_0.new --sparse --ignore-failed-read --totals --exclude-from /usr/local/etc/amanda/gtar/exclude.gtar . Total bytes written: 6206351360 (5.8GB, 852kB/s) . sendsize: getting size via gnutar for /mnt/hdg3 level 1 sendsize-gnutar: error opening /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg3_0: No such file or directory sendsize: spawning /usr/local/libexec/runtar in pipeline sendsize: argument list: /bin/gtar --create --file /dev/null --directory /mnt/hdg3 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg3_1.new --sparse --ignore-failed-read --totals --exclude-from /usr/local/etc/amanda/gtar/exclude.gtar . Total bytes written: 6206351360 (5.8GB, 985kB/s) . calculating for amname '/mnt/hdg4', dirname '/mnt/hdg4' sendsize: getting size via gnutar for /mnt/hdg2 level 0 sendsize: spawning /usr/local/libexec/runtar in pipeline sendsize: argument list: /bin/gtar --create --file /dev/null --directory /mnt/hdg2 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg2_0.new --sparse --ignore-failed-read --totals --exclude-from /usr/local/etc/amanda/gtar/exclude.gtar . Total bytes written: 5412874240 (5.0GB, 21MB/s) . sendsize: getting size via gnutar for /mnt/hdg2 level 1 sendsize-gnutar: error opening /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg2_0: No such file or directory sendsize: spawning /usr/local/libexec/runtar in pipeline sendsize: argument list: /bin/gtar --create --file /dev/null --directory /mnt/hdg2 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg2_1.new --sparse --ignore-failed-read --totals --exclude-from /usr/local/etc/amanda/gtar/exclude.gtar . Total bytes written: 5412874240 (5.0GB, 29MB/s) . calculating for amname '/mnt/sdb2', dirname '/mnt/sdb2' sendsize: getting size via gnutar for /mnt/hdg4 level 0 sendsize: spawning /usr/local/libexec/runtar in pipeline sendsize: argument list: /bin/gtar --create --file /dev/null --directory /mnt/hdg4 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg4_0.new --sparse --ignore-failed-read --totals --exclude-from /usr/local/etc/amanda/gtar/exclude.gtar . Total bytes written: 5310976000 (4.9GB, 22MB/s) . sendsize: getting size via gnutar for /mnt/hdg4 level 1 sendsize-gnutar: error opening /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg4_0: No such file or directory sendsize: spawning /usr/local/libexec/runtar in pipeline sendsize: argument list: /bin/gtar --create --file /dev/null --directory /mnt/hdg4 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar-lists/lx_mnt_hdg4_1.new --sparse --ignore-failed-read --totals --exclude-from /usr/local/etc/amanda/gtar/exclude.gtar . Total bytes written: 5310976000 (4.9GB, 23MB/s) . sendsize: getting size via gnutar for /mnt/sdb2 level 0 calculating for amname '/mnt/sda2', dirname '/mnt/sda2' sendsize: spawning /usr/local/libexec/runtar in pipeline sendsize: argument list: /bin/gtar --create --file /dev/null --directory /mnt/sdb2 --one-file-system --listed-incremental /usr/local/var/amanda/gnutar-lists/lx_mnt_sdb2_0.new --sparse --ignore-failed-read --totals --exclude-from /usr/local/etc/amanda/gtar/exclude.gtar . Total bytes written: 4256307200 (4.0GB, 5.2MB/s) . sendsize: getting size via gnutar for /mnt/sdb2 level 1 sendsize-gnutar: error opening /usr/local/var/amanda/gnutar-lists/lx_mnt_sdb2_0: No such file or directory sendsize: spawning /usr/local/libexec/runtar in
Re: sendsize appears to rerun disklist twice?
thats interesting, considering that i have dumpcycle 0 and dumpcycle 0 in dumptype global. Is there a way to stop the second scan, third, 4th, and so on ? John R. Jackson wrote: I waited 4 hours for /mnt/hde4 to complete the first run of sendsize. When it completes it appears to run the sendsize again on /mnt/hde4. This would take another 4 hours, and i'm wondering why? Because the first request was for a level 0 and the second was for a level 1. And Amanda may ask for a third estimate (level 2 in this case) if it's time (bumpdays) that a bump to the next level might be possible. John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
Re: i think that there is a (small) error in amanda(8) docs
Generally, u try the program. After a week of fustration, maybe then you think that you should have RTFM first. But so far neither method is doing me tooo mcchhh ggg. One might say that I can tar out the 69gb much faster with a simpler tar. Eventually i will find the 'key' to your methods of madness, and i will be content. Wow, you must be really bored to actually be reading the documentation and finding bugs like that :-) :-) :-). BTW. It seems that the sendsize finishes in a little over 4 hours. The send-backup gets an idle timeout ( which from the log is ~1802 seconds ) which is surprising in that it idle'd for a half hour - i suspect something is amiss, but i'm not at that part of the manual yet. :-/
what does etimeout really represent
does etimeout specify the time that sendsize will use to estimate the time needed to do 'its thing', or does it represent the sum total of time ( estimate dumping ) of a filesystem ?
Re: what does etimeout really represent
Then my current impression is that the feature does not work - exactly
as stated. There are a few file systems on one 'errant' system, where
one filesystem has taken over 224 minuts( wall time ) to complete just
the estimate. I had changed the time to be some 3600 ( an hour ) which,
if one beleives the man page, would leave some 8hours ( wall time ) for
all 8 partitions to complete. I think the log had 2 timeout errors of
some 3800. The longest, and the next to longest partition did not
complete :-{
I dont think that anyone wants to know who the 'planner' is, or
'sendsize' or even their relationship at a user, or administrative
level. But i suspect that one has to give the 'highest' possible
estimate on a per partition basis.
Its not too rational for the observer program 'planner?' to multiply the
estimate if u can have multiple ( or even a single ) 'sendsize's running
on the client machine ( now set at 8 * 6hrs ) Its just too long to wait
for some failed communication! ( it also ruins the concept of a daily
backup )
John R. Jackson wrote:
etimeout specifies that amount of time amdump will wait to hear back after
sending a sendsize request to a particular host. At least, I think it's
per-host.
This is covered in the man page:
Default: 300 seconds. Amount of time per disk on a
given client that the planner step of amdump will wait
to get the dump size estimates. For instance, with the
default of 300 seconds and four disks on client A,
planner will wait up to 20 minutes for that machine. A
negative value will be interpreted as a total amount of
time, instead of a per-disk value.
Note that, when positive, it is per disk. When negative, it is per
host (and the man page needs a minor tweak to make that point).
Joshua Baker-LePain
John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
Re: what does etimeout really represent
Sorry, for every NEW run, i would like a set of new logs just for that run just so that i know are from just that run. from my 'novice' eyes, its just to much data to figure out where the previous run completed, and the new one began. But if i ( ever ) get a complete backup ( after setting it for -6hrs ) i will go back and put it back to 3600. /gat John R. Jackson wrote: ... I had changed the time to be some 3600 ( an hour ) which, if one beleives the man page, would leave some 8hours ( wall time ) for all 8 partitions to complete. ... Right. That's the way it's supposed to work, and the way it has worked for myself and others. So what do you suggest? From an admin point of view I would like every disk on my disklist to be backed up. The time that it takes to do it is irrelevant. Time becomes relevant if the avenues of communication is severed between the parent/children ( maxdumps 1 ) of sendsize, and the communication between client and server becomes severed. How do u know that communication has been lost ? would a 'ping' or keep-alive concept have any use here ? But there are also times when the 'sizer' program may be stuck, spinning to no usefull end. I suppose that in this unusual case/scenario it would be up to the administrator to take the extraordinary action to determine what is causing the 'sizer' failure ( ie is it a bug? is tar backing up /dev/zero ? ) dtimeout represents idle time, why cant etimeout also represent idle time?
[Fwd: it did not complete, it took over 1.3 days to run out of 4 tapes!( 80gig uncompressed!)
---BeginMessage--- apparently it did not complete all filesystems. some appear to be network reliability problems. ( host reset by peer ) some appear to be that the server run out of time [ data timeout ] some appear to be because is cant open a jibberish filename !. are there things i can do ? /gat BTW i get a lot of these messages. I had to up the loop count in xinetd so that it would not be disabled. BTW#2 is this the right group, or would it be better in the amanda/users group ar 24 08:06:33 MyLaptop amandad[24106]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24105]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24111]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24118]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24121]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24119]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24117]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24115]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24114]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24122]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24120]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24116]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[24113]: error receiving message: timeout Mar 24 08:06:33 MyLaptop amandad[3275]: error receiving message: timeout Mar Mail.amanda Description: Binary data ---End Message---
