Backup of large disk arrays with AMANDA?????
Dear Sirs. I run into problems with AMANDA backing up a large disk array (250 GB). I use 'gnutar' as dumping tool defined by GTAR within amanda in the hope that AMANDA is capable to spread a big backup image over several tapes like TAR does this when used without AMANDA (multi volume facility). I asked about AMANDA's capabilty of backing up large arrays before and I was told to split the big array into smaller pieces. It seems to me, by the way, that AMANDA is only capable of backing up arrays or drive areas they are about the size of one tape. We use a tape changer with 6 slots of DDS-4 cardridges. But the suggestion splitting up the big array into pieces is impossible! The only way I could do is to order users in alphabetical order and save them splitted (the big array is the HOMES partition of our institute). But several users have enormous data fluctuations, today I run into problems with the daily backup due the fact one of the users load scientific data of about 30 GB - that is to much for one tape and since 8 hours amanda is trying to back up this area trying each tape, filling it up to its limits, then makes an error and take the next tape - a silly behaviour. Well, I would like to ask for some hints or simply a fact-statement: Is AMANDA capable of spreading a large volume over several tapes with gtar like gtar does this using the multivolume mechanism? How to configure this ability if possible? Please be aware of the following fact: I definitely can not split this array into smaller pieces because it is not possible to split them due the high data fluctuation. Thank you in adavnce, Oliver -- MfG O. Hartmann [EMAIL PROTECTED] IT-Administration des Institut fuer Physik der Atmosphaere (IPA) Johannes Gutenberg Universitaet Mainz Becherweg 21 55099 Mainz Tel: +496131/3924662 (Maschinensaal) Tel: +496131/3924144 FAX: +496131/3923532
RE: chg-manual poor man's changer problems
On Thu, 19 Apr 2001, Bort, Paul wrote: Hello, here I'm again. Well, the solution for my problem is really simple, but I have had to read the script first as Paul Bort implied. FreeBSD's 'mt' command reports in a different way than other OS' mt and for that, the variable $ONLINEREGEX in the script chg-manual contains no identifying expression to detect that a tape is 'online'. I added there a string which is only present if a tape has been sucessfully loaded into the drive and now, voila, it works fine also for FreeBSD driven UNIX boxes ;-) Thanks a lot ... :(Warning: Stream-Of-Trace) : :OK, I'm getting closer. The reason that it keeps asking for a tape in the :same slot is because it doesn't know what slot it's on. The message: : : - loaded : :Should really have a slot number in the , but doesn't. The variable $used :in chg-manual is empty for unknown reasons. : :I wish I knew more shell script. : :OK, it looks like the line before the "loaded" message is the relevant one: : :used=`$MT $MTF $tape status 21 | tee -a $dbglog | $EGREP "$ONLINEREGEX"` : :This is placing the status message in the log, but it looks like no status :message I've ever seen. : :My status messages look like this: : :[amanda@tape configdir]$ mt -f /dev/nst0 status :SCSI 2 tape drive: :File number=0, block number=1, partition=0. :Tape block size 0 bytes. Density code 0x0 (default). :Soft error count since last status=0 :General status bits on (101): : ONLINE IM_REP_EN : :I think that the grep is looking for 'ONLINE' and not finding it in your :status message, so it doens't think the tape drive is ready, and asks you to :insert a tape. : :Next steps would be to find out what AMANDA is invoking with $MT, and find :out if you have an mt command like mine, and how to bring the two closer :together. : :Sorry this took so long. I'm still learning shell script. : : :-Original Message----- :From: Hartmann, O. [mailto:[EMAIL PROTECTED]] :Sent: Thursday, April 19, 2001 8:55 AM :To: Bort, Paul :Cc: [EMAIL PROTECTED] :Subject: RE: chg-manual "poor man's" changer problems : : :On Wed, 18 Apr 2001, Bort, Paul wrote: : : :Of course, you're right, but this problem is reported by AMANDA. This error :seems nasty. : :All right, I'll try to explain again the configuration and you'll see that :many things are clear as a canonical declaration- there must be an small :error or a microscopic misconfiguration ... :-( : :Operating system: FreeBSD 4.3-RC :AMANDA: Amanda 2.4.2p1 : :Tapedrive:HP C1533A 9503 (4GB DDS-2 DAT drive) : The DAT drive is online and accessible! : AMANDA user 'operator' has full access to : the device. : Actually I can read and write normal cpio/pax/tar : from and to this drive as 'operator'. : :Tape cardridges: HP and TDK DDS-2 DAT tapes, NOT labeled! : :Configuration:/etc/inetd.conf is sdet up already, also :/etc/services. : Within "operator's" home .amandahosts exists, but it : is not used usually for local administrative :operations. : As defined at compiling time /etc/amandates exists :and : ist operator-writeable. : A amanda.conf file exists and is clearly read out by : AMANDA. The important configuration tags are: : : tpchanger "chg-manual" : tapedev "/dev/nsa0" : #rawtapedev "/dev/nrsa0" : changerfile :"/usr/local/etc/amanda/klima/changer.conf" : : The file 'changer.conf' exists, but it is empty. for :any tag I put : in there, I get an error by running amcheck so it :seems to be : right to leave this file clear, but it must exists :due the fact : the driver script 'chg-manual' needs it as the base :former for the : following also existent files: : : changer.confempty : changer.conf-access 22 (times of access) : changer.conf-clean 1 : changer.conf-slot 1 (reflects the slot :to use), : changed this :temporarily to 0, : doesn't matter ... :What I did, do and what the logs are: : :With the above mentioned configuration I tried to label or check tapes. For :instance of :labeling, I type exactly : : amlabel -f CONFIG_NAME CONFIG_TAG : :and gets: : : insert tape into slot 0 and press return : :I hit return, but nothing happened and the message occurs again, and again,
RE: chg-manual poor man's changer problems
dev/nsa0 - status /dev/nsa0 Mode Density Blocksize bpi Compression Current: 0x24:DDS-2 variable 61000DCLZ -available modes- 0:0x24:DDS-2 variable 61000DCLZ 1:0x24:DDS-2 variable 61000DCLZ 2:0x24:DDS-2 variable 61000DCLZ 3:0x24:DDS-2 variable 61000DCLZ - Current Driver State: at rest. - File Number: 0 Record Number: 0Residual Count 0 - loaded END-- That seems really normal to me, the tape is inserted, loaded and accessible and recognized by the tape subdevice and amanda recognizes it. That seems more than courios to me :-( :Usually if AMANDA kicks out a tape during amcheck it is either because the :tape isn't labeled or isn't the labeled tape she was expecting (for example, :if you forgot to change the tape.) : :With no offense intended, have you already amlabel'ed the tape you are :inserting during amcheck? : :Could you post everything that amcheck returns up (and including) the insert :message? This might give a clue about why it is deciding to reject that tape :and ask for another one. : : : :-Original Message- :From: Hartmann, O. [mailto:[EMAIL PROTECTED]] :Sent: Wednesday, April 18, 2001 4:50 PM :To: Bort, Paul :Cc: [EMAIL PROTECTED] :Subject: RE: chg-manual "poor man's" changer problems : : :On Wed, 18 Apr 2001, Bort, Paul wrote: : :Hello. :Thanks for responding. : :Our systems are all based on FreeBSD 4.3-RC3, I forgot to mention that. :well, the backup operator is 'operator' and this user has full read/write :permissions to the tape device (/dev/nsa0, /dev/nrsa0). : :whenever I put any line into "changer.conf" as configured with "changerfile" :in amanda.conf I receive errors about that line - so I omit all lines, I :simple :"touch" a changerfile 'changer.conf' to get the control files as expected :from :"chg-manual". : :When trying 'amcheck config' I get some infos about the holding disk and I :was :offered to insert a tape in slot 0 - and I should hit return. either I do :immediately :while the tape is already loaded into the HP DAT (4 GB DDS-2 drive) or I :eject and :reload it again, wait for the tape to be initialised and the hit return. :Always :with the same effect: I was offered to hit return again and load a tape into :slot 0 - but nothing happened to the tape inserted into the tapedrive. : :When checking /tmp/amanda/changer.debug then I get a big file with this: : :Mode Density Blocksize bpi Compression :Current: 0x24:DDS-2 variable 61000DCLZ :-available modes- :0:0x24:DDS-2 variable 61000DCLZ :1:0x24:DDS-2 variable 61000DCLZ :2:0x24:DDS-2 variable 61000DCLZ :3:0x24:DDS-2 variable 61000DCLZ :- :Current Driver State: at rest. :- :File Number: 0 Record Number: 0Residual Count 0 : - loaded : - rewind /dev/nsa0 : - status /dev/nsa0 :Mode Density Blocksize bpi Compression :Current: 0x24:DDS-2 variable 61000DCLZ :-available modes- :0:0x24:DDS-2 variable 61000DCLZ :1:0x24:DDS-2 variable 61000DCLZ :2:0x24:DDS-2 variable 61000DCLZ :3:0x24:DDS-2 variable 61000DCLZ :- :Current Driver State: at rest. :- :File Number: 0 Record Number: 0Residual Count 0 : - loaded : - rewind /dev/nsa0 : - status /dev/nsa0 :Mode Density Blocksize bpi Compression :Current: 0x24:DDS-2 variable 61000DCLZ :-available modes- :0:0x24:DDS-2 variable 61000DCLZ :1:0x24:DDS-2 variable 61000DCLZ :2:0x24:DDS-2 variable 61000DCLZ :3:0x24:DDS-2 variable 61000DCLZ :- :Current Driver State: at rest. :- :File Number: 0 Record Number: 0Residual Count 0 : - loaded : :-END- : :Well, it seems to be all right, tape is loaded, clear to be written on - but :whats going on with the changer? : :It seems not to be willing to write anything :-( : :A normal tar, mt operations and so on work well with the tape and as user :'operator', :so that is not a kind of a permission issue. : ::The config stuff looks OK to me, I would suggest leaving off the slot part ::of the amlabel command. When I label tapes through chg-manual, I always :just ::`amlabel Co
chg-manual poor man's changer problems
Dear Sirs. I already set up a HP SureStore 6x40i changer based tape device backing up our main server, now I tried to install on a secondary machine with a simple HP DAT drive a second piece of amanda, but without success. I followed the steps described in the really confusing manuals and other FAQ-based stuff. If I'm right, I have to setup the following in amanda.conf: (here are only the relevant entries) tpchanger "chg-manual" tapedev "/dev/nsa0" changerfile "/usr/local/perfector/amanda/services-backup/changer.conf" The entry "changerfile" should be set due the fact, that chg-manual driver script searches for several status-semaphores created with the name of the base changerfile. If I omit this, I receive an error. The targeted changerfile is either empty or I configure some parameters like slot /dev/nsa0 etc. But I receive allways the same result. After typing "amlabel -f CONFIG CONFIGLABEL slot [0-9] (I tried slot 0 to 9, but configured either slot 0 or 1, for testing purposes to see whether the result changes) I get the message that I'm to be to insert a cartridge into drive 0 or drive 1 (depends on what I configured or what slot I gave amlabel). I inserted already a cartridge, but I always get the same message after hitting RETURN. I checked /tmp/amanda/ amlabel.debug and there is a mt output I receive by 'mt status' - and this seems absolutely normal as it shows some informations about the used tape (DDS-2 and so on). Can anybody help? What to be aware of when configuring a chg-manual device? -- MfG O. Hartmann [EMAIL PROTECTED] IT-Administration des Institut fuer Physik der Atmosphaere (IPA) Johannes Gutenberg Universitaet Mainz Becherweg 21 55099 Mainz Tel: +496131/3924662 (Maschinensaal) Tel: +496131/3924144 FAX: +496131/3923532
RE: chg-manual poor man's changer problems
On Wed, 18 Apr 2001, Bort, Paul wrote: Hello. Thanks for responding. Our systems are all based on FreeBSD 4.3-RC3, I forgot to mention that. well, the backup operator is 'operator' and this user has full read/write permissions to the tape device (/dev/nsa0, /dev/nrsa0). whenever I put any line into "changer.conf" as configured with "changerfile" in amanda.conf I receive errors about that line - so I omit all lines, I simple "touch" a changerfile 'changer.conf' to get the control files as expected from "chg-manual". When trying 'amcheck config' I get some infos about the holding disk and I was offered to insert a tape in slot 0 - and I should hit return. either I do immediately while the tape is already loaded into the HP DAT (4 GB DDS-2 drive) or I eject and reload it again, wait for the tape to be initialised and the hit return. Always with the same effect: I was offered to hit return again and load a tape into slot 0 - but nothing happened to the tape inserted into the tapedrive. When checking /tmp/amanda/changer.debug then I get a big file with this: Mode Density Blocksize bpi Compression Current: 0x24:DDS-2 variable 61000DCLZ -available modes- 0:0x24:DDS-2 variable 61000DCLZ 1:0x24:DDS-2 variable 61000DCLZ 2:0x24:DDS-2 variable 61000DCLZ 3:0x24:DDS-2 variable 61000DCLZ - Current Driver State: at rest. - File Number: 0 Record Number: 0Residual Count 0 - loaded - rewind /dev/nsa0 - status /dev/nsa0 Mode Density Blocksize bpi Compression Current: 0x24:DDS-2 variable 61000DCLZ -available modes- 0:0x24:DDS-2 variable 61000DCLZ 1:0x24:DDS-2 variable 61000DCLZ 2:0x24:DDS-2 variable 61000DCLZ 3:0x24:DDS-2 variable 61000DCLZ - Current Driver State: at rest. - File Number: 0 Record Number: 0Residual Count 0 - loaded - rewind /dev/nsa0 - status /dev/nsa0 Mode Density Blocksize bpi Compression Current: 0x24:DDS-2 variable 61000DCLZ -available modes- 0:0x24:DDS-2 variable 61000DCLZ 1:0x24:DDS-2 variable 61000DCLZ 2:0x24:DDS-2 variable 61000DCLZ 3:0x24:DDS-2 variable 61000DCLZ - Current Driver State: at rest. - File Number: 0 Record Number: 0Residual Count 0 - loaded -END- Well, it seems to be all right, tape is loaded, clear to be written on - but whats going on with the changer? It seems not to be willing to write anything :-( A normal tar, mt operations and so on work well with the tape and as user 'operator', so that is not a kind of a permission issue. :The config stuff looks OK to me, I would suggest leaving off the slot part :of the amlabel command. When I label tapes through chg-manual, I always just :`amlabel Config2 NextTapeName`, and if it asks me to put a tape in a :specific slot, I just re-insert the tape in the drive, wait for it to settle :down (it's an EXB-8505, they take a while) and hit enter. Solid as a rock. : :The only other thing that comes to mind is to make sure that the amanda user :has read and write permissions on the changerfile. : :Good Luck. : : :-----Original Message- :From: Hartmann, O. [mailto:[EMAIL PROTECTED]] :Sent: Wednesday, April 18, 2001 4:03 PM :To: [EMAIL PROTECTED] :Subject: chg-manual "poor man's" changer problems : : :Dear Sirs. : :I already set up a HP SureStore 6x40i changer based tape device :backing up our main server, now I tried to install on a secondary :machine with a simple HP DAT drive a second piece of amanda, but :without success. : :I followed the steps described in the really confusing manuals and :other FAQ-based stuff. : :If I'm right, I have to setup the following in amanda.conf: : :(here are only the relevant entries) : :tpchanger "chg-manual" :tapedev "/dev/nsa0" :changerfile :"/usr/local/perfector/amanda/services-backup/changer.conf" : :The entry "changerfile" should be set due the fact, that chg-manual driver :script :searches for several status-semaphores created with the name of the base :changerfile. :If I omit this, I receive an error. The targeted changerfile is either empty :or :I configure some parameters like slot /dev/nsa0 etc. : :But I receive allways the same result. After typing "amlabel -f CONFIG :CONFIGLABEL slot [0-9]
GTAR Tape Changer problems
Dear Sirs. For our 250 GBytes RAID array backup on a FBSD 4.3 Server we use AMANDA 2.4.2p1. The tape changer is a HP SureStore 40x6i, which has a 6 cartridges magazine. I configured AMANDA that way to use the first time 8 magazines, each holding 5 tapes and one cleaning tape. For all filesystems GNU tar (GTAR) is used so I do not fall into problems when dumping images bigger than one tape. The first time this runs well, AMANDA saved all the images out to two or three tapes (like the multi volume mechanism when using a simple tar -M backing up a big filesystem ...). Now we changed several parameters (due to the lacking docimentation how to use GTAR we have had to recalibrate all parameters to the right size and want to start from the begining). amlabel labels each used cartridge used by AMANDA and in the manner of using dump as the backup/save tool this means that each run is made to tapes with increasing numbers. At this moment, our magazines are filled up with cartridges each labeled with the same label, means mag one is completely labeled throughout each cartridge with LABEL00, the second mag's tapes LABEL01 ... that means now, whenever GNU tar/AMANDA reaches the end of the first tape to continiue on the next one, it finds the same labeld tape in each slot, greps through the whole magazine and tells that it is out of tape. Well, I'm not very familiar how AMANDA/GTAR signs up each tape to keep in track with each one, but I want it to have 'clean' tapes because I think gtar is not continuing a LABEL00-tape on a second, like LABEL00:1, instead it needs LABEL01 - tape, or I'm wrong? How can I delete each label a fast way? ERASE is not a good idea (using mt) due the fact it takes a long time for each tape to be cleaned and delabeled ... -- MfG O. Hartmann [EMAIL PROTECTED] IT-Administration des Institut fuer Physik der Atmosphaere (IPA) Johannes Gutenberg Universitaet Mainz Becherweg 21 55099 Mainz Tel: +496131/3924662 (Maschinensaal) Tel: +496131/3924144 FAX: +496131/3923532
Backup with GNUTAR of big volumes ...
Dear Sirs. I installed amanda 2.4.2p1 and I use HP's SureStore 40x6i autoloader as backup system. For all backuped filesystems I wih to use gnutar - and use it with success - on drives with a capacity less than one tape's capacity! Each cartridges is labeled with an individual label, and counts up to 30, five cartridges per magazine plus one slot cleaning cartridge per magazine. Our user space RAID is about 200 GB in size - and I wish to backup this amount by using the multivolume facility of TAR - but amanda reports a failure - dump size is bigger than tape size (?). I do not use dump, I configured tar and I do not understand why it is impossible to span a backuped volume over multiple tapes. it seems, that amanda wish to backup one filesystem completely on one labeled tape - in this case, it is more than seneseless for me to backup our RAID system. Do I have to label a complete magazine with each slot (6 times) with the same label to target this? -- MfG O. Hartmann [EMAIL PROTECTED] IT-Administration des Institut fuer Physik der Atmosphaere (IPA) Johannes Gutenberg Universitaet Mainz Becherweg 21 55099 Mainz Tel: +496131/3924662 (Maschinensaal) Tel: +496131/3924144 FAX: +496131/3923532
