Re: Amanda from RPMs
Matt Hyclak wrote: On Fri, Jul 22, 2005 at 01:05:28PM -0500, Vicki Stanfield enlightened us: Matt Hyclak wrote: It's a little later than I promised, however I've pulled together a page covering some of the RPM changes I've proposed for amanda, along with some instructions for those unfamiliar with rebuilding RPMs. http://www.math.ohiou.edu/~hyclak/casit/amanda/ Feel free to link to this page, steal the files, whatever :-) Matt Looks good, Matt, but how about adding a specific example of changing the --with-user config option since that is commonly done (and gave me trouble). Since Debian (and maybe others???) uses the backup user instead of the amanda user, it is not possible AFAIK to use the rpm without changing the --with-user option. Well, I'm not sure anyone would be using RPMs on Debian, but it's definitely a good point. I'll expand on that section a little bit. Thanks! Matt No, I meant that if you are running the server on Debian and you have some Red Hat clients, you'll run into a problem. Again AFAIK. Vicki
Re: Amanda from RPMs
Matt Hyclak wrote: It's a little later than I promised, however I've pulled together a page covering some of the RPM changes I've proposed for amanda, along with some instructions for those unfamiliar with rebuilding RPMs. http://www.math.ohiou.edu/~hyclak/casit/amanda/ Feel free to link to this page, steal the files, whatever :-) Matt Looks good, Matt, but how about adding a specific example of changing the --with-user config option since that is commonly done (and gave me trouble). Since Debian (and maybe others???) uses the backup user instead of the amanda user, it is not possible AFAIK to use the rpm without changing the --with-user option. Vicki
Re: ssh keys only
Gene Heskett wrote: Yes, Vicki. I haven't had to deal with that here, but perhaps someone else has rigged an ssh script to wrap amanda up in? You might consider shooting the co-worker :) Better yet, offload the problem onto the one that caused it by telling him there will be no backups of that server until he cuts a hole in things so amanda can continue to do her job. Point out how valuable the info on the server is & all that rot... The problem is fixed and the employee in question has been forced to work in a VAX environment for a week as punishment. Thanks. Vicki
ssh keys only
Most of our servers are accessible only via ssh with a root key. Does amanda work in such a setup or does the amanda user have to have regular login access? One of my coworkers changed one of our servers to only accept logins via ssh and now amanda doesn't seem to be able to get there and we get the following message: WARNING: /host/: selfcheck request timed out. Host down? I suspect that amanda simply doesn't like not being able to log in. Is this accurate? Vicki
Re: runtar: error [must be invoked by amanda]
Gene Heskett wrote: On Wednesday 13 July 2005 14:22, Jon LaBadie wrote: On Wed, Jul 13, 2005 at 12:22:56PM -0500, Vicki Stanfield wrote: Jon LaBadie wrote: On Wed, Jul 13, 2005 at 10:05:29AM -0500, Vicki Stanfield wrote: I have set up a client for amanda to backup. The amanda server runs as user backup, so I set up the client to be run by backup as well. I am getting errors in my amanda main report: moe/tmp/rt3.backup lev 0 FAILED [disk /tmp/rt3.backup, all estimate failed] In the sendsize log I see the following line: runtar: error [must be invoked by amanda] Looks like the executable was compiled with the default amanda user, "amanda". Thus, it expects to be run only by a user named amanda. After compile you decided to name the amanda user "backup". The excutables on the client must be compiled to know of this. configure --with-?user?=backup ... (Not certain of the exact "with" option). Actually I just used yum to install the amanda-client, so there was no compiling involved. So I assume that I need to use a tarball if the username is not amanda, then? Don't know yum. Yellowdog Updater, Modified. It can keep your system uptodate automaticly if you run it daily AND your distro supports it. But, he has just ran into one of the main reasons we always recommend building amanda from the tarball, that way you don't get stuck with somebody elses grand view of the ideal preferences. Or he could just add a user named amanda, and make that amanda a member of the group disk or backup. Putting the execute daily stuff into the user amanda's crontab by becomeing amanda and doing a crontab -e to add it. That _should_ work if he is not comfortable working with the compiler. Do some of those pre-built packages have a way of specifying local preferences? Or is that only with the source packages. But the problem there is that I inherited a working amanda server configuration which runs as backup. I would have to change all the exsting clients and the server to make them run as amanda, right? I can do the tarball thing. No problem. I just wanted to make sure that that was the only option. Vicki
Re: runtar: error [must be invoked by amanda]
Jon LaBadie wrote: On Wed, Jul 13, 2005 at 10:05:29AM -0500, Vicki Stanfield wrote: I have set up a client for amanda to backup. The amanda server runs as user backup, so I set up the client to be run by backup as well. I am getting errors in my amanda main report: moe/tmp/rt3.backup lev 0 FAILED [disk /tmp/rt3.backup, all estimate failed] In the sendsize log I see the following line: runtar: error [must be invoked by amanda] Looks like the executable was compiled with the default amanda user, "amanda". Thus, it expects to be run only by a user named amanda. After compile you decided to name the amanda user "backup". The excutables on the client must be compiled to know of this. configure --with-?user?=backup ... (Not certain of the exact "with" option). Actually I just used yum to install the amanda-client, so there was no compiling involved. So I assume that I need to use a tarball if the username is not amanda, then? Vicki
runtar: error [must be invoked by amanda]
I have set up a client for amanda to backup. The amanda server runs as user backup, so I set up the client to be run by backup as well. I am getting errors in my amanda main report: moe/tmp/rt3.backup lev 0 FAILED [disk /tmp/rt3.backup, all estimate failed] In the sendsize log I see the following line: runtar: error [must be invoked by amanda] Everything else in the logs looks to me like the backup user is running things and is authenticated. Am I misreading something, or is the problem elsewhere? amandad: time 0.001: bsd security: remote host morimoto.progeny.com user backup local user backup amandad: time 0.001: amandahosts security check passed amandad: time 0.001: running service "/usr/lib64/amanda/sendsize" TIA, Vicki
Re: The next tape Amanda expects to use is: DailySet104 and the DailSet104 is in the drive.
Chuck Amadi Systems Administrator wrote: Hi I don't have no tpchanger specified in my ~amanda.conf As below # Specify tape device and/or tape changer. If you don't have a tape # changer, and you don't want to use more than one tape per run of # amdump, just comment out the definition of tpchanger. I am only using one tape per run that does a full backup onto tape. Thus is there a way to check my tapes I did orderly label them but need to verify which tape is DailySet4. Cheers On Mon, 2005-06-27 at 10:51 -0500, Vicki Stanfield wrote: Chuck Amadi Systems Administrator wrote: Hi all I have running amanda for a week but Fridays tape DailySet4 didn't get backed up because The DailySet4 tape is in the drive but my amanda doesn't think So. What is the best way to check what amanda tape it is as I assume when labelling I may of made a mistake. Is there a amanda program that will check what tape I have in the drive. *** A TAPE ERROR OCCURRED: [not an amanda tape]. Some dumps may have been left in the holding disk. Run amflush to flush them to tape. The next tape Amanda expects to use is: DailySet104. Here is the message when I run amverify Defects file is /tmp/amanda/amverify.20028/defects amverify DailySet1 Mon Jun 27 15:41:39 BST 2005 Using device /dev/nst0 Waiting for device to go ready... Rewinding... Processing label... ** No amanda tape in slot So whats the best step to correct this. Cheers I would try amtape [config] show. Vicki Sorry. I didn't notice that in your earlier email. How about using dd to read the header info off the tape? Vicki
Re: The next tape Amanda expects to use is: DailySet104 and the DailSet104 is in the drive.
Chuck Amadi Systems Administrator wrote: Hi all I have running amanda for a week but Fridays tape DailySet4 didn't get backed up because The DailySet4 tape is in the drive but my amanda doesn't think So. What is the best way to check what amanda tape it is as I assume when labelling I may of made a mistake. Is there a amanda program that will check what tape I have in the drive. *** A TAPE ERROR OCCURRED: [not an amanda tape]. Some dumps may have been left in the holding disk. Run amflush to flush them to tape. The next tape Amanda expects to use is: DailySet104. Here is the message when I run amverify Defects file is /tmp/amanda/amverify.20028/defects amverify DailySet1 Mon Jun 27 15:41:39 BST 2005 Using device /dev/nst0 Waiting for device to go ready... Rewinding... Processing label... ** No amanda tape in slot So whats the best step to correct this. Cheers I would try amtape [config] show. Vicki
amandad still running/ RESULTS MISSING
I am having a problem with amanda whereby some of the dumps (DLE's on the amanda server itself) are showing as RESULTS MISSING in the logs. There is an amandad process which is still running after quite a long time (24 hours) and which has a defunct child process: backup3318 0.0 0.1 3044 1076 ?SMay26 0:00 amandad backup3320 0.0 0.0 00 ?ZMay26 0:00 [amandad ] I googled the RESULTS MISSING error and checked that the appropriate files have SUID set and that the entries in inetd are right (they have not been changed). The only change that I have made in the last couple days is to add a DLE on a remote system (which already had other DLE's being backed up). Anyone have a clue for me? TIA, Vicki
tape label problem
I relabeled a tape during the last run using this command: /usr/local/sbin/amlabel Progeny01 Progeny01-0039 slot 2 The process finished and when I did an amtape Progeny01 show, the correct label appeared to be on the tape. Now when I try to reuse that same tape, amanda thinks it is labelel what it was previously labeled (Progeny01-0040). I tried to label it again, sudo -H su backup -c "/usr/local/sbin/amlabel Progeny01 Progeny01-0039 slot 1" amlabel: label Progeny01-0039 already on a tape And then I run the amtape show again, and it says the label is what it was before: sudo -H su backup -c "/usr/local/sbin/amtape Progeny01 show" amtape: scanning all 2 slots in tape-changer rack: slot 1: date 20050419 label Progeny01-0038 slot 2: date 20050418 label Progeny01-0040 What am I doing wrong? Vicki
Re: ERROR: new tape not found in rack
Alexander Jolk wrote: Vicki Stanfield wrote: I edited the tapelist to make sure the should-be-current tape was the last one listed: I moved to the top of the file and zeroed out the dates on the entries for tapes that had been used since the tarball was made. According to your last mail, you have successfully un-stuck your problem, but just a note: a date stamp of zero means to amanda that the tape has never been used before. Since you had two tapes with non-zero dates I believe, amanda would prefer the unused tapes to them. Alex Thanks. I hadn't thought that succinctly through that part of it. Vicki
Re: ERROR: new tape not found in rack
Frank Smith wrote: --On Tuesday, May 17, 2005 16:10:16 -0500 Vicki Stanfield <[EMAIL PROTECTED]> wrote: I was deleting old amanda installs that I'd inherited. I accidently deleted part of the current installation. I was able to untar the file again and reconfigure it. I had the configuration files in a tarball, so I just had to untar that to get pretty much back to normal. I edited the tapelist to make sure the should-be-current tape was the last one listed: I moved to the top of the file and zeroed out the dates on the entries for tapes that had been used since the tarball was made. I then ran amcheck again, and now I am getting: Amanda Tape Server Host Check - Holding disk /var/spool/amanda: 36514724 KB disk space available, that's plenty amcheck-server: slot 1: date 20050415 label Server01-0034 (active tape) amcheck-server: slot 2: date 20050415 label Server01-0035 (active tape) ERROR: new tape not found in rack (expecting a new tape) NOTE: skipping tape-writable test Server check took 0.716 seconds How do I get it to restart at tape Server01-0034 ? Amanda still thinks it is in use. Actually, it thinks all your tapes are in use and is asking for a new tape. Does your tapelist still contain as many tapes as your tapecycle? Frank Vicki P.S. I am sorry to be asking so similar a question to one that I have asked before, but I thought I'd done what I was told to do before, and I am still stuck. I got it. I just manufactured dates for the entries that were moved. Amanda seems happy now, although I don't suppose I'll know for sure until the backups run tonight. Thanks for your help. Vicki
Re: ERROR: new tape not found in rack
Frank Smith wrote: --On Tuesday, May 17, 2005 16:10:16 -0500 Vicki Stanfield <[EMAIL PROTECTED]> wrote: I was deleting old amanda installs that I'd inherited. I accidently deleted part of the current installation. I was able to untar the file again and reconfigure it. I had the configuration files in a tarball, so I just had to untar that to get pretty much back to normal. I edited the tapelist to make sure the should-be-current tape was the last one listed: I moved to the top of the file and zeroed out the dates on the entries for tapes that had been used since the tarball was made. I then ran amcheck again, and now I am getting: Amanda Tape Server Host Check - Holding disk /var/spool/amanda: 36514724 KB disk space available, that's plenty amcheck-server: slot 1: date 20050415 label Server01-0034 (active tape) amcheck-server: slot 2: date 20050415 label Server01-0035 (active tape) ERROR: new tape not found in rack (expecting a new tape) NOTE: skipping tape-writable test Server check took 0.716 seconds How do I get it to restart at tape Server01-0034 ? Amanda still thinks it is in use. Actually, it thinks all your tapes are in use and is asking for a new tape. Does your tapelist still contain as many tapes as your tapecycle? Frank Vicki P.S. I am sorry to be asking so similar a question to one that I have asked before, but I thought I'd done what I was told to do before, and I am still stuck. Yes, number of entries in tapelist equals the number specified for tapecycle. Vicki
ERROR: new tape not found in rack
I was deleting old amanda installs that I'd inherited. I accidently deleted part of the current installation. I was able to untar the file again and reconfigure it. I had the configuration files in a tarball, so I just had to untar that to get pretty much back to normal. I edited the tapelist to make sure the should-be-current tape was the last one listed: I moved to the top of the file and zeroed out the dates on the entries for tapes that had been used since the tarball was made. I then ran amcheck again, and now I am getting: Amanda Tape Server Host Check - Holding disk /var/spool/amanda: 36514724 KB disk space available, that's plenty amcheck-server: slot 1: date 20050415 label Server01-0034 (active tape) amcheck-server: slot 2: date 20050415 label Server01-0035 (active tape) ERROR: new tape not found in rack (expecting a new tape) NOTE: skipping tape-writable test Server check took 0.716 seconds How do I get it to restart at tape Server01-0034 ? Vicki P.S. I am sorry to be asking so similar a question to one that I have asked before, but I thought I'd done what I was told to do before, and I am still stuck.
Re: very large dumps in holding area
Alexander Jolk wrote: Vicki Stanfield wrote: I have searched the list archives and don't see a way (using GNU mt 2.4.2) to permanently disable hardware compression. I thought I had seen a post on it fairly recently, but I didn't find it with my search. I see the mt option defcompression, but that apparently isn't supported in my version of mt. There seem to be a few different versions of mt, with slightly different syntax. Mine uses `mt datcompression 0', other, as you say, `defcompression', and there also seems to be a version that uses plain `compression'. You are on Linux, or something else? Alex I am on Debian Woody (Linux). The version I have doesn't seem to support the defcompression or the compression options, so I am unsure how to make the change permanent. In the meantime, I am tracking down what has made the dumps I mentioned so large. Hopefully, I will be able to split them up. Vicki
Re: very large dumps in holding area
Jon LaBadie wrote: On Thu, Apr 21, 2005 at 10:49:00AM -0400, Vicki Stanfield wrote: If the dump is 28GB, should it not fit on a 20GB tape if it is compressed. Aren't the tapes basically 20/40GB? Not in my mind. They are 20GB tapes. I.e. they can hold 20GB of ones and zero bits. What those ones and zeros represent "could be" 40GB of data compressed ton 20GB of zeros and ones. I.e. 50% compression. But the level of compression varies greatly with the type of data being compressed and the algorithm being used. In fact, the compressor used by your tape drive can actually expand some types of data rather than compressing it. Using gzip as the software compressor I've tested data and gotten 0, 3, 10, 25, 50, 75, even 90+ percent compression. If the data in your dump compresses 50%, fine, it will fit on a 20GB tape. But if it only compresses 3%, I doubt it. And if it was already compressed by gzip (or has lots of gzipped files), it WILL EXPAND if the tapedrive compressor works on it. Okay, I see that I asked my initial question badly. I know that you very seldom if ever get 2-1 compression. I just meant that I thought that 1.4 to 1 wasn't too much to ask. Perhaps that is incorrect. Anyway, I am perfectly willing to stop using hardware compression altogether but haven't figured out how to make the setting permanent yet using GNU-mt 2.4.2. Vicki
Re: very large dumps in holding area
Alexander Jolk wrote:
Vicki Stanfield wrote:
I'm guessing that the holding area isn't big enough (which I had been
told before I admit, but since it's on its own partition I haven't
done anything about it). I remember hearing that you can combine two
holding areas as one; is that possible for me to add space from
another drive and combine it with my current holding area?
Just add a second holding disk to your amanda.conf, amand will use
both of them.
That's not your problem though; please follow Joshua's advice first.
Your dump simply is larger than your tape, and there's nothing a
holding disk can do about that. You need to split your DLE into
smaller pieces using GNUTAR and possibly exclude lists, and you'll
need to delete this too-large dump file from your holding disk. And
you probably have hardware and software compression mixed up, but we
cannot be sure without getting more info.
Alex
Well, I didn't remember to turn off software compression before this
run, so that is obviously a problem.
The whole tape definition is:
# our tape definition
define tapetype HP-C5683A {
comment "DDS-4 DAT tape"
length 16584 mbytes
filemark 452 kbytes
speed 2613 kbytes
}
The length was set before I got here, but I googled around and it seemed
to be in keeping with the other HP-C5683A entries that I found. Is it
not correct?
If the dump is 28GB, should it not fit on a 20GB tape if it is
compressed. Aren't the tapes basically 20/40GB?
Vicki
Re: very large dumps in holding area
Joshua Baker-LePain wrote: On Thu, 21 Apr 2005 at 9:24am, Vicki Stanfield wrote I am having a problem with amanda leaving very large dumps in the holding area and not writing them to tape. The tapes are 20/40GB tapes, but a 23GB dump doesn't get written even though there is a blank tape available. Even if I try to flush this dump, it runs for a long time and then errors again with "No space left on device." Are you using software or hardware compression? What does your tapetype look like? The dumps were flushed to tapes Progeny01-0043, Progeny01-0044. *** A TAPE ERROR OCCURRED: [[writing file: No space left on device]]. Some dumps may have been left in the holding disk. Run amflush again to flush them to tape. The next 2 tapes Amanda expects to used are: Progeny01-0020, Progeny01-0021. *snip* You clipped the *really* interesting part -- the line in the NOTES section from taper. What's that line look like? Okay. The tapetype is: HP-C5683A. I'm guessing that the holding area isn't big enough (which I had been told before I admit, but since it's on its own partition I haven't done anything about it). I remember hearing that you can combine two holding areas as one; is that possible for me to add space from another drive and combine it with my current holding area? Where can I find instructions on how to do that? NOTES: driver: WARNING: /var/spool/amanda: 36505600 KB requested, but only 8085788 KB available. planner: Incremental of pollux:/net/pollux/project/new-ps-archive/60 bumped to level 3. planner: Incremental of morimoto:/net/morimoto/home bumped to level 6. planner: Incremental of morimoto:/var bumped to level 2. planner: morimoto /net/morimoto/home 20050421 0 [dump larger than tape, 39666051 KB, full dump delayed] Thanks, Vicki
very large dumps in holding area
I am having a problem with amanda leaving very large dumps in the
holding area and not writing them to tape. The tapes are 20/40GB tapes,
but a 23GB dump doesn't get written even though there is a blank tape
available. Even if I try to flush this dump, it runs for a long time and
then errors again with "No space left on device."
Here is the log that I got.
The dumps were flushed to tapes Progeny01-0043, Progeny01-0044.
*** A TAPE ERROR OCCURRED: [[writing file: No space left on device]].
Some dumps may have been left in the holding disk.
Run amflush again to flush them to tape.
The next 2 tapes Amanda expects to used are: Progeny01-0020, Progeny01-0021.
FAILURE AND STRANGE DUMP SUMMARY:
morimoto /net/morimoto/home lev 5 FAILED [out of tape]
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:00
Run Time (hrs:min) 3:39
Dump Time (hrs:min)0:00 0:00 0:00
Output Size (meg) 0.00.00.0
Original Size (meg) 0.00.00.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped0 0 0
Avg Dump Rate (k/s) -- -- --
Tape Time (hrs:min)0:03 0:00 0:03
Tape Size (meg) 361.20.0 361.2
Tape Used (%) 2.20.02.2 (level:#disks ...)
Filesystems Taped14 0 14 (1:13 3:1)
Avg Tp Write Rate (k/s) 2408.5-- 2408.5
USAGE BY TAPE:
LabelTime Size %Nb
Progeny01-0043 0:03 361.22.214
Progeny01-0044 0:00 0.00.0 0
There are two amanda directories in my holding area:
20050420 is 28G
20050421 is 3.2G
How do I get amanda to readjust what gets backed up when to make it more
even over the full tape cycle? Is it possible with amadmin? When I do an
amadmin balance, I get this:
due-date #fsorig KB out KB balance
--
4/21 Thu3 37603890 26119168 +330.0%
4/24 Sun1 14116820 14116820 +132.4%
4/25 Mon1 12436100 11076062+82.3%
4/26 Tue174643407464340+22.9%
4/30 Sat2 12133630 11246536+85.1%
5/01 Sun3 536890 452308-92.6%
5/02 Mon251054604698652-22.7%
5/03 Tue6 128834408596350+41.5%
5/04 Wed4 104575608194571+34.9%
5/07 Sat3 24676810 23458226 +286.2%
--
TOTAL 26 137414940 115423033 6074896
(estimated 19 runs per dumpcycle)
(3 filesystems overdue, the most being overdue 146 days)
which doesn't seem balanced at all.
Vicki
Restart back at tape 0038
I am using amanda and have been happy with it, but now there is a 20GB dump in my holding area and amanda has repeatedly tried flush it to tape and flushed nothing to the tape. I am taking care of the 20GB dump file, but I now need to reset amanda to expect tape 0038 which had no data dumped to it. How do I do this? I am getting ready to split up my amanda config to prevent the problem in the future, although truthfully, I do not understand how this problem arose. I am investigating and will likely have a question about it in the future. Vicki
Re: Why include parent directory in disklist?
Jon LaBadie wrote: On Wed, Apr 06, 2005 at 04:07:42PM +0100, Dave Ewart wrote: On Wednesday, 06.04.2005 at 10:03 -0400, Vicki Stanfield wrote: Thanks Dave, So then if /net/mysystem/home is on the same partition as /net/mysystem/username, is /net/mysystem/username being backed up twice. It sounds to me like it is. Erm, not sure, because /net/mysystem/username is not mentioned in your disklist. From the above entries, /net/mysystem/username lies under / and would thus be backed up. /net is in my experience used for nfs style mounts. Thus it would not be backed up with a DLE of /. If on the otherhand you had a DLE of /net/mysystem (mysystem being the amanda server), which would correspond to the / directory of the myserver, then some double backups might be getting done. If you are doing indexing (most amanda sites are), then you can look at the index files and see what files are being backed up. As to the 131 day old DLE without a backup, I suspect it is being reported in your daily reports as "failed" or some such. Due to bad administration I have a few such beasts. Some home remodeling caused me to shutdown a couple of systems and store them in a closet. They have not been brought online since and I failed to adjust amanda's config. This can also happen with laptops that keep missing the backup window, multi-OS-boot computers where one OS never seems to get used when amanda runs, or following some kind of reorg of file systems and an old DLE corresponds to a file system or directory that no longer exists. Is there a way to trace the output of the amadmin to a specific disklist entry? Like I see the data for the 4/06 backup at +384.2%, but I am unsure what is to be backed up on 4/06. How do I know what Amanda plans to backup on this day? I also see a big zero in the balance for 4/08. What is that about? I know that I can force a full backup, but I'd rather let Amanda balance the load. I just need to understand what disklist entries are being backed up when. Vicki Vicki
Why include parent directory in disklist?
I am still trying to understand and improve my inherited amanda configuration. I notice that in the disklist, there are parent directories listed before their children. Here is an example: myhost/ comp-low-tar myhost/usrcomp-low-tar myhost/varcomp-med-tar myhost/net/myhost/home comp-med-tar Does Amanda backup the subdirectories of an item listed in the disklist? And does that mean that I am backing up /usr twice? I suspect that the answer is no, but I would like to confirm this. I also notice that some things in the disklist sometimes show up in amadmin balance as being extraordinarily overdue (like 131 days). due-date #fsorig KB out KB balance -- 4/06 Wed5 56836750 46326642 +384.2% 4/07 Thu2 13680830 13680830+43.0% 4/08 Fri0 0 0 --- 4/09 Sat379283506604199-31.0% 4/10 Sun390110809011080 -5.8% 4/11 Mon5 141177008890351 -7.1% 4/12 Tue6 11165980 10039560 +4.9% 4/13 Wed6 110576709254644 -3.3% 4/16 Sat893263508127764-15.1% 4/17 Sun4 11941010 10722435+12.1% 4/18 Mon1 14422360 14422360+50.7% 4/19 Tue3 14011020 11719132+22.5% 4/20 Wed6 112328508034076-16.0% 4/23 Sat 32 10841180 10841180+13.3% 4/24 Sun1 14116820 14116820+47.5% -- TOTAL 85 209689950 181791073 9567951 (estimated 19 runs per dumpcycle) (3 filesystems overdue, the most being overdue 131 days) I am suspicious that it is the parent directories that are so much overdue, but I don't know how to confirm what is overdue. Additionally, I am looking for a way to determine which disklist item is going to be fully backed up when. They don't seem to correspond that way. Is there an option to amadmin that I missed? Vicki
Re: Adding new stuff to disklist
Peter Kunst wrote: Vicki Stanfield wrote: I discovered that some things on a remote machine were not being backed up. Some things on that same machine were but not everything that should be. that stuff that wasn't backed up, was there an entry (DLE) of it in the disklist of your current config ? No, not until I added it. i read "NOTE:" in front of those lines. nothing to wonder about, those dirs will be created on the next amanda amdump run. "FATAL", "ERROR" or the like would be much more bad Thanks. That clears it up. I didn't expect to have a file on my amanda host to correspond to the remote files being backed up. I need to read up on the Amanda client package I guess. I obviously have another knowledge gap when it comes to how amanda works. I expected that it would go to the system in question with ssh and tar across the necessary stuff. Evidently I am wrong. Can someone give me a brief explanation of the process that amanda goes through to back up a remote system? using tar dumptypes or (ufs)dump dumptypes, amanda connects to amandad of a client, runs gtar, dump/ufsdump locally and streams output of the client to your amanda server host. for Windows shares, amanda uses smbclient, which can run on your amanda server or another amanda client which connects to the Windows share and puts it onto your amanda server. Thanks again. That is pretty clear. amanda takes some time to understand. for me, it tooks around 6 weeks, and i still do not understand how amanda schedules fulls/incrementals. I DO know that amanda does a very good job here. Peter Thanks very much. It makes sense to me now. I feel like Amanda is doing a good job, but I think there is a lot more to know. ;-) Vicki
Adding new stuff to disklist
I discovered that some things on a remote machine were not being backed up. Some things on that same machine were but not everything that should be. I tried to add the lines to the disklist by copying the ones that were there and editing them. I then ran amcheck and got this: NOTE: info dir /var/lib/amanda/Progeny01/curinfo/pollux/_net_pollux_project_new-ps-archive_50: does not exist NOTE: index dir /var/lib/amanda/Progeny01/index/pollux/_net_pollux_project_new-ps-archive_50: does not exist NOTE: info dir /var/lib/amanda/Progeny01/curinfo/pollux/_net_pollux_project_new-ps-archive_49: does not exist NOTE: index dir /var/lib/amanda/Progeny01/index/pollux/_net_pollux_project_new-ps-archive_49: does not exist I obviously have another knowledge gap when it comes to how amanda works. I expected that it would go to the system in question with ssh and tar across the necessary stuff. Evidently I am wrong. Can someone give me a brief explanation of the process that amanda goes through to back up a remote system? For the record, the lines I added to disklist look like this: pollux /net/pollux/project/new-ps-archive/49 med-tar pollux /net/pollux/project/new-ps-archive/50 med-tar /net/pollux/project/new-ps-archive/49 is an existing directory on pollux as is 50. What am I missing here? Vicki
Re: Determining appropriate backup strategy
Jon LaBadie wrote: On Thu, Mar 31, 2005 at 03:47:51PM -0500, Vicki Stanfield wrote: P.S. I think I asked before but didn't get an answer. Is there a source for an explanation of the different priorities of backups. I have the following defined from an inherited amanda.conf file: always-full (obvious) comp-low-tar comp-med-tar comp-high-tar low-tar med-tar high-tar comp-low-dump comp-med-dump comp-high-dump low-dump med-dump high-dump The explanations in the amanda.conf are insufficient. I need to know what decisions amanda makes when determining how often to back up each dumptype. A good additional place to read is the man pages. Under priority this appears: priority "string" Default: medium. When there is no tape to write to, Amanda will do incremental backups in priority order to the holding disk. The priority may be high(2), medium(1), low(0) or a number of your choice. So the difference appears important only when saving ONLY to holding disk. Then you can define which come earlier or later incase the holding disk fills. I think what I am really talking about is the dumptype. I have the ones I listed defined. I think I might be confusing them with priority. I suppose the dumptype could only mean level of compression. But then I need more information about how Amanda decides what to back up when. I'll look back to the amanda.org site. Vicki
Determining appropriate backup strategy
I am using amanda to back up my servers. I have the following situation: There is about 42G of data to be backed up on one particular machine. This data is mostly static data in a directory structure which creates a new directory whenever 1000 files are in the current directory and numbers the new directory incrementally. It looks like this: /project/newarchive/0 contains files 0-999 /project/newarchive/1 contains files 1000-1999 /project/newarchive/1 contains files 2000-2999 etc. The previous directories are necessary for the program to run successfully but will remain static once the next directory is created. I need a backup solution that is appropriate for this. What is appropriate for this situation? Vicki P.S. I think I asked before but didn't get an answer. Is there a source for an explanation of the different priorities of backups. I have the following defined from an inherited amanda.conf file: always-full (obvious) comp-low-tar comp-med-tar comp-high-tar low-tar med-tar high-tar comp-low-dump comp-med-dump comp-high-dump low-dump med-dump high-dump The explanations in the amanda.conf are insufficient. I need to know what decisions amanda makes when determining how often to back up each dumptype.
huge file showing up
I have noticed that two large files which are of type GNU tar archive and named in the format: hostname._net_hostname_home_user.20041221.2 (exactly matches an entry in disklist except that the slashes in the path are now underscores and the date and another number are appended) I assume that Amanda is creating these files, and that I just don't completely understand the process. My backups which begin at 2:00 AM are still running when I get here at 8:30. They used to finish much earlier (like 5:00 or so). I looked through the faq on amanda.org and didn't see anything about this. Maybe there is a step by step explanation of the process amanda goes through somewhere? Thanks, Vicki
Using more tapes than before
I have recently changed my Amanda config (upgraded to 2.4.4p4 and added
a second tape drive - chg-multi) and now the backups require more tapes
than before. I hate to waste the space if this is just bad configuration
on my part. I have looked at the faq but didn't see this problem
addressed even though I am sure it has been asked on the list before.
Here is my amanda.conf:
=
# for an explanation of the parameters refer to amanda(8)
org "Progeny01" # your organization name for reports
mailto "[EMAIL PROTECTED]" # list of operators at your site
dumpuser "backup" # the user to run dumps under
inparallel 8# maximum dumpers that will run in parallel
#netusage 1 # maximum net bandwidth for Amanda, in KB/s
netusage 180 Mbps # maximum net bandwidth for Amanda, in KB/s
dumporder ""
taperalgo largestfit
# a filesystem is due for a full backup once every days
dumpcycle 19 days # the number of days in the normal dump cycle
tapecycle 43 tapes # the number of tapes in rotation
# amanda will fail to back up some stuff if we don't give it enough
# time to estimate the big stuff. lets give it eight hours. i will
# specify in a negative (-) number which sets a hard limit. a
# positive number would be per disk...
etimeout -28800
# gnu tar takes a long time to get started when there are lots of
# files sitting around. the normal data timeout is thrity
# minutes, bumped to one hour.
dtimeout 7200
bumpsize 100 MB # minimum savings (threshold) to bump level 1 ->
2
bumpdays 2 # minimum days at each level
bumpmult 4 # threshold = bumpsize * (level-1)**bumpmult
#Added by Vicki to configure for two tape drives
runtapes 2 #number of tapes to be used in a single run
tpchanger "chg-multi" #tape-changer glue script
changerfile
"/usr/local/src/amanda-2.4.4p4/etc/amanda/Progeny01/chg-multi.conf"
tapedev "mychanger" # the non-rewinding device to write tapes
amrecover_changer "mychanger"
amrecover_check_label yes
amrecover_do_fsf yes
tapetype HP-C5683A # what kind of tape it is (see tapetypes below)
# label constraint regex: all tapes must match
labelstr "Progeny01-[0-9][0-9][0-9][0-9]*$"
holdingdisk disk1 {
comment "Primary holding disk"
directory "/var/spool/amanda"
use 35650 MB
chunksize 1024 MB
}
infofile "/var/lib/amanda/Progeny01/curinfo"# database filename
logfile "/var/log/amanda/Progeny01/log"# log filename
indexdir "/var/lib/amanda/Progeny01/index" # index files
# our tape definition
define tapetype HP-C5683A {
comment "DDS-4 DAT tape"
length 16584 mbytes
filemark 452 kbytes
speed 2613 kbytes
}
# dumptypes
#
# These are referred to by the disklist file. The dumptype specifies
# a number of options.
define dumptype always-full {
comment "full dump of this filesystem always"
options no-compress
priority high
dumpcycle 0
maxcycle 0
}
define dumptype comp-low-tar {
program "GNUTAR"
comment "compressed low priority tar of filesystem"
options compress-fast, index, exclude-list ".amanda-exclude"
priority low
}
define dumptype comp-med-tar {
program "GNUTAR"
comment "compressed medium priority tar of filesystem"
options compress-fast, index, exclude-list ".amanda-exclude"
priority medium
}
define dumptype comp-high-tar {
program "GNUTAR"
comment "compressed high priority tar of filesystem"
options compress-fast, index, exclude-list ".amanda-exclude"
priority high
}
define dumptype low-tar {
program "GNUTAR"
comment "low priority tar of filesystem"
options no-compress, index, exclude-list ".amanda-exclude"
priority low
}
define dumptype med-tar {
program "GNUTAR"
comment "medium priority tar of filesystem"
options no-compress, index, exclude-list ".amanda-exclude"
priority medium
}
define dumptype high-tar {
program "GNUTAR"
comment "high priority tar of filesystem"
options no-compress, index, exclude-list ".amanda-exclude"
priority high
}
define dumptype comp-low-dump {
comment "compressed low priority dump of filesystem"
options compress-fast, index
priority low
}
define dumptype comp-med-dump {
comment "compressed medium priority dump of filesystem"
options compress-fast, index
priority medium
}
define dumptype comp-high-dump {
comment "compressed high priority dump of filesystem"
options compress-fast, index
priority high
}
define dumptype low-dump {
comment "low priority dump of filesystem"
options no-compress, index
priority low
}
define dumptype med-dump {
comment "medium priority dump of filesystem"
options no-com
Re: How chg-multi selects which tape to backup to.....
On Mon, 2005-02-14 at 12:07 -0500, Jon LaBadie wrote: > Vicki, > just an off-list comment. > > I don't think chg-multi is used by a large number of installations. > I could certainly be wrong there. But if correct, I would not be > at all surprised that you may be wringing out a few sticky points. > (aka defects, less politely aka bugs). > Good for me then, huh? It really is all I need where I am since this is a small company. No sense spending the money for an actual changer. Vicki
How chg-multi selects which tape to backup to.....
I am using chg-multi with two tape drives. Sometimes my backups only require one tape, other times both. After the weekend, I use amtape current to determine which tape drive is current and put the first of the two tapes asked for by amcheck in it. It doesn't always seem, however, that amanda is writing to the next tape in sequence in the tapelist file. I would appreciate confirmation that "amtape DailySet current" is intended to specify which tape drive amanda expects to search for the next tape, as specified in tapelist. example: tape 0001 and 0002 were in last night with 0001 in /dev/nst0 and 0002 in /dev/nst1. I had some stuff in the holding area from the weekend, so I ran amflush to the first tape this morning. Now amanda says: The dumps were flushed to tape DailySet-0001. The next 2 tapes Amanda expects to used are: DailySet-0002, DailySet-0003. "amcheck DailySet current" returns the following: amtape: scanning current slot in tape-changer rack: slot 1: chg-multi: slot is empty I expected amanda to look in /dev/nst1 and see tape DailySet-0002. Do I have to move tape DailySet-0002 to /dev/nst0? Vicki
Re: Running multiple instances of amanda
I got a little further on my attempts to run backups via the new install. It appears to backup using the new binary except that the changes to /etc/inetd.conf seem to have caused some of the backups not to work. For some reason, amcheck is reporting that there is no .amanda-exclude file on each entry in the amanda disklist file (for a certain machine at least). That machine is indeed up and that file doesn't exist there, but it didn't when I was running 2.4.2p2-4 either and I never got these errors. Did something significant change between those versions in this area? The output from the dump last night indicated that these backups had failed. ERROR: morimoto: [Can't open exclude file '/net/morimoto/client/.amanda-exclude': No such file or directory] ERROR: morimoto: [Can't open exclude file '/net/morimoto/project/gelato/.amanda-exclude': No such file or directory] ERROR: morimoto: [Can't open exclude file '/net/morimoto/project/redhat/redhat-updates/.amanda-exclude': No such file or directory] And more. However some on a different filesystem did succeed. What am I missing here? Vicki
Re: Running multiple instances of amanda
On Wed, 2005-02-02 at 15:38 +, Gavin Henry wrote: > -- > Kind Regards, > > Gavin Henry. > Managing Director. > > T +44 (0) 1467 624141 > M +44 (0) 7930 323266 > F +44 (0) 1224 742001 > E [EMAIL PROTECTED] > > Open Source. Open Solutions(tm). > > http://www.suretecsystems.com/ > > > morimoto:/usr/local/src/amanda-2.4.4p4/sbin# ./amrecover -C Progeny01 > AMRECOVER Version 2.4.4p4. Contacting server on morimoto.progeny.com ... > 220 morimoto AMANDA index server (2.4.2p2) ready. > > 500 Access not allowed: [access as backup not allowed from > > [EMAIL PROTECTED] amandahostsauth failed > > > > See the FAQs Top Ten: > > http://www.amanda.org/docs/topten.html#id2518896 > > Particularly: > > .amandahosts > Okay, if I add morimoto.progeny.com root to /var/backups/.amandahosts, I can use the amrestore command, but I don't see the backups I have done with the new version of amanda, only the stuff backed up before I started using the new version. Where does amanda store information about what gets backed up and when it was and to which tape? I thought the answer was in /var/lib/amanda, but I don't see anything with dates, so I am not sure where amanda stores this information. Google hasn't produced the answer as yet, although I am still looking. Vicki
Running multiple instances of amanda
I am testing a new version of amanda before putting it into play here at work. The old version was 2.4.2p2. I can do the backups fine (I think), but when I try to check them with amrecover, I get the following error: morimoto:/usr/local/src/amanda-2.4.4p4/sbin# ./amrecover -C Progeny01 AMRECOVER Version 2.4.4p4. Contacting server on morimoto.progeny.com ... 220 morimoto AMANDA index server (2.4.2p2) ready. 500 Access not allowed: [access as backup not allowed from [EMAIL PROTECTED] amandahostsauth failed I am logging in as root, although I tried logging in as backup also. I checked the previous binary and the permissions and ownership are the same. To make this more clear, I have /etc/amanda/Progeny01 and /usr/sbin/am* from the 2.4.2p2 build and /usr/local/src/amanda-2.4.4p4/etc/amanda/Progeny01 and /usr/local/src/amanda-2.4.4p4/sbin/am* from the 2.4.4p2 build. What do I need to do to specify that I want the new one? Vicki
Re: Problem with chg-multi
On Mon, 2005-01-24 at 15:51 +0100, Paul Bijnens wrote: > Vicki Stanfield wrote: > > Are you saying that there should be a line like this in amanda.conf? > > changerfile "chg-multi.conf" > > Yes, indeed. In my config, I have entered the full pathname, instead > of a relative. > I found it. There was an added character on the tape definition. Now everything makes sense except for how it can tell me what is in a file and then deny that the file exists! Oh well, it appears to be ready to backup now. Thanks for the help. Vicki
Problem with chg-multi
I am reconfiguring my amanda setup to use two tape drives. I have
made what I believed to be the appropriate changes to my amanda
configuration, added the second drive to the scsi bus, and made the
appropriate devices (/dev/nst2, etc). The differences between the old
amanda.conf and the new are these:
< #Added by Vicki to configure for two tape drives
< tpchanger "chg-multi"
<
< tapedev "/dev/nst{1,2}" # the non-rewinding device to write
tapes
---
> tapedev "/dev/nst1" # the non-rewinding device to write
> tapes
52c49
< define tapetype HP-C5683Ax2 {
---
> define tapetype HP-C5683A {
When I run the amcheck command, I get the following error:
"/etc/amanda/Progeny01/amanda.conf", line 36: tapetype HP-C5683A not
defined
amcheck: could not find config file "/etc/amanda/Progeny01/amanda.conf"
Obviously, I have done something wrong here. Can I buy a clue?
Vicki
Re: Data loss if configuration of amanda.conf is not right.
On Wed, 2005-01-19 at 10:38 -0500, Jon LaBadie wrote: > On Wed, Jan 19, 2005 at 09:57:22AM -0500, Vicki Stanfield wrote: > > > > Thanks. I was unsure of whether this was a potential problem or not. As > > long as Amanda tells me that there was a problem and dumps to the spool, > > I am happy. > > Make sure your "holding disk" space is sufficiently large and > that your "reserve" parameter is set to some value smaller > than the default 100%. The reserve aspect is to reserve the > holding disk for incrementals only when there are tape drive > problems. If your holding disk is sufficiently large, you > can also hold some full backups so the reserve should be > decreased. I have enough holding disk for several days > of normal dumps (> 1wk) and use a reserve of 20%. > > jl I'm not sure that I correctly understand the reserve parameter. I have heard that it is used to get amanda to degrade full dumps to merely incremental dumps if the holding disk gets full. I assume the percentage refers to how full. So how does degrading to incrementals help if the holding disk is 100% full? What am I misunderstanding here? I can understand saying that if the holding disk is 75% full, switch to incrementals though. Vicki
Re: Data loss if configuration of amanda.conf is not right.
On Tue, 2005-01-18 at 10:10 -0500, Jon LaBadie wrote: > On Tue, Jan 18, 2005 at 09:39:56AM -0500, Vicki Stanfield wrote: > > I have set up an alternate amanda.conf to run with two tape drives > > instead of the one that I currently have. I am wondering whether there > > is some way to rerun an incremental backup if I run it and something in > > my new configuration is out of whack. Since there is stateful data > > involved which presumably would not be available after the backup ran, > > is there a way to get it back so that problem could be corrected and the > > backup rerun? > > In trying to think of potential problems, I don't really come up with > any which would cause amanda to think it has had a successful run > when it failed unless some component was "lying" about its success; > e.g. a tape system saying it was accepting the data when the tape > was sitting there broken and spinning. So I don't think the stateful > data would reflect a false situation. Maybe you or someone else > would think of other scenarios than I've come up with. > > Although amdump, which starts a backup, is normally run from a cron job, > there is no reason why you can't start it manually if the cron job fails. > > jl Thanks. I was unsure of whether this was a potential problem or not. As long as Amanda tells me that there was a problem and dumps to the spool, I am happy. Vicki
Data loss if configuration of amanda.conf is not right.
I have set up an alternate amanda.conf to run with two tape drives instead of the one that I currently have. I am wondering whether there is some way to rerun an incremental backup if I run it and something in my new configuration is out of whack. Since there is stateful data involved which presumably would not be available after the backup ran, is there a way to get it back so that problem could be corrected and the backup rerun? Vicki
Re: Amanda config for two drives
On Thu, 2005-01-13 at 15:50 -0500, Jon LaBadie wrote: > On Thu, Jan 13, 2005 at 12:06:56PM -0500, Vicki Stanfield wrote: > > > I would be amazed that amcheck did not complain :-) > > > > It hasn't yet been given the chance to. :-) > > Should be the first thing after any config changes. If you are ready to deploy. In this case the other tape drive was on order, so I made the changes in a copied config file. > > > > > Okay, so I knew that I needed to use the chg-multi changer, but I was > > unsure as to how. I have found some more information, but I don't know > > where to get the script. I saw reference to it in an example directory > > somewhere. I can't find it though. > > It is an included component of an amanda install. In my case it installed > in /usr/local/libexec. YMMV > > jl Thanks. I found it now. Now the follow on question is: is there a way to tell amanda to rerun the backup that it just ran just in case it doesn't work? Or is it as simple as running the backup command again? I assume that it changes the states of the backups each time to prevent the full backup of everything in the disklist. If the state information has been updated, is there a way to rerun exactly what was run before? Thanks, Vicki
Re: Amanda config for two drives
On Thu, 2005-01-13 at 17:22 +0100, Paul Bijnens wrote:
> Vicki Stanfield wrote:
> > I am setting up to transition an existing Amanda config on a Debian
> > Woody Linux system with one drive to a two drive setup. I have been
>
> Use the chg-multi changer: it emulates a tape changer using
> two or more tape drives.
>
>
> > trying to research this and have come up with what I believe is the
> > correct amanda config:
> >
> >
> >
> > tapedev "/dev/nst{1,2},2}" # the non-rewinding device to write
>
> What's that???
Evidently I wasn't paying attention when I edited my working file. I
meant this:
tapedev "/dev/nst{1,2}" # the non-rewinding device to write
>
>
>
> > tapes
> >
> >
> > # our tape definition
> > define tapetype HP-C5683Ax2 {
> > comment "DDS-4 DAT tape"
> > length 16584 mbytes
> > filemark 452 kbytes
> > speed 2613 kbytes
> >
>
> I also believe you have hardware compression on, together with
> software compression. A capacity of about 16 Gbyte instead of a little
> lower than 20 Gbyte is typical for those setups.
> Doing hard+soft compression results in a lower tapecapacity for
> most drives (all except LTO drives, AFAIK).
>
> Disable hardware compression. Search the list for how to do this,
> as this has been discussed numerous times here.
>
>
> > While I believe that this setup is correct, ...
>
> I would be amazed that amcheck did not complain :-)
It hasn't yet been given the chance to. :-)
Okay, so I knew that I needed to use the chg-multi changer, but I was
unsure as to how. I have found some more information, but I don't know
where to get the script. I saw reference to it in an example directory
somewhere. I can't find it though.
Vicki
Amanda config for two drives
I am setting up to transition an existing Amanda config on a Debian
Woody Linux system with one drive to a two drive setup. I have been
trying to research this and have come up with what I believe is the
correct amanda config:
tapedev "/dev/nst{1,2},2}" # the non-rewinding device to write
tapes
# our tape definition
define tapetype HP-C5683Ax2 {
comment "DDS-4 DAT tape"
length 16584 mbytes
filemark 452 kbytes
speed 2613 kbytes
While I believe that this setup is correct, I have not gotten a concrete
picture of how this will affect my backups. What I would like to see
happen is that after the first tape is full, instead of sending
subsequent backup data to the spooling area, it sends it to the second
tape drive. When the second tape drive is too full, I would like further
backup data to go to the spooling area. Is this the default setup or do
I need to change something else?
Thanks,
Vicki
dumps delayed
I am running Amanda on a network that I administer. I don't have a lot of Amanda experience, having always used afio scripts that I wrote. I inherited this backup config and am trying to get up to speed on it. I see several of these messages stating that the dump is being delayed. Obviously I want to ensure that the dump eventually makes it to tape. Is there an amanda command to tell me this? planner: Dump larger than tape: full dump of +mymachine:/home delayed. Vicki
