RE: Please help -- planner askfor: lev out of range -1..10: 10
This is just for the archives, so someone in the future can find the fix. No need to respond. > From: Daniel Lorenzini [mailto:[EMAIL PROTECTED]] > > Check to see if any of your dumps is at level 9. If so, > force a level 0. This is a planner bug. That turned out to be the problem. I looked through my the daily email reports from amanda, and found that in the DUMP SUMMARY that one disk had been at level 9 for many days: HOSTNAME DISKL ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s -- - beowulf -a/mm5/2002 9 42346884234688 --4:4714754.8 14:564724.8 After issuing the following command, su amanda -c "amadmin Daily force beowulf /data/mm5/2002" the next night amanda ran successfully. I still have to work out the problems associated with having too many disks to dump, but at least amanda is running again. Thanks to all those who responded and helped me wrangle with amanda! Bart -- Bart Brashers MFG Inc. Air Quality Meteorologist 19203 36th Ave W Suite 101 [EMAIL PROTECTED]Lynnwood WA 98036-5707 http://www.mfgenv.com 425.921.4000 Fax: 425.921.4040
RE: Please help -- planner askfor: lev out of range -1..10: 10
--On Monday, April 22, 2002 11:42:46 -0600 "Brashers, Bart -- MFG, Inc."
<[EMAIL PROTECTED]> wrote:
> Hi Frank, thanks for your response!
>
>> To summarize what I think you've said, you are trying to back up
>> ~200GB on a 40GB tape, and are dynamicly rewriting your disklist
>> entries into ~25GB chunks. You don't say how many tapes you have
>> in your runtapes and tapecycle, or if you have some kind of changer
>> configured (even if it is chg-manual).
>
>> > The relevant parts of my amanda.conf file are:
>> >
>> > dumpcycle 3 weeks
>> > runspercycle 15
>> > tapecycle 17 tapes
>> > define dumptype mydumptype {
>> > program "GNUTAR"
>> > compress none
>> > } (plus more of the usual options)
>
> runtapes 1
> tapedev "/dev/nst0"
>
> I'm trying to avoid using chg-manual, and let amanda run on 1 tape per
> night. I'd like to be able to use my tape drive for other non-amanda tasks
> during the day ;-).
You could still use it during the day, as long as amdump or amcheck
weren't trying to use it. If you really don't want to set up a changer,
and if you have enough holding disk, you could lie to Amanda about
your tapetype and say it is 80GB, then planner will dump up to that
much. You will get a tape error in your daily report (since it will
hit EOT around 40GB), but you can then amflush it whenever it is
convenient.
>>> FAILURE AND STRANGE DUMP SUMMARY:
>>> beowulf/home/proj/mm5 lev 1 FAILED [dump larger than tape, skipping
>>> incremental]
>>> beowulf/var/mail lev 1 FAILED [dumps way too big, must skip
>>> incremental dumps]
>>> (and so on, for ~30 or so directories)
>>
>> This says an incremental of /home/proj/mm5 is >40GB so it can't write
>> it to tape. The /var/mail error (dumps way too big) indicates that
>> the day's backup can't fit on the tapes allotted to it, so Amanda is
>> having to skip backups of those filesystems.
>
> Ok, I'm confused. Each of these errors "dump larger than tape" and "dumps
> way too big" refers not to the overall amanda run, BUT TO THAT PARTICULAR
> DISK? That should be impossible. Each night _before_ amanda is run, my
> ammakelist script would break up /home/proj/mm5 into sub-directories that
> are each less than 25Gb. So it's impossible for the lev 1 to be larger than
> 25Gb, and it should fit on a tape. Not necessarily this tape, since the
> amount to be written on any one night is generally larger than my
> tapedrive...
>
> [root]% du -h /var/spool/mail/
> 360k/var/spool/mail
>
> There's no way the lev 1 on /var/spool/mail is larger than a tape. So these
> errors "dump larger than tape" and "dumps way too big" must refer to the
> total amount dumped to tape.
"dumps way too big" does refer to total backup size, but the other
warning, "dump larger than tape" refers to the individual filesystem.
Check the size of /home/proj/mm5. Perhaps its size changed between
the time your disklist was created and when amdump ran.
>> Increasing runtapes should help (although you may have to adjust
>> tapecycle and runspercycle unless you have extra tapes).
>
> So my idea that I should add tapes (I recently went from 12 to 17) was
> heading in the right direction. I have extra tapes in-house. Can I go to
>
> dumpcycle 4 weeks
> runspercycle 20
> tapecycle 22 tapes
> runtapes 1
>
> or do I have to go to something like
>
> dumpcycle 2 weeks
> runspercycle 10
> tapecycle 22 tapes
> runtapes 2
Unless you can use compression to get more on a tape, it will take
quite a while to get everything backed up if you only use one tape
(5 or 6 days to do the first set of fulls without any incrementals).
I prefer having more than one full backup around, so I would prefer
the shorter dumpcycle. If your data doesn't change too much it might
only require two tapes for the first couple of weeks and then it
will level out and fit on one tape again. Definitely look into
compression, some data (especially text) compresses amazingly well.
If you are using hardware compression you may be able to increase
the size in your tapetype if you are not getting EOT errors.
>
Frank
--
Frank Smith[EMAIL PROTECTED]
Systems Administrator Voice: 512-374-4673
Hoover's Online Fax: 512-374-4501
RE: Please help -- planner askfor: lev out of range -1..10: 10
> > Thanks for your responses, Jon and Paul, > > I don't think going to 'runtapes 2' in amanda.conf is going > to fix the "lev > out of range" error, since that part happens long after my > error. The email > message starts out with > > FAILURE AND STRANGE DUMP SUMMARY: > planner: FATAL error [planner askfor: lev out of range -1..10: 10] > beowulf/usr/local RESULTS MISSING > beowulf/var/mail RESULTS MISSING > > and there's nothing in /tmp/amanda/dump/20020419/ and the > like. The planner > fails, so nothing is ever gtar'd, so there's nothing to write > to tape(s). > > Any ideas on how to clear the "lev out of range" error? Then > I'll tackle > the other errors stemming from having way to much data to > backup compared to > my tapesize. > Bart, increasing the number of tapes should address this problem, because the planner is trying to figure out what level to run for each backup, and for unknown reasons is bouncing that backup down to a lower level than dump can handle. (Or one part of planner understands that 10 is the limit and another thinks that 9 is the limit.) Additional tape space will allow planner to go with a higher level for that (and other) backups. Paul
Re: Please help -- planner askfor: lev out of range -1..10: 10
To summarize what I think you've said, you are trying to back up
~200GB on a 40GB tape, and are dynamicly rewriting your disklist
entries into ~25GB chunks. You don't say how many tapes you have
in your runtapes and tapecycle, or if you have some kind of changer
configured (even if it is chg-manual).
From the looks of the errors, Amanda is trying to do full
dumps and your disklist isn't doing what you think:
>> FAILURE AND STRANGE DUMP SUMMARY:
>> beowulf/home/proj/mm5 lev 1 FAILED [dump larger than tape, skipping
>> incremental]
>> beowulf/var/mail lev 1 FAILED [dumps way too big, must skip
>> incremental dumps]
>> (and so on, for ~30 or so directories)
This says an incremental of /home/proj/mm5 is >40GB so it can't write
it to tape. The /var/mail error (dumps way too big) indicates that
the day's backup can't fit on the tapes allotted to it, so Amanda is
having to skip backups of those filesystems.
If this was a formerly working setup, best guess is that the daily
volume of data to backup has exceeded your capacity (~40GB x runtapes).
Increasing runtapes should help (although you may have to adjust
tapecycle and runspercycle unless you have extra tapes).
I would also suggest splitting up your directories to under 20GB
instead of 25GB. If it writes one 25GB dump to tape and then starts
on another level 0, it will write 15 GB and hit EOT, and have to
start over on another tape. With a smaller directory size it will
waste less space on each tape.
Good luck,
Frank
--On Monday, April 22, 2002 09:52:10 -0600 "Brashers, Bart -- MFG, Inc."
<[EMAIL PROTECTED]> wrote:
>
> I didn't get any responses from my posting last Thursday, so I'm guessing
> I'm really up the creek on this one.
>
> If anyone has any suggestions as to how to fix this problem, please let me
> know. I've now gone 4 days without a backup, and I'm afraid I'm going to
> have to delete everything and start from scratch!
>
> Thanks,
>
> Bart
>
>
>
> After many months of operation (THANK YOU to all the developers!) last
> night's amanda Daily run gave me this error:
>
> FAILURE AND STRANGE DUMP SUMMARY:
> planner: FATAL error [planner askfor: lev out of range -1..10: 10]
> beowulf/usr/local RESULTS MISSING
> beowulf/var/mail RESULTS MISSING
> (and so on, for every element of the disklist)
>
> The only debug files from this (9 PM) run were
> /tmp/amanda.amtrmlog.*.debug
> /tmp/amanda/amtrmidx.*.debug
> but they showed nothing particularly interesting or helpful. The latter
> said
> could not open index directory "/var/lib/amanda/Daily/index/db02/_root/"
> but that's because I recently added that machine-directory combination
> (oops, forgot it) and amanda hasn't managed to schedule a full dump, and
> won't incrementally dump a new disk.
>
> The relevant parts of my amanda.conf file are:
>
> dumpcycle 3 weeks
> runspercycle 15
> tapecycle 17 tapes
> define dumptype mydumptype {
> program "GNUTAR"
> compress none
> } (plus more of the usual options)
>
> I searched the archive, but could not find anything directly relevant. This
> "lev out of range" error happened to me once before, when I was first
> learning and setting up amanda. That time, I just deleted all the logs and
> tapelists and everything, and started from scratch. I would like to avoid
> repeating that.
>
> The only other (possibly) relevant detail is that I constantly get errors
> like:
>
> FAILURE AND STRANGE DUMP SUMMARY:
> beowulf/home/proj/mm5 lev 1 FAILED [dump larger than tape, skipping
> incremental]
> beowulf/var/mail lev 1 FAILED [dumps way too big, must skip
> incremental dumps]
> (and so on, for ~30 or so directories)
>
> I've written a script that runs (via crontab) every afternoon and
> re-generates the disklist. It starts with a list of directories (/etc
> /var/log /var/lib /usr/spool /root /home /data). For every directory it
> finds that contains more than 25Gb, it recursively breaks it up and adds the
> sub-directories to the disklist. This is because /home is currently 153Gb,
> and would never get dumped since it's larger than a tape.
>
> I have about 200Gb to backup to a DLT8000 (40 Gb) tape drive. I recently
> went from a 2 to 3 week dumpcycle, hoping that the extra space would get rid
> of the "dumps way too big" error.
>
> Please cc me with any hints as to how to clear this up, as I only get daily
> digests and I don't want to wait until tomorrow to fix this...
>
> Bart
> --
> Bart Brashers MFG Inc.
> Air Quality Meteorologist 19203 36th Ave W Suite 101
> [EMAIL PROTECTED]Lynnwood WA 98036-5707
> http://www.mfgenv.com 425.921.4000 Fax: 425.921.4040
--
Frank Smith[EMAIL PROTECTED]
Systems Administrator Voice: 512-374-4673
Hoover's Online Fax: 512-374-4501
RE: Please help -- planner askfor: lev out of range -1..10: 10
You could at least temporarily go to 'runtapes 2' in amanda.conf to make
more room in each run. If you can afford more tapes, using two a day might
help, or using two tape drives with chg-multi might be good too.
Lengthening the tapecycle is going to take a while (about two tapecycles,
I'm guessing) to level out as you would expect. It sounds like you need more
tape now, which a second tape and/or a second drive could do.
Also, rather than regenerating the disklist every night, you might not
confuse amanda as much if you simply checked the directory structure every
night, and warned when a disklist entry is close to the limit. This would
allow you to plan splits better, and having your disk list entries change
less often is a good thing if you have to restore.
> -Original Message-
> From: Brashers, Bart -- MFG, Inc. [mailto:[EMAIL PROTECTED]]
> Sent: Monday, April 22, 2002 11:52 AM
> To: '[EMAIL PROTECTED]'
> Subject: Please help -- planner askfor: lev out of range -1..10: 10
>
>
>
> I didn't get any responses from my posting last Thursday, so
> I'm guessing
> I'm really up the creek on this one.
>
> If anyone has any suggestions as to how to fix this problem,
> please let me
> know. I've now gone 4 days without a backup, and I'm afraid
> I'm going to
> have to delete everything and start from scratch!
>
> Thanks,
>
> Bart
>
>
>
> After many months of operation (THANK YOU to all the developers!) last
> night's amanda Daily run gave me this error:
>
> FAILURE AND STRANGE DUMP SUMMARY:
> planner: FATAL error [planner askfor: lev out of range -1..10: 10]
> beowulf/usr/local RESULTS MISSING
> beowulf/var/mail RESULTS MISSING
> (and so on, for every element of the disklist)
>
> The only debug files from this (9 PM) run were
> /tmp/amanda.amtrmlog.*.debug
> /tmp/amanda/amtrmidx.*.debug
> but they showed nothing particularly interesting or helpful.
> The latter
> said
> could not open index directory
> "/var/lib/amanda/Daily/index/db02/_root/"
> but that's because I recently added that machine-directory combination
> (oops, forgot it) and amanda hasn't managed to schedule a
> full dump, and
> won't incrementally dump a new disk.
>
> The relevant parts of my amanda.conf file are:
>
> dumpcycle 3 weeks
> runspercycle 15
> tapecycle 17 tapes
> define dumptype mydumptype {
> program "GNUTAR"
> compress none
> } (plus more of the usual options)
>
> I searched the archive, but could not find anything directly
> relevant. This
> "lev out of range" error happened to me once before, when I was first
> learning and setting up amanda. That time, I just deleted
> all the logs and
> tapelists and everything, and started from scratch. I would
> like to avoid
> repeating that.
>
> The only other (possibly) relevant detail is that I
> constantly get errors
> like:
>
> FAILURE AND STRANGE DUMP SUMMARY:
> beowulf/home/proj/mm5 lev 1 FAILED [dump larger than
> tape, skipping
> incremental]
> beowulf/var/mail lev 1 FAILED [dumps way too big, must skip
> incremental dumps]
> (and so on, for ~30 or so directories)
>
> I've written a script that runs (via crontab) every afternoon and
> re-generates the disklist. It starts with a list of directories (/etc
> /var/log /var/lib /usr/spool /root /home /data). For every
> directory it
> finds that contains more than 25Gb, it recursively breaks it
> up and adds the
> sub-directories to the disklist. This is because /home is
> currently 153Gb,
> and would never get dumped since it's larger than a tape.
>
> I have about 200Gb to backup to a DLT8000 (40 Gb) tape drive.
> I recently
> went from a 2 to 3 week dumpcycle, hoping that the extra
> space would get rid
> of the "dumps way too big" error.
>
> Please cc me with any hints as to how to clear this up, as I
> only get daily
> digests and I don't want to wait until tomorrow to fix this...
>
> Bart
> --
> Bart Brashers MFG Inc.
> Air Quality Meteorologist 19203 36th Ave W Suite 101
> [EMAIL PROTECTED]Lynnwood WA 98036-5707
> http://www.mfgenv.com 425.921.4000 Fax: 425.921.4040
>
