Re: balance calculation -- was Re: separate spindle for dump use
In a recent discussion on widely out of balance dumps and
frequent promotions to level 0 dumps I noted this about
my configuration.
On Thu, Feb 13, 2020 at 11:50:34PM -0500, Jon LaBadie wrote:
...
>
> My default {dump/runsper}cycle is 7 days, set in amanda.conf.
> But more than 1/2 of my DLEs have custom dumpcycles (14,21,and
> 28 days) set in their dumptypes. You can not set custom values
> for runspercycle in dumptypes (bummer!!).
>
I decided to give amanda's planner a different scenario, long
dumpcycles and daily runspercycle to see if that would have an
effect on the balances and promotions. It seems to be working.
Using "amadmin balance", the report shows the
expected high balance over the next 28 days only exceeds 57%
one time. On that one date, 5 DLEs are scheduled to dump 350%
of the balanced amount. But the 3 days before that are well
under balance, so perhaps the planner will promote some of
the 5 DLEs.
Here are the changes I made.
In "amanda.conf", dumpcycle and runspercycle were both changed
from 7 to 28 days.
In "disklist" I had many entries that defaulted to 7 day cycles
like:
cyber.jgcomp.com Root/comp-root-tar
and many othere entries with longer dumpcycles like:
cyber.jgcomp.com Vault-Monthly/opt/vault-monthly {
comp-user-tar
dumpcycle 28
}
I made NO changes to disklist. Thus it seems like Root above
would get the default 28 day dumpcycle rather than the desired
7 day. To fix this I edited the "dumptypes" file. The dumptype
"global" is incorporated into nearly every other dumptype so
I added a dumpcycle line.
define dumptype global {
comment "Global definitions"
index yes
record yes
auth "bsdtcp"
# added jhl 2/13/2020
dumpcycle 7
}
So now, DLE Root gets its dumpcyle from the global dumptype
(7 days), Vault-Monthly overrides the global value to 28 days,
and amanda's planner uses the default value from amanda.conf,
also 28 days.
The primary downside is an administrative one. If someone
adds a new DLE they might expect it to get the default from
amanda.conf when that is actually overridden by the global
dumptype.
If I see any surprise effects from these changes I'll report
back. But so far (12 days) it has been fine.
Jon
--
Jon H. LaBadie j...@jgcomp.com
11226 South Shore Rd. (703) 787-0688 (H)
Reston, VA 20190 (703) 935-6720 (C)
Re: balance calculation -- was Re: separate spindle for dump use
On Thursday 13 February 2020 23:50:34 Jon LaBadie wrote:
> On Wed, Feb 12, 2020 at 05:40:31PM -0500, Nathan Stratton Treadway
wrote:
> > Looks like there we had some discussion here on the list back in
> > November 2018 about the unbalanced runs you were seeing, which got
> > as far as my theorizing that perhaps you had a few DLEs that were so
> > much larger than all the others that Amanda's normal algorithm for
> > adjusting the balance is unable to achieve that goal
> >
> > I don't know if there is anyone left on this list who understands
> > the inner workings of those algorithms, but if you really want to
> > get to the bottom of the problem you'll probably have to spend some
> > time combing through the balance calculation section of the planner
> > log files
>
> Asking if there is anyone left is really the purpose of my post.
> Following Gene's original thread I started playing with some
> configuration edits and "amadmin balance". Here is a
> sample of the last few lines of the report:
>
> TOTAL 18 975202 704008100572
> BALANCED 573868 311613 44516
> DISTINCT2714809641187912
> (estimated 7 runs per dumpcycle)
>
> The last value on the 2nd line above (44.5GB) is what amadmin
> considers a balanced output of level 0 dumps over the next
> 7 day dumpcycle.
>
> My default {dump/runsper}cycle is 7 days, set in amanda.conf.
> But more than 1/2 of my DLEs have custom dumpcycles (14,21,and
> 28 days) set in their dumptypes. You can not set custom values
> for runspercycle in dumptypes (bummer!!).
>
> Based on my playing, I do not think the above command considers
> custom dumpcycles in calculating a balance value. I wonder if
> the planner also fails to consider custom dumpcycles?
>
> Where is a code reader when you want one :)
>
> Jon
Excellent question Jon.
Acc my own balance report amanda will go all aglay monday and use about
1.5 tapes, then it will go bonkers promoting stuff to fail at filling
Tuesdays tape. Do I need to add a gig to the vtape? Or cut the
bumpsize_threshold in half, so I theoreticly made it easier to bump
since that 2T disk is at 88% now.
But the balance report is unaffected by any of this!
??
Copyright 2019 by Maurice E. Heskett
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
Genes Web page
balance calculation -- was Re: separate spindle for dump use
On Wed, Feb 12, 2020 at 05:40:31PM -0500, Nathan Stratton Treadway wrote:
>
> Looks like there we had some discussion here on the list back in
> November 2018 about the unbalanced runs you were seeing, which got as
> far as my theorizing that perhaps you had a few DLEs that were so much
> larger than all the others that Amanda's normal algorithm for adjusting
> the balance is unable to achieve that goal
>
> I don't know if there is anyone left on this list who understands the
> inner workings of those algorithms, but if you really want to get to the
> bottom of the problem you'll probably have to spend some time combing
> through the balance calculation section of the planner log files
Asking if there is anyone left is really the purpose of my post.
Following Gene's original thread I started playing with some
configuration edits and "amadmin balance". Here is a
sample of the last few lines of the report:
TOTAL 18 975202 704008100572
BALANCED 573868 311613 44516
DISTINCT2714809641187912
(estimated 7 runs per dumpcycle)
The last value on the 2nd line above (44.5GB) is what amadmin
considers a balanced output of level 0 dumps over the next
7 day dumpcycle.
My default {dump/runsper}cycle is 7 days, set in amanda.conf.
But more than 1/2 of my DLEs have custom dumpcycles (14,21,and
28 days) set in their dumptypes. You can not set custom values
for runspercycle in dumptypes (bummer!!).
Based on my playing, I do not think the above command considers
custom dumpcycles in calculating a balance value. I wonder if
the planner also fails to consider custom dumpcycles?
Where is a code reader when you want one :)
Jon
--
Jon H. LaBadie j...@jgcomp.com
11226 South Shore Rd. (703) 787-0688 (H)
Reston, VA 20190 (703) 935-6720 (C)
