I solved this with help of another individual on my team. In case anyone
comes across this same issue, here's what solved it for me (feel free to
correct me if I'm wrong on anything below).
We had incorrectly assigned the Shares to child accounts without giving the
correct amount of Shares to parent accounts. An account's normalized
shares is a percent of the shares of all accounts at the same level , with
the starting value being that of the parent.
Basic formula I used is this:
Normalized shares of account = pNShares * ( aShares / sShares )
pNShares = parent normalized shares
aShares = account Shares value
sShares = sum of all Shares values at same level in hierarchy
We also found that in our case the normalized shares of some accounts were
distorted because users belonged to the parent and then a sub-account would
have lower than expected normalized shares. We found the best solution for
us was to only assign users to accounts that have no sub-accounts
The end results:
# sshare
Account User Raw Shares Norm Shares Raw Usage Effectv
Usage FairShare
-------------------- ---------- ---------- ----------- -----------
------------- ----------
root 1.000000 82779413
1.000000 0.870551
root root 1 0.000323 0
0.000000 1.000000
grid 1 0.000323 4915
0.000060 0.974743
cms 10 0.000269 4915
0.000000 1.000000
suragrid 1 0.000027 0
0.000000 1.000000
tamu 3096 0.999354 82774497
0.999940 0.870480
agriculture 20 0.006671 0
0.000000 1.000000
aglife 1 0.003336 0
0.000000 1.000000
genomics 1 0.003336 0
0.000000 1.000000
engineering 10 0.003336 211171
0.002554 0.899279
pete 1 0.003336 211171
0.002554 0.899279
general 10 0.003336 4139196
0.050014 0.125109
geo 10 0.003336 0
0.000000 1.000000
atmo 1 0.003336 0
0.000000 1.000000
liberalarts 128 0.042696 39949222
0.482769 0.208566
idhmc 1 0.042696 39949222
0.482769 0.208566
mgmt 2058 0.686472 5335
0.000065 0.999987
science 760 0.253508 38469571
0.464610 0.775638
acad 10 0.003336 0
0.000000 1.000000
chem 10 0.003336 0
0.000000 1.000000
iamcs 10 0.003336 1396559
0.016930 0.494785
math-dept 20 0.006671 14429182
0.174490 0.026624
math 10 0.003336 14429182
0.174490 0.000709
secant 10 0.003336 0
0.000000 1.000000
physics 700 0.233494 22643828
0.409482 0.784180
hepx 700 0.233494 22643828
0.409482 0.784180
stat 10 0.003336 0
0.000000 1.000000
carroll 10 0.003336 0
0.000000 1.000000
I went the route of assigning Share values based on CPUs that exist in
cluster and giving stakeholders a Shares value equal to number of CPUs they
have funded. All remaining CPUs were assigned to mgmt so that all
non-stakeholders would have the same Normalized Shares value (with a few
extra given to all non-stakeholders so their normalized shares would be
slightly elevated).
If anyone comes across this and wants some help finding a good way to work
out FairShare values for their hierarchy, feel free to ping me off-list for
a spread sheet I developed in LibreOffice that mimics sshare output but
allows for tweaking of values to get a desired end-result.
Thanks,
- Trey
=============================
Trey Dockendorf
Systems Analyst I
Texas A&M University
Academy for Advanced Telecommunications and Learning Technologies
Phone: (979)458-2396
Email: [email protected]
Jabber: [email protected]
On Fri, Feb 20, 2015 at 2:42 PM, Trey Dockendorf <[email protected]> wrote:
> I'm concerned by output of sshare in regard to using DEPTH_OBVLIVIOUS. It
> seems the depth of our accounts are causing our primary stakeholder to have
> a drastically reduced normalized share value.
>
> Here's output of 'sshare -l' (sorry if distorts readability of this email)
>
> Account User Raw Shares Norm Shares Raw Usage Norm
> Usage Effectv Usage FairShare GrpCPUMins CPURunMins
> -------------------- ---------- ---------- ----------- -----------
> ----------- ------------- ---------- ----------- ---------------
> root 1.000000 43769249
> 1.000000 0.870551 11154213
> root root 1 0.333333 0
> 0.000000 0.000000 1.000000 0
> grid 1 0.333333 6300
> 0.000145 0.000145 0.999940 0
> cms 10 0.277778 6300
> 0.000145 0.000000 1.000000 0
> suragrid 1 0.027778 0
> 0.000000 0.000000 1.000000 0
> tamu 1 0.333333 43762948
> 0.999855 0.999855 0.659794 11154213
> aglife 10 0.114943 28
> 0.000001 0.225816 0.761587 0
> genomics 10 0.011610 0
> 0.000000 0.000000 1.000000 0
> engineering 1 0.011494 12410
> 0.000285 0.029560 0.700111 0
> pete 10 0.011494 12410
> 0.000285 0.029560 0.700111 0
> general 5 0.057471 44499
> 0.000939 0.145837 0.703435 112
> geo 1 0.011494 0
> 0.000000 0.000000 1.000000 0
> atmo 10 0.011494 0
> 0.000000 0.000000 1.000000 0
> liberalarts 1 0.011494 30971933
> 0.706947 0.706408 0.000199 10842867
> idhmc 128 0.011494 30971933
> 0.706947 0.706408 0.000199 10842867
> mgmt 10 0.114943 18
> 0.000000 0.222637 0.764512 0
> science 1 0.011494 12734058
> 0.291684 0.291684 0.029661 311234
> acad 1 0.000348 0
> 0.000000 0.000000 1.000000 0
> chem 10 0.003483 0
> 0.000000 0.000000 1.000000 0
> iamcs 10 0.003483 3349720
> 0.076799 0.088341 0.029717 22086
> math 10 0.003483 3655568
> 0.083454 0.088369 0.029684 167191
> secant 10 0.000995 0
> 0.000000 0.000000 1.000000 0
> physics 1 0.000348 5728769
> 0.131431 0.131431 0.000000 121956
> hepx 700 0.000348 5728769
> 0.131431 0.131431 0.000000 121956
> stat 1 0.000348 0
> 0.000000 0.000000 1.000000 0
> carroll 10 0.000348 0
> 0.000000 0.000000 1.000000 0
>
>
> What concerns me is the FareShare for 'hepx' is 0.0 and is non-zero for
> 'idhmc' when idhmc has fewer raw shares and much more usage.
>
> Right now these are our configs on 14.03.10
>
> PriorityFlags=DEPTH_OBLIVIOUS,SMALL_RELATIVE_TO_TIME
> PriorityType=priority/multifactor
> PriorityDecayHalfLife=1-0
>
> The group 'idhmc' has been only users running jobs for the past few days
> and I wanted to check that if hepx (primary stakeholder) began running
> they'd have priority. It seems the depth of 'hepx' is effecting their
> normalized shares , unless sshare does not take into account
> DEPTH_OBLIVIOUS.
>
> The Raw Shares value is the number of CPUs funded by a stakeholder, or 10
> for non-stakeholders. I have never really found a good value so just went
> with something tangible.
>
> Our account structure is also probably overly complicated and may be such
> we need to just move all accounts containing users to be directly under
> "tamu".
>
> The goal is that usage within an account adjust a person's fairshare but
> the location in the hierarchy have no impact on the account's fairshare.
> The account vs account usage is still important, ie if hepx used most of
> the cluster for a day then their priority should be lower than that of a
> non-stakeholder who hasn't run any jobs recently.
>
> Any insight is welcome.
>
> Thanks,
> - Trey
>
> =============================
>
> Trey Dockendorf
> Systems Analyst I
> Texas A&M University
> Academy for Advanced Telecommunications and Learning Technologies
> Phone: (979)458-2396
> Email: [email protected]
> Jabber: [email protected]
>
