The sshare reports the same even now (after few hours) - I have not changed
anything after the previous test.
[root@slurm-login slurm-scripts]# sshare
Account User Raw Shares Norm Shares Raw Usage Effectv
Usage FairShare
-------------------- ---------- ---------- ----------- -----------
------------- ----------
root 1.000000 79
1.000000 0.500000
premium1 50 0.500000 36
0.455696 0.531672
premium2 50 0.500000 43
0.544304 0.470215
Ryan,
sshare doesn't actually have access to the cputime that is accounted for. All
sacct does is (elapsed time * CPU count), something that you can check in the
manpage or sshare code. This can be slightly different than what was actually
accounted for in _apply_new_usage(). Slurm doesn't do very well at granularity
in the seconds (not necessarily bad thing).
On SLURM not being good at granularity in seconds - the delta between 36 and 43
from 60 (actual runtime) is significant isnt it?
Thanks!
Roshan
________________________________________
From: Lech Nieroda <lech.nier...@uni-koeln.de>
Sent: 26 November 2014 15:36
To: slurm-dev
Subject: [slurm-dev] Re: [ sshare ] RAW Usage
Hello Mathew,
just to check the basics - did you wait a few minutes before executing sshare?
As far as I remember the RawUsage value is updated every 5 minutes (per
default), so these erroneous values might be caused by a measurement that was
taken while the jobs were still running.
Regards,
Lech
Am 26.11.2014 um 12:14 schrieb Roshan Mathew <r.t.mat...@bath.ac.uk>:
> My fairshare test scenario - As it stand the farishare is not distributed
> correctly
>
> *Accounts*
>
> [root@slurm-login slurm-scripts]# sacctmgr list accounts
> Account Descr Org
> ---------- -------------------- --------------------
> premium1 primary account root
> premium2 primary account root
> root default root account root
>
> *Users*
>
> [root@slurm-login slurm-scripts]# sacctmgr list users
> User Def Acct Admin
> ---------- ---------- ---------
> mm339 premium1 None
> sy223 premium2 None
>
> *Initial Shares*
>
> [root@slurm-login slurm-scripts]# sshare
> Account User Raw Shares Norm Shares Raw Usage Effectv
> Usage FairShare
> -------------------- ---------- ---------- ----------- -----------
> ------------- ----------
> root 1.000000 0
> 1.000000 0.500000
> premium1 50 0.500000 0
> 0.000000 1.000000
> premium2 50 0.500000 0
> 0.000000 1.000000
>
>
> *Job script*
>
> [root@slurm-login slurm-scripts]# cat stress.slurm
> #!/bin/bash
>
> #SBATCH --nodes=1
> #SBATCH --ntasks=1
> #SBATCH --job-name=stress
> #SBATCH --time=10
> #SBATCH --output=stress.%j-out
> #SBATCH --error=stress.%j-out
>
> time /opt/shared/apps/stress/app/bin/stress --cpu 1 --timeout 1m
>
>
> *Job Submission*
>
> [root@slurm-login slurm-scripts]# runuser sy223 -c 'sbatch stress.slurm'
> Submitted batch job 2
> [root@slurm-login slurm-scripts]# runuser mm339 -c 'sbatch stress.slurm'
> Submitted batch job 3
>
>
> *SACCT Information*
>
> [root@slurm-login slurm-scripts]# sacct --format
> Jobid,jobname,partition,account,alloccpus,state,exitcode,cputimeraw
> JobID JobName Partition Account AllocCPUS State ExitCode
> CPUTimeRAW
> ------------ ---------- ---------- ---------- ---------- ---------- --------
> ----------
> 2 stress batch premium2 1 COMPLETED 0:0
> 60
> 2.batch batch premium2 1 COMPLETED 0:0
> 60
> 3 stress batch premium1 1 COMPLETED 0:0
> 60
> 3.batch batch premium1 1 COMPLETED 0:0
> 60
>
>
> *SSHARE Information*
>
> [root@slurm-login slurm-scripts]# sshare
> Account User Raw Shares Norm Shares Raw Usage Effectv
> Usage FairShare
> -------------------- ---------- ---------- ----------- -----------
> ------------- ----------
> root 1.000000 79
> 1.000000 0.500000
> premium1 50 0.500000 36
> 0.455696 0.531672
> premium2 50 0.500000 43
> 0.544304 0.470215
>
>
> *Slurm.conf - priority/multifactor*
>
> # Activate the Multi-factor Job Priority Plugin with decay
> PriorityType=priority/multifactor
>
> # apply no decay
> PriorityDecayHalfLife=0
> PriorityCalcPeriod=1
> # reset usage after 1 month
> PriorityUsageResetPeriod=MONTHLY
>
> # The larger the job, the greater its job size priority.
> PriorityFavorSmall=NO
>
> # The job's age factor reaches 1.0 after waiting in the
> # queue for 2 weeks.
> PriorityMaxAge=14-0
>
> # This next group determines the weighting of each of the
> # components of the Multi-factor Job Priority Plugin.
> # The default value for each of the following is 1.
> PriorityWeightAge=0
> PriorityWeightFairshare=100
> PriorityWeightJobSize=0
> PriorityWeightPartition=0
> PriorityWeightQOS=0 # don't use the qos factor
>
>
> *Questions*
>
> 1. Given that I have set the PriorityDecayHalfLife=0, i.e no decay applied at
> any stage, shouldnt both the jobs have the same RAW Usage reported by SSHARE?
>
> 2. Also shouldnt CPUTimeRAW in sacct be same as RAW Usage in sshare?
>
>
> From: Skouson, Gary B <gary.skou...@pnnl.gov>
> Sent: 25 November 2014 21:09
> To: slurm-dev
> Subject: [slurm-dev] Re: [ sshare ] RAW Usage
>
> I believe that the info share data is kept by slurmctld in memory. As far as
> I could tell from the code, it should be checkpointing the info to the
> assoc_usage file wherever slurm is saving state information. I couldn't find
> any docs on that, you'd have to check the code for more information.
>
> However, if you just want to see what was used, you can get the raw usage
> using sacct. For example, for a given job, you can do something like:
>
> sacct -X -a -j 1182128 --format
> Jobid,jobname,partition,account,alloccpus,state,exitcode,cputimeraw
>
> -----
> Gary Skouson
>
>
> From: Roshan Mathew [mailto:r.t.mat...@bath.ac.uk]
> Sent: Tuesday, November 25, 2014 9:51 AM
> To: slurm-dev
> Subject: [slurm-dev] Re: [ sshare ] RAW Usage
>
> Thanks Ryan,
>
> Is this value stored anywhere in the SLURM accounting DB? I could not find
> any value for the JOB that corresponds to this RAW usage.
>
> Roshan
> From: Ryan Cox <ryan_...@byu.edu>
> Sent: 25 November 2014 17:43
> To: slurm-dev
> Subject: [slurm-dev] Re: [ sshare ] RAW Usage
>
> Raw usage is a long double and the time added by jobs can be off by a few
> seconds. You can take a look at _apply_new_usage() in
> src/plugins/priority/multifactor/priority_multifactor.c to see exactly what
> happens.
>
> Ryan
>
> On 11/25/2014 10:34 AM, Roshan Mathew wrote:
> Hello SLURM users,
>
> http://slurm.schedmd.com/sshare.html
> Raw Usage
> The number of cpu-seconds of all the jobs that charged the account by the
> user. This number will decay over time when PriorityDecayHalfLife is defined.
> I am getting different RAW Usage values for the same job every time it is
> executed. The Job am using is a CPU stress test for 1 minute.
>
> It would be very useful to understand the formula for how this RAW Usage is
> calculated when we are using the plugin PriorityType=priority/multifactor.
>
> Snip of my slurm.conf file:-
>
> # Activate the Multi-factor Job Priority Plugin with decay
> PriorityType=priority/multifactor
>
> # apply no decay
> PriorityDecayHalfLife=0
>
> PriorityCalcPeriod=1
> PriorityUsageResetPeriod=MONTHLY
>
> # The larger the job, the greater its job size priority.
> PriorityFavorSmall=NO
>
> # The job's age factor reaches 1.0 after waiting in the
> # queue for 2 weeks.
> PriorityMaxAge=14-0
>
> # This next group determines the weighting of each of the
> # components of the Multi-factor Job Priority Plugin.
> # The default value for each of the following is 1.
> PriorityWeightAge=0
> PriorityWeightFairshare=100
> PriorityWeightJobSize=0
> PriorityWeightPartition=0
> PriorityWeightQOS=0 # don't use the qos factor
>
> Thanks!
>
> <image001.jpg>
> <image001.jpg>
>
>
