I think you need to set AccountingStorageEnforce to 'limits' or order to get slurm to enforce this.
From: "Hanby, Mike" <mha...@uab.edu> Reply-To: slurm-dev <slurm-dev@schedmd.com> Date: Wednesday, January 25, 2017 at 3:01 PM To: slurm-dev <slurm-dev@schedmd.com> Subject: [slurm-dev] sacctmgr user maxjobs not working Howdy, I'm trying to temporarily disable a user's ability to either submit or start (haven't decided which way I want to go yet) in the event that the user is abusing the cluster. I've tried both of the following: sacctmgr --immediate update user joeblow set maxsubmitjobs=0 sacctmgr --immediate update user joeblow set maxjobs=0 Both of these commands successfully update the user and can be confirmed looking at: sacctmgr list assoc user=joeblow However, neither of these prevent the user from either submitting (the former) or starting (the latter) jobs. Quick bit of info, while attempting the "update user" command the first time, I noticed none of our users (who are in LDAP) have Slurm user entries. So I had to start off by creating the user: sacctmgr --immediate add Account joeblow sacctmgr_admin --immediate add User Accounts=joeblow joeblow It appears we need to adjust our new user account script to also add the Slurm account and user. Is there something in the slurm.conf I need to enable in order to have the scheduler enforce user settings? Below is our slurm.conf file ------------------------------ ClusterName=SLURM_CLUSTER SlurmUser=slurm SlurmctldPort=6817 SlurmdPort=6818 AuthType=auth/munge StateSaveLocation=/cm/shared/apps/slurm/var/cm/statesave SlurmdSpoolDir=/cm/local/apps/slurm/var/spool SwitchType=switch/none MpiDefault=none SlurmctldPidFile=/var/run/slurmctld.pid SlurmdPidFile=/var/run/slurmd.pid ProctrackType=proctrack/cgroup PluginDir=/cm/shared/apps/slurm/current/lib64/slurm JobSubmitPlugins=job_submit/require_timelimit CacheGroups=0 ReturnToService=2 TaskPlugin=task/cgroup SlurmctldTimeout=300 SlurmdTimeout=300 InactiveLimit=0 MinJobAge=300 KillWait=30 Waittime=0 SlurmctldDebug=3 SlurmctldLogFile=/var/log/slurmctld SlurmdDebug=3 SlurmdLogFile=/var/log/slurmd JobAcctGatherType=jobacct_gather/linux JobAcctGatherFrequency=30 AccountingStorageType=accounting_storage/slurmdbd AccountingStorageUser=slurm SelectType=select/cons_res SelectTypeParameters=CR_CORE_Memory TmpFs=/scratch/local SchedulerType=sched/backfill ControlMachine=cheaha-master01 ControlAddr=cheaha-master01 AccountingStorageHost=cheaha-master01 NodeName=login001 NodeName=c[0001-0036] Procs=24 CoresPerSocket=12 RealMemory=128830 Sockets=2 ThreadsPerCore=1 NodeName=c[0089-0092] Procs=24 CoresPerSocket=12 RealMemory=128830 Sockets=2 ThreadsPerCore=1 Gres=gpu:2 NodeName=c[0093-0096] Procs=24 CoresPerSocket=12 RealMemory=128830 Sockets=2 ThreadsPerCore=1 Gres=mic:1 Feature=michost NodeName=c[0037-0074] Procs=24 CoresPerSocket=12 RealMemory=257854 Sockets=2 ThreadsPerCore=1 NodeName=c[0075-0088] Procs=24 CoresPerSocket=12 RealMemory=386878 Sockets=2 ThreadsPerCore=1 PartitionName=interactive Default=NO MinNodes=1 MaxNodes=1 MaxTime=02:00:00 AllowGroups=ALL Priority=1 DisableRootJobs=NO RootOnly=NO Hidden=NO Shared=NO GraceTime=0 PreemptMode=OFF ReqResv=NO AllowAccounts=ALL AllowQos=ALL LLN=NO ExclusiveUser=NO PriorityJobFactor=1 PriorityTier=1 OverSubscribe=NO State=UP Nodes=c[0001-0096] PartitionName=short Default=NO MinNodes=1 MaxNodes=44 MaxTime=12:00:00 AllowGroups=ALL Priority=1 DisableRootJobs=NO RootOnly=NO Hidden=NO Shared=NO GraceTime=0 PreemptMode=OFF ReqResv=NO AllowAccounts=ALL AllowQos=ALL LLN=NO ExclusiveUser=NO PriorityJobFactor=1 PriorityTier=1 OverSubscribe=NO State=UP Nodes=c[0001-0096] PartitionName=long Default=NO MinNodes=1 MaxNodes=22 MaxTime=6-06:00:00 AllowGroups=ALL Priority=1 DisableRootJobs=NO RootOnly=NO Hidden=NO Shared=NO GraceTime=0 PreemptMode=OFF ReqResv=NO AllowAccounts=ALL AllowQos=ALL LLN=NO ExclusiveUser=NO PriorityJobFactor=1 PriorityTier=1 OverSubscribe=NO State=UP Nodes=c[0001-0096] PartitionName=medium Default=NO MinNodes=1 MaxNodes=44 MaxTime=2-02:00:00 AllowGroups=ALL Priority=1 DisableRootJobs=NO RootOnly=NO Hidden=NO Shared=NO GraceTime=0 PreemptMode=OFF ReqResv=NO AllowAccounts=ALL AllowQos=ALL LLN=NO ExclusiveUser=NO PriorityJobFactor=1 PriorityTier=1 OverSubscribe=NO State=UP Nodes=c[0001-0096] PartitionName=express Default=NO MinNodes=1 MaxTime=02:01:00 AllowGroups=ALL Priority=1 DisableRootJobs=NO RootOnly=NO Hidden=NO Shared=NO GraceTime=0 PreemptMode=OFF ReqResv=NO AllowAccounts=ALL AllowQos=ALL LLN=NO ExclusiveUser=NO PriorityJobFactor=1 PriorityTier=1 OverSubscribe=NO State=UP Nodes=c[0001-0096] PartitionName=beta Default=NO MinNodes=1 AllowGroups=atlab Priority=1 DisableRootJobs=NO RootOnly=NO Hidden=YES Shared=NO GraceTime=0 PreemptMode=OFF ReqResv=NO AllowAccounts=ALL AllowQos=ALL LLN=NO ExclusiveUser=NO PriorityJobFactor=1 PriorityTier=1 OverSubscribe=NO State=UP GresTypes=gpu,mic PrologSlurmctld=/cm/local/apps/cmd/scripts/prolog-prejob Prolog=/cm/local/apps/cmd/scripts/prolog Epilog=/cm/local/apps/cmd/scripts/epilog FastSchedule=1 SuspendTime=-1 # this disables power saving SuspendTimeout=30 ResumeTimeout=60 SuspendProgram=/cm/local/apps/cluster-tools/wlm/scripts/slurmpoweroff ResumeProgram=/cm/local/apps/cluster-tools/wlm/scripts/slurmpoweron MaxArraySize=10001 EnforcePartLimits=yes Licenses=lsdyna:512 ------------------------------ ---------------- Mike Hanby mhanby @ uab.edu Systems Analyst II - Enterprise Research Computing Services / IT Infrastructure The University of Alabama at Birmingham
