Re: [slurm-users] Jobs in pending state

2018-04-29 Thread Paul Edmon
It sounds like your second partition is getting primarily scheduled by the backfill scheduler.  I would try the partition_job_depth option as otherwise the main loop only looks at priority order and not by partition. -Paul Edmon- On 4/29/2018 5:32 AM, Zohar Roe MLM wrote: Hello. I am having

Re: [slurm-users] Include some cores of the head node to a partition

2018-04-29 Thread Chris Samuel
On Sunday, 29 April 2018 6:57:58 PM AEST Mahmood Naderan wrote: > [root@rocks7 ~]# scontrol show config | fgrep -i rocks7 Ah, I'd forgotten that wouldn't list the NodeName lines from your config file. Sorry. > Chris, > Regarding this section > > NodeName=DEFAULT State=UNKNOWN >

[slurm-users] Jobs in pending state

2018-04-29 Thread Zohar Roe MLM
Hello. I am having 2 cluster in my slurm.conf: CLUS_WORK1 server1 server2 server3 CLUS_WORK2 pc1 pc2 pc3 When I'm sending 10,000 jobs to CLUS_WORK1 they are good and start running while a few are in pending state (which is ok). But if I send new jobs to CLUS_WORK2 which is idle, I see that the

Re: [slurm-users] Include some cores of the head node to a partition

2018-04-29 Thread Mahmood Naderan
[root@rocks7 ~]# scontrol show config | fgrep -i rocks7 AccountingStorageHost = rocks7 ControlMachine = rocks7 JobCompHost = rocks7 Slurmctld(primary/backup) at rocks7/(NULL) are UP/DOWN Chris, Regarding this section NodeName=DEFAULT State=UNKNOWN NodeName=rocks7

Re: [slurm-users] Include some cores of the head node to a partition

2018-04-29 Thread Chris Samuel
Hi Mahmood, Not quite what I meant sorry. What does this say? scontrol show config | fgrep -i rocks7 cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC

Re: [slurm-users] Include some cores of the head node to a partition

2018-04-29 Thread Mahmood Naderan
SlurmUser=root SlurmdUser=root SlurmctldPort=6817 SlurmdPort=6818 AuthType=auth/munge CryptoType=crypto/munge StateSaveLocation=/var/spool/slurm.state SlurmdSpoolDir=/var/spool/slurmd SwitchType=switch/none MpiDefault=none SlurmctldPidFile=/var/run/slurmctld.pid SlurmdPidFile=/var/run/slurmd.pid

Re: [slurm-users] Include some cores of the head node to a partition

2018-04-29 Thread Chris Samuel
On Sunday, 29 April 2018 4:11:39 PM AEST Mahmood Naderan wrote: > So, I don't know why only 1 core included What do you have in your slurm.conf for rocks7? -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC

Re: [slurm-users] Include some cores of the head node to a partition

2018-04-29 Thread Mahmood Naderan
[root@rocks7 ~]# slurmd -C NodeName=rocks7 slurmd: Considering each NUMA node as a socket CPUs=32 Boards=1 SocketsPerBoard=4 CoresPerSocket=8 ThreadsPerCore=1 RealMemory=64261 UpTime=15-21:30:53 [root@rocks7 ~]# scontrol show node rocks7 NodeName=rocks7 Arch=x86_64 CoresPerSocket=1 CPUAlloc=0