Hi Chris,
I suggest that you configure SlurmctldDebug to "verbose"; it will
probably tell you why the decision is being made.
(BTW, is it possible that the user doesn't have an account on the
compute nodes yet?)
Andy
On 04/03/2017 03:30 PM, Chris Woelkers - NOAA Affiliate wrote:
I am running a small HPC, only 24 nodes, via slurm and am having an
issue where one of the users is unable to submit any jobs.
The user is new and whenever a job is submitted it shows the "job
requeued in held state" state and is never actually ran. We have left
the job sitting for over three days and it does not start. We have
tried releasing the job and it does not start. Here are the log
entries after an attempted release:
[2017-04-03T19:16:24.173] sched: update_job: releasing hold for job_id
1938 uid 0
[2017-04-03T19:16:24.174] _slurm_rpc_update_job complete JobId=1938
uid=0 usec=375
[2017-04-03T19:16:24.919] sched: Allocate JobId=1938
NodeList=rhinonode[07-14] #CPUs=192
[2017-04-03T19:16:25.017] _slurm_rpc_requeue: Processing RPC:
REQUEST_JOB_REQUEUE from uid=0
[2017-04-03T19:16:25.035] Requeuing JobID=1938 State=0x0 NodeCnt=0
The user has the same permissions as the older users that can run jobs.
The script that is being run is a simple test script and no matter
where the output is redirected, an NFS mount(for our SAN), the local
home directory, or the tmp directory, the result is the same.
Any idea as to what might be happening?
Thanks,
Chris Woelkers
Caelum Research Corp.
Linux Server and Network Administrator
NOAA GLERL
--
Andy Riebs
andy.ri...@hpe.com
Hewlett-Packard Enterprise
High Performance Computing Software Engineering
+1 404 648 9024
My opinions are not necessarily those of HPE
May the source be with you!
