[slurm-users] getting started with job_submit_lua
Hi all,
I'm trying to get started with the lua job_submit feature and I have a
really dumb question. This job_submit Lua script:
function slurm_job_submit( job_desc, part_list, submit_uid )
slurm.log_info("submit called lua plugin")
for k,v in pairs(job_desc) do
slurm.log_info("submit %s", k)
end
slurm.log_info("submit completed lua plugin")
return slurm.SUCCESS
end
function slurm_job_modify(job_desc, job_rec, part_list, modify_uid)
slurm.log_info("submit called lua plugin2")
return slurm.SUCCESS
end
Prints:
Sep 15 18:00:57 quack1 slurmctld[9617]: job_submit.lua: submit called lua
plugin
Sep 15 18:00:57 quack1 slurmctld[9617]: job_submit.lua: submit completed lua
plugin
When I:
srun --pty -p test.q bash
I was expecting job_desc to be iterated over and show some interesting
stuff in my log, but it looks empty. Any ideas why that might be, please?
Slurm 19.05.7-1 / Lua 5.1.4 / CentOS 7.8 / x86_64
Thanks,
Mark
Re: [slurm-users] getting started with job_submit_lua
Il 15/09/20 19:47, Mark Dixon ha scritto: > I was expecting job_desc to be iterated over and show some interesting > stuff in my log, but it looks empty. Any ideas why that might be, please? Same thing I noticed. I needed it to check which fields were available, but it seems ipairs doesn't work as advertised (unless I misunderstood LUA docs). >From the source it seems these fields are available: account comment direct_set_prio gres job_id Always nil ? Maybe no JobID yet? job_state licenses max_cpus max_nodes min_cpus min_nodes nice partition priority req_switch time_limit time_min wait4switch wckey If you access 'em directly, you'll find that some are actually populated. -- Diego Zuccato DIFA - Dip. di Fisica e Astronomia Servizi Informatici Alma Mater Studiorum - Università di Bologna V.le Berti-Pichat 6/2 - 40127 Bologna - Italy tel.: +39 051 20 95786
Re: [slurm-users] getting started with job_submit_lua
On Wed, 16 Sep 2020, Diego Zuccato wrote:
...
From the source it seems these fields are available:
account
comment
direct_set_prio
gres
job_id Always nil ? Maybe no JobID yet?
job_state
licenses
max_cpus
max_nodes
min_cpus
min_nodes
nice
partition
priority
req_switch
time_limit
time_min
wait4switch
wckey
If you access 'em directly, you'll find that some are actually populated.
Hi Diego, thanks for replying :)
I gave this alternative a go:
function slurm_job_submit( job_desc, part_list, submit_uid )
slurm.log_info("submit called lua plugin")
slurm.log_info("submit -- account %s", job_desc.account)
slurm.log_info("submit -- gres %s", job_desc.gres)
slurm.log_info("submit completed lua plugin")
return slurm.SUCCESS
end
function slurm_job_modify(job_desc, job_rec, part_list, modify_uid)
slurm.log_info("submit called lua plugin2")
return slurm.SUCCESS
end
And got:
Sep 16 09:36:58 quack1 slurmctld[9617]: job_submit.lua: submit called lua
plugin
Sep 16 09:36:58 quack1 slurmctld[9617]: error: job_submit/lua:
/usr/local/slurm/19.05.7-1/etc/job_submit.lua: [string "slurm.log (0,
string.format(unpack({...})))"]:1: bad argument #2 to 'format' (no value)
It seems "pairs" wasn't lying, job_desc really is empty?
A job_submit function isn't much use without any information about the
job!
Please help!
Mark
Re: [slurm-users] getting started with job_submit_lua
If you explicitely specify the account, f.ex. 'sbatch -A myaccount'
then 'slurm.log_info("submit -- account %s", job_desc.account)'
works.
/Niels Carl
On Wed, 16 Sep 2020, Mark Dixon wrote:
> On Wed, 16 Sep 2020, Diego Zuccato wrote:
> ...
> > From the source it seems these fields are available:
> >account
> >comment
> >direct_set_prio
> >gres
> >job_id Always nil ? Maybe no JobID yet?
> >job_state
> >licenses
> >max_cpus
> >max_nodes
> >min_cpus
> >min_nodes
> >nice
> >partition
> >priority
> >req_switch
> >time_limit
> >time_min
> >wait4switch
> >wckey
> >
> > If you access 'em directly, you'll find that some are actually populated.
>
> Hi Diego, thanks for replying :)
>
> I gave this alternative a go:
>
> function slurm_job_submit( job_desc, part_list, submit_uid )
>
> slurm.log_info("submit called lua plugin")
> slurm.log_info("submit -- account %s", job_desc.account)
> slurm.log_info("submit -- gres %s", job_desc.gres)
> slurm.log_info("submit completed lua plugin")
>
> return slurm.SUCCESS
> end
>
> function slurm_job_modify(job_desc, job_rec, part_list, modify_uid)
> slurm.log_info("submit called lua plugin2")
> return slurm.SUCCESS
> end
>
> And got:
>
> Sep 16 09:36:58 quack1 slurmctld[9617]: job_submit.lua: submit called lua
> plugin
> Sep 16 09:36:58 quack1 slurmctld[9617]: error: job_submit/lua:
> /usr/local/slurm/19.05.7-1/etc/job_submit.lua: [string "slurm.log (0,
> string.format(unpack({...})))"]:1: bad argument #2 to 'format' (no value)
>
> It seems "pairs" wasn't lying, job_desc really is empty?
>
> A job_submit function isn't much use without any information about the job!
>
> Please help!
>
> Mark
>
Re: [slurm-users] getting started with job_submit_lua
Il 16/09/20 10:44, Mark Dixon ha scritto:
> It seems "pairs" wasn't lying, job_desc really is empty?
Nope. In my case, at least some fields are populated: .partition,
.max_cpus (usually "a lot"), .max_nodes, .min_cpus, .min_nodes .
I also tried adding
local j = {}
j.uid = submit_uid or -1
j.acc = job_desc.account or "N/A"
j.cmt = job_desc.comment or "N/A"
j.dsp = job_desc.direct_set_prio or 0
j.res = job_desc.gres or "N/A"
--j.jid = job_desc.job_id or -1
--j.jst = job_desc.job_state or 0
j.lic = job_desc.licenses or {}
j.cpu = job_desc.max_cpus or 0
j.nod = job_desc.max_nodes or 0
j.mcp = job_desc.min_cpus or 0
j.mnd = job_desc.min_nodes or 0
j.nic = job_desc.nice or 0
j.par = job_desc.partition or 0
j.pri = job_desc.priority or 0
--j. = job_desc.req_switch or 0
j.tim = job_desc.time_limit or 0
j.mti = job_desc.time_min or 0
--j. = job_desc.wait4switch or 0
--j. = job_desc.wckey or 0
but ipairs() iteration over j still gives nothing even if logging
individual values works!
> A job_submit function isn't much use without any information about the job!
Yup :)
--
Diego Zuccato
DIFA - Dip. di Fisica e Astronomia
Servizi Informatici
Alma Mater Studiorum - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786
Re: [slurm-users] getting started with job_submit_lua
On Wed, 16 Sep 2020, Niels Carl W. Hansen wrote:
If you explicitely specify the account, f.ex. 'sbatch -A myaccount'
then 'slurm.log_info("submit -- account %s", job_desc.account)'
works.
Great, thanks - that's working!
Of course I have other problems... :(
Cheers,
Mark
--
Mark Dixon Tel: +44(0)191 33 41383
Advanced Research Computing (ARC), Durham University, UK
