Coordinator permissions from the man pages:
coordinator
A special privileged user, usually an account manager, that can add
users or sub-accounts to the account they are coordinator over. This
should be a trusted person since they can change limits on account
and user associations, as well as cancel, requeue or reassign
accounts of jobs inside their realm.
So, I read that as it manages accounts in slurmdb with minimal access to
the jobs themselves. So you would be stuck with cancel/requeue. I see no
mention of hold, but if that is one of the permissions, I would say,
yes, our approach does what you want within the limits of what the
default permissions of a coordinator can do.
Of course, that still may not work if there are other
accounts/partitions/users with higher priority jobs than User B.
Specifically if those jobs can use the same resources A's jobs are
running on.
Brian Andrus
On 5/17/2023 10:49 AM, Groner, Rob wrote:
I'm not sure what you mean by "if they have the permissions". I'm
talking about someone who is specifically designated as "coordinator"
of an account in slurm. With that designation, and no other admin
level changes, I'm not aware that they can directly change the
priority of jobs associated with the account.
If you're talking about additional permissions or admin levels...we're
not looking into that as an option. We want to purely use the
coordinator role to have them manipulate stuff.
------------------------------------------------------------------------
*From:* slurm-users <slurm-users-boun...@lists.schedmd.com> on behalf
of Brian Andrus <toomuc...@gmail.com>
*Sent:* Wednesday, May 17, 2023 12:58 PM
*To:* slurm-users@lists.schedmd.com <slurm-users@lists.schedmd.com>
*Subject:* Re: [slurm-users] On the ability of coordinators
If they have the permissions, you can just raise the priority of user
B's jobs to be higher than whatever A's currently are. Then they will
run next.
That will work if you are able to wait for some jobs to finish and you
can 'skip the line' for the priority jobs.
If you need to preempt running jobs, that would take a bit more effort
to set up, but is an alternative.
Brian Andrus
On 5/17/2023 6:40 AM, Groner, Rob wrote:
I was asked to see if coordinators could do anything in this scenario:
* Within the account that they coordinated, User A submitted 1000s
of jobs and left for the day.
* Within the same account, User B wanted to run a few jobs really
quickly. Once submitted, his jobs were of course behind User A's
jobs.
* The coordinator wanted to see the results of User B's runs.
Reading the docs and doing some experiments, here is what I determined:
* The coordinator could put a hold on all of User A's jobs in the
pending queue. This won't affect any jobs User A has that aren't
tied to the coordinated account.
* With User A's jobs held, then User B's jobs would be next to run.
* If the coordinator was particularly impatient, he could scancel
User A's currently running jobs so that User B's jobs immediately
started.
* The coordinator would need to remember to release the held jobs,
or put them in a uhold so that User A could release them eventually.
It seems like the easiest way for the coordinator to elevate User B's
jobs to the top of the queue would be if he could "scontrol top"
those jobs. But my testing indicates that the coordinator doesn't
have that permission. Is there some reason that a coordinator can't
use "scontrol top" to change the priority of jobs within the account
that he coordinates?
Thanks.
Rob
