[slurm-dev] Re: Job Groups
2013/6/19 Paul Edmon ped...@cfa.harvard.edu: I have a group here that wants to submit a ton of jobs to the queue, but want to restrict how many they have running at any given time so that they don't torch their fileserver. They were using bgmod -L in LSF to do this, but they were wondering if there was a similar way in SLURM to do so. I know you can do this via the accounting interface but it would be good if I didn't have to apply it as a blanket to all their jobs and if they could manage it themselves. If nothing exists in SLURM to do this that's fine. One can always engineer around it. I figured I would ping the dev list first before putting a nail in it. From my look at the documentation I don't see anyway to do this other than what I stated above. I'm not familiar with LSF, but.. if you are using accounts (need database accounting backend) you can simply create account for them, and limit number of running jobs with: GrpJobs= The total number of jobs able to run at any given time from this association and its children. If this limit is reached new jobs will be queued but only allowed to run after previous jobs complete from this group. Another possibility may be, if users want to set the limit himself, is to create allocation and then submit jobs to this allocation. cheers, marcin
[slurm-dev] Re: Job Groups
Could you just create a dedicated queue for those jobs, and then configure its priority and max simultaneous settings? Then all they would have to do is ensure they submit those jobs to that queue. On Jun 19, 2013, at 8:36 AM, Paul Edmon ped...@cfa.harvard.edu wrote: I have a group here that wants to submit a ton of jobs to the queue, but want to restrict how many they have running at any given time so that they don't torch their fileserver. They were using bgmod -L in LSF to do this, but they were wondering if there was a similar way in SLURM to do so. I know you can do this via the accounting interface but it would be good if I didn't have to apply it as a blanket to all their jobs and if they could manage it themselves. If nothing exists in SLURM to do this that's fine. One can always engineer around it. I figured I would ping the dev list first before putting a nail in it. From my look at the documentation I don't see anyway to do this other than what I stated above. -Paul Edmon-
[slurm-dev] Re: Job Groups
Sounds like something you would use a QOS for. That way you get all the limits from accounting but only applies to certain jobs. On 06/19/13 09:03, Ralph Castain wrote: Could you just create a dedicated queue for those jobs, and then configure its priority and max simultaneous settings? Then all they would have to do is ensure they submit those jobs to that queue. On Jun 19, 2013, at 8:36 AM, Paul Edmon ped...@cfa.harvard.edu wrote: I have a group here that wants to submit a ton of jobs to the queue, but want to restrict how many they have running at any given time so that they don't torch their fileserver. They were using bgmod -L in LSF to do this, but they were wondering if there was a similar way in SLURM to do so. I know you can do this via the accounting interface but it would be good if I didn't have to apply it as a blanket to all their jobs and if they could manage it themselves. If nothing exists in SLURM to do this that's fine. One can always engineer around it. I figured I would ping the dev list first before putting a nail in it. From my look at the documentation I don't see anyway to do this other than what I stated above. -Paul Edmon-
[slurm-dev] Re: Job Groups
On 06/19/2013 10:36 AM, Paul Edmon wrote: I have a group here that wants to submit a ton of jobs to the queue, but want to restrict how many they have running at any given time so that they don't torch their fileserver. The licenses feature might work OK for this. Create a license for the fileserver with as many seats as max jobs, and jobs hitting the fileserver would request one (or more) licenses. Regards, John
[slurm-dev] Re: Job Groups
Paul, We were discussing this yesterday due to a user not limiting the amount of jobs hammering our storage. A QOS with a GrpJobs limit sounds like the best approach for both us and you. Ryan On 06/19/2013 09:36 AM, Paul Edmon wrote: I have a group here that wants to submit a ton of jobs to the queue, but want to restrict how many they have running at any given time so that they don't torch their fileserver. They were using bgmod -L in LSF to do this, but they were wondering if there was a similar way in SLURM to do so. I know you can do this via the accounting interface but it would be good if I didn't have to apply it as a blanket to all their jobs and if they could manage it themselves. If nothing exists in SLURM to do this that's fine. One can always engineer around it. I figured I would ping the dev list first before putting a nail in it. From my look at the documentation I don't see anyway to do this other than what I stated above. -Paul Edmon- -- Ryan Cox Operations Director Fulton Supercomputing Lab Brigham Young University
[slurm-dev] Re: Job Groups
Thanks for the input. Can GrpJobs be modified from the user side? -Paul Edmon- On 06/19/2013 12:15 PM, Ryan Cox wrote: Paul, We were discussing this yesterday due to a user not limiting the amount of jobs hammering our storage. A QOS with a GrpJobs limit sounds like the best approach for both us and you. Ryan On 06/19/2013 09:36 AM, Paul Edmon wrote: I have a group here that wants to submit a ton of jobs to the queue, but want to restrict how many they have running at any given time so that they don't torch their fileserver. They were using bgmod -L in LSF to do this, but they were wondering if there was a similar way in SLURM to do so. I know you can do this via the accounting interface but it would be good if I didn't have to apply it as a blanket to all their jobs and if they could manage it themselves. If nothing exists in SLURM to do this that's fine. One can always engineer around it. I figured I would ping the dev list first before putting a nail in it. From my look at the documentation I don't see anyway to do this other than what I stated above. -Paul Edmon-
[slurm-dev] Re: Job Groups
Okay, thanks. -Paul Edmon- On 06/19/2013 04:32 PM, Ryan Cox wrote: Not that I'm aware of. I don't know of a way to give users control over a QOS like you can do with account coordinators for accounts. Ryan On 06/19/2013 10:55 AM, Paul Edmon wrote: Thanks for the input. Can GrpJobs be modified from the user side? -Paul Edmon- On 06/19/2013 12:15 PM, Ryan Cox wrote: Paul, We were discussing this yesterday due to a user not limiting the amount of jobs hammering our storage. A QOS with a GrpJobs limit sounds like the best approach for both us and you. Ryan On 06/19/2013 09:36 AM, Paul Edmon wrote: I have a group here that wants to submit a ton of jobs to the queue, but want to restrict how many they have running at any given time so that they don't torch their fileserver. They were using bgmod -L in LSF to do this, but they were wondering if there was a similar way in SLURM to do so. I know you can do this via the accounting interface but it would be good if I didn't have to apply it as a blanket to all their jobs and if they could manage it themselves. If nothing exists in SLURM to do this that's fine. One can always engineer around it. I figured I would ping the dev list first before putting a nail in it. From my look at the documentation I don't see anyway to do this other than what I stated above. -Paul Edmon-
[slurm-dev] Re: Job Groups
Not that I'm aware of. I don't know of a way to give users control over a QOS like you can do with account coordinators for accounts. Ryan On 06/19/2013 10:55 AM, Paul Edmon wrote: Thanks for the input. Can GrpJobs be modified from the user side? -Paul Edmon- On 06/19/2013 12:15 PM, Ryan Cox wrote: Paul, We were discussing this yesterday due to a user not limiting the amount of jobs hammering our storage. A QOS with a GrpJobs limit sounds like the best approach for both us and you. Ryan On 06/19/2013 09:36 AM, Paul Edmon wrote: I have a group here that wants to submit a ton of jobs to the queue, but want to restrict how many they have running at any given time so that they don't torch their fileserver. They were using bgmod -L in LSF to do this, but they were wondering if there was a similar way in SLURM to do so. I know you can do this via the accounting interface but it would be good if I didn't have to apply it as a blanket to all their jobs and if they could manage it themselves. If nothing exists in SLURM to do this that's fine. One can always engineer around it. I figured I would ping the dev list first before putting a nail in it. From my look at the documentation I don't see anyway to do this other than what I stated above. -Paul Edmon- -- Ryan Cox Operations Director Fulton Supercomputing Lab Brigham Young University
