Thanks Reuti, the problem I have is that the queue system is used to handle jobs submitted through a web service. Each job is an array job, and if 10 web users submit a job, the last one in submitting has to wait until the rest is done for seeing at least some of their tasks processed. As the web users are not registered in the host system, GE see all jobs as from the same user (now thinking: is there any way to use fake users in GE?).
The max_aj_instances would mitigate the problem, but if the number of slots is lower than the number of webusers, some array jobs will have to wait anyways. cheers, -jaime On 10 August 2016 at 20:58, Reuti <[email protected]> wrote: > Hi, > > Am 10.08.2016 um 20:04 schrieb Jaime Huerta Cepas: > > > Hi there, > > I've been struggling for a while to find a configuration that allows me > to balance multiple arrays from the same user evenly. > > > > I tried adjusting all priorities to the same level, but array jobs are > always processed in submission order. I also tried assigning jobs to > different projects. Although this works for normal jobs, array jobs seem to > ignore the config (once an array job start being executed, all their tasks > are processed until the next array job is executed). > > Correct. > > > > am I missing something obvious? Is there any way to get that behaviour? > I would just need that the tasks from multiple array jobs from the same > user are distributed evenly, not following the submission order. > > AFAICS this won't speed up the overall workflow as it's the same user. The > last instance of the last array job will finish at the same time, even if > two ore more jobs are interleaved. > > Nevertheless you can limit the number of instances which are executed of > each particular array job to avoid that one job fills all empty slots in > the cluster. > > a) `qsub -tc <max_running_tasks> ...` on a per job basis (could even be > put in a JSV) > > b) the setting "max_aj_instances" in SGE's configuration `qconf -mconf` > > HTH -- Reuti > > > > thanks a lot in advance, > > -jaime > > _______________________________________________ > > users mailing list > > [email protected] > > https://gridengine.org/mailman/listinfo/users > >
_______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
