I appreciate the input, Daniel. Both SLURM_HINT=nomultithread and the srun alias are options, though it would be preferable to roll this into the global SLURM config.
I'm not sure that CR_Core will give the desired behavior. It sounds as if all hyperthreads would be allocated to a job given that each thread is counted as an independent CPU. A little more digging into consumable resources makes me wonder if we could get the desired behavior with either SelectType by using the following SelectTypeParameter: > CR_ONE_TASK_PER_CORE Allocate one task per core by default. Without this > option, by default one task will be allocated per thread on nodes with more > than one ThreadsPerCore configured. Perhaps just setting ThreadsPerCore=1 would work. It's not clear how these SelectTypeParameters would play with --hint=multithread. Does anyone have any experience with this? would a user be allowed to override these settings with --hint=multithread if all hyperthreads were desired? It's also not clear how these options affect hybrid MPI+OpenMP jobs. The --hint=nomultithread option seems to prevent OMP threads from using multiple threads per core, which is the desired behavior. It's not obvious that restricting jobs to a single task per core with CR_ONE_TASK_PER_CORE or by using ThreadsPerCore=1 would give the same result. Maybe this calls for some experimentation. Thanks again for the input. -Kris On Wed, Jan 6, 2016 at 2:46 AM, Daniel Letai <[email protected]> wrote: > Just a couple of observations > 1) Naively you can create a skeleton sbatch template which normal jobs > would use by default, and another that multi-thread jobs must specifically > request. Populate the templates via a wrapper script or submit plugin - use > only sbatch, not srun. Other option would be to alias srun='srun > --hint=nomultithread' and use the non-aliased srun by specifying full path. > Not recommended, unless you educate your users to always use your > wrappers/aliases or rename the srun/sbatch to hide the underlying actual > commands. > 2) I'd look into slurm.conf's *SelectTypeParameters=* > *CR_Core *According to the man page: > > Cores are consumable resources. On nodes with hyper-threads, each thread > is counted as a CPU to satisfy a job's resource requirement, but multiple > jobs are not allocated threads on the same core. The count of CPUs > allocated to a job may be rounded up to account for every CPU on an > allocated core. > > So if you know the use case and you always need multithreads for same job, > this might solve your issue. > > > > On 01/06/2016 01:33 AM, Kris Kersten wrote: > > I'm trying to better understand the use of SLURM with hyperthreads. On a > system with hyperthreads enabled (i.e., in BIOS), by default SLURM presents > all hyperthreads to a user application. The user can restrict an > application to using a single hyperthread per core with the srun > --hint=nomultithread option. > > The documentation also states that the reverse is possible, that a user > can request to use all hyperthreads per core by calling srun > --hint=multithread. What I can't find is a configuration option to make > the --hint=nomultithread behavior the default, so that multiple > hyperthreads per core are made available only when specifically requested > with --hint=multithread. Is this possible? something in slurm.conf? or > elsewhere? > > Thanks, > Kris > > >
