(clumsy fingers) if I understand your question correctly, but maybe;
srun --cpu_bind=threads On Wed, Feb 8, 2017 at 4:02 PM, andrealphus <andrealp...@gmail.com> wrote: > srun --cpu_bind=cores > > On Wed, Feb 8, 2017 at 1:08 PM, Brendan Moloney <moloney.bren...@gmail.com > > wrote: > >> Hi, >> >> I want to allocate at the level of logical cores (each serial job gets >> one thread on a hyperthreading system), which seems to be achievable only >> by not setting threads_per_core on each node, and instead just setting >> CPUs=<total_logical_cores>. >> >> However with core binding, this will pack two tasks onto the same >> physical core while all other cores are left idle. On a system with 20 >> cores and 40 threads I see this behavior: >> >> $ srun bash -c "lstopo | head -n 6 ; sleep 10" & >> NUMANode L#0 (P#0 47GB) + Package L#0 + L3 L#0 (25MB) + L2 L#0 (256KB) >> + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 + PU L#0 (P#0) >> NUMANode L#1 (P#1 47GB) >> HostBridge L#0 >> PCIBridge >> PCI 15b3:1003 >> $ srun bash -c "lstopo | head -n 6 ; sleep 10" & >> NUMANode L#0 (P#0 47GB) + Package L#0 + L3 L#0 (25MB) + L2 L#0 (256KB) >> + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 + PU L#0 (P#20) >> NUMANode L#1 (P#1 47GB) >> HostBridge L#0 >> PCIBridge >> PCI 15b3:1003 >> >> I expected to see the second job get logical core #1 (on the second >> physical core) but instead it gets logical core #20 (the second thread on >> the first physical core). I can't imagine that this is ever the desired >> behavior, but I guess I could be missing some use case. >> >> I have spent quite a bit of time reading the documentation/mailing list >> and experimenting with different options, all to no avail. Is it possible >> to achieve my desired setup with Slurm? >> >> I also experimented with setting threads_per_core=2 and then setting >> OverSubscribe=FORCE:2, but I am pretty unhappy with the results. I think >> it is confusing that you request one cpu and end up with two (with your >> --mem-per-cpu doubled), and best I can tell there is no way to only >> oversubscribe if the user requested 1 core instead of two. >> >> Thanks for your time, >> Brendan >> >> >> >> >
