Re: [slurm-users] Unconfigured GPUs being allocated

2023-07-19 Thread Wilson, Steven M
I found that this is actually a known bug in Slurm so I'll note it here in case anyone comes across this thread in the future:   https://bugs.schedmd.com/show_bug.cgi?id=10598 Steve From: slurm-users on behalf of Wilson, Steven M Sent: Tuesday, Ju

Re: [slurm-users] Unconfigured GPUs being allocated

2023-07-18 Thread Wilson, Steven M
eem to have an effect upon the actual environment of the job. Steve ____________ From: Wilson, Steven M Sent: Friday, July 14, 2023 4:10 PM To: slurm-users@lists.schedmd.com Subject: Re: [slurm-users] Unconfigured GPUs being allocated It's not so much whether a job may o

Re: [slurm-users] Unconfigured GPUs being allocated

2023-07-14 Thread Wilson, Steven M
Does Slurm support this? Best, Feng Best, Feng On Tue, Jun 27, 2023 at 9:54 AM Wilson, Steven M wrote: > > Hi, > > I manually configure the GPUs in our Slurm configuration (AutoDetect=off in > gres.conf) and everything works fine when all the GPUs in a node are > configure

Re: [slurm-users] Unconfigured GPUs being allocated

2023-07-14 Thread Wilson, Steven M
aring data On 7/14/23 10:20 am, Wilson, Steven M wrote: > I upgraded Slurm to 23.02.3 but I'm still running into the same problem. > Unconfigured GPUs (those absent from gres.conf and slurm.conf) are still > being made available to jobs so we end up with compute jobs being run on >

Re: [slurm-users] Unconfigured GPUs being allocated

2023-07-14 Thread Wilson, Steven M
Steve From: Wilson, Steven M Sent: Tuesday, June 27, 2023 9:50 AM To: slurm-users@lists.schedmd.com Subject: Unconfigured GPUs being allocated Hi, I manually configure the GPUs in our Slurm configuration (AutoDetect=off in gres.conf) and everything works fine when all the GPUs in a

[slurm-users] Unconfigured GPUs being allocated

2023-06-27 Thread Wilson, Steven M
Hi, I manually configure the GPUs in our Slurm configuration (AutoDetect=off in gres.conf) and everything works fine when all the GPUs in a node are configured in gres.conf and available to Slurm. But we have some nodes where a GPU is reserved for running the display and is specifically not co