[slurm-users] How to request ONLY one CPU instead of one socket or one node?

[Wang, Liaoyuan]

Dear there,

I wrote an analytic program to analyze my data. The analysis costs around 
twenty days to analyze all data for one species. When I submit my job to the 
cluster, it always request one node instead of one CPU. I am wondering how I 
can ONLY request one CPU using "sbatch" command? Below is my batch file. Any 
comments and help would be highly appreciated.

Appreciatively,
Leon

#!/bin/sh

#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH -t 45-00:00:00
#SBATCH -J 9625%j
#SBATCH -o 9625.out
#SBATCH -e 9625.err

/home/scripts/wcnqn.auto.pl
===
Where wcnqn.auto.pl is my program. 9625 denotes the species number.

[slurm-users] How to request ONLY one CPU instead of one socket or one node?

[Wang, Liaoyuan]

Dear there,

I wrote an analytic program to analyze my data. The analysis costs around 
twenty days to analyze all data for one species. When I submit my job to the 
cluster, it always request one node instead of one CPU. I am wondering how I 
can ONLY request one CPU using "sbatch" command? Below is my batch file. Any 
comments and help would be highly appreciated.

Appreciatively,
Leon

#!/bin/sh

#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH -t 45-00:00:00
#SBATCH -J 9625%j
#SBATCH -o 9625.out
#SBATCH -e 9625.err

/home/scripts/wcnqn.auto.pl
===
Where wcnqn.auto.pl is my program. 9625 denotes the species number.

Re: [slurm-users] How to request ONLY one CPU instead of one socket or one node?

[Henkel, Andreas]

Hi Leon,
If the partition is defined to run jobs exclusive you always get a full node.
You’ll have to try to either split up your analysis in independent subtasks to 
be run in parallel by dividing the data or make use of some Perl 
parallelization package like parallel::Forkmanager to run steps of a for-loop 
in parallel. Then you would make use of the whole node and have your analysis 
finish earlier.
Best

Andreas

Am 13.02.2019 um 05:40 schrieb Wang, Liaoyuan 
mailto:wan...@alfred.edu>>:

Dear there,

I wrote an analytic program to analyze my data. The analysis costs around 
twenty days to analyze all data for one species. When I submit my job to the 
cluster, it always request one node instead of one CPU. I am wondering how I 
can ONLY request one CPU using “sbatch” command? Below is my batch file. Any 
comments and help would be highly appreciated.

Appreciatively,
Leon

#!/bin/sh

#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH -t 45-00:00:00
#SBATCH -J 9625%j
#SBATCH -o 9625.out
#SBATCH -e 9625.err

/home/scripts/wcnqn.auto.pl
===
Where wcnqn.auto.pl is my program. 9625 denotes the species number.

Re: [slurm-users] How to request ONLY one CPU instead of one socket or one node?

[Marcus Wagner]

Hi Leon,

depends on how the admins configured slurm. If they set select/linear, 
you have no chance to get just a core, as slurm will schedule only 
complete nodes.


Nonetheless, you omitted to tell slurm, how much memory you need (at 
least there is nothing in your script). Slurm will then also schedule a 
complete node.
Please try something like --mem=1024. This will restrict your job to 1 
GB memory. If you need more, you'll have to specify more.



Best
Marcus

On 2/13/19 5:19 AM, Wang, Liaoyuan wrote:


Dear there,

I wrote an analytic program to analyze my data. The analysis costs 
around twenty days to analyze all data for one species. When I submit 
my job to the cluster, it always request one node instead of one CPU. 
I am wondering how I can ONLY request one CPU using “sbatch” command? 
Below is my batch file. Any comments and help would be highly 
appreciated.


Appreciatively,

Leon



#!/bin/sh

#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH -t 45-00:00:00
#SBATCH -J 9625%j
#SBATCH -o 9625.out
#SBATCH -e 9625.err

/home/scripts/wcnqn.auto.pl

===

Where wcnqn.auto.pl is my program. 9625 denotes the species number.



--
Marcus Wagner, Dipl.-Inf.

IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
wag...@itc.rwth-aachen.de
www.itc.rwth-aachen.de

Re: [slurm-users] How to request ONLY one CPU instead of one socket or one node?

[David Rhey]

Hello,

Are you sure you're NOT getting 1 CPU when you run your job? You might want
to put some echo logic into your job to look at Slurm env variables of the
node your job lands on as a way of checking. E.g.:

echo $SLURM_CPUS_ON_NODE
echo $SLURM_JOB_CPUS_PER_NODE

I don't see anything wrong with your script. As a test I took the basic
parameters you've outlined and ran an interactive `srun` session,
requesting 1 CPU per task and 4 CPUs per task, and then looked at the
aforementioned variable output within each session. For example, requesting
1 CPU per task:

[drhey@beta-login ~]$ srun --cpus-per-task=1 --ntasks-per-node=1
--partition=standard --mem=1G --pty bash
[drhey@bn19 ~]$ echo $SLURM_CPUS_ON_NODE
1

And again, running this command now asking for 4 CPUs per task and then
echoing the env var:

[drhey@beta-login ~]$ srun --cpus-per-task=4 --ntasks-per-node=1
--partition=standard --mem=1G --pty bash
[drhey@bn19 ~]$ echo $SLURM_CPUS_ON_NODE
4

HTH!

David

On Wed, Feb 13, 2019 at 9:24 PM Wang, Liaoyuan  wrote:

> Dear there,
>
>
>
> I wrote an analytic program to analyze my data. The analysis costs around
> twenty days to analyze all data for one species. When I submit my job to
> the cluster, it always request one node instead of one CPU. I am wondering
> how I can ONLY request one CPU using “sbatch” command? Below is my batch
> file. Any comments and help would be highly appreciated.
>
>
>
> Appreciatively,
>
> Leon
>
> 
>
> #!/bin/sh
>
> #SBATCH --ntasks=1
> #SBATCH --cpus-per-task=1
> #SBATCH -t 45-00:00:00
> #SBATCH -J 9625%j
> #SBATCH -o 9625.out
> #SBATCH -e 9625.err
>
> /home/scripts/wcnqn.auto.pl
>
> ===
>
> Where wcnqn.auto.pl is my program. 9625 denotes the species number.
>
>
>


-- 
David Rhey
---
Advanced Research Computing - Technology Services
University of Michigan

Re: [slurm-users] How to request ONLY one CPU instead of one socket or one node?

[Merlin Hartley]

Seems like you aren't specifying a --mem option, so the default would be to ask 
for a whole-node’s worth of RAM thus you would use the whole node for each job.

Hope this is useful!


Merlin
--
Merlin Hartley
Computer Officer
MRC Mitochondrial Biology Unit
University of Cambridge
Cambridge, CB2 0XY
United Kingdom

> On 14 Feb 2019, at 02:21, Wang, Liaoyuan  > wrote:
> 
> Dear there,
>  
> I wrote an analytic program to analyze my data. The analysis costs around 
> twenty days to analyze all data for one species. When I submit my job to the 
> cluster, it always request one node instead of one CPU. I am wondering how I 
> can ONLY request one CPU using “sbatch” command? Below is my batch file. Any 
> comments and help would be highly appreciated.
>  
> Appreciatively,
> Leon
> 
> #!/bin/sh 
> 
> #SBATCH --ntasks=1 
> #SBATCH --cpus-per-task=1 
> #SBATCH -t 45-00:00:00 
> #SBATCH -J 9625%j 
> #SBATCH -o 9625.out 
> #SBATCH -e 9625.err 
> 
> /home/scripts/wcnqn.auto.pl
> ===
> Where wcnqn.auto.pl is my program. 9625 denotes the species number.



--
Merlin Hartley
Computer Officer
MRC Mitochondrial Biology Unit
University of Cambridge
Cambridge, CB2 0XY
United Kingdom

Re: [slurm-users] How to request ONLY one CPU instead of one socket or one node?

[Wang, Liaoyuan]

Dear David,

Thank you for your reply. I did it. It did request one CPU.

I added your command into my job file
#!/bin/sh
#SBATCH --partition=defq
#SBATCH --ntasks-per-node=2
#SBATCH -n 1
#SBATCH --mem-per-cpu=5gb
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH -t 45-00:00:00
#SBATCH -J 9673%j
#SBATCH -o 9673.out
#SBATCH -e 9673.err

echo $SLURM_CPUS_ON_NODE
echo $SLURM_JOB_CPUS_PER_NODE


time /home/scripts/wcnqn.auto.pl


Here is the results for
echo $SLURM_CPUS_ON_NODE   ==>   40
echo $SLURM_JOB_CPUS_PER_NODE  ==>  40

Here, the cluster has 8 nodes, each node has 2 sockets, each socket has 20 
cores, and each core has 2 threads. Thus, the value is 40.

If I use your command “srun --cpus-per-task=1 --ntasks-per-node=1 
--partition=standard --mem=1G wcnqn.auto.pl”, I cannot get any output.


I am wondering whether the configuration of the cluster is improper or not. Any 
further suggestion would be highly appreciated.

Appreciatively,
Leon


From: David Rhey 
Sent: Friday, February 15, 2019 07:54
To: Slurm User Community List 
Subject: Re: [slurm-users] How to request ONLY one CPU instead of one socket or 
one node?

Hello,

Are you sure you're NOT getting 1 CPU when you run your job? You might want to 
put some echo logic into your job to look at Slurm env variables of the node 
your job lands on as a way of checking. E.g.:

echo $SLURM_CPUS_ON_NODE
echo $SLURM_JOB_CPUS_PER_NODE

I don't see anything wrong with your script. As a test I took the basic 
parameters you've outlined and ran an interactive `srun` session, requesting 1 
CPU per task and 4 CPUs per task, and then looked at the aforementioned 
variable output within each session. For example, requesting 1 CPU per task:

[drhey@beta-login ~]$ srun --cpus-per-task=1 --ntasks-per-node=1 
--partition=standard --mem=1G --pty bash
[drhey@bn19 ~]$ echo $SLURM_CPUS_ON_NODE
1

And again, running this command now asking for 4 CPUs per task and then echoing 
the env var:

[drhey@beta-login ~]$ srun --cpus-per-task=4 --ntasks-per-node=1 
--partition=standard --mem=1G --pty bash
[drhey@bn19 ~]$ echo $SLURM_CPUS_ON_NODE
4

HTH!

David

On Wed, Feb 13, 2019 at 9:24 PM Wang, Liaoyuan 
mailto:wan...@alfred.edu>> wrote:
Dear there,

I wrote an analytic program to analyze my data. The analysis costs around 
twenty days to analyze all data for one species. When I submit my job to the 
cluster, it always request one node instead of one CPU. I am wondering how I 
can ONLY request one CPU using “sbatch” command? Below is my batch file. Any 
comments and help would be highly appreciated.

Appreciatively,
Leon

#!/bin/sh

#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH -t 45-00:00:00
#SBATCH -J 9625%j
#SBATCH -o 9625.out
#SBATCH -e 9625.err

/home/scripts/wcnqn.auto.pl<http://wcnqn.auto.pl>
===
Where wcnqn.auto.pl<http://wcnqn.auto.pl> is my program. 9625 denotes the 
species number.



--
David Rhey
---
Advanced Research Computing - Technology Services
University of Michigan