Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Mike Mikailov]

Hi Paul,Thank you for the explanations. Actually this was not the main point of the question asked. I think we can close the discussions. The main point was: why a job is running more efficiently using sbatch than salloc.Thank you all for the contributions. - MikeSent from my iPhoneOn Jul 5, 2023, at 12:33 PM, Paul H. Hargrove  wrote:Mike,I think your definitions are probably in the minority on this list.To be clear, I am not saying you (or SGE) are wrong, just that the folk here use different terms for what you are asking for.I think of it like dialects of English where the same food might be a "cookie" or a "biscuit" depending on the local dialect.To me, "interactive" means "keyboard interactive".If I want the most immediate access then "immediate" is the term I use.Fwiw, the `srun` and `salloc` commands have an `--immediate=time` option to bound how long they wait for resources.It is possible to define an "immediate" or "interactive" (or any other name you may choose) QOS to ensure that interactive jobs are scheduled on a distinct partition of the cluster and/or are given higher priority in scheduling of nodes which are also used for non-interactive jobs.I've just confirmed that LSF and PBS documentation use the term "interactive" in the same way as Slurm.-PaulOn Wed, Jul 5, 2023 at 7:06 AM Mike Mikailov  wrote:Thank you Loris, for the further feedback. 

“Reasonable” for SGE is within a few minutes, would be nice if it could be adjusted. 

Still interactive means the user has almost immediate access to the system, not queued. 

Sent from my iPhone

> On Jul 5, 2023, at 9:43 AM, Loris Bennett  wrote:
> 
> Mike Mikailov  writes:
> 
>> Thank you Loris, for the further clarifications. The only question is
>> who will wait forever in interactive mode? And how practical is it?
>> 
>> Interactive mode is what its name implies - interactive, not queueing.
> 
> To me, "interactive" is the alternative to "batch" - queueing happens
> with both.  Whether I submit an interactive job or a batch job, the job
> can only start if the requested resources are available and the job has
> a sufficiently high priority.
> 
> If you have enough nodes, you might be able to spare some to be
> essentially idle most of the time, so that jobs sent to these nodes
> will, in general, start quickly.  If you have a small cluster, this
> might not be an option. 
> 
>> It would make more sense if the default setting for deadline would be set to a reasonable time not indefinite in Slurm. 
> 
> One person's "reasonable" may be another person's "ridiculously long".
> A "reasonable time" may even vary for the same person depending on the
> circumstances.  If you submit an interactive job before lunch, you might
> be prepared to wait longer than if the building is going to close in 30
> minutes and you will have to leave.  With '--deadline' you can decide
> case by case.
> 
> Cheers,
> 
> Loris
> 
>> Sent from my iPhone
>> 
 On Jul 5, 2023, at 1:43 AM, Loris Bennett  wrote:
>>> 
>>> Mike Mikailov  writes:
>>> 
 About the last point. In the case of sbatch the jobs wait in the
 queue as long as it takes until the resources are available. In the
 case of interactive jobs
 (at least using Son of Grid Engine) they fail after a short time if no resources available. 
>>> 
>>> But you were referring to 'salloc' and not something SGE does.  The
>>> default for 'salloc' is to wait indefinitely.  You can change this
>>> behaviour with the option '--deadline':
>>> 
>>>  --deadline=
>>>          remove the job if no ending is possible before this deadline
>>>          (start > (deadline - time[-min])).  Default is no deadline.
>>>          Valid time formats are:
>>>          HH:MM[:SS] [AM|PM]
>>>          MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
>>>          MM/DD[/YY]-HH:MM[:SS]
>>>          -MM-DD[THH:MM[:SS]]]
>>>          now[+count[seconds(default)|minutes|hours|days|weeks]]
>>> 
>>> [snip (36 lines)]
 
 Queuing system  No  Yes  
 
 I am not sure what you mean with the last point, since 'salloc' is also
 handled by the queueing system.  If the resources requested are
 currently not available, 'salloc' will wait until they are.
>>> 
>>> [snip (42 lines)]
>>> 
>>> -- 
>>> Dr. Loris Bennett (Herr/Mr)
>>> ZEDAT, Freie Universität Berlin
>>> 
> -- 
> Dr. Loris Bennett (Herr/Mr)
> ZEDAT, Freie Universität Berlin
> 

-- Paul H. Hargrove Pronouns: he, him, hisComputer Languages & Systems Software (CLaSS) GroupComputer Science DepartmentLawrence Berkeley National Laboratory

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Paul H. Hargrove]

Mike,

I think your definitions are probably in the minority on this list.
To be clear, I am *not* saying you (or SGE) are wrong, just that the folk
here use different terms for what you are asking for.
I think of it like dialects of English where the same food might be a
"cookie" or a "biscuit" depending on the local dialect.

To me, "interactive" means "keyboard interactive".

If I want the most immediate access then "immediate" is the term I use.
Fwiw, the `srun` and `salloc` commands have an `--immediate=time` option to
bound how long they wait for resources.

It is possible to define an "immediate" or "interactive" (or any other name
you may choose) QOS to ensure that interactive jobs are scheduled on a
distinct partition of the cluster and/or are given higher priority in
scheduling of nodes which are also used for non-interactive jobs.

I've just confirmed that LSF and PBS documentation use the term
"interactive" in the same way as Slurm.

-Paul

On Wed, Jul 5, 2023 at 7:06 AM Mike Mikailov  wrote:

> Thank you Loris, for the further feedback.
>
> “Reasonable” for SGE is within a few minutes, would be nice if it could be
> adjusted.
>
> Still interactive means the user has almost immediate access to the
> system, not queued.
>
> Sent from my iPhone
>
> > On Jul 5, 2023, at 9:43 AM, Loris Bennett 
> wrote:
> >
> > Mike Mikailov  writes:
> >
> >> Thank you Loris, for the further clarifications. The only question is
> >> who will wait forever in interactive mode? And how practical is it?
> >>
> >> Interactive mode is what its name implies - interactive, not queueing.
> >
> > To me, "interactive" is the alternative to "batch" - queueing happens
> > with both.  Whether I submit an interactive job or a batch job, the job
> > can only start if the requested resources are available and the job has
> > a sufficiently high priority.
> >
> > If you have enough nodes, you might be able to spare some to be
> > essentially idle most of the time, so that jobs sent to these nodes
> > will, in general, start quickly.  If you have a small cluster, this
> > might not be an option.
> >
> >> It would make more sense if the default setting for deadline would be
> set to a reasonable time not indefinite in Slurm.
> >
> > One person's "reasonable" may be another person's "ridiculously long".
> > A "reasonable time" may even vary for the same person depending on the
> > circumstances.  If you submit an interactive job before lunch, you might
> > be prepared to wait longer than if the building is going to close in 30
> > minutes and you will have to leave.  With '--deadline' you can decide
> > case by case.
> >
> > Cheers,
> >
> > Loris
> >
> >> Sent from my iPhone
> >>
>  On Jul 5, 2023, at 1:43 AM, Loris Bennett 
> wrote:
> >>>
> >>> Mike Mikailov  writes:
> >>>
>  About the last point. In the case of sbatch the jobs wait in the
>  queue as long as it takes until the resources are available. In the
>  case of interactive jobs
>  (at least using Son of Grid Engine) they fail after a short time if
> no resources available.
> >>>
> >>> But you were referring to 'salloc' and not something SGE does.  The
> >>> default for 'salloc' is to wait indefinitely.  You can change this
> >>> behaviour with the option '--deadline':
> >>>
> >>>  --deadline=
> >>>  remove the job if no ending is possible before this deadline
> >>>  (start > (deadline - time[-min])).  Default is no deadline.
> >>>  Valid time formats are:
> >>>  HH:MM[:SS] [AM|PM]
> >>>  MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
> >>>  MM/DD[/YY]-HH:MM[:SS]
> >>>  -MM-DD[THH:MM[:SS]]]
> >>>  now[+count[seconds(default)|minutes|hours|days|weeks]]
> >>>
> >>> [snip (36 lines)]
> 
>  Queuing system  No  Yes
> 
>  I am not sure what you mean with the last point, since 'salloc' is
> also
>  handled by the queueing system.  If the resources requested are
>  currently not available, 'salloc' will wait until they are.
> >>>
> >>> [snip (42 lines)]
> >>>
> >>> --
> >>> Dr. Loris Bennett (Herr/Mr)
> >>> ZEDAT, Freie Universität Berlin
> >>>
> > --
> > Dr. Loris Bennett (Herr/Mr)
> > ZEDAT, Freie Universität Berlin
> >
>
>

-- 
Paul H. Hargrove 
Pronouns: he, him, his
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department
Lawrence Berkeley National Laboratory

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Mike Mikailov]

Thank you Loris, for the further feedback. 

“Reasonable” for SGE is within a few minutes, would be nice if it could be 
adjusted. 

Still interactive means the user has almost immediate access to the system, not 
queued. 

Sent from my iPhone

> On Jul 5, 2023, at 9:43 AM, Loris Bennett  wrote:
> 
> Mike Mikailov  writes:
> 
>> Thank you Loris, for the further clarifications. The only question is
>> who will wait forever in interactive mode? And how practical is it?
>> 
>> Interactive mode is what its name implies - interactive, not queueing.
> 
> To me, "interactive" is the alternative to "batch" - queueing happens
> with both.  Whether I submit an interactive job or a batch job, the job
> can only start if the requested resources are available and the job has
> a sufficiently high priority.
> 
> If you have enough nodes, you might be able to spare some to be
> essentially idle most of the time, so that jobs sent to these nodes
> will, in general, start quickly.  If you have a small cluster, this
> might not be an option. 
> 
>> It would make more sense if the default setting for deadline would be set to 
>> a reasonable time not indefinite in Slurm. 
> 
> One person's "reasonable" may be another person's "ridiculously long".
> A "reasonable time" may even vary for the same person depending on the
> circumstances.  If you submit an interactive job before lunch, you might
> be prepared to wait longer than if the building is going to close in 30
> minutes and you will have to leave.  With '--deadline' you can decide
> case by case.
> 
> Cheers,
> 
> Loris
> 
>> Sent from my iPhone
>> 
 On Jul 5, 2023, at 1:43 AM, Loris Bennett  
 wrote:
>>> 
>>> Mike Mikailov  writes:
>>> 
 About the last point. In the case of sbatch the jobs wait in the
 queue as long as it takes until the resources are available. In the
 case of interactive jobs
 (at least using Son of Grid Engine) they fail after a short time if no 
 resources available. 
>>> 
>>> But you were referring to 'salloc' and not something SGE does.  The
>>> default for 'salloc' is to wait indefinitely.  You can change this
>>> behaviour with the option '--deadline':
>>> 
>>>  --deadline=
>>>  remove the job if no ending is possible before this deadline
>>>  (start > (deadline - time[-min])).  Default is no deadline.
>>>  Valid time formats are:
>>>  HH:MM[:SS] [AM|PM]
>>>  MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
>>>  MM/DD[/YY]-HH:MM[:SS]
>>>  -MM-DD[THH:MM[:SS]]]
>>>  now[+count[seconds(default)|minutes|hours|days|weeks]]
>>> 
>>> [snip (36 lines)]
 
 Queuing system  No  Yes  
 
 I am not sure what you mean with the last point, since 'salloc' is also
 handled by the queueing system.  If the resources requested are
 currently not available, 'salloc' will wait until they are.
>>> 
>>> [snip (42 lines)]
>>> 
>>> -- 
>>> Dr. Loris Bennett (Herr/Mr)
>>> ZEDAT, Freie Universität Berlin
>>> 
> -- 
> Dr. Loris Bennett (Herr/Mr)
> ZEDAT, Freie Universität Berlin
> 

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Loris Bennett]

Mike Mikailov  writes:

> Thank you Loris, for the further clarifications. The only question is
> who will wait forever in interactive mode? And how practical is it?
>
> Interactive mode is what its name implies - interactive, not queueing.

To me, "interactive" is the alternative to "batch" - queueing happens
with both.  Whether I submit an interactive job or a batch job, the job
can only start if the requested resources are available and the job has
a sufficiently high priority.

If you have enough nodes, you might be able to spare some to be
essentially idle most of the time, so that jobs sent to these nodes
will, in general, start quickly.  If you have a small cluster, this
might not be an option. 

> It would make more sense if the default setting for deadline would be set to 
> a reasonable time not indefinite in Slurm. 

One person's "reasonable" may be another person's "ridiculously long".
A "reasonable time" may even vary for the same person depending on the
circumstances.  If you submit an interactive job before lunch, you might
be prepared to wait longer than if the building is going to close in 30
minutes and you will have to leave.  With '--deadline' you can decide
case by case.

Cheers,

Loris

> Sent from my iPhone
>
>> On Jul 5, 2023, at 1:43 AM, Loris Bennett  wrote:
>> 
>> Mike Mikailov  writes:
>> 
>>> About the last point. In the case of sbatch the jobs wait in the
>>> queue as long as it takes until the resources are available. In the
>>> case of interactive jobs
>>> (at least using Son of Grid Engine) they fail after a short time if no 
>>> resources available. 
>> 
>> But you were referring to 'salloc' and not something SGE does.  The
>> default for 'salloc' is to wait indefinitely.  You can change this
>> behaviour with the option '--deadline':
>> 
>>   --deadline=
>>   remove the job if no ending is possible before this deadline
>>   (start > (deadline - time[-min])).  Default is no deadline.
>>   Valid time formats are:
>>   HH:MM[:SS] [AM|PM]
>>   MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
>>   MM/DD[/YY]-HH:MM[:SS]
>>   -MM-DD[THH:MM[:SS]]]
>>   now[+count[seconds(default)|minutes|hours|days|weeks]]
>> 
>> [snip (36 lines)]
>>> 
>>> Queuing system  No  Yes  
>>> 
>>> I am not sure what you mean with the last point, since 'salloc' is also
>>> handled by the queueing system.  If the resources requested are
>>> currently not available, 'salloc' will wait until they are.
>> 
>> [snip (42 lines)]
>> 
>> -- 
>> Dr. Loris Bennett (Herr/Mr)
>> ZEDAT, Freie Universität Berlin
>> 
-- 
Dr. Loris Bennett (Herr/Mr)
ZEDAT, Freie Universität Berlin

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Mike Mikailov]

Thank you Loris, for the further clarifications. The only question is who will 
wait forever in interactive mode? And how practical is it?

Interactive mode is what its name implies - interactive, not queueing.

It would make more sense if the default setting for deadline would be set to a 
reasonable time not indefinite in Slurm. 


Sent from my iPhone

> On Jul 5, 2023, at 1:43 AM, Loris Bennett  wrote:
> 
> Mike Mikailov  writes:
> 
>> About the last point. In the case of sbatch the jobs wait in the queue as 
>> long as it takes until the resources are available. In the case of 
>> interactive jobs
>> (at least using Son of Grid Engine) they fail after a short time if no 
>> resources available. 
> 
> But you were referring to 'salloc' and not something SGE does.  The
> default for 'salloc' is to wait indefinitely.  You can change this
> behaviour with the option '--deadline':
> 
>   --deadline=
>   remove the job if no ending is possible before this deadline
>   (start > (deadline - time[-min])).  Default is no deadline.
>   Valid time formats are:
>   HH:MM[:SS] [AM|PM]
>   MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
>   MM/DD[/YY]-HH:MM[:SS]
>   -MM-DD[THH:MM[:SS]]]
>   now[+count[seconds(default)|minutes|hours|days|weeks]]
> 
> [snip (36 lines)]
>> 
>> Queuing system  No  Yes  
>> 
>> I am not sure what you mean with the last point, since 'salloc' is also
>> handled by the queueing system.  If the resources requested are
>> currently not available, 'salloc' will wait until they are.
> 
> [snip (42 lines)]
> 
> -- 
> Dr. Loris Bennett (Herr/Mr)
> ZEDAT, Freie Universität Berlin
> 

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Loris Bennett]

Mike Mikailov  writes:

> About the last point. In the case of sbatch the jobs wait in the queue as 
> long as it takes until the resources are available. In the case of 
> interactive jobs
> (at least using Son of Grid Engine) they fail after a short time if no 
> resources available. 

But you were referring to 'salloc' and not something SGE does.  The
default for 'salloc' is to wait indefinitely.  You can change this
behaviour with the option '--deadline':

   --deadline=
   remove the job if no ending is possible before this deadline
   (start > (deadline - time[-min])).  Default is no deadline.
   Valid time formats are:
   HH:MM[:SS] [AM|PM]
   MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
   MM/DD[/YY]-HH:MM[:SS]
   -MM-DD[THH:MM[:SS]]]
   now[+count[seconds(default)|minutes|hours|days|weeks]]
 
[snip (36 lines)]
>
>  Queuing system  No  Yes  
>
>  I am not sure what you mean with the last point, since 'salloc' is also
>  handled by the queueing system.  If the resources requested are
>  currently not available, 'salloc' will wait until they are.

[snip (42 lines)]

-- 
Dr. Loris Bennett (Herr/Mr)
ZEDAT, Freie Universität Berlin

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Mike Mikailov]

Nodes for salloc could also be allowed to be oversubscribed or overloaded. There are a number of tools that can be used to study task performance bottlenecks on HPC clusters. Some of these tools include:SLURM Profiler: The SLURM Profiler is a tool that can be used to collect performance data for SLURM jobs. This data can be used to identify bottlenecks in the job execution process, such as slow nodes, I/O bottlenecks, and memory contention.Ganglia: Ganglia is a monitoring system that can be used to collect performance data for HPC clusters. This data can be used to identify bottlenecks in the cluster, such as overloaded nodes, high network traffic, and slow storage devices.PerfTools: PerfTools is a set of tools that can be used to collect performance data for Linux systems. This data can be used to identify bottlenecks in the system, such as CPU usage, memory usage, and I/O activity.VTune Amplifier: VTune Amplifier is a tool from Intel that can be used to collect performance data for Intel processors. This data can be used to identify bottlenecks in the processor, such as cache misses, branch mispredictions, and memory latency.HPCToolkit: HPCToolkit is a suite of tools that can be used to collect performance data for HPC applications. This data can be used to identify bottlenecks in the application, such as inefficient algorithms, memory leaks, and threading issues.The best tool for a particular situation will depend on the specific needs of the user. However, all of the tools listed above can be used to identify task performance bottlenecks on HPC clusters.In addition to these tools, there are a number of other things that can be done to study task performance bottlenecks on HPC clusters. These include:Reviewing the job submission scripts: The job submission scripts can be reviewed to ensure that they are using the correct resources and that they are submitting the jobs to the correct nodes.Monitoring the job execution: The job execution can be monitored to track the progress of the jobs and to identify any potential problems.Analyzing the performance data: The performance data can be analyzed to identify the specific bottlenecks that are impacting the performance of the jobs.Tuning the jobs: The jobs can be tuned to improve their performance. This may involve changing the parameters of the jobs, using different algorithms, or using different libraries.By taking these steps, it is possible to identify and address task performance bottlenecks on HPC clusters. This can help to improve the performance of the jobs and to get the most out of the HPC resources.Sent from my iPhoneOn Jul 4, 2023, at 9:04 AM, Татьяна Озерова  wrote:Thank you for your answer! And if slurm workers are identical, what can be the reason? Can interactive mode affect the performance? I have submitted the task with the help of "srun {{ name_of_task }} --pty bash", and the result is the same as for launching with salloc. Thanks in advance!вт, 4 июл. 2023 г. в 15:51, Mike Mikailov :They should not affect the task performance.May be the cluster configuration allocated slow machines for salloc. salloc and sbatch have different purposes:salloc is used to allocate a set of resources to a job. Once the resources have been allocated, the user can run a command or script on the allocated resources.sbatch is used to submit a batch script to Slurm. The batch script contains a list of commands or scripts that will be executed on the allocated resources.In general, salloc is used for jobs that need to be run interactively, such as jobs that require a shell or jobs that need to be debugged. sbatch is used for jobs that can be run in the background, such as long-running jobs or jobs that are submitted by a queuing system.Here is a table that summarizes the key differences between salloc and sbatch:FeaturesallocsbatchPurposeAllocate resources and run a command or scriptSubmit a batch scriptInteractiveYesNoBackgroundNoYesQueuing systemNoYesHere are some examples of how to use salloc and sbatch:To allocate 2 nodes with 4 CPUs each and run the command ls, you would use the following command:Code snippetsalloc -N 2 -c 4 ls
To submit a batch script called my_job.sh that contains the command python my_script.py, you would use the following command:Code snippetsbatch my_job.sh
For more information on salloc and sbatch, please see the following documentation:salloc documentation: https://slurm.schedmd.com/salloc.htmlsbatch documentation: https://slurm.schedmd.com/sbatch.htmlSent from my iPhoneOn Jul 4, 2023, at 8:22 AM, Татьяна Озерова  wrote:Hello! I have question about way of launching tasks in Slurm. I use the service in cloud and submit an application with sbatch or salloc. As far as I am concerned, the commands are similar: they allocate resources for counting users tasks and run them. However, I have received different results in cluster performance for the same task (task execution time is too long in 

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Mike Mikailov]

About the last point. In the case of sbatch the jobs wait in the queue as long 
as it takes until the resources are available. In the case of interactive jobs 
(at least using Son of Grid Engine) they fail after a short time if no 
resources available. 

Sent from my iPhone

> On Jul 4, 2023, at 9:11 AM, Loris Bennett  wrote:
> 
> Mike Mikailov  writes:
> 
>> They should not affect the task performance.
>> 
>> May be the cluster configuration allocated slow machines for salloc. 
>> 
>> salloc and sbatch have different purposes:
>> 
>> * salloc is used to allocate a set of resources to a job. Once the resources 
>> have been allocated, the user can run a command or script on the allocated
>> resources.
>> * sbatch is used to submit a batch script to Slurm. The batch script 
>> contains a list of commands or scripts that will be executed on the allocated
>> resources.
>> 
>> In general, salloc is used for jobs that need to be run interactively, such 
>> as jobs that require a shell or jobs that need to be debugged. sbatch is used
>> for jobs that can be run in the background, such as long-running jobs or 
>> jobs that are submitted by a queuing system.
>> 
>> Here is a table that summarizes the key differences between salloc and 
>> sbatch:
>> 
>> Feature  salloc  sbatch  
>> Purpose  Allocate resources and run a command or script  Submit a batch 
>> script  
>> Interactive  Yes  No  
>> Background  No  Yes  
>> Queuing system  No  Yes  
> 
> I am not sure what you mean with the last point, since 'salloc' is also
> handled by the queueing system.  If the resources requested are
> currently not available, 'salloc' will wait until they are.
> 
> Cheers,
> 
> Loris
> 
>> Here are some examples of how to use salloc and sbatch:
>> 
>> * To allocate 2 nodes with 4 CPUs each and run the command ls, you would use 
>> the following command:
>> 
>> Code snippet
>> salloc -N 2 -c 4 ls
>> 
>> * To submit a batch script called my_job.sh that contains the command python 
>> my_script.py, you would use the following command:
>> 
>> Code snippet
>> sbatch my_job.sh
>> 
>> For more information on salloc and sbatch, please see the following 
>> documentation:
>> 
>> * salloc documentation: https://slurm.schedmd.com/salloc.html
>> * sbatch documentation: https://slurm.schedmd.com/sbatch.html
>> 
>> Sent from my iPhone
>> 
>> On Jul 4, 2023, at 8:22 AM, Татьяна Озерова  
>> wrote:
>> 
>> Hello! I have question about way of launching tasks in Slurm. I use the 
>> service in cloud and submit an application with sbatch or salloc. As far as I
>> am concerned, the commands are similar: they allocate resources for counting 
>> users tasks and run them. However, I have received different
>> results in cluster performance for the same task (task execution time is too 
>> long in case of salloc). So my question is what is the difference
>> between these two commands, that can affect on task performance? Thank you 
>> beforehand.
>> 
> -- 
> Dr. Loris Bennett (Herr/Mr)
> ZEDAT, Freie Universität Berlin
> 

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Mike Mikailov]

Performance depends only on the worker node capabilities along with IO. If the worker nodes are the same then may be the nodes under salloc use network drive (main storage) for IO which may slow down the tasks.There are many tools available to localize the bottleneck in the task performance. You may also measure bandwidth between the local node and the main storage using “dd” command for instance.Sent from my iPhoneOn Jul 4, 2023, at 9:04 AM, Татьяна Озерова  wrote:Thank you for your answer! And if slurm workers are identical, what can be the reason? Can interactive mode affect the performance? I have submitted the task with the help of "srun {{ name_of_task }} --pty bash", and the result is the same as for launching with salloc. Thanks in advance!вт, 4 июл. 2023 г. в 15:51, Mike Mikailov :They should not affect the task performance.May be the cluster configuration allocated slow machines for salloc. salloc and sbatch have different purposes:salloc is used to allocate a set of resources to a job. Once the resources have been allocated, the user can run a command or script on the allocated resources.sbatch is used to submit a batch script to Slurm. The batch script contains a list of commands or scripts that will be executed on the allocated resources.In general, salloc is used for jobs that need to be run interactively, such as jobs that require a shell or jobs that need to be debugged. sbatch is used for jobs that can be run in the background, such as long-running jobs or jobs that are submitted by a queuing system.Here is a table that summarizes the key differences between salloc and sbatch:FeaturesallocsbatchPurposeAllocate resources and run a command or scriptSubmit a batch scriptInteractiveYesNoBackgroundNoYesQueuing systemNoYesHere are some examples of how to use salloc and sbatch:To allocate 2 nodes with 4 CPUs each and run the command ls, you would use the following command:Code snippetsalloc -N 2 -c 4 ls
To submit a batch script called my_job.sh that contains the command python my_script.py, you would use the following command:Code snippetsbatch my_job.sh
For more information on salloc and sbatch, please see the following documentation:salloc documentation: https://slurm.schedmd.com/salloc.htmlsbatch documentation: https://slurm.schedmd.com/sbatch.htmlSent from my iPhoneOn Jul 4, 2023, at 8:22 AM, Татьяна Озерова  wrote:Hello! I have question about way of launching tasks in Slurm. I use the service in cloud and submit an application with sbatch or salloc. As far as I am concerned, the commands are similar: they allocate resources for counting users tasks and run them. However, I have received different results in cluster performance for the same task (task execution time is too long in case of salloc). So my question is what is the difference between these two commands, that can affect on task performance? Thank you beforehand.

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Татьяна Озерова]

Thank you for your answer! And if slurm workers are identical, what can be
the reason? Can interactive mode affect the performance? I have submitted
the task with the help of "srun {{ name_of_task }} --pty bash", and the
result is the same as for launching with salloc. Thanks in advance!


вт, 4 июл. 2023 г. в 15:51, Mike Mikailov :

> They should not affect the task performance.
>
> May be the cluster configuration allocated slow machines for *salloc*.
>
> *salloc* and *sbatch* have different purposes:
>
>- *salloc* is used to allocate a set of resources to a job. Once the
>resources have been allocated, the user can run a command or script on the
>allocated resources.
>- *sbatch* is used to submit a batch script to Slurm. The batch script
>contains a list of commands or scripts that will be executed on the
>allocated resources.
>
> In general, *salloc* is used for jobs that need to be run interactively,
> such as jobs that require a shell or jobs that need to be debugged.
> *sbatch* is used for jobs that can be run in the background, such as
> long-running jobs or jobs that are submitted by a queuing system.
>
> Here is a table that summarizes the key differences between salloc and
> sbatch:
> Featuresallocsbatch
> Purpose Allocate resources and run a command or script Submit a batch
> script
> Interactive Yes No
> Background No Yes
> Queuing system No Yes
>
> Here are some examples of how to use salloc and sbatch:
>
>- To allocate 2 nodes with 4 CPUs each and run the command ls, you
>would use the following command:
>
> Code snippet
>
> salloc -N 2 -c 4 ls
>
>
>- To submit a batch script called my_job.sh that contains the command 
> python
>my_script.py, you would use the following command:
>
> Code snippet
>
> sbatch my_job.sh
>
> For more information on salloc and sbatch, please see the following
> documentation:
>
>- salloc documentation: https://slurm.schedmd.com/salloc.html
>- sbatch documentation: https://slurm.schedmd.com/sbatch.html
>
>
>
> Sent from my iPhone
>
> On Jul 4, 2023, at 8:22 AM, Татьяна Озерова 
> wrote:
>
> 
> Hello! I have question about way of launching tasks in Slurm. I use the
> service in cloud and submit an application with sbatch or salloc. As far as
> I am concerned, the commands are similar: they allocate resources for
> counting users tasks and run them. However, I have received different
> results in cluster performance for the same task (task execution time is
> too long in case of salloc). So my question is what is the difference
> between these two commands, that can affect on task performance? Thank you
> beforehand.
>
>

[slurm-users] Slurm commands for launching tasks: salloc and sbatch

[Татьяна Озерова]

Hello! I have question about way of launching tasks in Slurm. I use the
service in cloud and submit an application with sbatch or salloc. As far as
I am concerned, the commands are similar: they allocate resources for
counting users tasks and run them. However, I have received different
results in cluster performance for the same task (task execution time is
too long in case of salloc). So my question is what is the difference
between these two commands, that can affect on task performance? Thank you
beforehand.