Re: [slurm-users] How do you make --export=NONE the default behavior for our cluster?

[Ward Poelmans]

Hi,

We're using a cli filter for doing this. But it's more tricky then just 
`--export=NONE`. For a srun inside a sbatch, you want `--export=ALL` again 
because MPI will break otherwise.

We have this in our cli filter:

function slurm_cli_pre_submit(options, pack_offset)

        local default_env = "HOME,USER,TERM,PATH=/bin:/sbin"



        if options["type"] == "sbatch" or (options["type"] == "srun" and options["pty"] 
== "set") then

        -- for sbatch and interactive srun, set our minimal environment

            if options["export"] == nil then

                options["export"] = default_env

                slurm.log_debug("Setting export to %s", 
tostring(options["export"]))

            end

        elseif options["type"] == "srun" then

        -- in case of srun, and we're in our minimal environment, pass again 
everything (for mpi)

            if options["export"] == default_env then

                options["export"] = "ALL"

                slurm.log_debug("Setting export to %s", 
tostring(options["export"]))

            end

        end

end

So we set `--export=HOME,USER,TERM,PATH=/bin:/sbin` if the user has not 
specified anything else. For a srun inside a sbatch, we switch again to 
`--export=ALL`.

Ward

On 3/06/2022 20:39, Ransom, Geoffrey M. wrote:

Hello

     We recently added new architectures to our compute and submit nodes and a 
PATH gets generated based on the type of machine our users log into. 
Unfortunately, this PATH is architecture dependent but it is getting copied 
with slurm jobs to the compute nodes which could be a different architecture as 
the submit node. Right now, running srun inside the sbatch script will fail 
with linking errors if you submit from a RHEL 8 box and land on a RHEL 7 box 
since it tries to run the RHEL 8 compiled version due to the PATH environment 
variable.

Adding “--export=NONE” to the job avoids the problem, but I’m not seeing a way 
to change this default behavior for the whole cluster.

How do you change/set default options for the cluster for all users so 
“--export=NONE” is the default behavior?

Is this change likely to break anything unexpected or cause problems?

Our users have NFS mounted home directories so if we removed the ssh non-admin 
user block they would get a similar/functional environment as they do on the 
submit nodes.

Thanks.

smime.p7s
Description: S/MIME Cryptographic Signature