[slurm-users] Prologslurmctld env variable issue

[Joe Teumer]

Apologies, I was not able to reply to the previous thread.[slurm-users]
Prologslurmctld environment variables (google.com)


I was not able to use "spank_job_control_setenv" successfully with sbatch.
https://slurm.schedmd.com/spank.html

I'm using a spank option named 'env':
sbatch --env=test

My example environment.c script called the following functions as a test:

slurm_spank_init -> called with sbatch -> however, "test" value is Null
slurm_spank_job_prolog -> not called with sbatch
slurm_spank_init_post_opt -> called with sbatch -> however, "test" value is
Null
slurm_spank_local_user_init -> not called with sbatch
slurm_spank_user_init -> not called with sbatch
slurm_spank_task_init_privileged -> not called with sbatch
slurm_spank_task_post_fork -> not called with sbatch

if (spank_job_control_setenv(sp, "env", global_variable, 1) !=
ESPANK_SUCCESS)
{
slurm_info("init | Unable to set env");
slurm_info("Value: %s",global_variable);
}
else {
slurm_info("init | Able to set env");
slurm_info("Value: %s",global_variable);
}

Output:
sbatch: init | Able to set BIOS
sbatch: Value:

How can I reference the "--env=test" value in my c script?
Is it possible to run 'spank_job_control_setenv' with sbatch?

Re: [slurm-users] Secondary Unix group id of users not being issued in interactive srun command

[Timo Rothenpieler]

Are you using LDAP for your users?
This sounds exactly like what I was seeing on our cluster when 
nsswitch.conf was not properly set up.


In my case, I was missing a line like

> initgroups: files [SUCCESS=continue] ldap

Just adding ldap to group: was not enough, and only got the primary 
group to work, exactly like in your case.



On 21/09/2021 09:11, Amjad Syed wrote:

Hello all

We have users who have have defined unix secondary id on our login nodes.

vas20xhu@login01 ~]$ groups

BIO_pg BIO_AFMAKAY_LAB_USERS



But when we run interactive  and go to compute node , the user does not 
have secondary  group of BIO_AFMAKAY_LAB_USERS



vas20xhu@c0077 ~]$ groups

BIO_pg


This is our interactive script


alias interactive='srun -n1 -p interactive -J interactive 
--time=12:00:00 --mem-per-cpu=4G --pty bash --login'


/usr/bin/srun



When we ssh directly into node without using interactive script there 
are no issues  with groups.



Anything we are missing in that interactive script?


Amjad

Re: [slurm-users] Secondary Unix group id of users not being issued in interactive srun command

[Bjørn-Helge Mevik]

Amjad Syed  writes:

> We have users who have have defined unix secondary id on our login nodes.
>
> vas20xhu@login01 ~]$ groups
> BIO_pg BIO_AFMAKAY_LAB_USERS
>
> But when we run interactive  and go to compute node , the user does not
> have secondary  group of BIO_AFMAKAY_LAB_USERS
>
> vas20xhu@c0077 ~]$ groups
> BIO_pg

[...]
> When we ssh directly into node without using interactive script there are
> no issues  with groups.

Have you set up your Slurm to be NSS provider for user and group info?
I believe that will only send primary group to the job step processes.
See the enable_nss_slurm LaunchParameters in man slurm.conf, and the URL
in that description.

-- 
Regards,
Bjørn-Helge Mevik



signature.asc
Description: PGP signature

Re: [slurm-users] Secondary Unix group id of users not being issued in interactive srun command

[Sternberger, Sven]

Hi!

I tried it (I normally would use sacct and ssh) but even with "srun bash 
--login" 
the "id" command gives me all groups. And we are using rfc2307bis.

And I don't think that slurm stores any group information. So it seems
to be the way the session starts on your node. But at this point
no idea where to look.

best regards

- Ursprüngliche Mail -
> Von: "Amjad Syed" 
> An: slurm-us...@schedmd.com
> Gesendet: Dienstag, 21. September 2021 09:11:33
> Betreff: [slurm-users] Secondary Unix group id of users not being issued in 
> interactive srun command

> Hello all
> 
> We have users who have have defined unix secondary id on our login nodes.
> 
> vas20xhu@login01 ~]$ groups
> 
> BIO_pg BIO_AFMAKAY_LAB_USERS
> 
> 
> 
> But when we run interactive  and go to compute node , the user does not
> have secondary  group of BIO_AFMAKAY_LAB_USERS
> 
> 
> vas20xhu@c0077 ~]$ groups
> 
> BIO_pg
> 
> 
> This is our interactive script
> 
> 
> alias interactive='srun -n1 -p interactive -J interactive --time=12:00:00
> --mem-per-cpu=4G --pty bash --login'
> 
> /usr/bin/srun
> 
> 
> 
> When we ssh directly into node without using interactive script there are
> no issues  with groups.
> 
> 
> Anything we are missing in that interactive script?
> 
> 
> Amjad


smime.p7s
Description: S/MIME Cryptographic Signature

Re: [slurm-users] Secondary Unix group id of users not being issued in interactive srun command

[Loris Bennett]

Ole Holm Nielsen  writes:

> On 9/21/21 9:11 AM, Amjad Syed wrote:
>> We have users who have have defined unix secondary id on our login nodes.
>>
>> vas20xhu@login01 ~]$ groups
>>
>> BIO_pg BIO_AFMAKAY_LAB_USERS
>>
>> But when we run interactive  and go to compute node , the user does not have
>> secondary  group of BIO_AFMAKAY_LAB_USERS
>>
>> vas20xhu@c0077 ~]$ groups
>>
>> BIO_pg
>
> I believe that Slurm creates users in the database using the primary UNIX 
> group
> name.  Slurm would not know about any secondary UNIX groups.
>
> There must be a uniform user and group name space (including UIDs and GIDs)
> across the cluster.  It is your own responsibility to configure users and 
> groups
> in the passwd and group databases consistently, see
> https://slurm.schedmd.com/quickstart_admin.html (search for GIDs).
>
> FWIW, I have some information about creation of users and groups in this Wiki
> page:
> https://wiki.fysik.dtu.dk/niflheim/Slurm_accounting#create-accounts-and-users

I would have thought that this maybe does not have anything to do with
Slurm.

Assuming you are using SSSD, it looks to me more like the settings in
sssd.conf on the nodes might be incorrect.  In our sssd.conf I found the
following note to myself:


  # LB: rfc2307bis should not be used if memberUid is used for group membership
  # otherwise secondary groups fail
  # ldap_schema = rfc2307bis

What schema you need depends on how your group information is stored.
rfc2307 assumes the groups just have a memberUID, whereas with rfc2307bis 
the users also have a memberOf attribute.

I don't understand LDAP well enough to understand why rfc2307bis causes
the secondary group resolution to fail, even though the groups still
have the information via memberUID, but my experience was that it does
indeed fail.

Cheers,

Loris

-- 
Dr. Loris Bennett (Hr./Mr.)
ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de

Re: [slurm-users] Secondary Unix group id of users not being issued in interactive srun command

[Ole Holm Nielsen]

On 9/21/21 9:11 AM, Amjad Syed wrote:

We have users who have have defined unix secondary id on our login nodes.

vas20xhu@login01 ~]$ groups

BIO_pg BIO_AFMAKAY_LAB_USERS

But when we run interactive  and go to compute node , the user does not 
have secondary  group of BIO_AFMAKAY_LAB_USERS


vas20xhu@c0077 ~]$ groups

BIO_pg


I believe that Slurm creates users in the database using the primary UNIX 
group name.  Slurm would not know about any secondary UNIX groups.


There must be a uniform user and group name space (including UIDs and 
GIDs) across the cluster.  It is your own responsibility to configure 
users and groups in the passwd and group databases consistently, see 
https://slurm.schedmd.com/quickstart_admin.html (search for GIDs).


FWIW, I have some information about creation of users and groups in this 
Wiki page:

https://wiki.fysik.dtu.dk/niflheim/Slurm_accounting#create-accounts-and-users

/Ole

[slurm-users] Secondary Unix group id of users not being issued in interactive srun command

[Amjad Syed]

Hello all

We have users who have have defined unix secondary id on our login nodes.

vas20xhu@login01 ~]$ groups

BIO_pg BIO_AFMAKAY_LAB_USERS



But when we run interactive  and go to compute node , the user does not
have secondary  group of BIO_AFMAKAY_LAB_USERS


vas20xhu@c0077 ~]$ groups

BIO_pg


This is our interactive script


alias interactive='srun -n1 -p interactive -J interactive --time=12:00:00
--mem-per-cpu=4G --pty bash --login'

/usr/bin/srun



When we ssh directly into node without using interactive script there are
no issues  with groups.


Anything we are missing in that interactive script?


Amjad