[slurm-dev] Re: Build error on CentOS 5.10

[Dennis Zheleznyak]

Hi Chris,

* file /usr/bin/perl*
/usr/bin/perl: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for
GNU/Linux 2.6.9, dynamically linked (uses shared libs), stripped

*ls -l /usr/bin/perl*
-rwxr-xr-x 2 root root 19208 Nov  7  2013 /usr/bin/perl

*getfacl /usr/bin/perl*
getfacl: Removing leading '/' from absolute path names
# file: usr/bin/perl
# owner: root
# group: root
user::rwx
group::r-x
other::r-x

*getenforce*
Disabled

Thank you,
Dennis.


On Thu, Oct 2, 2014 at 9:43 AM, Christopher Samuel 
wrote:

>
> On 02/10/14 16:19, Dennis Zheleznyak wrote:
>
> > + /var/tmp/slurm-14.03.8-1/usr/bin/sjobexitmod --roff
> > /var/tmp/rpm-tmp.79119: /var/tmp/slurm-14.03.8-1/usr/bin/sjobexitmod:
> > /usr/bin/perl: bad interpreter: Permission denied
> > error: Bad exit status from /var/tmp/rpm-tmp.79119 (%install)
>
> What do the following commands say?
>
> file /usr/bin/perl
> ls -l /usr/bin/perl
> getfacl /usr/bin/perl
>
> Do you have SE Linux enabled?
>
> cheers,
> Chris
> --
>  Christopher SamuelSenior Systems Administrator
>  VLSCI - Victorian Life Sciences Computation Initiative
>  Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
>  http://www.vlsci.org.au/  http://twitter.com/vlsci
>

[slurm-dev] Unsubscribe

[Roman Sirokov]

blank

[slurm-dev] Re: Build error on CentOS 5.10

[Chris Samuel]

On Thu, 2 Oct 2014 12:15:10 AM Dennis Zheleznyak wrote:

> Hi Chris,

Hiya,

Those all look fine, but I've just noticed you're building RPMs there and 
that's something we don't do with Slurm so I'm afraid I'm not sure I can help 
there, sorry! :-(

All the best,
Chris
-- 
 Christopher SamuelSenior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/  http://twitter.com/vlsci

[slurm-dev] Authentication and invoking slurm commands from web app

[José Román Bilbao Castro]

Hi all,

First of all, this is my very first message to the list and don't even know
if this is the proper place to port this message.

I am facing a simple project that should allow a slurm user to monitor his
jobs running on a slurm server. I have been looking at the Slurm
authentication API but I cannot find anything useful for me as this seems
to be applied to users already logged in the system. My question is where
to start looking at (technologies, web development frameworks, etc...) to
be able to enter a user/password on the web browser that coincides with
that of the Linux user, send the credentials to the server, execute a slurm
command on behalf of that user and print results back...

May be this is a very complex question, but I have not much experience in
web development and how it should be done to link slurm commands execution,
specific user authorization, etc...

Thanks in advance,

Jose

-- 


*José Román Bilbao Castro*

Ingeniero Consultor
+34 901009188

*jrbc...@idiria.com **http://www.idiria.com
* <*http:// www.idiria.com/
*>

--
Idiria Sociedad Limitada - Aviso legal

Este mensaje, su contenido y cualquier fichero transmitido con él está
dirigido únicamente a su destinatario y es confidencial. Por ello, se
informa a quien lo reciba por error ó tenga conocimiento del mismo sin ser
su destinatario, que la información contenida en él es reservada y su uso
no autorizado, por lo que en tal caso le rogamos nos lo comunique por la
misma  vía o por teléfono (+ 34 690207492), así como que se abstenga de
reproducir el mensaje mediante cualquier medio o remitirlo o entregarlo a
otra persona, procediendo a su borrado de manera inmediata.

Idiria Sociedad Limitada se reserva las acciones legales que le
correspondan contra todo tercero que acceda de forma  ilegítima al
contenido de cualquier mensaje externo procedente del mismo.

Para información y consultas visite nuestra web http://www.idiria.com



Idiria Sociedad Limitada - Disclaimer
This message, its content and any file attached thereto is for the intended
recipient only and is confidential. If you have received this e-mail in
error or had access to it, you should note that the information in it is
private and any use thereof is unauthorised. In such an event please notify
us by e-mail or by telephone (+ 34 690207492). Any reproduction of this
e-mail by whatsoever means and any transmission or dissemination thereof to
other persons is prohibited. It should be deleted immediately from your
system.

Idiria Sociedad Limitada reserves the right to take legal action against
any persons unlawfully gaining access to the content of any external
message it has emitted.

For additional information, please visit our website http://www.idiria.com

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[Brian B.]

Hello Jose,

It is never a good idea to have the public facing credentials be the same as 
the private credentials. That is if your public facing server is compromised 
your internal system is compromised. The limited cases where direct internal 
access are needed (e.g. SSH) should be handled by hardened servers. 

Allowing users to input executable commands on a webpage is also not a good 
security practice. This is essentially how the shellshock bug works. 

This is just my take on things but I would suggest building a different system. 

--
Regards,
Brian

> On Oct 2, 2014, at 06:40, José Román Bilbao Castro  wrote:
> 
> Hi all,
> 
> First of all, this is my very first message to the list and don't even know 
> if this is the proper place to port this message. 
> 
> I am facing a simple project that should allow a slurm user to monitor his 
> jobs running on a slurm server. I have been looking at the Slurm 
> authentication API but I cannot find anything useful for me as this seems to 
> be applied to users already logged in the system. My question is where to 
> start looking at (technologies, web development frameworks, etc...) to be 
> able to enter a user/password on the web browser that coincides with that of 
> the Linux user, send the credentials to the server, execute a slurm command 
> on behalf of that user and print results back... 
> 
> May be this is a very complex question, but I have not much experience in web 
> development and how it should be done to link slurm commands execution, 
> specific user authorization, etc... 
> 
> Thanks in advance,
> 
> Jose
> 
> -- 
> 
> José Román Bilbao Castro
> 
> Ingeniero Consultor
> +34 901009188
> jrbc...@idiria.com
> http://www.idiria.com 
> 
> --
> Idiria Sociedad Limitada - Aviso legal
> 
> Este mensaje, su contenido y cualquier fichero transmitido con él está 
> dirigido únicamente a su destinatario y es confidencial. Por ello, se informa 
> a quien lo reciba por error ó tenga conocimiento del mismo sin ser su 
> destinatario, que la información contenida en él es reservada y su uso no 
> autorizado, por lo que en tal caso le rogamos nos lo comunique por la misma  
> vía o por teléfono (+ 34 690207492), así como que se abstenga de reproducir 
> el mensaje mediante cualquier medio o remitirlo o entregarlo a otra persona, 
> procediendo a su borrado de manera inmediata. 
>  
> Idiria Sociedad Limitada se reserva las acciones legales que le correspondan 
> contra todo tercero que acceda de forma  ilegítima al contenido de cualquier 
> mensaje externo procedente del mismo. 
> 
> Para información y consultas visite nuestra web http://www.idiria.com 
> 
>  
> 
> Idiria Sociedad Limitada - Disclaimer
> This message, its content and any file attached thereto is for the intended 
> recipient only and is confidential. If you have received this e-mail in error 
> or had access to it, you should note that the information in it is private 
> and any use thereof is unauthorised. In such an event please notify us by 
> e-mail or by telephone (+ 34 690207492). Any reproduction of this e-mail by 
> whatsoever means and any transmission or dissemination thereof to other 
> persons is prohibited. It should be deleted immediately from your system.
> 
> Idiria Sociedad Limitada reserves the right to take legal action against any 
> persons unlawfully gaining access to the content of any external message it 
> has emitted.
> 
> For additional information, please visit our website http://www.idiria.com 
> 
>  
> 

[slurm-dev] Re: Build error on CentOS 5.10

[Dennis Zheleznyak]

Thank you for trying :)

On Thu, Oct 2, 2014 at 1:40 PM, Chris Samuel  wrote:

>
> On Thu, 2 Oct 2014 12:15:10 AM Dennis Zheleznyak wrote:
>
> > Hi Chris,
>
> Hiya,
>
> Those all look fine, but I've just noticed you're building RPMs there and
> that's something we don't do with Slurm so I'm afraid I'm not sure I can
> help
> there, sorry! :-(
>
> All the best,
> Chris
> --
>  Christopher SamuelSenior Systems Administrator
>  VLSCI - Victorian Life Sciences Computation Initiative
>  Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
>  http://www.vlsci.org.au/  http://twitter.com/vlsci
>

[slurm-dev] Re: Changed behaviour of --exclusive in srun (job step context)

[Magnus Jonsson]

Is no one else affected by this?

/Magnus

On 2014-09-11 14:46, Magnus Jonsson wrote:

Hi!

A user found a "strange" new behaviour when using --exclusive with srun.

I have an example submit-script[1] that shows this.

I have tested this on 2.6.4 with the output [2] & [3] (stderr) and on
14.03.7 with the output [4] & [5] (stderr).

In 14.03.7 without --exclusive behaves like 2.6.4 with --exclusive.
In 14.03.7 with --exclusive get some kind of "node exclusive" within the
job. --overcommit gets the same behaviour on both versions.

in 14.03.7 -c3 does not seams to work at all in job step context I see
warnings about this in the man page for srun but in 2.6.4 this works as
I aspect. Stderr output from srun: "srun: error: Unable to create job
step: Requested node configuration is not available"


If you need more information please let me know.

Best regards,
Magnus

1, http://www.hpc2n.umu.se/staff/magnus/slurm/submit.sh
2, http://www.hpc2n.umu.se/staff/magnus/slurm/stdout.2.6.4
3, http://www.hpc2n.umu.se/staff/magnus/slurm/stderr.2.6.4
4, http://www.hpc2n.umu.se/staff/magnus/slurm/stdout.14.03.7
5, http://www.hpc2n.umu.se/staff/magnus/slurm/stderr.14.03.7



--
Magnus Jonsson, Developer, HPC2N, Umeå Universitet



smime.p7s
Description: S/MIME Cryptographic Signature

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[José Román Bilbao Castro]

Thanks Brian,

So you propose to have something like an intermediate database that maps
web portal users to system users and make all calls internally from the
webserver, right?. I just wanted to avoid the intermediate step for
simplicity, but it seems to be a bad practice.

So, regarding the second step... what is the safest and more logical manner
of invoking slurm commands from the webserver?. I mean, at the end I must
pass some credentials for the right user... Or should I have a tomcat user
that belongs to the sudo group and call invoke commands as another user?. I
am totally lost and need some thread to start pulling from it.

Thanks again,

Jose


2014-10-02 13:23 GMT+02:00 Brian B. :

> Hello Jose,
>
> It is never a good idea to have the public facing credentials be the same
> as the private credentials. That is if your public facing server is
> compromised your internal system is compromised. The limited cases where
> direct internal access are needed (e.g. SSH) should be handled by hardened
> servers.
>
> Allowing users to input executable commands on a webpage is also not a
> good security practice. This is essentially how the shellshock bug works.
>
> This is just my take on things but I would suggest building a different
> system.
>
> --
> Regards,
> Brian
>
> On Oct 2, 2014, at 06:40, José Román Bilbao Castro 
> wrote:
>
> Hi all,
>
> First of all, this is my very first message to the list and don't even
> know if this is the proper place to port this message.
>
> I am facing a simple project that should allow a slurm user to monitor his
> jobs running on a slurm server. I have been looking at the Slurm
> authentication API but I cannot find anything useful for me as this seems
> to be applied to users already logged in the system. My question is where
> to start looking at (technologies, web development frameworks, etc...) to
> be able to enter a user/password on the web browser that coincides with
> that of the Linux user, send the credentials to the server, execute a slurm
> command on behalf of that user and print results back...
>
> May be this is a very complex question, but I have not much experience in
> web development and how it should be done to link slurm commands execution,
> specific user authorization, etc...
>
> Thanks in advance,
>
> Jose
>
> --
>
>
> *José Román Bilbao Castro*
>
> Ingeniero Consultor
> +34 901009188
>
> *jrbc...@idiria.com **http://www.idiria.com
> * <*http:// www.idiria.com/
> *>
>
> --
> Idiria Sociedad Limitada - Aviso legal
>
> Este mensaje, su contenido y cualquier fichero transmitido con él está
> dirigido únicamente a su destinatario y es confidencial. Por ello, se
> informa a quien lo reciba por error ó tenga conocimiento del mismo sin ser
> su destinatario, que la información contenida en él es reservada y su uso
> no autorizado, por lo que en tal caso le rogamos nos lo comunique por la
> misma  vía o por teléfono (+ 34 690207492), así como que se abstenga de
> reproducir el mensaje mediante cualquier medio o remitirlo o entregarlo a
> otra persona, procediendo a su borrado de manera inmediata.
>
> Idiria Sociedad Limitada se reserva las acciones legales que le
> correspondan contra todo tercero que acceda de forma  ilegítima al
> contenido de cualquier mensaje externo procedente del mismo.
>
> Para información y consultas visite nuestra web http://www.idiria.com
>
>
>
> Idiria Sociedad Limitada - Disclaimer
> This message, its content and any file attached thereto is for the
> intended recipient only and is confidential. If you have received this
> e-mail in error or had access to it, you should note that the information
> in it is private and any use thereof is unauthorised. In such an event
> please notify us by e-mail or by telephone (+ 34 690207492). Any
> reproduction of this e-mail by whatsoever means and any transmission or
> dissemination thereof to other persons is prohibited. It should be deleted
> immediately from your system.
>
> Idiria Sociedad Limitada reserves the right to take legal action against
> any persons unlawfully gaining access to the content of any external
> message it has emitted.
>
> For additional information, please visit our website http://www.idiria.com
>
>
>
>
>


-- 


*José Román Bilbao Castro*

Ingeniero Consultor
+34 901009188

*jrbc...@idiria.com **http://www.idiria.com
* <*http:// www.idiria.com/
*>

--
Idiria Sociedad Limitada - Aviso legal

Este mensaje, su contenido y cualquier fichero transmitido con él está
dirigido únicamente a su destinatario y es confidencial. Por ello, se
informa a quien lo reciba por error ó tenga conocimiento del mismo sin ser
su destinatario, que la información contenida en él es reservada y su uso
no autorizado, por lo que en tal caso le rogamos nos lo comunique por la
misma  vía o por teléfono (+ 34 690207492), así como que se abstenga de
reproducir el 

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[José Román Bilbao Castro]

Thanks Lech,

That is something to start with. The problem is that I plan to add
submission in the future and don't want to start something that will have
to be changed too much with time. So I would prefer to be able to firstly
execute any slurm command from my webserver and for any user...

Regards,

Jose

2014-10-02 15:28 GMT+02:00 Lech Nieroda :

> Hello José,
>
> you might be interested in ubmod or its successor open xdmod. It's a
> system that queries SLURM regularly, writes the data into its own database
> and makes it available via webserver. You'd probably have to implement
> proper security measures for user  management.
>
> Regards,
> Lech
>
> (sent from mobile)
> Am 02.10.2014 14:38 schrieb =?ISO-8859-1?Q?Jos=E9_Rom=E1n_Bilbao_Castro?= <
> jrbc...@idiria.com>:
>
>  Thanks Brian,
>
> So you propose to have something like an intermediate database that maps
> web portal users to system users and make all calls internally from the
> webserver, right?. I just wanted to avoid the intermediate step for
> simplicity, but it seems to be a bad practice.
>
> So, regarding the second step... what is the safest and more logical
> manner of invoking slurm commands from the webserver?. I mean, at the end I
> must pass some credentials for the right user... Or should I have a tomcat
> user that belongs to the sudo group and call invoke commands as another
> user?. I am totally lost and need some thread to start pulling from it.
>
> Thanks again,
>
> Jose
>
>
> 2014-10-02 13:23 GMT+02:00 Brian B. :
>
>> Hello Jose,
>>
>> It is never a good idea to have the public facing credentials be the same
>> as the private credentials. That is if your public facing server is
>> compromised your internal system is compromised. The limited cases where
>> direct internal access are needed (e.g. SSH) should be handled by hardened
>> servers.
>>
>> Allowing users to input executable commands on a webpage is also not a
>> good security practice. This is essentially how the shellshock bug works.
>>
>> This is just my take on things but I would suggest building a different
>> system.
>>
>> --
>> Regards,
>> Brian
>>
>> On Oct 2, 2014, at 06:40, José Román Bilbao Castro 
>> wrote:
>>
>> Hi all,
>>
>> First of all, this is my very first message to the list and don't even
>> know if this is the proper place to port this message.
>>
>> I am facing a simple project that should allow a slurm user to monitor
>> his jobs running on a slurm server. I have been looking at the Slurm
>> authentication API but I cannot find anything useful for me as this seems
>> to be applied to users already logged in the system. My question is where
>> to start looking at (technologies, web development frameworks, etc...) to
>> be able to enter a user/password on the web browser that coincides with
>> that of the Linux user, send the credentials to the server, execute a slurm
>> command on behalf of that user and print results back...
>>
>> May be this is a very complex question, but I have not much experience in
>> web development and how it should be done to link slurm commands execution,
>> specific user authorization, etc...
>>
>> Thanks in advance,
>>
>> Jose
>>
>> --
>>
>>
>> *José Román Bilbao Castro*
>>
>> Ingeniero Consultor
>> +34 901009188
>>
>> *jrbc...@idiria.com **http://www.idiria.com
>> * <*http:// www.idiria.com/
>> *>
>>
>> --
>> Idiria Sociedad Limitada - Aviso legal
>>
>> Este mensaje, su contenido y cualquier fichero transmitido con él está
>> dirigido únicamente a su destinatario y es confidencial. Por ello, se
>> informa a quien lo reciba por error ó tenga conocimiento del mismo sin ser
>> su destinatario, que la información contenida en él es reservada y su uso
>> no autorizado, por lo que en tal caso le rogamos nos lo comunique por la
>> misma  vía o por teléfono (+ 34 690207492), así como que se abstenga de
>> reproducir el mensaje mediante cualquier medio o remitirlo o entregarlo a
>> otra persona, procediendo a su borrado de manera inmediata.
>>
>> Idiria Sociedad Limitada se reserva las acciones legales que le
>> correspondan contra todo tercero que acceda de forma  ilegítima al
>> contenido de cualquier mensaje externo procedente del mismo.
>>
>> Para información y consultas visite nuestra web http://www.idiria.com
>>
>>
>>
>> Idiria Sociedad Limitada - Disclaimer
>> This message, its content and any file attached thereto is for the
>> intended recipient only and is confidential. If you have received this
>> e-mail in error or had access to it, you should note that the information
>> in it is private and any use thereof is unauthorised. In such an event
>> please notify us by e-mail or by telephone (+ 34 690207492). Any
>> reproduction of this e-mail by whatsoever means and any transmission or
>> dissemination thereof to other persons is prohibited. It should be deleted
>> immediately from your system.
>>
>> Idiria Sociedad Limitada reserves the right to take legal

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[jette]

Brigham Young University has developed a number of web interfaces to  
SLurm. See:

https://marylou.byu.edu/documentation/slurm/script-generator
https://marylou.byu.edu/utilization/

Their Javascript tool to generate batch job scripts is here:
https://github.com/BYUHPC/BYUJobScriptGenerator




Quoting José Román Bilbao Castro :


Thanks Lech,

That is something to start with. The problem is that I plan to add
submission in the future and don't want to start something that will have
to be changed too much with time. So I would prefer to be able to firstly
execute any slurm command from my webserver and for any user...

Regards,

Jose

2014-10-02 15:28 GMT+02:00 Lech Nieroda :


Hello José,

you might be interested in ubmod or its successor open xdmod. It's a
system that queries SLURM regularly, writes the data into its own database
and makes it available via webserver. You'd probably have to implement
proper security measures for user  management.

Regards,
Lech

(sent from mobile)
Am 02.10.2014 14:38 schrieb =?ISO-8859-1?Q?Jos=E9_Rom=E1n_Bilbao_Castro?= <
jrbc...@idiria.com>:

 Thanks Brian,

So you propose to have something like an intermediate database that maps
web portal users to system users and make all calls internally from the
webserver, right?. I just wanted to avoid the intermediate step for
simplicity, but it seems to be a bad practice.

So, regarding the second step... what is the safest and more logical
manner of invoking slurm commands from the webserver?. I mean, at the end I
must pass some credentials for the right user... Or should I have a tomcat
user that belongs to the sudo group and call invoke commands as another
user?. I am totally lost and need some thread to start pulling from it.

Thanks again,

Jose


2014-10-02 13:23 GMT+02:00 Brian B. :


Hello Jose,

It is never a good idea to have the public facing credentials be the same
as the private credentials. That is if your public facing server is
compromised your internal system is compromised. The limited cases where
direct internal access are needed (e.g. SSH) should be handled by hardened
servers.

Allowing users to input executable commands on a webpage is also not a
good security practice. This is essentially how the shellshock bug works.

This is just my take on things but I would suggest building a different
system.

--
Regards,
Brian

On Oct 2, 2014, at 06:40, José Román Bilbao Castro 
wrote:

Hi all,

First of all, this is my very first message to the list and don't even
know if this is the proper place to port this message.

I am facing a simple project that should allow a slurm user to monitor
his jobs running on a slurm server. I have been looking at the Slurm
authentication API but I cannot find anything useful for me as this seems
to be applied to users already logged in the system. My question is where
to start looking at (technologies, web development frameworks, etc...) to
be able to enter a user/password on the web browser that coincides with
that of the Linux user, send the credentials to the server, execute a slurm
command on behalf of that user and print results back...

May be this is a very complex question, but I have not much experience in
web development and how it should be done to link slurm commands execution,
specific user authorization, etc...

Thanks in advance,

Jose

--


*José Román Bilbao Castro*

Ingeniero Consultor
+34 901009188

*jrbc...@idiria.com **http://www.idiria.com
* <*http:// www.idiria.com/
*>

--
Idiria Sociedad Limitada - Aviso legal

Este mensaje, su contenido y cualquier fichero transmitido con él está
dirigido únicamente a su destinatario y es confidencial. Por ello, se
informa a quien lo reciba por error ó tenga conocimiento del mismo sin ser
su destinatario, que la información contenida en él es reservada y su uso
no autorizado, por lo que en tal caso le rogamos nos lo comunique por la
misma  vía o por teléfono (+ 34 690207492), así como que se abstenga de
reproducir el mensaje mediante cualquier medio o remitirlo o entregarlo a
otra persona, procediendo a su borrado de manera inmediata.

Idiria Sociedad Limitada se reserva las acciones legales que le
correspondan contra todo tercero que acceda de forma  ilegítima al
contenido de cualquier mensaje externo procedente del mismo.

Para información y consultas visite nuestra web http://www.idiria.com



Idiria Sociedad Limitada - Disclaimer
This message, its content and any file attached thereto is for the
intended recipient only and is confidential. If you have received this
e-mail in error or had access to it, you should note that the information
in it is private and any use thereof is unauthorised. In such an event
please notify us by e-mail or by telephone (+ 34 690207492). Any
reproduction of this e-mail by whatsoever means and any transmission or
dissemination thereof to other persons is prohibited. It should be deleted
immediately from your system.

Idiria Soci

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[José Román Bilbao Castro]

Nice !!, I think this gives a much more detailed insight into the problem I
am facing !.

Thanks a lot!

2014-10-02 15:51 GMT+02:00 :

>
> Brigham Young University has developed a number of web interfaces to
> SLurm. See:
> https://marylou.byu.edu/documentation/slurm/script-generator
> https://marylou.byu.edu/utilization/
>
> Their Javascript tool to generate batch job scripts is here:
> https://github.com/BYUHPC/BYUJobScriptGenerator
>
>
>
>
>
> Quoting José Román Bilbao Castro :
>
>  Thanks Lech,
>>
>> That is something to start with. The problem is that I plan to add
>> submission in the future and don't want to start something that will have
>> to be changed too much with time. So I would prefer to be able to firstly
>> execute any slurm command from my webserver and for any user...
>>
>> Regards,
>>
>> Jose
>>
>> 2014-10-02 15:28 GMT+02:00 Lech Nieroda :
>>
>>  Hello José,
>>>
>>> you might be interested in ubmod or its successor open xdmod. It's a
>>> system that queries SLURM regularly, writes the data into its own
>>> database
>>> and makes it available via webserver. You'd probably have to implement
>>> proper security measures for user  management.
>>>
>>> Regards,
>>> Lech
>>>
>>> (sent from mobile)
>>> Am 02.10.2014 14:38 schrieb =?ISO-8859-1?Q?Jos=E9_Rom=E1n_Bilbao_Castro?=
>>> <
>>> jrbc...@idiria.com>:
>>>
>>>  Thanks Brian,
>>>
>>> So you propose to have something like an intermediate database that maps
>>> web portal users to system users and make all calls internally from the
>>> webserver, right?. I just wanted to avoid the intermediate step for
>>> simplicity, but it seems to be a bad practice.
>>>
>>> So, regarding the second step... what is the safest and more logical
>>> manner of invoking slurm commands from the webserver?. I mean, at the
>>> end I
>>> must pass some credentials for the right user... Or should I have a
>>> tomcat
>>> user that belongs to the sudo group and call invoke commands as another
>>> user?. I am totally lost and need some thread to start pulling from it.
>>>
>>> Thanks again,
>>>
>>> Jose
>>>
>>>
>>> 2014-10-02 13:23 GMT+02:00 Brian B. :
>>>
>>>  Hello Jose,

 It is never a good idea to have the public facing credentials be the
 same
 as the private credentials. That is if your public facing server is
 compromised your internal system is compromised. The limited cases where
 direct internal access are needed (e.g. SSH) should be handled by
 hardened
 servers.

 Allowing users to input executable commands on a webpage is also not a
 good security practice. This is essentially how the shellshock bug
 works.

 This is just my take on things but I would suggest building a different
 system.

 --
 Regards,
 Brian

 On Oct 2, 2014, at 06:40, José Román Bilbao Castro 
 wrote:

 Hi all,

 First of all, this is my very first message to the list and don't even
 know if this is the proper place to port this message.

 I am facing a simple project that should allow a slurm user to monitor
 his jobs running on a slurm server. I have been looking at the Slurm
 authentication API but I cannot find anything useful for me as this
 seems
 to be applied to users already logged in the system. My question is
 where
 to start looking at (technologies, web development frameworks, etc...)
 to
 be able to enter a user/password on the web browser that coincides with
 that of the Linux user, send the credentials to the server, execute a
 slurm
 command on behalf of that user and print results back...

 May be this is a very complex question, but I have not much experience
 in
 web development and how it should be done to link slurm commands
 execution,
 specific user authorization, etc...

 Thanks in advance,

 Jose

 --


 *José Román Bilbao Castro*

 Ingeniero Consultor
 +34 901009188

 *jrbc...@idiria.com **http://www.idiria.com
 * <*http:// www.idiria.com/
 *>


 --
 Idiria Sociedad Limitada - Aviso legal

 Este mensaje, su contenido y cualquier fichero transmitido con él está
 dirigido únicamente a su destinatario y es confidencial. Por ello, se
 informa a quien lo reciba por error ó tenga conocimiento del mismo sin
 ser
 su destinatario, que la información contenida en él es reservada y su
 uso
 no autorizado, por lo que en tal caso le rogamos nos lo comunique por la
 misma  vía o por teléfono (+ 34 690207492), así como que se abstenga de
 reproducir el mensaje mediante cualquier medio o remitirlo o entregarlo
 a
 otra persona, procediendo a su borrado de manera inmediata.

 Idiria Sociedad Limitada se reserva las acciones legales que le
 correspondan contra todo tercero que acceda de forma  ilegít

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[José Román Bilbao Castro]

It seems I was to fast... They don't seem to have open-sourced code. In
fact, they ask for specific Keys for each implementation so I suppose this
is a closed project for their users only... :-(

2014-10-02 15:51 GMT+02:00 :

>
> Brigham Young University has developed a number of web interfaces to
> SLurm. See:
> https://marylou.byu.edu/documentation/slurm/script-generator
> https://marylou.byu.edu/utilization/
>
> Their Javascript tool to generate batch job scripts is here:
> https://github.com/BYUHPC/BYUJobScriptGenerator
>
>
>
>
>
> Quoting José Román Bilbao Castro :
>
>  Thanks Lech,
>>
>> That is something to start with. The problem is that I plan to add
>> submission in the future and don't want to start something that will have
>> to be changed too much with time. So I would prefer to be able to firstly
>> execute any slurm command from my webserver and for any user...
>>
>> Regards,
>>
>> Jose
>>
>> 2014-10-02 15:28 GMT+02:00 Lech Nieroda :
>>
>>  Hello José,
>>>
>>> you might be interested in ubmod or its successor open xdmod. It's a
>>> system that queries SLURM regularly, writes the data into its own
>>> database
>>> and makes it available via webserver. You'd probably have to implement
>>> proper security measures for user  management.
>>>
>>> Regards,
>>> Lech
>>>
>>> (sent from mobile)
>>> Am 02.10.2014 14:38 schrieb =?ISO-8859-1?Q?Jos=E9_Rom=E1n_Bilbao_Castro?=
>>> <
>>> jrbc...@idiria.com>:
>>>
>>>  Thanks Brian,
>>>
>>> So you propose to have something like an intermediate database that maps
>>> web portal users to system users and make all calls internally from the
>>> webserver, right?. I just wanted to avoid the intermediate step for
>>> simplicity, but it seems to be a bad practice.
>>>
>>> So, regarding the second step... what is the safest and more logical
>>> manner of invoking slurm commands from the webserver?. I mean, at the
>>> end I
>>> must pass some credentials for the right user... Or should I have a
>>> tomcat
>>> user that belongs to the sudo group and call invoke commands as another
>>> user?. I am totally lost and need some thread to start pulling from it.
>>>
>>> Thanks again,
>>>
>>> Jose
>>>
>>>
>>> 2014-10-02 13:23 GMT+02:00 Brian B. :
>>>
>>>  Hello Jose,

 It is never a good idea to have the public facing credentials be the
 same
 as the private credentials. That is if your public facing server is
 compromised your internal system is compromised. The limited cases where
 direct internal access are needed (e.g. SSH) should be handled by
 hardened
 servers.

 Allowing users to input executable commands on a webpage is also not a
 good security practice. This is essentially how the shellshock bug
 works.

 This is just my take on things but I would suggest building a different
 system.

 --
 Regards,
 Brian

 On Oct 2, 2014, at 06:40, José Román Bilbao Castro 
 wrote:

 Hi all,

 First of all, this is my very first message to the list and don't even
 know if this is the proper place to port this message.

 I am facing a simple project that should allow a slurm user to monitor
 his jobs running on a slurm server. I have been looking at the Slurm
 authentication API but I cannot find anything useful for me as this
 seems
 to be applied to users already logged in the system. My question is
 where
 to start looking at (technologies, web development frameworks, etc...)
 to
 be able to enter a user/password on the web browser that coincides with
 that of the Linux user, send the credentials to the server, execute a
 slurm
 command on behalf of that user and print results back...

 May be this is a very complex question, but I have not much experience
 in
 web development and how it should be done to link slurm commands
 execution,
 specific user authorization, etc...

 Thanks in advance,

 Jose

 --


 *José Román Bilbao Castro*

 Ingeniero Consultor
 +34 901009188

 *jrbc...@idiria.com **http://www.idiria.com
 * <*http:// www.idiria.com/
 *>


 --
 Idiria Sociedad Limitada - Aviso legal

 Este mensaje, su contenido y cualquier fichero transmitido con él está
 dirigido únicamente a su destinatario y es confidencial. Por ello, se
 informa a quien lo reciba por error ó tenga conocimiento del mismo sin
 ser
 su destinatario, que la información contenida en él es reservada y su
 uso
 no autorizado, por lo que en tal caso le rogamos nos lo comunique por la
 misma  vía o por teléfono (+ 34 690207492), así como que se abstenga de
 reproducir el mensaje mediante cualquier medio o remitirlo o entregarlo
 a
 otra persona, procediendo a su borrado de manera inmediata.

 Idiria Sociedad Limitada se reserva la

[slurm-dev] jobs not using available resources (stuck in queue)

[Dr. Eddie]

I have a job:

scontrol show jobid 49
JobId=49
Name=20141001_17h23_C4_ArXe13nm_dt002.as_nbions00309_nbions00252_SyesIyesEyesTnoRyes_potSymmetric_b01.20_I01.5000e+14_WL13.7nm_FW10.0fs_Lrelaxed_A0.0_ALmd_oclAmd_deepthought

   UserId=zhart(1001) GroupId=zhart(1001)
   Priority=4294901712 Account=(null) QOS=(null)
   JobState=PENDING Reason=Resources Dependency=(null)
   Requeue=1 Restarts=0 BatchFlag=1 ExitCode=0:0
   RunTime=00:00:00 TimeLimit=UNLIMITED TimeMin=N/A
   SubmitTime=2014-10-01T17:23:40 EligibleTime=2014-10-01T17:23:40
   StartTime=2015-09-30T08:10:54 EndTime=Unknown
   PreemptTime=None SuspendTime=None SecsPreSuspend=0
   Partition=mem AllocNode:Sid=deepthought:658
   ReqNodeList=(null) ExcNodeList=(null)
   NodeList=(null)
   NumNodes=1 NumCPUs=16 CPUs/Task=1 ReqS:C:T=*:*:*
   MinCPUsNode=1 MinMemoryNode=0 MinTmpDiskNode=0
   Features=(null) Gres=(null) Reservation=(null)
   Shared=0 Contiguous=0 Licenses=(null) Network=(null)

Command=/raid/zhart/code/md/output/20141001_17h23_C4_ArXe13nm_dt002.as_nbions00309_nbions00252_SyesIyesEyesTnoRyes_potSymmetric_b01.20_I01.5000e+14_WL13.7nm_FW10.0fs_Lrelaxed_A0.0_ALmd_oclAmd_deepthought/slurm_20141001_17h23.sh


WorkDir=/raid/zhart/code/md/output/20141001_17h23_C4_ArXe13nm_dt002.as_nbions00309_nbions00252_SyesIyesEyesTnoRyes_potSymmetric_b01.20_I01.5000e+14_WL13.7nm_FW10.0fs_Lrelaxed_A0.0_ALmd_oclAmd_deepthought


says it does not have the resources needed to execute.  However, I have
four nodes with a configuration of

NodeName=node8 Arch=x86_64 CoresPerSocket=16
   CPUAlloc=16 CPUErr=0 CPUTot=32 CPULoad=15.98 Features=(null)
   Gres=gpu:4
   NodeAddr=node8 NodeHostName=node8
   OS=Linux RealMemory=257951 AllocMem=0 Sockets=2 Boards=1
   State=MIXED ThreadsPerCore=1 TmpDisk=0 Weight=1
   BootTime=2014-09-29T08:54:37 SlurmdStartTime=2014-09-29T08:55:08
   CurrentWatts=0 LowestJoules=0 ConsumedJoules=0
   ExtSensorsJoules=n/s ExtSensorsWatts=0 ExtSensorsTemp=n/s

which says only 16/32 cpus are in use, so job 49 should have resources to
run.

As you can see the node is in a mixed state since it is already running a
gres=4 cpu=16 run.

my slurm.conf is
ControlMachine=deepthought
#ControlAddr=
#BackupController=
#BackupAddr=
#
AuthType=auth/munge
CacheGroups=0
#CheckpointType=checkpoint/none
CryptoType=crypto/munge
#DisableRootJobs=NO
#EnforcePartLimits=NO
#Epilog=
#PrologSlurmctld=
#FirstJobId=1
#MaxJobId=99
GresTypes=gpu
#GroupUpdateForce=0
#GroupUpdateTime=600
#JobCheckpointDir=/var/slurm/checkpoint
#JobCredentialPrivateKey=
#JobCredentialPublicCertificate=
#JobFileAppend=0
#JobRequeue=1
#JobSubmitPlugins=1
#KillOnBadExit=0
#Licenses=foo*4,bar
#MailProg=/bin/mail
#MaxJobCount=5000
#MaxStepCount=4
#MaxTasksPerNode=128
MpiDefault=none
#MpiParams=ports=#-#
#PluginDir=
#PlugStackConfig=
#PrivateData=jobs
ProctrackType=proctrack/pgid
#Prolog=
#PrologSlurmctld=
#PropagatePrioProcess=0
#PropagateResourceLimits=
#PropagateResourceLimitsExcept=
ReturnToService=2
#SallocDefaultCommand=
SlurmctldPidFile=/var/run/slurm/slurmctld.pid
SlurmctldPort=6817
SlurmdPidFile=/var/run/slurm/slurmd.pid
SlurmdPort=6818
SlurmdSpoolDir=/tmp/slurm/slurmd
SlurmUser=slurm
#SrunEpilog=
#SrunProlog=
StateSaveLocation=/tmp/slurm
SwitchType=switch/none
#TaskEpilog=
TaskPlugin=task/none
#TaskPluginParam=
#TaskProlog=
#TopologyPlugin=topology/tree
#TmpFs=/tmp
#TrackWCKey=no
#TreeWidth=
#UnkillableStepProgram=
#UsePAM=0
#
#
# TIMERS
#BatchStartTimeout=10
#CompleteWait=0
#EpilogMsgTime=2000
#GetEnvTimeout=2
#HealthCheckInterval=0
#HealthCheckProgram=
InactiveLimit=0
KillWait=30
#MessageTimeout=10
#ResvOverRun=0
MinJobAge=300
#OverTimeLimit=0
SlurmctldTimeout=120
SlurmdTimeout=300
#UnkillableStepTimeout=60
#VSizeFactor=0
Waittime=0
#
#
# SCHEDULING
#DefMemPerCPU=0
FastSchedule=1
#MaxMemPerCPU=0
#SchedulerRootFilter=1
#SchedulerTimeSlice=30
SchedulerType=sched/backfill
SchedulerPort=7321
SelectType=select/cons_res
#SelectTypeParameters=
#
#
# JOB PRIORITY
#PriorityType=priority/basic
#PriorityDecayHalfLife=
#PriorityCalcPeriod=
#PriorityFavorSmall=
#PriorityMaxAge=
#PriorityUsageResetPeriod=
#PriorityWeightAge=
#PriorityWeightFairshare=
#PriorityWeightJobSize=
#PriorityWeightPartition=
#PriorityWeightQOS=
#
#
# LOGGING AND ACCOUNTING
#AccountingStorageEnforce=0
#AccountingStorageHost=
#AccountingStorageLoc=
#AccountingStoragePass=
#AccountingStoragePort=
AccountingStorageType=accounting_storage/none
#AccountingStorageUser=
AccountingStoreJobComment=YES
ClusterName=deepthought
#DebugFlags=
#JobCompHost=
#JobCompLoc=
#JobCompPass=
#JobCompPort=
JobCompType=jobcomp/none
#JobCompUser=
JobAcctGatherFrequency=30
JobAcctGatherType=jobacct_gather/none
SlurmctldDebug=3
SlurmctldLogFile=/var/log/slurm/slurmctld.log
SlurmdDebug=3
SlurmdLogFile=/var/log/slurm/slurmd.log
#SlurmSchedLogFile=
#SlurmSchedLogLevel=
#
#
# POWER SAVE SUPPORT FOR IDLE NODES (optional)
#SuspendProgram=
#ResumeProgram=
#SuspendTimeout=
#ResumeTimeout=
#ResumeRate=
#SuspendExcNodes=
#SuspendExcParts=
#SuspendRat

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[Lech Nieroda]

Hello José,
you might be interested in ubmod or its successor open xdmod. It's a system 
that queries SLURM regularly, writes the data into its own database and makes 
it available via webserver. You'd probably have to implement proper security 
measures for user  management.
Regards,
Lech
(sent from mobile)
Am 02.10.2014 14:38 schrieb =?ISO-8859-1?Q?Jos=E9_Rom=E1n_Bilbao_Castro?= 
: 
Re: [slurm-dev] Re: Authentication and invoking slurm commands from
 web app
Thanks Brian,So you propose to have something like an intermediate database 
that maps web portal users to system users and make all calls internally from 
the webserver, right?. I just wanted to avoid the intermediate step for 
simplicity, but it seems to be a bad practice. So, regarding the second step... 
what is the safest and more logical manner of invoking slurm commands from the 
webserver?. I mean, at the end I must pass some credentials for the right 
user... Or should I have a tomcat user that belongs to the sudo group and call 
invoke commands as another user?. I am totally lost and need some thread to 
start pulling from it.Thanks again,Jose2014-10-02 13:23 GMT+02:00 Brian B. 
:Hello Jose,It is never a good idea to have the public facing 
credentials be the same as the private credentials. That is if your public 
facing server is compromised your internal system is compromised. The limited 
cases where direct internal access are needed (e.g. SSH) should be handled by 
hardened servers. Allowing users to input executable commands on a webpage is 
also not a good security practice. This is essentially how the shellshock bug 
works. This is just my take on things but I would suggest building a different 
system. --Regards,BrianOn Oct 2, 2014, at 06:40, José Román Bilbao Castro 
 wrote: 
Hi all,First of all, this is my very first message to the list and don't even 
know if this is the proper place to port this message. I
 am facing a simple project that should allow a slurm user to monitor 
his jobs running on a slurm server. I have been looking at the Slurm 
authentication API but I cannot find anything useful for me as this 
seems to be applied to users already logged in the system. My question 
is where to start looking at (technologies, web development frameworks, 
etc...) to be able to enter a user/password on the web browser that 
coincides with that of the Linux user, send the credentials to the 
server, execute a slurm command on behalf of that user and print results
 back... May be this is a very complex question, but I 
have not much experience in web development and how it should be done to
 link slurm commands execution, specific user authorization, etc... Thanks in 
advance,Jose-- *José Román Bilbao Castro*Ingeniero Consultor+34 
901009188jrbcast@idiria.com_http://www.idiria.com_ 
--Idiria Sociedad Limitada - Aviso legalEste mensaje, 
su contenido y cualquier fichero transmitido con él está dirigido únicamente a 
su destinatario y es confidencial. Por ello, se informa a quien lo reciba por 
error ó tenga conocimiento del mismo sin ser su destinatario, que la 
información contenida en él es reservada y su uso no autorizado, por lo que en 
tal caso le rogamos nos lo comunique por la misma  vía o por teléfono (+ 34 
690207492), así como que se abstenga de reproducir el mensaje mediante 
cualquier medio o remitirlo o entregarlo a otra persona, procediendo a su 
borrado de manera inmediata.  Idiria Sociedad Limitada se reserva las acciones 
legales que le correspondan contra todo tercero que acceda de forma  ilegítima 
al contenido de cualquier mensaje externo procedente del mismo. Para 
información y consultas visite nuestra web http://www.idiria.com  Idiria 
Sociedad Limitada - DisclaimerThis message, its content and any file attached 
thereto is for the intended recipient only and is confidential. If you have 
received this e-mail in error or had access to it, you should note that the 
information in it is private and any use thereof is unauthorised. In such an 
event please notify us by e-mail or by telephone (+ 34 690207492). Any 
reproduction of this e-mail by whatsoever means and any transmission or 
dissemination thereof to other persons is prohibited. It should be deleted 
immediately from your system.Idiria Sociedad Limitada reserves the right to 
take legal action against any persons unlawfully gaining access to the content 
of any external message it has emitted.For additional information, please visit 
our website http://www.idiria.com  
-- *José Román Bilbao Castro*Ingeniero Consultor+34 
901009188jrbcast@idiria.com_http://www.idiria.com_ 
--Idiria Sociedad Limitada - Aviso legalEste mensaje, 
su contenido y cualquier fichero transmitido con él está dirigido únicamente a 
su destinatario y es confidencial. Por ello, se informa a quien lo reciba por 
error ó tenga conocimiento del mismo sin ser su destinatario, que la 
información contenida en él es reservada y su uso no a

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[Ryan Cox]

What keys are you talking about?  Are you referring to the script 
generator that Moe linked to?  It's on github as LGPL with no keys of 
any kind: https://github.com/BYUHPC/BYUJobScriptGenerator. It only 
creates a script but doesn't submit it.  We could easily add that 
capability for our own site but we haven't gotten around to it since 
copy-paste then "sbatch thefilename" isn't exactly hard.


Unfortunately a lot of our internal stuff isn't available as open source 
since it's way too tied to internal systems.  Utilization graphs are 
pretty easy.  You can get some information directly from the database 
but we prefer to have more advanced information available.  Some of it 
involves running scontrol to periodically populate tables (we should 
have used the perl API but we hadn't looked at it yet... oh well).


We also have a pretty substantial web services API which is also too 
integrated into our systems to release it.  Among many other things, it 
allows for querying information about all jobs, specific jobs, nodes, 
etc.  Users can submit jobs and admins can modify node state, etc.  The 
key for security is to have a good authentication method and have your 
commands only take well-sanitized input.  In other words, if you want to 
do something like call "scontrol show job" directly, make sure that it 
accepts one parameter from the user, an integer that you have verified 
is only an integer.  Even then, bash may happen :)


Creating an API or doing something like that more directly on the web 
server isn't a trivial task.  You may want to look for existing 
solutions like those mentioned by Lech and others, though I haven't 
looked at those myself.  Having done this ourselves, I know that it can 
take a long time to do it right.


Ryan


On 10/02/2014 08:20 AM, José Román Bilbao Castro wrote:
Re: [slurm-dev] Re: Authentication and invoking slurm commands from 
web app
It seems I was to fast... They don't seem to have open-sourced code. 
In fact, they ask for specific Keys for each implementation so I 
suppose this is a closed project for their users only... :-(


2014-10-02 15:51 GMT+02:00 mailto:je...@schedmd.com>>:


Brigham Young University has developed a number of web interfaces
to SLurm. See:
https://marylou.byu.edu/documentation/slurm/script-generator
https://marylou.byu.edu/utilization/

Their Javascript tool to generate batch job scripts is here:
https://github.com/BYUHPC/BYUJobScriptGenerator





Quoting José Román Bilbao Castro mailto:jrbc...@idiria.com>>:

Thanks Lech,

That is something to start with. The problem is that I plan to add
submission in the future and don't want to start something
that will have
to be changed too much with time. So I would prefer to be able
to firstly
execute any slurm command from my webserver and for any user...

Regards,

Jose

2014-10-02 15:28 GMT+02:00 Lech Nieroda
mailto:lech.nier...@uni-koeln.de>>:

Hello José,

you might be interested in ubmod or its successor open
xdmod. It's a
system that queries SLURM regularly, writes the data into
its own database
and makes it available via webserver. You'd probably have
to implement
proper security measures for user  management.

Regards,
Lech

(sent from mobile)
Am 02.10.2014 14:38 schrieb
=?ISO-8859-1?Q?Jos=E9_Rom=E1n_Bilbao_Castro?= <
jrbc...@idiria.com >:

 Thanks Brian,

So you propose to have something like an intermediate
database that maps
web portal users to system users and make all calls
internally from the
webserver, right?. I just wanted to avoid the intermediate
step for
simplicity, but it seems to be a bad practice.

So, regarding the second step... what is the safest and
more logical
manner of invoking slurm commands from the webserver?. I
mean, at the end I
must pass some credentials for the right user... Or should
I have a tomcat
user that belongs to the sudo group and call invoke
commands as another
user?. I am totally lost and need some thread to start
pulling from it.

Thanks again,

Jose


2014-10-02 13:23 GMT+02:00 Brian B. mailto:for...@gmail.com>>:

Hello Jose,

It is never a good idea to have the public facing
credentials be the same
as the private credentials. That is if your public
facing server is
compromised your internal system is compromised. The
limited cases where
direct internal access are neede

[slurm-dev] Re: Authentication and invoking slurm commands from web app

[José Román Bilbao Castro]

Well, under:

https://marylou.byu.edu/documentation/apps/api/

At the API Keys section. Is is why I supposed this is for internal use only
and therefore it shouldn't be open-sourced.

The script generator, although very useful is not the purpose of my
research at the moment.

So I understand that you uses control to get periodic status on jobs and
infrastructure. That is right but I am still intrigued with mechanisms
needed to perform a job submission. But as you explained it is not
implemented yet. Bad luck for me :).

Anyway, I will keep on investigating on this issue. Not only on the slurm
issue but on the way that people make this kind of tasks involving system
users and commands .

Thanks a lot,

Jose

Enviado desde mi iPad

El 2/10/2014, a las 17:44, Ryan Cox  escribió:

 What keys are you talking about?  Are you referring to the script
generator that Moe linked to?  It's on github as LGPL with no keys of any
kind: https://github.com/BYUHPC/BYUJobScriptGenerator.  It only creates a
script but doesn't submit it.  We could easily add that capability for our
own site but we haven't gotten around to it since copy-paste then "sbatch
thefilename" isn't exactly hard.

Unfortunately a lot of our internal stuff isn't available as open source
since it's way too tied to internal systems.  Utilization graphs are
pretty easy.  You can get some information directly from the database but
we prefer to have more advanced information available.  Some of it
involves running scontrol to periodically populate tables (we should have
used the perl API but we hadn't looked at it yet... oh well).

We also have a pretty substantial web services API which is also too
integrated into our systems to release it.  Among many other things, it
allows for querying information about all jobs, specific jobs, nodes,
etc.  Users can submit jobs and admins can modify node state, etc.  The
key for security is to have a good authentication method and have your
commands only take well-sanitized input.  In other words, if you want to
do something like call "scontrol show job" directly, make sure that it
accepts one parameter from the user, an integer that you have verified is
only an integer.  Even then, bash may happen :)

Creating an API or doing something like that more directly on the web
server isn't a trivial task.  You may want to look for existing solutions
like those mentioned by Lech and others, though I haven't looked at those
myself.  Having done this ourselves, I know that it can take a long time
to do it right.

Ryan


On 10/02/2014 08:20 AM, José Román Bilbao Castro wrote:

Re: [slurm-dev] Re: Authentication and invoking slurm commands from web app
It seems I was to fast... They don't seem to have open-sourced code. In
fact, they ask for specific Keys for each implementation so I suppose this
is a closed project for their users only... :-(

2014-10-02 15:51 GMT+02:00 :

>
> Brigham Young University has developed a number of web interfaces to
> SLurm. See:
> https://marylou.byu.edu/documentation/slurm/script-generator
> https://marylou.byu.edu/utilization/
>
> Their Javascript tool to generate batch job scripts is here:
> https://github.com/BYUHPC/BYUJobScriptGenerator
>
>
>
>
>
> Quoting José Román Bilbao Castro :
>
>Thanks Lech,
>>
>> That is something to start with. The problem is that I plan to add
>> submission in the future and don't want to start something that will have
>> to be changed too much with time. So I would prefer to be able to firstly
>> execute any slurm command from my webserver and for any user...
>>
>> Regards,
>>
>> Jose
>>
>> 2014-10-02 15:28 GMT+02:00 Lech Nieroda :
>>
>>Hello José,
>>>
>>> you might be interested in ubmod or its successor open xdmod. It's a
>>> system that queries SLURM regularly, writes the data into its own
>>> database
>>> and makes it available via webserver. You'd probably have to implement
>>> proper security measures for user  management.
>>>
>>> Regards,
>>> Lech
>>>
>>> (sent from mobile)
>>> Am 02.10.2014 14:38 schrieb =?ISO-8859-1?Q?José_Román_Bilbao_Castro?= <
>>> jrbc...@idiria.com>:
>>>
>>> Â Thanks Brian,
>>>
>>> So you propose to have something like an intermediate database that maps
>>> web portal users to system users and make all calls internally from the
>>> webserver, right?. I just wanted to avoid the intermediate step for
>>> simplicity, but it seems to be a bad practice.
>>>
>>> So, regarding the second step... what is the safest and more logical
>>> manner of invoking slurm commands from the webserver?. I mean, at the
>>> end I
>>> must pass some credentials for the right user... Or should I have a
>>> tomcat
>>> user that belongs to the sudo group and call invoke commands as another
>>> user?. I am totally lost and need some thread to start pulling from it.
>>>
>>> Thanks again,
>>>
>>> Jose
>>>
>>>
>>> 2014-10-02 13:23 GMT+02:00 Brian B. :
>>>
>>>Hello Jose,

 It is never a good idea to have the public facing credential

[slurm-dev] Re: Compiling SLURM on Solaris 11.2

[Jan van der Lugt]

Hi,

Yes, that seems to be the problem. Can someone point me in the right
direction to change the compile scripts? I'm willing to put some effort
into this, but I don't really know where to start.

- Jan

On Tue, Sep 30, 2014 at 10:59 PM, Blomqvist Janne 
wrote:

>  Hi,
>
> xcgroup.c (src/slurmd/common/xcgroup.c) handles the Linux-specific cgroups
> (https://en.wikipedia.org/wiki/Cgroups) stuff. Judging from your error
> message it appears that the solaris mount() function is different from the
> Linux one. But it doesn't really matter anyway, since cgroups don't exist
> on Solaris so mounting a cgroup filesystem is pointless there. I guess the
> real bug is that there is some missing autotools-foo to not compile/link
> xcgroups.c on non-Linux platforms (perhaps substituting some dummy no-op
> xcgroups.c implementation or such?).
>
> --
> Janne Blomqvist
>   --
> *From:* Jan van der Lugt [janl...@gmail.com]
> *Sent:* Wednesday, October 01, 2014 1:35
> *To:* slurm-dev
> *Subject:* [slurm-dev] Re: Compiling SLURM on Solaris 11.2
>
>   Hi all,
>
>  Does anyone have an idea about this problem? Both version 2.5.7 (which
> we use) and version 14.03.8 give me the following error:
>  xcgroup.c: In function ‘xcgroup_ns_mount’:
> xcgroup.c:225:15: error: ‘MS_NOEXEC’ undeclared (first use in this
> function)
>  MS_NOSUID|MS_NOEXEC|MS_NODEV, options))
>^
> xcgroup.c:225:15: note: each undeclared identifier is reported only once
> for each function it appears in
> xcgroup.c:225:25: error: ‘MS_NODEV’ undeclared (first use in this function)
>  MS_NOSUID|MS_NOEXEC|MS_NODEV, options))
>  ^
> xcgroup.c:225:5: warning: passing argument 3 of ‘mount’ makes integer from
> pointer without a cast [enabled by default]
>  MS_NOSUID|MS_NOEXEC|MS_NODEV, options))
>  ^
> In file included from xcgroup.c:57:0:
> /usr/include/sys/mount.h:56:5: note: expected ‘int’ but argument is of
> type ‘char *’
>  int mount(const char *, const char *, int, ...);
>  ^
> xcgroup.c: In function ‘xcgroup_lock’:
> xcgroup.c:417:21: error: ‘LOCK_EX’ undeclared (first use in this function)
>   if (flock(cg->fd,  LOCK_EX) < 0) {
>  ^
> xcgroup.c: In function ‘xcgroup_unlock’:
> xcgroup.c:432:21: error: ‘LOCK_UN’ undeclared (first use in this function)
>   if (flock(cg->fd,  LOCK_UN) < 0) {
>
>  Thanks in advance for any help.
>
>  - Jan
>
> On Tue, Sep 23, 2014 at 2:52 PM, Jan van der Lugt 
> wrote:
>
>> Hi all,
>>
>>  I'm trying to compile SLURM on Solaris 11.2, but I'm having some
>> trouble disabling certain Linux-specific features, such as everything
>> related to cgroups. What is the best way to achieve this?
>>
>>  Thanks in advance for your help.
>>
>>  - Jan
>>
>
>

[slurm-dev] Re: Build error on CentOS 5.10

[Michael Jennings]

On Wed, Oct 1, 2014 at 11:18 PM, Dennis Zheleznyak 
wrote:

>  + /var/tmp/slurm-14.03.8-1/usr/bin/sjobexitmod --roff
> /var/tmp/rpm-tmp.79119: /var/tmp/slurm-14.03.8-1/usr/bin/sjobexitmod:
> /usr/bin/perl: bad interpreter: Permission denied
> error: Bad exit status from /var/tmp/rpm-tmp.79119 (%install)
>

Is your /var/tmp mounted "noexec" by any chance?

Michael

-- 
Michael Jennings 
Senior HPC Systems Engineer
High-Performance Computing Services
Lawrence Berkeley National Laboratory
Bldg 50B-3209EW: 510-495-2687
MS 050B-3209  F: 510-486-8615

[slurm-dev] Building Slurm 14.03.x with FreeIPMI 1.4.5?

[Andy Riebs]

Trying to build Slurm 14.03.6 with FreeIPMI, using

./configure --with-freeipmi=[PATH-to-FREEIPMI]

The configure log showed that the FreeIPMI test program compiles but 
fails to link, with a bunch of missing symbols. After some 
head-scratching, I discovered that I had to add "-lfreeipmi -lgcrypt 
-lm" in order to get the test program to compile.


I worked around this by patching 2 spots in the configure script. I was 
actually going to go after after auxdir/x_ac_freeipmi.m4, but autogen.sh 
complained that it couldn't find AM_PATH_GLIB_2_0 or AM_PATH_GTK_2_0, 
and I ran out of time.


It would help the world's FreeIPMI users if someone would fix this 
appropriately (which looks like it may need to have a check for which 
version of FreeIPMI is being used).


Andy

--
Andy Riebs
Hewlett-Packard Company
High Performance Computing
+1 404 648 9024
My opinions are not necessarily those of HP

[slurm-dev] Re: Building Slurm 14.03.x with FreeIPMI 1.4.5?

[Nathan Harper]

We're using 14.03.7 with FreeIPMI 1.4.5 and didn't have to do anything
unusual to get it built.

FreeIPMI was built from source into an RPM, then SLURM itself built into an
RPM.


-- 
*Nathan Harper* // IT Systems Architect

*e: * nathan.har...@cfms.org.uk // *t: * 0117 906 1104 // *m: * 07875 510891 //
*w: * www.cfms.org.uk  // [image: Linkedin grey
icon scaled] 
CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // Emersons
Green // Bristol // BS16 7FR

[image: 4.2 CFMS_Artwork_RGB] 

--
CFMS Services Ltd is registered in England and Wales No 05742022 - a
subsidiary of CFMS Ltd
CFMS Services Ltd registered office // Victoria House // 51 Victoria Street
// Bristol // BS1 6AD

On 2 October 2014 19:23, Andy Riebs  wrote:

>
> Trying to build Slurm 14.03.6 with FreeIPMI, using
>
> ./configure --with-freeipmi=[PATH-to-FREEIPMI]
>
> The configure log showed that the FreeIPMI test program compiles but fails
> to link, with a bunch of missing symbols. After some head-scratching, I
> discovered that I had to add "-lfreeipmi -lgcrypt -lm" in order to get the
> test program to compile.
>
> I worked around this by patching 2 spots in the configure script. I was
> actually going to go after after auxdir/x_ac_freeipmi.m4, but autogen.sh
> complained that it couldn't find AM_PATH_GLIB_2_0 or AM_PATH_GTK_2_0, and I
> ran out of time.
>
> It would help the world's FreeIPMI users if someone would fix this
> appropriately (which looks like it may need to have a check for which
> version of FreeIPMI is being used).
>
> Andy
>
> --
> Andy Riebs
> Hewlett-Packard Company
> High Performance Computing
> +1 404 648 9024
> My opinions are not necessarily those of HP
>

[slurm-dev] Question about sattach usage

[Andrew Gontarek]

Hello,

When the sattach command is used, will the old IO streams of the original srun 
command still be available? Or will the IO streams be replaced?


Thanks.

-Andrew

--
Andrew Gontarek
PE Debuggers
Ph: 651-605-9176
Cell: 651-895-0295

[slurm-dev] Question about sattach usage

[Andrew Gontarek]

Hello,

When the sattach command is used, will the old IO streams of the original srun 
command still be available? Or will the IO streams be replaced?


Thanks.

-Andrew

--
Andrew Gontarek
PE Debuggers
Ph: 651-605-9176

[slurm-dev] Re: Question about sattach usage

[jette]

Quoting Andrew Gontarek :


Hello,

When the sattach command is used, will the old IO streams of the  
original srun command still be available?


Yes



Or will the IO streams be replaced?

Thanks.

-Andrew

--
Andrew Gontarek
PE Debuggers
Ph: 651-605-9176
Cell: 651-895-0295



--
Morris "Moe" Jette
CTO, SchedMD LLC

[slurm-dev] Re: Build error on CentOS 5.10

[Christopher Samuel]

On 03/10/14 03:05, Michael Jennings wrote:

> Is your /var/tmp mounted "noexec" by any chance?

I wondered that but I don't think that'll give the errors that Dennis is
seeing, he's seeing -EACCES for /usr/bin/perl, not for something in
/var/tmp. :-(

-- 
 Christopher SamuelSenior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/  http://twitter.com/vlsci

[slurm-dev] Re: Build error on CentOS 5.10

[Michael Jennings]

On Thu, Oct 2, 2014 at 4:37 PM, Christopher Samuel
 wrote:

>> Is your /var/tmp mounted "noexec" by any chance?
>
> I wondered that but I don't think that'll give the errors that Dennis is
> seeing, he's seeing -EACCES for /usr/bin/perl, not for something in
> /var/tmp. :-(

It's trying to run a script in /var/tmp that has #!/usr/bin/perl in
it.  So I suspect /usr is mounted noexec.

Michael

-- 
Michael Jennings 
Senior HPC Systems Engineer
High-Performance Computing Services
Lawrence Berkeley National Laboratory
Bldg 50B-3209EW: 510-495-2687
MS 050B-3209  F: 510-486-8615

[slurm-dev] Re: Build error on CentOS 5.10

[Christopher Samuel]

On 03/10/14 09:53, Michael Jennings wrote:

> It's trying to run a script in /var/tmp that has #!/usr/bin/perl in
> it.  So I suspect /usr is mounted noexec.

Hmm, that would make life, umm, interesting. :-)

Dennis, what does "cat /proc/mounts" on this machine say?

cheers!
Chris
-- 
 Christopher SamuelSenior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/  http://twitter.com/vlsci

[slurm-dev] Unsubscribe

[Fabricio Silva Kyt]

Unsubscribe

[slurm-dev] Re: Build error on CentOS 5.10

[Michael Jennings]

On Thu, Oct 2, 2014 at 5:13 PM, Christopher Samuel
 wrote:

>> It's trying to run a script in /var/tmp that has #!/usr/bin/perl in
>> it.  So I suspect /usr is mounted noexec.
>
> Hmm, that would make life, umm, interesting. :-)

Yes, indeed!  :-)

That's the only way I've been able to reproduce that error, though
(apart from what you already covered) -- either noexec on the
filesystem the interpreter is on, or execute permissions on a parent
directory being missing -- and both would be fairly odd for
/usr/bin/perl!  :-)

Michael

-- 
Michael Jennings 
Senior HPC Systems Engineer
High-Performance Computing Services
Lawrence Berkeley National Laboratory
Bldg 50B-3209EW: 510-495-2687
MS 050B-3209  F: 510-486-8615