Re: [easybuild] Easybuild as containers?

2016-12-18 Thread Christopher Samuel 
On 17/12/16 02:09, Alan O'Cais wrote:

> I think Singularity is a more interesting prospect here (come to the
> EasyBuild User Meeting in February, one of the developers will be
> there). I don't know any large HPC sites that allow docker containers
> (mainly due to security problems).

This is why Shifter from NERSC exists, to convert/disarm Docker
containers to make them safe to use on HPC systems:

http://www.nersc.gov/research-and-development/user-defined-images/

https://github.com/NERSC/shifter/

We've got both deployed here.

All the best,
Chris
-- 
 Christopher SamuelSenior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/  http://twitter.com/vlsci

Re: [easybuild] Easybuild as containers?

2016-12-17 Thread Kenneth Hoste 



On 16/12/16 16:47, Siddiqui, Shahzeb wrote:


I will try to attend, is this conference hosted by Easybuild on Google 
Hangout.




The EasyBuild User Meeting is not a regular EasyBuild conf call, it's an 
actual 3-day meeting, see https://indico-jsc.fz-juelich.de/event/30/ .
We do have some plans to try and record the presentations, but nothing 
100% sure yet.


It would be interesting to see how docker works in RHEL7. I recall 
that Redhat is now providing their own docker images, I would suspect 
they would be taking security into consideration.




Docker is not a good fit in an HPC context, for several (mostly 
security-related) reasons.

RedHat may be providing Docker images, but certainly not in an HPC context.
See 
http://singularity.lbl.gov/faq#how-does-singularity-relatediffer-from-docker 
for some information on why Singularity is a more viable solution in an 
HPC context.


We have some plans for working together with the Singularity community 
to see how EasyBuild can be used to populate Singularity 
containers/images, which is probably what you are after.




regards,

Kenneth

*From:*easybuild-requ...@lists.ugent.be 
[mailto:easybuild-requ...@lists.ugent.be] *On Behalf Of *Alan O'Cais

*Sent:* Friday, December 16, 2016 10:10 AM
*To:* easybuild@lists.ugent.be
*Subject:* Re: [easybuild] Easybuild as containers?

Dear Shahzeb ,

On 16 December 2016 at 15:33, Siddiqui, Shahzeb 
mailto:shahzeb.siddi...@pfizer.com>> wrote:


I meant to say is there any benefit of using docker container vs rpm 
for deploying software packages on a HPC system. If so, I don’t see 
any support for creating docker images from eb directly.


I think Singularity is a more interesting prospect here (come to the 
EasyBuild User Meeting in February, one of the developers will be 
there). I don't know any large HPC sites that allow docker containers 
(mainly due to security problems).


Alan


--

Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de <mailto:a.oc...@fz-juelich.de>
WWW: http://www.fz-juelich.de/ias/jsc/EN 
<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.fz-2Djuelich.de_ias_jsc_EN&d=DgMFaQ&c=UE1eNsedaKncO0Yl_u8bfw&r=RMJdCm7m5fiPWhajwKUnEW5yn4eK2YdUWW-MLVShghg&m=lFMhPQiNfrpQjcyb2-WBHLT_mr1LR4AGfoILbEsDv_M&s=eoCJDWs53lpUewTDQLDgJa15sVrrVpuMrjmIFz-7e2E&e=>






Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

RE: [easybuild] Easybuild as containers?

2016-12-16 Thread Siddiqui, Shahzeb 
I will try to attend, is this conference hosted by Easybuild on Google Hangout.

It would be interesting to see how docker works in RHEL7. I recall that Redhat 
is now providing their own docker images, I would suspect they would be taking 
security into consideration.



From: easybuild-requ...@lists.ugent.be 
[mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Alan O'Cais
Sent: Friday, December 16, 2016 10:10 AM
To: easybuild@lists.ugent.be
Subject: Re: [easybuild] Easybuild as containers?

Dear Shahzeb ,

On 16 December 2016 at 15:33, Siddiqui, Shahzeb 
mailto:shahzeb.siddi...@pfizer.com>> wrote:

I meant to say is there any benefit of using docker container vs rpm for 
deploying software packages on a HPC system. If so, I don’t see any support for 
creating docker images from eb directly.

I think Singularity is a more interesting prospect here (come to the EasyBuild 
User Meeting in February, one of the developers will be there). I don't know 
any large HPC sites that allow docker containers (mainly due to security 
problems).

Alan


--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de<mailto:a.oc...@fz-juelich.de>
WWW:
http://www.fz-juelich.de/ias/jsc/EN<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.fz-2Djuelich.de_ias_jsc_EN&d=DgMFaQ&c=UE1eNsedaKncO0Yl_u8bfw&r=RMJdCm7m5fiPWhajwKUnEW5yn4eK2YdUWW-MLVShghg&m=lFMhPQiNfrpQjcyb2-WBHLT_mr1LR4AGfoILbEsDv_M&s=eoCJDWs53lpUewTDQLDgJa15sVrrVpuMrjmIFz-7e2E&e=>




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Easybuild as containers?

2016-12-16 Thread Alan O'Cais 
Dear Shahzeb ,

On 16 December 2016 at 15:33, Siddiqui, Shahzeb 
mailto:shahzeb.siddi...@pfizer.com>> wrote:

I meant to say is there any benefit of using docker container vs rpm for 
deploying software packages on a HPC system. If so, I don’t see any support for 
creating docker images from eb directly.

I think Singularity is a more interesting prospect here (come to the EasyBuild 
User Meeting in February, one of the developers will be there). I don't know 
any large HPC sites that allow docker containers (mainly due to security 
problems).

Alan


--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

RE: [easybuild] Easybuild as containers?

2016-12-16 Thread Siddiqui, Shahzeb 
This is good example, but I think I need to clarify what I am looking for. This 
example is good for deploying Easybuild as container.

I meant to say is there any benefit of using docker container vs rpm for 
deploying software packages on a HPC system. If so, I don’t see any support for 
creating docker images from eb directly.

It seems like a standard process for creating docker container. One will need 
to contain the files specified by –install-path of that package along with the 
module file generated from easybuild. Right now, easybuild does that 
automatically when enabling --package

I think docker would be particularly useful when you finish a easybuild 
installation that requires starting a daemon, web-server that is part of a 
software.

One example is Material Studio, this software is installed on a server that 
runs as a service on a particular port that is used by Material Studio client 
that runs on Windows to connect to the server via the MStudio service.



From: easybuild-requ...@lists.ugent.be 
[mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Robert Schmidt
Sent: Thursday, December 15, 2016 4:28 PM
To: easybuild@lists.ugent.be
Subject: Re: [easybuild] Easybuild as containers?

I'm just heading out the door, but look at my docker container here:

https://hub.docker.com/r/rjeschmi/easybuild-centos7/<https://urldefense.proofpoint.com/v2/url?u=https-3A__hub.docker.com_r_rjeschmi_easybuild-2Dcentos7_&d=DgMFaQ&c=UE1eNsedaKncO0Yl_u8bfw&r=RMJdCm7m5fiPWhajwKUnEW5yn4eK2YdUWW-MLVShghg&m=_ddjGMozf6REHpMrninRnn222p9BN7VOqzcnGyiZqPs&s=lGH5H9oUlpoDgOz0KrVTCSlQlkBNEQK6tRV-wnyGlUk&e=>
 (especially the 3.0.0 tag)

I didn't have too much trouble importing that into singularity as well.

I'll try to provide some more information later.

On Thu, Dec 15, 2016 at 4:21 PM Siddiqui, Shahzeb 
mailto:shahzeb.siddi...@pfizer.com>> wrote:
Hello,

I would like to find out if there is a way to make Easybuild installs into 
containers. I am looking at ways to self-contain all the dependencies in 
container objects to properly deploy software packages and its dependencies. We 
are trying to build a software collection that contains all of our software 
builds, either through rpms or docker containers that can be used by our 
HPC-users to customize their installation on their VM.

Does anyone have any experience with this?

Regards,

Shahzeb Siddiqui
HPC Linux Engineer
B2220-447.2
Groton, CT

Re: [easybuild] Easybuild as containers?

2016-12-15 Thread Robert Schmidt 
I'm just heading out the door, but look at my docker container here:

https://hub.docker.com/r/rjeschmi/easybuild-centos7/ (especially the 3.0.0
tag)

I didn't have too much trouble importing that into singularity as well.

I'll try to provide some more information later.

On Thu, Dec 15, 2016 at 4:21 PM Siddiqui, Shahzeb <
shahzeb.siddi...@pfizer.com> wrote:

> Hello,
>
>
>
> I would like to find out if there is a way to make Easybuild installs into
> containers. I am looking at ways to self-contain all the dependencies in
> container objects to properly deploy software packages and its
> dependencies. We are trying to build a software collection that contains
> all of our software builds, either through rpms or docker containers that
> can be used by our HPC-users to customize their installation on their VM.
>
>
>
> Does anyone have any experience with this?
>
>
>
> Regards,
>
>
>
> Shahzeb Siddiqui
>
> HPC Linux Engineer
>
> B2220-447.2
>
> Groton, CT
>
>
>