Hi Di,

Your first link doesn’t work it is referencing your gmail. Please provide 
another link.

Right now eb shares easyconfig files that can be used by any HPC sites to build 
the packages on their own environment. If we decide to go in a containerized 
software stack, we are looking at sharing binaries instead of easyconfig files. 
A singularity container is just a binary that can be executed as any normal 
program.

If EB decides to with containerization then I foresee all the builds to happen 
inside a container that can produce a new container with only the application 
that was built along with dependencies. There would be a container naming 
format similar to RPM naming convention to allow for multiple software & 
toolchain version to coexist in a repository. This would be extremely useful 
for deploying apps like R, Python, perl, ruby in a container so that you can 
make your runscript like the following

%runscript
         echo "Arguments received: $*"
         exec /usr/bin/python "$@"

Currently EB installs packages in non-standard paths, we would need to 
translate module file environment in runscript to allow the user to access the 
binaries inside the container in their PATH environment. If there was only a 
way to redirect container output into your host shell environment, we might be 
able to do module loads inside the container runscript and redirect output to 
host shell.

From: easybuild-requ...@lists.ugent.be 
[mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Di Pe
Sent: Monday, May 8, 2017 1:43 PM
To: easybuild@lists.ugent.be
Subject: Re: [easybuild] eb in singularity

Interesting discussion about Singularity, there was an older thread here

https://mail.google.com/mail/u/0/#search/singularity/159045c3913ad572<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.google.com_mail_u_0_-23search_singularity_159045c3913ad572&d=DwMFaQ&c=UE1eNsedaKncO0Yl_u8bfw&r=RMJdCm7m5fiPWhajwKUnEW5yn4eK2YdUWW-MLVShghg&m=U1Ye_dBtCfKKvzdeYTPD9IUfYLhVCCQYcTW04W3jMFI&s=5Y8Aj2dkooBU_izF82b21XY7ckGP_7CRyAjXFR29TUA&e=>

and it would be great to learn what EB leadership thinks about integration of 
containers in EB?

To look a little bit into our requirements we would like to make it easy to 
share the software stacks we build using EB with

 1) other researchers outside our organization who are also running things on 
HPC systems
 2) other software platforms in the same organization (outside HPC clusters)

In both cases we want to support a certain version of a containerized software 
stack for a certain period of time. Take for example a data science stack. We 
would bundle a certain version of R with a certain version of Python and some 
other ancillary tools. We would build a container that has one version of 
Python2 and one version of R and another container that has Python3 with R.
I would not need or even want any Lmod environment inside the container because 
we may need to tie R and python together. For example R argparse is linked to a 
specific version of Python and Python rpy2 is linked to a specific version or 
R. This cannot be handled well with lmod.

Another issue that plays into this is size of the container. We want to keep 
the container relatively small to facilitate sharing but the easybuild 
environment is actually pretty large with sources and everything. if there was 
a good way to remove all the unnecessary things including compiler after the 
container is created that would be great.

Yet another issue is that non-HPC people are not used to lmod. So if I have 
hadoop or devops folks they need to know a little bit of lmod to be able to 
troubleshoot the software stack and because that is not the case (at least 
here) they are building stuff by hand or are using crappy Ubuntu debs even 
through the HPC person one floor down has already done the (better) job using 
EB. In that sense it would be great if we could use EB to create a container 
stack that support only one version of each software and wherethe software 
inside the container simply installs to /usr/local to they can use it as a base 
and then fuzz with it further if they need to.

Another interesting discussion was Singularity vs Docker. After reading this 
https://www.nextplatform.com/2017/04/10/singularity-containers-hpc-reproducibility-mobility/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.nextplatform.com_2017_04_10_singularity-2Dcontainers-2Dhpc-2Dreproducibility-2Dmobility_&d=DwMFaQ&c=UE1eNsedaKncO0Yl_u8bfw&r=RMJdCm7m5fiPWhajwKUnEW5yn4eK2YdUWW-MLVShghg&m=U1Ye_dBtCfKKvzdeYTPD9IUfYLhVCCQYcTW04W3jMFI&s=vQBRkyNKZ0AuhN1k27RTgwrznsvOcwJfGydKEpPx5y4&e=>
 and trying out singularity I think the answer is that we want to use both. 
Docker plain will probably never be usable on HPC systems (See the story about 
rejected pull requests) so you have to take a detour and use Shifter (Shifter 
needs to be integrated by a HPC sysadmin requiring root access and a test plan. 
It is optimized towards and tested with Slurm.)   Singularity works out of the 
box on HPC and standalone systems. The benefit of Shifter seems to be that it 
can use  Docker images natively while singularity requires you to either build 
singularity images or import docker images. In my ad-hoc testing I was having 
much more problems with building singularity images (e.g FS corruption ) than 
importing docker images with singularity. The latter worked really well and 
fast.

So perhaps it makes sense to focus on integrating Docker builds into EB as one 
could deliver a more comprehensive solution to bigdata, devops and HPC people.

Interested to hear some thoughts

DP







On Thu, May 4, 2017 at 9:53 AM, Siddiqui, Shahzeb 
<shahzeb.siddi...@pfizer.com<mailto:shahzeb.siddi...@pfizer.com>> wrote:
Hello folks,

I would like to find out how Easybuild and Singularity are going to work 
together. I am trying to design an eb environment in Singularity as a container 
solution to host all of the eb apps in a prod environment. Is there anyone in 
the HPC community that is working on this?

Currently, I have setup a container environment that can build packages in 
container and also install RPMs from easybuild via Artifactory that is done 
through the bootstrap process.

One of things that puzzles me is how to setup an isolated container environment 
that runs /etc/profile for container. I’ve noticed I need to do this inorder to 
get module command to work in container. Currently, I have to do this manually 
after shelling in.


-bash-4.2$ singularity shell /nfs/grid/software/testing.img
Singularity: Invoking an interactive shell within container...

Singularity.testing.img> module --version
sh: module: command not found
Singularity.testing.img> env | grep MODULEPATH
MODULEPATH_ROOT=/usr/share/modulefiles
MODULEPATH=/modulefiles/Core/:/nfs/grid/software/RHEL7-BUILD/easybuild/modules/all/:/nfs/grid/software/RHEL7/non-easybuild/modules/all/:/nfs/grid/software/RHEL7/easybuild/modules/all/Core:/nfs/grid/software/moduledomains:/etc/modulefiles:/usr/share/modulefiles:/usr/share/modulefiles/Linux:/usr/share/modulefiles/Core:/usr/share/lmod/lmod/modulefiles/Core
Singularity.testing.img> unset MODULEPATH
Singularity.testing.img> . /etc/profile
Singularity.testing.img> . /nfs/grid/software/
module-setup.sh  RHEL7/           RHEL7-BUILD/
Singularity.testing.img> . /nfs/grid/software/module-setup.sh
Singularity.testing.img> ml av

----------------------------------------------------------- 
/usr/share/lmod/lmod/modulefiles/Core 
------------------------------------------------------------
   lmod/6.5.1    settarg/6.5.1

---------------------------------------------------- 
/nfs/grid/software/RHEL7-BUILD/easybuild/modules/all 
----------------------------------------------------
   EasyBuild/3.1.2

---------------------------------------------------- 
/nfs/grid/software/RHEL7/easybuild/modules/all/Core 
-----------------------------------------------------
   Advisor/2017_update1        IGV/2.3.80-Java-1.8.0_92          Java/1.8.0_92  
       gaussian/16-AVX         iompi/2017.01      tbb/2017.2.132
   GCC/5.4.0-2.27              Inspector/2017_update1            
VTune/2017_update1    gaussian/16-SSE2 (D)    ipp/2017.1.132
   GCC/6.2.0-2.27       (D)    IntelClusterChecker/2017.1.016    
daal/2017.1.132       intel/2017.01           itac/2017.1.024

  Where:
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching 
any of the "keys".

Singularity.testing.img> ml EasyBuild
Singularity.testing.img> eb --version
This is EasyBuild 3.1.2 (framework: 3.1.2, easyblocks: 3.1.2) on host 
amrndhl1157.pfizer.com<http://amrndhl1157.pfizer.com>.
Singularity.testing.img> eb --show-config
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: 
configuration file)
#
buildpath      (D) = /home/siddis14/.local/easybuild/build
installpath    (D) = /home/siddis14/.local/easybuild
repositorypath (D) = /home/siddis14/.local/easybuild/ebfiles_repo
robot-paths    (D) = 
/nfs/grid/software/RHEL7-BUILD/easybuild/software/EasyBuild/3.1.2/lib/python2.7/site-packages/easybuild_easyconfigs-3.1.2-py2.7.egg/easybuild/easyconfigs
sourcepath     (D) = /home/siddis14/.local/easybuild/sources
Singularity.testing.img> eb zlib-1.2.8.eb
== temporary log file in case of crash /tmp/eb-BfnLvm/easybuild-ybJc4_.log
== zlib/1.2.8 is already installed (module found), skipping
== No easyconfigs left to be built.
== Build succeeded for 0 out of 0
== Temporary log file(s) /tmp/eb-BfnLvm/easybuild-ybJc4_.log* have been removed.
== Temporary directory /tmp/eb-BfnLvm has been removed.
Singularity.testing.img> eb zlib-1.2.8.eb --rebuild
== temporary log file in case of crash /tmp/eb-IMN3vl/easybuild-lWCjI1.log
== processing EasyBuild easyconfig 
/nfs/grid/software/RHEL7-BUILD/easybuild/software/EasyBuild/3.1.2/lib/python2.7/site-packages/easybuild_easyconfigs-3.1.2-py2.7.egg/easybuild/easyconfigs/z/zlib/zlib-1.2.8.eb
== building and installing 
zlib/1.2.8.<https://urldefense.proofpoint.com/v2/url?u=http-3A__1.2.8.&d=DwMFaQ&c=UE1eNsedaKncO0Yl_u8bfw&r=RMJdCm7m5fiPWhajwKUnEW5yn4eK2YdUWW-MLVShghg&m=U1Ye_dBtCfKKvzdeYTPD9IUfYLhVCCQYcTW04W3jMFI&s=2DbV7b8CLu85Uado7mlKmdBIcJvdmaZr0IIo3nO0XtU&e=>..
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) 
/home/siddis14/.local/easybuild/software/zlib/1.2.8/easybuild/easybuild-zlib-1.2.8-20170504.163248.log
== Build succeeded for 1 out of 1
== Temporary log file(s) /tmp/eb-IMN3vl/easybuild-lWCjI1.log* have been removed.
== Temporary directory /tmp/eb-IMN3vl has been removed.

Singularity.testing.img> module use $HOME/.local/easybuild/modules/all

Due to MODULEPATH changes the following have been reloaded:
  1) EasyBuild/3.1.2

Singularity.testing.img> module av

-------------------------------------------------------- 
/home/siddis14/.local/easybuild/modules/all 
---------------------------------------------------------
   EasyBuild/3.1.2 (L,D)    M4/1.4.17 (D)    zlib/1.2.8 (D)

----------------------------------------------------------- 
/usr/share/lmod/lmod/modulefiles/Core 
------------------------------------------------------------
   lmod/6.5.1    settarg/6.5.1

---------------------------------------------------- 
/nfs/grid/software/RHEL7-BUILD/easybuild/modules/all 
----------------------------------------------------
   EasyBuild/3.1.2

---------------------------------------------------- 
/nfs/grid/software/RHEL7/easybuild/modules/all/Core 
-----------------------------------------------------
   Advisor/2017_update1        IGV/2.3.80-Java-1.8.0_92          Java/1.8.0_92  
       gaussian/16-AVX         iompi/2017.01      tbb/2017.2.132
   GCC/5.4.0-2.27              Inspector/2017_update1            
VTune/2017_update1    gaussian/16-SSE2 (D)    ipp/2017.1.132
   GCC/6.2.0-2.27       (D)    IntelClusterChecker/2017.1.016    
daal/2017.1.132       intel/2017.01           itac/2017.1.024

  Where:
   L:  Module is loaded
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching 
any of the "keys".



Shahzeb Siddiqui
HPC Linux Engineer
B2220-447.2
Groton, CT


Reply via email to