Re: [easybuild] Can't get local install to work

[Niek de Klein]

Dear Kenneth,

I had easyconfigs that I used on an old cluster mixed into the new
easyconfig files which I was installing on a new cluster, which is why the
files were still from EasyBuild 3.x. I have removed all the old easyconfigs
that I had included and am now remaking them using the new easyconfig info.
For the easyconfig that I had the previous problem this fixes it.

Thanks a lot for your help,
Niek

On Tue, Nov 19, 2019 at 4:40 PM Kenneth Hoste 
wrote:

> Dear Niek,
>
> On 19/11/2019 22:28, Niek de Klein wrote:
> > Dear Kenneth,
> >
> > Attached the log.
>
> Thank you, that's helpful to figure this out.
>
> So, here's what's going on...
>
> When resolving dependencies for M4-1.4.17-foss-2018b.eb, EasyBuild goes
> looking for the easyconfigs for the subtoolchains of foss/2018b,
> starting with gompi/2018b.
>
> It finds an easyconfig for gompi/2018b at
>
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/g/gompi/gompi-2018b.eb
>
> This easyconfig file specifies that gompi/2018b consists of GCC 4.9.3,
> binutils 2.25 and OpenMPI 1.8.8, which is very incorrect...
>
> This makes things go horribly wrong when it tries to find the easyconfig
> for OpenBLAS, eventually.
>
>
> So, question is why that gompi-2018b.eb easyconfig file isn't what it's
> supposed to be (see
>
> https://github.com/easybuilders/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/g/gompi/gompi-2018b.eb
> ).
>
> The warnings you're hitting also show that you're using a copy of the
> easyconfig files from an EasyBuild 3.x version.
>
> The easyconfigs included in EasyBuild 4.0.x have been cleaned up to
> avoid those warnings...
>
>
>
> regards,
>
> Kenneth
> >
> > On Tue, Nov 19, 2019 at 4:00 PM Kenneth Hoste  > <mailto:kenneth.ho...@ugent.be>> wrote:
> >
> > Dear Niek,
> >
> > On 19/11/2019 20:42, Niek de Klein wrote:
> >  > I get
> >  >
> >  > $ eb M4-1.4.17-foss-2018b.eb -D
> >
> >
> > Please try running this with debugging enabled, and then zip up the
> log
> > file from /tmp (see path printed at the line like shown below):
> >
> > eb --debug M4-1.4.17-foss-2018b.eb -D
> >
> >
> >  > == temporary log file in case of crash
> > /tmp/eb-ZCIA8v/easybuild-JS80fY.log
> >
> >  >
> >  > 
> >  >
> >  > ERROR: Failed to process easyconfig
> >  >
> >
>  
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb:
> >
> >  > Failed to determine minimal toolchain for dep OpenBLAS
> > 0.3.1-GCC-7.3.0-2.30
> >  >
> >  > and a bunch of warnings (full log below).
> >  >
> >  > eb -S OpenBLAS-0.3.1-GCC-7.3.0-2.30
> >  >
> >  > gives
> >  >
> >  >
> >
>  
> CFGS1=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenBLAS
> >  >   * $CFGS1/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb
> >  >
> >  > and
> >  >
> >  > eb OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb -D
> >  > == temporary log file in case of crash
> > /tmp/eb-VAv5nH/easybuild-9UafJT.log
> >  > Dry run: printing build status of easyconfigs and dependencies
> >  >
> >
>  
> CFGS=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs
> >  >   * [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18)
> >  >   * [x] $CFGS/z/zlib/zlib-1.2.11.eb (module: zlib/1.2.11)
> >  >   * [x] $CFGS/h/help2man/help2man-1.47.4.eb (module:
> help2man/1.47.4)
> >  >   * [x] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
> >  >   * [x] $CFGS/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4)
> >  >   * [x] $CFGS/f/flex/flex-2.6.4.eb (module: flex/2.6.4)
> >  >   * [x] $CFGS/b/binutils/binutils-2.30.eb (module: binutils/2.30)
> >  >   * [x] $CFGS/g/GCCcore/GCCcore-7.3.0.eb (module: GCCcore/7.3.0)
> >  >   * [x] $CFGS/z/zlib/zlib-1.2.11-GCCcore-7.3.0.eb (module:
> >  > zlib/1.2.11-GCCcore-7.3.0)
> >  >   * [x] $CFGS/h/help2man/help2man-1.47.4-GCCcore-7.3.0.eb (module:
> >  > help2man/1.47.4-GCCcore-7.3.0)
> >  >   * [x] $CFGS/m/M4/M4-1.4.18-GCCcore-7.3.0.eb (module:
> >  > M4/1.4.18-GCCcore-7.3.0)
> >  >   * [x] $CFGS/b/Bison/Bison-3.0.4-GCCcore-7.3.0.eb (module:
> >  > Bison/3.0

Re: [easybuild] Can't get local install to work

[Niek de Klein]

dthedocs.io/en/latest/Easyconfig-files-local-variables.html
.


WARNING: Use of 2 unknown easyconfig parameters detected in
GNU-4.9.3-2.25.eb: binutilsver, gccver
If these are just local variables please rename them to start with
'local_', or try using --fix-deprecated-easyconfigs to do this
automatically.
For more information, see
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
.

ERROR: Failed to process easyconfig
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb:
Failed to determine minimal toolchain for dep OpenBLAS 0.3.1-GCC-7.3.0-2.30

On Mon, Nov 18, 2019 at 4:28 PM Kenneth Hoste 
wrote:

> On 18/11/2019 21:50, Niek de Klein wrote:
> > Hi Kenneth,
> >
> > I spoke too soon, some of my installations work but not all. When I try
> > to isntall Autoconf-2.69-foss-2018.eb I get
> >
> > ERROR: Failed to process easyconfig
> >
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-foss-2018b.eb:
>
> > Failed to determine minimal toolchain for dep M4 1.4.17
> >
> > with eb --search I get:
> >
> > eb --search M4-1.4.17-foss-2018b
> >   *
> >
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/m/M4/M4-1.4.17-foss-2018b.eb
> >
> > eb --search ^foss-2018b.eb
> >   *
> >
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/f/foss/foss-2018b.eb
> >
> > So it can find both the M4-1.4.17-foss-2018b.eb file and the
> > foss-2018.eb file.
>
> Can you try "eb M4-1.4.17-foss-2018b.eb -D"?
>
>
> >
> > Kind regards,
> > Niek
> >
> >
> >
> >
> >
> > On Mon, Nov 18, 2019 at 3:21 PM Niek de Klein  > <mailto:niekdekl...@gmail.com>> wrote:
> >
> > Hi Kenneth,
> >
> > Thanks for your help!
> >
> > 1. is solved now with your suggestion of
> > EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1
> >
> > Now that the bootstrap.py ran correctly it can also find the
> > easybuild files in the location that I have configured, so this
> > solves 1 and 2 as well.
> >
> > Thanks,
> > Niek
> >
> > On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste
> > mailto:kenneth.ho...@ugent.be>> wrote:
> >
> > Dear Niek,
> >
> > Sorry that you're hitting these issues, let's try and get them
> > sorted...
> >
> > On 18/11/2019 20:00, Niek de Klein wrote:
> >  > Hi all,
> >  >
> >  > I am trying to have my own installation of EasyBuild, but I
> > have 3 problems:
> >  >
> >  > 1. When I try to run bootstrap.py it gives an error that
> > md5sums are
> >  > missing, but I'm not sure where to add them as I thought that
> > should
> >  > then have been included in the EasyBuild-4.0.1.eb file:
> >  >
> >  >  easybuild.tools.build_log.EasyBuildError: "Build of
> >  > /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed
> > (first 300
> >  > chars): Missing checksum for
> easybuild-framework-4.0.1.tar.gz')"
> >
> > Sorry for not responding to the GitHub issue [1] you opened on
> this
> > (will do so now with the answer below).
> >
> > We still need to clean up the bootstrap script a bit after the
> > EasyBuild
> > v4.0 release, it's still doing a bunch of stuff it shouldn't
> > anymore,
> > and it's not compatible yet with Python 3 (see also
> > https://github.com/easybuilders/easybuild-framework/pull/3028).
> >
> >
> > Your active EasyBuild configuration has --enforce-checksums
> > enabled (via
> > $EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're
> > seeing.
> >
> > The bootstrap script auto-generates an easyconfig file, but it
> > doesn't
> > include checksums (it could I guess, with a bit more work).
> >
> > You can dance around this though, by temporarily disabling the
> > checksums
> > requirement, only for the bootstrap:
> >
> > EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap 
> >
> >
> > If you want to, you can bootstrap EasyBuild to a tem

Re: [easybuild] Can't get local install to work

[Niek de Klein]

Hi Kenneth,

I spoke too soon, some of my installations work but not all. When I try to
isntall Autoconf-2.69-foss-2018.eb I get

ERROR: Failed to process easyconfig
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-foss-2018b.eb:
Failed to determine minimal toolchain for dep M4 1.4.17

with eb --search I get:

eb --search M4-1.4.17-foss-2018b
 *
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/m/M4/M4-1.4.17-foss-2018b.eb

eb --search ^foss-2018b.eb
 *
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/f/foss/foss-2018b.eb

So it can find both the M4-1.4.17-foss-2018b.eb file and the foss-2018.eb
file.

Kind regards,
Niek





On Mon, Nov 18, 2019 at 3:21 PM Niek de Klein  wrote:

> Hi Kenneth,
>
> Thanks for your help!
>
> 1. is solved now with your suggestion of
> EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1
>
> Now that the bootstrap.py ran correctly it can also find the easybuild
> files in the location that I have configured, so this solves 1 and 2 as
> well.
>
> Thanks,
> Niek
>
> On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste 
> wrote:
>
>> Dear Niek,
>>
>> Sorry that you're hitting these issues, let's try and get them sorted...
>>
>> On 18/11/2019 20:00, Niek de Klein wrote:
>> > Hi all,
>> >
>> > I am trying to have my own installation of EasyBuild, but I have 3
>> problems:
>> >
>> > 1. When I try to run bootstrap.py it gives an error that md5sums are
>> > missing, but I'm not sure where to add them as I thought that should
>> > then have been included in the EasyBuild-4.0.1.eb file:
>> >
>> >  easybuild.tools.build_log.EasyBuildError: "Build of
>> > /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed (first 300
>> > chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"
>>
>> Sorry for not responding to the GitHub issue [1] you opened on this
>> (will do so now with the answer below).
>>
>> We still need to clean up the bootstrap script a bit after the EasyBuild
>> v4.0 release, it's still doing a bunch of stuff it shouldn't anymore,
>> and it's not compatible yet with Python 3 (see also
>> https://github.com/easybuilders/easybuild-framework/pull/3028).
>>
>>
>> Your active EasyBuild configuration has --enforce-checksums enabled (via
>> $EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're seeing.
>>
>> The bootstrap script auto-generates an easyconfig file, but it doesn't
>> include checksums (it could I guess, with a bit more work).
>>
>> You can dance around this though, by temporarily disabling the checksums
>> requirement, only for the bootstrap:
>>
>> EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap 
>>
>>
>> If you want to, you can bootstrap EasyBuild to a temporary location (for
>> example /tmp/$USER), and then use that installation to pull in an
>> easyconfig file for EasyBuild v4.0.1 that does have checksums, using "eb
>> --from-pr 9149 --installpath " (which pulls in the
>> easyconfig from
>> https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files)
>> for the actual installation you intend to use.
>>
>> >
>> >
>> > 2. I changed the robo and installation path to be different, but I keep
>> > getting "Failed to determine minimal toolchain for dep ..." errors. The
>> > eb --show-config output is at the end of this e-mail, and shows that my
>> > robot-paths is
>> >
>> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,
>>
>> > which is where I cloned the easyconfigs repository.
>>
>> What happens if you use "eb --search" with this setup, can 'eb' find
>> easyconfig files?
>>
>> Try something like "eb --search tensorflow-1.14" for example.
>>
>>
>> >
>> > 3. Even if the robo path wasn't working, tools that are already in my
>> > module avail are not being picked up. For example, EasyBuild that is
>> not
>> > working uses "toolchain = {'name': 'foss', 'version': '2018b'}" and has
>> > as one of the dependencies "('ncurses', '6.0'),". When I try to install
>> > it I get
>> >
>> > ERROR: Failed to process easyconfig
>> >
>> /groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:
>>
>> > Failed to determin

Re: [easybuild] Can't get local install to work

[Niek de Klein]

Hi Kenneth,

Thanks for your help!

1. is solved now with your suggestion of
EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1

Now that the bootstrap.py ran correctly it can also find the easybuild
files in the location that I have configured, so this solves 1 and 2 as
well.

Thanks,
Niek

On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste 
wrote:

> Dear Niek,
>
> Sorry that you're hitting these issues, let's try and get them sorted...
>
> On 18/11/2019 20:00, Niek de Klein wrote:
> > Hi all,
> >
> > I am trying to have my own installation of EasyBuild, but I have 3
> problems:
> >
> > 1. When I try to run bootstrap.py it gives an error that md5sums are
> > missing, but I'm not sure where to add them as I thought that should
> > then have been included in the EasyBuild-4.0.1.eb file:
> >
> >  easybuild.tools.build_log.EasyBuildError: "Build of
> > /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed (first 300
> > chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"
>
> Sorry for not responding to the GitHub issue [1] you opened on this
> (will do so now with the answer below).
>
> We still need to clean up the bootstrap script a bit after the EasyBuild
> v4.0 release, it's still doing a bunch of stuff it shouldn't anymore,
> and it's not compatible yet with Python 3 (see also
> https://github.com/easybuilders/easybuild-framework/pull/3028).
>
>
> Your active EasyBuild configuration has --enforce-checksums enabled (via
> $EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're seeing.
>
> The bootstrap script auto-generates an easyconfig file, but it doesn't
> include checksums (it could I guess, with a bit more work).
>
> You can dance around this though, by temporarily disabling the checksums
> requirement, only for the bootstrap:
>
> EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap 
>
>
> If you want to, you can bootstrap EasyBuild to a temporary location (for
> example /tmp/$USER), and then use that installation to pull in an
> easyconfig file for EasyBuild v4.0.1 that does have checksums, using "eb
> --from-pr 9149 --installpath " (which pulls in the
> easyconfig from
> https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files)
> for the actual installation you intend to use.
>
> >
> >
> > 2. I changed the robo and installation path to be different, but I keep
> > getting "Failed to determine minimal toolchain for dep ..." errors. The
> > eb --show-config output is at the end of this e-mail, and shows that my
> > robot-paths is
> >
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,
>
> > which is where I cloned the easyconfigs repository.
>
> What happens if you use "eb --search" with this setup, can 'eb' find
> easyconfig files?
>
> Try something like "eb --search tensorflow-1.14" for example.
>
>
> >
> > 3. Even if the robo path wasn't working, tools that are already in my
> > module avail are not being picked up. For example, EasyBuild that is not
> > working uses "toolchain = {'name': 'foss', 'version': '2018b'}" and has
> > as one of the dependencies "('ncurses', '6.0'),". When I try to install
> > it I get
> >
> > ERROR: Failed to process easyconfig
> >
> /groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:
>
> > Failed to determine minimal toolchain for dep ncurses 6.0
> >
> > but when I do ml avail ncur I get:
> >
> > --
> > /groups/umcg-biogen/tmp01/apps/modules/devel
> > ---
> > ncurses/6.0-foss-2018b
> >
> > ---
> > /apps/modules/devel
> > ---
> > ncurses/6.0ncurses/6.1-foss-2018bncurses/6.1-GCCcore-7.3.0
> (D)
> >
> > So the ncurses module is already available
> >
> >
> > Am I missing something in my configuration?
>
> This is a consequence of having --minimal-toolchains enabled, which
> requires that EasyBuild is able to find the easyconfig file for the
> toolchain (foss-2018b.eb in this case).
>
> It needs to figure out what the (versions of) the subtoolchains are for
> foss/2018b, so it can check for ncurses installed with GCCcore first,
> before gompi, before foss.
>
> So this is basically the same issu

[easybuild] Can't get local install to work

[Niek de Klein]

Hi all,

I am trying to have my own installation of EasyBuild, but I have 3 problems:

1. When I try to run bootstrap.py it gives an error that md5sums are
missing, but I'm not sure where to add them as I thought that should then
have been included in the EasyBuild-4.0.1.eb file:

easybuild.tools.build_log.EasyBuildError: "Build of
/tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed (first 300
chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"


2. I changed the robo and installation path to be different, but I keep
getting "Failed to determine minimal toolchain for dep ..." errors. The eb
--show-config output is at the end of this e-mail, and shows that my
robot-paths is
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,
which is where I cloned the easyconfigs repository.

3. Even if the robo path wasn't working, tools that are already in my
module avail are not being picked up. For example, EasyBuild that is not
working uses "toolchain = {'name': 'foss', 'version': '2018b'}" and has as
one of the dependencies "('ncurses', '6.0'),". When I try to install it I
get

ERROR: Failed to process easyconfig
/groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:
Failed to determine minimal toolchain for dep ncurses 6.0

but when I do ml avail ncur I get:

--
/groups/umcg-biogen/tmp01/apps/modules/devel
---
   ncurses/6.0-foss-2018b

---
/apps/modules/devel
---
   ncurses/6.0ncurses/6.1-foss-2018bncurses/6.1-GCCcore-7.3.0 (D)

So the ncurses module is already available


Am I missing something in my configuration?

Thanks for your help!
Niek de Klein


#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F:
configuration file)
#
buildpath   (E) =
/groups/umcg-biogen/tmp01/apps//.tmp/easybuild/builds/
containerpath   (D) =
/home/umcg-ndeklein/.local/easybuild/containers
enforce-checksums   (E) = True
installpath (E) = /groups/umcg-biogen/tmp01/apps/
minimal-toolchains  (E) = True
repositorypath  (D) =
/home/umcg-ndeklein/.local/easybuild/ebfiles_repo
robot-paths (E) =
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/
set-gid-bit (E) = True
sourcepath  (E) = /groups/umcg-biogen/tmp01/apps//sources/
umask   (E) = 002
verify-easyconfig-filenames (E) = True

Re: [easybuild] Python Package sanity check import other name

[Niek de Klein]

Thanks!

On Wed, Jun 29, 2016 at 3:44 PM, Kenneth Hoste 
wrote:

> Hi Niek,
>
> On 29/06/16 15:41, Niek de Klein wrote:
>
>> Dear list,
>>
>> I am writing an easyconfig for BeautifulSoup4 (
>> https://pypi.python.org/pypi/beautifulsoup4).
>> The sanity check tries to import it with import beautifulsoup4. However,
>> to import beautifulsoup the actual command is
>>
>>from bs4 import BeautifulSoup
>>
>> How can I change how it tries to import for sanity checking? I now tried
>> to overwrite it with setting sanity_commands, but it does not overwrite. I
>> can change the name to bs4, but then it can not find bs4-4.4.1.tar.gz.
>>
>
> The import check is not done via sanity check commands, it's separate.
>
> You can control the module name being used for the import check with:
>
> options = {'modulename': 'bs4'}
>
> See for example
> https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/p/pyTables/pyTables-3.2.2-intel-2015b-Python-2.7.10.eb
> .
>
>
> regards,
>
> Kenneth
>

[easybuild] Python Package sanity check import other name

[Niek de Klein]

Dear list,

I am writing an easyconfig for BeautifulSoup4 (
https://pypi.python.org/pypi/beautifulsoup4).
The sanity check tries to import it with import beautifulsoup4. However, to
import beautifulsoup the actual command is

   from bs4 import BeautifulSoup

How can I change how it tries to import for sanity checking? I now tried to
overwrite it with setting sanity_commands, but it does not overwrite. I can
change the name to bs4, but then it can not find bs4-4.4.1.tar.gz.

Thanks,
Niek

Re: [easybuild] Error copying files

[Niek de Klein]

Hi Kenneth,

Thanks a lot. I have opened an issue with the developers of the software
who kept that in for developement, and they said they will fix it in their
next release.

Regards,
Niek

On Wed, Jun 15, 2016 at 1:37 PM, Kenneth Hoste 
wrote:

> Hi Niek,
>
> I've issued a PR to make the Binary (and MakeCp) easyblock aware of the
> general 'keepsymlinks' easyconfig parameter (
> https://github.com/hpcugent/easybuild-easyblocks/pull/959), so with this
> in place you should be able to fix your problem by including this in your
> easyconfig file:
>
> keepsymlinks = True
>
> That does leave the question whether it makes sense to leave broken
> symlinks in place though...
>
>
> regards,
>
> Kenneth
>
>
> On 14/06/16 22:05, Kenneth Hoste wrote:
>
>> Hi Niek,
>>
>> Basically, the tarball is broken, is contains a bunch of broken symlinks,
>> for example:
>>
>> $ ls -l db/test/microbiome/bacteria/
>> total 0
>> lrwxrwxrwx 1 vsc40023 vsc40023 50 Jun 14 21:17 bacteria.00.idx ->
>> /u/home/s/serghei/project/Bacteria/bacteria.00.idx
>> lrwxrwxrwx 1 vsc40023 vsc40023 50 Jun 14 21:17 bacteria.00.nhd ->
>> /u/home/s/serghei/project/Bacteria/bacteria.00.nhd
>> lrwxrwxrwx 1 vsc40023 vsc40023 50 Jun 14 21:17 bacteria.00.nhi ->
>> /u/home/s/serghei/project/Bacteria/bacteria.00.nhi
>>
>>
>> The can also be considered a bug in the Binary easyblock (which is
>> leveraged by PackagedBinary you are using), since it should be calling
>> copytree with symlinks=True to make it preserve symlinks rather than
>> resolve them (cfr.
>> https://docs.python.org/2/library/shutil.html#shutil.copytree).
>> With that setting, it will happily copy the broken symlinks...
>>
>>
>> regards,
>>
>> Kenneth
>>
>> On 13/06/16 18:49, Niek de Klein wrote:
>>
>>> Hi all,
>>>
>>> I am trying to install ROP from here:
>>> https://github.com/smangul1/rop/archive/1.0.2.tar.gz. I attached the
>>> .eb file I'm using. I get errors of the type
>>>
>>> ('rop-1.0.2/source/vprofile/G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6.csv',
>>> '/apps/software/ROP/1.0.2-Python-2.7.11-foss-2015b/source/vprofile/G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6.csv',
>>> "[Errno 2] No such file or directory:
>>> 'rop-1.0.2/source/vprofile/G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6.csv'"),
>>>
>>> See attached .log file for full log.
>>>
>>> Our install buildpath is in /apps/.tmp/, and there I can find the file
>>> that should be copied, i.e this is the path to the existing file
>>> /apps/.tmp/easybuild/builds/ROP/1.0.2/dummy-dummy-Python-2.7.11-foss-2015b/rop-1.0.2/source/vprofile/G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6.csv,
>>> and the path it should copy to also exists:
>>>
>>>  ls
>>> /apps/software/ROP/1.0.2-Python-2.7.11-foss-2015b/source/vprofile/
>>>  extractViralReads2.py  extractViralReads.py
>>> G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6_filtered.csv
>>>
>>> And it did copy some of the files to that folder.
>>>
>>> Any idea what I am doing wrong?
>>>
>>> Thanks,
>>> Niek
>>>
>>
>>
>

[easybuild] Error copying files

[Niek de Klein]

Hi all,

I am trying to install ROP from here:
https://github.com/smangul1/rop/archive/1.0.2.tar.gz. I attached the .eb
file I'm using. I get errors of the type


('rop-1.0.2/source/vprofile/G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6.csv',
'/apps/software/ROP/1.0.2-Python-2.7.11-foss-2015b/source/vprofile/G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6.csv',
"[Errno 2] No such file or directory:
'rop-1.0.2/source/vprofile/G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6.csv'"),

See attached .log file for full log.

Our install buildpath is in /apps/.tmp/, and there I can find the file that
should be copied, i.e this is the path to the existing file
/apps/.tmp/easybuild/builds/ROP/1.0.2/dummy-dummy-Python-2.7.11-foss-2015b/rop-1.0.2/source/vprofile/G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6.csv,
and the path it should copy to also exists:

 ls /apps/software/ROP/1.0.2-Python-2.7.11-foss-2015b/source/vprofile/
 extractViralReads2.py  extractViralReads.py
G32965.GTEX-XAJ8-0726-SM-47JY5.3.unmapped_after_rRNA_virus_blastFormat6_filtered.csv

And it did copy some of the files to that folder.

Any idea what I am doing wrong?

Thanks,
Niek


ROP_1.0.2-Python-2.7.11-foss-2015b.eb
Description: Binary data


easybuild-HK9aos.log
Description: Binary data

[easybuild] Missing info in Deprecated-functionality.html?

[Niek de Klein]

When I try to install an easyconfig that does not include an easyblock I
get

ERROR: [...] NO LONGER SUPPORTED since v2.0: [...] see
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html
for more information

When I go to the deprecated functionality page there is nothing documented
for easyconfig authors:

For authors of easyconfig files:

*(nothing yet)*


Should I not get the "no longer supported" error, or is the deprecated
functionality page not updated?

Thanks,
Niek

Re: [easybuild] How to copy a file from the install directory?

[Niek de Klein]

That works, thanks!

On Wed, Mar 23, 2016 at 11:55 AM, Kenneth Hoste  wrote:
> Hi Niek,
>
> The configure step is skipped by default by MakeCp, so defining
> preconfigopts is not going to help; prebuildopts is what you're looking for.
>
> Besides that, you need to make sure that the 'make' command being executed
> doesn't get mangled; this is typically done using '&&' at the end.
>
> So, try this:
>
> prebuildopts = "cp make.inc.example make.inc && "
>
>
> regards,
>
> Kenneth
>
>
> On 23/03/16 10:49, Niek de Klein wrote:
>>
>> Hi all,
>>
>> I am trying to make an easyconfig for CLAPACK
>> (http://www.netlib.org/clapack/). The tar includes make.inc.example
>> which has to be copied to make.inc. I tried
>>
>>  preconfigopts = 'cp make.inc.example make.inc'
>>  preconfigopts = 'cp /path/to/extracted_tar/make.inc.example
>> /path/to/extracted_tar
>>  preinstallcmds = 'cp make.inc.example make.inc'
>>  preinstallcmds = 'cp /path/to/extracted_tar/make.inc.example
>> /path/to/extracted_tar
>>
>> but these did not work. Anyone know what command I should use? Below
>> is the easybuild file I have
>>
>> Thanks!
>> Niek
>>
>>
>>
>> easyblock = 'MakeCp'
>>
>> name = 'CLAPACK'
>> version = '3.2.1'
>> name_lower = name.lower()
>>
>> homepage = 'http://www.netlib.org/clapack/'
>> description = """The CLAPACK library was built using a Fortran to C
>> conversion utility called f2c.
>>   The entire Fortran 77 LAPACK library is run through
>> f2c to obtain C code,
>>   and then modified to improve readability.
>>   CLAPACK's goal is to provide LAPACK for someone who
>> does not have access to a Fortran compiler. """
>>
>> toolchain = {'name': 'foss', 'version': '2015b'}
>> toolchainopts = {'optarch': True, 'usempi': True}
>>
>> dependencies = [('CMake','3.2.2')]
>>
>> # http://www.netlib.org/clapack/clapack-3.2.1-CMAKE.tgz
>> source_urls = ['http://www.netlib.org/%s' % name_lower]
>> sources = ['%s-%s-CMAKE.tgz' % (name_lower, version)]
>>
>> preinstallcmds = ['cp
>> /apps/.tmp/easybuild/builds/CLAPACK/3.2.1/foss-2015b/make.inc.example
>> /apps/.tmp/easybuild/builds/CLAPACK/3.2.1/foss-2015b/make.inc']
>>
>>
>> files_to_copy = [('clapack','bin/')]
>>
>> moduleclass = 'tools'
>
>

[easybuild] How to copy a file from the install directory?

[Niek de Klein]

Hi all,

I am trying to make an easyconfig for CLAPACK
(http://www.netlib.org/clapack/). The tar includes make.inc.example
which has to be copied to make.inc. I tried

preconfigopts = 'cp make.inc.example make.inc'
preconfigopts = 'cp /path/to/extracted_tar/make.inc.example
/path/to/extracted_tar
preinstallcmds = 'cp make.inc.example make.inc'
preinstallcmds = 'cp /path/to/extracted_tar/make.inc.example
/path/to/extracted_tar

but these did not work. Anyone know what command I should use? Below
is the easybuild file I have

Thanks!
Niek



easyblock = 'MakeCp'

name = 'CLAPACK'
version = '3.2.1'
name_lower = name.lower()

homepage = 'http://www.netlib.org/clapack/'
description = """The CLAPACK library was built using a Fortran to C
conversion utility called f2c.
 The entire Fortran 77 LAPACK library is run through
f2c to obtain C code,
 and then modified to improve readability.
 CLAPACK's goal is to provide LAPACK for someone who
does not have access to a Fortran compiler. """

toolchain = {'name': 'foss', 'version': '2015b'}
toolchainopts = {'optarch': True, 'usempi': True}

dependencies = [('CMake','3.2.2')]

# http://www.netlib.org/clapack/clapack-3.2.1-CMAKE.tgz
source_urls = ['http://www.netlib.org/%s' % name_lower]
sources = ['%s-%s-CMAKE.tgz' % (name_lower, version)]

preinstallcmds = ['cp
/apps/.tmp/easybuild/builds/CLAPACK/3.2.1/foss-2015b/make.inc.example
/apps/.tmp/easybuild/builds/CLAPACK/3.2.1/foss-2015b/make.inc']


files_to_copy = [('clapack','bin/')]

moduleclass = 'tools'

Re: [easybuild] overwite default sanity check command for PythonPackage easyblock

[Niek de Klein]

Thanks that worked!

On Mon, Feb 22, 2016 at 11:08 PM, Jack Perdue  wrote:
> Howdy Niek,
>
> Try this:
>
> 1) add the line
>
> options = {'modulename': 'vcf'}
>
> 2) remove your "sanity_check_commands" line.
> By default, the PythonPackage easyblock will
> try to import it (using the name above) during
> sanity checking
>
> See Biopython (among others) for an example.
>
> Jack Perdue
> Lead Systems Administrator
> High Performance Research Computing
> TAMU Division of Research
> j-per...@tamu.edu    http://sc.tamu.edu
> SC Helpdesk: h...@sc.tamu.edu
>
>
> On 02/22/2016 03:53 PM, Niek de Klein wrote:
>>
>> Hi alI,
>>
>> I am trying to install PyVCF
>> (https://pypi.python.org/pypi/PyVCF/0.6.7) which is imported with
>>
>>  import vcf
>>
>> instead of
>>
>>  import pyvcf
>>
>> The problem I have is that when I use
>>
>> sanity_check_commands = [('python', '-c "import vcf"')]
>>
>> the easyblock PythonPackage still uses
>>
>>  python -c "import pyvcf"
>>
>> as sanity command as well.
>>
>> I could only find an open issue
>> (https://github.com/hpcugent/easybuild-easyconfigs/issues/1205)
>> regarding this, anyone know how I can work around this?
>>
>> Cheers,
>> Niek
>
>

[easybuild] overwite default sanity check command for PythonPackage easyblock

[Niek de Klein]

Hi alI,

I am trying to install PyVCF
(https://pypi.python.org/pypi/PyVCF/0.6.7) which is imported with

import vcf

instead of

import pyvcf

The problem I have is that when I use

   sanity_check_commands = [('python', '-c "import vcf"')]

the easyblock PythonPackage still uses

python -c "import pyvcf"

as sanity command as well.

I could only find an open issue
(https://github.com/hpcugent/easybuild-easyconfigs/issues/1205)
regarding this, anyone know how I can work around this?

Cheers,
Niek

Re: [easybuild] Sourceforge download

[Niek de Klein]

Hi Martin,


Thanks for making the .eb file. However, I am getting the below error for
the plugin patch, do you know how to fix this?

Thanks,
Niek

== 2015-10-08 21:23:25,613 main ERROR cmd "patch -b -p1 -i
/apps/sources/EasyBuild/custom/Bridger-r2014-12-01-fix-plugin-Makefiles.patch"
exited with exitcode 1 and output:
patching file plugins/fastool/Makefile
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file
plugins/fastool/Makefile.rej
patching file plugins/rsem/makefile
Hunk #2 FAILED at 50.
Hunk #3 FAILED at 83.
Hunk #4 FAILED at 93.
Hunk #5 FAILED at 127.
4 out of 5 hunks FAILED -- saving rejects to file plugins/rsem/makefile.rej
patching file plugins/rsem/sam/Makefile
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 25.
Hunk #3 FAILED at 48.
3 out of 3 hunks FAILED -- saving rejects to file
plugins/rsem/sam/Makefile.rej
patching file plugins/rsem/sam/bcftools/Makefile
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 11.
Hunk #3 FAILED at 31.
3 out of 3 hunks FAILED -- saving rejects to file
plugins/rsem/sam/bcftools/Makefile.rej
patching file plugins/rsem/sam/misc/Makefile
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 11.
Hunk #3 FAILED at 20.
Hunk #4 FAILED at 28.
4 out of 4 hunks FAILED -- saving rejects to file
plugins/rsem/sam/misc/Makefile.rej
 (at easybuild/tools/run.py:359 in parse_cmd_output)
== 2015-10-08 21:23:25,647 main.easyblock WARNING build failed (first 300
chars): cmd "patch -b -p1 -i
/apps/sources/EasyBuild/custom/Bridger-r2014-12-01-fix-plugin-Makefiles.patch"
exited with exitcode 1 and output:
patching file plugins/fastool/Makefile
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file
plugins/fastool/Makefile.rej
patching file plugins/rsem/
== 2015-10-08 21:23:25,647 main.MakeCp INFO Closing log for application
name Bridger version r2014-12-01

On Thu, Oct 8, 2015 at 5:49 PM, Martin  wrote:

> The software derives stuff from BOOST_ROOT existing as an environment
> variable
>
> Anyhow I put some work into it as we also needed Bridger, sorry for it not
> being a pull request but maybe it is of some use to Niek de Klein.
>
> Mind you: Not my best piece of work, but it gets the job done. Also Kudos
> to Arnau for beating me to it :)
>
> best,
> Martin
>
>
> Files are attached
> On Wed, Oct 7, 2015 at 5:11 PM, Kenneth Hoste 
> wrote:
>
>> Why the $BOOST_ROOT, and not just use
>> --configopts="--with-boost=$EBROOTBOOST" ?
>>
>
>
>
>
> --
> http://www.xing.com/profile/Martin_Marcher
> http://www.linkedin.com/in/martinmarcher
> Mobil: +43 / 660 / 62 45 103
> UID: ATU68801424
>

Re: [easybuild] Sourceforge download

[Niek de Klein]

It's in there, albeit hard coded:

configopts =
'--with-boost=/apps/software/Boost/1.57.0-goolf-1.7.20-Python-2.7.9/'


On Wed, Oct 7, 2015 at 1:49 PM, Martin  wrote:

> If you go thru creating a BOOST_ROOT shouldn't you do this also:
>
> configopts = '--with-boost=$BOOST_ROOT'
>
>
> /Martin
>
> On Wed, Oct 7, 2015 at 10:49 AM, Niek de Klein 
> wrote:
>
>> Hi Arnau,
>>
>> Here is the easyconfig file I have but it is not working yet because it
>> makes Boost instead of using the already installed one.
>>
>> Cheers,
>> Niek
>>
>>
>> easyblock = 'ConfigureMake'
>>
>> name = 'Bridger'
>> version = 'r2014-12-01'
>>
>> homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
>> description = """Bridger is an efficient de novo trascriptome assembler
>> for RNA-Seq data. """
>>
>> toolchain = {'name': 'goolf', 'version': '1.7.20'}
>> toolchainopts = {'pic': True, 'usempi': True}
>>
>> preconfigopts = 'export BOOST_ROOT=$EBROOTBOOST;'
>> preconfigopts += 'export BOOST_INCLUDEDIR=$EBROOTBOOST/include;'
>> preconfigopts += 'export BOOST_LIBRARYDIR=$EBROOTBOOST/lib;'
>>
>> configopts =
>> '--with-boost=/apps/software/Boost/1.57.0-goolf-1.7.20-Python-2.7.9/'
>>
>> sources = ['%(name)s_%(version)s.tar.gz']
>> source_urls = [('http://downloads.sourceforge.net/project/rnaseqassembly'
>> )]
>>
>> dependencies = [
>> ('zlib', '1.2.8'),
>> ('Boost', '1.57.0','-Python-2.7.9'),
>> ]
>>
>>
>> files_to_copy = [('Assemble','bin/')]
>>
>> moduleclass = 'bio'
>>
>> On Wed, Oct 7, 2015 at 10:39 AM, Arnau  wrote:
>>
>>> Hi Niek,
>>>
>>> would you mind sharing your easyconfig file for Bridger? :-)
>>>
>>> TIA,
>>> Arnau
>>>
>>> 2015-09-24 11:20 GMT+02:00 Niek de Klein :
>>>
>>>> That works, thanks!
>>>>
>>>> On Thu, Sep 24, 2015 at 11:11 AM, Jens Timmerman <
>>>> jens.timmer...@ugent.be> wrote:
>>>>
>>>>> Hi Niek,
>>>>>
>>>>> SOURCELOWER_TAR_BZ2 results in 'bridger-r2014.12.01'
>>>>> so this wil  not work in the sources field.
>>>>>
>>>>> your version seems wrong aswell
>>>>> This seems to work for me:
>>>>>
>>>>>
>>>>> easyblock = 'MakeCp'
>>>>>
>>>>> name = 'Bridger'
>>>>> version = 'r2014-12-01'
>>>>>
>>>>> homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
>>>>> description = """Bridger is an efficient de novo trascriptome
>>>>> assembler for RNA-Seq data. """
>>>>>
>>>>> toolchain = {'name': 'goolf', 'version': '1.7.20'}
>>>>> toolchainopts = {'pic': True, 'usempi': True}
>>>>>
>>>>>
>>>>> sources = ['%(name)s_%(version)s.tar.gz']
>>>>> source_urls = [('
>>>>> http://downloads.sourceforge.net/project/rnaseqassembly')]
>>>>>
>>>>>
>>>>> files_to_copy = [('Assemble','bin/')]
>>>>>
>>>>> moduleclass = 'bio'
>>>>>
>>>>> On 24/09/15 10:55, Niek de Klein wrote:
>>>>>
>>>>> Hi easybuild list,
>>>>>
>>>>> I'm trying to install Bridger from sourceforge. The sourceforge link
>>>>> is:
>>>>>
>>>>>
>>>>> sourceforge.net/projects/rnaseqassembly/files/Bridger_r2014-12-01.tar.gz/download
>>>>>
>>>>>
>>>>>
>>>>> So I tried using
>>>>>
>>>>> name = 'Bridger'
>>>>> version = 'r2014.12.01'
>>>>> [..]
>>>>> source_urls = [SOURCEFORGE_SOURCE]
>>>>> sources = ['%%(name)s_%s.tar.gz' % '-'.join(version.split('.'))]
>>>>>
>>>>> but this does not find the file to download. Looking at other progra

Re: [easybuild] Sourceforge download

[Niek de Klein]

Hi Arnau,

Here is the easyconfig file I have but it is not working yet because it
makes Boost instead of using the already installed one.

Cheers,
Niek


easyblock = 'ConfigureMake'

name = 'Bridger'
version = 'r2014-12-01'

homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
description = """Bridger is an efficient de novo trascriptome assembler for
RNA-Seq data. """

toolchain = {'name': 'goolf', 'version': '1.7.20'}
toolchainopts = {'pic': True, 'usempi': True}

preconfigopts = 'export BOOST_ROOT=$EBROOTBOOST;'
preconfigopts += 'export BOOST_INCLUDEDIR=$EBROOTBOOST/include;'
preconfigopts += 'export BOOST_LIBRARYDIR=$EBROOTBOOST/lib;'

configopts =
'--with-boost=/apps/software/Boost/1.57.0-goolf-1.7.20-Python-2.7.9/'

sources = ['%(name)s_%(version)s.tar.gz']
source_urls = [('http://downloads.sourceforge.net/project/rnaseqassembly')]

dependencies = [
('zlib', '1.2.8'),
('Boost', '1.57.0','-Python-2.7.9'),
]


files_to_copy = [('Assemble','bin/')]

moduleclass = 'bio'

On Wed, Oct 7, 2015 at 10:39 AM, Arnau  wrote:

> Hi Niek,
>
> would you mind sharing your easyconfig file for Bridger? :-)
>
> TIA,
> Arnau
>
> 2015-09-24 11:20 GMT+02:00 Niek de Klein :
>
>> That works, thanks!
>>
>> On Thu, Sep 24, 2015 at 11:11 AM, Jens Timmerman > > wrote:
>>
>>> Hi Niek,
>>>
>>> SOURCELOWER_TAR_BZ2 results in 'bridger-r2014.12.01'
>>> so this wil  not work in the sources field.
>>>
>>> your version seems wrong aswell
>>> This seems to work for me:
>>>
>>>
>>> easyblock = 'MakeCp'
>>>
>>> name = 'Bridger'
>>> version = 'r2014-12-01'
>>>
>>> homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
>>> description = """Bridger is an efficient de novo trascriptome assembler
>>> for RNA-Seq data. """
>>>
>>> toolchain = {'name': 'goolf', 'version': '1.7.20'}
>>> toolchainopts = {'pic': True, 'usempi': True}
>>>
>>>
>>> sources = ['%(name)s_%(version)s.tar.gz']
>>> source_urls = [('http://downloads.sourceforge.net/project/rnaseqassembly
>>> ')]
>>>
>>>
>>> files_to_copy = [('Assemble','bin/')]
>>>
>>> moduleclass = 'bio'
>>>
>>> On 24/09/15 10:55, Niek de Klein wrote:
>>>
>>> Hi easybuild list,
>>>
>>> I'm trying to install Bridger from sourceforge. The sourceforge link is:
>>>
>>>
>>> sourceforge.net/projects/rnaseqassembly/files/Bridger_r2014-12-01.tar.gz/download
>>>
>>>
>>>
>>> So I tried using
>>>
>>> name = 'Bridger'
>>> version = 'r2014.12.01'
>>> [..]
>>> source_urls = [SOURCEFORGE_SOURCE]
>>> sources = ['%%(name)s_%s.tar.gz' % '-'.join(version.split('.'))]
>>>
>>> but this does not find the file to download. Looking at other programs
>>> installed from sourceforge it looks for this
>>>
>>>
>>> sourceforge.net/projects/rnaseqassembly/files/bridger//2014.12.01/Bridger_2014-12-01.tar.gz/download
>>>
>>> so I think because this does not make use of
>>> // SOURCEFORGE_SOURCE won't work,
>>> correct?
>>>
>>> So then I tried with
>>>
>>> source_urls = ['sourceforge.net/projects/rnaseqassembly/files/']
>>>  sources = ['%%(name)s_%s.tar.gz/download' %
>>> '-'.join(version.split('.'))]
>>>
>>> But this gives the error
>>>  Unknown file type for file
>>> /apps/sources/b/Bridger/Bridger_r2014-12-01.tar.gz/download (['download'])")
>>>
>>> Without /download at the end it doesn't find the url.
>>>
>>> Other things I have tried:
>>>
>>>source_urls = ['sourceforge.net/projects/rnaseqassembly/files/']
>>>sources = [('%%(name)s_%s.tar.gz' %
>>> '-'.join(version.split('.')),'download')]
>>>
>>>
>>>   sources = [SOURCELOWER_TAR_BZ2]
>>>   source_urls = [('http://sourceforge.net/projects/rnaseqassembly/files/',
>>> 'download')]
>>>
>>> and
>>>
>>>   sources = [SOURCELOWER_TAR_BZ2]
>>>   source_urls = [('http://sourceforge.net/projects/rnaseqassembly/',
>>> 'download')]
>>>
>>> But then the files can't be found. I don't know what else to try, how
>>> can I get this downloaded from sourceforge? Below the full .eb file for
>>> replication:
>>>
>>> easyblock = 'MakeCp'
>>>
>>> name = 'Bridger'
>>> version = 'r2014.12.01'
>>>
>>> homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
>>> description = """Bridger is an efficient de novo trascriptome assembler
>>> for RNA-Seq data. """
>>>
>>> toolchain = {'name': 'goolf', 'version': '1.7.20'}
>>> toolchainopts = {'pic': True, 'usempi': True}
>>>
>>>
>>> sources = [SOURCELOWER_TAR_BZ2]
>>> source_urls = [('http://sourceforge.net/projects/rnaseqassembly/files',
>>> 'download')]
>>>
>>>
>>> files_to_copy = [('Assemble','bin/')]
>>>
>>> moduleclass = 'bio'
>>>
>>>
>>>
>>>
>>>
>>>
>>
>

Re: [easybuild] Sourceforge download

[Niek de Klein]

That works, thanks!

On Thu, Sep 24, 2015 at 11:11 AM, Jens Timmerman 
wrote:

> Hi Niek,
>
> SOURCELOWER_TAR_BZ2 results in 'bridger-r2014.12.01'
> so this wil  not work in the sources field.
>
> your version seems wrong aswell
> This seems to work for me:
>
>
> easyblock = 'MakeCp'
>
> name = 'Bridger'
> version = 'r2014-12-01'
>
> homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
> description = """Bridger is an efficient de novo trascriptome assembler
> for RNA-Seq data. """
>
> toolchain = {'name': 'goolf', 'version': '1.7.20'}
> toolchainopts = {'pic': True, 'usempi': True}
>
>
> sources = ['%(name)s_%(version)s.tar.gz']
> source_urls = [('http://downloads.sourceforge.net/project/rnaseqassembly
> ')]
>
>
> files_to_copy = [('Assemble','bin/')]
>
> moduleclass = 'bio'
>
> On 24/09/15 10:55, Niek de Klein wrote:
>
> Hi easybuild list,
>
> I'm trying to install Bridger from sourceforge. The sourceforge link is:
>
>
> sourceforge.net/projects/rnaseqassembly/files/Bridger_r2014-12-01.tar.gz/download
>
>
>
> So I tried using
>
> name = 'Bridger'
> version = 'r2014.12.01'
> [..]
> source_urls = [SOURCEFORGE_SOURCE]
> sources = ['%%(name)s_%s.tar.gz' % '-'.join(version.split('.'))]
>
> but this does not find the file to download. Looking at other programs
> installed from sourceforge it looks for this
>
>
> sourceforge.net/projects/rnaseqassembly/files/bridger//2014.12.01/Bridger_2014-12-01.tar.gz/download
>
> so I think because this does not make use of
> // SOURCEFORGE_SOURCE won't work,
> correct?
>
> So then I tried with
>
> source_urls = ['sourceforge.net/projects/rnaseqassembly/files/']
>  sources = ['%%(name)s_%s.tar.gz/download' %
> '-'.join(version.split('.'))]
>
> But this gives the error
>  Unknown file type for file
> /apps/sources/b/Bridger/Bridger_r2014-12-01.tar.gz/download (['download'])")
>
> Without /download at the end it doesn't find the url.
>
> Other things I have tried:
>
>source_urls = ['sourceforge.net/projects/rnaseqassembly/files/']
>sources = [('%%(name)s_%s.tar.gz' %
> '-'.join(version.split('.')),'download')]
>
>
>   sources = [SOURCELOWER_TAR_BZ2]
>   source_urls = [('http://sourceforge.net/projects/rnaseqassembly/files/',
> 'download')]
>
> and
>
>   sources = [SOURCELOWER_TAR_BZ2]
>   source_urls = [('http://sourceforge.net/projects/rnaseqassembly/',
> 'download')]
>
> But then the files can't be found. I don't know what else to try, how can
> I get this downloaded from sourceforge? Below the full .eb file for
> replication:
>
> easyblock = 'MakeCp'
>
> name = 'Bridger'
> version = 'r2014.12.01'
>
> homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
> description = """Bridger is an efficient de novo trascriptome assembler
> for RNA-Seq data. """
>
> toolchain = {'name': 'goolf', 'version': '1.7.20'}
> toolchainopts = {'pic': True, 'usempi': True}
>
>
> sources = [SOURCELOWER_TAR_BZ2]
> source_urls = [('http://sourceforge.net/projects/rnaseqassembly/files',
> 'download')]
>
>
> files_to_copy = [('Assemble','bin/')]
>
> moduleclass = 'bio'
>
>
>
>
>
>

[easybuild] Sourceforge download

[Niek de Klein]

Hi easybuild list,

I'm trying to install Bridger from sourceforge. The sourceforge link is:


sourceforge.net/projects/rnaseqassembly/files/Bridger_r2014-12-01.tar.gz/download

So I tried using

name = 'Bridger'
version = 'r2014.12.01'
[..]
source_urls = [SOURCEFORGE_SOURCE]
sources = ['%%(name)s_%s.tar.gz' % '-'.join(version.split('.'))]

but this does not find the file to download. Looking at other programs
installed from sourceforge it looks for this


sourceforge.net/projects/rnaseqassembly/files/bridger//2014.12.01/Bridger_2014-12-01.tar.gz/download

so I think because this does not make use of
// SOURCEFORGE_SOURCE won't work,
correct?

So then I tried with

source_urls = ['sourceforge.net/projects/rnaseqassembly/files/']
 sources = ['%%(name)s_%s.tar.gz/download' %
'-'.join(version.split('.'))]

But this gives the error
 Unknown file type for file
/apps/sources/b/Bridger/Bridger_r2014-12-01.tar.gz/download (['download'])")

Without /download at the end it doesn't find the url.

Other things I have tried:

   source_urls = ['sourceforge.net/projects/rnaseqassembly/files/']
   sources = [('%%(name)s_%s.tar.gz' %
'-'.join(version.split('.')),'download')]


  sources = [SOURCELOWER_TAR_BZ2]
  source_urls = [('http://sourceforge.net/projects/rnaseqassembly/files/',
'download')]

and

  sources = [SOURCELOWER_TAR_BZ2]
  source_urls = [('http://sourceforge.net/projects/rnaseqassembly/',
'download')]

But then the files can't be found. I don't know what else to try, how can I
get this downloaded from sourceforge? Below the full .eb file for
replication:

easyblock = 'MakeCp'

name = 'Bridger'
version = 'r2014.12.01'

homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
description = """Bridger is an efficient de novo trascriptome assembler for
RNA-Seq data. """

toolchain = {'name': 'goolf', 'version': '1.7.20'}
toolchainopts = {'pic': True, 'usempi': True}


sources = [SOURCELOWER_TAR_BZ2]
source_urls = [('http://sourceforge.net/projects/rnaseqassembly/',
'download')]


files_to_copy = [('Assemble','bin/')]

moduleclass = 'bio'

[easybuild]

[Niek de Klein]

Hi all,

I can't find an easyblock for this but maybe I have overlooked it so:
I'm trying to install HOMER, which is installed by putting
"configureHomer.pl" in the folder where you want to have HOMER installed
(so this should not go to /sourcres/h/Homer but to the installation
folder), and then have to run

perl configureHomer.pl

I haven't found an EasyBlock that uses perl to install programs though, is
one already available? Here is the EB I have so far:


easyblock = 'EB_HOMER'# tried to make my own easyblock that uses perl
but did not work
name = 'HOMER'
version = 'master'

homepage = 'http://homer.salk.edu/homer/'
description = """HOMER (Hypergeometric Optimization of Motif EnRichment) is
a suite of tools for Motif Discovery."""

toolchain = {'name': 'goolf', 'version': '1.7.20'}

source_urls = ['http://homer.salk.edu/homer/']
sources = ['configureHomer.pl']


executables =
['makeTagDirectory','findPeaks','mergePeaks','genomeOntology','getPeakTags','makeUCSCfile',
'getDifferentialPeaks', 'assignGenomeAnnotation',
'homerTools','homer','homer2','analyzeHiC',
'getGenomeTilingPeaks']

files_to_copy = [(executables,'bin')]

dependencies = [
('Ghostscript', '9.16'),
('Weblogo', '2.8.2', '', True),
('BLAT', '3.5'),
]

moduleclass = 'bio'

Thanks!
Niek

Re: [easybuild] install from git clone

[Niek de Klein]

Ah I didn't see it also has releases, got it installed now, thanks!


On Fri, Aug 28, 2015 at 3:26 PM, Ward Poelmans  wrote:
> On Fri, Aug 28, 2015 at 2:42 PM, Niek de Klein  wrote:
>> git clone https://github.com/voutcn/megahit.git
>
> They have releases so you can simplify use
> https://github.com/voutcn/megahit/archive/v1.0.2.tar.gz as source url?
>
> Ward

[easybuild] install from git clone

[Niek de Klein]

Hi all,

I want to install MEGAHIT which has the following install procedure:

git clone https://github.com/voutcn/megahit.git
cd megahit
make
./megahit -1 pe_1.fq.gz -2 pe_2.fq.gz -o megahit_out


Is there an EasyBlock that can install from git clone? I tried cloning
megahit manually (like the GATK easyconfig) to the same folder as the
other sources get downloaded to, but when I use the MakeCp easyblock
it can't find it.

Thanks,
Niek

[easybuild] Add .R script to path

[Niek de Klein]

Hi easybuild list,

I'm installing a program that includes a .R script. The files that I want
in my path

executables = ['run_dREG.bsh', 'writeBed.bsh', 'run_dREG.R', 'bed_merge.pl
','writeBed.bsh']

files_to_copy = [(executables, 'bin')]

The .bsh files I can run after module load dREG with bash run_dREG.bsh.
However, I when I try Rscript run_dREG.bsh I get Fatal error: cannot open
file 'run_dREG.R': No such file or directory.

When I do Rscript /path/to/install_folder/dREG/bin/run_dREG.R it does work.
So how can I get the R script to be added to the path?

Thanks,
Niek

Re: [easybuild] Easier way of installing R packages

[Niek de Klein]

> I would not know how to accomplish this without using R's own dependency 
> handling mechanism and AFAIK their is no way to do this in Python.

Could this be done with rpy?

Niek

On Wed, Jun 24, 2015 at 12:53 PM, Pieter Neerincx
 wrote:
> Hi Kenneth,
>
> On Jun 24, 2015, at 11:10 AM, Kenneth Hoste  wrote:
>
>> Hi Pieter,
>>
>> On 24/06/15 10:58, Pieter Neerincx wrote:
>>> Hi,
>>>
>>> On Jun 24, 2015, at 7:34 AM, Kenneth Hoste  wrote:
>>>
 Hi Riccardo,

 On 19/06/15 10:32, Riccardo Murri wrote:
> Hi Ken, all,
>
> (Kenneth Hoste, Thu, Jun 18, 2015 at 04:32:27PM +0200:)
>> We deliberately do not enable 'dependency=True', to ensure we can 
>> reproduce
>> the exact same installation later; if you let R automagically resolve
>> dependencies and let it pick the most recent version at that time for the
>> dependencies, you can't do an exact reproduced install.
> Could this be worked around by having EB save the list of installed R
> packages + versions (like `pip freeze`) upon first install, then re-use
> the saved list if available.
>
> Something along these lines (pseudo-code):
>
> package_list_file = easyconfig_file_name.replace('.eb', 
> '.pkglist.csv')
> if not exists(package_list_file):
># 1. install packages by name, let R choose version
># 2. dump list of installed packages to file: (pkgname, 
> version)
> else:
># 1. read in the pkglist.csv file
># 2. use R's install_version() to install the exact same pkg 
> and version
 I like this idea...
>>> +1
>>>
 We could have an easyconfig parameter like 'auto_deps = True', and when 
 that is set, EasyBuild lets R resolve dependencies by itself.

 Once the full list of easyconfigs has been installed, EasyBuild could then 
 see what got installed, and which versions, and adjust the list of 
 extensions included in the easyconfig file. The tricky bit may be the 
 order, but there's probably a solution for that (see Stefano's R script).

 Now, who's up for implementing this? :-)
>>> Yesterday I started working on an R-script that generates an *.eb from an 
>>> existing R installation. This is based on Stefano's code (Many thanks for 
>>> sharing that!) and adds some commandline switches, help and code to figure 
>>> out whether a package came from a CRAN repo or from BioConductor. Hence 
>>> this is slightly different compared to the above as it generates a complete 
>>> *.eb from pure R, but I can easily add an option to only generate a pkglist 
>>> file, which may then be used by an existing *.eb.
>>>
>>> Is it Ok if I add this with a README to the /r/R/ dir in the 
>>> EasyConfigs repo and create a pull request or would you prefer this kind of 
>>> supplementary code to go elsewhere?
>>
>> It would be better if you could integrate this in the R easyblock somehow (I 
>> realize that may be a lot to ask for).
>
> The small amount of code that generates the *.eb could be in Python and hence 
> that could go in the easyblock, but the majority of the code is R based on 
> Stefano's script to generate the list of R packages their versions & the 
> repos they came from and then re-order the list based on the dependencies. I 
> would not know how to accomplish this without using R's own dependency 
> handling mechanism and AFAIK their is no way to do this in Python.
>
> So the best I can imagine for now is a modified R easyblock that can execute 
> an external R script to create a pkglist... and then generate an updated 
> easyconfig.
>
>> We currently don't have a good location for this; I don't think we should 
>> start adding 'random' scripts to the easyconfigs repo...
>
> The .../r/R/ dir in the EasyConfigs repo would be the first place where I 
> would look for examples and a way to create new updated R easyconfigs, so in 
> that sense it isn't a random location, but I agree that it is random in the 
> sense that it isn't EasyBuild style Python code and R specific. Preferably we 
> would not need a kludge like this :o
>
>> Maybe go with https://github.com/fgeorgatos/easybuild.experimental for now 
>> to share it with others?
>
> That looks like a good place to start, so we can have a first version of an 
> pure R easyconfig generator by the end of this week...
>
>> We should look into a contrib directory in the main EasyBuild repo 
>> (https://github.com/hpcugent/easybuild), where people can get stuff included 
>> without too much reviewing or very strict requirements, we already have a 
>> couple of PRs for stuff like that (see 
>> https://github.com/hpcugent/easybuild-framework/pull/1190, 
>> https://github.com/hpcugent/easybuild-framework/pull/1180).
>
> Check, let me know when there is a contrib dir in place.
>
> Cheers,
>
> Pi
>
>> regards,
>>
>> Kenneth
>
> -
> phone: +31 6 143 

Re: [easybuild] Binary of a python package

[Niek de Klein]

HI Oliver,

You can try the parameter executables and files_to_copy, like so:

executables = ['binary_file_name']
files_to_copy = [(executables, 'bin')]

Cheers,
Niek


On Tue, Jun 23, 2015 at 5:29 PM, Stolpe, Oliver
 wrote:
> Hello list,
>
> I wrote an easyconfig file to install a python package. The installation is
> successful besides the fact that easybuild throws away the binary that is
> generated (or main entrance point for the standalone). When I install the
> software on my machine with python setup.py build/install --user, it puts
> the binary to ~/.local/bin, but I don't know where easybuild puts this file.
> When I look into the easybuild/software/MYSOFTWARE directory, there is no
> bin folder.
>
> Please help me save the binary :-)
>
> Thanks,
>   Oliver
>
> The easyconfig file:
>
> easyblock = 'PythonPackage'
>
> name = 'misopy'
> version = '0.5.3'
>
> homepage = 'http://genes.mit.edu/burgelab/miso/index.html'
> description = """MISO (Mixture of Isoforms) is a probabilistic framework
> that
>  quantitates the expression level of alternatively spliced genes from
> RNA-Seq
>  data, and identifies differentially regulated isoforms or exons across
> samples.
>  By modeling the generative process by which reads are produced from
> isoforms in
>  RNA-Seq, the MISO model uses Bayesian inference to compute the probability
> that
>  a read originated from a particular isoform."""
>
> toolchain = {'name': 'goolf', 'version': '1.4.10'}
>
> source_urls = ['http://pypi.python.org/packages/source/m/misopy']
> sources = ['%(namelower)s-%(version)s.tar.gz']
>
> python = 'Python'
> pythonver = '2.7.6'
> versionsuffix = '-%s-%s' % (python, pythonver)
>
> dependencies = [
> ('matplotlib', '1.3.1', versionsuffix),
> (python, pythonver),
> ]
>
> moduleclass = 'bio'
>

Re: [easybuild] Easier way of installing R packages

[Niek de Klein]

Thanks all!

On Thu, Jun 18, 2015 at 5:16 PM, Pablo Escobar Lopez
 wrote:
> Hi Niek,
>
> I think many people in this list has suffer the R dependencies nightmare.
>
> I wrote an small scirpt which if you already have a R easyconfig will try to
> give you which are the latest versions available in R repos. It's not
> perfect but it helps a little bit at least. Here you have it in case it's
> helpfful for you
> https://github.com/pescobar/random-scripts/blob/master/easybuild-update-R-libs.py
>
> regards
> Pablo.
>
> 2015-06-18 16:38 GMT+02:00 Jack Perdue :
>>
>> Howdy Niek,
>>
>> We have a non-Easybuild module tree.  The attachment
>> from before lives in /software/tamusc/modulefiles/R.
>> It loads the EB R and then sets the needed env vars
>> for building extensions.
>>
>> Note that I don't maintain the R extensions so don't
>> have details.  Here are the scripts written by my colleague
>> Maikel Pennings for such things.  You'd have to ask him to
>> explain them.
>>
>> Jack Perdue
>> Lead Systems Administrator
>> High Performance Research Computing
>> TAMU Division of Research
>> j-per...@tamu.eduhttp://sc.tamu.edu
>> SC Helpdesk: h...@sc.tamu.edu
>>
>> On 06/18/2015 09:27 AM, Niek de Klein wrote:
>> > Hi Jack,
>> >
>> > Thank you for your answer. I don't understand the attached file. How
>> > do you use that to install extensions, and how do you keep it so that
>> > if you want to migrate to another cluster you can install R with the
>> > exact same R libraries installed? Do you keep a separate file with all
>> > the R packages you installed extra?
>> >
>> > Thanks,
>> > Niek
>> >
>> > On Thu, Jun 18, 2015 at 4:20 PM, Jack Perdue  wrote:
>> >> Howdy Niek,
>> >>
>> >> FWIW, here is our latest build of R (with lots of
>> >> dependencies added):
>> >>
>> >>
>> >> http://www.siliconslick.com/easybuild/ebfiles_repo_cleaned/ada/R/R-3.2.0-intel-2015B-default-mt.eb
>> >>
>> >> For extensions, we have another (non-EB) module (attached)
>> >> for installing extensions to the above by hand.
>> >>
>> >> Between the two, we've:
>> >>
>> >> a) tried to make sure we aren't using system libs (e.g. TCL)
>> >> b) are able to satisfy user requests for misc. R extensions
>> >>  (automagically)
>> >>
>> >> Jack Perdue
>> >> Lead Systems Administrator
>> >> High Performance Research Computing
>> >> TAMU Division of Research
>> >> j-per...@tamu.eduhttp://sc.tamu.edu
>> >> SC Helpdesk: h...@sc.tamu.edu
>> >>
>> >>
>> >> On 06/18/2015 08:57 AM, Niek de Klein wrote:
>> >>> Hi mailinglist,
>> >>>
>> >>> I'm probably missing something here and unnecessarily complicating
>> >>> things. I am trying to get R installed with just a few packages we
>> >>> use. One of the packages, ggplot2, has a lot of dependencies. So when
>> >>> I try to install R with one of the existing EasyBuild files and have
>> >>> ggplot2 as the dependency it will throw an error due to the
>> >>> dependencies of ggplot2 not being installed, and I have to add the
>> >>> dependencies to the .eb file. And then add the dependencies of the
>> >>> dependencies to the .eb file, etc etc.
>> >>>
>> >>> When installing libraries from within R you can simply give the option
>> >>> "dependency=True" and it will automagically install all dependencies
>> >>> for you. Is there an option like that when installing R and some R
>> >>> libraries?
>> >>>
>> >>> Thanks,
>> >>> Niek
>> >>
>>
>
>
>
> --
> Pablo Escobar López
> HPC systems engineer
> Biozentrum, University of Basel
> Swiss Institute of Bioinformatics SIB
> Email: pablo.escobarlo...@unibas.ch
> Phone: +41 61 267 21 80
> http://www.biozentrum.unibas.ch

Re: [easybuild] Easier way of installing R packages

[Niek de Klein]

Hi Jack,

Thank you for your answer. I don't understand the attached file. How
do you use that to install extensions, and how do you keep it so that
if you want to migrate to another cluster you can install R with the
exact same R libraries installed? Do you keep a separate file with all
the R packages you installed extra?

Thanks,
Niek

On Thu, Jun 18, 2015 at 4:20 PM, Jack Perdue  wrote:
> Howdy Niek,
>
> FWIW, here is our latest build of R (with lots of
> dependencies added):
>
> http://www.siliconslick.com/easybuild/ebfiles_repo_cleaned/ada/R/R-3.2.0-intel-2015B-default-mt.eb
>
> For extensions, we have another (non-EB) module (attached)
> for installing extensions to the above by hand.
>
> Between the two, we've:
>
> a) tried to make sure we aren't using system libs (e.g. TCL)
> b) are able to satisfy user requests for misc. R extensions
> (automagically)
>
> Jack Perdue
> Lead Systems Administrator
> High Performance Research Computing
> TAMU Division of Research
> j-per...@tamu.eduhttp://sc.tamu.edu
> SC Helpdesk: h...@sc.tamu.edu
>
>
> On 06/18/2015 08:57 AM, Niek de Klein wrote:
>>
>> Hi mailinglist,
>>
>> I'm probably missing something here and unnecessarily complicating
>> things. I am trying to get R installed with just a few packages we
>> use. One of the packages, ggplot2, has a lot of dependencies. So when
>> I try to install R with one of the existing EasyBuild files and have
>> ggplot2 as the dependency it will throw an error due to the
>> dependencies of ggplot2 not being installed, and I have to add the
>> dependencies to the .eb file. And then add the dependencies of the
>> dependencies to the .eb file, etc etc.
>>
>> When installing libraries from within R you can simply give the option
>> "dependency=True" and it will automagically install all dependencies
>> for you. Is there an option like that when installing R and some R
>> libraries?
>>
>> Thanks,
>> Niek
>
>

[easybuild] Easier way of installing R packages

[Niek de Klein]

Hi mailinglist,

I'm probably missing something here and unnecessarily complicating
things. I am trying to get R installed with just a few packages we
use. One of the packages, ggplot2, has a lot of dependencies. So when
I try to install R with one of the existing EasyBuild files and have
ggplot2 as the dependency it will throw an error due to the
dependencies of ggplot2 not being installed, and I have to add the
dependencies to the .eb file. And then add the dependencies of the
dependencies to the .eb file, etc etc.

When installing libraries from within R you can simply give the option
"dependency=True" and it will automagically install all dependencies
for you. Is there an option like that when installing R and some R
libraries?

Thanks,
Niek

[easybuild] how to set dependency to not use compiler?

[Niek de Klein]

I have an easyconfig file (for HOMER) where I use

toolchain = {'name': 'goolf', 'version': '1.7.20'}

And in the dependencies I have a dependency (Weblogo) that is built
without compiler. When I try to install HOMER with easybuild I get the
error:

ERROR: Irresolvable dependencies encountered: Weblogo/2.8.2-goolf-1.7.20

when it should be using Weblogo/2.8.2.

My dependencies in the HOMER.eb file look like this:

dependencies = [
('Ghostscript', '9.16'),
('Weblogo', '2.8.2'),
('BLAT', '3.5'),
]

How can I set in dependency that it should use Weblogo/2.8.2, not
Weblogo/2.8.2-goolf-1.7.20?

Thanks,
Niek

Re: [easybuild] Sending text to the command line during installation

[Niek de Klein]

Thanks Ward and Kenneth, enough options to use

Niek

On Thu, Jun 11, 2015 at 10:39 AM, Kenneth Hoste 
wrote:

> Hi Niek,
>
>
> On 11/06/15 10:31, Niek de Klein wrote:
>
>> Hi all,
>>
>> I'm trying to write an .eb file for Anacona, which has a binary file
>> installer. During installation you have to send to the command line:
>> , yes, path/to/install/. I use easyblock = 'Binary'.
>> The first two steps (pressing enter and saying yes) I can do by using
>>
>>   install_cmd = 'sh %(name)s-%(version)s.sh | yes'
>>
>> However, I don't know how to send the install path to the command line
>> after pressing yes. Is there already support in EasyBuild for sending text
>> to the command line during installation?
>>
>
> You have a couple of options here (in order of my preference)
>
> i) You can enhance the BInary generic easyblock so you can specify a
> string to be passed to stdin of the install command; you can do this via
> the 'inp' named argument in the run_cmd function used by the Binary
> easyblock
>
> ii) You can write an easyblock that derives from Binary, and uses the
> 'inp' argument in run_cmd to pass 'yes' and the install dir.
>
> iii) You can write an easyblock that uses run_cmd_qa, to interact with the
> interactive installer (see
> https://github.com/hpcugent/easybuild-easyblocks/search?utf8=%E2%9C%93&q=run_cmd_qa
> ).
>
>
> regards,
>
> Kenneth
>

[easybuild] Sending text to the command line during installation

[Niek de Klein]

Hi all,

I'm trying to write an .eb file for Anacona, which has a binary file
installer. During installation you have to send to the command line:
, yes, path/to/install/. I use easyblock = 'Binary'.
The first two steps (pressing enter and saying yes) I can do by using

  install_cmd = 'sh %(name)s-%(version)s.sh | yes'

However, I don't know how to send the install path to the command line
after pressing yes. Is there already support in EasyBuild for sending text
to the command line during installation?

Thanks,
Niek

Re: [easybuild] Setting up own easyblock repository

[Niek de Klein]

Hello Oliver,

I'm not part of the EasyBuild team but I think I see what goes wrong.
I think you forgot to add the easyblock name. What is the class name
you used in FastQC.py? For example, if in FastQC.py, under the imports
you have

class EB_FastQC(EasyBlock):
   ...

You should add

easyblock = 'EB_FastQC'

in your FastQC-0.11.3.eb.

Hope this helps,
Niek

On Tue, Jun 9, 2015 at 1:43 PM, Stolpe, Oliver  wrote:
> Hello Mailinglist,
>
> I wanted to setup my own EasyBlock repository. I followed the instructions
> from
> .
> Somehow it ignores my easyblock when I try to execute the config file. Any
> help is appreciated!
>
> Cheers,
>   Oliver
>
> This is what I got:
>
> /tools/easybuild.local/easybuild/easyblocks
> /tools/easybuild.local/easybuild/easyblocks/FastQC.py
> /tools/easybuild.local/easybuild/easyblocks/__init__.py
>
> $ echo $PYTHONPATH
> /tools/easybuild/software/EasyBuild/2.1.1/lib/python2.7/site-packages:/tools/easybuild.local
>
> $ head FastQC-0.11.3.eb
> name = 'FastQC'
> version = '0.11.3'
>
> homepage = 'http://www.bioinformatics.babraham.ac.uk/projects/fastqc'
>
> $ eb FastQC-0.11.3.eb
> == temporary log file in case of crash /tmp/eb-hJyLxt/easybuild-NGWQ6S.log
> ERROR: Failed to process easyconfig
> /tools/easybuild.local/easyconfigs/f/FastQC/FastQC-0.11.3.eb: NO LONGER
> SUPPORTED since v2.0: Fallback to default easyblock ConfigureMake (from
> easybuild.easyblocks.generic.configuremake); use "easyblock =
> 'ConfigureMake'" in easyconfig file?; see
> http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for
> more information
>

Re: [easybuild] Where in the manual can I find what each EasyBlock does, and how to make my own?

[Niek de Klein]

How can I write these easyblocks in a custom directory? I followed the
steps from
https://github.com/hpcugent/easybuild/wiki/Setting-up-your-own-easyblocks-repository.
The folder where I saved the python file is in the $MYEBDIR folder:

   $ readlink -f easybuild/easyblocks/anaconda.py
   /apps/sources/EasyBuild/custom/easybuild/easyblocks/anaconda.py
   $ $MYEBDIR
   -bash: /apps/sources/EasyBuild/custom/easybuild/easyblocks/: is a
directory

and the folder is also in PYTHONPATH:

   $ $PYTHONPATH
   -bash:
/apps/software/EasyBuild/2.1.1/lib/python2.6/site-packages:/apps/sources/EasyBuild/custom/easybuild/easyblocks/:
No such file or directory

I double checked this by importing anaconda.py from Python, which works

   >>> import anaconda
   >>> dir(anaconda)
   ['CUSTOM', 'EB_Anaconda', 'EasyBlock', 'MANDATORY', '__builtins__',
'__doc__', '__file__', '__name__', '__package__', '__warningregistry__',
'env', 'os', 'run_cmd', 'toolchain']

In my anaconda.eb file I use easyblock = "EB_Anaconda". When I try to
install with eb anaconda.eb --robot I get the error:

   ERROR: Failed to process easyconfig
/apps/sources/EasyBuild/custom/anaconda-1.8.0.eb: Failed to obtain class
for EB_Anaconda easyblock (not available?): No module named anaconda

Thanks,
Niek

On Wed, Jun 3, 2015 at 6:14 PM, Alan O'Cais  wrote:

>  This script (that comes with EasyBuild) will create a commented template
> for you to work on:
>
> https://github.com/hpcugent/easybuild-framework/blob/master/easybuild/scripts/mk_tmpl_easyblock_for.py
>
> On 3 June 2015 at 18:07, Robert Schmidt  wrote:
>
>> I don't think there is any easy one like this (but you could write your
>> own easyblock).
>>
>> You might look at CmdCp as an idea. It could work for what you want too
>> (you could specify a source file in the map, but never use the variable in
>> the command).
>>
>>
>> https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/cmdcp.py
>>
>>
>>  On Wed, Jun 3, 2015 at 11:52 AM Niek de Klein 
>> wrote:
>>
>>> Hi all,
>>>
>>>  I have been going through the manual and I can't find a description of
>>> the steps that each of the easyblocks make. I can only find the list of
>>> easyblocks, an introduction and specifications. Specifically, I need an
>>> easyblock that only does "sh installer.sh". I can't find if any of the
>>> easyblocks already does that, or if not, how I can make an easyblock
>>> myself.
>>>
>>>  Can you point me where I can find this in the manual?
>>>
>>>  Thanks,
>>> Niek
>>>
>>
>
>
>  --
>  Dr. Alan O'Cais
> Application Support
> Juelich Supercomputing Centre
> Forschungszentrum Juelich GmbH
> 52425 Juelich, Germany
>
> Phone: +49 2461 61 5213
> Fax: +49 2461 61 6656
> E-mail: a.oc...@fz-juelich.de
> WWW:http://www.fz-juelich.de/ias/jsc/EN
>
>
>
> 
>
> 
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
>
> 
>
> 
>
>

[easybuild] Where in the manual can I find what each EasyBlock does, and how to make my own?

[Niek de Klein]

Hi all,

I have been going through the manual and I can't find a description of the
steps that each of the easyblocks make. I can only find the list of
easyblocks, an introduction and specifications. Specifically, I need an
easyblock that only does "sh installer.sh". I can't find if any of the
easyblocks already does that, or if not, how I can make an easyblock
myself.

Can you point me where I can find this in the manual?

Thanks,
Niek

[easybuild] How to get the correct file structure when software and folder have identical names?

[Niek de Klein]

Hi all,

I'm trying to write an .eb file for verifyBamID. The folder where the
executable verifyBamID is located in is also called verifyBamId.

So now when I use

executables = ['verifyBamID']
files_to_copy = [(executables, 'bin')]

The path to the executable is
$root/verifyBamID/1.1.2-goolf-1.7.20/bin/verifyBamID/bin/verifyBamID. This
should be $root/verifyBamID/1.1.2-goolf-1.7/20/bin/verifyBamID, both for
clarity and because $root/verifyBamID/1.1.2-goolf-1.7/20/bin/ is added to
PATH.

I have the same problem for other software where the folder name is the
same as the executable. For software where the rest of the .eb file is
exactly the same but where the folder name and executable name are
different it works as expected.

How can I get folder structure to be set correctly when the folder and
software name are the same?

Thanks!
Niek