Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
On 29/10/14 16:50, Backeljauw Franky wrote: Op 29-okt.-2014, om 15:58 heeft Ward Poelmans het volgende geschreven: In that case, then what’s the benefit of having multiple CMake modules, e.g. CMake-3.0.0-intel-2014b, CMake-3.0.0-foss-2014b, CMake-3.0.0-GCC-4.8.3? I would think that, depending on the version you load, CMake would also know it has to take the compiler with which it was built… There is no benefit of having multiple CMake with different toolchains. We already discussed multiple times what to do with toolchain independent modules (stuff like autotools, cmake, bison, ...) and haven't come up with the perfect solution. They way EB currently works, we need to build CMake for each toolchain. What CMake-3.0.0-intel-2014b means is: CMake 3.0 build with intel 2014b toolchain, not CMake 3.0 which always uses intel 2014b toolchain to build stuff. I understand — it’s an EB “issue” we have to live with. No problem. Just to know: is that subject to change anywhere in the future (e.g., starting from EB version 2 or so)? There a couple of things we can do here: * make toolchain modules define $CC, $CXX, etc. (optionally); something like this is being supported in the Intel MPI easyblock for example (cfr. https://github.com/hpcugent/easybuild-easyblocks/pull/493) * make EasyBuild also consider subtoolchains when resolving dependencies (e.g. also consider a CMake built with a GCC toolchain when resolving dependencies for a goolf build); this will not fix the runtime CMake behavior, but it will avoid the need for a CMake module per toolchain (or at least make that significantly easier) We don't have to wait for this until EasyBuild v2, since we can disable these features by default and provide switches to turn it on. However, someone needs to step up and actually look into this... If easy build takes care of this, then what does it do ‘extra’? It sets a bunch of environmental variables and gives some arguments to CMake where to look for includes, libraries, etc. Run easybuild with the -l option, and you will see what it does (or just look into the cmakemake class). As long as we are sure that this happens, it’s fine for us, so we will check those to see what happens exactly… Thanks for the pointer. If you want CMake to do something in particular, specify it hard, e.g. using -DCMAKE_C_COMPILER and friends (see https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/cmakemake.py#L87). There's a similar issue for the MPI wrappers commands, e.g. the mpicc provided by Intel MPI which defaults to using GCC (!). Adding support for making these wrappers behave more sensibly is the point of https://github.com/hpcugent/easybuild-easyblocks/pull/493. K. The fastest way to build something with the intel toolchain using a CMake build system is: CC=icc CXX=icpc cmake . I guess once we find out what happens exactly we’ll be making something similar available to our users. — Regards, Franky
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
On 10/29/2014 04:50 PM, Backeljauw Franky wrote: I understand — it’s an EB “issue” we have to live with. No problem. Just to know: is that subject to change anywhere in the future (e.g., starting from EB version 2 or so)? I believe this could be fixed if we were to make the cmake builds smarter, e.g. you could tell cmake to only look in certain subdirectories when looking for files, and ignore /usr/ However, I'm not a CMake specialist and further investigation is needed, but I did have a talk with the cmake developers who assured me this could be done. Jens
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
> Op 29-okt.-2014, om 15:58 heeft Ward Poelmans het > volgende geschreven: > >> In that case, then what’s the benefit of having multiple CMake modules, e.g. >> CMake-3.0.0-intel-2014b, CMake-3.0.0-foss-2014b, CMake-3.0.0-GCC-4.8.3? I >> would think that, depending on the version you load, CMake would also know >> it has to take the compiler with which it was built… > > There is no benefit of having multiple CMake with different > toolchains. We already discussed multiple times what to do with > toolchain independent modules (stuff like autotools, cmake, bison, > ...) and haven't come up with the perfect solution. They way EB > currently works, we need to build CMake for each toolchain. What > CMake-3.0.0-intel-2014b means is: CMake 3.0 build with intel 2014b > toolchain, not CMake 3.0 which always uses intel 2014b toolchain to > build stuff. I understand — it’s an EB “issue” we have to live with. No problem. Just to know: is that subject to change anywhere in the future (e.g., starting from EB version 2 or so)? >> If easy build takes care of this, then what does it do ‘extra’? > > It sets a bunch of environmental variables and gives some arguments to > CMake where to look for includes, libraries, etc. Run easybuild with > the -l option, and you will see what it does (or just look into the > cmakemake class). As long as we are sure that this happens, it’s fine for us, so we will check those to see what happens exactly… Thanks for the pointer. > The fastest way to build something with the intel toolchain using a > CMake build system is: CC=icc CXX=icpc cmake . I guess once we find out what happens exactly we’ll be making something similar available to our users. — Regards, Franky
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
On Wed, Oct 29, 2014 at 3:40 PM, Backeljauw Franky wrote: >> Well, you have to tell it about icc/icpc. By default, CMake will look >> for gcc and if it finds gcc, it will use it. EB sets several env >> variables to point CMake in the right direction (CC, CXX, CFLAGS, >> CXXFLAGS, ...). You also have to set these variables for CMake to use >> the intel compilers. Just setting CC=icc CXX=icpc will probably do the >> trick. If you build something with easybuild, this is all taken care >> of. > > > In that case, then what’s the benefit of having multiple CMake modules, e.g. > CMake-3.0.0-intel-2014b, CMake-3.0.0-foss-2014b, CMake-3.0.0-GCC-4.8.3? I > would think that, depending on the version you load, CMake would also know it > has to take the compiler with which it was built… There is no benefit of having multiple CMake with different toolchains. We already discussed multiple times what to do with toolchain independent modules (stuff like autotools, cmake, bison, ...) and haven't come up with the perfect solution. They way EB currently works, we need to build CMake for each toolchain. What CMake-3.0.0-intel-2014b means is: CMake 3.0 build with intel 2014b toolchain, not CMake 3.0 which always uses intel 2014b toolchain to build stuff. > If easy build takes care of this, then what does it do ‘extra’? It sets a bunch of environmental variables and gives some arguments to CMake where to look for includes, libraries, etc. Run easybuild with the -l option, and you will see what it does (or just look into the cmakemake class). The fastest way to build something with the intel toolchain using a CMake build system is: CC=icc CXX=icpc cmake . Ward
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
Hello Ward, > Op 29-okt.-2014, om 12:13 heeft Ward Poelmans het > volgende geschreven: > > On Wed, Oct 29, 2014 at 12:05 PM, Backeljauw Franky > wrote: > >> We’ve managed to get CMake to build on our local filesystem, but when I load >> it, it falls back to the system installed gcc instead of the intel toolchain: > >> /tmp/hello_world> cmake . >> -- The C compiler identification is GNU 4.4.7 >> -- The CXX compiler identification is GNU 4.4.7 >> -- Check for working C compiler: /usr/bin/cc >> -- Check for working C compiler: /usr/bin/cc -- works >> -- Detecting C compiler ABI info >> -- Detecting C compiler ABI info - done >> -- Check for working CXX compiler: /usr/bin/c++ >> -- Check for working CXX compiler: /usr/bin/c++ -- works >> -- Detecting CXX compiler ABI info >> -- Detecting CXX compiler ABI info - done >> -- Configuring done >> -- Generating done >> -- Build files have been written to: /tmp/hello_world > > > Well, you have to tell it about icc/icpc. By default, CMake will look > for gcc and if it finds gcc, it will use it. EB sets several env > variables to point CMake in the right direction (CC, CXX, CFLAGS, > CXXFLAGS, ...). You also have to set these variables for CMake to use > the intel compilers. Just setting CC=icc CXX=icpc will probably do the > trick. If you build something with easybuild, this is all taken care > of. In that case, then what’s the benefit of having multiple CMake modules, e.g. CMake-3.0.0-intel-2014b, CMake-3.0.0-foss-2014b, CMake-3.0.0-GCC-4.8.3? I would think that, depending on the version you load, CMake would also know it has to take the compiler with which it was built… If easy build takes care of this, then what does it do ‘extra’? — Regards, Franky
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
On Wed, Oct 29, 2014 at 12:05 PM, Backeljauw Franky wrote: > We’ve managed to get CMake to build on our local filesystem, but when I load > it, it falls back to the system installed gcc instead of the intel toolchain: > /tmp/hello_world> cmake . > -- The C compiler identification is GNU 4.4.7 > -- The CXX compiler identification is GNU 4.4.7 > -- Check for working C compiler: /usr/bin/cc > -- Check for working C compiler: /usr/bin/cc -- works > -- Detecting C compiler ABI info > -- Detecting C compiler ABI info - done > -- Check for working CXX compiler: /usr/bin/c++ > -- Check for working CXX compiler: /usr/bin/c++ -- works > -- Detecting CXX compiler ABI info > -- Detecting CXX compiler ABI info - done > -- Configuring done > -- Generating done > -- Build files have been written to: /tmp/hello_world Well, you have to tell it about icc/icpc. By default, CMake will look for gcc and if it finds gcc, it will use it. EB sets several env variables to point CMake in the right direction (CC, CXX, CFLAGS, CXXFLAGS, ...). You also have to set these variables for CMake to use the intel compilers. Just setting CC=icc CXX=icpc will probably do the trick. If you build something with easybuild, this is all taken care of. Ward
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
Hello Ward, We’ve managed to get CMake to build on our local filesystem, but when I load it, it falls back to the system installed gcc instead of the intel toolchain: /tmp/hello_world> module list Currently Loaded Modulefiles: 2) GCC/4.8.3 3) icc/2013.5.192-GCC-4.8.3 4) ifort/2013.5.192-GCC-4.8.3 5) iccifort/2013.5.192-GCC-4.8.3 6) impi/4.1.3.049-iccifort-2013.5.192-GCC-4.8.3 7) iimpi/5.5.3-GCC-4.8.3 8) imkl/11.1.2.144-iimpi-5.5.3-GCC-4.8.3 9) intel/2014b 10) ncurses/5.9-intel-2014b 11) CMake/3.0.0-intel-2014b /tmp/hello_world> cmake . -- The C compiler identification is GNU 4.4.7 -- The CXX compiler identification is GNU 4.4.7 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Configuring done -- Generating done -- Build files have been written to: /tmp/hello_world Since you mention it below, I’m wondering: shouldn’t it be using the intel compilers instead? In the logfile for CMake, it shows: loading initial cache file /tmp/CMake-easybuild/CMake/3.0.0/intel-2014b/cmake-3.0.0/Bootstrap.cmk/InitialCacheFlags.cmake -- The C compiler identification is Intel 13.1.0.20130607 -- The CXX compiler identification is Intel 13.1.0.20130607 -- Check for working C compiler: /apps/antwerpen/ivybridge/sl6/icc/2013.5.192-GCC-4.8.3/bin/intel64/icc -- Check for working C compiler: /apps/antwerpen/ivybridge/sl6/icc/2013.5.192-GCC-4.8.3/bin/intel64/icc -- works It finds the icc executable when building the module, so why isn’t it being used after the module is loaded? -- Thanks for your reply, Franky Op 28-okt.-2014, om 12:06 heeft Ward Poelmans het volgende geschreven: > On Tue, Oct 28, 2014 at 12:04 PM, Pablo Escobar Lopez > wrote: >> I verified it when I tried the workaround and it seems to work fine, at >> least in this case. > > That's because you are using a gcc toolchain and it is in the path. I > suspect this would fall back to system gcc with an ictce toolchain. > > Ward
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
Ha! Well I'm glad this was useful. Honestly I was surprised to see this
bug again -- it was really obscure. Maybe as you say it only affects
machines named hopper :)
From: Backeljauw Franky
Reply-To: "easybuild@lists.ugent.be"
List-Post: easybuild@lists.ugent.be
Date: Tuesday, October 28, 2014 at 8:15 AM
To: "easybuild@lists.ugent.be"
Subject: Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
> Hello again,
> Todd,
>
> We have found the following alert on IBM¹s website regarding GPFS:
>
> * http://www.gpfsusergroup.org/news/gpfs-3-5-announcments
> * http://www-01.ibm.com/support/docview.wss?uid=isg3T1021392
>
>> IBM has identified a problem with GPFS 3.5.0.20 and GPFS 4.1.0.2 where GPFS
>> may fail to correctly handle multiple vectors passed via the writev() system
>> call. When a {NULL, 0} is passed as the first vector, an EINVAL error may be
>> incorrectly returned. This would cause the user application to fail
>> unexpectedly when writev() is called to write to a GPFS file. User data are
>> not affected. The writev() call is most likely to have been automatically
>> generated by the library or compiler.
>
> Guess what we are running GPFS 3.5.0.20 on our compute nodes :-( And it
> would also explain why we do not have it on another (non-compute) node, since
> that is still running GPFS 3.5.0.19... That¹s what they call bad luck!
>
> Thanks a lot for pointing is in the right direction. Now let¹s hope that we
> get a GFPS fix soon!
>
> -- Regards,
>
> Franky Backeljauw
>
>
>
> Op 28-okt.-2014, om 13:46 heeft Backeljauw Franky
> het volgende geschreven:
>
>> Hello Todd,
>>
>> You¹re spot on! Here the file testC(XX)Compiler.c was empty indeed. We are
>> building on one of our GPFS filesystems on a compute node which is running SL
>> 6.4. Previously, we managed to build on another node with RHEL 6.4, so we¹ll
>> be checking for differences between these two nodes.
>>
>> On SL 6.4, when building on /tmp, the build is fine, but the install fails
>> when it¹s copying
>> /tmp/easybuild/CMake/2.8.12.1/intel-2014a/cmake-2.8.12.1/Copyright.txt to its
>> destination (on the GPFS filesystem). When executing the copy ourselves, it¹s
>> working fine.
>>
>> I¹m puzzled... Do you know whether they solved the issue and how?
>>
>> P.s.: The name of our new cluster is Hopper as well, yet we doubt whether
>> this has anything to do with it ;-)
>>
>> -- Many thanks,
>>
>> Franky
>>
>>
>>
>> Op 28-okt.-2014, om 11:08 heeft Todd Gamblin het volgende
>> geschreven:
>>
>>> I've seen this error on machines where the filesystem was having issues,
>>> specifically on the home filesystem on NERSC's hopper machine.
>>>
>>> The problem was that try_compile was generating an empty testCCompiler.c --
>>> have you looked at the size of this file in your build output?
>>>
>>> The root cause of the problem was that ostream::operator<< was generating
>>> calls to fwrite with more than 1023 bytes, but for whatever reason the
>>> filesystem was failing to execute that call. Running strace on CMake showed
>>> the buggy call -- I'm attaching the C++ and CMake reproducers I made for
>>> NERSC's system. You can run the cmake one with cmake -E, and you'll need to
>>> compile th C++ one. Strace them and see if you can see the problem I'm
>>> describing. Or try building in a different filesystem and see if the
>>> problem persists.
>>>
>>> You may actually have a different problem, but the output you're reporting
>>> looks awfully familiar to me. I don't think it's easy build's fault.
>>>
>>> -Todd
>>>
>>>
>>>
>>> Here's a more detailed description of the problem that I sent to the PETSc
>>> developers a few weeks ago:
>>>
>>> Hi guys,
>>>
>>> After playing around with this, the problem is the C++ implementation on
>>> hopper. Something is screwy with fstream it can¹t write chunks larger
>>> than 1023 bytes. Attached are two reproducers for your support request.
>>> One reproduces the problem in CMake; one reproduces it in C++.
>>>
>>> If you dig in the CMakeFiles directory, you¹ll see that the C file used to
>>> do the compiler identification is actually 0 bytes, which is what gets you
>>> the undefined reference to main in your error log, causing the compiler
>>> test to fail:
>>>
>>>
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
ring, it fails too. Take one character off the string and it works. So, something is botched with Hopper’s C++ libs, or maybe with writev. I imagine that more than CMake is broken, at least on the front-end nodes. I don’t know of many programs that write large chunks to ofstreams, so maybe all is not lost. Still makes me suspicious of the hopper machine. -Todd From: Backeljauw Franky mailto:franky.backelj...@uantwerpen.be>> Reply-To: "easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be>" mailto:easybuild@lists.ugent.be>> List-Post: easybuild@lists.ugent.be Date: Tuesday, October 28, 2014 at 2:56 AM To: "easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be>" mailto:easybuild@lists.ugent.be>> Subject: [easybuild] CMake-3.3.0-intel-2014b : problem building Hello all, We have great difficulty in installing CMake on RHEL 6.4. We have tried both with CMake-2.8.12.1-intel-2014a.eb and CMake-3.3.0-intel-2014b.eb which is included in EasyBuild 1.15.2. The same problem occurs with the foss-toolchains as well as with (e.g.) CMake-2.8.12.1-GCC-4.8.2.eb. We get the following errors: -- The C compiler identification is unknown CMake Error at Modules/CMakeDetermineCCompiler.cmake:170 (configure_file): configure_file Problem configuring file Call Stack (most recent call first): CMakeLists.txt:16 (project) -- The CXX compiler identification is unknown CMake Error at Modules/CMakeDetermineCXXCompiler.cmake:168 (configure_file): configure_file Problem configuring file Call Stack (most recent call first): CMakeLists.txt:16 (project) -- Check for working C compiler: /apps/antwerpen/ivybridge/sl6/icc/2013.5.192-GCC-4.8.3/bin/intel64/icc CMake Error at Modules/CMakeTestCCompiler.cmake:47 (try_compile): Unknown extension ".c" for file /apps/antwerpen/easybuild/build/CMake/3.0.0/intel-2014b/cmake-3.0.0/CMakeFiles/CMakeTmp/testCCompiler.c try_compile() works only for enabled languages. Currently these are: C CXX I have included the full log for the CMake-3.0.0 build (cmake.log). I hope someone can help us out here. -- Many thanks for your reply, Franky Backeljauw
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
Hello Todd,
You’re spot on! Here the file testC(XX)Compiler.c was empty indeed. We are
building on one of our GPFS filesystems on a compute node which is running SL
6.4. Previously, we managed to build on another node with RHEL 6.4, so we’ll be
checking for differences between these two nodes.
On SL 6.4, when building on /tmp, the build is fine, but the install fails when
it’s copying
/tmp/easybuild/CMake/2.8.12.1/intel-2014a/cmake-2.8.12.1/Copyright.txt to its
destination (on the GPFS filesystem). When executing the copy ourselves, it’s
working fine.
I’m puzzled... Do you know whether they solved the issue and how?
P.s.: The name of our new cluster is Hopper as well, yet we doubt whether this
has anything to do with it ;-)
-- Many thanks,
Franky
Op 28-okt.-2014, om 11:08 heeft Todd Gamblin
mailto:tgamb...@llnl.gov>> het volgende geschreven:
I've seen this error on machines where the filesystem was having issues,
specifically on the home filesystem on NERSC's hopper machine.
The problem was that try_compile was generating an empty testCCompiler.c --
have you looked at the size of this file in your build output?
The root cause of the problem was that ostream::operator<< was generating calls
to fwrite with more than 1023 bytes, but for whatever reason the filesystem was
failing to execute that call. Running strace on CMake showed the buggy call --
I'm attaching the C++ and CMake reproducers I made for NERSC's system. You can
run the cmake one with cmake -E, and you'll need to compile th C++ one. Strace
them and see if you can see the problem I'm describing. Or try building in a
different filesystem and see if the problem persists.
You may actually have a different problem, but the output you're reporting
looks awfully familiar to me. I don't think it's easy build's fault.
-Todd
Here's a more detailed description of the problem that I sent to the PETSc
developers a few weeks ago:
Hi guys,
After playing around with this, the problem is the C++ implementation on
hopper. Something is screwy with fstream — it can’t write chunks larger
than 1023 bytes. Attached are two reproducers for your support request.
One reproduces the problem in CMake; one reproduces it in C++.
If you dig in the CMakeFiles directory, you’ll see that the C file used to
do the compiler identification is actually 0 bytes, which is what gets you
the undefined reference to main in your error log, causing the compiler
test to fail:
$ l CMakeFiles/2.8.11.1/CompilerIdC/CMakeCCompilerId.c
-rw-r--r-- 1 tgamblin tgamblin 0 Sep 30 21:34
CMakeFiles/2.8.11.1/CompilerIdC/CMakeCCompilerId.c
I figured something was wrong with the CMake after the OS upgrade Mark
mentioned, and I tried to build a fresh one, but that gave the same error
*in the bootstrap script* — I couldn’t even build cmake.
If you dig around to the spot in CMake where it generates the files to use
for compiler testing, you find that CMake reads a file into a variable,
filters it, and writes it out. Printing the variable using message() pre-
and post-filtering works fine. But writing the variable still gets you a
0 byte file.
Making a simple reproducer that writes a small string succeeds. If,
however, you make a string > 1024 bytes, you get zero-byte output. Test
that by running my file:
cmake -P test.cmake
If you strace that, you notice this:
$ strace cmake -P test.cmake
[ … snip … ]
writev(3, [{NULL, 0}, {"01234567890123456789012345678901"..., 1024}], 2)
= -1 EINVAL (Invalid argument)
Smaller messages use a write() and not a writev(), so they succeed. But
that’s not cmake’s fault. ofstream does that. If you run the attached
C++ program, which uses ofstream to write a 1024-byte string, it fails
too. Take one character off the string and it works.
So, something is botched with Hopper’s C++ libs, or maybe with writev. I
imagine that more than CMake is broken, at least on the front-end nodes.
I don’t know of many programs that write large chunks to ofstreams, so
maybe all is not lost.
Still makes me suspicious of the hopper machine.
-Todd
From: Backeljauw Franky
mailto:franky.backelj...@uantwerpen.be>>
Reply-To: "easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be>"
mailto:easybuild@lists.ugent.be>>
List-Post: easybuild@lists.ugent.be
Date: Tuesday, October 28, 2014 at 2:56 AM
To: "easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be>"
mailto:easybuild@lists.ugent.be>>
Subject: [easybuild] CMake-3.3.0-intel-2014b : problem building
Hello all,
We have great difficulty in installing CMake on RHEL 6.4. We have tried both
with CMake-2.8.12.1-intel-2014a.eb and CMake-3.3.0-intel-2014b.eb which is
included in EasyBuild 1.15.2. The same problem occurs with the foss-toolchains
as well as with (e.g.) CMake-2.8.12.1-GCC-4.8.2.eb.
We get the following errors:
-- The C compiler identificati
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
You are right but that would also depend if in the makefile CC and CXX are being used, isn't it? In this specific example of freebayes the build system is a little bit weird. It includes different git submodules and make is calling cmake to build dependencies so I didn't want to spend too much time with itthe workaround worked at the first try for me so that was enough for my needs -- Pablo Escobar López HPC systems engineer Biozentrum, University of Basel Swiss Institute of Bioinformatics SIB Email: pablo.escobarlo...@unibas.ch Phone: +41 61 267 21 80 http://www.biozentrum.unibas.ch 2014-10-28 12:06 GMT+01:00 Ward Poelmans : > On Tue, Oct 28, 2014 at 12:04 PM, Pablo Escobar Lopez > wrote: > > I verified it when I tried the workaround and it seems to work fine, at > > least in this case. > > That's because you are using a gcc toolchain and it is in the path. I > suspect this would fall back to system gcc with an ictce toolchain. > > Ward > -- Pablo Escobar López HPC systems engineer Biozentrum, University of Basel Swiss Institute of Bioinformatics SIB Email: pablo.escobarlo...@unibas.ch Phone: +41 61 267 21 80 http://www.biozentrum.unibas.ch
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
On Tue, Oct 28, 2014 at 12:04 PM, Pablo Escobar Lopez wrote: > I verified it when I tried the workaround and it seems to work fine, at > least in this case. That's because you are using a gcc toolchain and it is in the path. I suspect this would fall back to system gcc with an ictce toolchain. Ward
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
I verified it when I tried the workaround and it seems to work fine, at least in this case. Here is my build log == 2014-10-27 10:36:55,249 main.run INFO cmd "unset CC && unset CXX && make -j 1 " exited with exitcode 0 and output: cat: src/version_git.h: No such file or directory wget -q http://hypervolu.me/freebayes/build/ & cd src && make make[1]: Entering directory `/tmp/easybuild/freebayes/0.9.18-1-g4233a23-dirty/goolf-1.4.10/freebayes/src' DETECTED_VERSION = v0.9.18-1-g4233a23-dirty CURRENT_VERSION = Updating version file. cd ../bamtools && mkdir -p build && cd build && cmake .. && make -- The C compiler identification is GNU 4.7.2 -- The CXX compiler identification is GNU 4.7.2 -- Check for working C compiler: /import/bc2/apps/GCC/4.7.2/bin/cc -- Check for working C compiler: /import/bc2/apps/GCC/4.7.2/bin/cc -- works By the way, I just sent a pull request for this easyconfig in case anyone want to look at it https://github.com/hpcugent/easybuild-easyconfigs/pull/1165 2014-10-28 11:54 GMT+01:00 Ward Poelmans : > On Tue, Oct 28, 2014 at 11:10 AM, Pablo Escobar > wrote: > > > recently I wrote a FreeBayes easyconfig and I had to add this to the > > easyconfig to avoid that error > > > > # workaround to avoid the error "The C compiler identification is > unknown" > > prebuildopts = "unset CC && unset CXX && " > > Don't you fall back to system gcc than? > > Ward > -- Pablo Escobar López HPC systems engineer Biozentrum, University of Basel Swiss Institute of Bioinformatics SIB Email: pablo.escobarlo...@unibas.ch Phone: +41 61 267 21 80 http://www.biozentrum.unibas.ch
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
On Tue, Oct 28, 2014 at 11:10 AM, Pablo Escobar wrote: > recently I wrote a FreeBayes easyconfig and I had to add this to the > easyconfig to avoid that error > > # workaround to avoid the error "The C compiler identification is unknown" > prebuildopts = "unset CC && unset CXX && " Don't you fall back to system gcc than? Ward
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
Hi Franky, recently I wrote a FreeBayes easyconfig and I had to add this to the easyconfig to avoid that error # workaround to avoid the error "The C compiler identification is unknown" prebuildopts = "unset CC && unset CXX && " maybe you can give it a try Pablo. 2014-10-28 10:56 GMT+01:00 Backeljauw Franky < franky.backelj...@uantwerpen.be>: > Hello all, > > We have great difficulty in installing CMake on RHEL 6.4. We have tried > both with CMake-2.8.12.1-intel-2014a.eb and CMake-3.3.0-intel-2014b.eb > which is included in EasyBuild 1.15.2. The same problem occurs with the > foss-toolchains as well as with (e.g.) CMake-2.8.12.1-GCC-4.8.2.eb. > > We get the following errors: > > -- The C compiler identification is unknown > CMake Error at Modules/CMakeDetermineCCompiler.cmake:170 (configure_file): > configure_file Problem configuring file > Call Stack (most recent call first): > CMakeLists.txt:16 (project) > > > -- The CXX compiler identification is unknown > CMake Error at Modules/CMakeDetermineCXXCompiler.cmake:168 > (configure_file): > configure_file Problem configuring file > Call Stack (most recent call first): > CMakeLists.txt:16 (project) > > > -- Check for working C compiler: > /apps/antwerpen/ivybridge/sl6/icc/2013.5.192-GCC-4.8.3/bin/intel64/icc > CMake Error at Modules/CMakeTestCCompiler.cmake:47 (try_compile): > Unknown extension ".c" for file > > > /apps/antwerpen/easybuild/build/CMake/3.0.0/intel-2014b/cmake-3.0.0/CMakeFiles/CMakeTmp/testCCompiler.c > >try_compile() works only for enabled languages. Currently these are: > > C CXX > > I have included the full log for the CMake-3.0.0 build (cmake.log). > > I hope someone can help us out here. > > -- Many thanks for your reply, > > Franky Backeljauw >
Re: [easybuild] CMake-3.3.0-intel-2014b : problem building
I've seen this error on machines where the filesystem was having issues,
specifically on the home filesystem on NERSC's hopper machine.
The problem was that try_compile was generating an empty testCCompiler.c --
have you looked at the size of this file in your build output?
The root cause of the problem was that ostream::operator<< was generating
calls to fwrite with more than 1023 bytes, but for whatever reason the
filesystem was failing to execute that call. Running strace on CMake showed
the buggy call -- I'm attaching the C++ and CMake reproducers I made for
NERSC's system. You can run the cmake one with cmake -E, and you'll need to
compile th C++ one. Strace them and see if you can see the problem I'm
describing. Or try building in a different filesystem and see if the
problem persists.
You may actually have a different problem, but the output you're reporting
looks awfully familiar to me. I don't think it's easy build's fault.
-Todd
Here's a more detailed description of the problem that I sent to the PETSc
developers a few weeks ago:
Hi guys,
After playing around with this, the problem is the C++ implementation on
hopper. Something is screwy with fstream it can¹t write chunks larger
than 1023 bytes. Attached are two reproducers for your support request.
One reproduces the problem in CMake; one reproduces it in C++.
If you dig in the CMakeFiles directory, you¹ll see that the C file used to
do the compiler identification is actually 0 bytes, which is what gets you
the undefined reference to main in your error log, causing the compiler
test to fail:
$ l CMakeFiles/2.8.11.1/CompilerIdC/CMakeCCompilerId.c
-rw-r--r-- 1 tgamblin tgamblin 0 Sep 30 21:34
CMakeFiles/2.8.11.1/CompilerIdC/CMakeCCompilerId.c
I figured something was wrong with the CMake after the OS upgrade Mark
mentioned, and I tried to build a fresh one, but that gave the same error
*in the bootstrap script* I couldn¹t even build cmake.
If you dig around to the spot in CMake where it generates the files to use
for compiler testing, you find that CMake reads a file into a variable,
filters it, and writes it out. Printing the variable using message() pre-
and post-filtering works fine. But writing the variable still gets you a
0 byte file.
Making a simple reproducer that writes a small string succeeds. If,
however, you make a string > 1024 bytes, you get zero-byte output. Test
that by running my file:
cmake -P test.cmake
If you strace that, you notice this:
$ strace cmake -P test.cmake
[ snip ]
writev(3, [{NULL, 0}, {"01234567890123456789012345678901"..., 1024}], 2)
= -1 EINVAL (Invalid argument)
Smaller messages use a write() and not a writev(), so they succeed. But
that¹s not cmake¹s fault. ofstream does that. If you run the attached
C++ program, which uses ofstream to write a 1024-byte string, it fails
too. Take one character off the string and it works.
So, something is botched with Hopper¹s C++ libs, or maybe with writev. I
imagine that more than CMake is broken, at least on the front-end nodes.
I don¹t know of many programs that write large chunks to ofstreams, so
maybe all is not lost.
Still makes me suspicious of the hopper machine.
-Todd
From: Backeljauw Franky
Reply-To: "easybuild@lists.ugent.be"
List-Post: easybuild@lists.ugent.be
Date: Tuesday, October 28, 2014 at 2:56 AM
To: "easybuild@lists.ugent.be"
Subject: [easybuild] CMake-3.3.0-intel-2014b : problem building
> Hello all,
>
> We have great difficulty in installing CMake on RHEL 6.4. We have tried both
> with CMake-2.8.12.1-intel-2014a.eb and CMake-3.3.0-intel-2014b.eb which is
> included in EasyBuild 1.15.2. The same problem occurs with the foss-toolchains
> as well as with (e.g.) CMake-2.8.12.1-GCC-4.8.2.eb.
>
> We get the following errors:
>
>> -- The C compiler identification is unknown
>> CMake Error at Modules/CMakeDetermineCCompiler.cmake:170 (configure_file):
>> configure_file Problem configuring file
>> Call Stack (most recent call first):
>> CMakeLists.txt:16 (project)
>>
>>
>> -- The CXX compiler identification is unknown
>> CMake Error at Modules/CMakeDetermineCXXCompiler.cmake:168 (configure_file):
>> configure_file Problem configuring file
>> Call Stack (most recent call first):
>> CMakeLists.txt:16 (project)
>>
>>
>> -- Check for working C compiler:
>> /apps/antwerpen/ivybridge/sl6/icc/2013.5.192-GCC-4.8.3/bin/intel64/icc
>> CMake Error at Modules/CMakeTestCCompiler.cmake:47 (try_compile):
>> Unknown extension ".c" for file
>>
>>
>> /apps/antwerpen/easybuild/build/CMake/3.0.0/intel-2014b/cmake-3.0.0/CMakeFile
>> s/CMakeTmp/testCCompiler.c
>>
>> try_compile() works only for enabled languages. Currently these are:
>>
>> C CXX
>>
> I have included the full log for the CMake-3.0.0 build (cmake.log).
>
> I hope someone can help us out here.
>
> -- Many thanks for your reply,
>
> Franky Backeljauw
test.cmake
Description: Binary data
test.C
Description: Binary data
[easybuild] CMake-3.3.0-intel-2014b : problem building
Hello all, We have great difficulty in installing CMake on RHEL 6.4. We have tried both with CMake-2.8.12.1-intel-2014a.eb and CMake-3.3.0-intel-2014b.eb which is included in EasyBuild 1.15.2. The same problem occurs with the foss-toolchains as well as with (e.g.) CMake-2.8.12.1-GCC-4.8.2.eb. We get the following errors: -- The C compiler identification is unknown CMake Error at Modules/CMakeDetermineCCompiler.cmake:170 (configure_file): configure_file Problem configuring file Call Stack (most recent call first): CMakeLists.txt:16 (project) -- The CXX compiler identification is unknown CMake Error at Modules/CMakeDetermineCXXCompiler.cmake:168 (configure_file): configure_file Problem configuring file Call Stack (most recent call first): CMakeLists.txt:16 (project) -- Check for working C compiler: /apps/antwerpen/ivybridge/sl6/icc/2013.5.192-GCC-4.8.3/bin/intel64/icc CMake Error at Modules/CMakeTestCCompiler.cmake:47 (try_compile): Unknown extension ".c" for file /apps/antwerpen/easybuild/build/CMake/3.0.0/intel-2014b/cmake-3.0.0/CMakeFiles/CMakeTmp/testCCompiler.c try_compile() works only for enabled languages. Currently these are: C CXX I have included the full log for the CMake-3.0.0 build (cmake.log). I hope someone can help us out here. -- Many thanks for your reply, Franky Backeljauw cmake.log Description: cmake.log
