Re: [Users] Problem with compiling Einstein Toolkit on Cray system
Ok, useful to know! Many thanks to everyone who's emailed me with help: I got considerably further with attempting to compile the Einstein Toolkit and whilst I haven't actually been successful yet (and have run out of time at this particular point to continue trying) I've certainly learned a lot about compiling. Next time I try I'll have a go at getting an account on Queen Bee 2. Best wishes and thanks again, Lucy On 2016-09-13 6:24 pm, Frank Loeffler wrote: > On Tue, Sep 13, 2016 at 05:02:09PM +0200, L.S. Oswald wrote: >> In fact that's what I tried originally, but I realised the commands >> only >> work if you're compiling on Queen Bee, not any other supercomputer. >> Google tells me that the Queen Bee supercomputer has been retired, so >> I >> wouldn't be able to obtain an account on it in any case. > > Google is right, but what we mean when referring to "Queen Bee" > nowadays > is "Queen Bee 2": > > http://www.hpc.lsu.edu/resources/hpc/system.php?system=QB2 > > We should be better about including the '2'. > > Frank ___ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Re: [Users] Continuation of problems downloading ET onto Dora
Ah ok, thanks for that. Following your advice definitely got me a step further with compiling. Lucy On 2016-09-13 6:26 pm, Frank Loeffler wrote: > On Tue, Sep 13, 2016 at 05:00:06PM +0200, L.S. Oswald wrote: >> I haven't been able to work out which gcc version I use, but I think >> you're >> correct that it's an old version. > > gcc --version > > should tell you the version > >> Could you advise me on how to direct the code to compile with the >> correct >> version of gcc? I attempted to do so by changing the CC flag in my >> configuration file but that didn't seem to do anything: the program >> compiled >> with gcc even when I changed the CC flag to a random word. > > That's what you usually put into the config file. > > That is, where you quoted: > > # Whenever this version string changes, the application is configured > > # and rebuilt from scratch > > VERSION = 2012-04-11 > > > > CPP = cpp > > FPP = cpp > > CC = gcc > > CXX = g++ > > #F77 = gfortran > > #F90 = gfortran > > F77 = gfortran-4.6 > > F90 = gfortran-4.6 > > If you change something there, and especially if you just start out, it > is safest to delete configs/YOURSPECIFIEDNAME before a retry. (Where > YOURSPECIFIEDNAME is 'sim' be default if you used simfactory). > > Frank ___ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Re: [Users] Problem with compiling Einstein Toolkit on Cray system
In fact that's what I tried originally, but I realised the commands only work if you're compiling on Queen Bee, not any other supercomputer. Google tells me that the Queen Bee supercomputer has been retired, so I wouldn't be able to obtain an account on it in any case. Lucy On 2016-09-12 4:33 pm, Erik Schnetter wrote: > Lucy > > Have you tried using the Einstein Toolkit on Queen Bee, as suggested > here <https://docs.einsteintoolkit.org/et-docs/Tutorial_for_New_Users > [4]>? > > -erik > > On Tue, Sep 6, 2016 at 11:56 AM, L.S. Oswald <lo...@cam.ac.uk> wrote: > >> Dear Erik >> >> In general I've been using the commands at >> > https://docs.einsteintoolkit.org/et-docs/Simplified_Tutorial_for_New_Users >> [1]. I have tried configuring with the debian.cfg file and also >> attempted creating my own configuration file. Moving onto a >> different computer solved my MPI issue but produced a new error, >> which I've written about in my reply to Rahul. I'm afraid I don't >> know what type of computer I'm using now but I should be able to >> find out tomorrow. >> >> Hopefully my other email gives an overview of my latest problem. I >> attach the full log of the output received when building the >> toolkit. >> >> Many thanks for your help, >> >> Lucy >> >> On 2016-09-06 5:41 pm, Erik Schnetter wrote: >> >> Lucy >> >> The Simulation Factory has several example option lists for Cray >> systems; look e.g. at bluewaters or edison. I don't recommend >> changing >> the "config-info" file, since this file will be overwritten. >> Instead, >> you can set such options in your option list. >> >> If you need more help, then we'll need more details: What kind of >> Cray >> system is this? What options did you pass when you configured the >> Einstein Toolkit? What output did you receive while configuring? >> etc. >> >> -erik >> >> On Mon, Aug 29, 2016 at 12:47 PM, L.S. Oswald <lo...@cam.ac.uk> >> wrote: >> >> Hello >> >> I'm attempting to install the Einstein Toolkit on Dora, which is a >> Cray >> system supercomputer. In doing so I came across the following >> error: >> >> Configuration script for thorn MPI returned exit code 5 >> Error message: 'MPI could not be configured: neither >> automatic nor >> manual configuration succeeded' >> >> In addition the log stated that: >> No MPI compiler wrapper found beneath MPI_DIR (MPI_DIR = >> /usr) >> >> I subsequently discovered that it was looking in the wrong place >> for the >> MPI root directory and that the reason it was looking under /usr >> instead >> of /opt/cray/mpt/7.3.2/gni/mpich-cray/8.3 was because that was the >> setting in the ubuntu configuration file that I had used when >> configuring SimFactory. On a reattempt therefore I did not use the >> ubuntu.cfg file when configuring SimFactory, and got the following >> error >> instead: >> >> Configuring with flesh MPI >> Warning: use of flesh MPI via MPI option is deprecated and >> should be replaced with the thorn ExternalLibraries/MPI and its >> MPI_DIR >> option >> MPI selected, but no known MPI method - what is >> "/opt/cray/mpt/7.3.2/gni/mpich-cray/8.3" ? >> >> Attempts at altering the Makefile and the config-info file resulted >> in >> either no change, or more errors. >> >> Please would you be able to advise me on the correct way to specify >> the >> root directory for MPI? >> >> Many thanks, >> >> Lucy Oswald >> ___ >> Users mailing list >> Users@einsteintoolkit.org >> http://lists.einsteintoolkit.org/mailman/listinfo/users [2] [1] >> >> -- >> >> Erik Schnetter <schnet...@cct.lsu.edu> >> http://www.perimeterinstitute.ca/personal/eschnetter/ [3] [2] >> >> Links: >> -- >> [1] http://lists.einsteintoolkit.org/mailman/listinfo/users [2] >> [2] http://www.perimeterinstitute.ca/personal/eschnetter/ [3] > > -- > > Erik Schnetter <schnet...@cct.lsu.edu> > http://www.perimeterinstitute.ca/personal/eschnetter/ [3] > > Links: > -- > [1] > https://docs.einsteintoolkit.org/et-docs/Simplified_Tutorial_for_New_Users > [2] http://lists.einsteintoolkit.org/mailman/listinfo/users > [3] http://www.perimeterinstitute.ca/personal/eschnetter/ > [4] https://docs.einsteintoolkit.org/et-docs/Tutorial_for_New_Users ___ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Re: [Users] Continuation of problems downloading ET onto Dora
I haven't been able to work out which gcc version I use, but I think you're correct that it's an old version. Could you advise me on how to direct the code to compile with the correct version of gcc? I attempted to do so by changing the CC flag in my configuration file but that didn't seem to do anything: the program compiled with gcc even when I changed the CC flag to a random word. Many thanks, Lucy On 2016-09-11 3:48 pm, Frank Loeffler wrote: > On Fri, Sep 09, 2016 at 04:58:43PM +0200, L.S. Oswald wrote: >> error: expected ‘#pragma omp’ clause before ‘collapse’ > > Which version of gcc (c compiler) do you use? Your option list mentions > only 'gcc'. Gcc older than 4.4 doens't support openMP 3.0, of which > collapse is part of. > > Since you seem to use gfortran-4.6, chances are, gcc-4.6 is installed > too. > > Frank ___ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
[Users] Continuation of problems downloading ET onto Dora
Hello again In previous emails I detailed the problems I was having downloading the Einstein Toolkit, first on the Dora supercomputer and then on a local computer. Following further efforts to solve my MPI issue on the Dora supercomputer I have come up against a new problem. This is the method I used to reach this point: I downloaded Cactus using the method described in https://docs.einsteintoolkit.org/et-docs/Simplified_Tutorial_for_New_Users I created my own configuration file based on the debian.cfg file which I have attached. This file specifies the path for the MPI directory (and this solves the MPI issue) I configured Simfactory using the following command: ./simfactory/bin/sim setup --optionlist=dora.cfg --runscript debian.sh I built the Einstein Toolkit with the following command: ./simfactory/bin/sim build --mdbkey make 'make -j2' --thornlist=manifest/einsteintoolkit.th This is the error I got. It seems to be some kind of syntax error: /users/lsoswald/old/Cactus/arrangements/CactusTest/TestLoop/src/TestLoopPointwise.c: In function ‘TestLoopPointwise_All’: /users/lsoswald/old/Cactus/arrangements/CactusTest/TestLoop/src/TestLoopPointwise.c:31: error: expected ‘#pragma omp’ clause before ‘collapse’ Creating /users/lsoswald/old/Cactus/configs/sim/lib/libthorn_TestLocalInterp2.a /users/lsoswald/old/Cactus/arrangements/CactusTest/TestLoop/src/TestLoopPointwise.c: In function ‘TestLoopPointwise_Int’: /users/lsoswald/old/Cactus/arrangements/CactusTest/TestLoop/src/TestLoopPointwise.c:90: error: expected ‘#pragma omp’ clause before ‘collapse’ /users/lsoswald/old/Cactus/arrangements/CactusTest/TestLoop/src/TestLoopPointwise.c: In function ‘TestLoopPointwise_Bnd’: /users/lsoswald/old/Cactus/arrangements/CactusTest/TestLoop/src/TestLoopPointwise.c:158: error: expected ‘#pragma omp’ clause before ‘collapse’ /users/lsoswald/old/Cactus/arrangements/CactusTest/TestLoop/src/TestLoopPointwise.c: In function ‘TestLoopPointwise_IntBnd’: /users/lsoswald/old/Cactus/arrangements/CactusTest/TestLoop/src/TestLoopPointwise.c:231: error: expected ‘#pragma omp’ clause before ‘collapse’ make[3]: *** [TestLoopPointwise.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: *** [/users/lsoswald/old/Cactus/configs/sim/lib/libthorn_TestLoop.a] Error 2 make[1]: *** Waiting for unfinished jobs make: *** [sim] Error 2 Could you advise me on what the issue with this might be and how I may solve it? Many thanks, Lucy# Configuration for a Debian installation, assuming the following list of # packages is installed: # # perl gfortran g++ libopenmpi-dev openmpi-bin make # # At the moment, an installed libmpich2 might cause problems if it is # set to superseed the openmpi installation # use 'ls -l /usr/lib/libmpi.so' and maybe 'ls -l /etc/alternatives/libmpi.so' # to find out what is actually used. # # In addition, installing the following list of packaed will prevent Cactus # from compiling it's own versions of these libraries: # # libfftw3-dev libgsl0-dev libatlas-base-dev libjpeg-dev libssl-dev libhdf5-serial-dev hdf5-tools # # Tools like GetComponents and Simfactory like to have the following list # installed too # # python subversion git # # For opencl-support, you have to install additional packages, depending on # your hardware. Here is an example using an intel/adm CPU and an nvidia GPU: # # ocl-icd-libopencl1 ocl-icd-opencl-dev # amd-opencl-icd # nvidia-opencl-common nvidia-opencl-icd # Whenever this version string changes, the application is configured # and rebuilt from scratch VERSION = 2012-04-11 CPP = cpp FPP = cpp CC = gcc CXX = g++ #F77 = gfortran #F90 = gfortran F77 = gfortran-4.6 F90 = gfortran-4.6 FPPFLAGS = -traditional CPPFLAGS = FPPFLAGS = CFLAGS = -g -std=gnu99 CXXFLAGS = -g -std=gnu++0x F77FLAGS = -g -fcray-pointer -m128bit-long-double -ffixed-line-length-none F90FLAGS = -g -fcray-pointer -m128bit-long-double -ffixed-line-length-none LDFLAGS = -rdynamic DEBUG = no CPP_DEBUG_FLAGS = FPP_DEBUG_FLAGS = C_DEBUG_FLAGS = -O0 CXX_DEBUG_FLAGS = -O0 F77_DEBUG_FLAGS = -O0 F90_DEBUG_FLAGS = -O0 OPTIMISE = yes CPP_OPTIMISE_FLAGS = FPP_OPTIMISE_FLAGS = C_OPTIMISE_FLAGS = -O2 CXX_OPTIMISE_FLAGS = -O2 F77_OPTIMISE_FLAGS = -O2 F90_OPTIMISE_FLAGS = -O2 PROFILE = no CPP_PROFILE_FLAGS = FPP_PROFILE_FLAGS = C_PROFILE_FLAGS = -pg CXX_PROFILE_FLAGS = -pg F77_PROFILE_FLAGS = -pg F90_PROFILE_FLAGS = -pg OPENMP = no CPP_OPENMP_FLAGS = -fopenmp FPP_OPENMP_FLAGS = -D_OPENMP C_OPENMP_FLAGS = -fopenmp CXX_OPENMP_FLAGS = -fopenmp F77_OPENMP_FLAGS = -fopenmp F90_OPENMP_FLAGS = -fopenmp WARN = yes CPP_WARN_FLAGS = -Wall FPP_WARN_FLAGS = -Wall C_WARN_FLAGS = -Wall CXX_WARN_FLAGS = -Wall F77_WARN_FLAGS = -Wall F90_WARN_FLAGS = -Wall PTHREADS_DIR = NO_BUILD MPI_DIR = /opt/cray/mpt/7.3.2/gni/mpich-cray/8.3 BLAS_DIR = NO_BUILD BLAS_LIBS = -mkl BOOST_DIR = BUILD GSL_DIR = /apps/GSL/1.1/ HDF5_DIR =
Re: [Users] Problem with compiling Einstein Toolkit on Cray system
Dear Erik In general I've been using the commands at https://docs.einsteintoolkit.org/et-docs/Simplified_Tutorial_for_New_Users. I have tried configuring with the debian.cfg file and also attempted creating my own configuration file. Moving onto a different computer solved my MPI issue but produced a new error, which I've written about in my reply to Rahul. I'm afraid I don't know what type of computer I'm using now but I should be able to find out tomorrow. Hopefully my other email gives an overview of my latest problem. I attach the full log of the output received when building the toolkit. Many thanks for your help, Lucy On 2016-09-06 5:41 pm, Erik Schnetter wrote: Lucy The Simulation Factory has several example option lists for Cray systems; look e.g. at bluewaters or edison. I don't recommend changing the "config-info" file, since this file will be overwritten. Instead, you can set such options in your option list. If you need more help, then we'll need more details: What kind of Cray system is this? What options did you pass when you configured the Einstein Toolkit? What output did you receive while configuring? etc. -erik On Mon, Aug 29, 2016 at 12:47 PM, L.S. Oswald <lo...@cam.ac.uk> wrote: Hello I'm attempting to install the Einstein Toolkit on Dora, which is a Cray system supercomputer. In doing so I came across the following error: Configuration script for thorn MPI returned exit code 5 Error message: 'MPI could not be configured: neither automatic nor manual configuration succeeded' In addition the log stated that: No MPI compiler wrapper found beneath MPI_DIR (MPI_DIR = /usr) I subsequently discovered that it was looking in the wrong place for the MPI root directory and that the reason it was looking under /usr instead of /opt/cray/mpt/7.3.2/gni/mpich-cray/8.3 was because that was the setting in the ubuntu configuration file that I had used when configuring SimFactory. On a reattempt therefore I did not use the ubuntu.cfg file when configuring SimFactory, and got the following error instead: Configuring with flesh MPI Warning: use of flesh MPI via MPI option is deprecated and should be replaced with the thorn ExternalLibraries/MPI and its MPI_DIR option MPI selected, but no known MPI method - what is "/opt/cray/mpt/7.3.2/gni/mpich-cray/8.3" ? Attempts at altering the Makefile and the config-info file resulted in either no change, or more errors. Please would you be able to advise me on the correct way to specify the root directory for MPI? Many thanks, Lucy Oswald ___ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users [1] -- Erik Schnetter <schnet...@cct.lsu.edu> http://www.perimeterinstitute.ca/personal/eschnetter/ [2] Links: -- [1] http://lists.einsteintoolkit.org/mailman/listinfo/users [2] http://www.perimeterinstitute.ca/personal/eschnetter/Using configuration: sim Updated thorn list for configuration sim Building sim Cactus - version: 4.3.0 Building configuration sim Reconfiguring thorns Reading ThornList... Parsing configuration files... Parsing interface files... ADMAnalysis Implements: ADMAnalysis Inherits: ADMBase StaticConformal Grid ADMMacros 6 Public, 0 Protected, 0 Private variable groups ADMBase Implements: ADMBase Inherits: grid 9 Public, 0 Protected, 0 Private variable groups ADMCoupling Implements: ADMCoupling 0 Public, 0 Protected, 0 Private variable groups ADMMacros Implements: ADMMacros 1 Public, 0 Protected, 0 Private variable groups ADMMass Implements: ADMMass Inherits: ADMBase ADMMacros StaticConformal SpaceMask 0 Public, 0 Protected, 6 Private variable groups AEILocalInterp Implements: AEILocalInterp 0 Public, 0 Protected, 0 Private variable groups AHFinder Implements: AHFinder Inherits: ADMBase StaticConformal SpaceMask Grid IO ADMMacros 6 Public, 0 Protected, 10 Private variable groups AHFinderDirect Implements: AHFinderDirect Inherits: Grid ADMBase StaticConformal SpaceMask SphericalSurface IO 0 Public, 0 Protected, 5 Private variable groups BLAS Implements: BLAS 0 Public, 0 Protected, 0 Private variable groups Boundary Implements: boundary 0 Public, 0 Protected, 0 Private variable groups CT_Analytic Implements: CT_Analytic Inherits: Boundary GenericFD Grid 40 Public, 0 Protected, 0 Private variable groups CT_MultiLevel Implements: CT_MultiLevel Inherits: Boundary Grid 0 Public, 0 Protected, 9 Private variable groups Cactus Implements: Cactus 0 Public, 0 Protected, 0 Private variable group
Re: [Users] Problem with compiling Einstein Toolkit on Cray system
Hi Rahul Thanks very much for your help. I hadn't tried that. I think I managed to partially solve the MPI issue by creating a new .cfg file specifying the path, but I have since come up against various other errors preventing me from compiling. I just tried including your line in the config-info file but no luck so far. Consequently I've been trying to install the Toolkit on a different computer, and this one has gone much farther. It seems to compile everything and then stop at the end, so that it doesn't create the executable. This is the error I get: /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: /home/ara/loswald/Cactus/configs/sim/lib/libthorn_SystemTopology.a(system_topology.cc.o): undefined reference to symbol 'opal_hwloc172_hwloc_topology_get_support' /usr/lib64/mpi/gcc/openmpi/lib64/libopen-pal.so.6: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status /home/ara/loswald/Cactus/lib/make/make.configuration:149: recipe for target '/home/ara/loswald/Cactus/exe/cactus_sim' failed I'm unclear on what hwloc actually is: is it a library in itself or should it be part of open-pal or MPI? Only one configuration file references hwloc specifically so we created a new configuration file which included the following lines: # OpenMPI contains a copy of hwloc, use it HWLOC_DIR = NO_BUILD HWLOC_INC_DIRS = /usr/lib64/mpi/gcc/openmpi/include/openmpi/opal/mca/hwloc /usr/lib64/mpi/gcc/openmpi/include/openmpi/o pal/mca/hwloc/hwloc172 /usr/lib64/mpi/gcc/openmpi/include/openmpi/opal/mca/hwloc/hwloc172/hwloc/include /usr/lib64/mpi/ gcc/openmpi/include/openmpi HWLOC_LIBS = -llibopen-pal but this had no effect and I got the same error. I don't suppose you could enlighten me on how I might go about adding the DSO to the command line, and indeed, what DSO I should be adding? Many thanks, Lucy On 2016-09-06 4:27 pm, rahul kashyap wrote: > Hi Lucy, > > Did you try including this line in /configs/my_config/config-info > > MPI_DIR=path/to/MPI/bin > > then do make my_config > > Best, > rahul > > On Mon, Aug 29, 2016 at 12:47 PM, L.S. Oswald <lo...@cam.ac.uk> wrote: > >> Hello >> >> I'm attempting to install the Einstein Toolkit on Dora, which is a >> Cray >> system supercomputer. In doing so I came across the following >> error: >> >> Configuration script for thorn MPI returned exit code 5 >> Error message: 'MPI could not be configured: neither >> automatic nor >> manual configuration succeeded' >> >> In addition the log stated that: >> No MPI compiler wrapper found beneath MPI_DIR (MPI_DIR = >> /usr) >> >> I subsequently discovered that it was looking in the wrong place >> for the >> MPI root directory and that the reason it was looking under /usr >> instead >> of /opt/cray/mpt/7.3.2/gni/mpich-cray/8.3 was because that was the >> setting in the ubuntu configuration file that I had used when >> configuring SimFactory. On a reattempt therefore I did not use the >> ubuntu.cfg file when configuring SimFactory, and got the following >> error >> instead: >> >> Configuring with flesh MPI >> Warning: use of flesh MPI via MPI option is deprecated and >> should be replaced with the thorn ExternalLibraries/MPI and its >> MPI_DIR >> option >> MPI selected, but no known MPI method - what is >> "/opt/cray/mpt/7.3.2/gni/mpich-cray/8.3" ? >> >> Attempts at altering the Makefile and the config-info file resulted >> in >> either no change, or more errors. >> >> Please would you be able to advise me on the correct way to specify >> the >> root directory for MPI? >> >> Many thanks, >> >> Lucy Oswald >> ___ >> Users mailing list >> Users@einsteintoolkit.org >> http://lists.einsteintoolkit.org/mailman/listinfo/users [1] > > > > Links: > -- > [1] http://lists.einsteintoolkit.org/mailman/listinfo/users ___ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
