Dear Jenny, There was a mistake in the list of my instructions, the last 2 sed commands in the MUMPS compilation should be
sed -i '/^\(OPTF.*=.*\)-O\(\>.*\)/s//\1-fallow-argument-mismatch -O3 -fPIC\2/' Makefile.inc sed -i '/^\(OPT[CL].*=.*\)-O\(\>.*\)/s//\1-O3 -fPIC\2/' Makefile.inc Which means, you need to add the -fPIC option in the compilation of MUMPS. An updated version of the entire recipe is attached. I also attach the editted Makefile.inc file for the compilation of MUMPS. Best regards Kostas On Wed, Dec 8, 2021 at 1:32 AM Chen,Jinzhen <jche...@mdanderson.org> wrote: > Dear. Dr. Poulios, > > > > I investigated the issue and found that libsmumps-5.3.so is a system > level mumps library that I installed from the rpm packages before. The > compiler used system level mumps libraries rather than what I built > mumps/5.4.1 in the previous built somehow. Hence, I rebuilt mumps/5.4.1 > from a node where there are no mumps or metis rpm packages installed. Then > I changed the getfem configuration to set LIBS having the same value as > LDFLAGS. > > > > ./configure --prefix=/risapps/rhel7/getfem-mpi/5.4.1-new \ > > LDFLAGS="-L/risapps/rhel7/gcc/9.3.0/lib64 > -L/risapps/rhel7/openmpi/4.1.1/lib -L/risapps/rhel7/mumps/5.4.1/lib > -L/risapps/rhel7/parmetis/4.0.3/lib -L/risapps/rhel7/metis/5.1.0/lib > -L/risapps/rhel7/scalapack/2.1.0/lib -L/risapps/rhel7/scotch/6.1.1/lib > -L/risapps/rhel7/qhull/2020.2/lib" \ > > CPPFLAGS="-I/risapps/rhel7/gcc/9.3.0/include > -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/mumps/5.4.1/include > -I/risapps/rhel7/parmetis/4.0.3/include > -I/risapps/rhel7/metis/5.1.0/include -I/risapps/rhel7/qhull/2020.2/include" > \ > > LIBS="-L/risapps/rhel7/gcc/9.3.0/lib64 -L/risapps/rhel7/openmpi/4.1.1/lib > -L/risapps/rhel7/mumps/5.4.1/lib -L/risapps/rhel7/parmetis/4.0.3/lib > -L/risapps/rhel7/metis/5.1.0/lib -L/risapps/rhel7/scalapack/2.1.0/lib > -L/risapps/rhel7/scotch/6.1.1/lib -L/risapps/rhel7/qhull/2020.2/lib" \ > > --with-blas="/risapps/rhel7/openblas/0.3.18/lib/libopenblas.a" \ > > --with-pic --with-optimization=-O3 --disable-metlab --enable-python > --enable-paralevel=2 --enable-shared > > > > With these changes, getfem can’t be built any more. Here is the last part > of compiling message. Please let me know if you need more information. > > > > ….. > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_matvec.o): > relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making > a shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_root_parallel.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zana_aux.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zana_aux_par.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_par_m.o): > relocation R_X86_64_32S against `.rodata.str1.1' can not be used when > making a shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zmumps_sol_es.o): > relocation R_X86_64_32S against `.rodata.str1.1' can not be used when > making a shared object; recompile with -fPIC > > /bin/ld: > /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zana_LDLT_preprocess.o): > relocation R_X86_64_32S against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_asm_ELT.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_lastrtnelind.o): > relocation R_X86_64_32S against `.rodata.str1.1' can not be used when > making a shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_mem_stack.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_root_parallel.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: > /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_type3_symmetrize.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zana_mtrans.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_bwd.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_fwd_aux.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: > /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_asm_master_ELT_m.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_asm_master_m.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: > /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_LU_type1.o): > relocation R_X86_64_32S against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: > /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_LU_type2.o): > relocation R_X86_64_32S against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: > /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_LDLT_type1.o): > relocation R_X86_64_32S against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: > /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_LDLT_type2.o): > relocation R_X86_64_32S against `.rodata.str1.1' can not be used when > making a shared object; recompile with -fPIC > > /bin/ld: > /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_type2_aux.o): > relocation R_X86_64_32 against `.rodata' can not be used when making a > shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_lr.o): relocation > R_X86_64_32S against `.rodata' can not be used when making a shared object; > recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_bwd_aux.o): > relocation R_X86_64_32S against `.rodata.str1.1' can not be used when > making a shared object; recompile with -fPIC > > /bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libdmumps.a(dmumps_c.o): > relocation R_X86_64_PC32 against symbol `strlen@@GLIBC_2.2.5' can not be > used when making a shared object; recompile with -fPIC > > /bin/ld: final link failed: Bad value > > collect2: error: ld returned 1 exit status > > make[2]: *** [libgetfem.la] Error 1 > > make[2]: Leaving directory `/risapps/build7/getfem-5.4.1/src' > > make[1]: *** [all-recursive] Error 1 > > make[1]: Leaving directory `/risapps/build7/getfem-5.4.1' > > make: *** [all] Error 2 > > > > Best Regards > > Jenny > > > > *From: *"Chen,Jinzhen" <jche...@mdanderson.org> > *Date: *Tuesday, December 7, 2021 at 11:34 AM > *To: *"getfem-users@nongnu.org" <getfem-users@nongnu.org> > *Subject: *Re: [EXT] Re: getfem installation > > > > Dear Dr. Poulios, > > > > I was able to follow your steps and built all dependencies. In addition, I > also built metis/5.1.0, as parmetis I built has no metis.h and metis > library but they are required for getfem. I passed the getfem compiling > with this configuration. I could not include some libraries on LIBS > somehow. However, I defined LD_LIBRARY_PATH environment variable which > contains all dependency libraries. So no problem on building getfem. > > ./configure --prefix=/risapps/rhel7/getfem-mpi/5.4.1-new \ > > LDFLAGS="-L/risapps/rhel7/gcc/9.3.0/lib64 > -L/risapps/rhel7/openmpi/4.1.1/lib -L/risapps/rhel7/mumps/5.4.1/lib > -L/risapps/rhel7/parmetis/4.0.3/lib -L/risapps/rhel7/metis/5.1.0/lib > -L/risapps/rhel7/scalapack/2.1.0/lib -L/risapps/rhel7/scotch/6.1.1/lib > -L/risapps/rhel7/qhull/2020.2/lib" \ > > CPPFLAGS="-I/risapps/rhel7/gcc/9.3.0/include > -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/mumps/5.4.1/include > -I/risapps/rhel7/parmetis/4.0.3/include > -I/risapps/rhel7/metis/5.1.0/include -I/risapps/rhel7/qhull/2020.2/include" > \ > > LIBS="-lmetis -lzmumps -ldmumps -lcmumps -lsmumps -lmumps_common -lpord > -lgfortran" \ > > --with-blas="/risapps/rhel7/openblas/0.3.18/lib/libopenblas.a" \ > > --with-pic --with-optimization=-O3 --disable-metlab --enable-python > --enable-paralevel=2 --enable-shared > > > > > > However, I got below error when I run a basic testing. I noticed that > most built dependency libraries are static libraries. I.e. *.a files. How > can I build shared libraries that getfem python needs at run time? Thank > you very much ! > > > > [jchen24@tdragon4 testmpi]$ mpirun -n 4 ./parallel.py > > Traceback (most recent call last): > > File > "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", > line 46, in <module> > > Traceback (most recent call last): > > File > "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", > line 46, in <module> > > Traceback (most recent call last): > > File > "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", > line 46, in <module> > > Traceback (most recent call last): > > File > "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", > line 46, in <module> > > from ._getfem import * > > ImportError: libsmumps-5.3.so: cannot open shared object file: No such > file or directory > > > > During handling of the above exception, another exception occurred: > > > > Traceback (most recent call last): > > from ._getfem import * > > File "/rsrch3/home/itops/jchen24/testmpi/./parallel.py", line 39, in > <module> > > ImportError: libsmumps-5.3.so: cannot open shared object file: No such > file or directory > > > > During handling of the above exception, another exception occurred: > > > > Traceback (most recent call last): > > File "/rsrch3/home/itops/jchen24/testmpi/./parallel.py", line 39, in > <module> > > from ._getfem import * > > ImportError: libsmumps-5.3.so: cannot open shared object file: No such > file or directory > > …. > > > > Actually, I was able to compile getfem using system level mumps and metis > with some code changes before I received your previous email. But I got > below errors in testing. I could be that the system level mumps was built > using different mpi as we are using or it missed other dependencies. This > is just FYI.. as I forgot to mention it in my previous email. > > > > [jchen24@tdragon4 testmpi]$ mpirun -n 4 ./parallel.py > > initializing ... > > numthread = 1 > > initializing ... > > numthread = 1 > > initializing ... > > numthread = 1 > > initializing ... > > numthread = 1 > > *** The MPI_Comm_rank() function was called before MPI_INIT was invoked. > > *** This is disallowed by the MPI standard. > > *** Your MPI job will now abort. > > [cdragon003:240842] Local abort before MPI_INIT completed completed > successfully, but am not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > > *** The MPI_Comm_rank() function was called before MPI_INIT was invoked. > > *** This is disallowed by the MPI standard. > > *** Your MPI job will now abort. > > [cdragon003:240843] Local abort before MPI_INIT completed completed > successfully, but am not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > > *** The MPI_Comm_rank() function was called before MPI_INIT was invoked. > > *** This is disallowed by the MPI standard. > > *** Your MPI job will now abort. > > [cdragon003:240844] Local abort before MPI_INIT completed completed > successfully, but am not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > > *** The MPI_Comm_rank() function was called before MPI_INIT was invoked. > > *** This is disallowed by the MPI standard. > > *** Your MPI job will now abort. > > > > Regards > > Jinzhen (Jenny) Chen > > > > *From: *"Chen,Jinzhen" <jche...@mdanderson.org> > *Date: *Monday, December 6, 2021 at 9:45 AM > *To: *Konstantinos Poulios <logar...@googlemail.com> > *Cc: *"getfem-users@nongnu.org" <getfem-users@nongnu.org> > *Subject: *Re: [EXT] Re: getfem installation > > > > Dear Dr. Poulios, > > > > Thank you so much for your so detailed information. It is definitely very > helpful. I will follow your steps and try again. > > > > Best Regards > > Jinzhen Chen > > > > *From: *Konstantinos Poulios <logar...@googlemail.com> > *Date: *Saturday, December 4, 2021 at 3:03 AM > *To: *"Chen,Jinzhen" <jche...@mdanderson.org> > *Cc: *"getfem-users@nongnu.org" <getfem-users@nongnu.org> > *Subject: *Re: [EXT] Re: getfem installation > > > > *WARNING: *This email originated from outside of MD Anderson. Please > validate the sender's email address before clicking on links or attachments > as they may not be safe. > > > > Dear Chen, > > > > Below you can see how I have compiled the parallel version of GetFEM with > MUMPS on our cluster (Scientific linux 7). Depending on the specific > versions of compilers and mpi, some parts of these steps need to be > modified. The basic idea that worked for me is to use the 3 environment > variables, LDFLAGS, CPPFLAGS and LIBS to pass any external dependencies not > detected by the configure script (header files, library directories, > libraries respectively). I also used the --with-blas= configure option to > link to my compiled openblas (which also includes lapack). (The use of the > PYTHONPATH variable is specific to a bug in our cluster's mpi4py module.) > > > > I hope this helps. You will probably need one more fix not included in my > instructions, when you come to the final step of building the getfem python > interface, but let's take that once you get there. > > > > Best regards > > Kostas > > > > module load python3/3.8.11 > module load mpi/4.1.1-gcc-10.3.0-binutils-2.36.1 > module load mpi4py/3.0.3-python-3.8.11-openmpi-4.1.1 > > wget > https://github.com/xianyi/OpenBLAS/releases/download/v0.3.18/OpenBLAS-0.3.18.tar.gz > <https://urldefense.com/v3/__https:/github.com/xianyi/OpenBLAS/releases/download/v0.3.18/OpenBLAS-0.3.18.tar.gz__;!!PfbeBCCAmug!33Kr3jrpM3WlGyBspYp_B85EGY6SFMnA9B5QSjClom5cH5VLOyeuU-fNizSJhic7UEc$> > wget > http://deb.debian.org/debian/pool/main/s/scalapack/scalapack_2.1.0.orig.tar.gz > <https://urldefense.com/v3/__http:/deb.debian.org/debian/pool/main/s/scalapack/scalapack_2.1.0.orig.tar.gz__;!!PfbeBCCAmug!33Kr3jrpM3WlGyBspYp_B85EGY6SFMnA9B5QSjClom5cH5VLOyeuU-fNizSJo2l0zMk$> > wget > http://deb.debian.org/debian/pool/main/s/scotch/scotch_6.1.1.orig.tar.xz > <https://urldefense.com/v3/__http:/deb.debian.org/debian/pool/main/s/scotch/scotch_6.1.1.orig.tar.xz__;!!PfbeBCCAmug!33Kr3jrpM3WlGyBspYp_B85EGY6SFMnA9B5QSjClom5cH5VLOyeuU-fNizSJRoaxqzg$> > wget > http://deb.debian.org/debian/pool/non-free/p/parmetis/parmetis_4.0.3.orig.tar.gz > <https://urldefense.com/v3/__http:/deb.debian.org/debian/pool/non-free/p/parmetis/parmetis_4.0.3.orig.tar.gz__;!!PfbeBCCAmug!33Kr3jrpM3WlGyBspYp_B85EGY6SFMnA9B5QSjClom5cH5VLOyeuU-fNizSJtQaQR4k$> > wget > http://deb.debian.org/debian/pool/main/m/mumps/mumps_5.4.1.orig.tar.gz > <https://urldefense.com/v3/__http:/deb.debian.org/debian/pool/main/m/mumps/mumps_5.4.1.orig.tar.gz__;!!PfbeBCCAmug!33Kr3jrpM3WlGyBspYp_B85EGY6SFMnA9B5QSjClom5cH5VLOyeuU-fNizSJbdI-Xl8$> > wget > http://deb.debian.org/debian/pool/main/q/qhull/qhull_2020.2.orig.tar.gz > <https://urldefense.com/v3/__http:/deb.debian.org/debian/pool/main/q/qhull/qhull_2020.2.orig.tar.gz__;!!PfbeBCCAmug!33Kr3jrpM3WlGyBspYp_B85EGY6SFMnA9B5QSjClom5cH5VLOyeuU-fNizSJo8KuqTo$> > wget > http://download-mirror.savannah.gnu.org/releases/getfem/stable/getfem-5.4.1.tar.gz > <https://urldefense.com/v3/__http:/download-mirror.savannah.gnu.org/releases/getfem/stable/getfem-5.4.1.tar.gz__;!!PfbeBCCAmug!33Kr3jrpM3WlGyBspYp_B85EGY6SFMnA9B5QSjClom5cH5VLOyeuU-fNizSJWGn8oEw$> > > tar -zxf OpenBLAS-0.3.18.tar.gz > tar -zxf scalapack_2.1.0.orig.tar.gz > tar -xf scotch_6.1.1.orig.tar.xz > tar -zxf parmetis_4.0.3.orig.tar.gz > tar -zxf mumps_5.4.1.orig.tar.gz > tar -zxf qhull_2020.2.orig.tar.gz > tar -zxf getfem-5.4.1.tar.gz > > cd OpenBLAS-0.3.18/ > CC=gcc make -j8 > CC=gcc make PREFIX=$HOME/tmp_opt/openblas install > > cd ../scalapack-2.1.0/ > sed '/^\(BLASLIB.*=\).*/ s//\1 > $${HOME}\/tmp_opt\/openblas\/lib\/libopenblas.a/' SLmake.inc.example | sed > '/^\(LAPACKLIB.*=\).*/ s//\1 /' > SLmake.inc > sed -i '/^\(NOOPT.*=.*-O0\)\(\>.*\)/s//\1 -fallow-argument-mismatch > -fPIC\2/' SLmake.inc > sed -i '/^\(FCFLAGS.*=.*-O3\)\(\>.*\)/s//\1 -fallow-argument-mismatch > -fPIC\2/' SLmake.inc > sed -i '/^\(CCFLAGS.*=.*-O3\)\(\>.*\)/s//\1 -fPIC\2/' SLmake.inc > sed -i '/^\(RANLIB.*=\).*/s//\1 echo/' SLmake.inc > make -j4 > mkdir -p $HOME/tmp_opt/scalapack/lib > cp libscalapack.a $HOME/tmp_opt/scalapack/lib/ > > cd ../parmetis-4.0.3/ > make config prefix=$HOME/tmp_opt/parmetis > make -j4 > make install > cp build/Linux-x86_64/libmetis/libmetis.a $HOME/tmp_opt/parmetis/lib/ > cp metis/include/metis.h $HOME/tmp_opt/parmetis/include/ > > cd ../scotch-6.1.1/src > cp Make.inc/Makefile.inc.x86-64_pc_linux2 Makefile.inc > sed -i '/^\(CC[SD].*=\).*/s//\1 mpicc/' Makefile.inc > make -j4 scotch > make -j4 esmumps > make -j4 ptscotch > make -j4 ptesmumps > mkdir -p $HOME/tmp_opt/scotch > cp -r ../lib $HOME/tmp_opt/scotch/ > cp -r ../include $HOME/tmp_opt/scotch/ > > > cd ../../MUMPS_5.4.1/ > cp Make.inc/Makefile.inc.generic Makefile.inc > sed -i '/^#\(SCOTCHDIR.*\)=.*/s//\1 = $${HOME}\/tmp_opt\/scotch/' > Makefile.inc > sed -i '/^#\(ISCOTCH.*\)=.*/s//\1 = -I$(SCOTCHDIR)\/include/' Makefile.inc > sed -i '/^#\(LSCOTCH.*-lpt.*\)/s//\1 -lscotch/' Makefile.inc > sed -i '/^#\(LMETISDIR.*\)=.*/s//\1 = $${HOME}\/tmp_opt\/parmetis\/lib/' > Makefile.inc > sed -i '/^#\(IMETIS.*\)=.*/s//\1 = > -I$${HOME}\/tmp_opt\/parmetis\/include/' Makefile.inc > sed -i '/^#\(LMETIS .*-lparmetis.*\)/s//\1/' Makefile.inc > sed -i '/^#\(ORDERINGSF .*-Dparmetis.*\)/s//\1/' Makefile.inc > sed -i '/^\(ORDERINGSF .*-Dpord$\)/s//#\1/' Makefile.inc > sed -i '/^\(CC.*= \)\(cc\)/s//\1mpi\2/' Makefile.inc > sed -i '/^\(FC.*= \)\(f90\)/s//\1mpi\2/' Makefile.inc > sed -i '/^\(FL.*= \)\(f90\)/s//\1mpi\2/' Makefile.inc > sed -i '/^\(RANLIB .*ranlib.*\)/s//#\1/' Makefile.inc > sed -i '/^#\(RANLIB .*echo.*\)/s//\1/' Makefile.inc > sed -i '/^\(LAPACK.*=\).*/s//\1/' Makefile.inc > sed -i '/^\(SCALAP.*=\).*/s//\1 > $${HOME}\/tmp_opt\/scalapack\/lib\/libscalapack.a/' Makefile.inc > sed -i '/^\(INCPAR.*=\).*/s//\1/' Makefile.inc > sed -i '/^\(\LIBPAR.*=\).*/s//\1 $(SCALAP)/' Makefile.inc > sed -i '/^\(LIBBLAS.*=\).*/s//\1 > $${HOME}\/tmp_opt\/openblas\/lib\/libopenblas.a/' Makefile.inc > sed -i '/^\(OPTF.*=.*\)-O\(\>.*\)/s//\1-fallow-argument-mismatch -O3\2/' > Makefile.inc > sed -i '/^\(OPT[CL].*=.*-O\)\(\>.*\)/s//\13\2/' Makefile.inc > make all > mkdir -p $HOME/tmp_opt/mumps > cp -r lib $HOME/tmp_opt/mumps/ > cp -r include $HOME/tmp_opt/mumps/ > > > cd ../qhull_2020.2/ > make -j4 > mkdir -p $HOME/tmp_opt/qhull > mkdir -p $HOME/tmp_opt/qhull/lib > mkdir -p $HOME/tmp_opt/qhull/include > mkdir -p $HOME/tmp_opt/qhull/include/libqhull > cp lib/libqhullstatic.a $HOME/tmp_opt/qhull/lib/libqhull.a > cp src/libqhull/*.h $HOME/tmp_opt/qhull/include/libqhull/ > > cd ../getfem-5.4.1 > sed -i '/^\(.*\/env \)python\($\)/s//\1python3\2/' bin/extract_doc > PYTHONPATH=/appl/mpi4py/3.0.3-openmpi-4.1.1-gcc-10.3.0-binutils-2.36.1-python-3.8.11/lib/python3.8/site-packages/:$PYTHONPATH > \ > ./configure --prefix="$HOME/tmp_opt" \ > LDFLAGS="-L$HOME/tmp_opt/mumps/lib -L$HOME/tmp_opt/parmetis/lib > -L$HOME/tmp_opt/scalapack/lib -L$HOME/tmp_opt/scotch/lib > -L$HOME/tmp_opt/qhull/lib" \ > CPPFLAGS="-I$HOME/tmp_opt/mumps/include -I$HOME/tmp_opt/parmetis/include > -I$HOME/tmp_opt/qhull/include" \ > LIBS="-lmumps_common -lpord -lgfortran -lscalapack -lptesmumps -lptscotch > -lptscotcherr -lscotch -lparmetis -lmetis -lmpi_usempif08 > -lmpi_usempi_ignore_tkr -lmpi_mpifh" \ > --with-blas="$HOME/tmp_opt/openblas/lib/libopenblas.a" \ > --with-pic --with-optimization=-O3 --disable-matlab --enable-python > --enable-paralevel=2 > make -j4 > make install > > > > > > On Tue, Nov 30, 2021 at 8:51 PM Chen,Jinzhen <jche...@mdanderson.org> > wrote: > > Dear Dr. Poulios, > > > > Thank you so much for your prompt response and information. They are very > helpful. I cc’ed this email to getfem-users@nongnu.org so that I don’t > need to send to your personal email in next one. After reading your > reference of the configure command, I started from scratch but still got > compiling errors. > > > > I installed metis and mumps in the system from EPEL reporistory. > > > > [root@ ~]# rpm -qa |grep metis > > metis-devel-5.1.0-12.el7.x86_64 > > metis-5.1.0-12.el7.x86_64 > > [root@ ~]# rpm -qa |grep metis > > metis-devel-5.1.0-12.el7.x86_64 > > metis-5.1.0-12.el7.x86_64 > > [root@ ~]# rpm -qa |grep MUMPS > > MUMPS-srpm-macros-5.3.5-1.el7.noarch > > MUMPS-5.3.5-1.el7.x86_64 > > MUMPS-devel-5.3.5-1.el7.x86_64 > > MUMPS-common-5.3.5-1.el7.noarch > > > > I used the openmpi/4.1.1, gcc/3.9.0, blas/3.8.0 and zlib/1.2.11 modules > for this build. Below is the module list command output. > > > > [ris_hpc_apps@r1drpswdev3 getfem-5.4.1]$ module list > > > > Currently Loaded Modules: > > 1) openmpi/4.1.1 2) python/3.7.3-anaconda 3) blas/3.8.0 4) > zlib/1.2.11 5) gcc/9.3.0 > > > > Here is my configuration command running on getfem-5.4.1 directory > > > > ./configure CXX="/risapps/rhel7/openmpi/4.1.1/bin/mpic++" > CC="/risapps/rhel7/openmpi/4.1.1/bin/mpicc" > FC="/risapps/rhel7/openmpi/4.1.1/bin/mpifort" > LIBS="-L/risapps/rhel7/gcc/9.3.0/lib64 -L/risapps/rhel7/openmpi/4.1.1/lib > -L/risapps/rhel7/blas/3.8.0 -L/risapps/rhel7/zlib/1.2.11/lib -L/usr/lib64 > -lmetis -lzmumps -ldmumps -lcmumps -lsmumps -lmumps_common" > CXXFLAGES="-I/risapps/rhel7/gcc/9.3.0/include > -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/zlib/1.2.11/include > -I/usr/include -I/usr/include/MUMPS" > CPPFLAGES="-I/risapps/rhel7/gcc/9.3.0/include > -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/zlib/1.2.11/include > -I/usr/include -I/usr/include/MUMPS" > CFLAGS="-I/risapps/rhel7/gcc/9.3.0/include > -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/zlib/1.2.11/include > -I/usr/include -I/usr/include/MUMPS" > PYTHON="/risapps/rhel7/python/3.7.3/bin/python" PYTHON_VERSION=3.7.3 > --with-mumps="-L/usr/lib64" > --with-mumps-include-dir="-I/usr/include/MUMPS" > --with-blas="-L/risapps/rhel7/blas/3.8.0" > --prefix=/risapps/rhel7/getfem-mpi/5.4.1 --enable-shared --enable-metis > --enable-par-mumps -enable-paralevel=2 > > > > The current errors on the make command are the following: > > …. > > libtool: compile: /risapps/rhel7/openmpi/4.1.1/bin/mpic++ -DHAVE_CONFIG_H > -I. -I.. -I../src -I../src -I.. -I/usr/local/include -DGETFEM_PARA_LEVEL=2 > -DGMM_USES_MPI -DGMM_USES_BLAS -DGMM_U SES_BLAS_INTERFACE > -I/usr/include/MUMPS -O3 -std=c++14 -MT dal_bit_vector.lo -MD -MP -MF > .deps/dal_bit_vector.Tpo -c dal_bit_vector.cc -fPIC -DPIC -o > .libs/dal_bit_vector.o > > In file included from ./gmm/gmm_kernel.h:49, > > from getfem/bgeot_config.h:50, > > from getfem/getfem_omp.h:46, > > from getfem/dal_basic.h:42, > > from getfem/dal_bit_vector.h:51, > > from dal_bit_vector.cc:23: > > ./gmm/gmm_matrix.h:956:32: error: ‘MPI_Datatype’ does not name a type > > 956 | template <typename T> inline MPI_Datatype mpi_type(T) > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:958:10: error: ‘MPI_Datatype’ does not name a type > > 958 | inline MPI_Datatype mpi_type(double) { return MPI_DOUBLE; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:959:10: error: ‘MPI_Datatype’ does not name a type > > 959 | inline MPI_Datatype mpi_type(float) { return MPI_FLOAT; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:960:10: error: ‘MPI_Datatype’ does not name a type > > 960 | inline MPI_Datatype mpi_type(long double) { return > MPI_LONG_DOUBLE; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:962:10: error: ‘MPI_Datatype’ does not name a type > > 962 | inline MPI_Datatype mpi_type(std::complex<float>) { return > MPI_COMPLEX; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:963:10: error: ‘MPI_Datatype’ does not name a type > > 963 | inline MPI_Datatype mpi_type(std::complex<double>) { return > MPI_DOUBLE_COMPLEX; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:965:10: error: ‘MPI_Datatype’ does not name a type > > 965 | inline MPI_Datatype mpi_type(int) { return MPI_INT; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:966:10: error: ‘MPI_Datatype’ does not name a type > > 966 | inline MPI_Datatype mpi_type(unsigned int) { return > MPI_UNSIGNED; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:967:10: error: ‘MPI_Datatype’ does not name a type > > 967 | inline MPI_Datatype mpi_type(long) { return MPI_LONG; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h:968:10: error: ‘MPI_Datatype’ does not name a type > > 968 | inline MPI_Datatype mpi_type(unsigned long) { return > MPI_UNSIGNED_LONG; } > > | ^~~~~~~~~~~~ > > ./gmm/gmm_matrix.h: In function ‘typename gmm::strongest_value_type3<V1, > V2, MATSP>::value_type gmm::vect_sp(const > gmm::mpi_distributed_matrix<MAT>&, const V1&, const V2&)’: > > ./gmm/gmm_matrix.h:1012:50: error: ‘MPI_SUM’ was not declared in this scope > > 1012 | MPI_Allreduce(&res, &rest, 1, mpi_type(T()), > MPI_SUM,MPI_COMM_WORLD); > > | ^~~~~~~ > > ./gmm/gmm_matrix.h: In function ‘void gmm::mult_add(const > gmm::mpi_distributed_matrix<MAT>&, const V1&, V2&)’: > > ./gmm/gmm_matrix.h:1023:20: error: there are no arguments to ‘MPI_Wtime’ > that depend on a template parameter, so a declaration of ‘MPI_Wtime’ must > be available [-fpermissive] > > 1023 | double t_ref = MPI_Wtime(); > > | ^~~~~~~~~ > > ./gmm/gmm_matrix.h:1023:20: note: (if you use ‘-fpermissive’, G++ will > accept your code, but allowing the use of an undeclared name is deprecated) > > ./gmm/gmm_matrix.h:1026:21: error: there are no arguments to ‘MPI_Wtime’ > that depend on a template parameter, so a declaration of ‘MPI_Wtime’ must > be available [-fpermissive] > > 1026 | double t_ref2 = MPI_Wtime(); > > | ^~~~~~~~~ > > ./gmm/gmm_matrix.h:1028:19: error: ‘MPI_SUM’ was not declared in this scope > > 1028 | MPI_SUM,MPI_COMM_WORLD); > > | ^~~~~~~ > > ./gmm/gmm_matrix.h:1029:18: error: there are no arguments to ‘MPI_Wtime’ > that depend on a template parameter, so a declaration of ‘MPI_Wtime’ must > be available [-fpermissive] > > 1029 | tmult_tot2 = MPI_Wtime()-t_ref2; > > | ^~~~~~~~~ > > ./gmm/gmm_matrix.h:1032:17: error: there are no arguments to ‘MPI_Wtime’ > that depend on a template parameter, so a declaration of ‘MPI_Wtime’ must > be available [-fpermissive] > > 1032 | tmult_tot = MPI_Wtime()-t_ref; > > | ^~~~~~~~~ > > make[2]: *** [Makefile:941: dal_bit_vector.lo] Error 1 > > make[2]: Leaving directory '/risapps/build7/getfem-5.4.1/src' > > make[1]: *** [Makefile:577: all-recursive] Error 1 > > make[1]: Leaving directory '/risapps/build7/getfem-5.4.1' > > make: *** [Makefile:466: all] Error 2 > > > > I really appreciate your help. Thank you again ! > > > > Best Regards > > Jinzhen Chen > > > > > > *From: *Konstantinos Poulios <k...@mek.dtu.dk> > *Date: *Tuesday, November 30, 2021 at 1:53 AM > *To: *"Chen,Jinzhen" <jche...@mdanderson.org> > *Subject: *[EXT] Re: getfem installation > > > > *WARNING: *This email originated from outside of MD Anderson. Please > validate the sender's email address before clicking on links or attachments > as they may not be safe. > > > > Dear Jinzhen Chen, > > > > Thanks for your question. Yes you should be able to compile GetFEM, also > the parallel version of it on Redhat. I haven't tried it but there isn't > anything distribution specific in the GetFEM code. > > > > Having said that, the parallel version has not been tested very recently, > and it might need some performance fixes from our side, to get a good > scaling for Anne Ceciles problem. Having compiled the parallel version of > GetFEM on your cluster is a good starting point in any case to detect > bottlenecks. > > > > The tricky parts for building GetFEM are normally how to link to mumps, > metis and other dependencies. If you could send me the compilation errors > that you get, either on this address or on the getfem mailing list > getfem-users@nongnu.org I can try to help you to resolve the issues. > > > > As an additional reference, some time ago, when I asked our cluster > administrators to compile GetFEM they used the following configure options: > > > > $ ../configure CXX=mpicxx CC=mpicc FC=mpifort LIBS= > -L/zdata/groups/common/nicpa/2018-feb/generic/build-tools/1.0/lib > -L/zdata/groups/common/nicpa/2018-feb/generic/build-tools/1.0/lib64 > -L/zdata/groups/common/nicpa/2018-feb/generic/gcc/7.3.0/lib > -L/zdata/groups/common/nicpa/2018-feb/generic/gcc/7.3.0/lib64 > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/zlib/1.2.11/gnu-7.3.0/lib > -L/zdata/groups/common/nicpa/2018-feb/generic/numactl/2.0.11/lib > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/libxml2/2.9.7/gnu-7.3.0/lib > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/hwloc/1.11.9/gnu-7.3.0/lib > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/openmpi/3.0.0/gnu-7.3.0/lib > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/parmetis/4.0.3/gnu-7.3.0/lib > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/scalapack/204/gnu-7.3.0/lib > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/scotch/6.0.4/gnu-7.3.0/lib > -L/zdata/g > roups/common/nicpa/2018-feb/XeonX5550/mumps/5.1.2/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/build-tools/1.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/build-tools/1.0/lib64 > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/gcc/7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/gcc/7.3.0/lib64 > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/zlib/1.2.11/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/numactl/2.0.11/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/libxml2/2.9.7/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/hwloc/1.11.9/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/openmpi/3.0.0/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/parmetis/4.0.3/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/scalapack/204/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/Xeon > X5550/openblas/0.2.20/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/scotch/6.0.4/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/mumps/5.1.2/gnu-7.3.0/lib > -lzmumps -ldmumps -lcmumps -lsmumps -lmumps_common -lpord -lesmumps > -lscotch -lscotcherr -lparmetis -lmetis > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib > -lopenblas --disable-openmp --enable-paralevel --enable-metis > --enable-par-mumps --enable-python --disable-boost --disable-matlab > --disable-scilab --with-blas= > -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib > -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib > -lopenblas --prefix=<change-here-for-your-installation-directory> > > > > Best regards > > Konstantinos > > > > On Tue, 2021-11-30 at 00:24 +0000, Chen,Jinzhen wrote: > > Dear Dr Konstantinos Poulios, > > > > My name is Jinzhen Chen, a HPC administrator on MD Anderson Cancer Center, > Houston, USA. I am helping a user (Dr. Lesage,Anne Cecile J) to install > getfem on our HPC cluster, which OS is rhel7.9. I got its basic > installed. However, When I tried to use option –enable-paralevel=2 of the > configure based onhttps://getfem.org/userdoc/parallel.html > <https://urldefense.com/v3/__https:/getfem.org/userdoc/parallel.html__;!!PfbeBCCAmug!ygWbJHCuteXJk48UUs0ySWQrr85DGqhePEiBMq7aoO3iDwtVzR_1URQh62UbmXurRzQ$>, > I kept getting compiling errors. Looks like some libraries are missing, not > visible or not compatible. I have metis and MUMPS installed on the system > and used openmpi/4.1.1 module. > > > > My question is it is possible to install the parallel version of getfem on > rhel7 ? if so, how can I contact developer or someone for the issues? I > really appreciate your help and am looking forward to hearing from you. > > > > Thank you very much ! > > > > *Regards* > > *Jinzhen Chen – HPC Team* > > *MD Anderson Cancer Center <http://www.mdanderson.org/>* > > *inside:Information Services > <http://inside.mdanderson.org/departments/information-services/> * > > *inside:**HPC Request <http://hpcrequest.mdanderson.edu/>* > > *Email: jche...@mdanderson.org <jche...@mdanderson.org> | Tel: > 713-745-6226* > > > > > > The information contained in this e-mail message may be privileged, > confidential, and/or protected from disclosure. This e-mail message may > contain protected health information (PHI); dissemination of PHI should > comply with applicable federal and state laws. If you are not the intended > recipient, or an authorized representative of the intended recipient, any > further review, disclosure, use, dissemination, distribution, or copying of > this message or any attachment (or the information contained therein) is > strictly prohibited. If you think that you have received this e-mail > message in error, please notify the sender by return e-mail and delete all > references to it and its contents from your systems. > > The information contained in this e-mail message may be privileged, > confidential, and/or protected from disclosure. This e-mail message may > contain protected health information (PHI); dissemination of PHI should > comply with applicable federal and state laws. If you are not the intended > recipient, or an authorized representative of the intended recipient, any > further review, disclosure, use, dissemination, distribution, or copying of > this message or any attachment (or the information contained therein) is > strictly prohibited. If you think that you have received this e-mail > message in error, please notify the sender by return e-mail and delete all > references to it and its contents from your systems. > > The information contained in this e-mail message may be privileged, > confidential, and/or protected from disclosure. This e-mail message may > contain protected health information (PHI); dissemination of PHI should > comply with applicable federal and state laws. If you are not the intended > recipient, or an authorized representative of the intended recipient, any > further review, disclosure, use, dissemination, distribution, or copying of > this message or any attachment (or the information contained therein) is > strictly prohibited. If you think that you have received this e-mail > message in error, please notify the sender by return e-mail and delete all > references to it and its contents from your systems. >
module load python3/3.8.11 module load mpi/4.1.1-gcc-10.3.0-binutils-2.36.1 module load mpi4py/3.0.3-python-3.8.11-openmpi-4.1.1 module load scipy/1.6.3-python-3.8.11 wget https://github.com/xianyi/OpenBLAS/releases/download/v0.3.18/OpenBLAS-0.3.18.tar.gz wget http://deb.debian.org/debian/pool/main/s/scalapack/scalapack_2.1.0.orig.tar.gz wget http://deb.debian.org/debian/pool/main/s/scotch/scotch_6.1.1.orig.tar.xz wget http://deb.debian.org/debian/pool/non-free/p/parmetis/parmetis_4.0.3.orig.tar.gz wget http://deb.debian.org/debian/pool/main/m/mumps/mumps_5.4.1.orig.tar.gz wget http://deb.debian.org/debian/pool/main/q/qhull/qhull_2020.2.orig.tar.gz wget http://download-mirror.savannah.gnu.org/releases/getfem/stable/getfem-5.4.1.tar.gz tar -zxf OpenBLAS-0.3.18.tar.gz tar -zxf scalapack_2.1.0.orig.tar.gz tar -xf scotch_6.1.1.orig.tar.xz tar -zxf parmetis_4.0.3.orig.tar.gz tar -zxf mumps_5.4.1.orig.tar.gz tar -zxf qhull_2020.2.orig.tar.gz tar -zxf getfem-5.4.1.tar.gz cd OpenBLAS-0.3.18/ CC=gcc make -j8 CC=gcc make PREFIX=$HOME/tmp_opt/openblas install cd ../scalapack-2.1.0/ sed '/^\(BLASLIB.*=\).*/ s//\1 $${HOME}\/tmp_opt\/openblas\/lib\/libopenblas.a/' SLmake.inc.example | sed '/^\(LAPACKLIB.*=\).*/ s//\1 /' > SLmake.inc sed -i '/^\(NOOPT.*=.*-O0\)\(\>.*\)/s//\1 -fallow-argument-mismatch -fPIC\2/' SLmake.inc sed -i '/^\(FCFLAGS.*=.*-O3\)\(\>.*\)/s//\1 -fallow-argument-mismatch -fPIC\2/' SLmake.inc sed -i '/^\(CCFLAGS.*=.*-O3\)\(\>.*\)/s//\1 -fPIC\2/' SLmake.inc sed -i '/^\(RANLIB.*=\).*/s//\1 echo/' SLmake.inc make -j1 mkdir -p $HOME/tmp_opt/scalapack/lib cp libscalapack.a $HOME/tmp_opt/scalapack/lib/ cd ../parmetis-4.0.3/ make config prefix=$HOME/tmp_opt/parmetis make -j4 make install cp build/Linux-x86_64/libmetis/libmetis.a $HOME/tmp_opt/parmetis/lib/ cp metis/include/metis.h $HOME/tmp_opt/parmetis/include/ cd ../scotch-6.1.1/src cp Make.inc/Makefile.inc.x86-64_pc_linux2 Makefile.inc sed -i '/^\(CC[SD].*=\).*/s//\1 mpicc/' Makefile.inc make -j4 scotch make -j4 esmumps make -j4 ptscotch make -j4 ptesmumps mkdir -p $HOME/tmp_opt/scotch cp -r ../lib $HOME/tmp_opt/scotch/ cp -r ../include $HOME/tmp_opt/scotch/ cd ../../MUMPS_5.4.1/ cp Make.inc/Makefile.inc.generic Makefile.inc sed -i '/^#\(SCOTCHDIR.*\)=.*/s//\1 = $${HOME}\/tmp_opt\/scotch/' Makefile.inc sed -i '/^#\(ISCOTCH.*\)=.*/s//\1 = -I$(SCOTCHDIR)\/include/' Makefile.inc sed -i '/^#\(LSCOTCH.*-lpt.*\)/s//\1 -lscotch/' Makefile.inc sed -i '/^#\(LMETISDIR.*\)=.*/s//\1 = $${HOME}\/tmp_opt\/parmetis\/lib/' Makefile.inc sed -i '/^#\(IMETIS.*\)=.*/s//\1 = -I$${HOME}\/tmp_opt\/parmetis\/include/' Makefile.inc sed -i '/^#\(LMETIS .*-lparmetis.*\)/s//\1/' Makefile.inc sed -i '/^#\(ORDERINGSF .*-Dparmetis.*\)/s//\1/' Makefile.inc sed -i '/^\(ORDERINGSF .*-Dpord$\)/s//#\1/' Makefile.inc sed -i '/^\(CC.*= \)\(cc\)/s//\1mpi\2/' Makefile.inc sed -i '/^\(FC.*= \)\(f90\)/s//\1mpi\2/' Makefile.inc sed -i '/^\(FL.*= \)\(f90\)/s//\1mpi\2/' Makefile.inc sed -i '/^\(RANLIB .*ranlib.*\)/s//#\1/' Makefile.inc sed -i '/^#\(RANLIB .*echo.*\)/s//\1/' Makefile.inc sed -i '/^\(LAPACK.*=\).*/s//\1/' Makefile.inc sed -i '/^\(SCALAP.*=\).*/s//\1 $${HOME}\/tmp_opt\/scalapack\/lib\/libscalapack.a/' Makefile.inc sed -i '/^\(INCPAR.*=\).*/s//\1/' Makefile.inc sed -i '/^\(\LIBPAR.*=\).*/s//\1 $(SCALAP)/' Makefile.inc sed -i '/^\(LIBBLAS.*=\).*/s//\1 $${HOME}\/tmp_opt\/openblas\/lib\/libopenblas.a/' Makefile.inc sed -i '/^\(OPTF.*=.*\)-O\(\>.*\)/s//\1-fallow-argument-mismatch -O3 -fPIC\2/' Makefile.inc sed -i '/^\(OPT[CL].*=.*\)-O\(\>.*\)/s//\1-O3 -fPIC\2/' Makefile.inc make all mkdir -p $HOME/tmp_opt/mumps cp -r lib $HOME/tmp_opt/mumps/ cp -r include $HOME/tmp_opt/mumps/ cd ../qhull_2020.2/ make -j4 mkdir -p $HOME/tmp_opt/qhull mkdir -p $HOME/tmp_opt/qhull/lib mkdir -p $HOME/tmp_opt/qhull/include mkdir -p $HOME/tmp_opt/qhull/include/libqhull cp lib/libqhullstatic.a $HOME/tmp_opt/qhull/lib/libqhull.a cp src/libqhull/*.h $HOME/tmp_opt/qhull/include/libqhull/ cd ../getfem-5.4.1 sed -i '/^\(.*\/env \)python\($\)/s//\1python3\2/' bin/extract_doc PYTHONPATH=/appl/mpi4py/3.0.3-openmpi-4.1.1-gcc-10.3.0-binutils-2.36.1-python-3.8.11/lib/python3.8/site-packages/:$PYTHONPATH \ ./configure --prefix="$HOME/tmp_opt" \ LDFLAGS="-L$HOME/tmp_opt/mumps/lib -L$HOME/tmp_opt/parmetis/lib -L$HOME/tmp_opt/scalapack/lib -L$HOME/tmp_opt/scotch/lib -L$HOME/tmp_opt/qhull/lib" \ CPPFLAGS="-I$HOME/tmp_opt/mumps/include -I$HOME/tmp_opt/parmetis/include -I$HOME/tmp_opt/qhull/include" \ LIBS="-lmumps_common -lpord -lgfortran -lscalapack -lptesmumps -lptscotch -lptscotcherr -lscotch -lparmetis -lmetis -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh" \ --with-blas="$HOME/tmp_opt/openblas/lib/libopenblas.a" \ --with-pic --with-optimization=-O3 --disable-matlab --enable-python --enable-paralevel=2 make -j4 make install
Makefile.inc
Description: Binary data
