It is very dangerous to use different compilers. I would make sure that all the compilers are the MPI compilers.
Thanks, Matt On Thu, Mar 14, 2019 at 8:46 AM Amneet Bhalla <mail2amn...@gmail.com> wrote: > Ah, Ok. Do serial compilers look OK to you? > > Can lib-32 and lib-64 (say -lm) operate simulataneously during runtime, or > this is my imagination? > > > > On Thu, Mar 14, 2019 at 5:36 AM Matthew Knepley <knep...@gmail.com> wrote: > >> On Thu, Mar 14, 2019 at 8:28 AM Amneet Bhalla <mail2amn...@gmail.com> >> wrote: >> >>> Matt -- SAMRAI, PETSc, and libMesh configure logs are attached with this >>> email. Also including some other log files in case they are useful. >>> >> >> Okay, PETSc is not sticking in a /usr/lib (or /usr/lib64). However, I can >> see that you mpif90 (and perhaps other of the wrappers) are >> reporting both /usr/lib64 AND /usr/lib flags, and I am guessing that is >> where the other configures are picking it up. >> >> Thanks, >> >> Matt >> >> >>> Thanks, >>> --Amneet >>> >>> On Thu, Mar 14, 2019 at 4:21 AM Matthew Knepley <knep...@gmail.com> >>> wrote: >>> >>>> In order to see why each flag was included, we need to see >>>> configure.log. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> On Thu, Mar 14, 2019 at 2:40 AM Amneet Bhalla via petsc-users < >>>> petsc-users@mcs.anl.gov> wrote: >>>> >>>>> Hi Folks, >>>>> >>>>> I am on a cluster that has -L/lib dir with 32-bit libraries and >>>>> -L/lib64 with 64-bit libraries. During compilation of some of >>>>> libraries required for my code (such as SAMRAI and libMesh) both paths >>>>> get picked -L/lib and -L/lib64. >>>>> >>>>> I am seeing some sporadic behavior in runtime when at some timesteps >>>>> PETSc does not converge. The same code with the same number of processors >>>>> run just fine on my workstation that has just 64-bit version of libraries. >>>>> >>>>> Even during the final linking stage of the executable, the linker >>>>> gives warnings like >>>>> >>>>> ld: skipping incompatible //lib/libm.so when searching for -lm >>>>> >>>>> ld: skipping incompatible /lib/libm.so when searching for -lm >>>>> >>>>> ld: skipping incompatible /lib/libm.so when searching for -lm >>>>> >>>>> ld: skipping incompatible //lib/libpthread.so when searching for >>>>> -lpthread >>>>> >>>>> ld: skipping incompatible /lib/libpthread.so when searching for >>>>> -lpthread >>>>> >>>>> ld: skipping incompatible /lib/libpthread.so when searching for >>>>> -lpthread >>>>> >>>>> ld: skipping incompatible //lib/libdl.so when searching for -ldl >>>>> >>>>> ld: skipping incompatible //lib/libc.so when searching for -lc >>>>> >>>>> ld: skipping incompatible /lib/libc.so when searching for -lc >>>>> >>>>> ld: skipping incompatible /lib/libc.so when searching for -lc >>>>> but the executable runs. >>>>> >>>>> >>>>> This is during config of SAMRAI when it picks both -L/lib and -L/lib64: >>>>> >>>>> checking whether we are using the GNU Fortran 77 compiler... no >>>>> >>>>> checking whether ifort accepts -g... yes >>>>> >>>>> checking how to get verbose linking output from ifort... -v >>>>> >>>>> checking for Fortran 77 libraries of ifort... >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/tbb/lib/intel64/gcc4.4 >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64 >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64 >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/ipp/lib/intel64 >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64_lin >>>>> -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/ >>>>> -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64 >>>>> -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/ >>>>> -L/lib/../lib64 >>>>> -L/lib/../lib64/ -L/usr/lib/../lib64 -L/usr/lib/../lib64/ >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/tbb/lib/intel64/gcc4.4/ >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/ >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64/ >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/ipp/lib/intel64/ >>>>> -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../ -L/lib64 -L/lib/ >>>>> -L/usr/lib64 -L/usr/lib -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc >>>>> -lpthread -lgcc -lgcc_s -lirc_s -ldl >>>>> >>>>> libMesh is also picking that path >>>>> >>>>> libmesh_optional_LIBS............ : -lhdf5 -lhdf5_cpp -lz >>>>> -L/home/asbhalla/softwares/PETSc-BitBucket/PETSc/linux-opt/lib >>>>> -Wl,-rpath,/home/asbhalla/softwares/PETSc-BitBucket/PETSc/linux-opt/lib >>>>> -Wl,-rpath,/opt/intel/mkl/lib/intel64 -L/opt/intel/mkl/lib/intel64 >>>>> -Wl,-rpath,/opt/mellanox/hcoll/lib -L/opt/mellanox/hcoll/lib >>>>> -Wl,-rpath,/opt/mellanox/mxm/lib -L/opt/mellanox/mxm/lib >>>>> -Wl,-rpath,/opt/intel/compilers_and_libraries_2018.2.199/linux/tbb/lib/intel64/gcc4.4 >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/tbb/lib/intel64/gcc4.4 >>>>> -Wl,-rpath,/opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64 >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64 >>>>> -Wl,-rpath,/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64 >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64 >>>>> -Wl,-rpath,/opt/intel/compilers_and_libraries_2018.2.199/linux/ipp/lib/intel64 >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/ipp/lib/intel64 >>>>> -Wl,-rpath,/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64_lin >>>>> -L/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64_lin >>>>> -lpetsc -lHYPRE -lmkl_intel_lp64 -lmkl_sequential -lmkl_core >>>>> -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lifport >>>>> -lifcoremt_pic -limf -lsvml -lm -lipgo -lirc -lpthread -lgcc_s -lirc_s >>>>> -lstdc++ -ldl -L/lib -Wl,-rpath,/lib >>>>> -Wl,-rpath,/usr/local/mpi/intel/openmpi-4.0.0/lib64 >>>>> -L/usr/local/mpi/intel/openmpi-4.0.0/lib64 >>>>> -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.8.5 >>>>> -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 >>>>> >>>>> Perhaps PETSc also picks up both versions (and there is a way to query >>>>> it from PETSc?), but I can't confirm this. Is there a way to instruct make >>>>> to select only -L/lib64? I want to rule out that 32-bit dynamic library is >>>>> not a culprit for the random non-convergence of PETSc solvers and the >>>>> eventual crash of the simulations. I have tried both gcc-7.3.0 and >>>>> intel-18 >>>>> compilers -- but the same thing is happening. >>>>> >>>>> >>>>> -- >>>>> --Amneet >>>>> >>>>> >>>>> >>>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> <http://www.cse.buffalo.edu/~knepley/> >>>> >>> >>> >>> -- >>> --Amneet >>> >>> >>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> <http://www.cse.buffalo.edu/~knepley/> >> > -- > --Amneet > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>