Yes, I had the hsa lib commented out but that did not help (appended). I now see that I had this problem in July and Junchao was helping. I was able to fix it with PrgEnv-gnu.
THe fortran test actually worked. Oh well, the application does their own linking so maybe that will fix it up. (They do use OMP). Thanks, Mark gmake[3]: [/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib/petsc/conf/rules:301: ex19.PETSc] Error 2 (ignored) *******************Error detected during compile or link!******************* See http://www.mcs.anl.gov/petsc/documentation/faq.html /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials ex19 ********************************************************************************* cc *-L/opt/rocm-4.2.0/lib -lhsa-runtime64* -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -Qunused-arguments -fvisibility=hidden -g -O2 -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -Qunused-arguments -fvisibility=hidden -g -O2 -I/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/include -I/opt/rocm-4.2.0/include ex19.c -Wl,-rpath,/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib -L/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib -Wl,-rpath,/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib -L/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib -Wl,-rpath,/opt/rocm-4.2.0/lib -L/opt/rocm-4.2.0/lib -Wl,-rpath,/opt/cray/pe/gcc/8.1.0/snos/lib64 -L/opt/cray/pe/gcc/8.1.0/snos/lib64 -Wl,-rpath,/opt/cray/pe/libsci/ 21.06.1.1/CRAY/9.0/x86_64/lib -L/opt/cray/pe/libsci/ 21.06.1.1/CRAY/9.0/x86_64/lib -Wl,-rpath,/opt/cray/pe/mpich/8.1.7/ofi/cray/10.0/lib -L/opt/cray/pe/mpich/8.1.7/ofi/cray/10.0/lib -Wl,-rpath,/opt/cray/pe/mpich/default/gtl/lib -L/opt/cray/pe/mpich/default/gtl/lib -Wl,-rpath,/opt/cray/pe/dsmml/0.1.5/dsmml/lib -L/opt/cray/pe/dsmml/0.1.5/dsmml/lib -Wl,-rpath,/opt/cray/pe/pmi/6.0.12/lib -L/opt/cray/pe/pmi/6.0.12/lib -Wl,-rpath,/opt/cray/pe/cce/12.0.1/cce/x86_64/lib -L/opt/cray/pe/cce/12.0.1/cce/x86_64/lib -Wl,-rpath,/opt/cray/xpmem/2.2.40-2.1_2.44__g3cf3325.shasta/lib64 -L/opt/cray/xpmem/2.2.40-2.1_2.44__g3cf3325.shasta/lib64 -Wl,-rpath,/opt/cray/pe/cce/12.0.1/cce-clang/x86_64/lib/clang/12.0.0/lib/linux -L/opt/cray/pe/cce/12.0.1/cce-clang/x86_64/lib/clang/12.0.0/lib/linux -Wl,-rpath,/opt/cray/pe/gcc/8.1.0/snos/lib/gcc/x86_64-suse-linux/8.1.0 -L/opt/cray/pe/gcc/8.1.0/snos/lib/gcc/x86_64-suse-linux/8.1.0 -Wl,-rpath,/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-unknown-linux-gnu/lib -L/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-unknown-linux-gnu/lib -Wl,-rpath,/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-pc-linux-gnu/..//x86_64-unknown-linux-gnu/lib -L/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-pc-linux-gnu/..//x86_64-unknown-linux-gnu/lib -lpetsc -lparmetis -lmetis -lhipsparse -lhipblas -lrocsparse -lrocsolver -lrocblas -lrocrand -lamdhip64* -lhsa-runtime64 *-lstdc++ -ldl -lmpifort_cray -lmpi_cray -lmpi_gtl_hsa -ldsmml -lpmi -lxpmem -lpgas-shmem -lquadmath -lcrayacc_amdgpu -lopenacc -lmodules -lfi -lcraymath -lf -lu -lcsup -lgfortran -lpthread -lgcc_eh -lm -lclang_rt.craypgo-x86_64 -lclang_rt.builtins-x86_64 -lquadmath -lstdc++ -ldl -o ex19 ld.lld: error: /gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib/libpetsc.so: undefined reference to .omp_offloading.img_start.cray_amdgcn-amd-amdhsa [--no-allow-shlib-undefined] ld.lld: error: /gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib/libpetsc.so: undefined reference to .omp_offloading.img_size.cray_amdgcn-amd-amdhsa [--no-allow-shlib-undefined] ld.lld: error: /gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib/libpetsc.so: undefined reference to .omp_offloading.img_cache.cray_amdgcn-amd-amdhsa [--no-allow-shlib-undefined] clang-12: error: linker command failed with exit code 1 (use -v to see invocation) gmake[4]: *** [<builtin>: ex19] Error 1 On Sun, Sep 19, 2021 at 9:44 AM Stefano Zampini <[email protected]> wrote: > Are you following the user advices here > https://docs.olcf.ornl.gov/systems/spock_quick_start_guide.html#compiling-with-the-cray-compiler-wrappers-cc-or-cc > ? > > Il giorno dom 19 set 2021 alle ore 16:30 Mark Adams <[email protected]> ha > scritto: > >> I am getting to see this error. It seems to be suggesting that I turn >> --no-allow-shlib-undefined off. >> Any ideas? >> Thanks, >> Mark >> >> 09:09 main= /gpfs/alpine/csc314/scratch/adams/petsc$ make >> PETSC_DIR=/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new >> PETSC_ARCH="" check >> Running check examples to verify correct installation >> Using >> PETSC_DIR=/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new >> and PETSC_ARCH= >> gmake[3]: >> [/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib/petsc/conf/rules:301: >> ex19.PETSc] Error 2 (ignored) >> *******************Error detected during compile or >> link!******************* >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials ex19 >> >> ********************************************************************************* >> cc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas >> -fstack-protector -Qunused-arguments -fvisibility=hidden -g -O2 -fPIC >> -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas >> -fstack-protector -Qunused-arguments -fvisibility=hidden -g -O2 >> >> -I/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/include >> -I/opt/rocm-4.2.0/include ex19.c >> >> -Wl,-rpath,/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib >> -L/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib >> -Wl,-rpath,/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib >> -L/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib >> -Wl,-rpath,/opt/rocm-4.2.0/lib -L/opt/rocm-4.2.0/lib >> -Wl,-rpath,/opt/cray/pe/gcc/8.1.0/snos/lib64 >> -L/opt/cray/pe/gcc/8.1.0/snos/lib64 -Wl,-rpath,/opt/cray/pe/libsci/ >> 21.06.1.1/CRAY/9.0/x86_64/lib -L/opt/cray/pe/libsci/ >> 21.06.1.1/CRAY/9.0/x86_64/lib >> -Wl,-rpath,/opt/cray/pe/mpich/8.1.7/ofi/cray/10.0/lib >> -L/opt/cray/pe/mpich/8.1.7/ofi/cray/10.0/lib >> -Wl,-rpath,/opt/cray/pe/mpich/default/gtl/lib >> -L/opt/cray/pe/mpich/default/gtl/lib >> -Wl,-rpath,/opt/cray/pe/dsmml/0.1.5/dsmml/lib >> -L/opt/cray/pe/dsmml/0.1.5/dsmml/lib -Wl,-rpath,/opt/cray/pe/pmi/6.0.12/lib >> -L/opt/cray/pe/pmi/6.0.12/lib >> -Wl,-rpath,/opt/cray/pe/cce/12.0.1/cce/x86_64/lib >> -L/opt/cray/pe/cce/12.0.1/cce/x86_64/lib >> -Wl,-rpath,/opt/cray/xpmem/2.2.40-2.1_2.44__g3cf3325.shasta/lib64 >> -L/opt/cray/xpmem/2.2.40-2.1_2.44__g3cf3325.shasta/lib64 >> -Wl,-rpath,/opt/cray/pe/cce/12.0.1/cce-clang/x86_64/lib/clang/12.0.0/lib/linux >> -L/opt/cray/pe/cce/12.0.1/cce-clang/x86_64/lib/clang/12.0.0/lib/linux >> -Wl,-rpath,/opt/cray/pe/gcc/8.1.0/snos/lib/gcc/x86_64-suse-linux/8.1.0 >> -L/opt/cray/pe/gcc/8.1.0/snos/lib/gcc/x86_64-suse-linux/8.1.0 >> -Wl,-rpath,/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-unknown-linux-gnu/lib >> -L/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-unknown-linux-gnu/lib >> -Wl,-rpath,/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-pc-linux-gnu/..//x86_64-unknown-linux-gnu/lib >> -L/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-pc-linux-gnu/..//x86_64-unknown-linux-gnu/lib >> -lpetsc -lparmetis -lmetis -lhipsparse -lhipblas -lrocsparse -lrocsolver >> -lrocblas -lrocrand -lamdhip64 -lstdc++ -ldl -lmpifort_cray -lmpi_cray >> -lmpi_gtl_hsa -ldsmml -lpmi -lxpmem -lpgas-shmem -lquadmath >> -lcrayacc_amdgpu -lopenacc -lmodules -lfi -lcraymath -lf -lu -lcsup >> -lgfortran -lpthread -lgcc_eh -lm -lclang_rt.craypgo-x86_64 >> -lclang_rt.builtins-x86_64 -lquadmath -lstdc++ -ldl -o ex19 >> >> >> *ld.lld: error: >> /gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib/libpetsc.so: >> undefined reference to .omp_offloading.img_start.cray_amdgcn-amd-amdhsa >> [--no-allow-shlib-undefined]ld.lld: error: >> /gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib/libpetsc.so: >> undefined reference to .omp_offloading.img_size.cray_amdgcn-amd-amdhsa >> [--no-allow-shlib-undefined]ld.lld: error: >> /gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib/libpetsc.so: >> undefined reference to .omp_offloading.img_cache.cray_amdgcn-amd-amdhsa >> [--no-allow-shlib-undefined]* >> clang-12: error: linker command failed with exit code 1 (use -v to see >> invocation) >> gmake[4]: *** [<builtin>: ex19] Error 1 >> *******************Error detected during compile or >> link!******************* >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials ex5f >> ********************************************************* >> ftn -fPIC -g -O2 -fPIC -g -O2 >> >> -I/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/include >> -I/opt/rocm-4.2.0/include ex5f.F90 >> >> -Wl,-rpath,/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib >> -L/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib >> -Wl,-rpath,/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib >> -L/gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray-new/lib >> -Wl,-rpath,/opt/rocm-4.2.0/lib -L/opt/rocm-4.2.0/lib >> -Wl,-rpath,/opt/cray/pe/gcc/8.1.0/snos/lib64 >> -L/opt/cray/pe/gcc/8.1.0/snos/lib64 -Wl,-rpath,/opt/cray/pe/libsci/ >> 21.06.1.1/CRAY/9.0/x86_64/lib -L/opt/cray/pe/libsci/ >> 21.06.1.1/CRAY/9.0/x86_64/lib >> -Wl,-rpath,/opt/cray/pe/mpich/8.1.7/ofi/cray/10.0/lib >> -L/opt/cray/pe/mpich/8.1.7/ofi/cray/10.0/lib >> -Wl,-rpath,/opt/cray/pe/mpich/default/gtl/lib >> -L/opt/cray/pe/mpich/default/gtl/lib >> -Wl,-rpath,/opt/cray/pe/dsmml/0.1.5/dsmml/lib >> -L/opt/cray/pe/dsmml/0.1.5/dsmml/lib -Wl,-rpath,/opt/cray/pe/pmi/6.0.12/lib >> -L/opt/cray/pe/pmi/6.0.12/lib >> -Wl,-rpath,/opt/cray/pe/cce/12.0.1/cce/x86_64/lib >> -L/opt/cray/pe/cce/12.0.1/cce/x86_64/lib >> -Wl,-rpath,/opt/cray/xpmem/2.2.40-2.1_2.44__g3cf3325.shasta/lib64 >> -L/opt/cray/xpmem/2.2.40-2.1_2.44__g3cf3325.shasta/lib64 >> -Wl,-rpath,/opt/cray/pe/cce/12.0.1/cce-clang/x86_64/lib/clang/12.0.0/lib/linux >> -L/opt/cray/pe/cce/12.0.1/cce-clang/x86_64/lib/clang/12.0.0/lib/linux >> -Wl,-rpath,/opt/cray/pe/gcc/8.1.0/snos/lib/gcc/x86_64-suse-linux/8.1.0 >> -L/opt/cray/pe/gcc/8.1.0/snos/lib/gcc/x86_64-suse-linux/8.1.0 >> -Wl,-rpath,/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-unknown-linux-gnu/lib >> -L/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-unknown-linux-gnu/lib >> -Wl,-rpath,/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-pc-linux-gnu/..//x86_64-unknown-linux-gnu/lib >> -L/opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-pc-linux-gnu/..//x86_64-unknown-linux-gnu/lib >> -lpetsc -lparmetis -lmetis -lhipsparse -lhipblas -lrocsparse -lrocsolver >> -lrocblas -lrocrand -lamdhip64 -lstdc++ -ldl -lmpifort_cray -lmpi_cray >> -lmpi_gtl_hsa -ldsmml -lpmi -lxpmem -lpgas-shmem -lquadmath >> -lcrayacc_amdgpu -lopenacc -lmodules -lfi -lcraymath -lf -lu -lcsup >> -lgfortran -lpthread -lgcc_eh -lm -lclang_rt.craypgo-x86_64 >> -lclang_rt.builtins-x86_64 -lquadmath -lstdc++ -ldl -o ex5f >> /opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-pc-linux-gnu/bin/ld: >> warning: alignment 128 of symbol >> `$host_init$$runtime_init_for_iso_c_binding$iso_c_binding_' in >> /opt/cray/pe/cce/12.0.1/cce/x86_64/lib/libmodules.so is smaller than 256 in >> /tmp/pe_39617/ex5f_1.o >> /opt/cray/pe/cce/12.0.1/binutils/x86_64/x86_64-pc-linux-gnu/bin/ld: >> warning: alignment 64 of symbol `$data_init$iso_c_binding_' in >> /opt/cray/pe/cce/12.0.1/cce/x86_64/lib/libmodules.so is smaller than 256 in >> /tmp/pe_39617/ex5f_1.o >> Fortran example src/snes/tutorials/ex5f run successfully with 1 MPI >> process >> Completed test examples >> 09:12 main= /gpfs/alpine/csc314/scratch/adams/petsc$ module list >> >> Currently Loaded Modules: >> 1) craype-x86-rome 4) perftools-base/21.05.0 >> 7) cray-pmi-lib/6.0.12 10) cray-dsmml/0.1.5 13) PrgEnv-cray/8.1.0 >> 16) rocm/4.2.0 19) autoconf/2.69 >> 2) libfabric/1.11.0.4.75 5) xpmem/2.2.40-2.1_2.44__g3cf3325.shasta >> 8) cce/12.0.1 11) cray-mpich/8.1.7 14) DefApps/default >> 17) emacs/27.2 20) automake/1.16.3 >> 3) craype-network-ofi 6) cray-pmi/6.0.12 >> 9) craype/2.7.8 12) cray-libsci/21.06.1.1 15) >> craype-accel-amd-gfx908 18) zlib/1.2.11 21) libtool/2.4.6 >> >> >> > > -- > Stefano >
