Re: [petsc-dev] building on Spock
Yes, the two files are just petsc headers.
--Junchao Zhang
On Fri, Aug 13, 2021 at 5:17 PM Mark Adams wrote:
> I seem to be getting Kokkos includes in my install but there is no kokkos
> in the configure and I started with a clean PETSc arch directory and
> install directory. Does this make sense?
>
> 18:12 main= /gpfs/alpine/csc314/scratch/adams/petsc$ ll
> /gpfs/alpine/phy122/proj-shared/spock/petsc/current/arch-opt-cray/include/
> | grep -i kokko
> -rw-rw-r-- 1 adams adams7074 May 27 06:57 petscdmda_kokkos.hpp
> -rw-rw-r-- 1 adams adams1636 May 27 06:57 petscvec_kokkos.hpp
> 18:12 main= /gpfs/alpine/csc314/scratch/adams/petsc$
>
>
> On Fri, Aug 13, 2021 at 2:27 PM Mark Adams wrote:
>
>>
>>
>> On Fri, Aug 13, 2021 at 2:01 PM Matthew Knepley
>> wrote:
>>
>>> On Fri, Aug 13, 2021 at 1:49 PM Mark Adams wrote:
>>>
I was building on Spock a few weeks ago, but am getting these errors
now.
I have a setup with this environment and get this error.
Any ideas?
>>>
>>> Here is the error:
>>>
>>> Executing: cc -o /tmp/petsc-6izfgpkr/config.setCompilers/conftest
>>> -L/opt/rocm-4.2.0/lib -lhsa-runtime64 -L${ROCM_PATH}/lib -lamdhip64
>>> -lhsa-runtime64 /tmp/petsc-6izfgpkr/config.setCompilers/conftest.o
>>> Possible ERROR while running linker: exit code 1
>>> stderr:
>>> ld.lld: error: unable to find library -lmpi_gtl_hsa
>>> clang-12: error: linker command failed with exit code 1 (use -v to see
>>> invocation)
>>> Linker output before filtering:
>>>
>>> ld.lld: error: unable to find library -lmpi_gtl_hsa
>>> clang-12: error: linker command failed with exit code 1 (use -v to see
>>> invocation)
>>> :
>>> Linker output after filtering:
>>>
>>> ld.lld: error: unable to find library -lmpi_gtl_hsa
>>> clang-12: error: linker command failed with exit code 1 (use -v to see
>>> invocation):
>>> Error testing C compiler: Cannot compile/link C with cc.
>>> Deleting "CC"
>>>
>>> That -l look like it is put in directly by 'cc'. How does anything work
>>> on that machine?
>>>
>>
>> It looks like this comes from an env variable that I have:
>>
>> export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa"
>>
>>
>>>
>>>Matt
>>>
>>>
Thanks,
Mark
module load craype-accel-amd-gfx908
module load rocm
module load emacs
module load zlib
module load autoconf automake libtool
## These must be set before compiling so the executable picks up GTL
export
PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib"
export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa"
## These must be set before running
export MPIR_CVAR_GPU_EAGER_DEVICE_MEM=0
export MPICH_GPU_SUPPORT_ENABLED=1
export MPICH_SMP_SINGLE_COPY_MODE=CMA
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>> https://www.cse.buffalo.edu/~knepley/
>>>
Re: [petsc-dev] building on Spock
On Fri, Aug 13, 2021 at 2:25 PM Aaron Scheinberg wrote: > Hey Mark, I believe the problem is in this line: > > export PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib" > "8.1.4" should be replaced with "default" since I think that version is in > flux. > Good catch. Its running now. > > On Fri, Aug 13, 2021 at 1:49 PM Mark Adams wrote: > >> I was building on Spock a few weeks ago, but am getting these errors now. >> I have a setup with this environment and get this error. >> >> Any ideas? >> >> Thanks, >> Mark >> >> module load craype-accel-amd-gfx908 >> module load rocm >> module load emacs >> module load zlib >> module load autoconf automake libtool >> >> ## These must be set before compiling so the executable picks up GTL >> export >> PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib" >> export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa" >> >> ## These must be set before running >> export MPIR_CVAR_GPU_EAGER_DEVICE_MEM=0 >> export MPICH_GPU_SUPPORT_ENABLED=1 >> export MPICH_SMP_SINGLE_COPY_MODE=CMA >> >
Re: [petsc-dev] building on Spock
Hey Mark, I believe the problem is in this line: export PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib" "8.1.4" should be replaced with "default" since I think that version is in flux. On Fri, Aug 13, 2021 at 1:49 PM Mark Adams wrote: > I was building on Spock a few weeks ago, but am getting these errors now. > I have a setup with this environment and get this error. > > Any ideas? > > Thanks, > Mark > > module load craype-accel-amd-gfx908 > module load rocm > module load emacs > module load zlib > module load autoconf automake libtool > > ## These must be set before compiling so the executable picks up GTL > export PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib" > export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa" > > ## These must be set before running > export MPIR_CVAR_GPU_EAGER_DEVICE_MEM=0 > export MPICH_GPU_SUPPORT_ENABLED=1 > export MPICH_SMP_SINGLE_COPY_MODE=CMA >
Re: [petsc-dev] building on Spock
On Fri, Aug 13, 2021 at 2:01 PM Matthew Knepley wrote:
> On Fri, Aug 13, 2021 at 1:49 PM Mark Adams wrote:
>
>> I was building on Spock a few weeks ago, but am getting these errors now.
>> I have a setup with this environment and get this error.
>>
>> Any ideas?
>>
>
> Here is the error:
>
> Executing: cc -o /tmp/petsc-6izfgpkr/config.setCompilers/conftest
> -L/opt/rocm-4.2.0/lib -lhsa-runtime64 -L${ROCM_PATH}/lib -lamdhip64
> -lhsa-runtime64 /tmp/petsc-6izfgpkr/config.setCompilers/conftest.o
> Possible ERROR while running linker: exit code 1
> stderr:
> ld.lld: error: unable to find library -lmpi_gtl_hsa
> clang-12: error: linker command failed with exit code 1 (use -v to see
> invocation)
> Linker output before filtering:
>
> ld.lld: error: unable to find library -lmpi_gtl_hsa
> clang-12: error: linker command failed with exit code 1 (use -v to see
> invocation)
> :
> Linker output after filtering:
>
> ld.lld: error: unable to find library -lmpi_gtl_hsa
> clang-12: error: linker command failed with exit code 1 (use -v to see
> invocation):
> Error testing C compiler: Cannot compile/link C with cc.
> Deleting "CC"
>
> That -l look like it is put in directly by 'cc'. How does anything work on
> that machine?
>
It looks like this comes from an env variable that I have:
export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa"
>
>Matt
>
>
>> Thanks,
>> Mark
>>
>> module load craype-accel-amd-gfx908
>> module load rocm
>> module load emacs
>> module load zlib
>> module load autoconf automake libtool
>>
>> ## These must be set before compiling so the executable picks up GTL
>> export
>> PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib"
>> export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa"
>>
>> ## These must be set before running
>> export MPIR_CVAR_GPU_EAGER_DEVICE_MEM=0
>> export MPICH_GPU_SUPPORT_ENABLED=1
>> export MPICH_SMP_SINGLE_COPY_MODE=CMA
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
>
Re: [petsc-dev] building on Spock
Here's a script I use for building Hypre that worked a few days ago. It may
help with PETSc.
https://github.com/hypre-space/hypre/issues/401
-Paul
On Fri, Aug 13, 2021 at 12:02 PM Matthew Knepley wrote:
> On Fri, Aug 13, 2021 at 1:49 PM Mark Adams wrote:
>
>> I was building on Spock a few weeks ago, but am getting these errors now.
>> I have a setup with this environment and get this error.
>>
>> Any ideas?
>>
>
> Here is the error:
>
> Executing: cc -o /tmp/petsc-6izfgpkr/config.setCompilers/conftest
> -L/opt/rocm-4.2.0/lib -lhsa-runtime64 -L${ROCM_PATH}/lib -lamdhip64
> -lhsa-runtime64 /tmp/petsc-6izfgpkr/config.setCompilers/conftest.o
> Possible ERROR while running linker: exit code 1
> stderr:
> ld.lld: error: unable to find library -lmpi_gtl_hsa
> clang-12: error: linker command failed with exit code 1 (use -v to see
> invocation)
> Linker output before filtering:
>
> ld.lld: error: unable to find library -lmpi_gtl_hsa
> clang-12: error: linker command failed with exit code 1 (use -v to see
> invocation)
> :
> Linker output after filtering:
>
> ld.lld: error: unable to find library -lmpi_gtl_hsa
> clang-12: error: linker command failed with exit code 1 (use -v to see
> invocation):
> Error testing C compiler: Cannot compile/link C with cc.
> Deleting "CC"
>
> That -l look like it is put in directly by 'cc'. How does anything work on
> that machine?
>
>Matt
>
>
>> Thanks,
>> Mark
>>
>> module load craype-accel-amd-gfx908
>> module load rocm
>> module load emacs
>> module load zlib
>> module load autoconf automake libtool
>>
>> ## These must be set before compiling so the executable picks up GTL
>> export
>> PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib"
>> export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa"
>>
>> ## These must be set before running
>> export MPIR_CVAR_GPU_EAGER_DEVICE_MEM=0
>> export MPICH_GPU_SUPPORT_ENABLED=1
>> export MPICH_SMP_SINGLE_COPY_MODE=CMA
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
>
Re: [petsc-dev] building on Spock
On Fri, Aug 13, 2021 at 1:49 PM Mark Adams wrote:
> I was building on Spock a few weeks ago, but am getting these errors now.
> I have a setup with this environment and get this error.
>
> Any ideas?
>
Here is the error:
Executing: cc -o /tmp/petsc-6izfgpkr/config.setCompilers/conftest
-L/opt/rocm-4.2.0/lib -lhsa-runtime64 -L${ROCM_PATH}/lib -lamdhip64
-lhsa-runtime64 /tmp/petsc-6izfgpkr/config.setCompilers/conftest.o
Possible ERROR while running linker: exit code 1
stderr:
ld.lld: error: unable to find library -lmpi_gtl_hsa
clang-12: error: linker command failed with exit code 1 (use -v to see
invocation)
Linker output before filtering:
ld.lld: error: unable to find library -lmpi_gtl_hsa
clang-12: error: linker command failed with exit code 1 (use -v to see
invocation)
:
Linker output after filtering:
ld.lld: error: unable to find library -lmpi_gtl_hsa
clang-12: error: linker command failed with exit code 1 (use -v to see
invocation):
Error testing C compiler: Cannot compile/link C with cc.
Deleting "CC"
That -l look like it is put in directly by 'cc'. How does anything work on
that machine?
Matt
> Thanks,
> Mark
>
> module load craype-accel-amd-gfx908
> module load rocm
> module load emacs
> module load zlib
> module load autoconf automake libtool
>
> ## These must be set before compiling so the executable picks up GTL
> export PE_MPICH_GTL_DIR_amd_gfx908="-L/opt/cray/pe/mpich/8.1.4/gtl/lib"
> export PE_MPICH_GTL_LIBS_amd_gfx908="-lmpi_gtl_hsa"
>
> ## These must be set before running
> export MPIR_CVAR_GPU_EAGER_DEVICE_MEM=0
> export MPICH_GPU_SUPPORT_ENABLED=1
> export MPICH_SMP_SINGLE_COPY_MODE=CMA
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
