Re: [petsc-users] seg fault with superlu

Thu, 06 Dec 2018 20:31:20 -0800

thanks, using " -mat_superlu_dist_fact SamePattern_SameRowPerm" works also with options other than  "-mat_superlu_dist_colperm NATURAL". The solution worked for 32bit integers but when I compile petsc using 64bit indices I get the following error regardless of the options used for mat_superlu_dist_fact and mat_superlu_dist_colperm
 
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] SuperLU_DIST:pzgssvx line 465 /home/cdfmat_marius/prog/petsc/master_build_gcc_superlu6/petsc/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c
[0]PETSC ERROR: [0] MatLUFactorNumeric_SuperLU_DIST line 314 /home/cdfmat_marius/prog/petsc/master_build_gcc_superlu6/petsc/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c
[0]PETSC ERROR: [0] MatLUFactorNumeric line 3080 /home/cdfmat_marius/prog/petsc/master_build_gcc_superlu6/petsc/src/mat/interface/matrix.c
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.10.2-723-g2a65a22  GIT Date: 2018-11-25 07:45:29 +0100
[0]PETSC ERROR: /home/cdfmat_marius/prog/transomat_latest_i8/transomat on a  named a122 by cdfmat_marius Fri Dec  7 11:52:52 2018
[0]PETSC ERROR: Configure options --prefix=/home/cdfmat_marius/prog/petsc/master_gcc_superlu6_debug_i8 --with-64-bit-indices=1 --CC=mpicc --COPTFLAGS="-O0 -march=native -g -std=c11 " --CXX=mpicxx --CXXOPTFLAGS="-O0 -march=native -g -std=c++11" --FC=mpif90 --FOPTFLAGS="-O
0 -march=native -g -fbacktrace -fdefault-integer-8" --with-mpi=1 --with-x=0 --download-parmetis=/home/cdfmat_marius/prog/petsc/master_build_gcc_superlu6/petsc/externalpackages/git.parmetis.tar.gz --download-metis=/home/cdfmat_marius/prog/petsc/master_build_gcc_superlu6/p
etsc/externalpackages/git.metis.tar.gz --with-blaslapack-lib=" -L/home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_ilp64 -lpthread 
-lm -ldl " --with-lapack-lib=" -L/home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_ilp64 -lpthread -lm -ldl " --with-blas-lib=" -L/
home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_ilp64 -lpthread -lm -ldl " --with-scalapack-lib=" -L/home/cdfmat_marius/intel/comp
ilers_and_libraries_2019.1.144/linux/mkl/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_ilp64 -lpthread -lm -ldl " --download-pastix=no --download-superlu_dist=/home/cdfmat_marius/prog/petsc/master_b
uild_gcc_superlu6/petsc/externalpackages/git.superlu_dist.tar.gz --download-superlu_dist-commit=HEAD --download-hwloc=/home/cdfmat_marius/prog/petsc/master_build_gcc_superlu6/petsc/externalpackages/hwloc-1.11.1.tar.gz --download-sowing=/home/cdfmat_marius/prog/petsc/mast
er_build_gcc_superlu6/petsc/externalpackages/git.sowing.tar.gz --with-openmp=0 --with-pthread=0 --download-elemental=/home/cdfmat_marius/prog/petsc/master_build_gcc_superlu6/petsc/externalpackages/git.elemental.tar.gz --download-elemental-commit=HEAD --with-cxx-dialect=c
++11 --with-scalar-type=complex --with-pic=1 --download-elemental-cmake-arguments="-DEL_USE_64BIT_INTS=ON -DEL_USE_64BIT_BLAS_INTS=ON" --with-debugging=1
[0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
 
 
Is superlu_dist supposed to work with 64 integers?
 
We just fixed a memory bug in 6.1.0 (which shows up in very rare situation), not sure whether it is related to your problem.
 
But "-mat_superlu_dist_colperm NATURAL" will be very slow, usually generate lots of fill-ins in the LU factors.
 
Sherry
 
On Thu, Dec 6, 2018 at 3:50 PM Marius Buerkle via petsc-users <petsc-users@mcs.anl.gov> wrote:
Thanks for the suggestion I will try this option. I used already "-mat_superlu_dist_colperm NATURAL" which seems to help but I did not test it thoroughly.
 

Betreff: Re: [petsc-users] seg fault with superlu
 
On Thu, Dec 6, 2018 at 2:55 AM Matthew Knepley via petsc-users <petsc-users@mcs.anl.gov> wrote:
Satish,
 
This is exactly the same method (pzgssvx) that had a problem before. Do you remember what happened?
 
Used to have a bug in the row permutation algorithm. The row permutation is not get populated correctly. It was fixed from PETSc-3.8.x.   If the current issue is the same as before, a simple "fix" is to use 
 
" -mat_superlu_dist_fact SamePattern_SameRowPerm". 
 
Fande,
 
 
 
  Thanks,
 
     Matt
 
On Wed, Dec 5, 2018 at 8:38 PM Smith, Barry F. via petsc-users <petsc-users@mcs.anl.gov> wrote:

   I would start with valgrind see if it finds anything. If not and if you can provide the code that reproduces the problem we can take a look at it.

    Barry

https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind



> On Dec 5, 2018, at 7:16 PM, Marius Buerkle via petsc-users <petsc-users@mcs.anl.gov> wrote:
>
> Hi,
> 
> I get the following error duing MatMatSolve. It works for smaller number of processes but if I  increade the number pf processes used it crashes.
> 
> [25]PETSC ERROR: ------------------------------------------------------------------------
> [25]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
> [25]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [25]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [25]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
> [25]PETSC ERROR: likely location of problem given in stack below
> [25]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
> [25]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
> [25]PETSC ERROR:       INSTEAD the line number of the start of the function
> [25]PETSC ERROR:       is given.
> [25]PETSC ERROR: [25] SuperLU_DIST:pzgssvx line 236 /home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c
> [25]PETSC ERROR: [25] MatMatSolve_SuperLU_DIST line 200 /home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c
> [25]PETSC ERROR: [25] MatMatSolve line 3396 /home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/src/mat/interface/matrix.c
> [25]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
> [25]PETSC ERROR: Signal received
> [25]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
> [25]PETSC ERROR: Petsc Development GIT revision: v3.10.2-723-g2a65a22  GIT Date: 2018-11-25 07:45:29 +0100
> [25]PETSC ERROR: /home/cdfmat_marius/prog/transomat_latest_test4/transomat on a  named a003 by cdfmat_marius Thu Dec  6 10:12:23 2018
> [25]PETSC ERROR: Configure options --prefix=/home/cdfmat_marius/prog/petsc/master_gcc_local_debug --with-64-bit-indices=0 --CC=mpicc --COPTFLAGS="-O0 -mtune=generic -g -std=c11 " --CXX=mpicxx --CXXOPTFLAGS="-O0 -mtune=generic -g -std=c++11 " --FC=mpif90 --FOPTFLAGS="-O0 -mtune=generic -g -fbacktrace -g " --with-mpi=1 --with-x=0 --download-parmetis=/home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/externalpackages/git.parmetis.tar.gz --download-metis=/home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/externalpackages/git.metis.tar.gz --with-blaslapack-lib=" -L/home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_lp64 -lpthread -lm -ldl " --with-scalapack-lib=" -L/home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_lp64 -lpthread -lm -ldl " --download-pastix=no --download-superlu_dist=/home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/externalpackages/git.superlu_dist.tar.gz --download-superlu_dist-commit=HEAD --download-hwloc=/home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/externalpackages/hwloc-1.11.1.tar.gz --download-sowing=/home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/externalpackages/git.sowing.tar.gz --with-mkl_pardiso-dir=/home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl --with-openmp=0 --with-pthread=0 --download-elemental=/home/cdfmat_marius/prog/petsc/master_build_gcc_local/petsc/externalpackages/git.elemental.tar.gz --download-elemental-commit=HEAD --with-mkl_cpardiso-dir=/home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl --with-mkl_sparse_optimize-dir=/home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl --with-mkl_sparse_sp2m-dir=/home/cdfmat_marius/intel/compilers_and_libraries_2019.1.144/linux/mkl --with-cxx-dialect=c++11 --with-scalar-type=complex --with-debugging=1
> 
> bestm
> marius
 
 
 
--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
 

Reply via email to