FW: [PETSC #18391] PETSc crash with memory allocation in ILU preconditioning
Hi,
I am seeing problems when trying to build petsc-dev code. My configure
line is below, same as I successfully did for 2.3.2-p10. I tried with
mkl 9 and mkl 10. Same errors. There are references to undefined
symbols. Please share with me if you have any experience with the issue
or suggestions to resolve it.
Thanks,
Ying
./config/configure.py --with-batch=1 --with-clanguage=C++
--with-vendor-compilers=intel '--CXXFLAGS=-g
-gcc-name=/usr/intel/pkgs/gcc/4.2.2/bin/g++ -gcc-version=420 '
'--LDFLAGS=-L/usr/lib64 -L/usr/intel/pkgs/gcc/4.2.2/lib -ldl -lpthread
-Qlocation,ld,/usr/intel/pkgs/gcc/4.2.2/x86_64-suse-linux/bin
-L/usr/intel/pkgs/icc/10.1.008e/lib -lirc' --with-cxx=$ICCDIR/bin/icpc
--with-fc=$IFCDIR/bin/ifort --with-mpi-compilers=0 --with-mpi-shared=0
--with-debugging=yes --with-mpi=yes --with-mpi-include=$MPIDIR/include
--with-mpi-lib=\[$MPIDIR/lib64/libmpi.a,$MPIDIR/lib64/libmpiif.a,$MPIDIR
/lib64/libmpigi.a\]
--with-blas-lapack-lib=\[$MKLLIBDIR/libguide.so,$MKLLIBDIR/libmkl_lapack
.so,$MKLLIBDIR/libmkl_solver.a,$MKLLIBDIR/libmkl.so\]
--with-scalapack=yes --with-scalapack-include=$MKLDIR/include
--with-scalapack-lib=$MKLLIBDIR/libmkl_scalapack.a --with-blacs=yes
--with-blacs-include=$MKLDIR/include
--with-blacs-lib=$MKLLIBDIR/libmkl_blacs_intelmpi_lp64.a
--with-umfpack=1
--with-umfpack-lib=\[$UMFPACKDIR/UMFPACK/Lib/libumfpack.a,$UMFPACKDIR/AM
D/Lib/libamd.a\] --with-umfpack-include=$UMFPACKDIR/UMFPACK/Include
--with-parmetis=1 --with-parmetis-dir=$PARMETISDIR --with-mumps=1
--download-mumps=$PETSC_DIR/externalpackages/MUMPS_4.6.3.tar.gz
--with-superlu_dist=1
--download-superlu_dist=$PETSC_DIR/externalpackages/superlu_dist_2.0.tar
.gz
/nfs/pdx/proj/dt/pdx_sde02/x86-64_linux26/petsc/petsc-dev/conftest.c:7:
undefined reference to `f2cblaslapack311_id_'
/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libguide.so:
undefined reference to `pthread_atfork'
--
You set a value for --with-blas-lapack-lib=lib, but
['/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libguide.so',
'/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libmkl_lapack.s
o',
'/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libmkl_solver.a
', '/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libmkl.so']
cannot be used
*
-Original Message-
From: Barry Smith [mailto:bsm...@mcs.anl.gov]
Sent: Thursday, October 09, 2008 12:39 PM
To: Rhew, Jung-hoon
Cc: PETSc-Maint Smith; Linton, Tom; Cea, Stephen M; Stettler, Mark
Subject: Re: [PETSC #18391] PETSc crash with memory allocation in ILU
preconditioning
We don't have all the code just right to use those packages with
64 bit integers. I will try to get them all
working by Monday and will let you know my progress. To use them you
will need to be using
petsc-dev
http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html
so you can switch to
that now if you are not yet using it in preparation for my updates.
Barry
On Oct 9, 2008, at 12:52 PM, Rhew, Jung-hoon wrote:
Hi,
I found that the root cause of malloc error was that our PETSc
library had been compiled without 64 bit flag on. Thus, PetscInt
was defined as int instead of long long and for large problems,
the memory allocation requires memory beyond the maximum of int and
causes integer overflow.
But when I tried to build using 64 bit flag (--with-64-bit-
indices=1), all files associated with the external libraries (such
as UMFPACK, and MUMPS) built with PETSc started failing in
compilation mainly due to the incompatibility between int in those
libraries and long long in PETSc.
I wonder if you can let us know how to resolve this conflict when
builing PETSc with 64 bit. The brute force way is to change the
source codes of those libraries where the conflicts occur but I
wonder if there is a neater way of doing this.
Thanks.
jr
Example:
libfast in: /nfs/ltdn/disks/td_disk49/usr.cdmg/jrhew/work/mds_work/
PETSC/mypetsc-2.3.2-p10/src/mat/impls/aij/seq/umfpack
umfpack.c(154): error: a value of type PetscInt={long long} *
cannot be used to initialize an entity of type int *
int m=A-rmap.n,n=A-cmap.n,*ai=mat-i,*aj=mat-
j,status,*ra,idx;
-Original Message-
From: Barry Smith [mailto:bsmith at mcs.anl.gov]
Sent: Tuesday, October 07, 2008 6:15 PM
To: Rhew, Jung-hoon
Cc: petsc-maint at mcs.anl.gov; Linton, Tom; Cea, Stephen M; Stettler,
Mark
Subject: Re: [PETSC #18391] PETSc crash with memory allocation in
ILU preconditioning
During the symbolic phase of ILU(N) there is no way in advance to
know how many new nonzeros are needed
in the factored version over the original matrix (this is tree for LU
too). We handle this by starting with a certain
amount of memory and then if that is not enough for for the symbolic
factor we double the
FW: [PETSC #18391] PETSc crash with memory allocation in ILU preconditioning
On Fri, Oct 10, 2008 at 6:29 PM, Deng, Ying ying.deng at intel.com wrote:
Hi,
I am seeing problems when trying to build petsc-dev code. My configure
line is below, same as I successfully did for 2.3.2-p10. I tried with
mkl 9 and mkl 10. Same errors. There are references to undefined
symbols. Please share with me if you have any experience with the issue
or suggestions to resolve it.
1) Please always send configure.log. The screen output does not tell us
enough to debug problems.
2) Specifying libraries directly is not usually a good idea since some packages,
like MKL, tend to depend on other libraries (like libguide,
libpthread). I would
use --with-blas-lapack-dir=$MKLDIR
3) Mail about install problems should go to petsc-maint at mcs.anl.gov.
petsc-dev
is for discussion of development.
Thanks,
Matt
Thanks,
Ying
./config/configure.py --with-batch=1 --with-clanguage=C++
--with-vendor-compilers=intel '--CXXFLAGS=-g
-gcc-name=/usr/intel/pkgs/gcc/4.2.2/bin/g++ -gcc-version=420 '
'--LDFLAGS=-L/usr/lib64 -L/usr/intel/pkgs/gcc/4.2.2/lib -ldl -lpthread
-Qlocation,ld,/usr/intel/pkgs/gcc/4.2.2/x86_64-suse-linux/bin
-L/usr/intel/pkgs/icc/10.1.008e/lib -lirc' --with-cxx=$ICCDIR/bin/icpc
--with-fc=$IFCDIR/bin/ifort --with-mpi-compilers=0 --with-mpi-shared=0
--with-debugging=yes --with-mpi=yes --with-mpi-include=$MPIDIR/include
--with-mpi-lib=\[$MPIDIR/lib64/libmpi.a,$MPIDIR/lib64/libmpiif.a,$MPIDIR
/lib64/libmpigi.a\]
--with-blas-lapack-lib=\[$MKLLIBDIR/libguide.so,$MKLLIBDIR/libmkl_lapack
.so,$MKLLIBDIR/libmkl_solver.a,$MKLLIBDIR/libmkl.so\]
--with-scalapack=yes --with-scalapack-include=$MKLDIR/include
--with-scalapack-lib=$MKLLIBDIR/libmkl_scalapack.a --with-blacs=yes
--with-blacs-include=$MKLDIR/include
--with-blacs-lib=$MKLLIBDIR/libmkl_blacs_intelmpi_lp64.a
--with-umfpack=1
--with-umfpack-lib=\[$UMFPACKDIR/UMFPACK/Lib/libumfpack.a,$UMFPACKDIR/AM
D/Lib/libamd.a\] --with-umfpack-include=$UMFPACKDIR/UMFPACK/Include
--with-parmetis=1 --with-parmetis-dir=$PARMETISDIR --with-mumps=1
--download-mumps=$PETSC_DIR/externalpackages/MUMPS_4.6.3.tar.gz
--with-superlu_dist=1
--download-superlu_dist=$PETSC_DIR/externalpackages/superlu_dist_2.0.tar
.gz
/nfs/pdx/proj/dt/pdx_sde02/x86-64_linux26/petsc/petsc-dev/conftest.c:7:
undefined reference to `f2cblaslapack311_id_'
/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libguide.so:
undefined reference to `pthread_atfork'
--
You set a value for --with-blas-lapack-lib=lib, but
['/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libguide.so',
'/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libmkl_lapack.s
o',
'/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libmkl_solver.a
', '/p/dt/sde/tools/x86-64_linux26/mkl/10.0.2.018/lib/em64t/libmkl.so']
cannot be used
*
-Original Message-
From: Barry Smith [mailto:bsmith at mcs.anl.gov]
Sent: Thursday, October 09, 2008 12:39 PM
To: Rhew, Jung-hoon
Cc: PETSc-Maint Smith; Linton, Tom; Cea, Stephen M; Stettler, Mark
Subject: Re: [PETSC #18391] PETSc crash with memory allocation in ILU
preconditioning
We don't have all the code just right to use those packages with
64 bit integers. I will try to get them all
working by Monday and will let you know my progress. To use them you
will need to be using
petsc-dev
http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html
so you can switch to
that now if you are not yet using it in preparation for my updates.
Barry
On Oct 9, 2008, at 12:52 PM, Rhew, Jung-hoon wrote:
Hi,
I found that the root cause of malloc error was that our PETSc
library had been compiled without 64 bit flag on. Thus, PetscInt
was defined as int instead of long long and for large problems,
the memory allocation requires memory beyond the maximum of int and
causes integer overflow.
But when I tried to build using 64 bit flag (--with-64-bit-
indices=1), all files associated with the external libraries (such
as UMFPACK, and MUMPS) built with PETSc started failing in
compilation mainly due to the incompatibility between int in those
libraries and long long in PETSc.
I wonder if you can let us know how to resolve this conflict when
builing PETSc with 64 bit. The brute force way is to change the
source codes of those libraries where the conflicts occur but I
wonder if there is a neater way of doing this.
Thanks.
jr
Example:
libfast in: /nfs/ltdn/disks/td_disk49/usr.cdmg/jrhew/work/mds_work/
PETSC/mypetsc-2.3.2-p10/src/mat/impls/aij/seq/umfpack
umfpack.c(154): error: a value of type PetscInt={long long} *
cannot be used to initialize an entity of type int *
int m=A-rmap.n,n=A-cmap.n,*ai=mat-i,*aj=mat-
j,status,*ra,idx;
-Original Message-
From: Barry Smith
