If using valgrind - I sugest using MPICH2 [installed with options --enable-g=meminit --enable-fast]
And valgrind can be invoked with: mpiexec -np 2 valgrind --tool=memcheck -q ./executable -exectuable-options Satish On Wed, 23 Apr 2008, Matthew Knepley wrote: > On Wed, Apr 23, 2008 at 9:07 AM, <Amit.Itagi at seagate.com> wrote: > > Barry, > > > > This is what valgrind gives me. Any idea ? What is confusing me is that I > > get the crash after several GMRES iterations. > > 1) Always start with the simplest case, meaning serial > > 2) When you run valgrind in parallel, you need --trace-children=yes, since > MPI usually spawns other processes > > 3) It is possible to corrupt memory so badly that valgrind crashes > like this, but it is hard. > > Matt > > > [2]PETSC ERROR: > > ------------------------------------------------------------------------ > > [3]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the > > batch system) has told this process to end > > [3]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > > ------------------------------------------------------------------------ > > [3]PETSC ERROR: or see > > http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[3]PETSC > > ERROR: or try http://valgrind.org on linux or man libgmalloc on Apple to > > find memory corruption errors > > [2]PETSC ERROR: Caught signal number 1 Hang up: Some other process (or the > > batch system) has told this process to end > > [2]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > > [2]PETSC ERROR: or see > > http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[2]PETSC > > ERROR: or try http://valgrind.org on linux or man libgmalloc on Apple to > > find memory corruption errors > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the > > batch system) has told this process to end > > [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > > [0]PETSC ERROR: or see > > http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[0]PETSC > > ERROR: or try http://valgrind.org on linux or man libgmalloc on Apple to > > find memory corruption errors > > [3]PETSC ERROR: likely location of problem given in stack below > > [3]PETSC ERROR: --------------------- Stack Frames > > ------------------------------------ > > ------------------------------------------------------------------------ > > [1]PETSC ERROR: [2]PETSC ERROR: Caught signal number 15 Terminate: Somet > > process (or the batch system) has told this process to end > > likely location of problem given in stack below > > [1]PETSC ERROR: [2]PETSC ERROR: Try option -start_in_debugger or > > -on_error_attach_debugger > > --------------------- Stack Frames ------------------------------------ > > [1]PETSC ERROR: or see > > http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[1]PETSC > > ERROR: or try http://valgrind.org on linux or man libgmalloc on Apple to > > find memory corruption errors > > [0]PETSC ERROR: likely location of problem given in stack below > > [0]PETSC ERROR: --------------------- Stack Frames > > ------------------------------------ > > [1]PETSC ERROR: likely location of problem given in stack below > > [1]PETSC ERROR: --------------------- Stack Frames > > ------------------------------------ > > [3]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, > > [3]PETSC ERROR: INSTEAD the line number of the start of the function > > [3]PETSC ERROR: is given. > > [2]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, > > [3]PETSC ERROR: [2]PETSC ERROR: INSTEAD the line number of the start > > of the function > > [2]PETSC ERROR: is given. > > [3] PCApply_Shell line 122 src/ksp/pc/impls/shell/shellpc.c > > [3]PETSC ERROR: [3] PCApply line 346 src/ksp/pc/interface/precon.c > > [3]PETSC ERROR: [3] PCApplyBAorAB line 539 src/ksp/pc/interface/precon.c > > [3]PETSC ERROR: [3] GMREScycle line 133 src/ksp/ksp/impls/gmres/gmres.c > > [2]PETSC ERROR: [0]PETSC ERROR: Note: The EXACT line numbers in the stack > > are not available, > > [2] PCApply_Shell line 122 src/ksp/pc/impls/shell/shellpc.c > > [0]PETSC ERROR: INSTEAD the line number of the start of the function > > [2]PETSC ERROR: [2] PCApply line 346 src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: is given. > > [2]PETSC ERROR: [2] PCApplyBAorAB line 539 src/ksp/pc/interface/precon.c > > [2]PETSC ERROR: [2] GMREScycle line 133 src/ksp/ksp/impls/gmres/gmres.c > > [1]PETSC ERROR: [0]PETSC ERROR: Note: The EXACT line numbers in the stack > > are not available, > > [0] PCApply_Shell line 122 src/ksp/pc/impls/shell/shellpc.c > > [1]PETSC ERROR: INSTEAD the line number of the start of the function > > [0]PETSC ERROR: [0] PCApply line 346 src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: [1]PETSC ERROR: is given. > > [0] PCApplyBAorAB line 539 src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: [0] GMREScycle line 133 src/ksp/ksp/impls/gmres/gmres.c > > [1]PETSC ERROR: [1] PCApply_Shell line 122 src/ksp/pc/impls/shell/shellpc.c > > [1]PETSC ERROR: [1] PCApply line 346 src/ksp/pc/interface/precon.c > > [1]PETSC ERROR: [3]PETSC ERROR: [1] PCApplyBAorAB line 539 > > src/ksp/pc/interface/precon.c > > --------------------- Error Message ------------------------------------ > > [1]PETSC ERROR: [1] GMREScycle line 133 src/ksp/ksp/impls/gmres/gmres.c > > [2]PETSC ERROR: --------------------- Error Message > > ------------------------------------ > > [0]PETSC ERROR: --------------------- Error Message > > ------------------------------------ > > [3]PETSC ERROR: Signal received! > > [3]PETSC ERROR: > > ------------------------------------------------------------------------ > > [3]PETSC ERROR: Petsc Release Version 2.3.3, Patch 8, Fri Nov 16 17:03:40 > > CST 2007 HG revision: 414581156e67e55c761739b0deb119f7590d0f4b > > [3]PETSC ERROR: See docs/changes/index.html for recent updates. > > [3]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > > > Thanks > > > > Rgds, > > Amit > > > > > >
