Could you give me some pointers on how to debug this? Currently I am creating/destroying a KSP that uses MUMPS, and sometimes when it is destroyed I get a crash. It only happens in parallel. Here is the backtrace from petsc-dev for a crashed process:
#0 0x00002b3a97f32c53 in PetscTrFreeDefault (aa=0x3ee0ec701867ea69, line=40, function=0x2b3a99509b34 "PetscComposedQuantitiesDestroy", file=0x2b3a99509888 "/home/jfe/local/petsc-dev/src/sys/objects/destroy.c", dir=0x2b3a995098bc "") at /home/jfe/local/petsc-dev/src/sys/memory/mtr.c:284 #1 0x00002b3a97f64b89 in PetscComposedQuantitiesDestroy (obj=0x1f090350) at /home/jfe/local/petsc-dev/src/sys/objects/destroy.c:40 #2 0x00002b3a980fc756 in VecScatterDestroy (ctx=0x19e6e938) at /home/jfe/local/petsc-dev/src/vec/vec/utils/vscat.c:1658 #3 0x00002b3a9884dd19 in MatDestroy_MUMPS (A=0x199a6cc0) at /home/jfe/local/petsc-dev/src/mat/impls/aij/mpi/mumps/mumps.c:484 #4 0x00002b3a981d376c in MatDestroy (A=0x14aaa5d0) at /home/jfe/local/petsc-dev/src/mat/interface/matrix.c:1046 #5 0x00002b3a98b158ba in PCReset_LU (pc=0x198cf370) at /home/jfe/local/petsc-dev/src/ksp/pc/impls/factor/lu/lu.c:174 #6 0x00002b3a98bfdd59 in PCReset (pc=0x198cf370) at /home/jfe/local/petsc-dev/src/ksp/pc/interface/precon.c:80 #7 0x00002b3a98cd0511 in KSPReset (ksp=0x14abb3f0) at /home/jfe/local/petsc-dev/src/ksp/ksp/interface/itfunc.c:718 #8 0x00002b3a98cd107d in KSPDestroy (ksp=0xd874a8) at /home/jfe/local/petsc-dev/src/ksp/ksp/interface/itfunc.c:755 I think a similar thing is happening in petsc-3.1-p8, here's a backtrace from there: #0 0x00002b20b2a964f8 in VMPI_Win_free () from /usr/local/encap/platform_mpi-8.01/lib/linux_amd64/libmpi.so.1 #1 0x00002b20b00d27ff in VecScatterDestroy_PtoP (ctx=0x1fe7bcf0) at vpscat.c:171 #2 0x00002b20b01018d5 in VecScatterDestroy (ctx=0x1fe7bcf0) at vscat.c:1638 #3 0x00002b20afd1249d in MatDestroy_MPIAIJ (mat=0x1cacb4d0) at mpiaij.c:909 #4 0x00002b20afc4c2cf in MatDestroy (A=0x1cacb4d0) at matrix.c:876 #5 0x00002b20afe972b0 in PCDestroy (pc=0x1032cf20) at precon.c:88 #6 0x00002b20afbbe78e in KSPDestroy (ksp=0x105f22d0) at itfunc.c:695 Is this likely memory corruption on my side or on the PETSc side, or something else entirely? I seem unable to reproduce it with an example, unfortunately. Running it through valgrind may take a week. Thanks, John -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120221/fa506be1/attachment.html>