problem with PetscSynchronizedFPrintf (fwd)
-- Forwarded message -- Date: Sun, 20 Jul 2008 22:39:13 -0700 (PDT) From: Shi Jin jinzish...@yahoo.com Subject: problem with PetscSynchronizedFPrintf To: petsc-dev at mcs.anl.gov Hi, I recently updated my petsc-dev and found my previously working code is having trouble running in parallel. I've narrowed it down to the calls to PetscSynchronizedFPrintf. Even if I don't have those, I will have errors at the end of my MPI runs like: [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: PetscMemoryShowUsage() line 367 in src/sys/memory/mtr.c [1]PETSC ERROR: PetscFinalize() line 675 in src/sys/objects/pinit.c [1]PETSC ERROR: completeSection() line 163 in main.cxx [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 Since the same code was running fine before the update. I am wondering if it is something recently introduced. Thanks a lot. -- Shi Jin, PhD
problem with PetscSynchronizedFPrintf (fwd)
Looks like this could be related to changeset 9e3fbe04a926 -- mprint.h -- - charstring[QUEUESTRINGSIZE]; + char *string; So now strings needs to use malloc()/free()? Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c Satish On Mon, 21 Jul 2008, Satish Balay wrote: -- Forwarded message -- Date: Sun, 20 Jul 2008 22:39:13 -0700 (PDT) From: Shi Jin jinzishuai at yahoo.com Subject: problem with PetscSynchronizedFPrintf To: petsc-dev at mcs.anl.gov Hi, I recently updated my petsc-dev and found my previously working code is having trouble running in parallel. I've narrowed it down to the calls to PetscSynchronizedFPrintf. Even if I don't have those, I will have errors at the end of my MPI runs like: [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: PetscMemoryShowUsage() line 367 in src/sys/memory/mtr.c [1]PETSC ERROR: PetscFinalize() line 675 in src/sys/objects/pinit.c [1]PETSC ERROR: completeSection() line 163 in main.cxx [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 Since the same code was running fine before the update. I am wondering if it is something recently introduced. Thanks a lot. -- Shi Jin, PhD
problem with PetscSynchronizedFPrintf (fwd)
I made this change because synchronized printing was cutting off my stuff. Now the queue is dynamically allocated so it can accomodate any size string. I tried ex7 and it runs fine for me, even under valgrind. Shi, can you reproduce this with a Sieve example? Matt On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay balay at mcs.anl.gov wrote: Looks like this could be related to changeset 9e3fbe04a926 -- mprint.h -- - charstring[QUEUESTRINGSIZE]; + char *string; So now strings needs to use malloc()/free()? Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c Satish On Mon, 21 Jul 2008, Satish Balay wrote: -- Forwarded message -- Date: Sun, 20 Jul 2008 22:39:13 -0700 (PDT) From: Shi Jin jinzishuai at yahoo.com Subject: problem with PetscSynchronizedFPrintf To: petsc-dev at mcs.anl.gov Hi, I recently updated my petsc-dev and found my previously working code is having trouble running in parallel. I've narrowed it down to the calls to PetscSynchronizedFPrintf. Even if I don't have those, I will have errors at the end of my MPI runs like: [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: PetscMemoryShowUsage() line 367 in src/sys/memory/mtr.c [1]PETSC ERROR: PetscFinalize() line 675 in src/sys/objects/pinit.c [1]PETSC ERROR: completeSection() line 163 in main.cxx [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 Since the same code was running fine before the update. I am wondering if it is something recently introduced. Thanks a lot. -- Shi Jin, PhD -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
problem with PetscSynchronizedFPrintf (fwd)
Matt - Perhaps ex7 is a different Problem. But it does crash for me. And I noticed the following code: ierr = PetscNew(struct _PrintfQueue,next);CHKERRQ(ierr); if (queue) {queue-next = next; queue = next;} else {queuebase = queue = next;} queuelength++; string = next-string; ierr = PetscMemzero(string,QUEUESTRINGSIZE);CHKERRQ(ierr); i.e 'next' is allocated - here is no malloc for 'next-string'. But its being zeroed in PetscMemzero() - hence the crash with ex7 for me. Satish - asterix:/home/balay/spetsc/src/dm/ao/examples/testsmpiexec -n 2 ./ex7 Number of elements in ordering 10 PETSc-App App-PETSc 00 00 15 12 21 24 36 36 42 48 57 51 63 63 78 75 84 87 99 99 [1]PETSC ERROR: - Error Message [1]PETSC ERROR: Null argument, when expecting valid pointer! [1]PETSC ERROR: Trying to zero at a null pointer! [1]PETSC ERROR: [1]PETSC ERROR: Petsc Development Version 2.3.3, Patch 13, unknown HG revision: unknown [1]PETSC ERROR: See docs/changes/index.html for recent updates. [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [1]PETSC ERROR: See docs/index.html for manual pages. [1]PETSC ERROR: [1]PETSC ERROR: ./ex7 on a asterix64 named asterix by balay Mon Jul 21 14:51:02 2008 [1]PETSC ERROR: Libraries linked from /home/balay/spetsc/lib [1]PETSC ERROR: Configure run at Mon Jul 21 10:38:52 2008 [1]PETSC ERROR: Configure options --with-mpi-dir=/home/balay/soft/linux64/mpich2-1.0.7 --with-shared=1 PETSC_ARCH=asterix64 [1]PETSC ERROR: [0] Number of indices in (stride) set 5 [0] 0 0 [0] 1 2 [0] 2 4 [0] 3 6 [0] 4 8 [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: ISView_Stride() line 231 in src/vec/is/impls/stride/stride.c [1]PETSC ERROR: ISView() line 398 in src/vec/is/interface/index.c [1]PETSC ERROR: main() line 31 in src/dm/ao/examples/tests/ex7.c application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1[cli_1]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1 [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 asterix:/home/balay/spetsc/src/dm/ao/examples/tests On Mon, 21 Jul 2008, Matthew Knepley wrote: I made this change because synchronized printing was cutting off my stuff. Now the queue is dynamically allocated so it can accomodate any size string. I tried ex7 and it runs fine for me, even under valgrind. Shi, can you reproduce this with a Sieve example? Matt On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay balay at mcs.anl.gov wrote: Looks like this could be related to changeset 9e3fbe04a926 -- mprint.h -- - charstring[QUEUESTRINGSIZE]; + char *string; So now strings needs to use malloc()/free()? Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c Satish
problem with PetscSynchronizedFPrintf (fwd)
Matt, You have to be careful here. You use PetscMalloc() here, are you positive this routine will never be called BEFORE PetscMalloc() is setup for (sometime in PetscInitialize), or after PetscMalloc() is taken down (somewhere in PetscFinalize())? Is there someway you can make sure that it is safe? In a few places on setup I have to use a raw malloc instead of PetscMalloc(), I don't want you to have to do that here. I'm not saying to change back the code, just to be careful and check it. Barry On Jul 21, 2008, at 2:47 PM, Matthew Knepley wrote: I made this change because synchronized printing was cutting off my stuff. Now the queue is dynamically allocated so it can accomodate any size string. I tried ex7 and it runs fine for me, even under valgrind. Shi, can you reproduce this with a Sieve example? Matt On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay balay at mcs.anl.gov wrote: Looks like this could be related to changeset 9e3fbe04a926 -- mprint.h -- - charstring[QUEUESTRINGSIZE]; + char *string; So now strings needs to use malloc()/free()? Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c Satish On Mon, 21 Jul 2008, Satish Balay wrote: -- Forwarded message -- Date: Sun, 20 Jul 2008 22:39:13 -0700 (PDT) From: Shi Jin jinzishuai at yahoo.com Subject: problem with PetscSynchronizedFPrintf To: petsc-dev at mcs.anl.gov Hi, I recently updated my petsc-dev and found my previously working code is having trouble running in parallel. I've narrowed it down to the calls to PetscSynchronizedFPrintf. Even if I don't have those, I will have errors at the end of my MPI runs like: [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: PetscMemoryShowUsage() line 367 in src/sys/memory/ mtr.c [1]PETSC ERROR: PetscFinalize() line 675 in src/sys/objects/pinit.c [1]PETSC ERROR: completeSection() line 163 in main.cxx [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 Since the same code was running fine before the update. I am wondering if it is something recently introduced. Thanks a lot. -- Shi Jin, PhD -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
problem with PetscSynchronizedFPrintf (fwd)
I will check all the occurences of SynchronizedPrintf() in the source. Matt On Mon, Jul 21, 2008 at 3:03 PM, Barry Smith bsmith at mcs.anl.gov wrote: Matt, You have to be careful here. You use PetscMalloc() here, are you positive this routine will never be called BEFORE PetscMalloc() is setup for (sometime in PetscInitialize), or after PetscMalloc() is taken down (somewhere in PetscFinalize())? Is there someway you can make sure that it is safe? In a few places on setup I have to use a raw malloc instead of PetscMalloc(), I don't want you to have to do that here. I'm not saying to change back the code, just to be careful and check it. Barry On Jul 21, 2008, at 2:47 PM, Matthew Knepley wrote: I made this change because synchronized printing was cutting off my stuff. Now the queue is dynamically allocated so it can accomodate any size string. I tried ex7 and it runs fine for me, even under valgrind. Shi, can you reproduce this with a Sieve example? Matt On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay balay at mcs.anl.gov wrote: Looks like this could be related to changeset 9e3fbe04a926 -- mprint.h -- - charstring[QUEUESTRINGSIZE]; + char *string; So now strings needs to use malloc()/free()? Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c Satish On Mon, 21 Jul 2008, Satish Balay wrote: -- Forwarded message -- Date: Sun, 20 Jul 2008 22:39:13 -0700 (PDT) From: Shi Jin jinzishuai at yahoo.com Subject: problem with PetscSynchronizedFPrintf To: petsc-dev at mcs.anl.gov Hi, I recently updated my petsc-dev and found my previously working code is having trouble running in parallel. I've narrowed it down to the calls to PetscSynchronizedFPrintf. Even if I don't have those, I will have errors at the end of my MPI runs like: [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: PetscMemoryShowUsage() line 367 in src/sys/memory/mtr.c [1]PETSC ERROR: PetscFinalize() line 675 in src/sys/objects/pinit.c [1]PETSC ERROR: completeSection() line 163 in main.cxx [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 Since the same code was running fine before the update. I am wondering if it is something recently introduced. Thanks a lot. -- Shi Jin, PhD -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
problem with PetscSynchronizedFPrintf (fwd)
Matt, While you are at it could you please fix the buggy code in mprint.c? make PETSC_DIR=/Users/bsmith/Src/petsc-dev-for-fixes PETSC_ARCH=arch- icc-ifort /Users/bsmith/Src/petsc-dev-for-fixes/arch-icc-ifort/bin/mpicc -o mprint.o -c -wd1572 -Qoption,cpp,--extended_float_type -g -I/Users/ bsmith/Src/petsc-dev-for-fixes -I/Users/bsmith/Src/petsc-dev-for-fixes/ arch-icc-ifort/include -I/Users/bsmith/Src/petsc-dev-for-fixes/include -I/usr/X11/include -I/Users/bsmith/Src/petsc-dev-for-fixes/arch-icc- ifort/include-D__SDIR__='src/sys/fileio/' mprint.c mprint.c(350): warning #592: variable fullLength is used before its value is set while(fullLength = next-size) { ^ On Jul 21, 2008, at 3:30 PM, Matthew Knepley wrote: Pushed a fix. I missed the Viewer when I corrected the raw call. That should not be separate code. Matt On Mon, Jul 21, 2008 at 2:54 PM, Satish Balay balay at mcs.anl.gov wrote: Matt - Perhaps ex7 is a different Problem. But it does crash for me. And I noticed the following code: ierr = PetscNew(struct _PrintfQueue,next);CHKERRQ(ierr); if (queue) {queue-next = next; queue = next;} else {queuebase = queue = next;} queuelength++; string = next-string; ierr = PetscMemzero(string,QUEUESTRINGSIZE);CHKERRQ(ierr); i.e 'next' is allocated - here is no malloc for 'next-string'. But its being zeroed in PetscMemzero() - hence the crash with ex7 for me. Satish - asterix:/home/balay/spetsc/src/dm/ao/examples/testsmpiexec -n 2 ./ ex7 Number of elements in ordering 10 PETSc-App App-PETSc 00 00 15 12 21 24 36 36 42 48 57 51 63 63 78 75 84 87 99 99 [1]PETSC ERROR: - Error Message [1]PETSC ERROR: Null argument, when expecting valid pointer! [1]PETSC ERROR: Trying to zero at a null pointer! [1]PETSC ERROR: [1]PETSC ERROR: Petsc Development Version 2.3.3, Patch 13, unknown HG revision: unknown [1]PETSC ERROR: See docs/changes/index.html for recent updates. [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [1]PETSC ERROR: See docs/index.html for manual pages. [1]PETSC ERROR: [1]PETSC ERROR: ./ex7 on a asterix64 named asterix by balay Mon Jul 21 14:51:02 2008 [1]PETSC ERROR: Libraries linked from /home/balay/spetsc/lib [1]PETSC ERROR: Configure run at Mon Jul 21 10:38:52 2008 [1]PETSC ERROR: Configure options --with-mpi-dir=/home/balay/soft/ linux64/mpich2-1.0.7 --with-shared=1 PETSC_ARCH=asterix64 [1]PETSC ERROR: [0] Number of indices in (stride) set 5 [0] 0 0 [0] 1 2 [0] 2 4 [0] 3 6 [0] 4 8 [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: ISView_Stride() line 231 in src/vec/is/impls/stride/ stride.c [1]PETSC ERROR: ISView() line 398 in src/vec/is/interface/index.c [1]PETSC ERROR: main() line 31 in src/dm/ao/examples/tests/ex7.c application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1[cli_1]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1 [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 asterix:/home/balay/spetsc/src/dm/ao/examples/tests On Mon, 21 Jul 2008, Matthew Knepley wrote: I made this change because synchronized printing was cutting off my stuff. Now the queue is dynamically allocated so it can accomodate any size string. I tried ex7 and it runs fine for me, even under valgrind. Shi, can you reproduce this with a Sieve example? Matt On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay balay at mcs.anl.gov wrote: Looks like this could be related to changeset 9e3fbe04a926 -- mprint.h -- - charstring[QUEUESTRINGSIZE]; + char *string; So now strings needs to use malloc()/free()? Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c Satish -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
problem with PetscSynchronizedFPrintf (fwd)
Fixed. Matt On Mon, Jul 21, 2008 at 3:32 PM, Barry Smith bsmith at mcs.anl.gov wrote: Matt, While you are at it could you please fix the buggy code in mprint.c? make PETSC_DIR=/Users/bsmith/Src/petsc-dev-for-fixes PETSC_ARCH=arch-icc-ifort /Users/bsmith/Src/petsc-dev-for-fixes/arch-icc-ifort/bin/mpicc -o mprint.o -c -wd1572 -Qoption,cpp,--extended_float_type -g -I/Users/bsmith/Src/petsc-dev-for-fixes -I/Users/bsmith/Src/petsc-dev-for-fixes/arch-icc-ifort/include -I/Users/bsmith/Src/petsc-dev-for-fixes/include -I/usr/X11/include -I/Users/bsmith/Src/petsc-dev-for-fixes/arch-icc-ifort/include -D__SDIR__='src/sys/fileio/' mprint.c mprint.c(350): warning #592: variable fullLength is used before its value is set while(fullLength = next-size) { ^ On Jul 21, 2008, at 3:30 PM, Matthew Knepley wrote: Pushed a fix. I missed the Viewer when I corrected the raw call. That should not be separate code. Matt On Mon, Jul 21, 2008 at 2:54 PM, Satish Balay balay at mcs.anl.gov wrote: Matt - Perhaps ex7 is a different Problem. But it does crash for me. And I noticed the following code: ierr = PetscNew(struct _PrintfQueue,next);CHKERRQ(ierr); if (queue) {queue-next = next; queue = next;} else {queuebase = queue = next;} queuelength++; string = next-string; ierr = PetscMemzero(string,QUEUESTRINGSIZE);CHKERRQ(ierr); i.e 'next' is allocated - here is no malloc for 'next-string'. But its being zeroed in PetscMemzero() - hence the crash with ex7 for me. Satish - asterix:/home/balay/spetsc/src/dm/ao/examples/testsmpiexec -n 2 ./ex7 Number of elements in ordering 10 PETSc-App App-PETSc 00 00 15 12 21 24 36 36 42 48 57 51 63 63 78 75 84 87 99 99 [1]PETSC ERROR: - Error Message [1]PETSC ERROR: Null argument, when expecting valid pointer! [1]PETSC ERROR: Trying to zero at a null pointer! [1]PETSC ERROR: [1]PETSC ERROR: Petsc Development Version 2.3.3, Patch 13, unknown HG revision: unknown [1]PETSC ERROR: See docs/changes/index.html for recent updates. [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [1]PETSC ERROR: See docs/index.html for manual pages. [1]PETSC ERROR: [1]PETSC ERROR: ./ex7 on a asterix64 named asterix by balay Mon Jul 21 14:51:02 2008 [1]PETSC ERROR: Libraries linked from /home/balay/spetsc/lib [1]PETSC ERROR: Configure run at Mon Jul 21 10:38:52 2008 [1]PETSC ERROR: Configure options --with-mpi-dir=/home/balay/soft/linux64/mpich2-1.0.7 --with-shared=1 PETSC_ARCH=asterix64 [1]PETSC ERROR: [0] Number of indices in (stride) set 5 [0] 0 0 [0] 1 2 [0] 2 4 [0] 3 6 [0] 4 8 [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: ISView_Stride() line 231 in src/vec/is/impls/stride/stride.c [1]PETSC ERROR: ISView() line 398 in src/vec/is/interface/index.c [1]PETSC ERROR: main() line 31 in src/dm/ao/examples/tests/ex7.c application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1[cli_1]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 85) - process 1 [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 asterix:/home/balay/spetsc/src/dm/ao/examples/tests On Mon, 21 Jul 2008, Matthew Knepley wrote: I made this change because synchronized printing was cutting off my stuff. Now the queue is dynamically allocated so it can accomodate any size string. I tried ex7 and it runs fine for me, even under valgrind. Shi, can you reproduce this with a Sieve example? Matt On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay balay at mcs.anl.gov wrote: Looks like this could be related to changeset 9e3fbe04a926 -- mprint.h -- - charstring[QUEUESTRINGSIZE]; + char *string; So now strings needs to use malloc()/free()? Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c Satish -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
problem with PetscSynchronizedFPrintf (fwd)
On Mon, Jul 21, 2008 at 3:03 PM, Barry Smith bsmith at mcs.anl.gov wrote: Matt, You have to be careful here. You use PetscMalloc() here, are you positive this routine will never be called BEFORE PetscMalloc() is setup for (sometime in PetscInitialize), or after PetscMalloc() is taken down (somewhere in PetscFinalize())? Is there someway you can make sure that it is safe? In a few places on setup I have to use a raw malloc instead of PetscMalloc(), I don't want you to have to do that here. I'm not saying to change back the code, just to be careful and check it. I looked at all the code. There are no calls in PetscInitialize() and I cannot find anything else that might be called before PetscMalloc is set. Matt Barry On Jul 21, 2008, at 2:47 PM, Matthew Knepley wrote: I made this change because synchronized printing was cutting off my stuff. Now the queue is dynamically allocated so it can accomodate any size string. I tried ex7 and it runs fine for me, even under valgrind. Shi, can you reproduce this with a Sieve example? Matt On Mon, Jul 21, 2008 at 2:37 PM, Satish Balay balay at mcs.anl.gov wrote: Looks like this could be related to changeset 9e3fbe04a926 -- mprint.h -- - charstring[QUEUESTRINGSIZE]; + char *string; So now strings needs to use malloc()/free()? Matt - you can reproduce this issue with dm/ao/examples/tests/ex7.c Satish On Mon, 21 Jul 2008, Satish Balay wrote: -- Forwarded message -- Date: Sun, 20 Jul 2008 22:39:13 -0700 (PDT) From: Shi Jin jinzishuai at yahoo.com Subject: problem with PetscSynchronizedFPrintf To: petsc-dev at mcs.anl.gov Hi, I recently updated my petsc-dev and found my previously working code is having trouble running in parallel. I've narrowed it down to the calls to PetscSynchronizedFPrintf. Even if I don't have those, I will have errors at the end of my MPI runs like: [1]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c [1]PETSC ERROR: PetscViewerASCIISynchronizedPrintf() line 859 in src/sys/viewer/impls/ascii/filev.c [1]PETSC ERROR: PetscMemoryShowUsage() line 367 in src/sys/memory/mtr.c [1]PETSC ERROR: PetscFinalize() line 675 in src/sys/objects/pinit.c [1]PETSC ERROR: completeSection() line 163 in main.cxx [0]0:Return code = 0, signaled with Interrupt [0]1:Return code = 85 Since the same code was running fine before the update. I am wondering if it is something recently introduced. Thanks a lot. -- Shi Jin, PhD -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener