I created a small toy example (attached) that suggests that the verification of
matching PetscFunctionBeginUser() and PetscFunctionReturn() fails when
PetscFunctionReturn() is missing or in some cases when different functions are
missing PetscFunctionBeginUser() or PetscFunctionReturn(). The cases where the
code crashes are much more complex and not easily reproduced in a small toy
example.
Here is the output for 4 cases of running my toy example:
CASE 1: Correct stack. No errors.
./check_petscstack
Testing correct stack...
Layer 0: Calling layer 1
Layer 1: Noop
CASE 2: Missing PetscFunctionBeginUser() in layer 0 is correctly detected and
error message is generated.
./check_petscstack missing_begin
Testing missing PetscFunctionBeginUser() in layer 0...
Layer 0: Calling layer 1
Layer 1: Noop
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Petsc has generated inconsistent data
[0]PETSC ERROR: Invalid stack size 0, pop layer0
/home/baagaard/src/cig/pylith/tests/libtests/utils/TestPetscStack.cc:54.
[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.19.2-475-g71252c2aa25 GIT
Date: 2023-06-15 13:06:55 -0600
[0]PETSC ERROR:
/home/baagaard/scratch/build/gcc-9.3/cig/pylith-debug/tests/libtests/utils/.libs/check_petscstack
on a arch-gcc-9.3_debug named igskcicguwarlng by baagaard Wed Jul 12 11:20:50
2023
[0]PETSC ERROR: Configure options --PETSC_ARCH=arch-gcc-9.3_debug
--with-debugging=1 --with-clanguage=c --with-mpi-compilers=1
--with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1
--with-lgrind=0 --download-chaco=1 --download-parmetis=1 --download-metis=1
--download-triangle --download-ml=1 --download-superlu=1 --with-fc=0
--download-f2cblaslapack --with-hdf5=1
--with-hdf5-dir=/software/baagaard/hdf5-1.12.1/gcc-9.3 --with-zlib=1 CFLAGS=-g
[0]PETSC ERROR: #1 layer0() at
/home/baagaard/src/cig/pylith/tests/libtests/utils/TestPetscStack.cc:54
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_SELF
with errorcode 77.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
CASE 3: Missing PetscFunctionReturn() in layer 1 is not detected.
./check_petscstack missing_return
Testing missing PetscFunctionReturn() in layer 1...
Layer 0: Calling layer 1
Layer 1: Noop
CASE 4: Missing PetscFunctionBeginUser() in layer 0 and missing
PetscFunctionReturn() in layer 1 are not detected.
./check_petscstack missing_both
Testing missing PetsFunctionBeginUser() in layer 0 and PetscFunctionReturn() in
layer 1...
Layer 0: Calling layer 1
Layer 1: Noop
________________________________________
From: Barry Smith <bsm...@petsc.dev>
Sent: Tuesday, July 11, 2023 10:45 PM
To: Aagaard, Brad T
Cc: petsc-users@mcs.anl.gov
Subject: [EXTERNAL] Re: [petsc-users] Verifying matching
PetscFunctionBeginUser() and PetscFunctionReturn()
This email has been received from outside of DOI - Use caution before clicking
on links, opening attachments, or responding.
#define PetscStackPop_Private(stack__, func__) \
do { \
PetscCheckAbort(!stack__.check || stack__.currentsize > 0,
PETSC_COMM_SELF, PETSC_ERR_PLIB, "Invalid stack size %d, pop %s %s:%d.\n",
stack__.currentsize, func__, __FILE__, __LINE__); \
if (--stack__.currentsize < PETSCSTACKSIZE) { \
PetscCheckAbort(!stack__.check ||
stack__.petscroutine[stack__.currentsize] != 1 ||
stack__.function[stack__.currentsize] == (const char *)(func__),
PETSC_COMM_SELF, PETSC_ERR_PLIB, "Invalid stack: push from %s %s:%d. Pop from
%s %s:%d.\n", \
stack__.function[stack__.currentsize],
stack__.file[stack__.currentsize], stack__.line[stack__.currentsize], func__,
__FILE__, __LINE__); \
It is checking on each pop (return) that the current function it is in is the
same function that the begin was called in.
You do not have to call through PetscCall() to have this stuff work (though
we strong recommend adding them).
Even if you have a return and no outstanding begins it should not crash
Maybe you need to see a couple of crashes so we try to figure out what is
going on.
On Jul 11, 2023, at 11:29 PM, Aagaard, Brad T via petsc-users
<petsc-users@mcs.anl.gov> wrote:
PETSc developers,
When I fail to have matching PetscFunctionBeginUser() and PetscFunctionReturn()
in my code, I get segfaults and valgrind reports invalid writes at places in
PETSc where memory is freed. As a result, it is difficult to track down the
actual source of the error. I know there used to be a command line argument for
checking for mismatches in PetscFunctionBeginUser() and PetscFunctionReturn(),
but Matt said it is no longer implemented and the reporting is supposed to be
automatic when debugging is turned on. This automatic reporting does not seem
to be working for me.
Is there a test in PETSc of this automatic reporting that I can run to verify
it works in my build?
Are there any additional requirements for this automatic reporting of
mismatched PetscFunctionBeginUser() and PetscFunctionReturn() in my code? Must
I wrap all PETSc function calls with PetscCall()?
Thanks,
Brad
#include "petsc.h"
#include <iostream>
#include <stdexcept>
#include <string>
PetscErrorCode layer0(const bool missingBegin,
const bool missingReturn);
PetscErrorCode layer1(const bool missingBegin,
const bool missingReturn);
int
main(int argc,
char* argv[]) {
PetscCall(PetscInitialize(&argc, &argv, NULL, NULL));
bool missingBegin = false;
bool missingReturn = false;
if (1 == argc) {
std::cout << "Testing correct stack..." << std::endl;
} else if ((2 == argc) && (0 == strcmp(argv[1], "missing_begin"))) {
std::cout << "Testing missing PetscFunctionBeginUser() in layer 0..." << std::endl;
missingBegin = true;
} else if ((2 == argc) && (0 == strcmp(argv[1], "missing_return"))) {
std::cout << "Testing missing PetscFunctionReturn() in layer 1..." << std::endl;
missingReturn = true;
} else if ((2 == argc) && (0 == strcmp(argv[1], "missing_both"))) {
std::cout << "Testing missing PetsFunctionBeginUser() in layer 0 and PetscFunctionReturn() in layer 1..." << std::endl;
missingBegin = true;
missingReturn = true;
} else {
std::cout << "Unknown argument" << std::endl;
return -1;
}
PetscCall(layer0(missingBegin, missingReturn));
PetscCall(PetscFinalize());
return 0;
}
PetscErrorCode
layer0(const bool missingBegin,
const bool missingReturn) {
if (!missingBegin) {
PetscFunctionBeginUser;
}
PetscErrorCode err = PETSC_SUCCESS;
std::cout << "Layer 0: Calling layer 1" << std::endl;
PetscCall(layer1(false, missingReturn));
PetscFunctionReturn(err);
}
PetscErrorCode
layer1(const bool missingBegin,
const bool missingReturn) {
if (!missingBegin) {
PetscFunctionBeginUser;
}
PetscErrorCode err = PETSC_SUCCESS;
std::cout << "Layer 1: Noop" << std::endl;
if (missingReturn) {
return err;
} else {
PetscFunctionReturn(err);
}
}
