Dear PETSc/Tao team, there is a bug in the voector interface: In the functionVecNorm, see, eg. https://petsc.org/release/src/vec/vec/interface/rvector.c.html#VecNorm line 197 the check for consistency in line 214 is done on the wrong communicator. The communicator should be PETSC_COMM_SELF.
Otherwise the program may hang when PetscCheck is executed.
Please find a minimal example attached. Kind regards, Stephan Köhler -- Stephan Köhler TU Bergakademie Freiberg Institut für numerische Mathematik und Optimierung Akademiestraße 6 09599 Freiberg Gebäudeteil Mittelbau, Zimmer 2.07 Telefon: +49 (0)3731 39-3188 (Büro)
#include "petscvec.h" int main(int argc, char **args) { PetscMPIInt size, rank; Vec vec; PetscReal norm; PetscBool flg = PETSC_FALSE, minflg = PETSC_FALSE; MPI_Comm comm; PetscScalar *xx; PetscCall(PetscInitialize(&argc, &args, PETSC_NULLPTR, PETSC_NULLPTR)); comm = PETSC_COMM_WORLD; PetscCallMPI(MPI_Comm_size(comm, &size)); PetscCallMPI(MPI_Comm_rank(comm, &rank)); PetscCheck(size > 1, comm, PETSC_ERR_ARG_WRONG, "example should be called with more than 1 MPI rank."); PetscCall(VecCreateMPI(comm, (rank+1)*10, PETSC_DETERMINE, &vec)); PetscCall(VecSet(vec, 1.0)); PetscCall(VecNorm(vec, NORM_INFINITY, &norm)); PetscSynchronizedPrintf(PETSC_COMM_WORLD, "rank = %d, size = %d, norm = %lf\n", rank, size, norm); PetscSynchronizedFlush(comm, PETSC_STDOUT); if(rank == 0) { PetscCall(VecGetArrayWrite(vec, &xx)); PetscCall(VecRestoreArrayWrite(vec, &xx)); } PetscCall(VecNormAvailable(vec, NORM_INFINITY, &flg, &norm)); PetscSynchronizedPrintf(comm, "rank = %d, size = %d, flg = %d, norm = %lf\n", rank, size, flg, norm); PetscSynchronizedFlush(comm, PETSC_STDOUT); PetscCall(MPIU_Allreduce(&flg, &minflg, 1, MPIU_BOOL, MPI_LAND, PetscObjectComm((PetscObject)vec))); /* wrong */ PetscCheck(flg == minflg, PetscObjectComm((PetscObject)vec), PETSC_ERR_ARG_WRONGSTATE, "Some MPI processes have cached norm, others do not. This may happen when some MPI processes call VecGetArray() and some others do not."); /* this is correct */ // PetscCheck(flg == minflg, PETSC_COMM_SELF, PETSC_ERR_ARG_WRONGSTATE, "Some MPI processes have cached norm, others do not. This may happen when some MPI processes call VecGetArray() and some others do not."); PetscCall(VecDestroy(&vec)); PetscFinalize(); return 0; }
OpenPGP_0xC9BF2C20DFE9F713.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature