On Sun, Jun 24, 2018 at 2:09 PM Smith, Barry F. <bsm...@mcs.anl.gov> wrote:
> > > > On Jun 24, 2018, at 11:15 AM, Mark Adams <mfad...@lbl.gov> wrote: > > > > First, -pc_gamg_threshold 0.8 is crazy. 0.08 is very high. probably > best to start with 0. > > > > This threshold will delete all of your edges in the coarsening graph, > probably, and GAMG will switch to a one level solve (silently although > -info will tell you). > > > > And yes, this looks like a 64 bit indices problem, we have: > > > > #define GAMG_HASH(key) ((((PetscInt)7)*key)%a_tab->size) > > > > We should probably cast everything to (long long) or (unsigned PetscInt) > I guess ... I'm not sure how this should be done exactly. Any ideas from > the peanut gallery? > > Mark, > > It should only be changed if we know the exact mechanism that causes > the failure. It is not obvious to me that just changing > ((((PetscInt)7)*key)%a_tab->size) to ((((PetscInt64)7)*key)%a_tab->size) > should matter. > If 7*key > 2B then you can get a negative number and %a_tab->size will still be a negative number, which triggers: ksp/pc/impls/gamg/agg.c: if (flid < 0) SETERRQ(PETSC_COMM_SELF,PETSC_ERR_PLIB,"Cannot find gid1 in table”); which is the second error that Pierre gets. I don't fully understand the first error but a messed up hash table is certainly a possibility. And maybe we should cast the hell out of it to be safe: #define GAMG_HASH(key) (PetscInt)((((PetscInt64)7)*(PetscInt64)key)% (PetscInt64)a_tab->size) I am a bit puzzled that this works. I would think this would be more of a problem but maybe everyone just uses 64 bit ints. I have dealt with this many times and I am puzzled that I left it in a state that looks so wrong to me now. Perhaps Pierre's complicated thing is causing this somehow. > Barry > > > > > > > > > On Sun, Jun 24, 2018 at 2:34 AM Pierre Jolivet < > pierre.joli...@enseeiht.fr> wrote: > > Hello, > > Any ideas on how to debug this? > > [406]PETSC ERROR: key 328214936 not found in table > > [406]PETSC ERROR: See > http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > [406]PETSC ERROR: Petsc Development GIT revision: v3.9.2-603-gceafe64 > GIT Date: 2018-06-10 12:46:16 -0500 > > [406]PETSC ERROR: > /ccc/work/cont003/rndm/rndm/FreeFem-sources/src/mpi/FreeFem++-mpi on a > arch-linux2-c-debug-bullxmpi named curie2698 by jolivetp Sun Jun 24 > 07:06:07 2018 > > [406]PETSC ERROR: Configure options --with-scalar-type=real > --with-hypre-dir=arch-linux2-c-opt-bullxmpi > --with-metis-dir=arch-linux2-c-opt-bullxmpi > --with-mumps-dir=arch-linux2-c-opt-bullxmpi --with-mkl_cpardiso=1 > --with-mkl_pardiso=1 --with-parmetis-dir=arch-linux2-c-opt-bullxmpi > --with-ptscotch-dir=arch-linux2-c-opt-bullxmpi > --with-sowing-dir=arch-linux2-c-opt-bullxmpi > --with-blacs-include=/ccc/products/mkl-18.0.1.163/default/ > 18.0.1.163/mkl/include > --with-blacs-lib=/ccc/products/mkl-18.0.1.163/default/ > 18.0.1.163/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so > --with-blaslapack-dir=/ccc/products/mkl-18.0.1.163/default/ > 18.0.1.163/mkl/lib/intel64 --with-x=0 --with-debugging=1 > --with-errorchecking=1 > --with-mkl_pardiso-dir=/ccc/products/mkl-18.0.1.163/default/18.0.1.163/mkl > --with-mkl_cpardiso-dir=/ccc/products/mkl-18.0.1.163/default/ > 18.0.1.163/mkl > --with-scalapack-include=/ccc/products/mkl-18.0.1.163/default/ > 18.0.1.163/mkl/include > --with-scalapack-lib="[/ccc/products/mkl-18.0.1.163/default/ > 18.0.1.163/mkl/lib/intel64/libmkl_scalapack_lp64.so,/ccc/products/mkl-18.0.1.163/default/18.0.1.163/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so]" > --with-fortran-bindings=0 PETSC_ARCH=arch-linux2-c-debug-bullxmpi > > [406]PETSC ERROR: #1 PCGAMGHashTableFind() line 106 in > /ccc/work/cont003/rndm/rndm/petsc/include/petsc/private/pcgamgimpl.h > > [406]PETSC ERROR: #2 smoothAggs() line 526 in > /ccc/work/cont003/rndm/rndm/petsc/src/ksp/pc/impls/gamg/agg.c > > [406]PETSC ERROR: #3 PCGAMGCoarsen_AGG() line 970 in > /ccc/work/cont003/rndm/rndm/petsc/src/ksp/pc/impls/gamg/agg.c > > [406]PETSC ERROR: #4 PCSetUp_GAMG() line 518 in > /ccc/work/cont003/rndm/rndm/petsc/src/ksp/pc/impls/gamg/gamg.c > > [406]PETSC ERROR: #5 PCSetUp() line 932 in > /ccc/work/cont003/rndm/rndm/petsc/src/ksp/pc/interface/precon.c > > [406]PETSC ERROR: #6 KSPSetUp() line 381 in > /ccc/work/cont003/rndm/rndm/petsc/src/ksp/ksp/interface/itfunc.c > > > > For smaller problems, this works fine. Should I compile using 64 bit > indices? > > I’m using -pc_type gamg -pc_gamg_threshold 0.8 -pc_gamg_agg_nsmooths 1 > -pc_gamg_sym_graph true -pc_gamg_square_graph 10 > > > > Thanks, > > Pierre > >