"Smith, Barry F. via petsc-users" <petsc-us...@mcs.anl.gov> writes:
> The reason this worked for 4 processes is that the largest count in that > case was roughly 6,653,750,976/4 which does fit into an int. PETSc only needs > to know the number of nonzeros on each process, it doesn't need to know the > amount across all the processors. In other words you may want to use a > different PETSC_ARCH (different configuration) for small number of processors > and large number depending on how large your problem is. Or you can always > use 64 bit integers at a little performance and memory cost. We could consider always using 64-bit ints for quantities like row starts, keeping column indices (the "heavy" part) in 32-bit. This may become a more frequent issue with fatter nodes and many GPUs potentially being driven by a single MPI rank.