Dear PETSc developers/users,
I’m solving linear PDEs on a regular grid with high-order finite
differences, assembling an MPIAIJ matrix to solve linear systems or
eigenvalue problems. I’ve been using vertex major, natural ordering for
the parallelism with PetscSplitOwnership (yielding rectangular slices of
the physical domain) and wanted to move to DMDA to have a more
square-ish domain decomposition and minimize communication between
processes.
However, my application is memory critical, and I have finely-tuned
matrix preallocation routines for allocating memory “optimally”. It
seems the memory of a DMDA matrix is allocated along the value of the
stencil width of DMDACreate and the manual says about it
“These DMDA stencils have nothing directly to do with any finite
difference stencils one might chose to use for a discretization”
And despite reading the manual pages there must be something I do not
understand in the DM topology, what is that "stencil width" for then? I
will not use ghost values for my FD-method, right?
I was then wondering if I could just create a MPIAIJ matrix, and with a
PETSc routine get the global indices of the domain for each process: in
other words, an equivalent of PetscSplitOwnership that gives me the DMDA
unknown ordering. So I can feed and loop on that in my preallocation and
assembly routines.
Thanks very much,
Thibaut
