On Wed, Dec 6, 2023 at 8:10 PM Barry Smith <bsm...@petsc.dev> wrote:
> > Depending on the serial library you may not need to split the vector > into DMDA vectors with DMStagVecSplitToDMDA() for each component. Just > global to natural and scatter to zero on the full vector, now the full > vector is on the first rank and you can access what you need in that one > vector if possible. > Does DMStag have a GlobalToNatural? Also, the serial code would have to have identical interleaving. Thanks, Matt > On Dec 6, 2023, at 6:37 PM, Colton Bryant < > coltonbryant2...@u.northwestern.edu> wrote: > > Ah excellent! I was not aware of the ability to preallocate the objects > and migrate them each time. > > Thanks! > -Colton > > On Wed, Dec 6, 2023 at 5:18 PM Matthew Knepley <knep...@gmail.com> wrote: > >> On Wed, Dec 6, 2023 at 5:54 PM Colton Bryant < >> coltonbryant2...@u.northwestern.edu> wrote: >> >>> Hello, >>> >>> I am working on a code in which a DMSTAG object is used to solve a fluid >>> flow problem and I need to gather this flow data on a single process to >>> interact with an existing (serial) library at each timestep of my >>> simulation. After looking around the solution I've tried is: >>> >>> -use DMStagVecSplitToDMDA to extract vectors of each component of the >>> flow >>> -use DMDACreateNaturalVector and DMDAGlobalToNatural to get the >>> components naturally ordered >>> -use VecScatterCreateToZero to set up and then do the scatter to gather >>> on the single process >>> >>> Unless I'm misunderstanding something this method results in a lot of >>> memory allocation/freeing happening at each step of the evolution and I was >>> wondering if there is a way to directly perform such a scatter from the >>> DMSTAG object without splitting as I'm doing here. >>> >> >> 1) You can see here: >> >> >> https://petsc.org/main/src/dm/impls/stag/stagda.c.html#DMStagVecSplitToDMDA >> >> that this function is small. You can do the DMDA creation manually, and >> then just call DMStagMigrateVecDMDA() each time, which will not create >> anything. >> >> 2) You can create the natural vector upfront, and just scatter each time. >> >> 3) You can create the serial vector upfront, and just scatter each time. >> >> This is some data movement. You can compress the g2n and 2zero scatters >> using >> >> https://petsc.org/main/manualpages/PetscSF/PetscSFCompose/ >> >> as an optimization. >> >> Thanks, >> >> Matt >> >> >>> Any advice would be much appreciated! >>> >>> Best, >>> Colton Bryant >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> <http://www.cse.buffalo.edu/~knepley/> >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>