On Thu, Jan 19, 2023 at 9:13 PM Nicholas Arnold-Medabalimi < narno...@umich.edu> wrote:
> Hi Matt > > Yep, that makes sense and is consistent. > > My question is a little more specific. So let's say I take an initial mesh > and distribute it and get the distribution SF with an overlap of one. > Consider a cell that is a root on process 0 and a leaf on process 1 after > the distribution. > > Will the distribution pointSF have an entry for the cell that is a leaf in > the ghost cell sense? > > I guess, in short does the distribution SF only have entries for the > movement of points that are roots in the ghost SF? > I do not understand the question. Suppose that a certain cell, say 0, in the original distribution goes to two different processes, say 0 and 1, and will happen when you distribute with overlap. Then the migration SF has two leaf entries for that cell, one from process 0 and one from process 1. They both point to root cell 0 on process 0. > Sorry if this is a little unclear. > > Maybe my usage will be a bit clearer. I am generating a distributionSF > (type 2 in your desc) then using that to generate a dof distribution(type > 3) using the section information. I then pass the information from the > initial distribution to new distribution with PetscSFBcast with > MPI_REPLACE. That scatters the vector to the new distribution. I then do > "stuff" and now want to redistribute back. So I pass the same dof > distributionSF but call PetscSFReduce with MPI_REPLACE. My concern is I am > only setting the root cell values on each partition. So if the ghost cells > are part of the distribution SF there will be multiple cells reducing to > the original distribution cell? > Yes, definitely. Thanks, Matt > Thanks > Nicholas > > > On Thu, Jan 19, 2023 at 8:28 PM Matthew Knepley <knep...@gmail.com> wrote: > >> On Thu, Jan 19, 2023 at 11:58 AM Nicholas Arnold-Medabalimi < >> narno...@umich.edu> wrote: >> >>> Hi Petsc Users >>> >>> I'm working with a distribution start forest generated by >>> DMPlexDistribute and PetscSFBcast and Reduce to move data between the >>> initial distribution and the distribution generated by DMPlex Distribute. >>> >>> I'm trying to debug some values that aren't being copied properly and >>> wanted to verify I understand how a redistribution SF works compared with a >>> SF that describes overlapped points. >>> >>> [0] 0 <- (0,7) point 0 on the distributed plex is point 7 on >>> process 0 on the initial distribution >>> [0] 1 <- (0,8) point 1 on the distributed plex is point 8 on >>> process 0 on the initial distribution >>> [0] 2 <- (0,9) >>> [0] 3 <- (0,10) >>> [0] 4 <- (0,11) >>> >>> [1] 0 <- (1,0) point 0 on the distributed plex is point 0 on >>> process 1 on the initial distribution >>> [1] 1 <- (1,1) >>> [1] 2 <- (1,2) >>> [1] 3 <- (0,0) point 3 on the distributed plex is point 0 on >>> process 0 on the initial distribution >>> [1] 4 <- (0,1) >>> [1] 5 <- (0,2) >>> >>> my confusion I think is how does the distributionSF inform of what >>> cells will be leafs on the distribution? >>> >> >> I should eventually write something to clarify this. I am using SF in (at >> least) two different ways. >> >> First, there is a familiar SF that we use for dealing with "ghost" >> points. These are replicated points where one process >> is said to "own" the point and another process is said to hold a "ghost". >> The ghost points are leaves in the SF which >> point back to the root point owned by another process. We call this the >> pointSF for a DM. >> >> Second, we have a migration SF. Here the root points give the original >> point distribution. The leaf points give the new >> point distribution. Thus a PetscSFBcast() pushes points from the original >> to new distribution, which is what we mean >> by a migration. >> >> Third, instead of point values, we might want to communicate fields over >> those points. For this we make new SFes, >> where the numbering does not refer to points, but rather to dofs. >> >> Does this make sense? >> >> Thanks, >> >> Matt >> >> >>> Sincerely >>> Nicholas >>> >>> -- >>> Nicholas Arnold-Medabalimi >>> >>> Ph.D. Candidate >>> Computational Aeroscience Lab >>> University of Michigan >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> <http://www.cse.buffalo.edu/~knepley/> >> > > > -- > Nicholas Arnold-Medabalimi > > Ph.D. Candidate > Computational Aeroscience Lab > University of Michigan > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>