2012/1/9 Jed Brown <jedbrown at mcs.anl.gov> > Cc'ing petsc-dev to get other opinions. > > Great.
> On Mon, Jan 9, 2012 at 06:56, Stefano Zampini <stefano.zampini at > gmail.com>wrote: > >> Because of the way I built the code, different coarse problem types set >> communicators and operators in a different way. Specifically, >> >> - MULTILEVEL_BDDC: mat_type=MATIS, coarse_comm=comm of parent, >> coarse_pc=PCBDDC. >> - PARALLEL_BDDC: mat_type=MATMPIAIJ, coarse_comm=comm of parent, >> coarse_pc=PCREDUNDANT. >> > > The only distinction here is the matrix type. > > >> - SEQUENTIAL_BDDC (only rank 0 creates it): mat_type=MATSEQAIJ, >> coarse_comm=PETSC_COMM_SELF, coarse_pc=PCLU. >> > > This should be handled through a mechanism like PCREDUNDANT so that the > caller does not need to know. This "reduce to subcommunicator, broadcast > the result" would be simple to add to PCREDUNDANT. > > >> - REPLICATED_BDDC (all ranks creates the same sequential ksp): >> mat_type=MATSEQAIJ, coarse_comm=PETSC_COMM_SELF, coarse_pc=PCLU. >> > > What does this have over using PCREDUNDANT as in case 2? > > >> >> >>> >>> >>>> indeed a possible user code >>>> >>>> PCBDDCGetCoarseKSP(pcbddc,&coarse_ksp) >>>> KSPGetPC(coarse_ksp,&coarse_pc) >>>> PCSetType(coarse_pc,PCBDDC) (or something else) >>>> PCSetup(pcbddc) >>>> >>>> breaks at the second line since pcbddc->coarse_ksp doesn't exist before >>>> calling PCSetup. >>>> >>> >>> You can make PCBDDCGetCoarseKSP() return a KSP that has not had its type >>> set yet. >>> >>> >> >> By creating the KSP object internally to the function call? In this case >> the problem remains since the user should need to know a priori the type of >> communicator associated with the KSP. Can we change the communicator after >> we created the KSP object? >> > > No, but I think that the coarse communicator should probably always be the > same as for the whole problem, and then use PCREDUNDANT or similar if we > want to reduce to a subcomm. > > Except for MULTILEVEL_BDDC, all other approaches solve exactly the coarse problem thus there are no differences in theoretical results. In practice, they are there for performances optimizations. REPLICATED_BDDC is performed directly using allgatherv operations and it is an heritage from my previous codes. I can drop it. If you can manage the SEQUENTIAL_BDDC case directly in PCREDUNDANT, then we will be half-way. Enums on coarse communications type will be eliminated too as communications can be all made using VecScatters. > But this brings up an issue that also appears in GAMG, how to expose user > access to coarse KSPs even though they are created on adaptively sized > subcomms. If we configure strictly through the options database, then we > are okay, but what if we have to pass any information directly to the KSP? > We can set operators and call PCSetUp(), then pull out levels, but what > should be returned on ranks that are not participating? > Coarse KSP in my multilevel BDDC implementation is created on the same comm of parent; ranks not partecipating in coarse comm will create all objects needed my the levels even if they have a local dimension equal to zero. -- Stefano -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120109/9ca2371e/attachment.html>