You'll usually have a GlobalToLocal operation for each rank to get the halo data it needs, then either a LocalToGlobal to collect the result (e.g., finite element methods) or the local compute will write directly into the owned portion of the global vector. If you're doing the communication "raw", then you may find VecScatter or PetscSF useful to perform the necessary communication.
"Tu, Jiannan" <jiannan...@uml.edu> writes: > Hi Barry, > > Following your instructions I implemented the matrix-free method to solve a > large linear equation system resulted from a surface integration equation. > The KSP solver works fine for single process, but it is problematic with > multiple processes. The problem is that each process only can access its own > part of solution vector so that each process only conducts part of > matrix-vector multiplication. MPI can be used to assemble these partial > matrix-vector multiplication together. Does Petsc provide any ways to > implement multi-process matrix-free method? > > Thanks, > Jiannan > ________________________________ > From: Barry Smith <bsm...@petsc.dev> > Sent: Tuesday, May 24, 2022 2:12 PM > To: Tu, Jiannan <jiannan...@uml.edu> > Cc: petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov> > Subject: Re: [petsc-users] Using matrix-free with KSP > > This e-mail originated from outside the UMass Lowell network. > ________________________________ > > You can use MatCreateMFFD > https://petsc.org/main/docs/manualpages/Mat/MatCreateMFFD/<https://urldefense.com/v3/__https://petsc.org/main/docs/manualpages/Mat/MatCreateMFFD/__;!!PVKG_VDCxu5g!stxq2NWGVjUihvHBQC9Rlk0aHVoowcy0PTQYMcQxQ4DqOEC05KSw6TmstSXKckiUgelHzy4ue-d10-zlXkw$> > MatMFFDSetFunction MatSetFromOptions MatMFFDSetBase and provide the matrix > to KSP. Note you will need to use -pc_type none or provide your own custom > preconditioner with > https://petsc.org/main/docs/manualpages/PC/PCSHELL/<https://urldefense.com/v3/__https://petsc.org/main/docs/manualpages/PC/PCSHELL/__;!!PVKG_VDCxu5g!stxq2NWGVjUihvHBQC9Rlk0aHVoowcy0PTQYMcQxQ4DqOEC05KSw6TmstSXKckiUgelHzy4ue-d1cA4fKGw$> > > > > On May 24, 2022, at 1:21 PM, Tu, Jiannan > <jiannan...@uml.edu<mailto:jiannan...@uml.edu>> wrote: > > I want to use a matrix-free matrix to solve a large linear equation system > because the matrix is too large to be stored. Petsc user manual describes > matrix-free method for SNES with examples. The matrix-free matrices section > explains how to set up such a matrix, but I can't find any example of > matrix-free method with KSP. I am wondering how to apply the method to KSP > iterative solver in parallel. > > I greatly appreciate your help for me to understand this topic. > > Jiannan Tu
