Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

The reason for the large number of RHS's is that the problem originally comes from having to do one solve with 1e10 size matrix. If we make some extra assumptions on our problem, that 1e10 matrix becomes block diagonal with blocks of size 1e6 and all the blocks are the same. The 1e6 is a spatial

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

> On 21 Dec 2023, at 7:38 PM, Junchao Zhang wrote: > > > > > On Thu, Dec 21, 2023 at 5:54 AM Matthew Knepley > wrote: >> On Thu, Dec 21, 2023 at 6:46 AM Sreeram R Venkat > > wrote: >>> Ok, I think the error I'm getting has something to

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

On Thu, Dec 21, 2023 at 5:54 AM Matthew Knepley wrote: > On Thu, Dec 21, 2023 at 6:46 AM Sreeram R Venkat > wrote: > >> Ok, I think the error I'm getting has something to do with how the >> multiple solves are being done in succession. I'll try to see if there's >> anything I'm doing wrong

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

On Thu, Dec 21, 2023 at 6:46 AM Sreeram R Venkat wrote: > Ok, I think the error I'm getting has something to do with how the > multiple solves are being done in succession. I'll try to see if there's > anything I'm doing wrong there. > > One question about the -pc_type lu -ksp_type preonly

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

Would using the CHOLMOD Cholesky factorization ( https://petsc.org/release/manualpages/Mat/MATSOLVERCHOLMOD/) let us do the factorization on device as well? On Wed, Dec 20, 2023 at 1:21 PM Pierre Jolivet wrote: > > > On 20 Dec 2023, at 8:42 AM, Sreeram R Venkat wrote: > > Ok, I think the

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

> On 20 Dec 2023, at 8:42 AM, Sreeram R Venkat wrote: > > Ok, I think the error I'm getting has something to do with how the multiple > solves are being done in succession. I'll try to see if there's anything I'm > doing wrong there. > > One question about the -pc_type lu -ksp_type preonly

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

Ok, I think the error I'm getting has something to do with how the multiple solves are being done in succession. I'll try to see if there's anything I'm doing wrong there. One question about the -pc_type lu -ksp_type preonly method: do you know which parts of the solve (factorization/triangular

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

Unfortunately, I am not able to reproduce such a failure with your input matrix. I’ve used ex79 that I linked previously and the system is properly solved. $ ./ex79 -pc_type hypre -ksp_type hpddm -ksp_hpddm_type cg -ksp_converged_reason -ksp_view_mat ascii::ascii_info -ksp_view_rhs

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

> On 14 Dec 2023, at 11:45 PM, Sreeram R Venkat wrote: > > Thanks, I will try to create a minimal reproducible example. This may take me > some time though, as I need to figure out how to extract only the relevant > parts (the full program this solve is used in is getting quite complex). You

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

Thanks, I will try to create a minimal reproducible example. This may take me some time though, as I need to figure out how to extract only the relevant parts (the full program this solve is used in is getting quite complex). I'll also try out some of the BoomerAMG options to see if that helps.

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

> On 14 Dec 2023, at 8:02 PM, Sreeram R Venkat wrote: > > Hello Pierre, > > Thank you for your reply. I tried out the HPDDM CG as you said, and it seems > to be doing the batched solves, but the KSP is not converging due to a NaN or > Inf being generated. I also noticed there are a lot of

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

Hello Sreeram, KSPCG (PETSc implementation of CG) does not handle solves with multiple columns at once. There is only a single native PETSc KSP implementation which handles solves with multiple columns at once: KSPPREONLY. If you use --download-hpddm, you can use a CG (or GMRES, or more advanced

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

N.B., AMGX interface is a bit experimental. Mark On Thu, Dec 7, 2023 at 4:11 PM Sreeram R Venkat wrote: > Oh, in that case I will try out BoomerAMG. Getting AMGX to build correctly > was also tricky so hopefully the HYPRE build will be easier. > > Thanks, > Sreeram > > On Thu, Dec 7, 2023, 3:03

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

Oh, in that case I will try out BoomerAMG. Getting AMGX to build correctly was also tricky so hopefully the HYPRE build will be easier. Thanks, Sreeram On Thu, Dec 7, 2023, 3:03 PM Pierre Jolivet wrote: > > > On 7 Dec 2023, at 9:37 PM, Sreeram R Venkat wrote: > > Thank you Barry and Pierre; I

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

> On 7 Dec 2023, at 9:37 PM, Sreeram R Venkat wrote: > > Thank you Barry and Pierre; I will proceed with the first option. > > I want to use the AMGX preconditioner for the KSP. I will try it out and see > how it performs. Just FYI, AMGX does not handle systems with multiple RHS, and thus

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

Thank you Barry and Pierre; I will proceed with the first option. I want to use the AMGX preconditioner for the KSP. I will try it out and see how it performs. Thanks, Sreeram On Thu, Dec 7, 2023 at 2:02 PM Pierre Jolivet wrote: > To expand on Barry’s answer, we have observed repeatedly that

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

To expand on Barry’s answer, we have observed repeatedly that MatMatMult with MatAIJ performs better than MatMult with MatMAIJ, you can reproduce this on your own with https://petsc.org/release/src/mat/tests/ex237.c.html. Also, I’m guessing you are using some sort of preconditioner within your

Re: [petsc-users] Matvecs and KSPSolves with multiple vectors

> On Dec 7, 2023, at 1:17 PM, Sreeram R Venkat wrote: > > I have 2 sequential matrices M and R (both MATSEQAIJCUSPARSE of size n x n) > and a vector v of size n*m. v = [v_1 , v_2 ,... , v_m] where v_i has size n. > The data for v can be stored either in column-major or row-major order. Now,

[petsc-users] Matvecs and KSPSolves with multiple vectors

I have 2 sequential matrices M and R (both MATSEQAIJCUSPARSE of size n x n) and a vector v of size n*m. v = [v_1 , v_2 ,... , v_m] where v_i has size n. The data for v can be stored either in column-major or row-major order. Now, I want to do 2 types of operations: 1. Matvecs of the form M*v_i =