Hi Hong, Thanks for looking into this. Here is the workflow that I might use:
MatLUFactorSymbolic(F,A,perm,iperm,&info); // get memory estimates from MUMPS e.g. INFO(3), INFOG(16), INFOG(17) // find available memory on the system e.g. RAM size if (estimated_memory > available_memory) { // inform and stop; or // switch MUMPS to out-of-core factorization ICNTL(22) = 1; } else { // set appropriate settings for in-core factorization } // Now we call the solve and inside if MatLUFactorSymbolic is already called then it should be skipped EPSSolve(eps); Thanks, Varun On Mon, Jan 17, 2022 at 9:18 AM Zhang, Hong <hzh...@mcs.anl.gov> wrote: > Varun, > I am trying to find a way to enable you to switch options after > MatLUFactorSymbolic(). > A hack is modifying the flag 'mumps->matstruc' > inside MatLUFactorSymbolic_AIJMUMPS() and MatFactorNumeric_MUMPS(). > > My understanding of what you want is: > // collect mumps memory info > ... > MatLUFactorSymbolic(F,A,perm,iperm,&info); > printMumpsMemoryInfo(F); > //--------- > if (memory is available) { > EPSSolve(eps); --> skip calling of MatLUFactorSymbolic() > } else { > //out-of-core (OOC) option in MUMPS > } > > Am I correct? I'll let you know once I work out a solution. > Hong > > ------------------------------ > *From:* Varun Hiremath <varunhirem...@gmail.com> > *Sent:* Sunday, January 16, 2022 10:10 PM > *To:* Zhang, Hong <hzh...@mcs.anl.gov> > *Cc:* Jose E. Roman <jro...@dsic.upv.es>; Peder Jørgensgaard Olesen via > petsc-users <petsc-users@mcs.anl.gov> > *Subject:* Re: [petsc-users] PETSc MUMPS interface > > Hi Jose, Hong, > > Thanks for the explanation. I have verified using -log_view that > MatLUFactorSymbolic > is indeed getting called twice. > > Hong, we use MUMPS solver for other things, and we typically run the > symbolic analysis first and get memory estimates to ensure that we have > enough memory available to run the case. If the available memory is not > enough, we can stop or switch to the out-of-core (OOC) option in MUMPS. We > wanted to do the same when using MUMPS via SLEPc/PETSc. Please let me know > if there are other ways of getting these memory stats and switching options > during runtime with PETSc. > Appreciate your help! > > Thanks, > Varun > > On Sun, Jan 16, 2022 at 4:01 PM Zhang, Hong <hzh...@mcs.anl.gov> wrote: > > Varun, > I believe Jose is correct. You may verify it by running your code with > option '-log_view', then check the number of calls to MatLUFactorSym. > > I guess I can add a flag in PCSetUp() to check if user has already called > MatLUFactorSymbolic() and wants to skip it. Normally, users simply allocate > sufficient memory in the symbolic factorization. Why do you want to check > it? > Hong > > ------------------------------ > *From:* Jose E. Roman <jro...@dsic.upv.es> > *Sent:* Sunday, January 16, 2022 5:11 AM > *To:* Varun Hiremath <varunhirem...@gmail.com> > *Cc:* Zhang, Hong <hzh...@mcs.anl.gov>; Peder Jørgensgaard Olesen via > petsc-users <petsc-users@mcs.anl.gov> > *Subject:* Re: [petsc-users] PETSc MUMPS interface > > Hong may give a better answer, but if you look at PCSetUp_LU() > https://petsc.org/main/src/ksp/pc/impls/factor/lu/lu.c.html#PCSetUp_LU > you will see that MatLUFactorSymbolic() is called unconditionally during > the first PCSetUp(). Currently there is no way to check if the user has > already called MatLUFactorSymbolic(). > > Jose > > > > El 16 ene 2022, a las 10:40, Varun Hiremath <varunhirem...@gmail.com> > escribió: > > > > Hi Hong, > > > > Thank you, this is very helpful! > > > > Using this method I am able to get the memory estimates before the > actual solve, however, I think my code may be causing the symbolic > factorization to be run twice. Attached is my code where I am using SLEPc > to compute eigenvalues, and I use MUMPS for factorization. I have commented > above the code that computes the memory estimates, could you please check > and tell me if this would cause the symbolic factor to be computed twice (a > second time inside EPSSolve?), as I am seeing a slight increase in the > overall computation time? > > > > Regards, > > Varun > > > > On Wed, Jan 12, 2022 at 7:58 AM Zhang, Hong <hzh...@mcs.anl.gov> wrote: > > PCFactorSetMatSolverType(pc,MATSOLVERMUMPS); > > PCFactorSetUpMatSolverType(pc); > > PCFactorGetMatrix(pc,&F); > > > > MatLUFactorSymbolic(F,A,...) > > You must provide row and column permutations etc, > petsc/src/mat/tests/ex125.c may give you a clue on how to get these inputs. > > > > Hong > > > > > > From: petsc-users <petsc-users-boun...@mcs.anl.gov> on behalf of > Junchao Zhang <junchao.zh...@gmail.com> > > Sent: Wednesday, January 12, 2022 9:03 AM > > To: Varun Hiremath <varunhirem...@gmail.com> > > Cc: Peder Jørgensgaard Olesen via petsc-users <petsc-users@mcs.anl.gov> > > Subject: Re: [petsc-users] PETSc MUMPS interface > > > > Calling PCSetUp() before KSPSetUp()? > > > > --Junchao Zhang > > > > > > On Wed, Jan 12, 2022 at 3:00 AM Varun Hiremath <varunhirem...@gmail.com> > wrote: > > Hi All, > > > > I want to collect MUMPS memory estimates based on the initial symbolic > factorization analysis before the actual numerical factorization starts to > check if the estimated memory requirements fit the available memory. > > > > I am following the steps from > https://petsc.org/main/src/ksp/ksp/tutorials/ex52.c.html > > > > PCFactorSetMatSolverType(pc,MATSOLVERMUMPS); > > PCFactorSetUpMatSolverType(pc); > > PCFactorGetMatrix(pc,&F); > > > > KSPSetUp(ksp); > > MatMumpsGetInfog(F,...) > > > > But it appears KSPSetUp calls both symbolic and numerical factorization. > So is there some other way to get these statistics before the actual > factorization starts? > > > > Thanks, > > Varun > > <slepc_eps_mumps_test.cpp> > >