We need all the error output for the errors you got below to understand why the errors are happening.
> On Feb 3, 2023, at 11:41 AM, Karthikeyan Chockalingam - STFC UKRI > <karthikeyan.chockalin...@stfc.ac.uk> wrote: > > Hello Barry, > > I would like to better understand pc_type redistribute usage. > > I am plan to use pc_type redistribute in the context of adaptive mesh > refinement on a structured grid in 2D. My base mesh (level 0) is indexed from > 0 to N-1 elements and refined mesh (level 1) is indexed from 0 to 4(N-1) > elements. When I construct system matrix A on (level 1); I probably only use > 20% of 4(N-1) elements, however the indexes are scattered in the range of 0 > to 4(N-1). That leaves 80% of the rows and columns of the system matrix A on > (level 1) to be zero. From your earlier response, I believe this would be a > use case for petsc_type redistribute. Indeed the linear solve will be more efficient if you use the redistribute solver. But I don't understand your plan. With adaptive refinement I would just create the two matrices, one for the initial grid on which you solve the system, this will be a smaller matrix and then create a new larger matrix for the refined grid (and discard the previous matrix). > > > Question (1) > > > If N is really large, I would have to allocate memory of size 4(N-1) for the > system matrix A on (level 1). How does pc_type redistribute help? Because, I > did end up allocating memory for a large system, where most of the rows and > columns are zeros. Is most of the allotted memory not wasted? Is this the > correct usage? See above > > > Question (2) > > > I tried using pc_type redistribute for a two level system. > I have attached the output only for (level 1) > The solution converges to right solution but still petsc outputs some error > messages. > > [0]PETSC ERROR: WARNING! There are option(s) set that were not used! Could be > the program crashed before they were used or a spelling mistake, etc! > [0]PETSC ERROR: Option left: name:-options_left (no value) > > But the there were no unused options > > #PETSc Option Table entries: > -ksp_type preonly > -options_left > -pc_type redistribute > -redistribute_ksp_converged_reason > -redistribute_ksp_monitor_true_residual > -redistribute_ksp_type cg > -redistribute_ksp_view > -redistribute_pc_type jacobi > #End of PETSc Option Table entries > There are no unused options. > Program ended with exit code: 0 > I cannot explain this > > Question (2) > > [0;39m[0;49m[0]PETSC ERROR: Object is in wrong state > [0]PETSC ERROR: Matrix is missing diagonal entry in row 0 (65792) > > What does this error message imply? Given I only use 20% of 4(N-1) indexes, I > can imagine most of the diagonal entrees are zero. Is my understanding > correct? > > > Question (3) > > [0]PETSC ERROR: #5 MatZeroRowsColumnsIS() at > /Users/karthikeyan.chockalingam/AMReX/SRC_PKG/petsc/src/mat/interface/matrix.c:6124 > > I am using MatZeroRowsColumnsIS to set the homogenous Dirichelet boundary. I > don’t follow why I get this error message as the linear system converges to > the right solution. > > Thank you for your help. > > Kind regards, > Karthik. > > > > From: Barry Smith <bsm...@petsc.dev <mailto:bsm...@petsc.dev>> > Date: Tuesday, 10 January 2023 at 18:50 > To: Chockalingam, Karthikeyan (STFC,DL,HC) > <karthikeyan.chockalin...@stfc.ac.uk > <mailto:karthikeyan.chockalin...@stfc.ac.uk>> > Cc: petsc-users@mcs.anl.gov <mailto:petsc-users@mcs.anl.gov> > <petsc-users@mcs.anl.gov <mailto:petsc-users@mcs.anl.gov>> > Subject: Re: [petsc-users] Eliminating rows and columns which are zeros > > > Yes, after the solve the x will contain correct values for ALL the > locations including the (zeroed out rows). You use case is exactly what > redistribute it for. > > Barry > > > > On Jan 10, 2023, at 11:25 AM, Karthikeyan Chockalingam - STFC UKRI > <karthikeyan.chockalin...@stfc.ac.uk > <mailto:karthikeyan.chockalin...@stfc.ac.uk>> wrote: > > Thank you Barry. This is great! > > I plan to solve using ‘-pc_type redistribute’ after applying the Dirichlet bc > using > MatZeroRowsColumnsIS(A, isout, 1, x, b); > > While I retrieve the solution data from x (after the solve) – can I index > them using the original ordering (if I may say that)? > > Kind regards, > Karthik. > > From: Barry Smith <bsm...@petsc.dev <mailto:bsm...@petsc.dev>> > Date: Tuesday, 10 January 2023 at 16:04 > To: Chockalingam, Karthikeyan (STFC,DL,HC) > <karthikeyan.chockalin...@stfc.ac.uk > <mailto:karthikeyan.chockalin...@stfc.ac.uk>> > Cc: petsc-users@mcs.anl.gov <mailto:petsc-users@mcs.anl.gov> > <petsc-users@mcs.anl.gov <mailto:petsc-users@mcs.anl.gov>> > Subject: Re: [petsc-users] Eliminating rows and columns which are zeros > > > https://petsc.org/release/docs/manualpages/PC/PCREDISTRIBUTE/#pcredistribute > -pc_type redistribute > > > It does everything for you. Note that if the right hand side for any of the > "zero" rows is nonzero then the system is inconsistent and the system does > not have a solution. > > Barry > > > > > On Jan 10, 2023, at 10:30 AM, Karthikeyan Chockalingam - STFC UKRI via > petsc-users <petsc-users@mcs.anl.gov <mailto:petsc-users@mcs.anl.gov>> wrote: > > Hello, > > I am assembling a MATIJ of size N, where a very large number of rows (and > corresponding columns), are zeros. I would like to potentially eliminate them > before the solve. > > For instance say N=7 > > 0 0 0 0 0 0 0 > 0 1 -1 0 0 0 0 > 0 -1 2 0 0 0 -1 > 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 -1 0 0 0 1 > > I would like to reduce it to a 3x3 > > 1 -1 0 > -1 2 -1 > 0 -1 1 > > I do know the size N. > > Q1) How do I do it? > Q2) Is it better to eliminate them as it would save a lot of memory? > Q3) At the moment, I don’t know which rows (and columns) have the zero > entries but with some effort I probably can find them. Should I know which > rows (and columns) I am eliminating? > > Thank you. > > Karthik. > This email and any attachments are intended solely for the use of the named > recipients. If you are not the intended recipient you must not use, disclose, > copy or distribute this email or any of its attachments and should notify the > sender immediately and delete this email from your system. UK Research and > Innovation (UKRI) has taken every reasonable precaution to minimise risk of > this email or any attachments containing viruses or malware but the recipient > should carry out its own virus and malware checks before opening the > attachments. UKRI does not accept any liability for any losses or damages > which the recipient may sustain due to presence of any viruses. > > > <petsc_redistribute.txt>
