On Dec 20, 2011, at 10:33 AM, Dave Nystrom wrote: > Hi Mark, > > I would like to try GAMG on some of my linear solves. Could you suggest how > to get started? Is it more complicated than something like: > > -ksp_type cg -pc_type gamg
This is a good start. for scalar SPD problems '-pc_gamg_type sa' is good (this might be the default, use -help to see. Mark > > I'm guessing I should first try it on one of my easier linear solves. I have > 5 of them that would have a block size of 1. Are the other GAMG option > defaults good to start with or should I be trying to configure them as well? > If so, I'm not familiar enough with multigrid to know off hand how to do > that. > > Thanks, > > Dave > > Mark F. Adams writes: >> >> On Dec 2, 2011, at 6:06 PM, Dave Nystrom wrote: >> >>> Mark F. Adams writes: >>>> It sounds like you have a symmetric positive definite systems like du/dt - >>>> div(alpha(x) grad)u. The du/dt term makes the systems easier to solve. >>>> I'm guessing your hard system does not have this mass term and so is >>>> purely elliptic. Multigrid is well suited for this type of problem, but >>>> the vector nature requires some thought. You could use PETSc AMG -pc_type >>>> gamg but you need to tell it that you have a system of two dof/vertex. >>>> You can do that with something like: >>>> >>>> ierr = MatSetBlockSize( mat, 2 ); CHKERRQ(ierr); >>>> >>>> For the best results from GAMG you need to give it null space information >>>> but we can worry about that later. >>> >>> Hi Mark, >>> >>> I have been interested in trying some of the multigrid capabilities in >>> petsc. I'm not sure I remember seeing GAMG so I guess I should go look for >>> that. >> >> GAMG is pretty new. >> >>> I have tried sacusp and sacusppoly but did not get good results on >>> this particular linear system. >>> In particular, sacusppoly seems broken. I >>> can't get it to work even with the petsc >>> src/ksp/ksp/examples/tutorials/ex2.c >>> example. Thrust complains about an invalid device pointer I believe. >>> Anyway, I can get the other preconditioners to work just fine on this petsc >>> example problem. When I try sacusp on this matrix for the case of >>> generating >>> a rhs from a known solution vector, the computed solution seems to diverge >>> from the exact solution. We also have an interface to an external agmg >>> package which is not able to solve this problem >>> but works well on the other 5 >>> linear solves. So I'd like to try more from the multigrid toolbox but do >>> not >>> know much about how to supply the extra stuff that these packages often >>> need. >>> >>> So, it sounds like you are suggesting that I try gamg and that I could at >>> least try it out without having to initially supply lots of additional info. >>> So I will take a look at gamg. >>> >> >> There are many things that can break a solver but most probably want to know >> that its a system so if you can set the block size and try gamg then that >> would be a good start. >> >> Mark >> >>> Thanks, >>> >>> Dave >>> >>>> Mark >>>> >>>> On Nov 30, 2011, at 8:15 AM, Matthew Knepley wrote: >>>> >>>>> On Wed, Nov 30, 2011 at 12:41 AM, Dave Nystrom <dnystrom1 at comcast.net> >>>>> wrote: >>>>> I have a linear system in a code that I have interfaced to petsc that is >>>>> taking about 80 percent of the run time per timestep. This linear system >>>>> is >>>>> a symmetric block banded matrix where the blocks are 2x2. The matrix >>>>> looks >>>>> as follows: >>>>> >>>>> 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 >>>>> 1X X Y Y Y >>>>> 2X X X Y Y Y >>>>> 3 X X X Y Y Y >>>>> 4 X X X Y Y Y >>>>> 5 X X X Y Y Y >>>>> 6 X X X Y Y Y >>>>> 7 X X X Y Y Y >>>>> 8 X X X Y Y Y >>>>> 9 X X X Y Y Y >>>>> 0 X X X Y Y Y >>>>> 1 X X X Y Y Y >>>>> 2 X X X Y Y Y >>>>> 3Z X X X Y Y Y >>>>> 4Z Z X X X Y Y Y >>>>> 5Z Z Z X X X Y Y Y >>>>> 6 Z Z Z X X X Y Y Y >>>>> 7 Z Z Z X X X Y Y Y >>>>> 8 Z Z Z X X X Y Y Y >>>>> 9 Z Z Z X X X Y Y Y >>>>> 0 Z Z Z X X X Y Y Y >>>>> >>>>> So in my diagram above, X, Y and Z are 2x2 blocks. The symmetry of the >>>>> matrix requires that X_ij = transpose(X_ji) and Y_ij = transpose(Z_ji). >>>>> So >>>>> far, I have just input this matrix to petsc without indicating that it was >>>>> block banded with 2x2 blocks. I have also not told petsc that the matrix >>>>> is >>>>> symmetric. And I have allowed petsc to decide the best way to store the >>>>> matrix. >>>>> >>>>> I can solve this linear system over the course of a run using -ksp_type >>>>> preonly -pc_type lu. But that will not scale very well to larger problems >>>>> that I want to solve. I can also solve this system over the course of a >>>>> run >>>>> using -ksp_type cg -pc_type jacobi -vec_type cusp -mat_type aijcusp. >>>>> However, over the course of a run, the iteration count ranges from 771 to >>>>> 47300. I have also tried sacusp, ainvcusp, sacusppoly, ilu(k) and icc(k) >>>>> with k=0. The sacusppoly preconditioner fails because of a thrust error >>>>> related to an invalid device pointer, if I am remembering correctly. I >>>>> reported this problem to petsc-maint a while back and have also reported >>>>> it >>>>> for the cusp bugtracker. But it does not appear that anyone has really >>>>> looked into the bug. For the other preconditioners of sacusp, ilu(k) and >>>>> icc(k), they do not result in convergence to a solution and the runs fail. >>>>> >>>>> All preconditioners are custom. Have you done a literature search for PCs >>>>> known to work for this problem? Can yu say anything about the spectrum of >>>>> the >>>>> operator? conditioning? what is the principal symbol (if its a PDE)? The >>>>> pattern >>>>> is not enough to recommend a PC. >>>>> >>>>> Matt >>>>> >>>>> I'm wondering if there are suggestions of other preconditioners in petsc >>>>> that >>>>> I should try. The only third party package that I have tried is the >>>>> txpetscgpu package. I have not tried hypre or any of the multigrid >>>>> preconditioners yet. I'm not sure how difficult it is to try those >>>>> packages. Anyway, so far I have not found a preconditioner available in >>>>> petsc that provides a robust solution to this problem and would be >>>>> interested >>>>> in any suggestions that anyone might have of things to try. >>>>> >>>>> I'd be happy to provide additional info and am planning on packaging up a >>>>> couple of examples of the matrix and rhs for people I am interacting with >>>>> at >>>>> Tech-X and EMPhotonics. So I'd be happy to provide the matrix examples >>>>> for >>>>> this forum as well if anyone wants a copy. >>>>> >>>>> Thanks, >>>>> >>>>> Dave >>>>> >>>>> -- >>>>> Dave Nystrom >>>>> >>>>> phone: 505-661-9943 (home office) >>>>> 505-662-6893 (home) >>>>> skype: dave.nystrom76 >>>>> email: dnystrom1 at comcast.net >>>>> smail: 219 Loma del Escolar >>>>> Los Alamos, NM 87544 >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which >>>>> their experiments lead. >>>>> -- Norbert Wiener >>>> >>> >> >
