And you can also always use -mg_levels_esteig_ksp_type cg On Tue, Oct 30, 2018 at 2:51 PM Mark Adams <mfad...@lbl.gov> wrote:
> The last two cases were cut off but that is OK. > > Lets just do -pc_gamg_square_graph 0 and please add/do -ksp_view - > mg_levels_ksp_type richardson -pc_mg_levels 5 -gamg_est_ksp_type cg > > Send the grep GAMG again, and the ksp_view output. > > And then do this again with: -pc_gamg_agg_nsmooths > > I think the eigen estimators are messed up on the coarse grids. > > > On Tue, Oct 30, 2018 at 12:39 PM Karin&NiKo <niko.ka...@gmail.com> wrote: > >> Ok, sorry for my misunderstanding and thank you for the clarification. >> >> Le mar. 30 oct. 2018 13:55, Mark Adams <mfad...@lbl.gov> a écrit : >> >>> Nicolas, >>> >>> Smoothed aggregation is fine with shells. see the original SA paper ( >>> https://link.springer.com/article/10.1007/BF02238511). >>> >>> The rotational modes, which are the non-trivial modes that must be >>> supplied, are used in the interpolation. >>> >>> Mark >>> >>> On Tue, Oct 30, 2018 at 5:22 AM Karin&NiKo <niko.ka...@gmail.com> wrote: >>> >>>> Manav, >>>> >>>> How are interpolated the rotational degrees of freedom? AFAIK, when >>>> using smoothed aggregation, the interpolation process tries to mimic linear >>>> interpolation, which can be OK for the displacement DOF but is not for the >>>> rotational DOF using some plate and shell formulations. >>>> This can explain poor convergence of a multilevel approach, which needs >>>> to restrict and extrapolate the unknowns. In order to check this >>>> hypothesis, you can try a test case with zero rotations. >>>> >>>> Nicolas >>>> >>>> Le lun. 29 oct. 2018 à 22:13, Mark Adams via petsc-users < >>>> petsc-users@mcs.anl.gov> a écrit : >>>> >>>>> * the two level results tell us that MG is not doing well on the >>>>> coarse grids. So the coarse grids are the problem. >>>>> >>>>> * Do not worry about timing now. Get the math correct. The two level >>>>> solve is not meant to be a solution just a diagnostic so don't try to >>>>> optimize it by squaring the graph. Use -pc_gamg_square_graph 0. >>>>> >>>>> * It looks like you don't need 4 smoothing steps but lets keep it and >>>>> we can dial it back later. >>>>> >>>>> * This table is interesting. First, you had about 12 iterations >>>>> earlier and I think your rtol was tighter than the default (so the >>>>> iteration could should go down not up). Do you know what change here? >>>>> >>>>> Note, even though -mg_levels_ksp_max_it is not in the ksp_view it does >>>>> work. It is syntactic sugar to just add it to all levels like you did >>>>> manually. >>>>> >>>>> Anyway, these number look reasonable. It is interesting that 3 levels >>>>> ran well but the 4th level ran poorly. This implies we want to slow down >>>>> coarsening on these levels, but ... >>>>> >>>>> First can you please rerun this experiment with -pc_gamg_square_graph >>>>> 0. >>>>> >>>>> Also, please run with -info. This is very noisy but you can grep on >>>>> "GAMG" and send that output to us (about 15 lines). >>>>> >>>>> Thanks, >>>>> Mark >>>>> >>>>> >>>>> >>>>> On Mon, Oct 29, 2018 at 3:34 PM Manav Bhatia <bhatiama...@gmail.com> >>>>> wrote: >>>>> >>>>>> Barry, >>>>>> >>>>>> Here are some quick numbers with the following options on 4 CPUs >>>>>> and 543,606 dofs: >>>>>> >>>>>> -mg_levels_ksp_max_it 4 -pc_gamg_square_graph 1 -pc_gamg_threshold 0. >>>>>> >>>>>> #levels | #KSP Iters >>>>>> ——————————— >>>>>> 2 | 18 >>>>>> 3 | 18 >>>>>> 4 | 40 >>>>>> 5 | 59 >>>>>> >>>>>> -Manav >>>>>> >>>>>> >>>>>> On Oct 29, 2018, at 2:06 PM, Smith, Barry F. <bsm...@mcs.anl.gov> >>>>>> wrote: >>>>>> >>>>>> >>>>>> Exactly how much does it increase with number of levels? Send a >>>>>> chart number of levels and number of iterations. With say >>>>>> -mg_levels_ksp_maxit 4 >>>>>> >>>>>> Thanks >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Oct 29, 2018, at 12:59 PM, Manav Bhatia <bhatiama...@gmail.com> >>>>>> wrote: >>>>>> >>>>>> Thanks for the clarification. >>>>>> >>>>>> I also observed that the number of KSP iterations increases with an >>>>>> increase in the levels of AMG. Is this true, in general, for all/most >>>>>> applications? >>>>>> >>>>>> -Manav >>>>>> >>>>>> On Oct 29, 2018, at 12:53 PM, Jed Brown <j...@jedbrown.org> wrote: >>>>>> >>>>>> Manav Bhatia <bhatiama...@gmail.com> writes: >>>>>> >>>>>> Thanks, Jed. >>>>>> >>>>>> The description says: “ Square the graph, ie. compute A'*A before >>>>>> aggregating it" >>>>>> >>>>>> What is A here? >>>>>> >>>>>> >>>>>> The original matrix, or its "graph" (your 6x6 blocks condensed to >>>>>> scalars). >>>>>> >>>>>> What is the impact of setting this to 0, which led to a very >>>>>> significant increase in the CPU time in my case? >>>>>> >>>>>> >>>>>> The aggregates are formed on the connectivity of your original matrix, >>>>>> so root nodes are aggregated only with their first neighbors, >>>>>> resulting >>>>>> in slower coarsening. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>