Hi all, I used git bisect to determine when the memory need increased. I found that the first "bad" commit is aa690a28a7284adb519c28cb44eae20a2c131c85.
Barry was right, this commit seems to be about an evolution of MatPtAPSymbolic_MPIAIJ_MPIAIJ. You mentioned the option "-matptap_via scalable" but I can't find any information about it. Can you tell me more? Thanks Myriam Le 03/11/19 à 14:40, Mark Adams a écrit : > Is there a difference in memory usage on your tiny problem? I assume no. > > I don't see anything that could come from GAMG other than the RAP > stuff that you have discussed already. > > On Mon, Mar 11, 2019 at 9:32 AM Myriam Peyrounette > <myriam.peyroune...@idris.fr <mailto:myriam.peyroune...@idris.fr>> wrote: > > The code I am using here is the example 42 of PETSc > > (https://www.mcs.anl.gov/petsc/petsc-3.9/src/ksp/ksp/examples/tutorials/ex42.c.html). > Indeed it solves the Stokes equation. I thought it was a good idea > to use an example you might know (and didn't find any that uses > GAMG functions). I just changed the PCMG setup so that the memory > problem appears. And it appears when adding PCGAMG. > > I don't care about the performance or even the result rightness > here, but only about the difference in memory use between 3.6 and > 3.10. Do you think finding a more adapted script would help? > > I used the threshold of 0.1 only once, at the beginning, to test > its influence. I used the default threshold (of 0, I guess) for > all the other runs. > > Myriam > > > Le 03/11/19 à 13:52, Mark Adams a écrit : >> In looking at this larger scale run ... >> >> * Your eigen estimates are much lower than your tiny test >> problem. But this is Stokes apparently and it should not work >> anyway. Maybe you have a small time step that adds a lot of mass >> that brings the eigen estimates down. And your min eigenvalue >> (not used) is positive. I would expect negative for Stokes ... >> >> * You seem to be setting a threshold value of 0.1 -- that is very >> high >> >> * v3.6 says "using nonzero initial guess" but this is not in >> v3.10. Maybe we just stopped printing that. >> >> * There were some changes to coasening parameters in going from >> v3.6 but it does not look like your problem was effected. (The >> coarsening algo is non-deterministic by default and you can see >> small difference on different runs) >> >> * We may have also added a "noisy" RHS for eigen estimates by >> default from v3.6. >> >> * And for non-symetric problems you can try -pc_gamg_agg_nsmooths >> 0, but again GAMG is not built for Stokes anyway. >> >> >> On Tue, Mar 5, 2019 at 11:53 AM Myriam Peyrounette >> <myriam.peyroune...@idris.fr >> <mailto:myriam.peyroune...@idris.fr>> wrote: >> >> I used PCView to display the size of the linear system in >> each level of the MG. You'll find the outputs attached to >> this mail (zip file) for both the default threshold value and >> a value of 0.1, and for both 3.6 and 3.10 PETSc versions. >> >> For convenience, I summarized the information in a graph, >> also attached (png file). >> >> As you can see, there are slight differences between the two >> versions but none is critical, in my opinion. Do you see >> anything suspicious in the outputs? >> >> + I can't find the default threshold value. Do you know where >> I can find it? >> >> Thanks for the follow-up >> >> Myriam >> >> >> Le 03/05/19 à 14:06, Matthew Knepley a écrit : >>> On Tue, Mar 5, 2019 at 7:14 AM Myriam Peyrounette >>> <myriam.peyroune...@idris.fr >>> <mailto:myriam.peyroune...@idris.fr>> wrote: >>> >>> Hi Matt, >>> >>> I plotted the memory scalings using different threshold >>> values. The two scalings are slightly translated (from >>> -22 to -88 mB) but this gain is neglectable. The >>> 3.6-scaling keeps being robust while the 3.10-scaling >>> deteriorates. >>> >>> Do you have any other suggestion? >>> >>> Mark, what is the option she can give to output all the GAMG >>> data? >>> >>> Also, run using -ksp_view. GAMG will report all the sizes of >>> its grids, so it should be easy to see >>> if the coarse grid sizes are increasing, and also what the >>> effect of the threshold value is. >>> >>> Thanks, >>> >>> Matt >>> >>> Thanks >>> >>> Myriam >>> >>> Le 03/02/19 à 02:27, Matthew Knepley a écrit : >>>> On Fri, Mar 1, 2019 at 10:53 AM Myriam Peyrounette via >>>> petsc-users <petsc-users@mcs.anl.gov >>>> <mailto:petsc-users@mcs.anl.gov>> wrote: >>>> >>>> Hi, >>>> >>>> I used to run my code with PETSc 3.6. Since I >>>> upgraded the PETSc version >>>> to 3.10, this code has a bad memory scaling. >>>> >>>> To report this issue, I took the PETSc script >>>> ex42.c and slightly >>>> modified it so that the KSP and PC configurations >>>> are the same as in my >>>> code. In particular, I use a "personnalised" >>>> multi-grid method. The >>>> modifications are indicated by the keyword >>>> "TopBridge" in the attached >>>> scripts. >>>> >>>> To plot the memory (weak) scaling, I ran four >>>> calculations for each >>>> script with increasing problem sizes and >>>> computations cores: >>>> >>>> 1. 100,000 elts on 4 cores >>>> 2. 1 million elts on 40 cores >>>> 3. 10 millions elts on 400 cores >>>> 4. 100 millions elts on 4,000 cores >>>> >>>> The resulting graph is also attached. The scaling >>>> using PETSc 3.10 >>>> clearly deteriorates for large cases, while the one >>>> using PETSc 3.6 is >>>> robust. >>>> >>>> After a few tests, I found that the scaling is >>>> mostly sensitive to the >>>> use of the AMG method for the coarse grid (line 1780 in >>>> main_ex42_petsc36.cc). In particular, the >>>> performance strongly >>>> deteriorates when commenting lines 1777 to 1790 (in >>>> main_ex42_petsc36.cc). >>>> >>>> Do you have any idea of what changed between >>>> version 3.6 and version >>>> 3.10 that may imply such degradation? >>>> >>>> >>>> I believe the default values for PCGAMG changed between >>>> versions. It sounds like the coarsening rate >>>> is not great enough, so that these grids are too large. >>>> This can be set using: >>>> >>>> >>>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGAMGSetThreshold.html >>>> >>>> There is some explanation of this effect on that page. >>>> Let us know if setting this does not correct the situation. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>> Let me know if you need further information. >>>> >>>> Best, >>>> >>>> Myriam Peyrounette >>>> >>>> >>>> -- >>>> Myriam Peyrounette >>>> CNRS/IDRIS - HLST >>>> -- >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they >>>> begin their experiments is infinitely more interesting >>>> than any results to which their experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> <http://www.cse.buffalo.edu/%7Eknepley/> >>> >>> -- >>> Myriam Peyrounette >>> CNRS/IDRIS - HLST >>> -- >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin >>> their experiments is infinitely more interesting than any >>> results to which their experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> <http://www.cse.buffalo.edu/%7Eknepley/> >> >> -- >> Myriam Peyrounette >> CNRS/IDRIS - HLST >> -- >> > > -- > Myriam Peyrounette > CNRS/IDRIS - HLST > -- > -- Myriam Peyrounette CNRS/IDRIS - HLST --
smime.p7s
Description: Signature cryptographique S/MIME