I built petsc 3.13.4 and got results similar to the old ones. I am
attaching the log-view output file here.
Mohammad
On Tue, Mar 23, 2021 at 6:49 PM Satish Balay via petsc-users <
[email protected]> wrote:
> On Tue, 23 Mar 2021, Matthew Knepley wrote:
>
> > On Tue, Mar 23, 2021 at 9:08 PM Junchao Zhang <[email protected]>
> > wrote:
> >
> > > In the new log, I saw
> > >
> > > Summary of Stages: ----- Time ------ ----- Flop ------ ---
> Messages --- -- Message Lengths -- -- Reductions --
> > > Avg %Total Avg %Total Count
> %Total Avg %Total Count %Total
> > > 0: Main Stage: 5.4095e+00 2.3% 4.3700e+03 0.0% 4.764e+05
> 3.0% 3.135e+02 1.0% 2.244e+04 12.6% 1: Solute_Assembly:
> 1.3977e+02 59.4% 7.3353e+09 4.6% 3.263e+06 20.7% 1.278e+03
> 26.9% 1.059e+04 6.0%
> > >
> > >
> > > But I didn't see any event in this stage had a cost close to 140s. What
> > > happened?
> > >
> >
> > This is true, but all the PETSc operations are speeding up by a factor
> 2x.
> > It is hard to believe these were run on the same machine.
> > For example, VecScale speeds up!?! So it is not network, or
> optimizations.
> > I cannot explain this.
>
> * Using C compiler:
> /home/mohammad/Programs/petsc/arch-linux-c-opt/bin/mpicc -Wall
> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector
> -fvisibility=hidden -Ofast -march=native -mtune=native
>
> Perhaps the CPU is new enough that '-march=native -mtune=native' makes a
> difference between '18.04 to 20.04'?
>
> You can build 3.13.4 again and see if the numbers are similar to the old
> or new numbers you currently have..
>
> * --download-fblaslapack --download-openblas
>
> You should use one or the other - but not both. Perhaps one is using
> openblas in thread mode [vs single thread for the other]?
>
> Satish
>
>
> >
> > Matt
> >
> > --- Event Stage 1: Solute_Assembly
> > >
> > > BuildTwoSided 3531 1.0 2.8025e+0026.3 0.00e+00 0.0 3.6e+05
> 4.0e+00 3.5e+03 1 0 2 0 2 1 0 11 0 33 0
> > > BuildTwoSidedF 3531 1.0 2.8678e+0013.2 0.00e+00 0.0 7.1e+05
> 3.6e+03 3.5e+03 1 0 5 17 2 1 0 22 62 33 0
> > > VecScatterBegin 7062 1.0 7.1911e-02 1.9 0.00e+00 0.0 7.1e+05
> 3.5e+02 0.0e+00 0 0 5 2 0 0 0 22 6 0 0
> > > VecScatterEnd 7062 1.0 2.1248e-01 3.0 1.60e+06 2.7 0.0e+00
> 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 73
> > > SFBcastOpBegin 3531 1.0 2.6516e-02 2.4 0.00e+00 0.0 3.6e+05
> 3.5e+02 0.0e+00 0 0 2 1 0 0 0 11 3 0 0
> > > SFBcastOpEnd 3531 1.0 9.5041e-02 4.7 0.00e+00 0.0 0.0e+00
> 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > > SFReduceBegin 3531 1.0 3.8955e-02 2.1 0.00e+00 0.0 3.6e+05
> 3.5e+02 0.0e+00 0 0 2 1 0 0 0 11 3 0 0
> > > SFReduceEnd 3531 1.0 1.3791e-01 3.9 1.60e+06 2.7 0.0e+00
> 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 112
> > > SFPack 7062 1.0 6.5591e-03 2.5 0.00e+00 0.0 0.0e+00
> 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > > SFUnpack 7062 1.0 7.4186e-03 2.1 1.60e+06 2.7 0.0e+00
> 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2080
> > > MatAssemblyBegin 3531 1.0 4.7846e+00 1.1 0.00e+00 0.0 7.1e+05
> 3.6e+03 3.5e+03 2 0 5 17 2 3 0 22 62 33 0
> > > MatAssemblyEnd 3531 1.0 1.5468e+00 2.7 1.68e+07 2.7 0.0e+00
> 0.0e+00 0.0e+00 0 0 0 0 0 1 2 0 0 0 104
> > > MatZeroEntries 3531 1.0 3.0998e-02 1.2 0.00e+00 0.0 0.0e+00
> 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > >
> > >
> > > --Junchao Zhang
> > >
> > >
> > >
> > > On Tue, Mar 23, 2021 at 5:24 PM Mohammad Gohardoust <
> [email protected]>
> > > wrote:
> > >
> > >> Thanks Dave for your reply.
> > >>
> > >> For sure PETSc is awesome :D
> > >>
> > >> Yes, in both cases petsc was configured with --with-debugging=0 and
> > >> fortunately I do have the old and new -log-veiw outputs which I
> attached.
> > >>
> > >> Best,
> > >> Mohammad
> > >>
> > >> On Tue, Mar 23, 2021 at 1:37 AM Dave May <[email protected]>
> wrote:
> > >>
> > >>> Nice to hear!
> > >>> The answer is simple, PETSc is awesome :)
> > >>>
> > >>> Jokes aside, assuming both petsc builds were configured with
> > >>> —with-debugging=0, I don’t think there is a definitive answer to your
> > >>> question with the information you provided.
> > >>>
> > >>> It could be as simple as one specific implementation you use was
> > >>> improved between petsc releases. Not being an Ubuntu expert, the
> change
> > >>> might be associated with using a different compiler, and or a more
> > >>> efficient BLAS implementation (non threaded vs threaded). However I
> doubt
> > >>> this is the origin of your 2x performance increase.
> > >>>
> > >>> If you really want to understand where the performance improvement
> > >>> originated from, you’d need to send to the email list the result of
> > >>> -log_view from both the old and new versions, running the exact same
> > >>> problem.
> > >>>
> > >>> From that info, we can see what implementations in PETSc are being
> used
> > >>> and where the time reduction is occurring. Knowing that, it should be
> > >>> clearer to provide an explanation for it.
> > >>>
> > >>>
> > >>> Thanks,
> > >>> Dave
> > >>>
> > >>>
> > >>> On Tue 23. Mar 2021 at 06:24, Mohammad Gohardoust <
> [email protected]>
> > >>> wrote:
> > >>>
> > >>>> Hi,
> > >>>>
> > >>>> I am using a code which is based on petsc (and also parmetis).
> Recently
> > >>>> I made the following changes and now the code is running about two
> times
> > >>>> faster than before:
> > >>>>
> > >>>> - Upgraded Ubuntu 18.04 to 20.04
> > >>>> - Upgraded petsc 3.13.4 to 3.14.5
> > >>>> - This time I installed parmetis and metis directly via petsc by
> > >>>> --download-parmetis --download-metis flags instead of installing
> them
> > >>>> separately and using --with-parmetis-include=... and
> > >>>> --with-parmetis-lib=... (the version of installed parmetis was
> 4.0.3 before)
> > >>>>
> > >>>> I was wondering what can possibly explain this speedup? Does anyone
> > >>>> have any suggestions?
> > >>>>
> > >>>> Thanks,
> > >>>> Mohammad
> > >>>>
> > >>>
> >
> >
>
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
/home/mohammad/Programs/parswms_v1.40/parswms on a arch-linux-c-opt named mohammad-box with 16 processors, by mohammad Tue Mar 23 23:07:52 2021
Using Petsc Release Version 3.13.4, Aug 01, 2020
Max Max/Min Avg Total
Time (sec): 4.621e+02 1.000 4.621e+02
Objects: 1.194e+04 1.000 1.194e+04
Flop: 1.062e+10 1.215 9.958e+09 1.593e+11
Flop/sec: 2.298e+07 1.215 2.155e+07 3.448e+08
MPI Messages: 1.770e+06 2.863 9.871e+05 1.579e+07
MPI Message Lengths: 1.512e+09 2.537 9.827e+02 1.552e+10
MPI Reductions: 1.777e+05 1.000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total Count %Total Avg %Total Count %Total
0: Main Stage: 1.2329e+01 2.7% 4.3700e+03 0.0% 4.764e+05 3.0% 3.135e+02 1.0% 2.244e+04 12.6%
1: Solute_Assembly: 2.5775e+02 55.8% 7.3353e+09 4.6% 3.263e+06 20.7% 1.278e+03 26.9% 1.059e+04 6.0%
2: Solute_Solve: 2.2628e+01 4.9% 5.1093e+10 32.1% 2.924e+06 18.5% 7.478e+02 14.1% 5.291e+04 29.8%
3: Water_Assembly: 1.3576e+02 29.4% 1.6183e+10 10.2% 4.744e+06 30.0% 1.214e+03 37.1% 1.874e+04 10.5%
4: Water_Solve: 3.3579e+01 7.3% 8.4711e+10 53.2% 4.387e+06 27.8% 7.420e+02 21.0% 7.300e+04 41.1%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
AvgLen: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage ---- Total
Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
BuildTwoSided 3532 1.0 3.0122e+00 3.7 0.00e+00 0.0 1.0e+02 4.0e+00 3.5e+03 0 0 0 0 2 17 0 0 0 16 0
BuildTwoSidedF 3531 1.0 3.0212e+00 3.7 0.00e+00 0.0 0.0e+00 0.0e+00 3.5e+03 0 0 0 0 2 17 0 0 0 16 0
VecScatterBegin 3533 1.0 3.7550e-01 2.4 0.00e+00 0.0 3.6e+05 3.5e+02 0.0e+00 0 0 2 1 0 2 0 75 83 0 0
VecScatterEnd 3533 1.0 4.0637e-02 2.6 4.54e+02 2.7 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0100 0 0 0 0
SFSetGraph 1 1.0 2.1458e-06 3.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFSetUp 1 1.0 2.0008e-03 4.8 0.00e+00 0.0 2.0e+02 8.9e+01 1.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFBcastOpBegin 3532 1.0 3.6941e-01 2.5 0.00e+00 0.0 3.6e+05 3.5e+02 0.0e+00 0 0 2 1 0 2 0 75 83 0 0
SFBcastOpEnd 3532 1.0 3.0472e-02 2.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFReduceBegin 1 1.0 1.7285e-04 2.4 0.00e+00 0.0 1.0e+02 3.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFReduceEnd 1 1.0 2.7103e-0393.2 4.54e+02 2.7 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0100 0 0 0 2
SFPack 3533 1.0 5.6019e-03 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFUnpack 3533 1.0 1.3416e-02 3.8 4.54e+02 2.7 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0100 0 0 0 0
MatAssemblyBegin 3531 1.0 3.0538e+00 3.6 0.00e+00 0.0 0.0e+00 0.0e+00 3.5e+03 0 0 0 0 2 17 0 0 0 16 0
MatAssemblyEnd 3531 1.0 2.0374e-02 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
--- Event Stage 1: Solute_Assembly
BuildTwoSided 3531 1.0 5.4293e+00 3.4 0.00e+00 0.0 3.6e+05 4.0e+00 3.5e+03 1 0 2 0 2 1 0 11 0 33 0
BuildTwoSidedF 3531 1.0 5.6386e+00 2.8 0.00e+00 0.0 7.1e+05 3.6e+03 3.5e+03 1 0 5 17 2 2 0 22 62 33 0
VecScatterBegin 7062 1.0 7.7945e-01 2.5 0.00e+00 0.0 7.1e+05 3.5e+02 0.0e+00 0 0 5 2 0 0 0 22 6 0 0
VecScatterEnd 7062 1.0 2.1868e+00 4.5 1.60e+06 2.7 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 7
SFBcastOpBegin 3531 1.0 3.9145e-01 2.8 0.00e+00 0.0 3.6e+05 3.5e+02 0.0e+00 0 0 2 1 0 0 0 11 3 0 0
SFBcastOpEnd 3531 1.0 1.4410e+00 8.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFReduceBegin 3531 1.0 3.7230e-01 2.5 0.00e+00 0.0 3.6e+05 3.5e+02 0.0e+00 0 0 2 1 0 0 0 11 3 0 0
SFReduceEnd 3531 1.0 1.6625e+00 5.7 1.60e+06 2.7 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 9
SFPack 7062 1.0 1.1888e-02 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFUnpack 7062 1.0 1.2406e-02 1.5 1.60e+06 2.7 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1244
MatAssemblyBegin 3531 1.0 9.6657e+00 1.1 0.00e+00 0.0 7.1e+05 3.6e+03 3.5e+03 2 0 5 17 2 4 0 22 62 33 0
MatAssemblyEnd 3531 1.0 2.7771e+00 2.2 1.68e+07 2.7 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 2 0 0 0 58
MatZeroEntries 3531 1.0 1.1192e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
--- Event Stage 2: Solute_Solve
VecMDot 24688 1.0 7.3788e+00 1.5 4.82e+08 1.2 0.0e+00 0.0e+00 2.5e+04 1 5 0 0 14 29 14 0 0 47 975
VecNorm 28219 1.0 5.0321e+00 1.1 9.61e+07 1.2 0.0e+00 0.0e+00 2.8e+04 1 1 0 0 16 21 3 0 0 53 285
VecScale 28033 1.0 5.6265e-02 1.2 4.77e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 12673
VecCopy 3531 1.0 4.2992e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 6876 1.0 6.1489e-02 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 3345 1.0 2.4401e-02 2.3 1.14e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6974
VecMAXPY 28033 1.0 2.6858e-01 1.3 5.66e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 1 17 0 0 0 31484
VecScatterBegin 28219 1.0 3.1789e+00 2.4 0.00e+00 0.0 2.9e+06 7.5e+02 0.0e+00 0 0 19 14 0 8 0100100 0 0
VecScatterEnd 28219 1.0 2.5083e+00 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 9 0 0 0 0 0
SFBcastOpBegin 28219 1.0 3.1346e+00 2.5 0.00e+00 0.0 2.9e+06 7.5e+02 0.0e+00 0 0 19 14 0 8 0100100 0 0
SFBcastOpEnd 28219 1.0 2.4744e+00 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 8 0 0 0 0 0
SFPack 28219 1.0 7.6354e-02 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFUnpack 28219 1.0 1.5168e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMult 24688 1.0 6.5519e+00 1.5 1.12e+09 1.2 2.6e+06 8.0e+02 0.0e+00 1 11 16 13 0 22 33 88 94 0 2605
MatSOR 24688 1.0 3.2379e+00 1.2 1.10e+09 1.3 0.0e+00 0.0e+00 0.0e+00 1 10 0 0 0 13 31 0 0 0 4958
KSPSetUp 3531 1.0 3.6850e-03 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 3531 1.0 2.0069e+01 1.0 3.41e+09 1.2 2.6e+06 8.0e+02 5.3e+04 4 32 16 13 30 88100 88 94100 2546
KSPGMRESOrthog 24688 1.0 7.5811e+00 1.5 9.63e+08 1.2 0.0e+00 0.0e+00 2.5e+04 1 9 0 0 14 29 28 0 0 47 1899
PCSetUp 3531 1.0 2.9278e-03 2.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
PCApply 24688 1.0 3.2604e+00 1.2 1.10e+09 1.3 0.0e+00 0.0e+00 0.0e+00 1 10 0 0 0 13 31 0 0 0 4923
--- Event Stage 3: Water_Assembly
BuildTwoSided 17548 1.0 1.2790e+01 2.4 0.00e+00 0.0 5.9e+05 4.0e+00 1.8e+04 2 0 4 0 10 7 0 12 0 94 0
BuildTwoSidedF 17547 1.0 1.3220e+01 2.2 0.00e+00 0.0 1.2e+06 3.6e+03 1.8e+04 2 0 7 27 10 7 0 25 74 94 0
VecSet 1 1.0 5.9605e-06 6.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 35094 1.0 4.2246e+00 2.7 0.00e+00 0.0 3.6e+06 4.2e+02 0.0e+00 1 0 23 10 0 2 0 75 26 0 0
VecScatterEnd 35094 1.0 1.8867e+01 3.0 2.66e+06 2.7 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 8 0 0 0 0 1
SFSetGraph 1 1.0 1.4305e-06 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFSetUp 1 1.0 6.9189e-04 1.3 0.00e+00 0.0 2.1e+02 2.0e+02 1.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFBcastOpBegin 29245 1.0 3.4950e+00 2.9 0.00e+00 0.0 3.0e+06 4.4e+02 0.0e+00 0 0 19 8 0 1 0 63 23 0 0
SFBcastOpEnd 29245 1.0 8.5878e+00 5.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 4 0 0 0 0 0
SFReduceBegin 5849 1.0 6.7392e-01 2.1 0.00e+00 0.0 5.9e+05 3.5e+02 0.0e+00 0 0 4 1 0 0 0 12 4 0 0
SFReduceEnd 5849 1.0 1.2553e+01 3.8 2.66e+06 2.7 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 4 0 0 0 0 2
SFPack 35094 1.0 5.5740e-02 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFUnpack 35094 1.0 5.8571e-02 1.7 2.66e+06 2.7 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 436
MatMult 5849 1.0 4.3462e+00 2.6 2.64e+08 1.2 6.1e+05 8.0e+02 0.0e+00 1 3 4 3 0 2 25 13 8 0 930
MatAssemblyBegin 17547 1.0 1.9566e+01 1.1 0.00e+00 0.0 1.2e+06 3.6e+03 1.8e+04 4 0 7 27 10 14 0 25 74 94 0
MatAssemblyEnd 17547 1.0 4.7569e+00 2.3 2.79e+07 2.7 2.1e+02 2.0e+02 1.0e+01 1 0 0 0 0 2 2 0 0 0 56
MatZeroEntries 5848 1.0 1.3598e-01 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
--- Event Stage 4: Water_Solve
VecMDot 30652 1.0 8.0163e+00 1.5 4.82e+08 1.2 0.0e+00 0.0e+00 3.1e+04 2 5 0 0 17 21 9 0 0 42 899
VecNorm 42350 1.0 7.6133e+00 1.1 1.44e+08 1.2 0.0e+00 0.0e+00 4.2e+04 2 1 0 0 24 21 3 0 0 58 283
VecScale 36501 1.0 6.6659e-02 1.2 6.21e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 13928
VecCopy 5849 1.0 3.2074e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 48198 1.0 1.5659e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 11697 1.0 3.7661e-02 1.4 3.98e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 15800
VecMAXPY 36500 1.0 1.8171e-01 1.2 5.87e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 6 0 0 0 1 10 0 0 0 48275
VecScatterBegin 42350 1.0 4.5137e+00 2.4 0.00e+00 0.0 4.4e+06 7.4e+02 0.0e+00 1 0 28 21 0 8 0100100 0 0
VecScatterEnd 42350 1.0 3.7564e+00 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 9 0 0 0 0 0
VecNormalize 36501 1.0 6.7361e+00 1.1 1.86e+08 1.2 0.0e+00 0.0e+00 3.7e+04 1 2 0 0 21 19 3 0 0 50 413
SFBcastOpBegin 42350 1.0 4.4548e+00 2.5 0.00e+00 0.0 4.4e+06 7.4e+02 0.0e+00 1 0 28 21 0 8 0100100 0 0
SFBcastOpEnd 42350 1.0 3.7086e+00 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 8 0 0 0 0 0
SFPack 42350 1.0 1.0332e-01 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFUnpack 42350 1.0 2.1708e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMult 36501 1.0 9.7851e+00 1.6 1.65e+09 1.2 3.8e+06 8.0e+02 0.0e+00 2 16 24 20 0 21 30 87 94 0 2579
MatSolve 42350 1.0 3.6468e+00 1.3 1.80e+09 1.3 0.0e+00 0.0e+00 0.0e+00 1 17 0 0 0 10 31 0 0 0 7215
MatLUFactorNum 5849 1.0 3.8069e+00 1.2 8.91e+08 1.2 0.0e+00 0.0e+00 0.0e+00 1 8 0 0 0 10 16 0 0 0 3547
MatILUFactorSym 1 1.0 4.3368e-04 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 7.1526e-07 3.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 3.8385e-05 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPSetUp 11698 1.0 5.7492e-03 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 5849 1.0 2.9424e+01 1.0 5.66e+09 1.2 3.8e+06 8.0e+02 7.3e+04 6 53 24 20 41 88100 87 94100 2879
KSPGMRESOrthog 30652 1.0 8.2116e+00 1.5 9.65e+08 1.2 0.0e+00 0.0e+00 3.1e+04 2 9 0 0 17 22 17 0 0 42 1756
PCSetUp 11698 1.0 3.8799e+00 1.2 8.91e+08 1.2 0.0e+00 0.0e+00 0.0e+00 1 8 0 0 0 10 16 0 0 0 3480
PCSetUpOnBlocks 5849 1.0 3.8700e+00 1.2 8.91e+08 1.2 0.0e+00 0.0e+00 0.0e+00 1 8 0 0 0 10 16 0 0 0 3489
PCApply 42350 1.0 4.0158e+00 1.3 1.80e+09 1.3 0.0e+00 0.0e+00 0.0e+00 1 17 0 0 0 11 31 0 0 0 6552
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vec Scatter 1 2 1600 0.
Vector 6 213 1854616 0.
Index Set 2 5 24400 0.
IS L to G Mapping 1 1 7840 0.
Star Forest Graph 1 2 2240 0.
Matrix 3 4 655356 0.
Krylov Solver 2 3 85688 0.
Preconditioner 2 3 2768 0.
Viewer 1 0 0 0.
--- Event Stage 1: Solute_Assembly
--- Event Stage 2: Solute_Solve
Vector 136 0 0 0.
--- Event Stage 3: Water_Assembly
Vec Scatter 1 0 0 0.
Vector 2 1 1688 0.
Index Set 2 2 3592 0.
Star Forest Graph 1 0 0 0.
--- Event Stage 4: Water_Solve
Vector 11768 11698 102848816 0.
Index Set 3 0 0 0.
Matrix 1 0 0 0.
Krylov Solver 1 0 0 0.
Preconditioner 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 4.76837e-08
Average time for MPI_Barrier(): 0.000191069
Average time for zero size MPI_Send(): 3.37809e-05
#PETSc Option Table entries:
-log_view :run8.py:ascii_info
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-cc=gcc --with-cxx=g++ --with-cxx-dialect=C++11 --with-fc=gfortran --with-debugging=0 COPTFLAGS="-Ofast -march=native -mtune=native" CXXOPTFLAGS="-Ofast -march=native -mtune=native" FOPTFLAGS="-Ofast -march=native -mtune=native" --download-mpich --download-fblaslapack --download-scalapack --download-hwloc --download-parmetis --download-metis --download-hypre
-----------------------------------------
Libraries compiled on 2021-03-24 05:57:06 on mohammad-box
Machine characteristics: Linux-5.4.0-67-generic-x86_64-with-glibc2.29
Using PETSc directory: /home/mohammad/Programs/petsc-3.13.4
Using PETSc arch: arch-linux-c-opt
-----------------------------------------
Using C compiler: /home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/bin/mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -Ofast -march=native -mtune=native
Using Fortran compiler: /home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/bin/mpif90 -Wall -ffree-line-length-0 -Wno-unused-dummy-argument -Ofast -march=native -mtune=native
-----------------------------------------
Using include paths: -I/home/mohammad/Programs/petsc-3.13.4/include -I/home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/include
-----------------------------------------
Using C linker: /home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/bin/mpicc
Using Fortran linker: /home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/bin/mpif90
Using libraries: -Wl,-rpath,/home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/lib -L/home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/lib -lpetsc -Wl,-rpath,/home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/lib -L/home/mohammad/Programs/petsc-3.13.4/arch-linux-c-opt/lib -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9 -Wl,-rpath,/usr/lib/x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu -L/lib/x86_64-linux-gnu -lHYPRE -lscalapack -lflapack -lfblas -lpthread -lhwloc -lparmetis -lmetis -lm -lstdc++ -ldl -lmpifort -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -ldl
-----------------------------------------
