Amneet,
Thanks for bringing this to our attention. The long term design goal in
PETSc is that the PetscOptions... calls are all made from the
XXXSetFromOptions() calls and not within the numerical solver portions.
Unfortunately this is not as easy to do as it might seem; hence there are a
bunch of them scattered within the solver portions.
In particular the worse culprit is KSPSolve(). You can run the following
experiment: edit src/ksp/ksp/interface/itfunc.c and locate the function
KSPSolve() now comment out all the lines with the work Option in them (I count
about 17 of them) now do make gnumake in that directory (of course with
optimized build) then rerun your exact same code that you report for from below.
How much faster is the total time and how much percentage are the
troublesome Options calls now? In other words how much does this change help? A
dramatic difference would motivate us to fix this problem sooner rather than
later.
Barry
> On Jan 15, 2016, at 5:33 PM, Bhalla, Amneet Pal S <[email protected]>
> wrote:
>
>
> Hi Barry,
>
> In our code at each timestep we build MG level smoothers using PETSc KSP
> solvers. We are using a PETSc function KSPSetFromOptions()
> after we set some default values to the KSP. However, the profiler is showing
> that PetscOptionsFindPair_Private() is taking about 14% of total runtime.
> We ran the code for 100 timesteps, and preconditioner is built everytime
> step. I am posting a sequence of calls to KSPSolve_Richardson that shows
> getting PETScOptions adds up to a lot of cost
>
> "KSPSolve_Richardson" 8.80e+05 12.7%
> "PCApplyBAorAB" 5.25e+05 7.6%
> "PCApply" 4.85e+05 7.0%
> "PCApply_ASM" 4.85e+05 7.0%
> "KSPSolve" 4.53e+05 6.6%
> "KSPSolve_PREONLY" 2.06e+05 3.0%
> "PetscObjectViewFromOptions" 3.19e+04 0.5%
> "PetscObjectViewFromOptions" 2.39e+04 0.3%
> "PetscOptionsGetBool" 2.39e+04 0.3%
> "PetscOptionsHasName" 2.39e+04 0.3%
> "PetscOptionsGetBool" 2.39e+04 0.3%
> "PetscOptionsHasName" 1.60e+04 0.2%
> "PetscOptionsGetBool" 1.60e+04 0.2%
> "PetscObjectViewFromOptions" 1.60e+04 0.2%
> "KSPReasonViewFromOptions" 1.60e+04 0.2%
> "PetscOptionsGetBool" 1.56e+04 0.2%
> "PetscObjectViewFromOptions" 7.98e+03 0.1%
> "KSPSetUpOnBlocks" 7.98e+03 0.1%
> "PetscOptionsGetBool" 7.98e+03 0.1%
> "VecSet" 7.97e+03 0.1%
> "PetscObjectViewFromOptions" 7.92e+03 0.1%
>
> Do you have some suggestions as to doing it in a fast way -- maybe parsing
> options only once in the simulation and making populating KSP
> options essentially a no-op?
>
> Thanks,
> --Amneet
