Re: [petsc-dev] Issue with -log_view
https://gitlab.com/petsc/petsc/-/merge_requests/6440 > El 9 may 2023, a las 15:31, Matthew Knepley escribió: > > On Tue, May 9, 2023 at 9:04 AM Jose E. Roman wrote: > I found the bug: the event MAT_MultHermitianTranspose is used but not > registered. > I will create a MR. > > Great! > > Thanks, > Matt > > Thanks Matt. > > > El 9 may 2023, a las 14:50, Matthew Knepley escribió: > > > > On Tue, May 9, 2023 at 8:41 AM Jose E. Roman wrote: > > But MatCreateShell() calls MatInitializePackage() (via MatCreate()) and > > also the main program creates a regular Mat. The events should have been > > registered by the time the shell matrix operations are invoked. > > > > The reason I say this is that PetscBarrier is the _first_ event, so if an > > event is called without initializing, it > > will show up as PetscBarrier. Maybe break in MatMultTranspose, to see who > > is calling it first? > > > > Thanks, > > > > Matt > > > > > El 9 may 2023, a las 14:13, Matthew Knepley escribió: > > > > > > On Tue, May 9, 2023 at 7:17 AM Jose E. Roman wrote: > > > Hi. > > > > > > We are seeing a strange thing in the -log_view output with one of the > > > SLEPc solvers. It is probably an issue with SLEPc, but we don't know how > > > to debug it. > > > > > > It can be reproduced for instance with > > > > > > $ ./ex45 -m 15 -n 20 -p 21 -svd_nsv 4 -svd_ncv 9 -log_view > > > > > > The log_view events are listed at the end of this email. The first one > > > (PetscBarrier) is wrong, because PetscBarrier is never called, if I place > > > a breakpoint in PetscBarrier() it will never be hit. Also, in that event > > > it reports some nonzero Mflop/s, which suggests that it corresponds to > > > another event (not PetscBarrier). Furthermore, the count of the > > > PetscBarrier event always matches the count of MatMultTranspose, so there > > > must be a connection. > > > > > > Does anyone have suggestions how to address this? > > > > > > Hi Jose, > > > > > > Here is my guess. PETSc sets all of the event ids (using Register) when > > > the dynamic libraries get loaded. If they are not loaded, > > > then the library initialization function is called when some function > > > from that library is used. My guess is that we put this init check > > > in MatCreate(), but that is not called when you create your shell matrix > > > and thus the events are not initialized correctly for you until > > > later. Can you check? > > > > > > Thanks, > > > > > > Matt > > > > > > Note: this is with 1 MPI process. > > > Note: the solver creates a shell matrix with MATOP_MULT_TRANSPOSE. > > > > > > Thanks. > > > Jose > > > > > > > > > PetscBarrier 16 1.0 7.5579e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > > > MatMult 82 1.0 6.1590e-01 1.0 9.25e+05 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 81 97 0 0 0 81 97 0 0 0 2 > > > MatMultTranspose 16 1.0 7.4625e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > > > MatAssemblyBegin 4 1.0 5.2452e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > MatAssemblyEnd 4 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > MatTranspose 2 1.0 1.6265e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > SVDSetUp 1 1.0 1.8686e-02 1.0 2.72e+02 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 > > > SVDSolve 1 1.0 5.5965e-01 1.0 7.95e+05 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 74 83 0 0 0 74 83 0 0 0 1 > > > EPSSetUp 1 1.0 9.5146e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > > > EPSSolve 1 1.0 5.4082e-01 1.0 7.94e+05 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 71 83 0 0 0 71 83 0 0 0 1 > > > STSetUp1 1.0 4.8406e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > > > STComputeOperatr 1 1.0 1.2653e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > STApply 24 1.0 6.1569e-01 1.0 8.84e+05 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > > STMatSolve24 1.0 6.1556e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > > KSPSetUp 1 1.0 2.8465e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > KSPSolve 24 1.0 6.1551e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > > KSPGMRESOrthog 480 1.0 4.3219e-01 1.0 3.98e+05 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 57 42 0 0 0 57 42 0 0 0 1 > > > PCSetUp1 1.0 3.8147e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0
Re: [petsc-dev] Issue with -log_view
On Tue, May 9, 2023 at 9:04 AM Jose E. Roman wrote: > I found the bug: the event MAT_MultHermitianTranspose is used but not > registered. > I will create a MR. > Great! Thanks, Matt > Thanks Matt. > > > El 9 may 2023, a las 14:50, Matthew Knepley > escribió: > > > > On Tue, May 9, 2023 at 8:41 AM Jose E. Roman wrote: > > But MatCreateShell() calls MatInitializePackage() (via MatCreate()) and > also the main program creates a regular Mat. The events should have been > registered by the time the shell matrix operations are invoked. > > > > The reason I say this is that PetscBarrier is the _first_ event, so if > an event is called without initializing, it > > will show up as PetscBarrier. Maybe break in MatMultTranspose, to see > who is calling it first? > > > > Thanks, > > > > Matt > > > > > El 9 may 2023, a las 14:13, Matthew Knepley > escribió: > > > > > > On Tue, May 9, 2023 at 7:17 AM Jose E. Roman > wrote: > > > Hi. > > > > > > We are seeing a strange thing in the -log_view output with one of the > SLEPc solvers. It is probably an issue with SLEPc, but we don't know how to > debug it. > > > > > > It can be reproduced for instance with > > > > > > $ ./ex45 -m 15 -n 20 -p 21 -svd_nsv 4 -svd_ncv 9 -log_view > > > > > > The log_view events are listed at the end of this email. The first one > (PetscBarrier) is wrong, because PetscBarrier is never called, if I place a > breakpoint in PetscBarrier() it will never be hit. Also, in that event it > reports some nonzero Mflop/s, which suggests that it corresponds to another > event (not PetscBarrier). Furthermore, the count of the PetscBarrier event > always matches the count of MatMultTranspose, so there must be a connection. > > > > > > Does anyone have suggestions how to address this? > > > > > > Hi Jose, > > > > > > Here is my guess. PETSc sets all of the event ids (using Register) > when the dynamic libraries get loaded. If they are not loaded, > > > then the library initialization function is called when some function > from that library is used. My guess is that we put this init check > > > in MatCreate(), but that is not called when you create your shell > matrix and thus the events are not initialized correctly for you until > > > later. Can you check? > > > > > > Thanks, > > > > > > Matt > > > > > > Note: this is with 1 MPI process. > > > Note: the solver creates a shell matrix with MATOP_MULT_TRANSPOSE. > > > > > > Thanks. > > > Jose > > > > > > > > > PetscBarrier 16 1.0 7.5579e-04 1.0 4.38e+03 1.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > > > MatMult 82 1.0 6.1590e-01 1.0 9.25e+05 1.0 0.0e+00 > 0.0e+00 0.0e+00 81 97 0 0 0 81 97 0 0 0 2 > > > MatMultTranspose 16 1.0 7.4625e-04 1.0 4.38e+03 1.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > > > MatAssemblyBegin 4 1.0 5.2452e-06 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > MatAssemblyEnd 4 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > MatTranspose 2 1.0 1.6265e-03 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > SVDSetUp 1 1.0 1.8686e-02 1.0 2.72e+02 1.0 0.0e+00 > 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 > > > SVDSolve 1 1.0 5.5965e-01 1.0 7.95e+05 1.0 0.0e+00 > 0.0e+00 0.0e+00 74 83 0 0 0 74 83 0 0 0 1 > > > EPSSetUp 1 1.0 9.5146e-03 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > > > EPSSolve 1 1.0 5.4082e-01 1.0 7.94e+05 1.0 0.0e+00 > 0.0e+00 0.0e+00 71 83 0 0 0 71 83 0 0 0 1 > > > STSetUp1 1.0 4.8406e-03 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > > > STComputeOperatr 1 1.0 1.2653e-03 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > STApply 24 1.0 6.1569e-01 1.0 8.84e+05 1.0 0.0e+00 > 0.0e+00 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > > STMatSolve24 1.0 6.1556e-01 1.0 8.80e+05 1.0 0.0e+00 > 0.0e+00 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > > KSPSetUp 1 1.0 2.8465e-03 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > KSPSolve 24 1.0 6.1551e-01 1.0 8.80e+05 1.0 0.0e+00 > 0.0e+00 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > > KSPGMRESOrthog 480 1.0 4.3219e-01 1.0 3.98e+05 1.0 0.0e+00 > 0.0e+00 0.0e+00 57 42 0 0 0 57 42 0 0 0 1 > > > PCSetUp1 1.0 3.8147e-06 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > PCApply 504 1.0 2.1615e-02 1.0 1.01e+04 1.0 0.0e+00 > 0.0e+00 0.0e+00 3 1 0 0 0 3 1 0 0 0 0 > > > BVCopy27 1.0 3.0560e-03 1.0 0.00e+00 0.0 0.0e+00 > 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > B
Re: [petsc-dev] Issue with -log_view
I found the bug: the event MAT_MultHermitianTranspose is used but not registered. I will create a MR. Thanks Matt. > El 9 may 2023, a las 14:50, Matthew Knepley escribió: > > On Tue, May 9, 2023 at 8:41 AM Jose E. Roman wrote: > But MatCreateShell() calls MatInitializePackage() (via MatCreate()) and also > the main program creates a regular Mat. The events should have been > registered by the time the shell matrix operations are invoked. > > The reason I say this is that PetscBarrier is the _first_ event, so if an > event is called without initializing, it > will show up as PetscBarrier. Maybe break in MatMultTranspose, to see who is > calling it first? > > Thanks, > > Matt > > > El 9 may 2023, a las 14:13, Matthew Knepley escribió: > > > > On Tue, May 9, 2023 at 7:17 AM Jose E. Roman wrote: > > Hi. > > > > We are seeing a strange thing in the -log_view output with one of the SLEPc > > solvers. It is probably an issue with SLEPc, but we don't know how to debug > > it. > > > > It can be reproduced for instance with > > > > $ ./ex45 -m 15 -n 20 -p 21 -svd_nsv 4 -svd_ncv 9 -log_view > > > > The log_view events are listed at the end of this email. The first one > > (PetscBarrier) is wrong, because PetscBarrier is never called, if I place a > > breakpoint in PetscBarrier() it will never be hit. Also, in that event it > > reports some nonzero Mflop/s, which suggests that it corresponds to another > > event (not PetscBarrier). Furthermore, the count of the PetscBarrier event > > always matches the count of MatMultTranspose, so there must be a connection. > > > > Does anyone have suggestions how to address this? > > > > Hi Jose, > > > > Here is my guess. PETSc sets all of the event ids (using Register) when the > > dynamic libraries get loaded. If they are not loaded, > > then the library initialization function is called when some function from > > that library is used. My guess is that we put this init check > > in MatCreate(), but that is not called when you create your shell matrix > > and thus the events are not initialized correctly for you until > > later. Can you check? > > > > Thanks, > > > > Matt > > > > Note: this is with 1 MPI process. > > Note: the solver creates a shell matrix with MATOP_MULT_TRANSPOSE. > > > > Thanks. > > Jose > > > > > > PetscBarrier 16 1.0 7.5579e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > > MatMult 82 1.0 6.1590e-01 1.0 9.25e+05 1.0 0.0e+00 0.0e+00 > > 0.0e+00 81 97 0 0 0 81 97 0 0 0 2 > > MatMultTranspose 16 1.0 7.4625e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > > MatAssemblyBegin 4 1.0 5.2452e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > MatAssemblyEnd 4 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > MatTranspose 2 1.0 1.6265e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > SVDSetUp 1 1.0 1.8686e-02 1.0 2.72e+02 1.0 0.0e+00 0.0e+00 > > 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 > > SVDSolve 1 1.0 5.5965e-01 1.0 7.95e+05 1.0 0.0e+00 0.0e+00 > > 0.0e+00 74 83 0 0 0 74 83 0 0 0 1 > > EPSSetUp 1 1.0 9.5146e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > > EPSSolve 1 1.0 5.4082e-01 1.0 7.94e+05 1.0 0.0e+00 0.0e+00 > > 0.0e+00 71 83 0 0 0 71 83 0 0 0 1 > > STSetUp1 1.0 4.8406e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > > STComputeOperatr 1 1.0 1.2653e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > STApply 24 1.0 6.1569e-01 1.0 8.84e+05 1.0 0.0e+00 0.0e+00 > > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > STMatSolve24 1.0 6.1556e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > KSPSetUp 1 1.0 2.8465e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > KSPSolve 24 1.0 6.1551e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > KSPGMRESOrthog 480 1.0 4.3219e-01 1.0 3.98e+05 1.0 0.0e+00 0.0e+00 > > 0.0e+00 57 42 0 0 0 57 42 0 0 0 1 > > PCSetUp1 1.0 3.8147e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > PCApply 504 1.0 2.1615e-02 1.0 1.01e+04 1.0 0.0e+00 0.0e+00 > > 0.0e+00 3 1 0 0 0 3 1 0 0 0 0 > > BVCopy27 1.0 3.0560e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > BVMultVec 38 1.0 3.7060e-03 1.0 8.88e+03 1.0 0.0e+00 0.0e+00 > > 0.0e+00 0 1 0 0 0 0 1 0 0 0
Re: [petsc-dev] Issue with -log_view
On Tue, May 9, 2023 at 8:41 AM Jose E. Roman wrote: > But MatCreateShell() calls MatInitializePackage() (via MatCreate()) and > also the main program creates a regular Mat. The events should have been > registered by the time the shell matrix operations are invoked. > The reason I say this is that PetscBarrier is the _first_ event, so if an event is called without initializing, it will show up as PetscBarrier. Maybe break in MatMultTranspose, to see who is calling it first? Thanks, Matt > > El 9 may 2023, a las 14:13, Matthew Knepley > escribió: > > > > On Tue, May 9, 2023 at 7:17 AM Jose E. Roman wrote: > > Hi. > > > > We are seeing a strange thing in the -log_view output with one of the > SLEPc solvers. It is probably an issue with SLEPc, but we don't know how to > debug it. > > > > It can be reproduced for instance with > > > > $ ./ex45 -m 15 -n 20 -p 21 -svd_nsv 4 -svd_ncv 9 -log_view > > > > The log_view events are listed at the end of this email. The first one > (PetscBarrier) is wrong, because PetscBarrier is never called, if I place a > breakpoint in PetscBarrier() it will never be hit. Also, in that event it > reports some nonzero Mflop/s, which suggests that it corresponds to another > event (not PetscBarrier). Furthermore, the count of the PetscBarrier event > always matches the count of MatMultTranspose, so there must be a connection. > > > > Does anyone have suggestions how to address this? > > > > Hi Jose, > > > > Here is my guess. PETSc sets all of the event ids (using Register) when > the dynamic libraries get loaded. If they are not loaded, > > then the library initialization function is called when some function > from that library is used. My guess is that we put this init check > > in MatCreate(), but that is not called when you create your shell matrix > and thus the events are not initialized correctly for you until > > later. Can you check? > > > > Thanks, > > > > Matt > > > > Note: this is with 1 MPI process. > > Note: the solver creates a shell matrix with MATOP_MULT_TRANSPOSE. > > > > Thanks. > > Jose > > > > > > PetscBarrier 16 1.0 7.5579e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > > MatMult 82 1.0 6.1590e-01 1.0 9.25e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 97 0 0 0 81 97 0 0 0 2 > > MatMultTranspose 16 1.0 7.4625e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > > MatAssemblyBegin 4 1.0 5.2452e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > MatAssemblyEnd 4 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > MatTranspose 2 1.0 1.6265e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > SVDSetUp 1 1.0 1.8686e-02 1.0 2.72e+02 1.0 0.0e+00 0.0e+00 > 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 > > SVDSolve 1 1.0 5.5965e-01 1.0 7.95e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 74 83 0 0 0 74 83 0 0 0 1 > > EPSSetUp 1 1.0 9.5146e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > > EPSSolve 1 1.0 5.4082e-01 1.0 7.94e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 71 83 0 0 0 71 83 0 0 0 1 > > STSetUp1 1.0 4.8406e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > > STComputeOperatr 1 1.0 1.2653e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > STApply 24 1.0 6.1569e-01 1.0 8.84e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > STMatSolve24 1.0 6.1556e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > KSPSetUp 1 1.0 2.8465e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > KSPSolve 24 1.0 6.1551e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > > KSPGMRESOrthog 480 1.0 4.3219e-01 1.0 3.98e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 57 42 0 0 0 57 42 0 0 0 1 > > PCSetUp1 1.0 3.8147e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > PCApply 504 1.0 2.1615e-02 1.0 1.01e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 3 1 0 0 0 3 1 0 0 0 0 > > BVCopy27 1.0 3.0560e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > BVMultVec 38 1.0 3.7060e-03 1.0 8.88e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 1 0 0 0 0 1 0 0 0 2 > > BVMultInPlace 3 1.0 1.0681e-04 1.0 4.32e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 040 > > BVDotVec 38 1.0 4.5121e-03 1.0 4.38e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 1 5 0 0 0 1 5 0 0 010 > > BVOrthogonalizeV 20 1.0 1.3182e-02 1.0 5.33e
Re: [petsc-dev] Issue with -log_view
But MatCreateShell() calls MatInitializePackage() (via MatCreate()) and also the main program creates a regular Mat. The events should have been registered by the time the shell matrix operations are invoked. > El 9 may 2023, a las 14:13, Matthew Knepley escribió: > > On Tue, May 9, 2023 at 7:17 AM Jose E. Roman wrote: > Hi. > > We are seeing a strange thing in the -log_view output with one of the SLEPc > solvers. It is probably an issue with SLEPc, but we don't know how to debug > it. > > It can be reproduced for instance with > > $ ./ex45 -m 15 -n 20 -p 21 -svd_nsv 4 -svd_ncv 9 -log_view > > The log_view events are listed at the end of this email. The first one > (PetscBarrier) is wrong, because PetscBarrier is never called, if I place a > breakpoint in PetscBarrier() it will never be hit. Also, in that event it > reports some nonzero Mflop/s, which suggests that it corresponds to another > event (not PetscBarrier). Furthermore, the count of the PetscBarrier event > always matches the count of MatMultTranspose, so there must be a connection. > > Does anyone have suggestions how to address this? > > Hi Jose, > > Here is my guess. PETSc sets all of the event ids (using Register) when the > dynamic libraries get loaded. If they are not loaded, > then the library initialization function is called when some function from > that library is used. My guess is that we put this init check > in MatCreate(), but that is not called when you create your shell matrix and > thus the events are not initialized correctly for you until > later. Can you check? > > Thanks, > > Matt > > Note: this is with 1 MPI process. > Note: the solver creates a shell matrix with MATOP_MULT_TRANSPOSE. > > Thanks. > Jose > > > PetscBarrier 16 1.0 7.5579e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > MatMult 82 1.0 6.1590e-01 1.0 9.25e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 97 0 0 0 81 97 0 0 0 2 > MatMultTranspose 16 1.0 7.4625e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > MatAssemblyBegin 4 1.0 5.2452e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatAssemblyEnd 4 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatTranspose 2 1.0 1.6265e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > SVDSetUp 1 1.0 1.8686e-02 1.0 2.72e+02 1.0 0.0e+00 0.0e+00 > 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 > SVDSolve 1 1.0 5.5965e-01 1.0 7.95e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 74 83 0 0 0 74 83 0 0 0 1 > EPSSetUp 1 1.0 9.5146e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > EPSSolve 1 1.0 5.4082e-01 1.0 7.94e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 71 83 0 0 0 71 83 0 0 0 1 > STSetUp1 1.0 4.8406e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > STComputeOperatr 1 1.0 1.2653e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > STApply 24 1.0 6.1569e-01 1.0 8.84e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > STMatSolve24 1.0 6.1556e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > KSPSetUp 1 1.0 2.8465e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > KSPSolve 24 1.0 6.1551e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > KSPGMRESOrthog 480 1.0 4.3219e-01 1.0 3.98e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 57 42 0 0 0 57 42 0 0 0 1 > PCSetUp1 1.0 3.8147e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > PCApply 504 1.0 2.1615e-02 1.0 1.01e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 3 1 0 0 0 3 1 0 0 0 0 > BVCopy27 1.0 3.0560e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > BVMultVec 38 1.0 3.7060e-03 1.0 8.88e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 1 0 0 0 0 1 0 0 0 2 > BVMultInPlace 3 1.0 1.0681e-04 1.0 4.32e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 040 > BVDotVec 38 1.0 4.5121e-03 1.0 4.38e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 1 5 0 0 0 1 5 0 0 010 > BVOrthogonalizeV 20 1.0 1.3182e-02 1.0 5.33e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 2 6 0 0 0 2 6 0 0 0 4 > BVScale 24 1.0 4.8089e-04 1.0 4.80e+02 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 1 > BVNormVec 4 1.0 5.4097e-04 1.0 3.67e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 7 > BVNormalize1 1.0 1.3154e-03
Re: [petsc-dev] Issue with -log_view
On Tue, May 9, 2023 at 7:17 AM Jose E. Roman wrote: > Hi. > > We are seeing a strange thing in the -log_view output with one of the > SLEPc solvers. It is probably an issue with SLEPc, but we don't know how to > debug it. > > It can be reproduced for instance with > > $ ./ex45 -m 15 -n 20 -p 21 -svd_nsv 4 -svd_ncv 9 -log_view > > The log_view events are listed at the end of this email. The first one > (PetscBarrier) is wrong, because PetscBarrier is never called, if I place a > breakpoint in PetscBarrier() it will never be hit. Also, in that event it > reports some nonzero Mflop/s, which suggests that it corresponds to another > event (not PetscBarrier). Furthermore, the count of the PetscBarrier event > always matches the count of MatMultTranspose, so there must be a connection. > > Does anyone have suggestions how to address this? > Hi Jose, Here is my guess. PETSc sets all of the event ids (using Register) when the dynamic libraries get loaded. If they are not loaded, then the library initialization function is called when some function from that library is used. My guess is that we put this init check in MatCreate(), but that is not called when you create your shell matrix and thus the events are not initialized correctly for you until later. Can you check? Thanks, Matt > Note: this is with 1 MPI process. > Note: the solver creates a shell matrix with MATOP_MULT_TRANSPOSE. > > Thanks. > Jose > > > PetscBarrier 16 1.0 7.5579e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > MatMult 82 1.0 6.1590e-01 1.0 9.25e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 97 0 0 0 81 97 0 0 0 2 > MatMultTranspose 16 1.0 7.4625e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 6 > MatAssemblyBegin 4 1.0 5.2452e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatAssemblyEnd 4 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatTranspose 2 1.0 1.6265e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > SVDSetUp 1 1.0 1.8686e-02 1.0 2.72e+02 1.0 0.0e+00 0.0e+00 > 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 > SVDSolve 1 1.0 5.5965e-01 1.0 7.95e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 74 83 0 0 0 74 83 0 0 0 1 > EPSSetUp 1 1.0 9.5146e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > EPSSolve 1 1.0 5.4082e-01 1.0 7.94e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 71 83 0 0 0 71 83 0 0 0 1 > STSetUp1 1.0 4.8406e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > STComputeOperatr 1 1.0 1.2653e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > STApply 24 1.0 6.1569e-01 1.0 8.84e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > STMatSolve24 1.0 6.1556e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > KSPSetUp 1 1.0 2.8465e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > KSPSolve 24 1.0 6.1551e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 81 92 0 0 0 81 92 0 0 0 1 > KSPGMRESOrthog 480 1.0 4.3219e-01 1.0 3.98e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 57 42 0 0 0 57 42 0 0 0 1 > PCSetUp1 1.0 3.8147e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > PCApply 504 1.0 2.1615e-02 1.0 1.01e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 3 1 0 0 0 3 1 0 0 0 0 > BVCopy27 1.0 3.0560e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > BVMultVec 38 1.0 3.7060e-03 1.0 8.88e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 1 0 0 0 0 1 0 0 0 2 > BVMultInPlace 3 1.0 1.0681e-04 1.0 4.32e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 040 > BVDotVec 38 1.0 4.5121e-03 1.0 4.38e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 1 5 0 0 0 1 5 0 0 010 > BVOrthogonalizeV 20 1.0 1.3182e-02 1.0 5.33e+04 1.0 0.0e+00 0.0e+00 > 0.0e+00 2 6 0 0 0 2 6 0 0 0 4 > BVScale 24 1.0 4.8089e-04 1.0 4.80e+02 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 1 > BVNormVec 4 1.0 5.4097e-04 1.0 3.67e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 7 > BVNormalize1 1.0 1.3154e-03 1.0 3.75e+03 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 3 > BVSetRandom1 1.0 9.1791e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > BVMatMultVec 19 1.0 4.6828e-01 1.0 6.99e+05 1.0 0.0e+00 0.0e+00 > 0.0e+00 62 73 0 0 0 62 73 0 0 0 1 > DSSolve
