Hi Matt, No, interations from both linear and nonlinear solvers are similar. The system administrator doubt that the latency in mpich makes the difference. We will test a petsc version with OpenMPI on that cluster to check if it makes difference.
Thanks, Danyang On October 29, 2020 6:05:53 p.m. PDT, Matthew Knepley <knep...@gmail.com> wrote: >On Thu, Oct 29, 2020 at 3:04 PM Su,D.S. Danyang <d...@eoas.ubc.ca> >wrote: > >> Dear PETSc users, >> >> >> >> This is a question bother me for some time. I have the same code >running >> on different clusters and both clusters have good speedup. However, I >> noticed some thing quite strange. On one cluster, the solver is quite >> stable in computing time while on another cluster, the solver is >unstable >> in computing time. As shown in the figure below, the local >calculation >> almost has no communication and the computing time in this part is >quite >> stable. However, PETSc solver on Cluster B jumps quite a lot and the >> performance is not as good as Cluster A, even though the local >calculation >> is a little better on Cluster B. There are some difference on >hardware and >> PETSc configuration and optimization. Cluster A uses OpenMPI + GCC >compiler >> and Cluster B uses MPICH + GCC compiler. The number of processors >used is >> 128 on Cluster A and 120 on Cluster B. I also tested different number >of >> processors but the problem is the same. Does anyone have any idea >which >> part might cause this problem? >> > >First question: Does the solver take more iterates when the time bumps >up? > > Thanks, > > Matt > > >> >> >> >> >> Thanks and regards, >> >> >> >> Danyang >> >> >> > > >-- >What most experimenters take for granted before they begin their >experiments is infinitely more interesting than any results to which >their >experiments lead. >-- Norbert Wiener > >https://www.cse.buffalo.edu/~knepley/ ><http://www.cse.buffalo.edu/~knepley/> -- Sent from my Android device with K-9 Mail. Please excuse my brevity.