On Tue, Nov 9, 2021 at 6:52 AM Nicolas Barnafi <nab...@gmail.com> wrote:
> Thanks for the help Matthew. > > > The next step would be to verify that the operators you get in those slots > are correct in parallel. I normally just print out the matrix > for a small problem. This is the only way I know to debug assembly. > > To check the solver, you set it to an exact solver and see that it takes 1 > iterate. That means using LU as the subsolver > for the first block, using the full Schur complement, and solving the > Schur system with a residual tolerance of something like 1e-10. > > > There is certainly something wrong in the nested matrix, as this is the > output in serial (LU + full Schur LU): > Yes, this is why I recommend going to just 2 or 4 cells, so that you can see explicitly what is wrong with the matrix. Thanks, Matt > 0 SNES Function norm 3.240309155660e+03 > 0 KSP Residual norm 3.240309155660e+03 > 1 KSP Residual norm 3.219893365194e-08 > 1 SNES Function norm 3.999872813081e+03 > 0 KSP Residual norm 1.214145680994e+04 > 1 KSP Residual norm 2.742817643080e-04 > 2 SNES Function norm 3.663148395735e+01 > 0 KSP Residual norm 3.980103473350e+03 > 1 KSP Residual norm 1.814754406828e-05 > 3 SNES Function norm 2.136252100504e+01 > 0 KSP Residual norm 6.970609974245e+01 > 1 KSP Residual norm 1.982821753925e-05 > 4 SNES Function norm 1.059694009734e-02 > 0 KSP Residual norm 2.136131725219e+01 > 1 KSP Residual norm 4.545878835451e-06 > 2 KSP Residual norm 8.841427426699e-08 > 3 KSP Residual norm 3.305898584949e-09 > 5 SNES Function norm 3.729120062850e-03 > 0 KSP Residual norm 3.846880232871e-03 > 1 KSP Residual norm 1.740268533385e-06 > 2 KSP Residual norm 3.013108797112e-08 > 3 KSP Residual norm 1.105602265959e-09 > 6 SNES Function norm 1.117697448247e-09 > > whereas in parallel it is this (two processes): > > 0 SNES Function norm 3.240309155660e+03 > 0 KSP Residual norm 3.240309155660e+03 > 1 KSP Residual norm 2.158162432432e-01 > 2 KSP Residual norm 2.156121452805e-01 > 3 KSP Residual norm 2.148639482595e-01 > 4 KSP Residual norm 2.148248193325e-01 > 5 KSP Residual norm 1.305140166188e-03 > Linear solve converged due to CONVERGED_RTOL iterations 5 > 1 SNES Function norm 4.774611203034e+42 > > > Best, > NB > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>