I am using PETSc to solve an adjoint system for a CFD code. The issue I am having is that if I try to run successive cfd/adjoint solutions, the adjoint fails to converge after the first cfd/adjoint solution cycle. i.e. Solve CFD (ok) Solve adjoint system with multiple RHS's (ok) Solve CFD at new point (ok) Solve adjoint system at new point with multiple RHS's (GMRES iterations stall)
PETSc is not used to solve the CFD system, only the adjoint system. I am currently using the following solver options: 'Adjoint solver type': 'GMRES',\ 'adjoint relative tolerance':1e-6,\ 'adjoint absolute tolerance':1e-16,\ 'adjoint max iterations': 1500,\ 'adjoint restart iteration' : 80,\ 'adjoint monitor step': 10,\ 'Preconditioner Side': 'right',\ 'Matrix Ordering': 'NestedDissection',\ 'Global Preconditioner Type': 'Additive Schwartz',\ 'Local Preconditioner Type' : 'ILU',\ 'ILU Fill Levels': 3,\ 'ASM Overlap' : 5,\ Also, I am using the fortran interface to PETSc. As a sample, I have included two convergence histories. Both are at the same converged CFD point(this case is ~1.2million cell so the PETSc system is approx 6million degrees of freedom). The first is an example where the adjoint system is solved the first time through the cycle. The second is an example of where the adjoint is solved the second time through the cycle: 1) as first point in cycle # ... KSP properties: # type : gmres # tolerances : rel = 1.0E-10 # abs = 1.0E-16 # div = 1.0E+05 # max.iter. : 1500 # precond.type: asm Solving ADjoint Transpose with PETSc... 0 KSP Residual norm 0.1392E+00 10 KSP Residual norm 0.6394E-01 20 KSP Residual norm 0.6106E-01 30 KSP Residual norm 0.6019E-01 40 KSP Residual norm 0.5941E-01 50 KSP Residual norm 0.5876E-01 60 KSP Residual norm 0.5602E-01 70 KSP Residual norm 0.4915E-01 80 KSP Residual norm 0.3994E-01 90 KSP Residual norm 0.3892E-01 100 KSP Residual norm 0.3854E-01 110 KSP Residual norm 0.3794E-01 120 KSP Residual norm 0.3717E-01 130 KSP Residual norm 0.3630E-01 140 KSP Residual norm 0.3415E-01 ... 900 KSP Residual norm 0.2437E-09 910 KSP Residual norm 0.1452E-09 920 KSP Residual norm 0.1025E-09 930 KSP Residual norm 0.6875E-10 940 KSP Residual norm 0.4141E-10 950 KSP Residual norm 0.2317E-10 960 KSP Residual norm 0.1559E-10 Solving ADjoint Transpose with PETSc time (s) = 391.43 Norm of error = 0.1366E-10 Iterations = 965 ------------------------------------------------ PETSc solver converged after 965 iterations. ------------------------------------------------ 2) As second point in cycle # ... KSP properties: # type : gmres # tolerances : rel = 1.0E-10 # abs = 1.0E-16 # div = 1.0E+05 # max.iter. : 1500 # precond.type: asm Solving ADjoint Transpose with PETSc... 0 KSP Residual norm 0.1392E+00 10 KSP Residual norm 0.6400E-01 20 KSP Residual norm 0.6140E-01 30 KSP Residual norm 0.6060E-01 40 KSP Residual norm 0.5995E-01 50 KSP Residual norm 0.5974E-01 60 KSP Residual norm 0.5971E-01 70 KSP Residual norm 0.5957E-01 80 KSP Residual norm 0.5906E-01 90 KSP Residual norm 0.5906E-01 100 KSP Residual norm 0.5906E-01 110 KSP Residual norm 0.5903E-01 120 KSP Residual norm 0.5901E-01 130 KSP Residual norm 0.5901E-01 140 KSP Residual norm 0.5901E-01 ... 1400 KSP Residual norm 0.5895E-01 1410 KSP Residual norm 0.5895E-01 1420 KSP Residual norm 0.5895E-01 1430 KSP Residual norm 0.5895E-01 1440 KSP Residual norm 0.5895E-01 1450 KSP Residual norm 0.5895E-01 1460 KSP Residual norm 0.5895E-01 1470 KSP Residual norm 0.5895E-01 1480 KSP Residual norm 0.5895E-01 1490 KSP Residual norm 0.5895E-01 1500 KSP Residual norm 0.5895E-01 Solving ADjoint Transpose with PETSc time (s) = 516.59 Norm of error = 0.5895E-01 Iterations = 1500 ------------------------------------------------ PETSc solver converged after 1500 iterations. ------------------------------------------------ I have tried both resetting the operators between cycles and completely destroying the KSP object between cycles. Both give the same result. Is there a step I am missing to properly reset the system for the subsequent solves? Thanks in advance for your input... Sandy -- C.A.(Sandy) Mader Ph.D Candidate Multidisciplinary Design Optimization Laboratory University of Toronto Institute for Aerospace Studies mader at utias.utoronto.ca
