> On Oct 25, 2016, at 5:39 PM, Olivier Mesnard <olivier.mesna...@gmail.com> > wrote: > > On 25 October 2016 at 17:51, Barry Smith <bsm...@mcs.anl.gov> wrote: > > Olivier, > > In theory you do not need to change anything else. Are you using a > different matrix object for the velocity_ksp object than the poisson_ksp > object? > > The matrix is different for the velocity_ksp and the poisson_ksp. > > The code change in PETSc is very little but we have a report from another > CFD user who also had problems with the change so there may be some subtle > bug that we can't figure out causing things to not behave properly. > > First run the 3.7.4 code with -poisson_ksp_view and verify that when it > prints the matrix information it prints something like has attached null > space if it does not print that it means that somehow the matrix is not > properly getting the matrix attached. > > When running with 3.7.4 and -poisson_ksp_view, the output shows that the > nullspace is not attached to the KSP (as it was with 3.5.4); however the > print statement is now under the Mat info (which is expected when moving from > KSPSetNullSpace to MatSetNullSpace?).
Good, this is how it should be. > > Though older versions had MatSetNullSpace() they didn't necessarily > associate it with the KSP so it was not expected to work as a replacement for > KSPSetNullSpace() with older versions. > > Because our other user had great difficulty trying to debug the issue > feel free to send us at petsc-ma...@mcs.anl.gov your code with instructions > on building and running and we can try to track down the problem. Better than > hours and hours spent with fruitless email. We will, of course, not > distribute the code and will delete in when we are finished with it. > > The code is open-source and hosted on GitHub > (https://github.com/barbagroup/PetIBM). > I just pushed the branches `feature-compatible-petsc-3.7` and > `revert-compatible-petsc-3.5` that I used to observe this problem. > Thanks, I'll get back to you if I discover anything > PETSc (both 3.5.4 and 3.7.4) was configured as follow: > export PETSC_ARCH="linux-gnu-dbg" > ./configure --PETSC_ARCH=$PETSC_ARCH \ > --with-cc=gcc \ > --with-cxx=g++ \ > --with-fc=gfortran \ > --COPTFLAGS="-O0" \ > --CXXOPTFLAGS="-O0" \ > --FOPTFLAGS="-O0" \ > --with-debugging=1 \ > --download-fblaslapack \ > --download-mpich \ > --download-hypre \ > --download-yaml \ > --with-x=1 > > Our code was built using the following commands: > mkdir petibm-build > cd petibm-build > export PETSC_DIR=<directory of PETSc> > export PETSC_ARCH="linux-gnu-dbg" > export PETIBM_DIR=<directory of PetIBM git repo> > $PETIBM_DIR/configure --prefix=$PWD \ > CXX=$PETSC_DIR/$PETSC_ARCH/bin/mpicxx \ > CXXFLAGS="-g -O0 -std=c++11" > make all > make install > > Then > cd examples > make examples > > The example of the lid-driven cavity I was talking about can be found in the > folder `examples/2d/convergence/lidDrivenCavity20/20/` > > To run it: > mpiexec -n N <path-to-petibm-build>/bin/petibm2d -directory <path-to-example> > > Let me know if you need more info. Thank you. > > Barry > > > > > > > > > > On Oct 25, 2016, at 4:38 PM, Olivier Mesnard <olivier.mesna...@gmail.com> > > wrote: > > > > Hi all, > > > > We develop a CFD code using the PETSc library that solves the Navier-Stokes > > equations using the fractional-step method from Perot (1993). > > At each time-step, we solve two systems: one for the velocity field, the > > other, a Poisson system, for the pressure field. > > One of our test-cases is a 2D lid-driven cavity flow (Re=100) on a 20x20 > > grid using 1 or 2 procs. > > For the Poisson system, we usually use CG preconditioned with GAMG. > > > > So far, we have been using PETSc-3.5.4, and we would like to update the > > code with the latest release: 3.7.4. > > > > As suggested in the changelog of 3.6, we replaced the routine > > `KSPSetNullSpace()` with `MatSetNullSpace()`. > > > > Here is the list of options we use to configure the two solvers: > > * Velocity solver: prefix `-velocity_` > > -velocity_ksp_type bcgs > > -velocity_ksp_rtol 1.0E-08 > > -velocity_ksp_atol 0.0 > > -velocity_ksp_max_it 10000 > > -velocity_pc_type jacobi > > -velocity_ksp_view > > -velocity_ksp_monitor_true_residual > > -velocity_ksp_converged_reason > > * Poisson solver: prefix `-poisson_` > > -poisson_ksp_type cg > > -poisson_ksp_rtol 1.0E-08 > > -poisson_ksp_atol 0.0 > > -poisson_ksp_max_it 20000 > > -poisson_pc_type gamg > > -poisson_pc_gamg_type agg > > -poisson_pc_gamg_agg_nsmooths 1 > > -poissonksp_view > > -poisson_ksp_monitor_true_residual > > -poisson_ksp_converged_reason > > > > With 3.5.4, the case runs normally on 1 or 2 procs. > > With 3.7.4, the case runs normally on 1 proc but not on 2. > > Why? The Poisson solver diverges because of an indefinite preconditioner > > (only with 2 procs). > > > > We also saw that the routine `MatSetNullSpace()` was already available in > > 3.5.4. > > With 3.5.4, replacing `KSPSetNullSpace()` with `MatSetNullSpace()` led to > > the Poisson solver diverging because of an indefinite matrix (on 1 and 2 > > procs). > > > > Thus, we were wondering if we needed to update something else for the KSP, > > and not just modifying the name of the routine? > > > > I have attached the output files from the different cases: > > * `run-petsc-3.5.4-n1.log` (3.5.4, `KSPSetNullSpace()`, n=1) > > * `run-petsc-3.5.4-n2.log` > > * `run-petsc-3.5.4-nsp-n1.log` (3.5.4, `MatSetNullSpace()`, n=1) > > * `run-petsc-3.5.4-nsp-n2.log` > > * `run-petsc-3.7.4-n1.log` (3.7.4, `MatSetNullSpace()`, n=1) > > * `run-petsc-3.7.4-n2.log` > > > > Thank you for your help, > > Olivier > > <run-petsc-3.5.4-n1.log><run-petsc-3.5.4-n2.log><run-petsc-3.5.4-nsp-n1.log><run-petsc-3.5.4-nsp-n2.log><run-petsc-3.7.4-n1.log><run-petsc-3.7.4-n2.log> > >
