On Dec 7, 2007 9:58 AM, Tim Stitt <timothy.stitt at ichec.ie> wrote: > Barry, > > I added the following lines to my Fortran code: > > call KSPGetPC(ksp,pc,error) > call KSPSetUp(ksp,error) > oneInt=1 > call PCBJacobiGetSubKSP(pc,oneInt,PETSC_NULL,kspSub,error) > call KSPGetPC(kspSub,pcSub,error) > call PCFactorSetShiftNonzero(pcSub,PETSC_DECIDE,error) > > Now the parallel code goes beyond the zero pivot problem I was getting > in the KSPSolve()...but only process 0 seems to complete the KSPSolve() > and Process 1 and higher never makes it out of the KSPSolve() i.e. > process 0 moves on and performs post-KSPSolve work (just some print > statements) while the other processes never get out of KSPSolve(). My > job only terminates once the requested wallclock expires. Again when > running with only 1 process everything terminates successfully.
This does not seem possible. BJacobi synchronizes at each step for a residual evaluation. Are you sure you did not call KSPSolve() on the inner KSP? Matt > Any ideas? Have I done something stupid with the instructions above? > > Thanks, > > Tim. > > Barry Smith wrote: > > > > On Dec 6, 2007, at 6:09 AM, Tim Stitt wrote: > > > >> Barry, > >> > >> I will be using these routines from Fortran..so I am assuming that > >> Fortran interfaces are available for each routine? > >> > >> Also, how do I know how many sub ksp's there will be? I am assuming I > >> need to dynamically allocate the subksp array in Fortran but do I > >> know the size in advance? Is this related to the value 'n' ? If so, > >> how do I calculate 'n'. > > > > There will always be one sub ksp be default. There will only be > > more than one if you use > > PCBJacobiSetLocalBlocks() or PCBJacobiSetTotalBlocks() or > > -pc_bjacobi_blocks. > > In general we recommend keeping it one. This means you do not need to > > allocate > > any KSP, just pass in a KSP variable > > > >> > >> > >> What is the significance of subksp[0]? Is it just the sub ksp at this > >> position I should be interested in? > > > > This is just the first one. If you have multiply ones then you must > > loop over them, but I > > recommend having just one. > >> > >> > >> Finally, which of the PCFactorSetxxxxxx routines should I be using? > > > > PCFactorSetZeroPivot() or PCFactorSetShiftNonzero() or > > PCFactorSetShiftPd() depending > > on what you want to have happen. > > > > Barry > > > >> > >> > >> Sorry for the twenty questions (well nearly) but I am just a bit > >> confused with this approach. > >> > >> Thanks, > >> > >> Tim. > >> > >> Barry Smith wrote: > >>> > >>> KSP *subksp; > >>> > >>> KSPGetPC(ksp,pc) > >>> PCBJacobiGetSubKSP(pc,&n,PETSC_NULL,&subksp) > >>> KSPGetPC(subksp[0],&subpc); > >>> PCFactorSetxxxxxx(subpc, .... > >>> > >>> Barry > >>> > >>> > >>> On Nov 25, 2007, at 1:10 PM, Tim Stitt wrote: > >>> > >>>> I should also add that the code executes without this error when > >>>> using 1 processor...but then displays the error when running in > >>>> parallel with more than one process. > >>>> > >>>> Tim Stitt wrote: > >>>>> Hi all, > >>>>> > >>>>> Can anyone suggest ways of overcoming the following pivot error I > >>>>> keep receiving in my PETSc code during a KSPSolve(). > >>>>> > >>>>> [1]PETSC ERROR: Detected zero pivot in LU factorization > >>>>> see > >>>>> http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#ZeroPivot! > >>>>> > >>>>> [1]PETSC ERROR: Zero pivot row 1801 value 0.00102826 tolerance > >>>>> 0.00165189 * rowsum 1.65189e+09! > >>>>> > >>>>> From checking the documentation....the error is in row 1801, which > >>>>> means it is most likely not a matrix assembly issue? > >>>>> > >>>>> I tried the following prior to the solve with no luck either..... > >>>>> > >>>>> call KSPGetPC(ksp,pc,error) > >>>>> call PCFactorSetShiftNonzero(pc,PETSC_DECIDE,error) > >>>>> > >>>>> Is there anything else I can try? > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Tim. > >>>>> > >>>> > >>>> > >>>> --Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie> > >>>> HPC Application Consultant - ICHEC (www.ichec.ie) > >>>> > >>>> Dublin Institute for Advanced Studies > >>>> 5 Merrion Square - Dublin 2 - Ireland > >>>> > >>>> +353-1-6621333 (tel) / +353-1-6621477 (fax) > >>>> > >>> > >> > >> > >> --Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie> > >> HPC Application Consultant - ICHEC (www.ichec.ie) > >> > >> Dublin Institute for Advanced Studies > >> 5 Merrion Square - Dublin 2 - Ireland > >> > >> +353-1-6621333 (tel) / +353-1-6621477 (fax) > >> > > > > > -- > Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie> > HPC Application Consultant - ICHEC (www.ichec.ie) > > Dublin Institute for Advanced Studies > 5 Merrion Square - Dublin 2 - Ireland > > +353-1-6621333 (tel) / +353-1-6621477 (fax) > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
