On Mon, Feb 6, 2023 at 11:45 AM Nicolas Barnafi <nbarn...@cmm.uchile.cl> wrote:
> Thank you Matt, > > Again, at the bottom of this message you will find the -info output. I > don't see any output related to the fields, > If the splits were done automatically, you would see an info message from here: https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L1595 Thus it must be setup here https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L380 There are info statements, but you do not see them, I do not see a way around using a small example to understand how you are setting up the system, since it is working as expected in the PETSc examples. Thanks, Matt > Best > > > ------ -info > > [0] <sys> PetscDetermineInitialFPTrap(): Floating point trapping is on by > default 13 > [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDeviceType > host available, initializing > [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDevice host > initialized, default device id 0, view FALSE, init type lazy > [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDeviceType > cuda not available > [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDeviceType > hip not available > [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDeviceType > sycl not available > [0] <sys> PetscInitialize_Common(): PETSc successfully started: number of > processors = 1 > [0] <sys> PetscGetHostName(): Rejecting domainname, likely is NIS > nico-santech.(none) > [0] <sys> PetscInitialize_Common(): Running on machine: nico-santech > [0] <sys> SlepcInitialize(): SLEPc successfully started > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960 > 94770087780768 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770087780768 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960 > is being unlinked from inner PETSc comm 94770087780768 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770087780768 > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960 > 94770087780768 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770087780768 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960 > is being unlinked from inner PETSc comm 94770087780768 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770087780768 > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960 > 94770087780768 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770087780768 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960 > is being unlinked from inner PETSc comm 94770087780768 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770087780768 > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960 > 94770087780768 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770087780768 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960 > is being unlinked from inner PETSc comm 94770087780768 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770087780768 > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960 > 94770087780768 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770087780768 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960 > is being unlinked from inner PETSc comm 94770087780768 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770087780768 > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960 > 94770087780768 max tags = 2147483647 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066936960 94770087780768 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 1219 X 1219; storage > space: 0 unneeded,26443 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 150 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 0)/(num_localrows 1219) < 0.6. Do not use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 1160 nodes out of 1219 rows. Not > using Inode routines > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066936960 94770087780768 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066936960 94770087780768 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066936960 94770087780768 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066936960 94770087780768 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066936960 94770087780768 > [0] <sys> PetscGetHostName(): Rejecting domainname, likely is NIS > nico-santech.(none) > [0] <pc> PCSetUp(): Setting up PC for first time > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 615 X 615; storage space: > 0 unneeded,9213 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 117 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 0)/(num_localrows 615) < 0.6. Do not use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 561 nodes out of 615 rows. Not > using Inode routines > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048 > 94770110251424 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770110251424 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066934048 > is being unlinked from inner PETSc comm 94770110251424 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770110251424 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770110251424 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 64 X 64; storage space: 0 > unneeded,0 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 0 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 64)/(num_localrows 64) > 0.6. Use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 13 nodes of 64. Limit used: 5. > Using Inode routines > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048 > 94770100861088 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770100861088 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066934048 > is being unlinked from inner PETSc comm 94770100861088 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770100861088 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770100861088 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 240 X 240; storage space: > 0 unneeded,2140 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 11 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 0)/(num_localrows 240) < 0.6. Do not use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 235 nodes out of 240 rows. Not > using Inode routines > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048 > 94770100861088 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770100861088 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066934048 > is being unlinked from inner PETSc comm 94770100861088 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770100861088 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770100861088 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 300 X 300; storage space: > 0 unneeded,2292 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 33 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 0)/(num_localrows 300) < 0.6. Do not use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 300 nodes out of 300 rows. Not > using Inode routines > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048 > 94770100861088 max tags = 2147483647 > [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc > communicator embedded in a user MPI_Comm 94770100861088 > [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066934048 > is being unlinked from inner PETSc comm 94770100861088 > [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770100861088 > [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an > MPI_Comm 94770100861088 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 615 X 1219; storage space: > 0 unneeded,11202 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 150 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 0)/(num_localrows 615) < 0.6. Do not use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 561 nodes out of 615 rows. Not > using Inode routines > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 64 X 1219; storage space: > 0 unneeded,288 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 6 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 0)/(num_localrows 64) < 0.6. Do not use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 64 nodes out of 64 rows. Not using > Inode routines > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 240 X 1219; storage space: > 0 unneeded,8800 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 78 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 0)/(num_localrows 240) < 0.6. Do not use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 235 nodes out of 240 rows. Not > using Inode routines > [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 300 X 1219; storage space: > 0 unneeded,6153 used > [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 89 > [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows > 0)/(num_localrows 300) < 0.6. Do not use CompressedRow routines. > [0] <mat> MatSeqAIJCheckInode(): Found 300 nodes out of 300 rows. Not > using Inode routines > [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048 > 94770100861088 max tags = 2147483647 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066934048 94770100861088 > 0 KSP Residual norm 2.541418258630e+01 > [0] <ksp> KSPConvergedDefault(): user has provided nonzero initial guess, > computing 2-norm of RHS > [0] <pc> PCSetUp(): Leaving PC with identical preconditioner since > operator is unchanged > [0] <pc> PCSetUp(): Leaving PC with identical preconditioner since > operator is unchanged > [0] <pc> PCSetUp(): Setting up PC for first time > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066934048 94770100861088 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066934048 94770100861088 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066934048 94770100861088 > [0] <pc> PCSetUp(): Leaving PC with identical preconditioner since > operator is unchanged > [0] <pc> PCSetUp(): Setting up PC for first time > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066934048 94770100861088 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066934048 94770100861088 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066934048 94770100861088 > [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator > 94770066934048 947701008610882 > > > On 03/02/23 21:11, Matthew Knepley wrote: > > On Fri, Feb 3, 2023 at 3:03 PM Nicolas Barnafi <nbarn...@cmm.uchile.cl> > wrote: > >> > There are a number of common errors: >> > >> > 1) Your PC has a prefix >> > >> > 2) You have not called KSPSetFromOptions() here >> > >> > Can you send the -ksp_view output? >> >> The PC at least has no prefix. I had to set ksp_rtol to 1 to get through >> the solution process, you will find both the petsc_rc and the ksp_view >> at the bottom of this message. >> >> Options are indeed being set from the options file, but there must be >> something missing at a certain level. Thanks for looking into this. >> > > Okay, the next step is to pass > > -info > > and send the output. This will tell us how the default splits were done. > If that > is not conclusive, we will have to use the debugger. > > Thanks, > > Matt > > >> Best >> >> ---- petsc_rc file >> >> -ksp_monitor >> -ksp_type gmres >> -ksp_view >> -mat_type aij >> -ksp_norm_type unpreconditioned >> -ksp_atol 1e-14 >> -ksp_rtol 1 >> -pc_type fieldsplit >> -pc_fieldsplit_type multiplicative >> >> ---- ksp_view >> >> KSP Object: 1 MPI process >> type: gmres >> restart=500, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> happy breakdown tolerance 1e-30 >> maximum iterations=10000, nonzero initial guess >> tolerances: relative=1., absolute=1e-14, divergence=10000. >> right preconditioning >> using UNPRECONDITIONED norm type for convergence test >> PC Object: 1 MPI process >> type: fieldsplit >> FieldSplit with MULTIPLICATIVE composition: total splits = 4 >> Solver info for each split is in the following KSP objects: >> Split number 0 Defined by IS >> KSP Object: (fieldsplit_0_) 1 MPI process >> type: preonly >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: (fieldsplit_0_) 1 MPI process >> type: ilu >> PC has not been set up so information may be incomplete >> out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 2.22045e-14 >> matrix ordering: natural >> matrix solver type: petsc >> matrix not yet factored; no additional information available >> linear system matrix = precond matrix: >> Mat Object: (fieldsplit_0_) 1 MPI process >> type: seqaij >> rows=615, cols=615 >> total: nonzeros=9213, allocated nonzeros=9213 >> total number of mallocs used during MatSetValues calls=0 >> not using I-node routines >> Split number 1 Defined by IS >> KSP Object: (fieldsplit_1_) 1 MPI process >> type: preonly >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: (fieldsplit_1_) 1 MPI process >> type: ilu >> PC has not been set up so information may be incomplete >> out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 2.22045e-14 >> matrix ordering: natural >> matrix solver type: petsc >> matrix not yet factored; no additional information available >> linear system matrix = precond matrix: >> Mat Object: (fieldsplit_1_) 1 MPI process >> type: seqaij >> rows=64, cols=64 >> total: nonzeros=0, allocated nonzeros=0 >> total number of mallocs used during MatSetValues calls=0 >> using I-node routines: found 13 nodes, limit used is 5 >> Split number 2 Defined by IS >> KSP Object: (fieldsplit_2_) 1 MPI process >> type: preonly >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: (fieldsplit_2_) 1 MPI process >> type: ilu >> PC has not been set up so information may be incomplete >> out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 2.22045e-14 >> matrix ordering: natural >> matrix solver type: petsc >> matrix not yet factored; no additional information available >> linear system matrix = precond matrix: >> Mat Object: (fieldsplit_2_) 1 MPI process >> type: seqaij >> rows=240, cols=240 >> total: nonzeros=2140, allocated nonzeros=2140 >> total number of mallocs used during MatSetValues calls=0 >> not using I-node routines >> Split number 3 Defined by IS >> KSP Object: (fieldsplit_3_) 1 MPI process >> type: preonly >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: (fieldsplit_3_) 1 MPI process >> type: ilu >> PC has not been set up so information may be incomplete >> out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 2.22045e-14 >> matrix ordering: natural >> matrix solver type: petsc >> matrix not yet factored; no additional information available >> linear system matrix = precond matrix: >> Mat Object: (fieldsplit_3_) 1 MPI process >> type: seqaij >> rows=300, cols=300 >> total: nonzeros=2292, allocated nonzeros=2292 >> total number of mallocs used during MatSetValues calls=0 >> not using I-node routines >> linear system matrix = precond matrix: >> Mat Object: 1 MPI process >> type: seqaij >> rows=1219, cols=1219 >> total: nonzeros=26443, allocated nonzeros=26443 >> total number of mallocs used during MatSetValues calls=0 >> not using I-node routines >> solving time: 0.00449609 >> iterations: 0 >> estimated error: 25.4142 >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>