Hi, Philip, Could you add -log_view and see what functions are used in the solve? Since it is CPU-only, perhaps with -log_view of different runs, we can easily see which functions slowed down.
--Junchao Zhang ________________________________ From: Fackler, Philip <fackle...@ornl.gov> Sent: Tuesday, January 17, 2023 4:13 PM To: xolotl-psi-developm...@lists.sourceforge.net <xolotl-psi-developm...@lists.sourceforge.net>; petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov> Cc: Mills, Richard Tran <rtmi...@anl.gov>; Zhang, Junchao <jczh...@mcs.anl.gov>; Blondel, Sophie <sblon...@utk.edu>; Roth, Philip <rot...@ornl.gov> Subject: Performance problem using COO interface In Xolotl's feature-petsc-kokkos branch I have ported the code to use petsc's COO interface for creating the Jacobian matrix (and the Kokkos interface for interacting with Vec entries). As the attached plots show for one case, while the code for computing the RHSFunction and RHSJacobian perform similarly (or slightly better) after the port, the performance for the solve as a whole is significantly worse. Note: This is all CPU-only (so kokkos and kokkos-kernels are built with only the serial backend). The dev version is using MatSetValuesStencil with the default implementations for Mat and Vec. The port version is using MatSetValuesCOO and is run with -dm_mat_type aijkokkos -dm_vec_type kokkos. The port/def version is using MatSetValuesCOO and is run with -dm_vec_type kokkos (using the default Mat implementation). So, this seems to be due be a performance difference in the petsc implementations. Please advise. Is this a known issue? Or am I missing something? Thank you for the help, Philip Fackler Research Software Engineer, Application Engineering Group Advanced Computing Systems Research Section Computer Science and Mathematics Division Oak Ridge National Laboratory
