Hi,
I'm working with a 3D DMDA, with 3 dof per "node", used to create a sparse
matrix Mat K. The Mat is modified repeatedly by the program, using the commands
(in that order) :
MatZeroEntries(K)
In a for loop : MatSetValuesLocal(K, 24, irow, 24, icol, vals, ADD_VALUES)
MatAssemblyBegin(K, MAT_FINAL_ASSEMBLY)
MatAssemblyEnd(K, MAT_FINAL_ASSEMBLY)
MatDiagonalScale(K, vec1, vec1)
MatDiagonalSet(K, vec2, ADD_VALUES)
Computing time seems high and I would like to improve it. Running tests with
"-log_view" tells me that MatScale() is the bottle neck (50% of total computing
time) . From manual pages, I've tried a few tweaks :
* DMSetMatType(da, MATMPIBAIJ) : "For problems with multiple degrees of
freedom per node, ... BAIJ can significantly enhance performance", Chapter
14.2.4
* Used MatMissingDiagonal() to confirm there is no missing diagonal entries
: "If the matrix Y is missing some diagonal entries this routine can be very
slow", MatDiagonalSet() manual
* Tried MatSetOption()
* MAT_NEW_NONZERO_LOCATIONS == PETSC_FALSE : to increase assembly
efficiency
* MAT_NEW_NONZERO_LOCATION_ERR == PETSC_TRUE : "When true, assembly
processes have one less global reduction"
* MAT_NEW_NONZERO_ALLOCATION_ERR == PETSC_TRUE : "When true, assembly
processes have one less global reduction"
* MAT_USE_HASH_TABLE == PETSC_TRUE : "Improve the searches during matrix
assembly"
According to "-log_view", assembly is fast (0% of total time), and the use of a
DMDA makes me believe preallocation isn't the cause of performance issue.
I would like to know how could I improve MatScale(). What are the best
practices (during allocation, when defining Vecs and Mats, the DMDA, etc.)?
Instead of MatDiagonalScale(), should I use another command to obtain the same
result faster?
Thank you very much!
Antoine Côté
