I think what confused me was the fact that using DMCreateMatrix(da,&A) created a 12x12 matrix with 144 elements but summing up nel*nen from each rank gives only 2*2+3*2+3*2+3*2=20 elements. So this means that DMDAGetElements returns the elements for the vector created on the mesh and this happens to be used for indexing the matrix (created using the same DM object) via the local indices. Luckily since this code just creates a tridiagonal matrix, this works here (since if I wanted to create a dense matrix I'd want to have access to all the indices which would happen if the addressable indices at each rank (for the submatrix stored locally) add up to the size of the total matrix). Is my understanding correct?
Thanks for the help! -- Sajid Ali Applied Physics Northwestern University
