Thank you very much! I already checked that its MPIAIJ matrix and for size I use MatGetInfo routine. You are right. With matrix of dim 100000x10000 i get sizes:
- serial: 5.603 Mb - 4 proc.: 7.626 Mb - 36 proc.: 7.834 Mb That looks fine to me. Thank you again for such quick response. I am really impressed with python interface to petsc and slepc. I think it is missing detailed documentation and that discouraged me to use it initially, so I was writing C code and then wrapping it with python. I am still confused, how for example to set MUMPS parameters from python code, but that is different topic. Lukas On Fri, Jan 10, 2020 at 5:21 PM Smith, Barry F. <[email protected]> wrote: > > Yes, with, for example, MATMPAIJ, the matrix entries are distributed > among the processes; first verify that you are using a MPI matrix, not Seq, > since Seq will keep an entire copy on each process. > > But the parallel matrices do come with some overhead for meta data. So > for small matrices like yours it can seem the memory grows unrealistically. > Try a much bigger matrix, say 100 times as big and look at the memory usage > then. You should see that the meta data is now a much smaller percentage of > the memory usage. > > Also be careful if you use top or other such tools for determining > memory usage; since malloced() is often not returned to the OS, the can > indicate much higher memory usage than is really taking place. You can run > PETSc with -log_view -log_view_memory to get a good idea of where PETSc is > allocating memory and how much > > Barry > > > > On Jan 10, 2020, at 7:52 AM, Lukas Razinkovas <[email protected]> > wrote: > > > > Hello, > > > > I am trying to use petsc4py and slepc4py for parallel sparse matrix > diagonalization. > > However I am a bit confused about matrix size increase when I switch > from single processor to multiple processors. For example 100 x 100 matrix > with 298 nonzero elements consumes > > 8820 bytes of memory (mat.getInfo()["memory"]), however on two processes > it consumes 20552 bytes of memory and on four 33528. My matrix is taken > from the slepc4py/demo/ex1.py, > > where nonzero elements are on three diagonals. > > > > Why memory usage increases with MPI processes number? > > I thought that each process stores its own rows and it should stay the > same. Or some elements are stored globally? > > > > Lukas > >
