Sorry for the delayed reply. I'm guessing that there's some kind of rooted collective operation occurring during MPI_INIT. Try doing a 1 byte MPI_GATHER to another rank in MCW (e.g., to rank 5) and see if the VmSS goes to the same size as MCW rank 0.
On Jul 8, 2011, at 5:17 AM, Eloi Gaudry wrote: > what i cannot understand is the reason why this extra memory would be > initialized on proc 0 only. > as far as i know, this doesn't make sense. > éloi > >> On 22/04/2011 08:52, Eloi Gaudry wrote: >>> it varies with the receive_queues specification *and* with the number of >>> mpi processes: memory_consumed = nb_mpi_process * nb_buffers * >>> (buffer_size + low_buffer_count_watermark + credit_window_size ) >>> >>> éloi >>> >>> >>> On 04/22/2011 12:26 AM, Jeff Squyres wrote: >>>> Does it vary exactly according to your receive_queues specification? >>>> >>>> On Apr 19, 2011, at 9:03 AM, Eloi Gaudry wrote: >>>> >>>>> hello, >>>>> >>>>> i would like to get your input on this: >>>>> when launching a parallel computation on 128 nodes using openib and the >>>>> "-mca btl_openib_receive_queues P,65536,256,192,128" option, i observe a >>>>> rather large resident memory consumption (2GB: 65336*256*128) on the >>>>> process with rank 0 (and only this process) just after a call to MPI_Init. >>>>> >>>>> i'd like to know why the other processes doesn't behave the same: >>>>> - other processes located on the same nodes don't use that amount of >>>>> memory >>>>> - all others processes (i.e. located on any other nodes) neither >>>>> >>>>> i'm using OpenMPI-1.4.2, built with gcc-4.3.4 and >>>>> '--enable-cxx-exceptions --with-pic --with-threads=posix' options. >>>>> >>>>> thanks for your help, >>>>> éloi > > _______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
