It's complicated, but the short answer is that a "short" message is defined as one where the cost of a memory copy doesn't matter.
You could always use mpi_alloc_mem to get registered memory. But I don't recall offhand if we check to see if the memory is already registered for short messages (because the cost of the memcpy is insignificant). Yes, the cost of mem reg is one of the main drivers for these kinds of optimizations. -jms Sent from my PDA. No type good. ----- Original Message ----- From: users-boun...@open-mpi.org <users-boun...@open-mpi.org> To: us...@open-mpi.org <us...@open-mpi.org> Sent: Mon Jan 04 11:09:10 2010 Subject: [OMPI users] Question about senting short MPI messages using RDMA Hi, guys. As I understand, to send short MPI messages, OpenMPI copies the messages to preallocated buffer and then uses RDMA. I was wondering if we can avoid the overhead of memory copy. If the user buffers for short messages are reused a lot, we can just register the user buffer instead of using preallocated buffer. Then we can do RDMA directly from the user buffer instead of the preallocated buffer. But if the user buffers are not reused, we will suffer from the overhead of memory registration. Besides the overhead of memory registration, is there any other reason that prevent you to do RDMA directly from the user buffer for short messages? Thank you. _______________________________________________ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
