Hi Folks, First initial disclaimer - I've looked through the open mpi faq and have been unable so far an answer to my question below.
I've been having a discussion with one of the other trilab folks about some issues with using PSM within mvpaich where the default cpu affinity behavior of PSM can cause problems. It turns out that the default behavior of PSM appears to be to set cpu affinity for a process which calls psm_ep_open if process affinity has not already been set. We're finding that it is necesary to use the PSM_EP_OPEN_AFFINITY_SKIP setting in the affinity field of the psm_opts struct that is passed to psm_ep_open in order to work around the problem. The problem has to do with singleton processes. If mvapich is using psm and multiple singleton jobs are scheduled on a node, they all by default end up binding to core 0. Setting the above option eliminates this problem. Could Open MPI also potentially have this same problem? If so, I'd want to add an mca param to set this option before calling psm_ep_open within psm mtl. Hmm.. maybe the ofi mtl supporter should talk with the libfabric psm provider folks about this. Thanks for any help, Howard
