Jeff Squyres writes:
> Assuming you built OMPI with PSM support:
>
> mpirun --mca pml cm --mca mtl psm
>
> (although probably just the pml/cm setting is sufficient -- the mtl/psm
> option will probably happen automatically)
For what it's worth, you needn't specify anything to get psm u
If you're using QLogic, you might want to try the native PSM Open MPI support
rather than the verbs support. QLogic cards only "sorta" support verbs in
order to say that they're OFED-complaint; their native PSM interface is more
performant than verbs for MPI.
Assuming you built OMPI with PSM s
Hi,
Thanks for getting back to me (and thanks to Jeff for the explanation
too).
On Thu, 2011-05-19 at 09:59 -0600, Samuel K. Gutierrez wrote:
> Hi,
>
> On May 19, 2011, at 9:37 AM, Robert Horton wrote
>
> > On Thu, 2011-05-19 at 08:27 -0600, Samuel K. Gutierrez wrote:
> >> Hi,
> >>
> >> Try th
What Sam is alluding to is that the OpenFabrics driver code in OMPI is sucking
up oodles of memory for each IB connection that you're using. The
receive_queues param that he sent tells OMPI to use all shared receive queues
(instead of defaulting to one per-peer receive queue and the rest shared
Hi,
On May 19, 2011, at 9:37 AM, Robert Horton wrote
> On Thu, 2011-05-19 at 08:27 -0600, Samuel K. Gutierrez wrote:
>> Hi,
>>
>> Try the following QP parameters that only use shared receive queues.
>>
>> -mca btl_openib_receive_queues S,12288,128,64,32:S,65536,128,64,32
>>
>
> Thanks for tha
On Thu, 2011-05-19 at 08:27 -0600, Samuel K. Gutierrez wrote:
> Hi,
>
> Try the following QP parameters that only use shared receive queues.
>
> -mca btl_openib_receive_queues S,12288,128,64,32:S,65536,128,64,32
>
Thanks for that. If I run the job over 2 x 48 cores it now works and the
performa
Hi,
Try the following QP parameters that only use shared receive queues.
-mca btl_openib_receive_queues S,12288,128,64,32:S,65536,128,64,32
Samuel K. Gutierrez
Los Alamos National Laboratory
On May 19, 2011, at 5:28 AM, Robert Horton wrote:
> Hi,
>
> I'm having problems getting the MPIRandomA
Hi,
I'm having problems getting the MPIRandomAccess part of the HPCC
benchmark to run with more than 32 processes on each node (each node has
4 x AMD 6172 so 48 cores total). Once I go past 32 processes I get an
error like:
[compute-1-13.local][[5637,1],18][../../../../../ompi/mca/btl/openib/conn
