Re: [OMPI users] using multiple IB connections between hosts

2015-02-01 Thread Gilles Gouaillardet
Dave, the QDR Infiniband uses the openib btl (by default : btl_openib_exclusivity=1024) i assume the RoCE 10Gbps card is using the tcp btl (by default : btl_tcp_exclusivity=100) that means that by default, when both openib and tcp btl could be used, the tcp btl is discarded. could you give a

[OMPI users] using multiple IB connections between hosts

2015-01-28 Thread Dave Turner
I ran some aggregate bandwidth tests between 2 hosts connected by both QDR InfiniBand and RoCE enabled 10 Gbps Mellanox cards. The tests measured the aggregate performance for 16 cores on one host communicating with 16 on the second host. I saw the same performance as with the QDR