On 11/22/2013 10:13 PM, Anuj Kalia wrote:
Update: I found ways to improve active side performance from 10
million RDMA writes per second to 20 million (which I believe is the
PCIe bottleneck):
1. Use inline payload - I think this reduces PCIe traffic.
Yes, without inline, each IO requires 2 PCIe transactions: 1 to fetch
(or push) the work request, and one to fetch the payload/data. If you
use inline, the data is included in the work request. So you cut the
required transactions in half.
2. Use non-signalled RDMA writes + don't poll for completion for every
write - I don't know if ibv_poll_cq() uses the PCIe much.
Each signaled work request generates a completion entry (CQE) which is
pushed from the adapter into the CQ in host memory. So reducing the
number of these required also reduces the PCIe transactions.
Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
