Doug Ledford wrote:
On Thu, 2007-10-11 at 12:39 -0500, Steve Wise wrote:
Doug Ledford wrote:
So, one of the options when creating a QP is the max inline data size.
If I understand this correctly, for any send up to that size, the
payload of that send will be transmitted to the receiving side along
with the request to send.
What it really means is the payload is DMA'd to the HW on the local side
in the work request itself as opposed being DMA'd down in a 2nd
transaction after the WR is DMA'd and processed. It has no end-to-end
significance. Other than to reduce the latency needed to transfer the data.
OK, that clears things up for me ;-)
This reduces back and forth packet counts on
the wire in the case that the receiving side is good to go, because it
basically just responds with "OK, got it" and you're done.
I don't think this is true. Definitely not with iWARP. INLINE is just
an optimization to push small amts of data downto the local adapter as
part of the work request, thus avoiding 2 DMA's.
Even though you create the QP with the inline option, only WRs that pass
in the IBV_SEND_INLINE flag will do inline processing, so you can
control this functionality at a per-WR basis.
Hmm..that raises a question on my part. You don't call ibv_reg_mr on
the wr itself, so if the data is pushed with the wr, do you still need
to call ibv_reg_mr on the data separately?
The WR DMA'd by the HW is actually built in memory that is setup for the
adapter to DMA from. Whether that is really done via ibv_reg_mr or some
other method is provider/vendor specific. So the WR you pass into
ibv_post_send() is always copied and munged into the HW-specific memory
and format. For inline sends, the data you pass in via the SGL is
copied into the HW-specific WR memory as well.
And from the man page on ibv_post_send(), I conclude you do _not_ have
to register the payload memory used in an INLINE send:
IBV_SEND_INLINE Send data in given gather list as inline data
in a send WQE. Valid only for Send and RDMA Write. The L_Key
will not be checked.
Steve.
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
