So, one of the options when creating a QP is the max inline data size. If I understand this correctly, for any send up to that size, the payload of that send will be transmitted to the receiving side along with the request to send. This reduces back and forth packet counts on the wire in the case that the receiving side is good to go, because it basically just responds with "OK, got it" and you're done. The trade off of course is that if there is a resource shortage on the receiving side, then it sends a RNR packet back, and however much payload data you sent over the wire with the original request to send was just wasted bandwidth as it was thrown away on the receiving side.
So, if my understanding of that is correct, then inline data improves
latency and maximum bandwidth up until the point where the receiving
side starts to have resource problems, then it wastes bandwidth and
doesn't help latency at all. So, if a person wanted to write their
program to use inline data up until this point of congestion, then quit
using it until the congestion clears, how would they go about doing
that? Would I have to set RNR retry count to something ridiculously
small and take the RNR error (along with the corresponding queue flush
and the pain that brings in terms of requeuing all the flushed events)
and do an ibv_modify_qp to turn off inline data until some number of
sends have completed without error? Or is there possibly a counter
somewhere that I can monitor? Or should I just forget about trying to
optimize this part of my code?
Separate question, when using an SRQ, and let's say you have more than 1
QP associate with that SRQ, then does a single QP going into QP_ERR
state flush the SRQ requests, or only the send requests on the QP that's
in error? And if you get down to only 1 QP left attached to the SRQ,
and you then set that QP to the error state, will it flush the SRQ
entries? Reading everything I can on SRQs, it's not clear to me how you
might flush one, especially since setting the SRQ itself to error state
specifically does *not* flush the posted and unused recv requests.
--
Doug Ledford <[EMAIL PROTECTED]>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford
Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband
signature.asc
Description: This is a digitally signed message part
_______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
