Ramaswamy Tummala wrote:
I have a few questions about the openfabrics CMA interfaces for iWARP.
I'd appreciate if anyone could clarify them.
- If RNIC's modify_qp() entry point is called to move the QP state to
CLOSING or
ERROR while there are some WQEs on SQ and RQ, does RNIC flush the
incomplete
WRs on the SQ or RQ?
It is really up to the device, but if the rnic supports the RDMAC verbs
then it will.
If so, does RNIC wait until the flush is complete
before returning modify_qp() to the caller? If RNIC does not wait for the
flush to complete how does the caller know when the flush is complete
(so that caller can poll CQ to retrieve the CQ entries)?
This is up to the provider also. I don't think the verbs specify that
the flush will be done by the time you return from modify_qp(). Its up
to the application to deal with knowing when the flush is done.
[ Another possibility is, when RNIC's modify_qp() entry point called to
move the QP state to CLOSING while there some WQEs on the SQ, the RNIC
would
internally move the QP state to ERROR. My question still is does RNIC
wait until the flushing of incomplete WRs from SQ and RQ are done before
returning modify_qp() to the caller even though it internally
transitioned
the QP state to ERROR. If RNIC does not wait for the flush to complete
how does the caller know when the flush is complete? ]
I think the answer is no, you cannot depend on the flush being complete
when you exit modify_qp...
- If RNIC's modify_qp() entry point called to move the QP state to CLOSING,
does RNIC just initiate LLP CLOSE and return to the caller?, or does
it wait
until LLP CLOSE is complete?.
It initiates the LLP CLOSE. It does not wait for the close to complete.
- It appears that RNIC should send IW_CM_EVENT_DISCONNECT event to CMA
prior
to the start of closing or aborting the connection (except in the case
where the disconnect has been initiated by CMA itself, for example by CMA
calling modify_qp entry point of RNIC to move the QP state to CLOSING or
ERROR). Is this correct?
I'm not sure I understand your question.
- It appears that RNIC should send IW_CM_EVENT_CLOSE event after the
connection
has been closed. Should this event be sent on both active and passive
sides
after the connection has been closed?
Yes.
- RNIC has add_ref(struct ib_qp *qp), and rem_ref(struct ib_qp *qp) entry
points. What is the expected use of CMA calling these entry points? My
general
thinking is that CMA can increase the reference count on QP (i.e.
add_ref)
to prevent the QP from being destroyed by RNIC. But, it is the CMA that
initiates destroying of QP by calling destroy_qp() entry point of RNIC.
So, CMA could maintain the reference count for QP in its own private data
(instead of calling RNIC's add_ref entry point) and not call
destroy_qp() entry point of RNIC if the reference count is not zero.
The iWCM keeps the ref on the QP while the QP is directly associated
with a iw_cm_id.
- It appears that if RNIC's accept() entry point is called to accept an
incoming connection, the RNIC, after successful processing of accept,
would send IW_CM_EVENT_ESTABLISHED event to CMA. What event RNIC should
send if the call to accept() succeeded, but later RNIC encountered some
error in sending MPA reply message to the remote peer or some other
error?
In this case although the call to accept() succeeded, the connection
could
still be not be established. So the RNIC can not send
IW_CM_EVENT_ESTABLISHED event.
It is ok to block in the provider until the connection and qp are bound
and in FPDU mode. That's what the chelsio device does. So when the
ESTABLISHED event is posted, the MPA reply was sent and ACKed and the QP
/connection moved into FPDU mode. Then any problems in the connection
would be posted as IW_CM_EVENT_CLOSE or IW_CM_EVENT_DISCONNECT.
- It appears that a client of CMA needs to call rdma_resolve_route() after
a successful rdma_resolve_addr(). Any reason for the existence of two
interfaces instead of one interface that combines the functionality of
both the interfaces?
Its an infiniband requirement, I think. iWARP doesn't do anything for
resolve route. The "next hop route" in iWARP terms is actually
determined as part to address resolution...
Steve.
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
