On Wed, Jan 13, 2010 at 07:57:26PM +0100, Bart Van Assche wrote: > On Wed, Jan 13, 2010 at 7:16 PM, Jason Gunthorpe > <jguntho...@obsidianresearch.com> wrote: > > > > On Wed, Jan 13, 2010 at 08:23:27AM +0100, Bart Van Assche wrote: > > > On Wed, Jan 13, 2010 at 12:24 AM, Jason Gunthorpe > > > <jguntho...@obsidianresearch.com> wrote: > > > > [ ... ] > > > > > > > > Also, I couldn't tell for sure from a cursory examination of the > > > > patch, but the initiator must be designed to not stall processing the > > > > RQ dependent on the SQ or the credit level, when using a credit scheme > > > > like this. Or you will deadlock. > > > > > > > > For instance it isn't clear to me how the control flow around > > > > srp_process_cred_req is ment to work - it tries to send a reply, but > > > > if it can't due to srp_get_txp_iu failing it just gives up forever? > > > > > > For a standards-conforming SRP target, srp_get_txp_iu() will never > > > fail. A quote from section 5.5.2 of the SRP r16a document: > > > > What guarantees that that the send completion for the reply is > > processed before a receive completion for the next request? > > Did you notice that I increased the number of receive slots reserved > for target-initiated requests from one to two ?
Yes, but I don't see how more receive slots helps exhausting send slots. The IBA doesn't guarentee ordering between completions originating from different work queues. Getting a receive completion does not guarantee that a related send completion has been received. Even if the work queues are mapped to the same CQ. Since srp_get_txp_iu depends on txp_head/tail which is incremented by send completions, but srp_get_txp_iu is called on the receive completion path, it is possible to call srp_get_txp_iu multiple times before the send completions are generated - even though the target is only generating new request sends in response to reply receives. The two main causes are ack coalescing algorithms and lost ack packets. That said, with SRP_TXP_SIZE set to 2, I think it would be really hard to break this in real life, with current hardware. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
