Whether iWARP or IB, there is a fixed number of RDMA Requests allowed to
be outstanding at any given time. If one posts more RDMA Read
requests than the fixed number, the transmit queue is stalled. This
is documented in both technology specifications. It is something
that all ULP should be aw
Todd, thanks for the set-up. I'm really glad we're having this discussion!
Let me give an NFS/RDMA example to illustrate why this upper layer,
at least, doesn't want the HCA doing its flow control, or resource
management.
NFS/RDMA is a credit-based protocol which allows many operations in
progres
Quoting r. Talpey, Thomas <[EMAIL PROTECTED]>:
> But, it really doesn't matter. The point is, an upper layer should be paying
> attention to the number of RDMA Reads it posts, or else suffer either the
> queue-stalling or connection-failing consequences. Bad stuff either way.
Queue-stalling is not
> Talpey, Thomas
> Sent: Tuesday, June 06, 2006 10:49 AM
>
> At 10:40 AM 6/6/2006, Roland Dreier wrote:
> >Thomas> This is the difference between "may" and "must". The
value
> >Thomas> is provided, but I don't see anything in the spec that
> >Thomas> makes a requirement on its enforc
Thomas> I don't see how strained has anything to do with it. It's
Thomas> not saying anything either way. So, a legal implementation
Thomas> can make either choice. We're talking about the spec!
I guess the reason I say it is strained is because the spec does have
the following complia
At 10:40 AM 6/6/2006, Roland Dreier wrote:
>Thomas> This is the difference between "may" and "must". The value
>Thomas> is provided, but I don't see anything in the spec that
>Thomas> makes a requirement on its enforcement. Table 107 says the
>Thomas> consumer can query it, that's a
Thomas> This is the difference between "may" and "must". The value
Thomas> is provided, but I don't see anything in the spec that
Thomas> makes a requirement on its enforcement. Table 107 says the
Thomas> consumer can query it, that's about as close as it
Thomas> comes. There's
At 08:56 AM 6/6/2006, Michael S. Tsirkin wrote:
>> The core spec does not require it. An implementation *may* enforce it,
>> but is not *required* to do so. And as pointed out in the other message,
>> there are repercussions of doing so.
>
>Interesting, I wasn't aware of such interpretation of the
Quoting r. Talpey, Thomas <[EMAIL PROTECTED]>:
> Subject: Re: Mellanox HCAs: outstanding RDMAs
>
> At 03:43 AM 6/6/2006, Michael S. Tsirkin wrote:
> >Quoting r. Talpey, Thomas <[EMAIL PROTECTED]>:
> >> Semantically, the provider is not required to provide any such flow control
> >> behavior by the
At 08:44 AM 6/6/2006, Michael S. Tsirkin wrote:
>> MST, are you disagreeing that RDMA Reads can stall the queue?
>
>I don't disagree with this of course. I was simply suggesting to ULP designers
>to read the chapter 9.5 and become aware of the rules, taking them
>into account at early stages of pro
Quoting r. Talpey, Thomas <[EMAIL PROTECTED]>:
> Subject: Re: Mellanox HCAs: outstanding RDMAs
>
> At 03:09 AM 6/6/2006, Michael S. Tsirkin wrote:
> >Quoting r. somenath <[EMAIL PROTECTED]>:
> >> possibility of stalling is scary!
> >
> >You might want to review chapter 9.5 TRANSACTION ORDERING for
At 03:09 AM 6/6/2006, Michael S. Tsirkin wrote:
>Quoting r. somenath <[EMAIL PROTECTED]>:
>> possibility of stalling is scary!
>
>You might want to review chapter 9.5 TRANSACTION ORDERING for info on when will
>ordering rules cause the IB QP to stall.
MST, are you disagreeing that RDMA Reads can s
At 03:43 AM 6/6/2006, Michael S. Tsirkin wrote:
>Quoting r. Talpey, Thomas <[EMAIL PROTECTED]>:
>> Semantically, the provider is not required to provide any such flow control
>> behavior by the way. The Mellanox one apparently does, but it is not
>> a requirement of the verbs, it's a requirement on
Quoting r. Talpey, Thomas <[EMAIL PROTECTED]>:
> Semantically, the provider is not required to provide any such flow control
> behavior by the way. The Mellanox one apparently does, but it is not
> a requirement of the verbs, it's a requirement on the upper layer. If more
> RDMA Reads are posted th
Quoting r. somenath <[EMAIL PROTECTED]>:
> possibility of stalling is scary!
You might want to review chapter 9.5 TRANSACTION ORDERING for info on when will
ordering rules cause the IB QP to stall.
--
MST
___
openib-general mailing list
openib-general
Quoting r. Manpreet Singh <[EMAIL PROTECTED]>:
> Subject: Re: Mellanox HCAs: outstanding RDMAs
>
> We have seen this happen over an IB analyzer. Recompiling the mthca driver
> with a high value like 64 or 128 works around this problem.
> When the condition hits, the HCA receiving the 4+ RDMAs gen
16 matches
Mail list logo
