RE: [openib-general] CM private data
All three of these have mechanisms whereby the message can be skipped, in which case applications should not depend on the private data (IB spec mentions that apps should not depend on them). For example, A inbound receive while in RTR state after having sent a REP can be treated as an RTU, in which case later arrival of the RTU is to be discarded by the CM. Todd Rimmer > -Original Message- > From: Tillier, Fabian > Sent: Wednesday, May 18, 2005 3:28 PM > To: 'Sean Hefty'; openib-general > Subject: RE: [openib-general] CM private data > > > > From: Sean Hefty [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, May 18, 2005 12:23 PM > > > > Do any applications make use of the private data in these > CM message: RTU, > > MRA, or DREP? I'm doubtful of the RTU or DREP, but not as > sure of the > MRA. > > > > Since no replies are generated in response to these > messages, the CM does > > not keep them after their sends complete. However, it may > need to resend > > the messages. For example, it will resend the RTU if a > duplicate REP is > > received. > > > > If no applications are using the private data, I will not > worry about > > storing the private data for the retransmissions at this time. > > If you're not going to store them, I would suggest removing > the private data > for the associated calls (RTU, MRA, DREP). That way it > becomes very clear > to applications wishing to use the private data that they can't. > > - Fab > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] CM private data
> From: Sean Hefty [mailto:[EMAIL PROTECTED] > Sent: Thursday, May 19, 2005 9:40 AM > > Fab Tillier wrote: > > I'm still for hiding the RTU private data. I think it's useless because > > it's unreliable - anything exchanged via private data in the RTU must > > also be exchanged by other means in case the connection is established > > before the RTU is received. Any ULPs that depend on the RTU private > > data are setting themselves up for potential failures. > > This depends on the implementation. If the server side of a connection > initiates the data transfer, it cannot do so until an RTU is received. Good point. So this gets back to the CM providing the user's RTU private data even in retries. - Fab ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
On Thu, May 19, 2005 at 09:34:16AM -0700, Fab Tillier wrote: > > From: Caitlin Bestler [mailto:[EMAIL PROTECTED] > > Sent: Thursday, May 19, 2005 9:25 AM > > > > So being predictably unreliable for one implementation > > stage is certainly something you can get away with. > > Even when you add support it might be quite acceptable > > to send the private data *only* on the first try, or to > > require the IT-API layer to do the retries. > > I'm still for hiding the RTU private data. I think it's useless because > it's unreliable - anything exchanged via private data in the RTU must also > be exchanged by other means in case the connection is established before the > RTU is received. Any ULPs that depend on the RTU private data are setting > themselves up for potential failures. I agree for exactly the reason you give, I can't think of a legitimate use for RTU private data. I'd get rid of it entirely, from the code as well as the spec, which is why I think it would be a waste of someones time to add the correct support for it. -Libor ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
Fab Tillier wrote: I'm still for hiding the RTU private data. I think it's useless because it's unreliable - anything exchanged via private data in the RTU must also be exchanged by other means in case the connection is established before the RTU is received. Any ULPs that depend on the RTU private data are setting themselves up for potential failures. This depends on the implementation. If the server side of a connection initiates the data transfer, it cannot do so until an RTU is received. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] CM private data
> From: Caitlin Bestler [mailto:[EMAIL PROTECTED] > Sent: Thursday, May 19, 2005 9:25 AM > > So being predictably unreliable for one implementation > stage is certainly something you can get away with. > Even when you add support it might be quite acceptable > to send the private data *only* on the first try, or to > require the IT-API layer to do the retries. Only sending the user's private data on the first try requires the user to support connection establishment with: - no private data (no RTU received) - zero private data (RTU retry) - remote peer's private data (first RTU) The receipt of private data does not mean that private data is actually what the remote side sent, and also requires users to never use all zero private data since that would make the second and third case above indistinguishable. So if the CM is going to expose the private data, it needs to put that private data in all retries. I'm still for hiding the RTU private data. I think it's useless because it's unreliable - anything exchanged via private data in the RTU must also be exchanged by other means in case the connection is established before the RTU is received. Any ULPs that depend on the RTU private data are setting themselves up for potential failures. Just my opinion, though... - Fab ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
In that case, I'll point out that no application can rely upon the final data being delivered because the first data can easily beat the RTU. When that happens the IT-API layer declares the connection established, and provides no private data. So being predictably unreliable for one implementation stage is certainly something you can get away with. Even when you add support it might be quite acceptable to send the private data *only* on the first try, or to require the IT-API layer to do the retries. I was mostly responding to the suggestion that the private data be eliminated from the API if nobody had any use for it. On 5/19/05, Sean Hefty <[EMAIL PROTECTED]> wrote: > Caitlin Bestler wrote: > > In any event, if no support is going to be provided for > > private data on the second it_ep_accept at the verb > > layer then that should be explicitly documented, and > > I'd suggest sending a 'heads up' to the IT-API authors, > > To clarify, I was only trying to determine when to implement this, not if. > Based on the feedback, I will try to fix this as part of my next set of > changes to the CM. > > Thanks, > Sean > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
Caitlin Bestler wrote: In any event, if no support is going to be provided for private data on the second it_ep_accept at the verb layer then that should be explicitly documented, and I'd suggest sending a 'heads up' to the IT-API authors, To clarify, I was only trying to determine when to implement this, not if. Based on the feedback, I will try to fix this as part of my next set of changes to the CM. Thanks, Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
With a two-way handshake only the passive side accepts the connection request (it_ep_accept() or dat_cr_accept()). IT-API defines an optional *three-way* handshake where the active side must *also* call it_ep_accept() before the final connection establishment can proceed. This does not map to iWARP/MPA in any standard way, so rather than defining a protocol to wrap the first handshake private data, DAT decided to stick with only two-way handshaking. There has been no screaming, so we can presume that *most* applications find two-way handshaking acceptable. However an IT-API provider is free to say that it supports three-way handshaking, and I believe it is implied (if not required) that InfiniBand providers do so. What I do not know is if any actual applications have ever made use of this capability, or if it is only a defensive encoding of a protocol option into an API that prefers not to avoid removal of options that are available at the wire level. DAT emphasized providing a clean API for transport neutral services, and so simply said "use two-way handshaking". In any event, if no support is going to be provided for private data on the second it_ep_accept at the verb layer then that should be explicitly documented, and I'd suggest sending a 'heads up' to the IT-API authors, On 5/19/05, Or Gerlitz <[EMAIL PROTECTED]> wrote: > Caitlin> I believe that an IT-API it_ep_accept() supplies private data > that it expects to be delivered to its peer when the three-way handshake > option is selected. > > Both IT-API it_ep_accept() and DAT dat_cr_accept() cause the provider at > the passive side to send a --REP--, > so how --RTU-- is related to the _accept calls? > > I think you ment to say that exposing two-way handshake means that the > consumer can not supply private data for > the RTU as he is not aware of it (the RTU) being sent? > > Or. > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Caitlin Bestler > Sent: Thursday, May 19, 2005 2:16 AM > To: Sean Hefty > Cc: openib-general > Subject: Re: [openib-general] CM private data > > I believe that an IT-API it_ep_accept() supplies private data that it > expects to be delivered to its peer when the three-way handshake option > is selected. > > DAT only exposes a two-way handshake, so there it never requires private > data on the RTU. > > I don't know if any IT-API applications actually require the three-way > handshake. > > > On 5/18/05, Sean Hefty <[EMAIL PROTECTED]> wrote: > > Do any applications make use of the private data in these CM message: > > RTU, MRA, or DREP? I'm doubtful of the RTU or DREP, but not as sure > of the MRA. > > > > Since no replies are generated in response to these messages, the CM > > does not keep them after their sends complete. However, it may need > > to resend the messages. For example, it will resend the RTU if a > > duplicate REP is received. > > > > If no applications are using the private data, I will not worry about > > storing the private data for the retransmissions at this time. > > > > - Sean > > ___ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] CM private data
Caitlin> I believe that an IT-API it_ep_accept() supplies private data that it expects to be delivered to its peer when the three-way handshake option is selected. Both IT-API it_ep_accept() and DAT dat_cr_accept() cause the provider at the passive side to send a --REP--, so how --RTU-- is related to the _accept calls? I think you ment to say that exposing two-way handshake means that the consumer can not supply private data for the RTU as he is not aware of it (the RTU) being sent? Or. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Caitlin Bestler Sent: Thursday, May 19, 2005 2:16 AM To: Sean Hefty Cc: openib-general Subject: Re: [openib-general] CM private data I believe that an IT-API it_ep_accept() supplies private data that it expects to be delivered to its peer when the three-way handshake option is selected. DAT only exposes a two-way handshake, so there it never requires private data on the RTU. I don't know if any IT-API applications actually require the three-way handshake. On 5/18/05, Sean Hefty <[EMAIL PROTECTED]> wrote: > Do any applications make use of the private data in these CM message: > RTU, MRA, or DREP? I'm doubtful of the RTU or DREP, but not as sure of the MRA. > > Since no replies are generated in response to these messages, the CM > does not keep them after their sends complete. However, it may need > to resend the messages. For example, it will resend the RTU if a > duplicate REP is received. > > If no applications are using the private data, I will not worry about > storing the private data for the retransmissions at this time. > > - Sean > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
Caitlin Bestler wrote: I believe that an IT-API it_ep_accept() supplies private data that it expects to be delivered to its peer when the three-way handshake option is selected. DAT only exposes a two-way handshake, so there it never requires private data on the RTU. I don't know if any IT-API applications actually require the three-way handshake. There may also be some lustre related implementations that make use of the private data in the RTU, so it sounds like this may be needed. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
I believe that an IT-API it_ep_accept() supplies private data that it expects to be delivered to its peer when the three-way handshake option is selected. DAT only exposes a two-way handshake, so there it never requires private data on the RTU. I don't know if any IT-API applications actually require the three-way handshake. On 5/18/05, Sean Hefty <[EMAIL PROTECTED]> wrote: > Do any applications make use of the private data in these CM message: RTU, > MRA, or DREP? I'm doubtful of the RTU or DREP, but not as sure of the MRA. > > Since no replies are generated in response to these messages, the CM does > not keep them after their sends complete. However, it may need to resend > the messages. For example, it will resend the RTU if a duplicate REP is > received. > > If no applications are using the private data, I will not worry about > storing the private data for the retransmissions at this time. > > - Sean > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
James Lentini wrote: If future CM users aren't aware of this, they are bound use this feature and encounter hard to find bugs. Did you consider using asserts to detect when clients uses unimplemented parts of the API? I did consider adding a failure if private data is sent in the MRA, DREP, or RTU. But I first want to know if any current applications are using private data for those messages. There is a subtle issue with the current retransmission mechanism that I need to fix that brought this to my attention, and if private data is needed, then I will just fix that now as well. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
On Wed, 18 May 2005, Sean Hefty wrote: Fab Tillier wrote: If you're not going to store them, I would suggest removing the private data for the associated calls (RTU, MRA, DREP). That way it becomes very clear to applications wishing to use the private data that they can't. That makes it more difficult to add in the support later, since it involves an API change and would effect all clients. As long as no one is using the private data yet, then the implementation of storing the private data can just be deferred until it is needed. If future CM users aren't aware of this, they are bound use this feature and encounter hard to find bugs. Did you consider using asserts to detect when clients uses unimplemented parts of the API? james ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] CM private data
Fab Tillier wrote: If you're not going to store them, I would suggest removing the private data for the associated calls (RTU, MRA, DREP). That way it becomes very clear to applications wishing to use the private data that they can't. That makes it more difficult to add in the support later, since it involves an API change and would effect all clients. As long as no one is using the private data yet, then the implementation of storing the private data can just be deferred until it is needed. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] CM private data
> From: Sean Hefty [mailto:[EMAIL PROTECTED] > Sent: Wednesday, May 18, 2005 12:23 PM > > Do any applications make use of the private data in these CM message: RTU, > MRA, or DREP? I'm doubtful of the RTU or DREP, but not as sure of the MRA. > > Since no replies are generated in response to these messages, the CM does > not keep them after their sends complete. However, it may need to resend > the messages. For example, it will resend the RTU if a duplicate REP is > received. > > If no applications are using the private data, I will not worry about > storing the private data for the retransmissions at this time. If you're not going to store them, I would suggest removing the private data for the associated calls (RTU, MRA, DREP). That way it becomes very clear to applications wishing to use the private data that they can't. - Fab ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] CM private data
Do any applications make use of the private data in these CM message: RTU, MRA, or DREP? I'm doubtful of the RTU or DREP, but not as sure of the MRA. Since no replies are generated in response to these messages, the CM does not keep them after their sends complete. However, it may need to resend the messages. For example, it will resend the RTU if a duplicate REP is received. If no applications are using the private data, I will not worry about storing the private data for the retransmissions at this time. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
