Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-15 Thread Michael Krause
At 12:49 PM 11/14/2005, Nitin Hande wrote: Michael Krause wrote: At 01:01 PM 11/11/2005, Nitin Hande wrote: Michael Krause wrote: At 10:28 AM 11/9/2005, Rick Frank wrote: Yes, the application is responsible for detecting lost msgs at the application level - the transport can not do this. RDS

RE: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-15 Thread Caitlin Bestler
In absence of any protocol level ack (and regardless of protocol level ack), it is the application which has to implement its own reliability. RDS becomes a passive channel passing packet back and forth including duplicate packets. The responsibility then shifts to the

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-14 Thread Nitin Hande
Michael Krause wrote: At 01:01 PM 11/11/2005, Nitin Hande wrote: Michael Krause wrote: At 10:28 AM 11/9/2005, Rick Frank wrote: Yes, the application is responsible for detecting lost msgs at the application level - the transport can not do this. RDS does not guarantee that a message has

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-11 Thread Michael Krause
At 10:28 AM 11/9/2005, Rick Frank wrote: Yes, the application is responsible for detecting lost msgs at the application level - the transport can not do this. RDS does not guarantee that a message has been delivered to the application - just that once the transport has accepted a msg it will

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-11 Thread Nitin Hande
Michael Krause wrote: At 10:28 AM 11/9/2005, Rick Frank wrote: Yes, the application is responsible for detecting lost msgs at the application level - the transport can not do this. RDS does not guarantee that a message has been delivered to the application - just that once the transport has

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-11 Thread Ranjit Pandit
On 11/11/05, Michael Krause [EMAIL PROTECTED] wrote: Please clarify the following which was in the document provided by Oracle. On page 3 of the RDS document, under the section RDP Interface, the 2nd and 3rd paragraphs are state: * RDP does not guarantee that a datagram is delivered to

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-10 Thread Michael Krause
At 02:09 PM 11/9/2005, Greg Lindahl wrote: On Wed, Nov 09, 2005 at 01:57:06PM -0800, Michael Krause wrote: What you indicate above is that RDS will implement a resync of the two sides of the association to determine what has been successfully sent. More accurate to say that it could

RE: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB

2005-11-10 Thread Caitlin Bestler
Mike Krause wrote in response to Greg Lindahl: If it is to be reasonably robust, then RDS should be required to support the resync between the two sides of the communication. This aligns with the stated objective of implementing reliability in one location in

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-10 Thread Rick Frank
Yes, this is the case. - Original Message - From: Caitlin Bestler [EMAIL PROTECTED] To: openib-general@openib.org Sent: Thursday, November 10, 2005 1:48 PM Subject: RE: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB Mike Krause wrote in response

RE: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB

2005-11-10 Thread Michael Krause
At 10:48 AM 11/10/2005, Caitlin Bestler wrote: Mike Krause wrote in response to Greg Lindahl: If it is to be reasonably robust, then RDS should be required to support the resync between the two sides of the communication. This aligns with the stated objective of implementing reliability in

RE: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB

2005-11-10 Thread Caitlin Bestler
My concern is the requirement that RDS resync the structures in the face of failureand know whether to re-transmit or will deal with duplicates. Having pre-posted bufferswill help enable the resync to be accomplished but should not be equated to pre-post equalsone can deal with

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
At 12:37 PM 11/8/2005, Hal Rosenstock wrote: On Tue, 2005-11-08 at 15:33, Ranjit Pandit wrote: Using APM is not useful because it doesn't provide failover across HCA's. Can't APM be made to work across HCAs ? No. It requires state that is only within the HCA and there are other aspects that

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
At 12:33 PM 11/8/2005, Ranjit Pandit wrote: Mike wrote: - RDS does not solve a set of failure models. For example, if a RNIC / HCA were to fail, then one cannot simply replay the operations on another RNIC / HCA without extracting state, etc. and providing some end-to-end sync of what was

RE: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Caitlin Bestler
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael KrauseSent: Tuesday, November 08, 2005 1:08 PMTo: Ranjit PanditCc: openib-general@openib.orgSubject: Re: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB At 12:33 PM

Re: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Greg Lindahl
Caitlin, Can you please use the standard quoting style? I can't tell which comments are yours. Thanks. -- greg ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Rick Frank
--- Original Message - From: Michael Krause To: Ranjit Pandit Cc: openib-general@openib.org Sent: Tuesday, November 08, 2005 4:08 PM Subject: Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB At 12:33 PM 11/8/2005, Ranjit Pandit

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Greg Lindahl
On Tue, Nov 08, 2005 at 01:08:13PM -0800, Michael Krause wrote: If an application takes any action assuming that send complete means it is delivered, then it is subject to silent data corruption. Right. That's the same as pretty much all other *transport* layers. I don't think anyone's

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
At 11:42 AM 11/9/2005, Greg Lindahl wrote: On Tue, Nov 08, 2005 at 01:08:13PM -0800, Michael Krause wrote: If an application takes any action assuming that send complete means it is delivered, then it is subject to silent data corruption. Right. That's the same as pretty much all other

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
- From: Michael Krause To: Ranjit Pandit Cc: openib-general@openib.org Sent: Tuesday, November 08, 2005 4:08 PM Subject: Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB At 12:33 PM 11/8/2005, Ranjit Pandit wrote: Mike wrote: - RDS does not solve

RE: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Caitlin Bestler
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael KrauseSent: Wednesday, November 09, 2005 12:21 PMTo: Rick Frank; Ranjit PanditCc: openib-general@openib.orgSubject: Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Greg Lindahl
On Wed, Nov 09, 2005 at 12:18:28PM -0800, Michael Krause wrote: So, things like HCA failure are not transparent and one cannot simply replay the operations since you don't know what was really seen by the other side unless the application performs the resync itself. I think you are

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
At 01:24 PM 11/9/2005, Greg Lindahl wrote: On Wed, Nov 09, 2005 at 12:18:28PM -0800, Michael Krause wrote: So, things like HCA failure are not transparent and one cannot simply replay the operations since you don't know what was really seen by the other side unless the application performs

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Greg Lindahl
On Wed, Nov 09, 2005 at 01:57:06PM -0800, Michael Krause wrote: What you indicate above is that RDS will implement a resync of the two sides of the association to determine what has been successfully sent. More accurate to say that it could implement that. I'm just kibbutzing on someone

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Ranjit Pandit
On 11/9/05, Michael Krause [EMAIL PROTECTED] wrote: I hadn't assumed anything. I'm simply trying to understand the assertions concerning availability and recovery. What you indicate above is that RDS will implement a resync of the two sides of the association to determine what has been

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Ranjit Pandit
On 11/9/05, Greg Lindahl [EMAIL PROTECTED] wrote: On Wed, Nov 09, 2005 at 01:57:06PM -0800, Michael Krause wrote: What you indicate above is that RDS will implement a resync of the two sides of the association to determine what has been successfully sent. More accurate to say that it

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Grant Grundler
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael Krause Sent: Wednesday, November 09, 2005 12:21 PM To: Rick Frank; Ranjit Pandit Cc: openib-general@openib.org Subject: Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets

RE: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Bob Woodruff
Rick wrote, I've atttached a draft proposal for RDS from Oracle which discusses some of the motivation for RDS. I assume that you have a driver that uses TCP sockets, Correct ? If so, have you compared the performance of RDS to SDP ? woody ___

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Rick Frank
04, 2005 11:35 AM Subject: RE: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB Rick wrote, I've atttached a draft proposal for RDS from Oracle which discusses some of the motivation for RDS. I assume that you have a driver that uses TCP sockets, Correct

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Sean Hefty
Rick Frank wrote: No we do not use TCP sockets - we use to many connections for this 100k+. Isn't RDS implemented on top of reliable IB/RDMA connections anyway? - Sean ___ openib-general mailing list openib-general@openib.org

RE: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Bob Woodruff
Rick wrote, I've atttached a draft proposal for RDS from Oracle which discusses some of the motivation for RDS. Couple of questions/comments on the spec. AF_INET_OFFLOAD should be renamed to something like AF_INET_RDS. Would something like SCTP provide the same type of capabilities (relaible

RE: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Caitlin Bestler
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Bob Woodruff Sent: Friday, November 04, 2005 2:15 PM To: 'Rick Frank'; Ranjit Pandit; Grant Grundler Cc: openib-general@openib.org Subject: RE: [openib-general] [ANNOUNCE] Contribute RDS

RE: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Bob Woodruff
, woody -Original Message- From: Caitlin Bestler [mailto:[EMAIL PROTECTED] Sent: Friday, November 04, 2005 2:31 PM To: Woodruff, Robert J; Rick Frank; Ranjit Pandit; Grant Grundler Cc: openib-general@openib.org Subject: RE: [openib-general] [ANNOUNCE] Contribute RDS

RE: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Rimmer, Todd
Bob wrote, Perhaps if tunneling udp packets over RC connections rather than UD connections provides better performance, as was seen in the RDS experiment, then why not just convert IPoIB to use a connected model (rather than datagrams) and then all existing IP upper level protocols would

RE: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Bob Woodruff
, November 04, 2005 2:31 PM To: Woodruff, Robert J; Rick Frank; Ranjit Pandit; Grant Grundler Cc: openib-general@openib.org Subject: RE: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED

RE: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Bob Woodruff
Rick wrote, SCTP is connection based - we have many dependencies on our connectionless datagram model. I think I get it now. I was just talking with Roy about SCTP, and he said the same thing, SCTP is a connected rather than datagram model, so SCTP does not seem to solve the problem since it

RE: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Bob Woodruff
Ranjit wrote, RDS is somewhat like SDP in that it offloads/accelerates SOCK_DGRAM instead of SOCK_STREAM. So back to the question from Roland that started this thread. When do you plan to re-work the code to use the OpenIB verbs and make it suitable for the kernel ? And do you plan to develop

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Rick Frank
- From: Bob Woodruff [EMAIL PROTECTED] To: 'Ranjit Pandit' [EMAIL PROTECTED] Cc: Rick Frank [EMAIL PROTECTED]; openib-general@openib.org Sent: Friday, November 04, 2005 6:58 PM Subject: RE: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB Ranjit wrote, RDS

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-04 Thread Roland Dreier
Rick We (Oracle) are currently investigating / working on an RDS Rick over Ethernet driver for Linux. Our current plans are to Rick produce a new verbs provider that registers with Gen 2 IB Rick verbs layer. This new driver will bind to a standard Rick ethernet nic driver and

RE: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-03 Thread Bob Woodruff
Grant wrote, 2) include some docs on it's use and why RDS is better than SDP. 3) nag people to review the ported code 4) post functional test results Looking at the code that is in the contrib branch, it looks like RDS uses connected channels, Is that correct ? If so, I do not see that it