Re: [openib-general] IB routing discussion summary

2007-02-26 Thread Michael Krause
At 11:49 AM 2/21/2007, Sean Hefty wrote: I sent a message on this topic to the IBTA several days ago, but I am still awaiting details (likely early next week). Unclear if that will occur. I just responded to some e-mail in the IBTA on the router subject as well.Given that discussion, I

Re: [openib-general] IB routing discussion summary

2007-02-20 Thread Michael Krause
At 02:05 PM 2/15/2007, Sean Hefty wrote: Is this first an IBTA problem to solve if you believe there is a problem? Based on my interpretation, I do not believe that there's an error in the architecture. It seems consistent. Additional clarification of what PathRecord fields mean when the GIDs

Re: [openib-general] Immediate data question

2007-02-15 Thread Michael Krause
At 09:37 PM 2/14/2007, Devesh Sharma wrote: On 2/14/07, Michael Krause [EMAIL PROTECTED] wrote: At 05:37 AM 2/13/2007, Devesh Sharma wrote: On 2/12/07, Devesh Sharma [EMAIL PROTECTED] wrote: On 2/10/07, Tang, Changqing [EMAIL PROTECTED] wrote: Not for the receiver, but the sender

Re: [openib-general] IB routing discussion summary

2007-02-15 Thread Michael Krause
At 11:39 AM 2/15/2007, Sean Hefty wrote: Ideas were presented around trying to construct an 'inter-subnet path record' that contained the following: - Side A GRH.SGID = active side's Port GID - Side A GRH.DGID = passive side's Port GID - Side A LRH.SLID = any active side's port LID

Re: [openib-general] IB routing discussion summary

2007-02-14 Thread Michael Krause
I do not see the need for any of this. The router protocol should be designed to work with each subnet's SM / SA to provide information on what GID prefix is on each router Port. This is used to look up the subnet local LRH fields. The only cross-subnet challenges are global based, e.g.

Re: [openib-general] Problem is routing CM REQ

2007-02-14 Thread Michael Krause
At 01:36 PM 2/14/2007, Sean Hefty wrote: Assume that the active and passive sides of a connection request are on different subnets and: Active side - LID 1 Active side router - LID 2 Passive side - LID 93 Passive side router - LID 94 What values are you suggesting are used for: Active side QP

Re: [openib-general] IB routing discussion summary

2007-02-14 Thread Michael Krause
At 02:02 PM 2/14/2007, Sean Hefty wrote: Mike, are you expecting that routers will modify CM messages as they flow between subnets? The router parses the GRH, strips the LRH, attaches a new LRH to the next hop with the contents of the LRH filled in per its internal policies. Nothing more for

Re: [openib-general] Problem is routing CM REQ

2007-02-13 Thread Michael Krause
At 03:48 PM 2/12/2007, Sean Hefty wrote: An endnode look up should be to find the address vector to the remote. A look up may return multiple vectors. The SLID would correspond to each local subnet router port that acts as a first-hop destination to the remote subnet.I don't see why the

Re: [openib-general] Immediate data question

2007-02-13 Thread Michael Krause
At 05:37 AM 2/13/2007, Devesh Sharma wrote: On 2/12/07, Devesh Sharma [EMAIL PROTECTED] wrote: On 2/10/07, Tang, Changqing [EMAIL PROTECTED] wrote: Not for the receiver, but the sender will be severely slowed down by having to wait for the RNR timeouts. RNR = Receiver Not Ready so

Re: [openib-general] Problem is routing CM REQ

2007-02-13 Thread Michael Krause
At 04:10 PM 2/12/2007, Jason Gunthorpe wrote: On Mon, Feb 12, 2007 at 03:31:15PM -0800, Michael Krause wrote: TClass is intended to communicate the end-to-end QoS desired. TClass is then mapped to a SL that is local to each subnet. A flow label is intended to much the same as in the IP

Re: [openib-general] Problem is routing CM REQ

2007-02-13 Thread Michael Krause
At 01:14 PM 2/13/2007, Sean Hefty wrote: It does not need to comprehend the remote subnet(s) LID. That is the router protocol to determine. CM also must understand the GIDs involved which the router will process to figure out its LID mapping to the next hop. The CM REQ carries the remote

Re: [openib-general] Problem is routing CM REQ

2007-02-13 Thread Michael Krause
At 02:02 PM 2/13/2007, Jason Gunthorpe wrote: On Tue, Feb 13, 2007 at 12:49:57PM -0800, Michael Krause wrote: Translated into a network with routers this means that for a RC flow to successfully work both the *forward* and *reverse* direction must traverse the same router *LID* not just

Re: [openib-general] dapl broken for iWARP

2007-02-12 Thread Michael Krause
Kanevsky email: [EMAIL PROTECTED] Network Appliance Inc. phone: 781-768-5395 1601 Trapelo Rd. - Suite 16.Fax: 781-895-1195 Waltham, MA 02451 central phone: 781-768-5300 -Original Message- From: Michael Krause [mailto:[EMAIL

Re: [openib-general] Immediate data question

2007-02-12 Thread Michael Krause
At 09:10 PM 2/11/2007, Devesh Sharma wrote: On 2/10/07, Tang, Changqing [EMAIL PROTECTED] wrote: Not for the receiver, but the sender will be severely slowed down by having to wait for the RNR timeouts. RNR = Receiver Not Ready so by definition, the data flow isn't going to progress

Re: [openib-general] Problem is routing CM REQ

2007-02-12 Thread Michael Krause
At 12:56 PM 2/12/2007, Jason Gunthorpe wrote: On Mon, Feb 12, 2007 at 09:23:06AM -0800, Sean Hefty wrote: Ah, I think I missed the key step in your scheme.. You plan to query the local SM for SGID=remote DGID=local? (ie reversed from 'normal'. I was thinking only about the SGID=local

Re: [openib-general] Problem is routing CM REQ

2007-02-12 Thread Michael Krause
At 02:47 PM 2/12/2007, Sean Hefty wrote: 1) What does the TClass and FlowLabel returned from SGID=local DGID=remote mean? Do you use it in the Node1 - Node2 direction or the Node2 - Node1 direction or both? Maybe it would help if we can agree on a set of expectations. These

Re: [openib-general] Immediate data question

2007-02-08 Thread Michael Krause
At 03:41 PM 2/7/2007, Roland Dreier wrote: Changqing What I mean is that, is there any performance penalty Changqing for receiver's overall performance if RNR happens Changqing continuously on one of the QP ? Not for the receiver, but the sender will be severely slowed down by

Re: [openib-general] dapl broken for iWARP

2007-02-08 Thread Michael Krause
At 07:43 AM 2/8/2007, Kanevsky, Arkady wrote: That is correct. I am working with Krishna on it. Expect patches soon. By the way the problem is not DAPL specific and so is a proposed solution. There are 3 aspects of the solution. One is APIs. We suggest that we do not augment these. That is a

Re: [openib-general] Problem is routing CM REQ was: Use a GRH when appropriate for unicast packets

2007-02-08 Thread Michael Krause
At 12:39 PM 2/8/2007, Hal Rosenstock wrote: On Thu, 2007-02-08 at 14:54, Sean Hefty wrote: Hum, you mean to meet the LID validation rules of 9.6.1.5? That is a huge PITA.. [IMHO, 9.6.1.5 C9-54 is a mistake, if there is a GRH then the LRH.SLID should not be validated against the QP

Re: [openib-general] BandWidth doubt

2006-11-10 Thread Michael Krause
At 02:02 AM 11/10/2006, john t wrote: Hi, I got following readings in one of my experiments: Single 64-bit xeon machine (2 dual-core 3.2 GHz Intel CPUs, linux FC4, OFED 1.0) with two Mellanox DDR (4x) HCAs (each having two ports and each connected to a PCI x8 interface) is connected to a

Re: [openib-general] ibstatus support for speed

2006-10-31 Thread Michael Krause
At 02:43 PM 10/30/2006, Hal Rosenstock wrote: On Mon, 2006-10-30 at 17:29, Michael Krause wrote: At 02:05 PM 10/30/2006, Roland Dreier wrote: Hal So rate = speed * width ? Yes, you should see the right think on DDR systems etc. Strange. Bandwidth = signaling rate * width

Re: [openib-general] ibstatus support for speed

2006-10-30 Thread Michael Krause
At 02:05 PM 10/30/2006, Roland Dreier wrote: Hal So rate = speed * width ? Yes, you should see the right think on DDR systems etc. Strange. Bandwidth = signaling rate * width. This of course is raw bandwidth prior to encoding, protocol, etc. overheads which will derate the effective

Re: [openib-general] IPoIB Question

2006-10-24 Thread Michael Krause
At 10:00 PM 10/23/2006, Greg Lindahl wrote: On Mon, Oct 23, 2006 at 07:53:06AM -0500, Hubbell, Sean C Contractor/Decibel wrote: I currently have several applications that uses a legacy IPv4 protocol and I use IPoIB to utilize my infiniband network which works great. I have completed some

Re: [openib-general] IPoIB Question

2006-10-23 Thread Michael Krause
At 10:19 AM 10/23/2006, Michael S. Tsirkin wrote: Quoting r. Sean Hubbell [EMAIL PROTECTED]: I am looking at libsdp for the TCP funcationality and wanted to know if libsdp supports UDP as well AFAIK, SDP can only emulate TCP sockets. SDP is defined to work with AF_INET applications. If

Re: [openib-general] APM support in openib stack

2006-10-13 Thread Michael Krause
At 11:24 AM 10/13/2006, Sean Hefty wrote: 3. in req_handler() we follow the same steps as we have done without APM.. i.e. create qpairs, change qp state to RTR and then send REP. however, when trying to change state to RTR usinb ib_modify_qp() I get an error (-22). two info: same code

Re: [openib-general] Dropping NETIF_F_SG since no checksum feature.

2006-10-11 Thread Michael Krause
At 02:46 AM 10/11/2006, Michael S. Tsirkin wrote: Quoting r. David Miller [EMAIL PROTECTED]: Subject: Re: Dropping NETIF_F_SG since no checksum feature. From: Michael S. Tsirkin [EMAIL PROTECTED] Date: Wed, 11 Oct 2006 11:05:04 +0200 So, it seems that if I set NETIF_F_SG but clear

Re: [openib-general] ipoib: ignores dma mapping errors on TX?

2006-10-10 Thread Michael Krause
At 10:24 AM 10/10/2006, Tom Tucker wrote: Does anyone know what might happen if a device tries to bus master bad_dma_address. Does it get a pci-abort, an NMI, a bus err interrupt, all of the above? It depends upon the platform. Some will enter a containment mode and, for example, shutdown the

Re: [openib-general] Multi-port HCA

2006-10-06 Thread Michael Krause
Off-line someone asked me to clarify my earlier e-mail. Given this discussion continues, perhaps this might help explain the performance a bit more. The Max Payload Size quoted here is what is typically implemented on x86 chipsets though other chipsets may use a larger value. From a pure

Re: [openib-general] Drop in performance on Mellanox MT25204 single port DDR HCA

2006-10-03 Thread Michael Krause
At 02:43 PM 10/2/2006, Roland Dreier wrote: Robert Yes. 1250Mbytes/sec is what we expect. You say the 128 Robert value comes from the BIOS ? If so, we need to discuss this Robert with our BIOS team to find out why they limit it to 128, Robert perhaps it is a BIOS bug. Yes, I

Re: [openib-general] [PATCH 0/10] [RFC] Support for SilverStorm Virtual Ethernet I/O controller (VEx)

2006-10-03 Thread Michael Krause
Silverstorm is executing a usage model that the IBTA used to develop the IB protocols. What is the problem with that? If it works and integrates into the stack, then this seems like an appropriate bit of functionality to support. The fact that one can use a standard ULP to communicate to

Re: [openib-general] A critique of RDMA PUT/GET in HPC

2006-08-29 Thread Michael Krause
At 08:56 AM 8/25/2006, Greg Lindahl wrote: On Fri, Aug 25, 2006 at 10:13:01AM -0500, Tom Tucker wrote: He does say this, but his analysis does not support this conclusion. His analysis revolves around MPI send/recv, not the MPI 2.0 get/put services. Nobody uses MPI put/get anyway, so leaving

Re: [openib-general] basic IB doubt

2006-08-28 Thread Michael Krause
At 10:14 AM 8/23/2006, Ralph Campbell wrote: On Wed, 2006-08-23 at 09:47 -0700, Caitlin Bestler wrote: [EMAIL PROTECTED] wrote: Quoting r. john t [EMAIL PROTECTED]: Subject: basic IB doubt Hi I have a very basic doubt. Suppose Host A is doing RDMA write (say 8 MB) to Host B.

Re: [openib-general] basic IB doubt

2006-08-28 Thread Michael Krause
At 02:58 PM 8/24/2006, Sean Hefty wrote: We're trying to create *inter-operable* hardware and software in this community. So we follow the IB standard. Atomic operations and RDD are optional, yet still part of the IB standard. An application that makes use of either of these isn't guaranteed to

Re: [openib-general] basic IB doubt

2006-08-25 Thread Michael Krause
At 11:55 AM 8/25/2006, Greg Lindahl wrote: On Fri, Aug 25, 2006 at 10:00:50AM -0400, Thomas Bachman wrote: Not that I have any stance on this issue, but is this is the text in the spec that is being debated? (page 269, section 9.5, Transaction Ordering): An application shall not depend

Re: [openib-general] basic IB doubt

2006-08-25 Thread Michael Krause
At 12:53 PM 8/25/2006, Talpey, Thomas wrote: At 03:23 PM 8/25/2006, Greg Lindahl wrote: On Fri, Aug 25, 2006 at 03:21:20PM -0400, [EMAIL PROTECTED] wrote: I presume you meant invalidate the cache, not flush it, before accessing DMA'ed data. Yes, this is what I meant. Sorry! Flush (sync

Re: [openib-general] basic IB doubt

2006-08-25 Thread Michael Krause
At 10:45 AM 8/25/2006, Tom Tucker wrote: On Fri, 2006-08-25 at 12:51 -0400, Talpey, Thomas wrote: At 12:40 PM 8/25/2006, Sean Hefty wrote: Thomas How does an adapter guarantee that no bridges or other Thomas intervening devices reorder their writes, or for that Thomas matter flush them

Re: [openib-general] basic IB doubt

2006-08-25 Thread Michael Krause
At 09:50 AM 8/25/2006, Caitlin Bestler wrote: [EMAIL PROTECTED] wrote: Thomas How does an adapter guarantee that no bridges or other Thomas intervening devices reorder their writes, or for that Thomas matter flush them to memory at all!? That's a good point. The HCA would have to do a

Re: [openib-general] Multicast traffic performace of OFED 1.0 ipoib

2006-08-02 Thread Michael Krause
Is the performance being measured on an identical topology and hardware set as before? Multicast by its very nature is sensitive to topology, hardware components used (buffer depth, latency, etc.) and workload occurring within the fabric. Loss occurs as a function of congestion or lack of

Re: [openib-general] [openfabrics-ewg] OFED 1.1 release - schedule and features

2006-07-13 Thread Michael Krause
At 03:49 PM 7/12/2006, Fabian Tillier wrote: Hi Mike, On 7/12/06, Michael Krause [EMAIL PROTECTED] wrote: At 09:48 AM 7/12/2006, Jeff Broughton wrote: Modifying the sockets API is just defining yet another RDMA API, and we have so many already I disagree. This effort has distilled the API

Re: [openib-general] OFED 1.1 release - schedule and features

2006-07-12 Thread Michael Krause
At 12:59 AM 7/12/2006, Tziporet Koren wrote: Scott Weitzenkamp (sweitzen) wrote: For SDP, I would like to see improved stability (maybe you have this in mind under beta quality), also how about AIO support? The rest of the list looks good. Yes - beta quality means improved stability. AIO

Re: [openib-general] [openfabrics-ewg] OFED 1.1 release - schedule and features

2006-07-12 Thread Michael Krause
of experience implementing Sockets and working with application developers. Mike -Jeff From: [EMAIL PROTECTED] [ mailto:[EMAIL PROTECTED]] On Behalf Of Michael Krause Sent: Wednesday, July 12, 2006 9:23 AM To: Tziporet Koren; Scott Weitzenkamp (sweitzen) Cc: OpenFabricsEWG; openib Subject: Re

Re: [openib-general] [PATCH v3 1/7] AMSO1100 Low Level Driver.

2006-06-23 Thread Michael Krause
At 10:14 AM 6/23/2006, Grant Grundler wrote: On Fri, Jun 23, 2006 at 04:04:31PM +0200, Arjan van de Ven wrote: I thought the posted write WILL eventually get to adapter memory. Not stall forever cached in a bridge. I'm wrong? I'm not sure there is a theoretical upper bound I'm not

Re: [openib-general] Mellanox HCAs: outstanding RDMAs

2006-06-15 Thread Michael Krause
As one of the authors of IB and iWARP, I can say that both Roland and Todd's responses are correct and the intent of the specifications. The number of outstanding RDMA Reads are bounded and that is communicated during session establishment. The ULP can choose to be aware of this requirement

Re: [openib-general] IB MTU tunable for uDAPL and/or Intel MPI?

2006-06-12 Thread Michael Krause
At 10:44 AM 6/9/2006, Scott Weitzenkamp (sweitzen) wrote: Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary=_=_NextPart_001_01C68BEC.6C768F57 Content-Transfer-Encoding: 7bit While we're talking about MTUs, is the IB MTU tunable in uDAPL and/or Intel MPI

Re: [openib-general] Re: Mellanox HCAs: outstanding RDMAs

2006-06-09 Thread Michael Krause
Whether iWARP or IB, there is a fixed number of RDMA Requests allowed to be outstanding at any given time. If one posts more RDMA Read requests than the fixed number, the transmit queue is stalled. This is documented in both technology specifications. It is something that all ULP should be

Re: [openib-general] QoS RFC - Resend using a friendly mailer

2006-05-30 Thread Michael Krause
High-level feedback: - An IB fabric could be used for a single ULP and still require QoS. The issue is how to differentiate flows on a given shared element within the fabric. - QoS controls must be dynamic. The document references initialization as the time when decisions are made but obviously

Re: [openib-general] re RDS missing features

2006-05-03 Thread Michael Krause
At 10:42 AM 5/1/2006, Ranjit Pandit wrote: On 5/1/06, Or Gerlitz [EMAIL PROTECTED] wrote: Can you elaborate on each of the features, specifically the following points are of interest to us: +1 so you running Oracle Loopback traffic over RDS sockets? if yes, what the issue here? the openib CMA

Re: [openib-general] re RDS missing features

2006-05-01 Thread Michael Krause
Given this is an extension to Sockets, should it not also be reviewed by the Sockets owners? What about the API itself? Any plans to make this portable to other OS / endnodes or have a spec and associated wire protocol that is reviewed perhaps in the IETF so it is applicable to more than just

Re: [openib-general] mthca FMR correctness (and memory windows)

2006-03-29 Thread Michael Krause
At 05:10 PM 3/20/2006, Fabian Tillier wrote: On 3/20/06, Talpey, Thomas [EMAIL PROTECTED] wrote: Ok, this is a longer answer. At 06:08 PM 3/20/2006, Fabian Tillier wrote: As to using FMRs to create virtually contiguous regions, the last data I saw about this related to SRP (not on OpenIB),

Re: [openib-general] Re: RFC: e2e credits

2006-03-24 Thread Michael Krause
At 03:31 AM 3/24/2006, Hal Rosenstock wrote: On Thu, 2006-03-23 at 12:34, Michael S. Tsirkin wrote: Quoting r. Hal Rosenstock [EMAIL PROTECTED]: Sean, just to wrap it up, the API at the verbs layer will look like the below, and then ULPs just put the value they want in

Re: [openib-general] mthca FMR correctness (and memory windows)

2006-03-23 Thread Michael Krause
At 04:30 PM 3/20/2006, Talpey, Thomas wrote: At 06:00 PM 3/20/2006, Sean Hefty wrote: Can you provide more details on this statement? When are you fencing the send queue when using memory windows? Infiniband 101, and VI before it. Memory windows fence later operations on the send queue until

Re: [openib-general] IPoIB and lid change

2006-02-10 Thread Michael Krause
At 09:43 AM 2/10/2006, Grant Grundler wrote: On Fri, Feb 10, 2006 at 11:05:34AM -0500, Hal Rosenstock wrote: Hi, Roland! One issue we have with IPoIB is that IPoIB may cache a remote node path for a long time. Remote LID may get changed e.g. if the SM is changed, and IPoIB might lose

Re: [dat-discussions] [openib-general] [RFC] DAT2.0immediatedataproposal

2006-02-09 Thread Michael Krause
At 03:36 PM 2/8/2006, Arlin Davis wrote: Roland Dreier wrote: Michael So, here we have a long discussion on attempting to Michael perpetuate a concept that is not universal across Michael transports and was deemed to have minimal value that most Michael wanted to see removed from the

Re: [openib-general] Re: [PATCH] [RFC] - example user moderdmaping/pongprogram using CMA

2006-02-08 Thread Michael Krause
At 11:04 AM 2/8/2006, Michael S. Tsirkin wrote: Quoting r. Steve Wise [EMAIL PROTECTED]: Subject: Re: [openib-general] Re: [PATCH] [RFC] - example user moderdmaping/pongprogram using CMA On Wed, 2006-02-08 at 19:10 +0200, Michael S. Tsirkin wrote: Quoting r. Sean Hefty [EMAIL PROTECTED]:

Re: [openib-general] Re: [PATCH] [RFC] - example user moderdmaping/pongprogram using CMA

2006-02-08 Thread Michael Krause
At 11:35 AM 2/8/2006, Steve Wise wrote: I just read this section in the 1.2 version of the spec, and I still don't understand what the issue really is? 9.7.7.2 talks about IBA doing flow control based on the RECV WQEs posted. rping always ensures that there is a RECV posted before the

RE: [dat-discussions] [openib-general] [RFC] DAT2.0immediatedataproposal

2006-02-08 Thread Michael Krause
At 09:16 PM 2/6/2006, Sean Hefty wrote: The requirement is to provide an API that supports RDMA writes with immediate data. A send that follows an RDMA write is not immediate data, and the API should not be constructed around trying to make it so. To be clear, I believe that write with immediate

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-15 Thread Michael Krause
At 12:49 PM 11/14/2005, Nitin Hande wrote: Michael Krause wrote: At 01:01 PM 11/11/2005, Nitin Hande wrote: Michael Krause wrote: At 10:28 AM 11/9/2005, Rick Frank wrote: Yes, the application is responsible for detecting lost msgs at the application level - the transport can not do this. RDS

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-11 Thread Michael Krause
At 10:28 AM 11/9/2005, Rick Frank wrote: Yes, the application is responsible for detecting lost msgs at the application level - the transport can not do this. RDS does not guarantee that a message has been delivered to the application - just that once the transport has accepted a msg it will

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-10 Thread Michael Krause
At 02:09 PM 11/9/2005, Greg Lindahl wrote: On Wed, Nov 09, 2005 at 01:57:06PM -0800, Michael Krause wrote: What you indicate above is that RDS will implement a resync of the two sides of the association to determine what has been successfully sent. More accurate to say that it could

RE: [openib-general] [ANNOUNCE] Contribute RDS ( ReliableDatagramSockets) to OpenIB

2005-11-10 Thread Michael Krause
At 10:48 AM 11/10/2005, Caitlin Bestler wrote: Mike Krause wrote in response to Greg Lindahl: If it is to be reasonably robust, then RDS should be required to support the resync between the two sides of the communication. This aligns with the stated objective of implementing reliability in

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
At 12:37 PM 11/8/2005, Hal Rosenstock wrote: On Tue, 2005-11-08 at 15:33, Ranjit Pandit wrote: Using APM is not useful because it doesn't provide failover across HCA's. Can't APM be made to work across HCAs ? No. It requires state that is only within the HCA and there are other aspects that

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
At 12:33 PM 11/8/2005, Ranjit Pandit wrote: Mike wrote: - RDS does not solve a set of failure models. For example, if a RNIC / HCA were to fail, then one cannot simply replay the operations on another RNIC / HCA without extracting state, etc. and providing some end-to-end sync of what was

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
At 11:42 AM 11/9/2005, Greg Lindahl wrote: On Tue, Nov 08, 2005 at 01:08:13PM -0800, Michael Krause wrote: If an application takes any action assuming that send complete means it is delivered, then it is subject to silent data corruption. Right. That's the same as pretty much all other

Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
- From: Michael Krause To: Ranjit Pandit Cc: openib-general@openib.org Sent: Tuesday, November 08, 2005 4:08 PM Subject: Re: [openib-general] [ANNOUNCE] Contribute RDS(ReliableDatagramSockets) to OpenIB At 12:33 PM 11/8/2005, Ranjit Pandit wrote: Mike wrote: - RDS does not solve

Re: [openib-general] [ANNOUNCE] Contribute RDS (ReliableDatagramSockets) to OpenIB

2005-11-09 Thread Michael Krause
At 01:24 PM 11/9/2005, Greg Lindahl wrote: On Wed, Nov 09, 2005 at 12:18:28PM -0800, Michael Krause wrote: So, things like HCA failure are not transparent and one cannot simply replay the operations since you don't know what was really seen by the other side unless the application performs

Re: [swg] Re: [openib-general] round 2 - proposal for socket based connectionmodel

2005-10-25 Thread Michael Krause
Just to correct one comment: A ULP written to TCP/IP can use RDMA transport without change. An example is SDP not that the ULP must use what SDP uses. Also, please keep in mind that SDP on iWARP uses the port mapper protocol to obtain the IP address and port to target for the connection

RE: [openib-general] TCP/IP connection service over IB

2005-10-24 Thread Michael Krause
At 12:50 PM 10/21/2005, Fab Tillier wrote: From: James Lentini [ mailto:[EMAIL PROTECTED]] Sent: Friday, October 21, 2005 12:38 PM On Fri, 21 Oct 2005, Sean Hefty wrote: sean version(8) | reserved(8) | src port (16) version(1) | reserved(1) | src port (2) sean src ip (16) sean

[openib-general] Re: [swg] Re: private data...

2005-10-20 Thread Michael Krause
This is really an IBTA issue to resolve and to insure that backward compatibility with existing applications is maintained. Hence, this exercise of who is broken or not is inherently flawed in that one cannot comprehend all implementations that may exist. Therefore, the spec should use either a

Re: [openib-general] I/O controllers

2005-10-19 Thread Michael Krause
At 10:41 PM 10/18/2005, Mohit Katiyar, Noida wrote: Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary=_=_NextPart_001_01C5D46F.B45AF930 Hi all, Can anyone tell me are there any specific I/O controller for the connection between the TCA and SCSI devices

Re: [openib-general] IB and FC

2005-10-17 Thread Michael Krause
These types of discussions should be taken up with IB technology / OEM vendors directly as they have nothing to do with development. Mike At 06:28 AM 10/15/2005, Mohit Katiyar, Noida wrote: Hi all, Sorry previous mail got scrapped due to HTML pictures so now with text pictures I just cant

RE: [openib-general] [RFC] IB address translation using ARP

2005-10-13 Thread Michael Krause
At 03:14 PM 10/12/2005, Caitlin Bestler wrote: -Original Message- From: [EMAIL PROTECTED] [ mailto:[EMAIL PROTECTED]] On Behalf Of Sean Hefty Sent: Wednesday, October 12, 2005 2:36 PM To: Michael Krause Cc: openib-general@openib.org Subject: Re: [openib-general] [RFC] IB address

RE: [openib-general] [RFC] IB address translation using ARP

2005-10-12 Thread Michael Krause
At 09:59 AM 10/12/2005, Caitlin Bestler wrote: From: [EMAIL PROTECTED] [ mailto:[EMAIL PROTECTED]] On Behalf Of Michael Krause Sent: Wednesday, October 12, 2005 8:24 AM To: Hal Rosenstock; Sean Hefty Cc: Openib Subject: RE: [openib-general] [RFC] IB address translation using ARP At 07

Re: [openib-general] IRQ sharing on PCIe bus

2005-10-11 Thread Michael Krause
At 02:05 PM 10/10/2005, Roland Dreier wrote: Roland BTW, for INTx emulation on PCI Express, there are no Roland physical interrupt lines -- interrupts are asserted and Roland deasserted with messages. So PCI Express interrupts are Roland unshared. Michael They are messages upstream that any

Re: [openib-general] Re: [PATCH] [CMA] RDMA CM abstraction module

2005-10-11 Thread Michael Krause
At 01:09 PM 10/10/2005, Christoph Hellwig wrote: On Mon, Oct 10, 2005 at 12:53:29PM -0700, Michael Krause wrote: standards. There are also the new standard Sockets extension API available today that might be extended sometime in the future to include explicit which is never going to get

RE: [openib-general] Re: [PATCH] [CMA] RDMA CM abstraction module

2005-10-10 Thread Michael Krause
At 12:13 PM 10/10/2005, Fab Tillier wrote: From: Sean Hefty [ mailto:[EMAIL PROTECTED]] Sent: Monday, October 10, 2005 11:16 AM Michael S. Tsirkin wrote: Maybe rdma_connection (these things encapsulate connectin state)? Or, rdma_sock or rdma_socket, since people are used to the fact

Re: [openib-general] [RFC] IB address translation using ARP

2005-10-10 Thread Michael Krause
At 10:40 AM 10/10/2005, Sean Hefty wrote: Hal Rosenstock wrote: What about the case of iWARP - IB ? Crossing IB shouldn't matter. iWarp should simply cross the IB subnet using IPoIB. You could build a gateway to make the transfer across IB more efficient, but it's not required. I don't

Re: [openib-general] IRQ sharing on PCIe bus

2005-10-10 Thread Michael Krause
At 09:22 AM 10/10/2005, Roland Dreier wrote: yipee Hi, My setup is a 3GHz Xeon (x86_64) with a 2.6.13.2 yipee kernel. A Mellanox memfree PCIe ddr HCA is connected. Why yipee do I see IRQ sharing although I'm using msi_x and PCIe? yipee Doesn't IRQ sharing only happen on older non PCIe

Re: [openib-general] [RFC] IB address translation using ARP

2005-10-10 Thread Michael Krause
At 01:59 PM 10/10/2005, Sean Hefty wrote: Michael Krause wrote: What about the case of iWARP - IB ? Crossing IB shouldn't matter. iWarp should simply cross the IB subnet using IPoIB. You could build a gateway to make the transfer across IB more efficient, but it's not required.I don't

RE: [openib-general] [RFC] IB address translation using ARP

2005-10-07 Thread Michael Krause
At 06:38 AM 9/30/2005, Caitlin Bestler wrote: -Original Message- From: [EMAIL PROTECTED] [ mailto:[EMAIL PROTECTED]] On Behalf Of Roland Dreier Sent: Thursday, September 29, 2005 6:50 PM To: Sean Hefty Cc: Openib Subject: Re: [openib-general] [RFC] IB address translation using

RE: [openib-general] [RFC] IB address translation using ARP

2005-10-07 Thread Michael Krause
At 06:24 AM 9/30/2005, Yaron Haviv wrote: -Original Message- From: Roland Dreier [ mailto:[EMAIL PROTECTED]] Sent: Thursday, September 29, 2005 9:50 PM To: Sean Hefty Cc: Yaron Haviv; Openib Subject: Re: [openib-general] [RFC] IB address translation using ARP I think the usage

RE: [openib-general] [RFC] libibverbs completion event handling

2005-09-22 Thread Michael Krause
At 03:33 PM 9/21/2005, Caitlin Bestler wrote: I'm not sure I follow what a completion channel is. My understanding is that work completions are stored in user-accessible memory (typically a ring buffer). This enables fast-path reaping of work completions. The OS has no involvement unless

Re: [openib-general][PATCH][RFC]: CMA IB implementation

2005-09-22 Thread Michael Krause
At 05:30 PM 9/21/2005, Caitlin Bestler wrote: On 9/21/05, Sean Hefty [EMAIL PROTECTED] wrote: Caitlin Bestler wrote: That's certainly an acceptably low overhead for iWARP IHVs, provided there are applications that want this control and *not* also need even more IB-specific CM control.

Re: [openib-general] Re: RDMA Generic Connection Management

2005-08-31 Thread Michael Krause
At 07:46 AM 8/31/2005, James Lentini wrote: On Tue, 30 Aug 2005, Roland Dreier wrote: I just committed this SRP fix, which should make sure we don't use a device after it's gone. And it actually simplifies the code a teeny bit... The device could still be used after it's gone. For example: -

Re: [openib-general] Re: uverbs comp events

2005-08-22 Thread Michael Krause
At 11:11 AM 8/19/2005, Roland Dreier wrote: Arlin Yes, this is certainly another option; albeit one that Arlin requires more system resources. Why not take full advantage Arlin of the FD resource we already have? It's your call, but Arlin uDAPL and other multi-thread applications could make

RE: [openib-general] [ANNOUNCE] Initial trunk checkin of ISERinitiator

2005-08-19 Thread Michael Krause
At 08:04 AM 8/19/2005, Yaron Haviv wrote: -Original Message- From: Christoph Hellwig [mailto:[EMAIL PROTECTED]] Sent: Friday, August 19, 2005 10:22 AM To: Roland Dreier Cc: Yaron Haviv; Christoph Hellwig; Grant Grundler; open- [EMAIL PROTECTED]; openib-general@openib.org Subject:

Re: [openib-general] Re:[ULP] how to choose appropriate ULPs for application

2005-07-13 Thread Michael Krause
At 06:39 AM 7/13/2005, James Lentini wrote: On Tue, 12 Jul 2005, xg wang wrote: Frankly speaking, I can not distinguish the function of SDP and DAPL. Since Lustre is a file system, it runs on kernel. So I think maybe kDAPL is better. SDP stands for the Sockets Direct Protocol. The protocol is

Re: [openib-general] Re:[ULP] how to choose appropriate ULPs for application

2005-07-13 Thread Michael Krause
At 11:18 AM 7/13/2005, James Lentini wrote: On Wed, 13 Jul 2005, Michael Krause wrote: At 06:39 AM 7/13/2005, James Lentini wrote: kDAPL was designed specifically for RDMA networks with lots of features that allow you to control how the network is used. This is good if you are writing new code

Re: [openib-general] Re: IP addressing on InfiniBand networks (Caitlin Bestler)

2005-07-11 Thread Michael Krause
At 06:37 AM 7/11/2005, James Lentini wrote: On Tue, 5 Jul 2005, Michael Krause wrote: The intention was to allow one to manage the fabric by having mapping functions from traditional IP management applications to IB GID to minimize the amount of work to enable IB within a solution. I

Re: [openib-general] vendor_id, vendor_part_id, hw_ver

2005-07-11 Thread Michael Krause
At 04:20 PM 7/8/2005, Kevin Reilly wrote: Mike, Ideally your right the ULP shouldn't care what HCA it's running on. There are some practical reasons why an ULP might want to know the vendor and part number it was using like for debug or taking advantage a perform nuance of a particular HCA.

Re: [openib-general] vendor_id, vendor_part_id, hw_ver

2005-07-08 Thread Michael Krause
At 02:00 PM 7/8/2005, Roland Dreier wrote: Kevin Is openIB going to do anything to enumerate the Kevin vendor_id,vendor_part_id and hw_ver in a common header Kevin fille or is it the responsiblity of ULP running ontop of Kevin the lib to understand these values? I don't have anything planned

RE: [openib-general] [iser]about the target

2005-07-06 Thread Michael Krause
/ plug-ins / etc. I do not want to get into a vision / marketing debate - was just explaining why we created iSER instead of just enhancing SRP. Mike Todd R. -Original Message- From: Michael Krause [ mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 05, 2005 12:28 PM To: Ian Jiang; openib

Re: [openib-general] Re: IP addressing on InfiniBand networks (Caitlin Bestler)

2005-07-05 Thread Michael Krause
At 10:49 AM 6/30/2005, Roland Dreier wrote: Michael Being the person who led the addressing definition for Michael IB, I can state quite clearly that GID are NOT IPv6 Michael addresses. They were intentionally defined to have a Michael similar look-n-feel since they were derived in large

Re: [openib-general] [iser]about the target

2005-07-05 Thread Michael Krause
At 06:07 PM 7/4/2005, Ian Jiang wrote: Hi! I am new to the iSER. On https://openib.org/tiki/tiki-index.php?page=iSER, it is said that iSER currently contains initiator only (no target). Will the target come out later? How did they test the iSER initiator without a iSER target? Could you give

Re: [openib-general] Re: IP addressing on InfiniBand networks (Caitlin Bestler)

2005-06-30 Thread Michael Krause
At 10:39 PM 6/29/2005, Bill Strahm wrote: -- Message: 2 Date: Wed, 29 Jun 2005 09:00:37 -0700 From: Roland Dreier [EMAIL PROTECTED] Subject: Re: [openib-general] IP addressing on InfiniBand networks To: Caitlin Bestler [EMAIL PROTECTED] Cc: Lentini, James [EMAIL

Re: [openib-general] mapping between IP address and device name

2005-06-28 Thread Michael Krause
At 10:30 AM 6/24/2005, Roland Dreier wrote: Thomas As I said - I am not attached to ATS. I would welcome an Thomas alternative. Sure, understood. I'm suggesting a slight tweak to the IB wire protocol. I don't think there's a difference in the security provided, and carrying the peer address in

Re: [openib-general] How about ib_send_page() ?

2005-06-07 Thread Michael Krause
At 12:13 PM 6/3/2005, Sean Hefty wrote: Fab Tillier wrote: Ok, so this question is from a noob, but here goes anyway. Why can't IPoIB advertise a larger MTU than the UD MTU, and then just fragment large IP packets up if they need to go over the IB UD transport? Is there any reason this couldn't

RE: [openib-general] How about ib_send_page() ?

2005-06-07 Thread Michael Krause
At 09:28 AM 6/7/2005, Fab Tillier wrote: From: Roland Dreier [ mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 07, 2005 8:38 AM Michael Why not just use the IETF draft for RC / UC based IP over Michael IB and not worry about creating something new? I think we've come full circle. The

Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux

2005-05-31 Thread Michael Krause
At 06:47 AM 5/28/2005, Christoph Hellwig wrote: On Sat, May 28, 2005 at 05:17:54AM -0700, Sukanta ganguly wrote: That's a pretty bold statement. Linux grew up to be popular via mass acceptance. Seems like that charter has changed and a few have control over Linux and its future. The My way

Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMA APIs and ULPs for Linux

2005-05-27 Thread Michael Krause
At 06:40 AM 5/27/2005, Sukanta ganguly wrote: Venkata, How will that work? If the RNIC offloads RDMA and TCP completely from the Operating System and does not share any state information then the application running on the host will never be in the position to utilize the socket interface to

Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMA APIs and ULPs for Linux

2005-05-27 Thread Michael Krause
At 09:29 AM 5/27/2005, Grant Grundler wrote: On Fri, May 27, 2005 at 07:24:44AM -0700, Michael Krause wrote: ... Again, Sockets is an application API and not how one communicates to a TOE or RDMA component. Mike, What address family is used to open a socket over iWARP? AF_INET? Or something

  1   2   >