Re: [openib-general] uDAPL: RDMA Write example

2007-02-19 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Hello, > > I'm trying to find a small sample program, that uses RDMA > Write instead of Send/Recv. In the sources there is no single > uDAPL example program and on the net neither. > Could someone please help me to find something useful? > > Thanks! > Christian > With

Re: [openib-general] new IB CM reject reason

2007-01-31 Thread Caitlin Bestler
Sean Hefty wrote: >> Is there a reason to distinquish between a connection that is being >> rejected because the listener crashed and a connection that is being >> rejected because the listener does not exist? > > This only covers the case for the REQ received state, and > could work for that stat

Re: [openib-general] new IB CM reject reason

2007-01-31 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > We've hit into an issue with the IB CM reject reason codes. > When a remote application crashes during connection > establishment, the connection will be rejected by the kernel > CM. Unfortunately, there's not a decent reject reason that > maps to this event. Currently,

Re: [openib-general] [Bug 325] New: RDMA_CM and address translation broken on sles9sp3

2007-01-26 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > https://bugs.openfabrics.org/show_bug.cgi?id=325 > >Summary: RDMA_CM and address translation broken on sles9sp3 >Product: OpenFabrics Linux >Version: 1.2 > Platform: X86-64 > OS/Version: SLES 9 > Status: N

Re: [openib-general] Minutes for January 15, 2007 teleconference about OFED 1.2 development progress toward code freeze

2007-01-22 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > > > when RDMA is used, a message is transferred > from card A (in node >> > A) to card B (in node B), card B delivers the message to to user >> buffer, > and sends ACK to card A, but ACK is lost due to switch >> fail. So process > on node A get fail for this

Re: [openib-general] Minutes for January 15, 2007 teleconference about OFED 1.2 development progress toward code freeze

2007-01-22 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Roland: > when RDMA is used, a message is transferred from card A (in node > A) to card B (in node B), card B delivers the message to to > user buffer, and sends ACK to card A, but ACK is lost due to > switch fail. So process on node A get fail for this transfer, >

Re: [openib-general] [PATCH RFC] return qp pointer as part of ib_wc

2007-01-09 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Roland Dreier wrote: >> This change makes sense to me. Does anyone object to queueing this >> for 2.6.21? > > Indeed, it makes much sense, do you any idea what would it > take to expose this capability also by libibverbs? > > Or. > Translating QP ID to a kernel point

Re: [openib-general] [PATCH 1/10] cxgb3 - main header files

2007-01-09 Thread Caitlin Bestler
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > Michael S. Tsirkin > Sent: Tuesday, January 09, 2007 5:57 AM > To: Steve Wise > Cc: netdev@vger.kernel.org; Roland Dreier; Divy Le Ray; > linux-kernel@vger.kernel.org; openib-general > Subject: R

Re: [openib-general] [PATCH] rdma_cm iWARP connection setup timeouts reported as rejects.

2007-01-08 Thread Caitlin Bestler
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Steve Wise > Sent: Monday, January 08, 2007 6:55 AM > To: Mirko Benz > Cc: openib-general@openib.org > Subject: Re: [openib-general] [PATCH] rdma_cm iWARP > connection setup timeouts reported as rej

Re: [openib-general] [RFC] [PATCH 6/7] rdma_cm 2.6.20: add support for RDMA_PS_UDP

2006-10-11 Thread Caitlin Bestler
On 10/10/06, Sean Hefty <[EMAIL PROTECTED]> wrote: > Add missing support for RDMA_PS_UDP. This allows the use of UD QPs > through the rdma_cm, which provides address translation services > over IB, even if not all RDMA transports support UD. > To the best of my knowledge every single iWARP RNIC fu

Re: [openib-general] Port reuse issue for rdma_cm/iwarp

2006-09-26 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Hi, > We are facing a problem while running back-to-back > applications using the same port number for rdma_cm over > iwarp (Ammasso). The port seems to be busy for about 60 > seconds after each disconnect. > > The first execution finishes without any problems or errors

Re: [openib-general] RDMA question

2006-09-12 Thread Caitlin Bestler
On 9/12/06, Makia Minich <[EMAIL PROTECTED]> wrote: > I'm looking for some information on whether or not you can set a service > level for RDMA packets (as a way to start working on a QoS design). > Transport independent QoS is not truly feasible. You'll have to apply QoS to the underlying transpo

Re: [openib-general] single rkey

2006-09-01 Thread Caitlin Bestler
On 8/31/06, yipee <[EMAIL PROTECTED]> wrote: > Hi, > > Is it possible for several memory registrations (using ibv_reg_mr) to have a > single rkey? > Can I add memory registrations to a previous rkey? > > You need to create the Memory Region as large as you think it will need to be. But there are t

Re: [openib-general] basic IB doubt

2006-08-28 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > At 03:39 AM 8/26/2006, Gleb Natapov wrote: >> On Fri, Aug 25, 2006 at 03:53:12PM -0400, Talpey, Thomas wrote: >>> Flush (sync for_device) before posting. >>> Invalidate (sync for_cpu) before processing. >>> >> So, before touching the data that was RDMAed into the buffer

Re: [openib-general] basic IB doubt

2006-08-25 Thread Caitlin Bestler
Woodruff, Robert J wrote: > Catlin wrote, >> Another point, even if a vendor were to implement the firmware you >> suggest, how does the Data Source know that it is safe to use just >> RDMA Writes? The enabling firmware is in the Data Sink. > > Huh, don't understand the question. > >> Application

Re: [openib-general] basic IB doubt

2006-08-25 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: >>Thomas> How does an adapter guarantee that no bridges or other >>Thomas> intervening devices reorder their writes, or for that >>Thomas> matter flush them to memory at all!? >> >> That's a good point. The HCA would have to do a read to flush the >> posted wr

Re: [openib-general] basic IB doubt

2006-08-25 Thread Caitlin Bestler
Woodruff, Robert J wrote: > Catlin wrote, > >> For iWARP there are network performance reasons why in-order memory >> writes will never be guaranteed. > > For iWarp, or any other RDMA over Ethernet protocol, the > behavior is not to guarantee all packets are written > in-order, just that the last

Re: [openib-general] basic IB doubt

2006-08-24 Thread Caitlin Bestler
Woodruff, Robert J wrote: > Catlin wrote, > >> For iWARP there are network performance reasons why in-order memory >> writes will never be guaranteed. > > For iWarp, or any other RDMA over Ethernet protocol, the > behavior is not to guarantee all packets are written > in-order, just that the last

Re: [openib-general] basic IB doubt

2006-08-24 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On 8/24/06, Woodruff, Robert J <[EMAIL PROTECTED]> wrote: >> If the feature gives them a huge advantage in performance (and it >> does) and all of the hardware vendors that they deal with already >> implement it, then yes, they will force, by defacto standard that all >>

Re: [openib-general] basic IB doubt

2006-08-23 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Quoting r. john t <[EMAIL PROTECTED]>: >> Subject: basic IB doubt >> >> Hi >> >> I have a very basic doubt. Suppose Host A is doing RDMA write (say 8 >> MB) to Host B. When data is copied into Host B's local > buffer, is it guaranteed that data will be copied starting >

Re: [openib-general] return error when rdma_listen fails

2006-08-17 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Tom Tucker wrote: >> I think this makes sense for IB, however, for TCP based transports, >> we should share the port space with TCP. > > My view is that the iWarp transport needs to provide the > mapping from an RDMA_PS_TCP to the actual TCP port space, > RDMA_PS_UDP to

Re: [openib-general] RDMA_READ SGE

2006-08-10 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Quoting r. Tom Tucker <[EMAIL PROTECTED]>: >> Subject: RDMA_READ SGE >> >> Roland: >> >> iWARP RNIC's have a different SGE limit for RDMA_READ response then >> they do for other SGE. To support iWARP, we need to add a >> max_read_sge attribute to the ib_device structure

Re: [openib-general] [PATCH 0/6] Tranport Neutral Verbs Proposal.

2006-07-31 Thread Caitlin Bestler
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Greg Lindahl > Sent: Monday, July 31, 2006 10:38 AM > To: James Lentini > Cc: openib-general > Subject: Re: [openib-general] [PATCH 0/6] Tranport Neutral > Verbs Proposal. > > On Mon, Jul 31, 2006

Re: [openib-general] posting send requests in RTR

2006-07-31 Thread Caitlin Bestler
> -Original Message- > From: Michael S. Tsirkin [mailto:[EMAIL PROTECTED] > Sent: Saturday, July 29, 2006 2:55 PM > To: Rimmer, Todd > Cc: Caitlin Bestler; Sean Hefty; Or Gerlitz; Roland Dreier; > openib-general@openib.org > Subject: Re: posting send requests

Re: [openib-general] [PATCH 0/6] Tranport Neutral Verbs Proposal.

2006-07-31 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On Mon, Jul 31, 2006 at 09:52:48AM -0500, Steve Wise wrote: > >> rdma_* is more descriptive than something like ofv_* or of_* in my >> opinion. I would think the prefix should help describe the >> functionality being implemented: Transport Neutral RDMA. > > Some funct

Re: [openib-general] posting send requests in RTR

2006-07-28 Thread Caitlin Bestler
Rimmer, Todd wrote: >> From: Caitlin Bestler [mailto:[EMAIL PROTECTED] >> >> That assumes that there is any valid reason for an application to >> post send requests before the connection is established. While there >> is clearly a need to post receive work reques

Re: [openib-general] posting send requests in RTR

2006-07-27 Thread Caitlin Bestler
Sean Hefty wrote: > >> Alternately, it would be reasonable to simply document that a receive >> completion *implied* a connection established event, and therefore >> the application could post to the send queue after it reaped a >> receive completion (or got a connection established event). > >

Re: [openib-general] posting send requests in RTR

2006-07-27 Thread Caitlin Bestler
Sean Hefty wrote: >> That assumes that there is any valid reason for an application to >> post send requests before the connection is established. While there >> is clearly a need to post receive work requests before the >> connection is established I cannot think of any reason why an >> applicatio

Re: [openib-general] posting send requests in RTR

2006-07-27 Thread Caitlin Bestler
Rimmer, Todd wrote: >> From: Caitlin Bestler [mailto:[EMAIL PROTECTED] >> >>> >>> Actually since WQEs are in memory, while in RTR the verbs driver >>> could build the WQEs, post them to the QP, just not issue the >>> doorbells. >> >&g

Re: [openib-general] posting send requests in RTR

2006-07-27 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: >> From: Sean Hefty [mailto:[EMAIL PROTECTED] >> >>> Not necessarily. CMA consumer at this point is a rare beast, but >>> since the issue is not specific to CMA, let's look at some IB >>> protocols: IPoIB connected mode can simply drop a packet. So can >>> SRP. SDP (pote

Re: [openib-general] posting send requests in RTR ( was Re: [PATCH 0/4] Dispatch communication related events to the IB CM)

2006-07-27 Thread Caitlin Bestler
Michael Tsirkin wrote: > > Not necessarily. CMA consumer at this point is a rare beast, > but since the issue is not specific to CMA, let's look at > some IB protocols: IPoIB connected mode can simply drop a > packet. So can SRP. SDP (potential CMA consumer!) simply > never needs to send any ap

Re: [openib-general] max_send_sge < max_sge

2006-06-28 Thread Caitlin Bestler
> > Yep. We could have an option to have the stack scale the > requested values down to some legal set instead of failing an > allocation. But we couldn't come up with a clean way to tell > the stack e.g. what should it round down: the SGE or WR > value. Do you think selecting something arbit

Re: [openib-general] Interface for getting RNIC's IP address

2006-06-26 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On Mon, 2006-06-26 at 14:22 -0400, Sundeep Narravula wrote: >> Is there any s/w interface to obtain the local RNIC's IP address? >> >> The current rdma cm examples, rping and cmatose, require the user to >> enter the ip address as a command line parameter. I am currentl

Re: [openib-general] [PATCH v2 1/7] AMSO1100 Low Level Driver.

2006-06-15 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On Thu, 2006-06-15 at 08:41 -0500, Steve Wise wrote: >> On Wed, 2006-06-14 at 20:35 -0500, Bob Sharp wrote: >> +void c2_ae_event(struct c2_dev *c2dev, u32 mq_index) { + >> >> >> + case C2_RES_IND_EP:{ + + struct c2wr_ae_connection_re

Re: [openib-general] [PATCH v2 1/2] iWARP Connection Manager.

2006-06-14 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On Tue, 2006-06-13 at 16:46 -0500, Steve Wise wrote: >> On Tue, 2006-06-13 at 14:36 -0700, Sean Hefty wrote: > Er...no. It will lose this event. Depending on the event...the > carnage varies. We'll take a look at this. > This behavior is consistent

RE: [openib-general] Re: [PATCH 1/2] iWARP Connection Manager.

2006-06-01 Thread Caitlin Bestler
>> >> There's a difference between trying to handle the user calling >> disconnect/destroy at the same time a call to accept/connect is >> active, versus the user calling disconnect/destroy after >> accept/connect have returned. In the latter case, I think you're >> fine. In the first case, thi

RE: [openib-general] Re: QoS RFC

2006-05-31 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Roland > > This thread also indirectly brings up the issue of > OpenFabrics, RNIC, and QoS. > > The RNIC devices don't have to be, but are typically unified > wire devices, i.e. have simultaneous support for regular > Ethernet NIC functions such as LSO/TSO and checksum

RE: [openib-general] RFC: CMA backlog

2006-05-30 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Hello, Sean! > I am looking at implementing the listen backlog parameter correctly. > Here's what this does in TCP: TCP counts the number of > connect requests at the specific local socket that were not > yet accepted by accept(). > Once this number exceeds the backlog sp

RE: [openib-general] Re: [PATCH 21 of 53] ipath - use phys_to_virt instead of bus_to_virt

2006-05-15 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Grant> Aren't remote addresses handled differently than local > Grant> ones? ULP has to map local addresses. We can't map remote > Grant> ones (remote host maps it). The ULP must know the > Grant> difference and can tell the lower level driver which is

RE: [openib-general] cma private data length

2006-05-12 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On Fri, 2006-05-12 at 12:03, Steve Wise wrote: >> Sean/IB experts: >> >> I'm running a version of rdma_bw from src/userspace/perftest that I >> ported to utilize the RDMA CMA library for connection setup (stay >> tuned for a patch to offer this to the trunk). The CMA ve

[openib-general] RE: rdma_cm.h: comment nits.

2006-05-11 Thread Caitlin Bestler
Tom Tucker wrote: > On Thu, 2006-05-11 at 10:56 +0300, Michael S. Tsirkin wrote: >> Quoting r. Tom Tucker <[EMAIL PROTECTED]>: >>> Subject: RE: rdma_cm.h: comment nits. >>> >>> On Wed, 2006-05-10 at 14:20 -0700, Caitlin Bestler wrote: >>>> To

[openib-general] RE: rdma_cm.h: comment nits.

2006-05-11 Thread Caitlin Bestler
Michael S. Tsirkin wrote: > Quoting r. Tom Tucker <[EMAIL PROTECTED]>: >> So... all that said, I could in fact support rdma_reject on an active >> side connection. But this would effectively reduce to a QP --> ERROR >> and I doubt this matches the semantics you're looking for. > > Why not? Sounds

RE: [openib-general] rdma_cm.h: comment nits.

2006-05-10 Thread Caitlin Bestler
Tom Tucker wrote: > > So... all that said, I could in fact support rdma_reject on > an active side connection. But this would effectively reduce > to a QP --> ERROR and I doubt this matches the semantics > you're looking for. > > And you could send an RST. There's just no way to send any user

RE: [openib-general] rdma_cm.h: comment nits.

2006-05-10 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Tom Tucker wrote: >>> Its OK to call rdma_reject on active side as well, isn't it? >> >> You'll get -EINVAL on iWARP if you do this > > For IB, rdma_reject can be called on the active side if the > user is managing their own QP states, or is SDP. How does iWarp > s

RE: [openib-general] Re: [PATCH 5 of 13] ipath - use proper address translation routine

2006-05-02 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > proper address translation routine > > On Tue, May 02, 2006 at 07:24:18AM -0700, Roland Dreier wrote: >> Christoph> Or stop doing the dma mapping in the IB upper level >> Christoph> drivers. I told you that we'll get broken hardware >> Christoph> that doesn'

RE: [openib-general] [PATCH] [RFC] dapltest change for iwarp

2006-04-28 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Steve Wise wrote: >> The Chelsio RNIC has this issue. If the server sends the first FPDU >> _before_ the client driver has moved the connection/qp into RDMA >> mode, then the data is placed as streaming data and the connection >> must be terminated (dapltest 6 exposes th

RE: [openib-general] [PATCH] [RFC] dapltest change for iwarp

2006-04-28 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: >> I'm confused. Is this an iWARP requirement or a Chelsio requirement? >> It sounds like iWARP supports data transfers being initiated by the >> server but the Chelsio implementation does not. >> > > This is an iWARP requirement. (I will _not_ argue that its a > reasona

RE: [openib-general] [PATCH] rdma_cm: let SDP control the SDP version in the hello header

2006-04-27 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: >> Go ahead. BTW, I'm reasonably sure CMA does not check MajV at least >> in incoming HelloAck. I think since you must check it in incoming >> Hello in CMA, its best to check it in incoming HelloAck in CMA as >> well. >> >> Another validation check needed in CMA: >> >> C

RE: [openib-general] [RFC] [PATCH 1/3] RDMA CM: addrdma_get/set_optioncalls to get/set path records

2006-04-26 Thread Caitlin Bestler
Sean Hefty wrote: >> What about for IB HCAs? Are there a large number of options that have >> not yet been exposed but which are device independent and *might* be >> desirable to control? If not, then why introduce a "catchall" >> interface as opposed to specific interfaces that have to justified >

RE: [openib-general] [RFC] [PATCH 1/3] RDMA CM: add rdma_get/set_optioncalls to get/set path records

2006-04-26 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Quoting r. Sean Hefty <[EMAIL PROTECTED]>: >>> How about doing copy_from_user in ucma, and implementing >>> rdma_set_path/rdma_get_path in cma? >> >> I don't think that we want to start adding a new set of APIs for >> every option that may eventually need to be supported

RE: [openib-general] Re: RFC: cma: need rdma_unbind

2006-04-24 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Michael S. Tsirkin wrote: >> No. A socket is a 5 tuple (proto, local addr, local port, remote >> addr, remote port). unbind just says that you can reuse local >> addresses, so e.g. a new connection request will connect to a new >> socket. > > I understand. But if both

RE: [openib-general] Re: [uDAPL] dat.conf generator

2006-04-24 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On Sun, 23 Apr 2006, Dotan Barak wrote: > >> On Wednesday 19 April 2006 22:17, Arlin Davis wrote: > OpenIB-cma u1.2 nonthreadsafe default > /usr/lib/libdaplcma.so mv_dapl.1.2 "mthca0 1" "" > OpenIB-cma0-1 u1.2 nonthreadsafe default > /usr/lib/libdaplcma.s

RE: [openib-general] RFC userspace / MPI multicast support

2006-04-20 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > I'd like to get some feedback regarding the following > approach to supporting multicast groups in userspace, and in > particular for MPI. Based on side conversations, I need to > know if this approach would meet the needs of MPI developers. > > To join / leave a multic

RE: [openib-general] How do we prevent starvation say between TCP over IPOIB / and SRP traffic ?

2006-04-19 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Hi Rick, > > On 4/19/06, Richard Frank <[EMAIL PROTECTED]> wrote: >> Some application level protocols - require higher QoS levels than >> others - for various communication and I/O operations. >> >> For example, cluster inter-node health msgs have fixed latency >> requi

RE: [openib-general] How do we prevent starvation say between TCP over IPOIB / and SRP traffic ?

2006-04-19 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Some application level protocols - require higher QoS levels than > others - for various communication and I/O operations. > > For example, cluster inter-node health msgs have fixed > latency requirements that if exceeded may result in > unexpected node removals from the

RE: [openib-general] uDAPL not supported on ppc64?

2006-04-12 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > I get this trying to compile uDAPL using install.sh with IBED > 1.0 rc3 on RHEL4 U2 2.6.9-22 ppc64: > > WARNING: Dapl is not supported on PPC64 arcitecture > WARNING: Dapl is not supported on PPC64 arcitecture > > Scott There are include files that map DAT-defined type

RE: [openib-general] Re: CM patch for 2.6.17 merge

2006-04-04 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Quoting r. Roland Dreier <[EMAIL PROTECTED]>: >> Subject: Re: CM patch for 2.6.17 merge >> >> Michael> The second is a security fix, its a must. >> >> Not sure I understand this. What's the exploit? > > Connecting from userspace to an SDP socket. People expect > s

RE: [openib-general] RDMA CM and loopback addresses

2006-03-30 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Jason Gunthorpe wrote: >> Well, this is what happens in the normal IP stack. To match normal IP >> semantics the source IP alone should never be mapped to a device, the >> full tuple should be passed through the route table to get to a >> device. This is what I was saying

RE: [openib-general] RDMA CM and loopback addresses

2006-03-30 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > I created a test program to determine how sockets handles > local system connections. The server side was configured to > listen in 1 of 3 ways: on any address, a local address, or > the loopback address. The client side connected in > 1 of 4 ways (bound address to serv

RE: [openib-general] Re: [PATCH] CMA: allow/require bind before connect

2006-03-29 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Michael S. Tsirkin wrote: >> I'm fine with not requiring bind before connect. >> However, CMA must *allow* bind before connect and it does not >> currently. > > It does permit this, but requires using an IP address that > matches with a local ipoib device. > Or a local

RE: [openib-general] Re: Problem configuring ipath_ether

2006-03-28 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On Mon, Mar 27, 2006 at 08:18:28AM -0800, Shirley Ma wrote: > >> Is it a good idea to send such a big messages size using UD mode? > > The usual reason for not doing this with UDP is that it turns > a small amount of packet loss into lots more lossage. This is > a real

RE: [openib-general] Re: [PATCH] CMA: allow/require bind beforeconnect

2006-03-28 Thread Caitlin Bestler
Tom Tucker wrote: > On Tue, 2006-03-28 at 09:50 -0800, Sean Hefty wrote: The app _usually_ doesn't care. See NFS discussion for a client app that does care. Also, providers DO care. Because of this issue, the chelsio iwarp provider right now has to allocate its own ephemeral

RE: [openib-general] Re: [PATCH] CMA: allow/require bind beforeconnect

2006-03-28 Thread Caitlin Bestler
Sean Hefty wrote: >>> The app _usually_ doesn't care. See NFS discussion for a client app >>> that does care. Also, providers DO care. Because of this issue, >>> the chelsio iwarp provider right now has to allocate its own >>> ephemeral ports at connect time. This logic should be moved into >>>

RE: [openib-general] Re: [PATCH] CMA: allow/require bind beforeconnect

2006-03-28 Thread Caitlin Bestler
Steve Wise wrote > > The app _usually_ doesn't care. See NFS discussion for a > client app that does care. Also, providers DO care. Because > of this issue, the chelsio iwarp provider right now has to > allocate its own ephemeral ports at connect time. This logic > should be moved into the IWC

RE: [openib-general] Re: [PATCH] CMA: allow/require bind beforeconnect

2006-03-27 Thread Caitlin Bestler
Roland Dreier wrote: > OK, fair enough. I was really replying to the first sentence of: > > Caitlin> From the perspective of any given host, IP addresses are > Caitlin> unique across all interface devices. A given connection > Caitlin> can therefore be identified by just the 4-tupl

RE: [openib-general] Re: [PATCH] CMA: allow/require bind beforeconnect

2006-03-27 Thread Caitlin Bestler
Sean Hefty wrote: > Roland Dreier wrote: >> OK, fair enough. I was really replying to the first sentence of: >> >> Caitlin> From the perspective of any given host, IP addresses >> are Caitlin> unique across all interface devices. A given >> connection Caitlin> can therefore be iden

RE: [openib-general] Re: [PATCH] CMA: allow/require bind beforeconnect

2006-03-27 Thread Caitlin Bestler
Roland Dreier wrote: > Caitlin> From the perspective of any given host, IP addresses are > Caitlin> unique across all interface devices. A given connection > Caitlin> can therefore be identified by just the 4-tuple, with no > Caitlin> need to explicitly state "via this device". > >

RE: [openib-general] Re: [PATCH] CMA: allow/require bind beforeconnect

2006-03-27 Thread Caitlin Bestler
Sean Hefty wrote: >>> It's correct that the CMA currently does not do this. I guess I'm >>> still unsure of why it needs to when running over IB. The CMA >>> should work fine if it assigned every active connection the same >>> port number. As long as the CMA manages its own port spaces, does >>>

RE: [openib-general] Re: [PATCH] CMA: allow/require bind before connect

2006-03-27 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > > It's correct that the CMA currently does not do this. I > guess I'm still unsure of why it needs to when running over > IB. The CMA should work fine if it assigned every active > connection the same port number. As long as the CMA manages > its own port spaces, doe

RE: [openib-general] Re: Re: IBM utilizing and testing openIB release 1 RCx

2006-03-27 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > ...unfortunately there's no easy way to detect drops on the receive > side. > > Resizing queues on the fly also means you have to modify your > CQ size as well. > Resizing also means that it could be CPU intense during the > resize operation, so I wouldn't resize too of

RE: [openib-general] Re: [PATCH 3/3] iWARP CM -iWARPConnectionManager

2006-03-22 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: >> The user, being the IWCM, will always see a CONNECT_REPLY to a >> connect downcall. > > I understand that a CONNECT_REPLY event is generated in > response to calling connect(). But can any events (e.g. > CLOSE) follow that without the user taking any other action? > >

RE: [openib-general] Re: [PATCH 3/3] iWARP CM - iWARPConnectionManager

2006-03-22 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: >>> For example, as soon as the user calls connect(), can they receive a >>> CLOSE event, even before the connect() call returns? >> >> No. connect results in a CONNECT_REPLY event always. Not a CLOSE >> event. > > What if the remote side sends the reply, then decides to

RE: [openib-general] Re: mthca FMR correctness (and memory windows)

2006-03-21 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On 3/21/06, Michael S. Tsirkin <[EMAIL PROTECTED]> wrote: >> Quoting Fabian Tillier <[EMAIL PROTECTED]>: >>> I'd still be interested in seeing regular registration calls >>> improved, as it's clear that an application that is sensitive about >>> its security must either r

RE: [openib-general] mthca FMR correctness (and memory windows)

2006-03-21 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On 3/21/06, Caitlin Bestler <[EMAIL PROTECTED]> wrote: >> [EMAIL PROTECTED] wrote: >>> > ...snip... >>> >>> I just want the fastest possible map and unmap. I guess that means >>> I want fast MTT's. >>&

RE: [openib-general] Re: mthca FMR correctness (and memory windows)

2006-03-21 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > At 03:41 AM 3/21/2006, Michael S. Tsirkin wrote: >> Which applications do register/unregister for each I/O? > > Storage! > >> Do you have a specific benchmark in mind? > > Storage! > > :-) > > Tom. > Any application where the ultimate consumer is not aware of the n

RE: [openib-general] mthca FMR correctness (and memory windows)

2006-03-21 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > At 08:24 PM 3/20/2006, Doug O'Neil wrote: >>> From iWarp RDMA Verbs Section 5.2 >> ... >> Tom, I read the above as an STag that represents a MR can be used by >> any QP with the same PD ID. STags that represent a MW must be used on >> the same QP that created them. > > T

RE: [openib-general] mthca FMR correctness (and memory windows)

2006-03-21 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Thomas> Yes, I know about binding on a separate queue. That > Thomas> doesn't work, because windows are semantically not > Thomas> fungible (for security reasons). > > Can you elaborate on the issue of fungibility? If one entity > has two QPs, one of which i

RE: [openib-general] mthca FMR correctness (and memory windows)

2006-03-21 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Thomas> If I can snoop or guess rkeys (not a huge challenge with > Thomas> 32 bits), and if I can use them on an arbitrary queuepair, > Thomas> then I can handily peek and poke at memory that does not > Thomas> belong to me. > > Thomas> For this reason,

RE: [openib-general] mthca FMR correctness (and memory windows)

2006-03-21 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Ok, this is a longer answer. > > At 06:08 PM 3/20/2006, Fabian Tillier wrote: >> You pre-alloc the MPT entry, but not the MTT entries. You then >> populate the MTT by doing posted writes to the HCA memory (or host >> memory for memfree HCAs). ... >> I don't know if allo

RE: [openib-general] mthca FMR correctness (and memory windows)

2006-03-21 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > At 06:00 PM 3/20/2006, Sean Hefty wrote: >> Can you provide more details on this statement? When are you fencing >> the send queue when using memory windows? > > Infiniband 101, and VI before it. Memory windows fence later > operations on the send queue until the bind c

RE: [openib-general] Re: [PATCH 3/3] iWARP CM

2006-03-16 Thread Caitlin Bestler
Sean Hefty wrote: > Tom Tucker wrote: > >> +cm_id_priv->id.state = IW_CM_STATE_ESTABLISHED; >> +} else >> +cm_id_priv->id.state = IW_CM_STATE_IDLE; >> + >> +spin_unlock_irqrestore(&cm_id_priv->lock, flags); + >> +/* Call the client CM handler */ >> +return

RE: [openib-general] Re: port_num

2006-03-16 Thread Caitlin Bestler
Roland Dreier wrote: > Roland> If one offload device can share an address with the host > Roland> stack, why can't I give the same address to another > Roland> offload device? > > Caitlin> How would you select which offload device to use? > > Routing I guess. With existing stack

RE: [openib-general] Re: [PATCH 3/3] iWARP CM

2006-03-16 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Tom Tucker wrote: >> The iWARP CM prevents this from happening by having a state >> (DESTROYING) that prevents events from being delivered after >> iw_destroy_cm_id has returned. This approach avoids having either the >> kernel application thread or the event thread block

RE: [openib-general] Re: Re: port_num

2006-03-16 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > On Thu, 2006-03-16 at 10:45 -0500, James Lentini wrote: >> >> On Thu, 16 Mar 2006, Steve Wise wrote: >> >> swise> Just to clarify: Tom is talking about iWARP devices here that >> swise> support the native stack -and- the rdma stack with the same >> swise> offload devic

RE: [openib-general] Re: port_num

2006-03-16 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: >> I'd like to suggest that CMA implement its own rule along the lines >> of: if port is 0, bind to a port number unused by CMA in this port >> space and on this device. > > To clarify: a user is permitted to bind to a specific port, > but use a wildcard IP address. Just

RE: [openib-general] Re: port_num

2006-03-16 Thread Caitlin Bestler
Roland Dreier wrote: > Caitlin> I can see how an offload device can share an IP address > Caitlin> with the host stack and hence need to prevent conflicting > Caitlin> uses of the TCP port. But how do *two* offload devices > Caitlin> share the same IP address with each other and/or

RE: [openib-general] Re: port_num

2006-03-15 Thread Caitlin Bestler
Sean Hefty wrote: > Caitlin Bestler wrote: >> How about something along the lines of stating that the transport >> specific CM is responsible for translating 0 to a number that does >> not conflict with the host stack, and then leave the details to be >> implemented as

RE: [openib-general] Re: port_num

2006-03-15 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Quoting r. Sean Hefty <[EMAIL PROTECTED]>: >> Subject: Re: [openib-general] Re: port_num >> >> Michael S. Tsirkin wrote: >>> OK, so if we all agree bind with port 0 will select a free one, I'll >>> proceed with this assumption and Sean will code up the IB part, >>> iWarp

RE: [openib-general] [PATCH 3/3] iWARP CM

2006-03-09 Thread Caitlin Bestler
Sean Hefty wrote: > Caitlin Bestler wrote: >> No it doesn't map to the physical port, it just typically maps to a >> physical port. It maps to an L2 endpoint. The ethernet portion of >> the device may be doing binding. >> >> In either case, the L2 endpoi

[openib-general] RE: Re: RFC: e2e credits

2006-03-09 Thread Caitlin Bestler
Sean Hefty wrote: >> What does the flow_control parameter in ib_cm and rdma_cma >> conn_param do? If that's what it is, I think its easier to just add >> API to ib verbs making it possible to implement it than invent a >> device-specific way to do it per driver. > > As far as I can tell, it doesn'

RE: [openib-general] [PATCH 3/3] iWARP CM

2006-03-08 Thread Caitlin Bestler
-Original Message > This relates back to the question on whether a port has > more than one address. It depends on your definition of a > port. If the definition of a port is a netdevice then it > is only necessary to provide one of the addresses. Existing > techniques allow fetching the

RE: [openib-general] [PATCH 3/3] iWARP CM

2006-03-08 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Tom Tucker wrote: >> +#include >> +#include > > Is this needed? > >> +#include >> + >> +#include "cm_msgs.h" > > Is this needed? > >> +struct iwcm_id_private; > > I think this can be removed. > >> +struct iwcm_device; >> +struct iwcm_port { >> +struct iwcm_de

RE: [openib-general] Re: RFC: e2e credits

2006-03-07 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > Quoting r. Michael S. Tsirkin <[EMAIL PROTECTED]>: >> Hello! >> As was recently discussed on this list, infiniband implements an >> optional hardware end to end credits mechanism, with credits encoded >> in the AETH field in an ACK packet. >> >> The result is that an app

RE: [openib-general] RE: [RFC] support transports whose native endpoint is not a socket

2006-03-07 Thread Caitlin Bestler
Or Gerlitz wrote: > Rewinding... and starting from scratch, the idea was to > confirm to the following scheme which is now (and ever to be) used by > open iscsi: > > The current user space code assumes that the transport is > using a socket and act as follows: > > +1 target discovery over TCP/IP

[openib-general] RE: [PATCH 2/6] IB: match connection requests based on private data

2006-03-06 Thread Caitlin Bestler
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Sean Hefty > Sent: Monday, March 06, 2006 11:04 AM > To: 'Roland Dreier' > Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org; > openib-general@openib.org > Subject: [PATCH 2/6] IB: match conn

RE: [openib-general] [PATCH 03/18] [RFC] Provider Registration and Methods

2006-03-06 Thread Caitlin Bestler
> -Original Message- > From: Sean Hefty [mailto:[EMAIL PROTECTED] > Sent: Monday, March 06, 2006 10:40 AM > To: Caitlin Bestler > Cc: Steve Wise; openib-general > Subject: Re: [openib-general] [PATCH 03/18] [RFC] Provider > Registration and Methods > > Ca

RE: [openib-general] [PATCH 03/18] [RFC] Provider Registration and Methods

2006-03-06 Thread Caitlin Bestler
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Sean Hefty > Sent: Monday, March 06, 2006 10:27 AM > To: Steve Wise > Cc: openib-general > Subject: Re: [openib-general] [PATCH 03/18] [RFC] Provider > Registration and Methods > > Steve Wise wrote

RE: [openib-general] [PATCH 09/18] [RFC] Provider iWARP Connection Management

2006-03-06 Thread Caitlin Bestler
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Steve Wise > Sent: Monday, March 06, 2006 10:07 AM > To: openib-general > Subject: [openib-general] [PATCH 09/18] [RFC] Provider iWARP > Connection Management > > ISSUES: > > - CM should pass down

RE: [openib-general] [PATCH 03/18] [RFC] Provider Registration and Methods

2006-03-06 Thread Caitlin Bestler
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Steve Wise > Sent: Monday, March 06, 2006 10:05 AM > To: openib-general > Subject: [openib-general] [PATCH 03/18] [RFC] Provider > Registration and Methods > > > ISSUES: > > - we're exporting the

[openib-general] RE: [RFC] support transports whose native endpoint is not a socket

2006-03-06 Thread Caitlin Bestler
> -- Forwarded message -- > From: Or Gerlitz <[EMAIL PROTECTED]> > Date: Mar 6, 2006 5:40 AM > Subject: [RFC] support transports whose native endpoint is > not a socket > To: [EMAIL PROTECTED] > > > The patch below is an initial drop (which compiles and work with the > open-is

  1   2   3   4   5   >