On Fri, 2 Sep 2005, Roland Dreier wrote:
Roland Yes, but what is the generic way?
James The generic way would be to handle this in a common
James layer. For the IB verbs + RDMA connection API to be as easy
James to use as the sockets API, then it needs to make this issue
On Tue, 30 Aug 2005, Roland Dreier wrote:
I just committed this SRP fix, which should make sure we don't use a
device after it's gone. And it actually simplifies the code a teeny bit...
The device could still be used after it's gone. For example:
- the user is configuring SRP via sysfs.
James The device could still be used after it's gone. For
James example:
James - the user is configuring SRP via sysfs. The thread in
James srp_create_target() has just called ib_sa_path_rec_get()
James [srp.c line 1209] and is waiting for the path record query
James to
-Original Message-
From: [EMAIL PROTECTED] [mailto:openib-general-
[EMAIL PROTECTED] On Behalf Of Roland Dreier
Sent: Tuesday, August 30, 2005 2:36 PM
To: Talpey, Thomas
Cc: openib-general@openib.org
Subject: Re: [openib-general] Re: RDMA Generic Connection Management
Thomas
Yaron Haviv wrote:
If all the ULPs need to do exactly the same, or the implementation is
different for IB/iWarp, than we should probably do it under the API like
its defined in kDAPL.
To do this means destroying QPs, CQs, PDs, MRs, etc. under the API. I don't see
that you want to do this.
At 02:06 PM 8/31/2005, Yaron Haviv wrote:
Also note that with Virtual machines this type of event may be more
frequent and we may want to decouple the ULPs from the actual hardware
s/may want/definitely want/
Tom.
___
openib-general mailing list
At 07:46 AM 8/31/2005, James Lentini wrote:
On Tue, 30 Aug 2005, Roland
Dreier wrote:
I just committed this SRP fix, which should make sure we don't use
a
device after it's gone. And it actually simplifies the code a
teeny bit...
The device could still be used after it's gone. For example:
-
On Mon, 2005-08-29 at 14:21 -0700, Sean Hefty wrote:
Michael S. Tsirkin wrote:
How is this different from what we have with ib_verbs now?
With ib_verbs, users receive notification of device addition/removal.
This interface doesn't require receiving that notification.
Why should it be part
On Tue, 30 Aug 2005, Guy German wrote:
I don't understand the difference between handling a device
received in cma_get_device and device received in ib_client.add ...
Consumers that receive a device via the ib_client.add
callback will receive a notification that it is no longer
available
Michael Wont users also activate verbs directly anyway, and so be
Michael required to handle this notification?
In the current verbs API, consumers only receive device pointers from
their client-add callback. They may use that pointer until they
return from the corresponding
Guy Couldn't consumers be registered as clients _before_ they
Guy call cma_get_device ?
Yes, consumers had better continue to use ib_register_client. The
question is how to define the cma_get_device API so that consumers can
handle hotplug correctly. And we need to make it simple enough
: RDMA Generic Connection Management
On Tue, 30 Aug 2005, Guy German wrote:
I don't understand the difference between handling a device
received in cma_get_device and device received in ib_client.add ...
Consumers that receive a device via the ib_client.add
callback will receive
Steve I thought all ULPs needed to register as an IB client
Steve regardless?
Right now they do, because there's no other way to get a struct
ib_device pointer. If we add a new API that returns a struct
ib_device pointer, then inevitably consumers will use it instead of
the current
not be a bad thing...
Stevo.
-Original Message-
From: Roland Dreier [mailto:[EMAIL PROTECTED]
Sent: Tuesday, August 30, 2005 12:13 PM
To: Steve Wise
Cc: 'James Lentini'; 'Guy German'; openib-general@openib.org
Subject: Re: [openib-general] Re: RDMA Generic Connection Management
At 01:12 PM 8/30/2005, Roland Dreier wrote:
Steve I thought all ULPs needed to register as an IB client
Steve regardless?
Right now they do, because there's no other way to get a struct
ib_device pointer. If we add a new API that returns a struct
ib_device pointer, then inevitably
Thomas Are you telling us that RPC/RDMA (for example) has to
Thomas handle hotplug events just to use IB? Isn't that the job
Thomas of a lower layer? NFS/Sockets don't have to deal with
Thomas these, f'rinstance.
Yes, if you want to talk directly to the device then you have to
Steve You could enforce that ulps must register as clients. Then
Steve get_ib_device() or whatever would only work if the client
Steve ULP is registered...
I don't think this really solves anything. You still have all sorts
of races to handle. For example, a device could be removed
At 02:01 PM 8/30/2005, Roland Dreier wrote:
Thomas Are you telling us that RPC/RDMA (for example) has to
Thomas handle hotplug events just to use IB? Isn't that the job
Thomas of a lower layer? NFS/Sockets don't have to deal with
Thomas these, f'rinstance.
Yes, if you want to talk
Thomas Verbs don't do that?
Not as they are currently defined. And I don't think we want to add
reference counting (aka cache-line pingpong) into every verbs call
including the fast path to make sure that a device doesn't go away in
the middle of the call.
- R.
At 02:15 PM 8/30/2005, Roland Dreier wrote:
Thomas Verbs don't do that?
Not as they are currently defined. And I don't think we want to add
reference counting (aka cache-line pingpong) into every verbs call
including the fast path to make sure that a device doesn't go away in
the middle of
Thomas Well, you're saying somebody has to do it, right? Is it
Thomas easier to fob this off to upper layers that (frankly)
Thomas don't care what hardware they're talking to!? This means
Thomas we have N copies of this, and N ways to do it. Talk about
Thomas cacheline
kDAPL does this!
:-)
At 02:35 PM 8/30/2005, Roland Dreier wrote:
Thomas Well, you're saying somebody has to do it, right? Is it
Thomas easier to fob this off to upper layers that (frankly)
Thomas don't care what hardware they're talking to!? This means
Thomas we have N copies of
, August 30, 2005 1:59 PM
To: Roland Dreier
Cc: Steve Wise; openib-general@openib.org
Subject: Re: [openib-general] Re: RDMA Generic Connection Management
kDAPL does this!
:-)
At 02:35 PM 8/30/2005, Roland Dreier wrote:
Thomas Well, you're saying somebody has to do it, right
Thomas kDAPL does this! :-)
Does what? As far as I can tell kDAPL just ignores hotplug and
routing and hopes the problems go away ;)
I do see some racy uses of atomic variables in kDAPL, but they don't
seem to protect against anything really.
- R.
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Steve Wise
Sent: Tuesday, August 30, 2005 12:01 PM
To: 'Talpey, Thomas'; 'Roland Dreier'
Cc: openib-general@openib.org
Subject: RE: [openib-general] Re: RDMA Generic Connection Management
At 03:08 PM 8/30/2005, Roland Dreier wrote:
Thomas kDAPL does this! :-)
Does what? As far as I can tell kDAPL just ignores hotplug and
routing and hopes the problems go away ;)
I was referring to kDAPL's architecture, which does in fact address
hotplug with async evd upcalls. In the early
At 04:10 PM 8/30/2005, Talpey, Thomas wrote:
At 03:08 PM 8/30/2005, Roland Dreier wrote:
Thomas kDAPL does this! :-)
Does what? As far as I can tell kDAPL just ignores hotplug and
routing and hopes the problems go away ;)
I was referring to kDAPL's architecture, which does in fact address
On Tue, 30 Aug 2005, Roland Dreier wrote:
Thomas kDAPL does this! :-)
Does what? As far as I can tell kDAPL just ignores hotplug and
routing and hopes the problems go away ;)
I do see some racy uses of atomic variables in kDAPL, but they don't
seem to protect against anything
-general@openib.org
Subject: Re: [openib-general] Re: RDMA Generic Connection Management
At 03:08 PM 8/30/2005, Roland Dreier wrote:
Thomas kDAPL does this! :-)
Does what? As far as I can tell kDAPL just ignores hotplug and
routing and hopes the problems go away ;)
I was referring
James For example when SRP's srp_add_one() function is called, I
James don't see a synchronization primitive setup to do
James this. Suppose for example that a thread was in SRP's CM
James callback handler, srp_cm_handler(), just before the call to
James ib_modify_qp() on line
I just committed this SRP fix, which should make sure we don't use a
device after it's gone. And it actually simplifies the code a teeny bit...
- R.
--- ib_srp.c(revision 3262)
+++ ib_srp.c(working copy)
@@ -1030,7 +1030,7 @@ static void srp_release_class_dev(struct
struct
Quoting r. Guy German [EMAIL PROTECTED]:
1. get_device returns device and port, according the local routing
tables, synchronously. ib_cma_connect calls the at module for address
resolving (cache handled) before calling the cm_connect.
How does one cancel address resolution request?
2.
Quoting r. Roland Dreier [EMAIL PROTECTED]:
Subject: Re: RDMA Generic Connection Management
James What happens if multiple devices can reach the destination
James address? How will they be enumerated to the consumer?
I guess we need to move towards the full horror of
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of
Michael S. Tsirkin
Sent: Monday, August 29, 2005 1:19 PM
To: Roland Dreier
Cc: openib-general@openib.org
Subject: [openib-general] Re: RDMA Generic Connection Management
I think that yes
Quoting r. Roland Dreier [EMAIL PROTECTED]:
Subject: Re: RDMA Generic Connection Management
Sean To focus on something a little different... do we want an
Sean API that returns a pointer to a device structure?
Sean Specifically, how does this affect device removal?
Hey, that's
Michael Roland, could you please explain what the problem is? If
Michael you have an outstanding request, and all devices went
Michael down, cant it simply be completed with an error status?
Something like:
get_device_for_route(device);
/* hot unplug device */
Quoting r. Sean Hefty [EMAIL PROTECTED]:
Subject: Re: RDMA Generic Connection Management
Guy German wrote:
- ib_cma_get_device (...) /* get device(1) or device+path(2) */
- pd = ib_alloc_pd(...) /* pd allocated in the given device */
- qp = ib_cma_create_qp(...) /* qp returned in init state
Quoting r. Roland Dreier [EMAIL PROTECTED]:
Subject: Re: RDMA Generic Connection Management
Michael Roland, could you please explain what the problem is? If
Michael you have an outstanding request, and all devices went
Michael down, cant it simply be completed with an error
Michael S. Tsirkin wrote:
How is this different from what we have with ib_verbs now?
With ib_verbs, users receive notification of device addition/removal.
This interface doesn't require receiving that notification.
I think that reasonable ULPs must register for hotplug events
in the ib
39 matches
Mail list logo
