Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Eric W. Biederman
Matt Leininger <[EMAIL PROTECTED]> writes: > On Thu, 2005-02-10 at 12:27 -0800, Grant Grundler wrote: > > On Thu, Feb 10, 2005 at 12:05:58PM -0800, Matt Leininger wrote: > > > uDAPL - Oracle, MPI > > > kDAPL - iSER, NFS over RDMA, Lustre? > > > > Lustre will use Sandia Portals AFAIK. > > Anyo

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Eric W. Biederman
"Kanevsky, Arkady" <[EMAIL PROTECTED]> writes: > For kDAPL: > The iSER has been submitted to Open Ib by Voltaire already. > NFS-RDMA is at http://sourceforge.net/projects/nfs-rdma/. > > For uDAPL: Oracle, DB2 and MPI. > I am not aware if there is an open source MPI version on uDAPL. > > These ar

[openib-general] enhanced libsdp

2005-02-10 Thread Tom Duffy
At the workshop this week, Libor mentioned to me that somebody (can't remember who, maybe at Mellanox?) had modified libsdp to create a transparent connect() and listen(). It would be *really cool* if we could get this into the repository. A pointer to the work would be ok too. Thanks, -tduffy

[openib-general] [PATCH] use module_param() instead of MODULE_PARM() in SDP

2005-02-10 Thread Tom Duffy
use module_param() instead of MODULE_PARM() Signed-off-by: Tom Duffy <[EMAIL PROTECTED]> Index: drivers/infiniband/ulp/sdp/sdp_inet.c === --- drivers/infiniband/ulp/sdp/sdp_inet.c (revision 1766) +++ drivers/infiniband/ulp/sdp/

[openib-general] [PATCH][Take2] Change AF_INET_SDP to 27

2005-02-10 Thread Tom Duffy
Oops. I only changed it one place. Why is it defined in two places? Signed-off-by: Tom Duffy <[EMAIL PROTECTED]> Index: drivers/infiniband/ulp/sdp/sdp_sock.h === --- drivers/infiniband/ulp/sdp/sdp_sock.h (revision 1766) +++ d

[openib-general] [PATCH] Fix init and exit functions in SDP

2005-02-10 Thread Tom Duffy
Signed-off-by: Tom Duffy <[EMAIL PROTECTED]> Index: drivers/infiniband/ulp/sdp/sdp_inet.c === --- drivers/infiniband/ulp/sdp/sdp_inet.c (revision 1766) +++ drivers/infiniband/ulp/sdp/sdp_inet.c (working copy) @@ -1707,7 +1

[openib-general] [PATCH] Change AF_INET_SDP to 27

2005-02-10 Thread Tom Duffy
It appears AF_LLC is already 26 in include/linux/socket.h. Change SDP to 27. Signed-off-by: Tom Duffy <[EMAIL PROTECTED]> Index: drivers/infiniband/ulp/sdp/sdp_sock.h === --- drivers/infiniband/ulp/sdp/sdp_sock.h (revision 176

[openib-general] IB now in Ubuntu

2005-02-10 Thread Roland Dreier
Ubuntu just pulled in a 2.6.11 snapshot for their next release, which is due in April: http://lists.ubuntu.com/archives/hoary-changes/2005-February/002512.html I checked and they have enabled IB in their configs. So now both Fedora and Ubuntu will be shipping IB. - R. _

Re: [openib-general] RFC on SDP checkin

2005-02-10 Thread Tom Duffy
On Thu, 2005-02-10 at 19:09 -0800, Libor Michalek wrote: > OK, I've gone with option number 3. Here is a one or two line > description of each file, and a TODO list. Both are checked in > along with the code in the infiniband/ulp/sdp directory. Of course, you will need this patch to plumb SDP in

Re: [openib-general] RFC on SDP checkin

2005-02-10 Thread Libor Michalek
On Fri, Feb 04, 2005 at 01:26:32PM -0800, Tom Duffy wrote: > On Fri, 2005-02-04 at 11:42 -0800, Libor Michalek wrote: > > You mean a quick description of the code, like the primary contents of > > each file and how to get it to do something ? > > That would be great start. OK, I've gone with

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Dhabaleswar Panda
> For kDAPL: > The iSER has been submitted to Open Ib by Voltaire already. > NFS-RDMA is at http://sourceforge.net/projects/nfs-rdma/. > > For uDAPL: Oracle, DB2 and MPI. > I am not aware if there is an open source MPI version on uDAPL. As I indicated in my talk at the OpenIB workshop, we are pro

RE: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Kanevsky, Arkady
For kDAPL: The iSER has been submitted to Open Ib by Voltaire already. NFS-RDMA is at http://sourceforge.net/projects/nfs-rdma/. For uDAPL: Oracle, DB2 and MPI. I am not aware if there is an open source MPI version on uDAPL. These are publicly known. As far as changing the uDAPL or kDAPL APIs. T

[openib-general] PayPal Verification

2005-02-10 Thread PayPal . com
  Dear valued PayPal® member:  It has come to our attention that your PayPal® account information needs to be updated as part of our continuing commitment to protect your account and to reduce the instance of fraud on our website.  If you could please take 5-10 minutes out of your online exper

[openib-general] PayPal Verification

2005-02-10 Thread PayPal . com
  Dear valued PayPal® member:  It has come to our attention that your PayPal® account information needs to be updated as part of our continuing commitment to protect your account and to reduce the instance of fraud on our website.  If you could please take 5-10 minutes out of your online exper

Re: [openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Johannes Erdfelt
On Thu, Feb 10, 2005, Grant Grundler <[EMAIL PROTECTED]> wrote: > On Thu, Feb 10, 2005 at 12:39:41PM -0800, Johannes Erdfelt wrote: > > The kernel code should be lean and unless there is an immediate or short > > term use for a feature, it probably shouldn't be there. We should be > > asking "why k

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Libor Michalek
On Thu, Feb 10, 2005 at 12:36:39PM -0800, Matt Leininger wrote: > On Thu, 2005-02-10 at 12:27 -0800, Grant Grundler wrote: > > On Thu, Feb 10, 2005 at 12:05:58PM -0800, Matt Leininger wrote: > > > uDAPL - Oracle, MPI > > > kDAPL - iSER, NFS over RDMA, Lustre? > > > > Lustre will use Sandia Por

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Grant Grundler
On Thu, Feb 10, 2005 at 12:39:17PM -0800, Matt Leininger wrote: >Good point. Lustre is open source, but only after the code is old. > I wish they would open up the project and do real open development. Oh > well. Ugh. Chrisptoph is right. I was under the illusion Lustre was a "real" open sou

Re: [openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Grant Grundler
On Thu, Feb 10, 2005 at 12:39:41PM -0800, Johannes Erdfelt wrote: > The kernel code should be lean and unless there is an immediate or short > term use for a feature, it probably shouldn't be there. We should be > asking "why keep it in the code?", not "why should we remove it from > the code?" en

[openib-general] Re: Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Michael S. Tsirkin
Quoting r. Sean Hefty <[EMAIL PROTECTED]>: > Subject: Re: Re: [PATCH (updated)] ipoib: dont lock tx on completion > > Michael S. Tsirkin wrote: > > >>I don't see a specific race -- I'm just not totally comfortable yet, > >>even with that fixed. The issue is not necessarily reads being > >>delaye

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Matt Leininger
On Thu, 2005-02-10 at 21:11 +0100, Christoph Hellwig wrote: > On Thu, Feb 10, 2005 at 12:05:58PM -0800, Matt Leininger wrote: > > kDAPL - iSER, NFS over RDMA, Lustre? > > Okay, we have iSER code and maybe there will be NFS code. Lustre > doesn't matter at all for any possible design because it'

[openib-general] Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [PATCH (updated)] ipoib: dont lock tx on completion > > Michael> Can they be moved to before the wmb? I think not. > > Why not? All wmb() does is guarantee that stores issued before the > wmb() will complete before stores issued af

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Matt Leininger
On Thu, 2005-02-10 at 12:27 -0800, Grant Grundler wrote: > On Thu, Feb 10, 2005 at 12:05:58PM -0800, Matt Leininger wrote: > > uDAPL - Oracle, MPI > > kDAPL - iSER, NFS over RDMA, Lustre? > > Lustre will use Sandia Portals AFAIK. > Anyone know what Portals will use? > They might directly progr

Re: [openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Johannes Erdfelt
On Thu, Feb 10, 2005, Grant Grundler <[EMAIL PROTECTED]> wrote: > Offline, I exchange email with Roland about this attribute. > Seems like we want to know the answer to these questions: > > o is there any code outside of openib.org using this attribute? > o if so, how useful is it? > > Roland is

Re: [openib-general] opensm segfaults after CL_INSUFFICIENT_MEMORY

2005-02-10 Thread Hal Rosenstock
On Thu, 2005-02-10 at 14:44, Bernhard Fischer wrote: > Hi, > > I'm seeing the segfault below when i try to run opensm. > > gen2 as of 09.02.2005, only thing i changed in order to try to > circumvent those > ib_mthca :04:00.0: CQ overrun on CQN 0082 > was setting IPOIB_NUM_WC to 1 as noted

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Libor Michalek
On Thu, Feb 10, 2005 at 12:05:58PM -0800, Matt Leininger wrote: > On Thu, 2005-02-10 at 10:51 -0800, Tom Duffy wrote: > > On Thu, 2005-02-10 at 18:46 +0100, Christoph Hellwig wrote: > > > > > - what are the users > > > > NFS over RDMA, maybe -- so RPC. Anybody else know others? > > > uDAPL -

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Grant Grundler
On Thu, Feb 10, 2005 at 12:05:58PM -0800, Matt Leininger wrote: > uDAPL - Oracle, MPI > kDAPL - iSER, NFS over RDMA, Lustre? Lustre will use Sandia Portals AFAIK. Anyone know what Portals will use? They might directly program to VAPI or something. grant ___

Re: [openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Grant Grundler
Offline, I exchange email with Roland about this attribute. Seems like we want to know the answer to these questions: o is there any code outside of openib.org using this attribute? o if so, how useful is it? Roland is certain none of the protocols in the proprietary Topspin stack uses this featu

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Christoph Hellwig
On Thu, Feb 10, 2005 at 12:05:58PM -0800, Matt Leininger wrote: > kDAPL - iSER, NFS over RDMA, Lustre? Okay, we have iSER code and maybe there will be NFS code. Lustre doesn't matter at all for any possible design because it's not freely available. __

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Matt Leininger
On Thu, 2005-02-10 at 10:51 -0800, Tom Duffy wrote: > On Thu, 2005-02-10 at 18:46 +0100, Christoph Hellwig wrote: > > Maybe you should lay down the requirement first. > > I'll take a crack at it. Let me know where I am off base. > > > - why do we need an intermediate API > > To get things work

[openib-general] Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Roland Dreier
Michael> Can they be moved to before the wmb? I think not. Why not? All wmb() does is guarantee that stores issued before the wmb() will complete before stores issued after the wmb(). It doesn't even say anything about when the stores have to complete, let alone the order of loads. The most

Re: [openib-general] Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Sean Hefty
Michael S. Tsirkin wrote: I don't see a specific race -- I'm just not totally comfortable yet, even with that fixed. The issue is not necessarily reads being delayed, since both an out-of-order CPU and a compiler may move reads much _earlier_ than they appear in the code (for example a read may be

[openib-general] opensm segfaults after CL_INSUFFICIENT_MEMORY

2005-02-10 Thread Bernhard Fischer
Hi, I'm seeing the segfault below when i try to run opensm. gen2 as of 09.02.2005, only thing i changed in order to try to circumvent those ib_mthca :04:00.0: CQ overrun on CQN 0082 was setting IPOIB_NUM_WC to 1 as noted here: http://openib.org/pipermail/openib-general/2004-December/00714

[openib-general] Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [PATCH (updated)] ipoib: dont lock tx on completion > > Michael> Hmm, I think I have it covered. > > Michael> If the read of netif_queue_stopped is delayed by a huge > Michael> margin, we see the interface stopped and wake i

[openib-general] Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Roland Dreier
Michael> Hmm, I think I have it covered. Michael> If the read of netif_queue_stopped is delayed by a huge Michael> margin, we see the interface stopped and wake it up. Michael> If the read of tx_tail is delayed, we see the tail Michael> updated and wake it up in the send routi

Re: [openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Sean Hefty
Roland Dreier wrote: Does this seem OK to apply to everyone? (The basic idea is to remove the rq_sig_type QP attribute, since it is a Tavor-specific extension that is not part of the IB spec, is currently unused, and that Mellanox doesn't want to support) I say go ahead and apply it. - Sean __

Re: [openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Roland Dreier
Does this seem OK to apply to everyone? (The basic idea is to remove the rq_sig_type QP attribute, since it is a Tavor-specific extension that is not part of the IB spec, is currently unused, and that Mellanox doesn't want to support) - R. ___ openib-g

Re: [openib-general] ib_device_cap_flags

2005-02-10 Thread Roland Dreier
Sean> Int is used because the returned value is a bitwise OR of Sean> the valid capabilities, and is not a value defined by the Sean> enum. We can change the enum values to #define if that Sean> would make it any easier. An enum type is an integer type, so it can represent the bit

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Tom Duffy
On Thu, 2005-02-10 at 18:46 +0100, Christoph Hellwig wrote: > Maybe you should lay down the requirement first. I'll take a crack at it. Let me know where I am off base. > - why do we need an intermediate API To get things working today with the code that people have already written. To create

Re: [openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Tom Duffy
On Thu, 2005-02-10 at 10:37 -0800, Sean Hefty wrote: > It looks like this leaves an extra comma at the end of the enum. Did > you compile/test these changes? In fact, I think this is preferred since then a new flag patch only needs to + one line. -tduffy signature.asc Description: This is a d

[openib-general] Re: Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Michael S. Tsirkin
Quoting r. Sean Hefty <[EMAIL PROTECTED]>: > Subject: Re: Re: FW: summary of my understanding on our common work on > openib.org > > Michael S. Tsirkin wrote: > > >Index: include/ib_verbs.h > >=== > >--- include/ib_verbs.h (re

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Grant Grundler
On Thu, Feb 10, 2005 at 11:28:54AM -0700, Ronald G. Minnich wrote: > > > On Thu, 10 Feb 2005, Grant Grundler wrote: > > > Well, that works best IFF one has time and a clue what to write. > > oops, that whole 'you have to have a clue' thing just ruled me out. yeah, Me too. But I can hammer some

Re: [openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Sean Hefty
Michael S. Tsirkin wrote: Index: include/ib_verbs.h === --- include/ib_verbs.h (revision 1759) +++ include/ib_verbs.h (working copy) @@ -73,7 +73,6 @@ enum ib_device_cap_flags { IB_DEVICE_RC_RNR_NAK_GEN= (1<<12),

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Ronald G. Minnich
On Thu, 10 Feb 2005, Grant Grundler wrote: > Well, that works best IFF one has time and a clue what to write. oops, that whole 'you have to have a clue' thing just ruled me out. To a large extent, I am writing tongue-in-cheek about the 'burn the spec' idea. My concern is that we avoid a slav

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Christoph Hellwig
On Thu, Feb 10, 2005 at 10:15:00AM -0800, Grant Grundler wrote: > On Thu, Feb 10, 2005 at 06:40:59PM +0100, Christoph Hellwig wrote: > > btw, what other RDMA transports that DAPL is supposed to layer above are > > actually available for Linux? (as in we'll see support for them in > > kernel.org soo

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Grant Grundler
On Thu, Feb 10, 2005 at 10:38:30AM -0700, Ronald G. Minnich wrote: > we're back into the "we did a spec, so the code can't change" loop. No we aren't. I consider it just a starting point. If someone volunteers a different starting point (like roland did for mthca), then let's use that. Either way

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Grant Grundler
On Thu, Feb 10, 2005 at 06:40:59PM +0100, Christoph Hellwig wrote: > btw, what other RDMA transports that DAPL is supposed to layer above are > actually available for Linux? (as in we'll see support for them in > kernel.org soon) I don't know for which definition of "soon": http://www.openrdma.org

[openib-general] Re: ib_device_cap_flags

2005-02-10 Thread Michael S. Tsirkin
Quoting r. Sean Hefty <[EMAIL PROTECTED]>: > Subject: Re: ib_device_cap_flags > > Roland Dreier wrote: > >Michael> Hi! enum ib_device_cap_flags from ib_verbs.h does not > >Michael> seem to be used anywhere. Specifically there does not > >Michael> seem to exist a way to find out the d

Re: [openib-general] ib_device_cap_flags

2005-02-10 Thread Sean Hefty
Roland Dreier wrote: Michael> Hi! enum ib_device_cap_flags from ib_verbs.h does not Michael> seem to be used anywhere. Specifically there does not Michael> seem to exist a way to find out the device capabilities. Michael> # grep -rIi ib_device_cap_flags . Michael> ./include/ib

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Christoph Hellwig
On Thu, Feb 10, 2005 at 09:43:07AM -0800, Tom Duffy wrote: > On Thu, 2005-02-10 at 10:38 -0700, Ronald G. Minnich wrote: > > we're back into the "we did a spec, so the code can't change" loop. > > burn the spec in your fireplace or wood stove > > Fair enough, the spec isn't a gold standard. But,

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Tom Duffy
On Thu, 2005-02-10 at 10:38 -0700, Ronald G. Minnich wrote: > we're back into the "we did a spec, so the code can't change" loop. > burn the spec in your fireplace or wood stove Fair enough, the spec isn't a gold standard. But, let's not burn the baby as well. Why don't we clean up the code and

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Christoph Hellwig
On Thu, Feb 10, 2005 at 10:38:30AM -0700, Ronald G. Minnich wrote: > we're back into the "we did a spec, so the code can't change" loop. > > To break out of the loop, one need merely: > - repeat "spec? what spec? we don't need no steenking spec" out loud* > - burn the spec in your fireplace or wo

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Ronald G. Minnich
On Thu, 10 Feb 2005, Christoph Hellwig wrote: > > The *DAPL API is already decided in a spec. If we change it, it will > > become lose compliance. > > Who cares? Specs don't matter at all for kernel APIs. The kDAPL API > as-is is won't go in the kernel, and no amount of cosmetic cleanup > ca

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Christoph Hellwig
On Thu, Feb 10, 2005 at 09:28:43AM -0800, Tom Duffy wrote: > On Thu, 2005-02-10 at 14:48 +0100, Christoph Hellwig wrote: > > Umm, you IB guys had lots of strange ideas on what code was acceptable > > until Roland stepped ahead and started the rewrite of the codebase. > > Don't worry. We have lear

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Tom Duffy
On Thu, 2005-02-10 at 14:48 +0100, Christoph Hellwig wrote: > Umm, you IB guys had lots of strange ideas on what code was acceptable > until Roland stepped ahead and started the rewrite of the codebase. Don't worry. We have learned. We will clean it up. The code with be in kernel normal form be

[openib-general] Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [PATCH (updated)] ipoib: dont lock tx on completion > > Michael> We are now only looking at tx_tail and > Michael> netif_queue_stopped outside the lock (tx_head is only > Michael> used inside the lock): > > Thanks, that help

[openib-general] Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Roland Dreier
Michael> We are now only looking at tx_tail and Michael> netif_queue_stopped outside the lock (tx_head is only Michael> used inside the lock): Thanks, that helps. Michael> We have a write barrier in between, so its guaranteed Michael> tx_tail is written before we test netif_qu

Re: FW: [openib-general] Minutes from DAPL BOF at OpenIB Workshop

2005-02-10 Thread Christoph Hellwig
On Thu, Feb 10, 2005 at 03:44:56PM +0200, Yaron Haviv wrote: > Christoph, > > See the following email from OpenIB mailing list > > Yaron Umm, you IB guys had lots of strange ideas on what code was acceptable until Roland stepped ahead and started the rewrite of the codebase. Similarly even if w

[openib-general] Re: FW: summary of my understanding on our common work on openib.org

2005-02-10 Thread Michael S. Tsirkin
Quoting r. Tziporet Koren <[EMAIL PROTECTED]>: > Subject: FW: summary of my understanding on our common work on openib.org > > please do it > > -Original Message- > From: Roland Dreier [mailto:[EMAIL PROTECTED] > Sent: Wednesday, February 09, 2005 9:00 AM > To: Tziporet Koren > Subject:

[openib-general] Re: [PATCH (updated)] ipoib: dont lock tx on completion

2005-02-10 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [PATCH (updated)] ipoib: dont lock tx on completion > > Michael> By testing again, and waking the net queue if completions > Michael> arrived, we can avoid taking the tx lock on send > Michael> completion in ip over ib, reduc