RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
> -Original Message- > From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > ow...@vger.kernel.org] On Behalf Of Steve Wise > Sent: Friday, July 18, 2014 1:39 AM > To: 'Hefty, Sean'; 'Shirley Ma'; Devesh Sharma; 'Roland Dreier' > Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com > Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider > module > > > > > -Original Message- > > From: Steve Wise [mailto:sw...@opengridcomputing.com] > > Sent: Thursday, July 17, 2014 2:56 PM > > To: 'Hefty, Sean'; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > > Cc: 'linux-rdma@vger.kernel.org'; 'chuck.le...@oracle.com' > > Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider > > module > > > > > > > > > -Original Message- > > > From: Hefty, Sean [mailto:sean.he...@intel.com] > > > Sent: Thursday, July 17, 2014 2:50 PM > > > To: Steve Wise; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > > > Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com > > > Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma > > > provider module > > > > > > > > So the rdma cm is expected to increase the driver reference > > > > > count > > > > (try_module_get) for > > > > > each new cm id, then deference count (module_put) when cm id is > > > > destroyed? > > > > > > > > > > > > > No, I think he's saying the rdma-cm posts a > RDMA_CM_DEVICE_REMOVAL > > > > event to each application with rdmacm objects allocated, and each > > > > application is expected to destroy all the objects it has > > > > allocated before returning from the event handler. > > > > > > This is almost correct. The applications do not have to destroy all > > > the objects that > it has > > > allocated before returning from their event handler. E.g. an app > > > can queue a work > item > > > that does the destruction. The rdmacm will block in its ib_client > > > remove handler > until all > > > relevant rdma_cm_id's have been destroyed. > > > > > > > Thanks for the clarification. > > > > And looking at xprtrdma, it does handle the DEVICE_REMOVAL event in > rpcrdma_conn_upcall(). > It sets ep->rep_connected to -ENODEV, wakes everybody up, and calls > rpcrdma_conn_func() for that endpoint, which schedules > rep_connect_worker... and I gave up following the code path at this point... > :) > > For this to all work correctly, it would need to destroy all the QPs, MRs, > CQs, > etc for that device _before_ destroying the rdma cm ids. Otherwise the > provider module could be unloaded too soon... Okay, Should I try to handle device removal in this proposed fashion and post the v1. > > Steve. > > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the > body of a message to majord...@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
> -Original Message- > From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > ow...@vger.kernel.org] On Behalf Of Steve Wise > Sent: Friday, July 18, 2014 2:29 AM > To: 'Chuck Lever' > Cc: 'Hefty, Sean'; 'Shirley Ma'; Devesh Sharma; 'Roland Dreier'; linux- > r...@vger.kernel.org > Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider > module > > > > > -Original Message- > > From: linux-rdma-ow...@vger.kernel.org > > [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Chuck Lever > > Sent: Thursday, July 17, 2014 3:42 PM > > To: Steve Wise > > Cc: Hefty, Sean; Shirley Ma; Devesh Sharma; Roland Dreier; > > linux-rdma@vger.kernel.org > > Subject: Re: [for-next 1/2] xprtrdma: take reference of rdma provider > > module > > > > > > On Jul 17, 2014, at 4:08 PM, Steve Wise > wrote: > > > > > > > > > > >> -Original Message- > > >> From: Steve Wise [mailto:sw...@opengridcomputing.com] > > >> Sent: Thursday, July 17, 2014 2:56 PM > > >> To: 'Hefty, Sean'; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > > >> Cc: 'linux-rdma@vger.kernel.org'; 'chuck.le...@oracle.com' > > >> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma > > >> provider module > > >> > > >> > > >> > > >>> -Original Message- > > >>> From: Hefty, Sean [mailto:sean.he...@intel.com] > > >>> Sent: Thursday, July 17, 2014 2:50 PM > > >>> To: Steve Wise; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > > >>> Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com > > >>> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma > > >>> provider module > > >>> > > > So the rdma cm is expected to increase the driver reference > > > count > > (try_module_get) for > > > each new cm id, then deference count (module_put) when cm id is > > destroyed? > > > > > > > No, I think he's saying the rdma-cm posts a > > RDMA_CM_DEVICE_REMOVAL event to each application with > rdmacm > > objects allocated, and each application is expected to destroy > > all the objects it has allocated before returning from the event > > handler. > > >>> > > >>> This is almost correct. The applications do not have to destroy > > >>> all the objects > that > > > it has > > >>> allocated before returning from their event handler. E.g. an app > > >>> can queue a work > > > item > > >>> that does the destruction. The rdmacm will block in its ib_client > > >>> remove handler > > > until all > > >>> relevant rdma_cm_id's have been destroyed. > > >>> > > >> > > >> Thanks for the clarification. > > >> > > > > > > And looking at xprtrdma, it does handle the DEVICE_REMOVAL event in > > rpcrdma_conn_upcall(). > > > It sets ep->rep_connected to -ENODEV, wakes everybody up, and calls > > rpcrdma_conn_func() > > > for that endpoint, which schedules rep_connect_worker... and I gave > > > up following the > > code > > > path at this point... :) > > > > > > For this to all work correctly, it would need to destroy all the > > > QPs, MRs, CQs, etc > for > > > that device _before_ destroying the rdma cm ids. Otherwise the > > > provider module could > > be > > > unloaded too soon. > > > > We can't really deal with a CM_DEVICE_REMOVE event while there are > > active NFS mounts. > > > > System shutdown ordering should guarantee (one would hope) that NFS > > mount points are unmounted before the RDMA/IB core infrastructure is > > torn down. Ordering shouldn't matter as long all NFS activity has > > ceased before the CM tries to remove the device. > > > > So if something is hanging up the CM, there's something xprtrdma is > > not cleaning up properly. > > > > > Devesh, how are you reproducing this? Are you just rmmod'ing the ocrdma > module while there are active mounts? Yes, I am issuing rmmod while there is an active mount. In my case rmmod ocrdma remains blocked forever. Off-the-course of this discussion: Is there a reasoning behind not using ib_register_client()/ib_unregister_client() framework? > > > > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the > body of a message to majord...@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] iw_cxgb4: fix for 64-bit integer division
From: Sergei Shtylyov
Date: Thu, 17 Jul 2014 22:44:42 +0400
> Hello.
>
> On 07/17/2014 09:01 PM, Hariprasad Shenai wrote:
>
>> Fixed error introduced in commit id 7730b4c (" cxgb4/iw_cxgb4: work
>> request
>> logging feature") while compiling on 32 bit architecture reported by
>> kbuild.
>
>> Signed-off-by: Hariprasad Shenai
>> ---
>> drivers/infiniband/hw/cxgb4/device.c |3 ++-
>> 1 files changed, 2 insertions(+), 1 deletions(-)
>
>> diff --git a/drivers/infiniband/hw/cxgb4/device.c
>> b/drivers/infiniband/hw/cxgb4/device.c
>> index df1f1b5..bb0de61 100644
>> --- a/drivers/infiniband/hw/cxgb4/device.c
>> +++ b/drivers/infiniband/hw/cxgb4/device.c
> [...]
>> @@ -150,7 +151,7 @@ static int wr_log_show(struct seq_file *seq, void
>> *v)
>> int prev_ts_set = 0;
>> int idx, end;
>>
>> -#define ts2ns(ts) ((ts) * dev->rdev.lldi.cclk_ps / 1000)
>> +#define ts2ns(ts) div64_ul((ts) * dev->rdev.lldi.cclk_ps , 1000)
>
>Don't need space before comma.
I killed the space when applying this patch, thanks everyone.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH net-next v2 0/4] cxgb4i: Add ipv6 support
From: Anish Bhatt Date: Thu, 17 Jul 2014 00:18:14 -0700 >The following patchset add ipv6 support for the cxgb4i(iscsi) driver. > > Patch 1 moves a define from the iw_cxgb4 to cxgb4 to prevent code duplication, > as it is used by cxgb4i and iw_cxgb4 both. > Patch 2 exports symbols needed by cxgb4i for ipv6 support from cxgb4. > Patch 3 & 4 deal with actual ipv6 code in libcxgbi/cxgb4i. > > Submitting to net-next as changes were needed in cxgb4/iw_cxgb4, and > submitting > to scsi separately would break compilation, cc-ing scsi & rdma. > > v2 : Cleanup macro that prints ip addresses by using %pIS in patch 3 Series applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [for-next 1/2] xprtrdma: take reference of rdma provider module
On 07/17/2014 01:41 PM, Chuck Lever wrote: > On Jul 17, 2014, at 4:08 PM, Steve Wise wrote: > >> > >> > >>> >> -Original Message- >>> >> From: Steve Wise [mailto:sw...@opengridcomputing.com] >>> >> Sent: Thursday, July 17, 2014 2:56 PM >>> >> To: 'Hefty, Sean'; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' >>> >> Cc: 'linux-rdma@vger.kernel.org'; 'chuck.le...@oracle.com' >>> >> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider >>> >> module >>> >> >>> >> >>> >> >>> -Original Message- >>> From: Hefty, Sean [mailto:sean.he...@intel.com] >>> Sent: Thursday, July 17, 2014 2:50 PM >>> To: Steve Wise; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' >>> Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com >>> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider >>> module >>> >> > So the rdma cm is expected to increase the driver reference count > (try_module_get) for >> > each new cm id, then deference count (module_put) when cm id is > destroyed? >> > > > No, I think he's saying the rdma-cm posts a RDMA_CM_DEVICE_REMOVAL > event > to each > application with rdmacm objects allocated, and each application is > expected > to destroy all > the objects it has allocated before returning from the event handler. >>> >>> This is almost correct. The applications do not have to destroy all >>> the objects that >> > it has >>> allocated before returning from their event handler. E.g. an app can >>> queue a work >> > item >>> that does the destruction. The rdmacm will block in its ib_client >>> remove handler >> > until all >>> relevant rdma_cm_id's have been destroyed. >>> >>> >> >>> >> Thanks for the clarification. >>> >> >> > >> > And looking at xprtrdma, it does handle the DEVICE_REMOVAL event in >> > rpcrdma_conn_upcall(). >> > It sets ep->rep_connected to -ENODEV, wakes everybody up, and calls >> > rpcrdma_conn_func() >> > for that endpoint, which schedules rep_connect_worker... and I gave up >> > following the code >> > path at this point... :) >> > >> > For this to all work correctly, it would need to destroy all the QPs, MRs, >> > CQs, etc for >> > that device _before_ destroying the rdma cm ids. Otherwise the provider >> > module could be >> > unloaded too soon… > We can’t really deal with a CM_DEVICE_REMOVE event while there are active > NFS mounts. > > System shutdown ordering should guarantee (one would hope) that NFS > mount points are unmounted before the RDMA/IB core infrastructure is > torn down. Ordering shouldn’t matter as long all NFS activity has > ceased before the CM tries to remove the device. > > So if something is hanging up the CM, there’s something xprtrdma is not > cleaning up properly. I saw a problem once, restart the system without umounting the NFS. CM was hung on waiting for completion. It looks like a bug in xprtrdma cleanup up. I couldn't reproduce it. Call Trace: [] schedule+0x29/0x70 [] schedule_timeout+0x165/0x200 [] ? wait_for_completion+0xcf/0x110 [] ? __lock_release+0x9e/0x1f0 [] ? wait_for_completion+0xcf/0x110 [] wait_for_completion+0xd7/0x110 [] ? try_to_wake_up+0x260/0x260 [] cma_process_remove+0xee/0x110 [rdma_cm] [] cma_remove_one+0x4c/0x60 [rdma_cm] [] ib_unregister_device+0x4f/0x100 [ib_core] [] mlx4_ib_remove+0x2e/0x260 [mlx4_ib] [] mlx4_remove_device+0x69/0x80 [mlx4_core] [] mlx4_unregister_interface+0x43/0x80 [mlx4_core] [] mlx4_ib_cleanup+0x10/0x23 [mlx4_ib] [] SyS_delete_module+0x183/0x1e0 [] ? __audit_syscall_entry+0x94/0x100 [] ? lockdep_sys_exit_thunk+0x35/0x67 [] system_call_fastpath+0x16/0x1b Shirley -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
> -Original Message- > From: linux-rdma-ow...@vger.kernel.org > [mailto:linux-rdma-ow...@vger.kernel.org] On > Behalf Of Chuck Lever > Sent: Thursday, July 17, 2014 3:42 PM > To: Steve Wise > Cc: Hefty, Sean; Shirley Ma; Devesh Sharma; Roland Dreier; > linux-rdma@vger.kernel.org > Subject: Re: [for-next 1/2] xprtrdma: take reference of rdma provider module > > > On Jul 17, 2014, at 4:08 PM, Steve Wise wrote: > > > > > > >> -Original Message- > >> From: Steve Wise [mailto:sw...@opengridcomputing.com] > >> Sent: Thursday, July 17, 2014 2:56 PM > >> To: 'Hefty, Sean'; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > >> Cc: 'linux-rdma@vger.kernel.org'; 'chuck.le...@oracle.com' > >> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider > >> module > >> > >> > >> > >>> -Original Message- > >>> From: Hefty, Sean [mailto:sean.he...@intel.com] > >>> Sent: Thursday, July 17, 2014 2:50 PM > >>> To: Steve Wise; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > >>> Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com > >>> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider > >>> module > >>> > > So the rdma cm is expected to increase the driver reference count > (try_module_get) for > > each new cm id, then deference count (module_put) when cm id is > destroyed? > > > > No, I think he's saying the rdma-cm posts a RDMA_CM_DEVICE_REMOVAL event > to each > application with rdmacm objects allocated, and each application is > expected > to destroy all > the objects it has allocated before returning from the event handler. > >>> > >>> This is almost correct. The applications do not have to destroy all the > >>> objects that > > it has > >>> allocated before returning from their event handler. E.g. an app can > >>> queue a work > > item > >>> that does the destruction. The rdmacm will block in its ib_client remove > >>> handler > > until all > >>> relevant rdma_cm_id's have been destroyed. > >>> > >> > >> Thanks for the clarification. > >> > > > > And looking at xprtrdma, it does handle the DEVICE_REMOVAL event in > rpcrdma_conn_upcall(). > > It sets ep->rep_connected to -ENODEV, wakes everybody up, and calls > rpcrdma_conn_func() > > for that endpoint, which schedules rep_connect_worker... and I gave up > > following the > code > > path at this point... :) > > > > For this to all work correctly, it would need to destroy all the QPs, MRs, > > CQs, etc for > > that device _before_ destroying the rdma cm ids. Otherwise the provider > > module could > be > > unloaded too soon. > > We can't really deal with a CM_DEVICE_REMOVE event while there are active > NFS mounts. > > System shutdown ordering should guarantee (one would hope) that NFS > mount points are unmounted before the RDMA/IB core infrastructure is > torn down. Ordering shouldn't matter as long all NFS activity has > ceased before the CM tries to remove the device. > > So if something is hanging up the CM, there's something xprtrdma is not > cleaning up properly. > Devesh, how are you reproducing this? Are you just rmmod'ing the ocrdma module while there are active mounts? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [for-next 1/2] xprtrdma: take reference of rdma provider module
On Jul 17, 2014, at 4:08 PM, Steve Wise wrote: > > >> -Original Message- >> From: Steve Wise [mailto:sw...@opengridcomputing.com] >> Sent: Thursday, July 17, 2014 2:56 PM >> To: 'Hefty, Sean'; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' >> Cc: 'linux-rdma@vger.kernel.org'; 'chuck.le...@oracle.com' >> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider module >> >> >> >>> -Original Message- >>> From: Hefty, Sean [mailto:sean.he...@intel.com] >>> Sent: Thursday, July 17, 2014 2:50 PM >>> To: Steve Wise; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' >>> Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com >>> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider module >>> > So the rdma cm is expected to increase the driver reference count (try_module_get) for > each new cm id, then deference count (module_put) when cm id is destroyed? > No, I think he's saying the rdma-cm posts a RDMA_CM_DEVICE_REMOVAL event to each application with rdmacm objects allocated, and each application is expected to destroy all the objects it has allocated before returning from the event handler. >>> >>> This is almost correct. The applications do not have to destroy all the >>> objects that > it has >>> allocated before returning from their event handler. E.g. an app can queue >>> a work > item >>> that does the destruction. The rdmacm will block in its ib_client remove >>> handler > until all >>> relevant rdma_cm_id's have been destroyed. >>> >> >> Thanks for the clarification. >> > > And looking at xprtrdma, it does handle the DEVICE_REMOVAL event in > rpcrdma_conn_upcall(). > It sets ep->rep_connected to -ENODEV, wakes everybody up, and calls > rpcrdma_conn_func() > for that endpoint, which schedules rep_connect_worker... and I gave up > following the code > path at this point... :) > > For this to all work correctly, it would need to destroy all the QPs, MRs, > CQs, etc for > that device _before_ destroying the rdma cm ids. Otherwise the provider > module could be > unloaded too soon… We can’t really deal with a CM_DEVICE_REMOVE event while there are active NFS mounts. System shutdown ordering should guarantee (one would hope) that NFS mount points are unmounted before the RDMA/IB core infrastructure is torn down. Ordering shouldn’t matter as long all NFS activity has ceased before the CM tries to remove the device. So if something is hanging up the CM, there’s something xprtrdma is not cleaning up properly. -- Chuck Lever chuck[dot]lever[at]oracle[dot]com -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [for-next 1/2] xprtrdma: take reference of rdma provider module
On 07/17/2014 12:55 PM, Steve Wise wrote: > > >> -Original Message- >> From: Hefty, Sean [mailto:sean.he...@intel.com] >> Sent: Thursday, July 17, 2014 2:50 PM >> To: Steve Wise; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' >> Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com >> Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider module >> So the rdma cm is expected to increase the driver reference count >>> (try_module_get) for each new cm id, then deference count (module_put) when cm id is >>> destroyed? >>> >>> No, I think he's saying the rdma-cm posts a RDMA_CM_DEVICE_REMOVAL event >>> to each >>> application with rdmacm objects allocated, and each application is expected >>> to destroy all >>> the objects it has allocated before returning from the event handler. >> >> This is almost correct. The applications do not have to destroy all the >> objects that it > has >> allocated before returning from their event handler. E.g. an app can queue >> a work item >> that does the destruction. The rdmacm will block in its ib_client remove >> handler until > all >> relevant rdma_cm_id's have been destroyed. >> > > Thanks for the clarification. Thanks, checked the cma code, the reference count maintains there. >>> And I think the ib_verbs core calls each ib_client's remove handler when an >>> rdma provider >>> unregisters with the core. >> >> yes > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 1/1] ibacm: Incorrect return value check in path resolution
thanks - merged -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
> -Original Message- > From: Steve Wise [mailto:sw...@opengridcomputing.com] > Sent: Thursday, July 17, 2014 2:56 PM > To: 'Hefty, Sean'; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > Cc: 'linux-rdma@vger.kernel.org'; 'chuck.le...@oracle.com' > Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider module > > > > > -Original Message- > > From: Hefty, Sean [mailto:sean.he...@intel.com] > > Sent: Thursday, July 17, 2014 2:50 PM > > To: Steve Wise; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > > Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com > > Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider module > > > > > > So the rdma cm is expected to increase the driver reference count > > > (try_module_get) for > > > > each new cm id, then deference count (module_put) when cm id is > > > destroyed? > > > > > > > > > > No, I think he's saying the rdma-cm posts a RDMA_CM_DEVICE_REMOVAL event > > > to each > > > application with rdmacm objects allocated, and each application is > > > expected > > > to destroy all > > > the objects it has allocated before returning from the event handler. > > > > This is almost correct. The applications do not have to destroy all the > > objects that it has > > allocated before returning from their event handler. E.g. an app can queue > > a work item > > that does the destruction. The rdmacm will block in its ib_client remove > > handler until all > > relevant rdma_cm_id's have been destroyed. > > > > Thanks for the clarification. > And looking at xprtrdma, it does handle the DEVICE_REMOVAL event in rpcrdma_conn_upcall(). It sets ep->rep_connected to -ENODEV, wakes everybody up, and calls rpcrdma_conn_func() for that endpoint, which schedules rep_connect_worker... and I gave up following the code path at this point... :) For this to all work correctly, it would need to destroy all the QPs, MRs, CQs, etc for that device _before_ destroying the rdma cm ids. Otherwise the provider module could be unloaded too soon... Steve. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
> -Original Message- > From: Hefty, Sean [mailto:sean.he...@intel.com] > Sent: Thursday, July 17, 2014 2:50 PM > To: Steve Wise; 'Shirley Ma'; 'Devesh Sharma'; 'Roland Dreier' > Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com > Subject: RE: [for-next 1/2] xprtrdma: take reference of rdma provider module > > > > So the rdma cm is expected to increase the driver reference count > > (try_module_get) for > > > each new cm id, then deference count (module_put) when cm id is > > destroyed? > > > > > > > No, I think he's saying the rdma-cm posts a RDMA_CM_DEVICE_REMOVAL event > > to each > > application with rdmacm objects allocated, and each application is expected > > to destroy all > > the objects it has allocated before returning from the event handler. > > This is almost correct. The applications do not have to destroy all the > objects that it has > allocated before returning from their event handler. E.g. an app can queue a > work item > that does the destruction. The rdmacm will block in its ib_client remove > handler until all > relevant rdma_cm_id's have been destroyed. > Thanks for the clarification. > > And I think the ib_verbs core calls each ib_client's remove handler when an > > rdma provider > > unregisters with the core. > > yes -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
> > So the rdma cm is expected to increase the driver reference count > (try_module_get) for > > each new cm id, then deference count (module_put) when cm id is > destroyed? > > > > No, I think he's saying the rdma-cm posts a RDMA_CM_DEVICE_REMOVAL event > to each > application with rdmacm objects allocated, and each application is expected > to destroy all > the objects it has allocated before returning from the event handler. This is almost correct. The applications do not have to destroy all the objects that it has allocated before returning from their event handler. E.g. an app can queue a work item that does the destruction. The rdmacm will block in its ib_client remove handler until all relevant rdma_cm_id's have been destroyed. > And I think the ib_verbs core calls each ib_client's remove handler when an > rdma provider > unregisters with the core. yes -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
> -Original Message-
> From: Shirley Ma [mailto:shirley...@oracle.com]
> Sent: Thursday, July 17, 2014 1:58 PM
> To: Hefty, Sean; Steve Wise; Devesh Sharma; Roland Dreier
> Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com
> Subject: Re: [for-next 1/2] xprtrdma: take reference of rdma provider module
>
>
>
> On 07/17/2014 09:06 AM, Hefty, Sean wrote:
> >> On 7/17/2014 9:01 AM, Devesh Sharma wrote:
> >>> If verndor driver is attempted for removal while xprtrdma still has an
> >>> active mount, the removal of driver may never complete and can cause
> >>> unseen races or in worst case system crash.
> >>>
> >>> To solve this, xprtrdma module should get reference of struct ib_device
> >>> structure for every mount. Reference is taken after local device address
> >>> resolution is completed successfuly.
> >>>
> >>> reference to the struct ib_device pointer is put just before cm_id
> >> destruction.
> >>>
> >>> Signed-off-by: Devesh Sharma
> >>
> >> This seems like an issue with the rdma-cm or rdma core, not xprtrdma. I
> >> see that user rdma applications cause a ref on the provider module here
> >> in ib_uverbs_open():
> >>
> >> if (!try_module_get(dev->ib_dev->owner)) {
> >> ret = -ENODEV;
> >> goto err;
> >>
> >>
> >> Maybe kernel applications that allocate device resources should cause a
> >> ref on the provider's module.
> >>
> >> Sean/Roland, is there some history here as to how rdma provider module
> >> removal should be handled?
> >
> > The kernel modules should are not expected to access the rdma devices after
> > their
> remove device callback has been invoked. The rdma cm basically forwards the
> device
> removal on a per id basis. Apps are expected to destroy the id after
> receiving that
callback.
> The rdma cm should block in the remove device call until all id's associated
> with the
> removed device have been destroyed.
>
> So the rdma cm is expected to increase the driver reference count
> (try_module_get) for
> each new cm id, then deference count (module_put) when cm id is destroyed?
>
No, I think he's saying the rdma-cm posts a RDMA_CM_DEVICE_REMOVAL event to
each
application with rdmacm objects allocated, and each application is expected to
destroy all
the objects it has allocated before returning from the event handler.
And I think the ib_verbs core calls each ib_client's remove handler when an
rdma provider
unregisters with the core.
Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [for-next 1/2] xprtrdma: take reference of rdma provider module
On 07/17/2014 09:06 AM, Hefty, Sean wrote:
>> On 7/17/2014 9:01 AM, Devesh Sharma wrote:
>>> If verndor driver is attempted for removal while xprtrdma still has an
>>> active mount, the removal of driver may never complete and can cause
>>> unseen races or in worst case system crash.
>>>
>>> To solve this, xprtrdma module should get reference of struct ib_device
>>> structure for every mount. Reference is taken after local device address
>>> resolution is completed successfuly.
>>>
>>> reference to the struct ib_device pointer is put just before cm_id
>> destruction.
>>>
>>> Signed-off-by: Devesh Sharma
>>
>> This seems like an issue with the rdma-cm or rdma core, not xprtrdma. I
>> see that user rdma applications cause a ref on the provider module here
>> in ib_uverbs_open():
>>
>> if (!try_module_get(dev->ib_dev->owner)) {
>> ret = -ENODEV;
>> goto err;
>>
>>
>> Maybe kernel applications that allocate device resources should cause a
>> ref on the provider's module.
>>
>> Sean/Roland, is there some history here as to how rdma provider module
>> removal should be handled?
>
> The kernel modules should are not expected to access the rdma devices after
> their remove device callback has been invoked. The rdma cm basically
> forwards the device removal on a per id basis. Apps are expected to destroy
> the id after receiving that callback. The rdma cm should block in the remove
> device call until all id's associated with the removed device have been
> destroyed.
So the rdma cm is expected to increase the driver reference count
(try_module_get) for each new cm id, then deference count (module_put) when cm
id is destroyed?
> - Sean
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] iw_cxgb4: fix for 64-bit integer division
Hello.
On 07/17/2014 09:01 PM, Hariprasad Shenai wrote:
Fixed error introduced in commit id 7730b4c (" cxgb4/iw_cxgb4: work request
logging feature") while compiling on 32 bit architecture reported by kbuild.
Signed-off-by: Hariprasad Shenai
---
drivers/infiniband/hw/cxgb4/device.c |3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/device.c
b/drivers/infiniband/hw/cxgb4/device.c
index df1f1b5..bb0de61 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
[...]
@@ -150,7 +151,7 @@ static int wr_log_show(struct seq_file *seq, void *v)
int prev_ts_set = 0;
int idx, end;
-#define ts2ns(ts) ((ts) * dev->rdev.lldi.cclk_ps / 1000)
+#define ts2ns(ts) div64_ul((ts) * dev->rdev.lldi.cclk_ps , 1000)
Don't need space before comma.
WBR, Sergei
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/1] ibacm: Incorrect return value check in path resolution
From: Kaike Wan
The returned variable "addr" should be checked instead of the variable "ep" in
the function acm_svr_resolve_path() when query parameters are given in a path
record (eg, slid/dlid, sgid/dgid, etc).
Signed-off-by: Kaike Wan
---
src/acm.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/src/acm.c b/src/acm.c
index fae9d0b..11fda4c 100644
--- a/src/acm.c
+++ b/src/acm.c
@@ -946,7 +946,7 @@ acm_svr_resolve_path(struct acmc_client *client, struct
acm_msg *msg)
msg->resolve_data[0].info.addr, sizeof *path);
acm_log(2, "path %s\n", log_data);
addr = acm_get_ep_address(&msg->resolve_data[0]);
- if (!ep) {
+ if (!addr) {
acm_log(0, "notice - unknown local end point address\n");
return acmc_resolve_response(client->index, msg,
ACM_STATUS_ESRCADDR);
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] iw_cxgb4: fix for 64-bit integer division
Fixed error introduced in commit id 7730b4c (" cxgb4/iw_cxgb4: work request
logging feature") while compiling on 32 bit architecture reported by kbuild.
Signed-off-by: Hariprasad Shenai
---
drivers/infiniband/hw/cxgb4/device.c |3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/device.c
b/drivers/infiniband/hw/cxgb4/device.c
index df1f1b5..bb0de61 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -33,6 +33,7 @@
#include
#include
#include
+#include
#include
@@ -150,7 +151,7 @@ static int wr_log_show(struct seq_file *seq, void *v)
int prev_ts_set = 0;
int idx, end;
-#define ts2ns(ts) ((ts) * dev->rdev.lldi.cclk_ps / 1000)
+#define ts2ns(ts) div64_ul((ts) * dev->rdev.lldi.cclk_ps , 1000)
idx = atomic_read(&dev->rdev.wr_log_idx) &
(dev->rdev.wr_log_size - 1);
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
> On 7/17/2014 9:01 AM, Devesh Sharma wrote:
> > If verndor driver is attempted for removal while xprtrdma still has an
> > active mount, the removal of driver may never complete and can cause
> > unseen races or in worst case system crash.
> >
> > To solve this, xprtrdma module should get reference of struct ib_device
> > structure for every mount. Reference is taken after local device address
> > resolution is completed successfuly.
> >
> > reference to the struct ib_device pointer is put just before cm_id
> destruction.
> >
> > Signed-off-by: Devesh Sharma
>
> This seems like an issue with the rdma-cm or rdma core, not xprtrdma. I
> see that user rdma applications cause a ref on the provider module here
> in ib_uverbs_open():
>
> if (!try_module_get(dev->ib_dev->owner)) {
> ret = -ENODEV;
> goto err;
>
>
> Maybe kernel applications that allocate device resources should cause a
> ref on the provider's module.
>
> Sean/Roland, is there some history here as to how rdma provider module
> removal should be handled?
The kernel modules should are not expected to access the rdma devices after
their remove device callback has been invoked. The rdma cm basically forwards
the device removal on a per id basis. Apps are expected to destroy the id
after receiving that callback. The rdma cm should block in the remove device
call until all id's associated with the removed device have been destroyed.
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 2/2] xprtrdma: fix deallocation sequence of pd
> -Original Message-
> From: Steve Wise [mailto:sw...@opengridcomputing.com]
> Sent: Thursday, July 17, 2014 8:36 PM
> To: Devesh Sharma; linux-rdma@vger.kernel.org
> Cc: chuck.le...@oracle.com
> Subject: Re: [for-next 2/2] xprtrdma: fix deallocation sequence of pd
>
> On 7/17/2014 9:01 AM, Devesh Sharma wrote:
> > xprtrdma tries to destroy pd after destruction of cm_id. However, pd
> > should be deallocated before destruction of cm_id.
>
> Why?
>
> I think you really mean that the pd dealloc needs to be done before the
> module deref that you added in the first patch. But let us see what Roland
> says about how device removal is supposed to be handled when kernel
> applications are using the device...
Partially Yes, PD de-allocation should not be done after taking out the module
reference. In the allocation
Sequence PD is allocated after cm_id allocation hence should be de-allocated
before cm-id destroy.
>
>
>
> > Signed-off-by: Devesh Sharma
> > ---
> > net/sunrpc/xprtrdma/verbs.c | 22 +-
> > 1 files changed, 13 insertions(+), 9 deletions(-)
> >
> > diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
> > index b00e55e..096e94b 100644
> > --- a/net/sunrpc/xprtrdma/verbs.c
> > +++ b/net/sunrpc/xprtrdma/verbs.c
> > @@ -544,7 +544,7 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct
> sockaddr *addr, int memreg)
> > if (rc) {
> > dprintk("RPC: %s: ib_query_device failed %d\n",
> > __func__, rc);
> > - goto out2;
> > + goto out3;
> > }
> >
> > if (devattr.device_cap_flags & IB_DEVICE_LOCAL_DMA_LKEY) { @@
> > -602,14 +602,14 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct
> sockaddr *addr, int memreg)
> > "phys register failed with %lX\n",
> > __func__, PTR_ERR(ia->ri_bind_mem));
> > rc = -ENOMEM;
> > - goto out2;
> > + goto out3;
> > }
> > break;
> > default:
> > printk(KERN_ERR "RPC: Unsupported memory "
> > "registration mode: %d\n", memreg);
> > rc = -ENOMEM;
> > - goto out2;
> > + goto out3;
> > }
> > dprintk("RPC: %s: memory registration strategy is %d\n",
> > __func__, memreg);
> > @@ -619,6 +619,9 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct
> > sockaddr *addr, int memreg)
> >
> > rwlock_init(&ia->ri_qplock);
> > return 0;
> > +out3:
> > + ib_dealloc_pd(ia->ri_pd);
> > + ia->ri_pd = NULL;
> > out2:
> > module_put(ia->ri_id->device->owner);
> > rdma_destroy_id(ia->ri_id);
> > @@ -643,19 +646,20 @@ rpcrdma_ia_close(struct rpcrdma_ia *ia)
> > dprintk("RPC: %s: ib_dereg_mr returned %i\n",
> > __func__, rc);
> > }
> > +
> > if (ia->ri_id != NULL && !IS_ERR(ia->ri_id)) {
> > if (ia->ri_id->qp)
> > rdma_destroy_qp(ia->ri_id);
> > +
> > + if (ia->ri_pd != NULL && !IS_ERR(ia->ri_pd)) {
> > + rc = ib_dealloc_pd(ia->ri_pd);
> > + dprintk("RPC: %s: ib_dealloc_pd returned %i\n",
> > + __func__, rc);
> > + }
> > module_put(ia->ri_id->device->owner);
> > rdma_destroy_id(ia->ri_id);
> > ia->ri_id = NULL;
> > }
> > -
> > - if (ia->ri_pd != NULL && !IS_ERR(ia->ri_pd)) {
> > - rc = ib_dealloc_pd(ia->ri_pd);
> > - dprintk("RPC: %s: ib_dealloc_pd returned %i\n",
> > - __func__, rc);
> > - }
> > }
> >
> > /*
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
Replying again, due to mail formatting issue:
> -Original Message-
> From: Chuck Lever [mailto:chuck.le...@oracle.com]
> Sent: Thursday, July 17, 2014 8:36 PM
> To: Steve Wise
> Cc: Devesh Sharma; Roland Dreier; Hefty, Sean; linux-rdma@vger.kernel.org
> Subject: Re: [for-next 1/2] xprtrdma: take reference of rdma provider
> module
>
>
> On Jul 17, 2014, at 11:01 AM, Steve Wise
> wrote:
>
> >
> >
> > On 7/17/2014 9:01 AM, Devesh Sharma wrote:
> >> If verndor driver is attempted for removal while xprtrdma still has
> >> an active mount, the removal of driver may never complete and can
> >> cause unseen races or in worst case system crash.
> >>
> >> To solve this, xprtrdma module should get reference of struct
> >> ib_device structure for every mount. Reference is taken after local
> >> device address resolution is completed successfuly.
> >>
> >> reference to the struct ib_device pointer is put just before cm_id
> destruction.
> >>
> >> Signed-off-by: Devesh Sharma
> >
> > This seems like an issue with the rdma-cm or rdma core, not xprtrdma.
>
> Good point. I was wondering if svcrdma might have a similar issue.
>
> Infrequently I see a hang during shutdown that appears to be related to the
> ordering of removal of one of the RDMA modules, but I've never had time to
> chase it.
>
> See also: https://bugzilla.linux-nfs.org/show_bug.cgi?id=252
>
>
> > I see that user rdma applications cause a ref on the provider module here
> in ib_uverbs_open():
> >
> >if (!try_module_get(dev->ib_dev->owner)) {
> >ret = -ENODEV;
> >goto err;
> >
> >
> > Maybe kernel applications that allocate device resources should cause a ref
> on the provider's module.
Yes, To me it looks like the right place to get reference of provider driver is
rdma_resolve_addr()-->acuire_ib_dev(). However this patch provides a workaround.
Let's wait for Roland/Sean's input.
> >
> > Sean/Roland, is there some history here as to how rdma provider module
> removal should be handled?
> >
> > Steve.
>
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [for-next 1/2] xprtrdma: take reference of rdma provider module
From: Steve Wise [sw...@opengridcomputing.com]
Sent: Thursday, July 17, 2014 8:31 PM
To: Devesh Sharma; Roland Dreier; Hefty, Sean
Cc: linux-rdma@vger.kernel.org; chuck.le...@oracle.com
Subject: Re: [for-next 1/2] xprtrdma: take reference of rdma provider module
On 7/17/2014 9:01 AM, Devesh Sharma wrote:
> If verndor driver is attempted for removal while xprtrdma still has an
> active mount, the removal of driver may never complete and can cause
> unseen races or in worst case system crash.
>
> To solve this, xprtrdma module should get reference of struct ib_device
> structure for every mount. Reference is taken after local device address
> resolution is completed successfuly.
>
> reference to the struct ib_device pointer is put just before cm_id
> destruction.
>
> Signed-off-by: Devesh Sharma
This seems like an issue with the rdma-cm or rdma core, not xprtrdma. I
see that user rdma applications cause a ref on the provider module here
in ib_uverbs_open():
if (!try_module_get(dev->ib_dev->owner)) {
ret = -ENODEV;
goto err;
Maybe kernel applications that allocate device resources should cause a
ref on the provider's module.
Yes, To me it looks like the right place to get reference of provider driver is
rdma_resolve_addr()-->acuire_ib_dev(). However this patch provides a workaround.
Lets wait for Roland/Sean's input.
Sean/Roland, is there some history here as to how rdma provider module
removal should be handled?
Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [for-next 1/2] xprtrdma: take reference of rdma provider module
On Jul 17, 2014, at 11:01 AM, Steve Wise wrote:
>
>
> On 7/17/2014 9:01 AM, Devesh Sharma wrote:
>> If verndor driver is attempted for removal while xprtrdma still has an
>> active mount, the removal of driver may never complete and can cause
>> unseen races or in worst case system crash.
>>
>> To solve this, xprtrdma module should get reference of struct ib_device
>> structure for every mount. Reference is taken after local device address
>> resolution is completed successfuly.
>>
>> reference to the struct ib_device pointer is put just before cm_id
>> destruction.
>>
>> Signed-off-by: Devesh Sharma
>
> This seems like an issue with the rdma-cm or rdma core, not xprtrdma.
Good point. I was wondering if svcrdma might have a similar issue.
Infrequently I see a hang during shutdown that appears to be related to
the ordering of removal of one of the RDMA modules, but I’ve never had
time to chase it.
See also: https://bugzilla.linux-nfs.org/show_bug.cgi?id=252
> I see that user rdma applications cause a ref on the provider module here in
> ib_uverbs_open():
>
>if (!try_module_get(dev->ib_dev->owner)) {
>ret = -ENODEV;
>goto err;
>
>
> Maybe kernel applications that allocate device resources should cause a ref
> on the provider's module.
>
> Sean/Roland, is there some history here as to how rdma provider module
> removal should be handled?
>
> Steve.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [for-next 2/2] xprtrdma: fix deallocation sequence of pd
On 7/17/2014 9:01 AM, Devesh Sharma wrote:
xprtrdma tries to destroy pd after destruction of cm_id. However,
pd should be deallocated before destruction of cm_id.
Why?
I think you really mean that the pd dealloc needs to be done before the
module deref that you added in the first patch. But let us see what
Roland says about how device removal is supposed to be handled when
kernel applications are using the device...
Signed-off-by: Devesh Sharma
---
net/sunrpc/xprtrdma/verbs.c | 22 +-
1 files changed, 13 insertions(+), 9 deletions(-)
diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
index b00e55e..096e94b 100644
--- a/net/sunrpc/xprtrdma/verbs.c
+++ b/net/sunrpc/xprtrdma/verbs.c
@@ -544,7 +544,7 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr
*addr, int memreg)
if (rc) {
dprintk("RPC: %s: ib_query_device failed %d\n",
__func__, rc);
- goto out2;
+ goto out3;
}
if (devattr.device_cap_flags & IB_DEVICE_LOCAL_DMA_LKEY) {
@@ -602,14 +602,14 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct
sockaddr *addr, int memreg)
"phys register failed with %lX\n",
__func__, PTR_ERR(ia->ri_bind_mem));
rc = -ENOMEM;
- goto out2;
+ goto out3;
}
break;
default:
printk(KERN_ERR "RPC: Unsupported memory "
"registration mode: %d\n", memreg);
rc = -ENOMEM;
- goto out2;
+ goto out3;
}
dprintk("RPC: %s: memory registration strategy is %d\n",
__func__, memreg);
@@ -619,6 +619,9 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr
*addr, int memreg)
rwlock_init(&ia->ri_qplock);
return 0;
+out3:
+ ib_dealloc_pd(ia->ri_pd);
+ ia->ri_pd = NULL;
out2:
module_put(ia->ri_id->device->owner);
rdma_destroy_id(ia->ri_id);
@@ -643,19 +646,20 @@ rpcrdma_ia_close(struct rpcrdma_ia *ia)
dprintk("RPC: %s: ib_dereg_mr returned %i\n",
__func__, rc);
}
+
if (ia->ri_id != NULL && !IS_ERR(ia->ri_id)) {
if (ia->ri_id->qp)
rdma_destroy_qp(ia->ri_id);
+
+ if (ia->ri_pd != NULL && !IS_ERR(ia->ri_pd)) {
+ rc = ib_dealloc_pd(ia->ri_pd);
+ dprintk("RPC: %s: ib_dealloc_pd returned %i\n",
+ __func__, rc);
+ }
module_put(ia->ri_id->device->owner);
rdma_destroy_id(ia->ri_id);
ia->ri_id = NULL;
}
-
- if (ia->ri_pd != NULL && !IS_ERR(ia->ri_pd)) {
- rc = ib_dealloc_pd(ia->ri_pd);
- dprintk("RPC: %s: ib_dealloc_pd returned %i\n",
- __func__, rc);
- }
}
/*
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [for-next 1/2] xprtrdma: take reference of rdma provider module
On 7/17/2014 9:01 AM, Devesh Sharma wrote:
If verndor driver is attempted for removal while xprtrdma still has an
active mount, the removal of driver may never complete and can cause
unseen races or in worst case system crash.
To solve this, xprtrdma module should get reference of struct ib_device
structure for every mount. Reference is taken after local device address
resolution is completed successfuly.
reference to the struct ib_device pointer is put just before cm_id destruction.
Signed-off-by: Devesh Sharma
This seems like an issue with the rdma-cm or rdma core, not xprtrdma. I
see that user rdma applications cause a ref on the provider module here
in ib_uverbs_open():
if (!try_module_get(dev->ib_dev->owner)) {
ret = -ENODEV;
goto err;
Maybe kernel applications that allocate device resources should cause a
ref on the provider's module.
Sean/Roland, is there some history here as to how rdma provider module
removal should be handled?
Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[ANNOUNCE] opensm 3.3.18 release
There is a new 3.3.18 release of OpenSM.
Tarball is available in:
http://www.openfabrics.org/downloads/management/
(listed in http://www.openfabrics.org/downloads/management/latest.txt)
md5sum:
642c2a2fda102a7d11add773a89265af opensm-3.3.18.tar.gz
All component versions are from recent master branch. Full list of
changes is below.
Albert Chu (3):
osm_perfmgr.c: Remove unnecessary log message
osm_perfmgr.c: Add log message when perfmgr sweep is skipped
osm_perfmgr.c: Fix perfmgr sweep_state race
Alex Netes (19):
ib_types.h: Fix shadow declaration warnings
Change discovery order of switch data
Better handling of topology changes in the fabric
osm_port_info_rcv.c: Reread pkeys from SP0 if switch rebooted during a
sweep
Resend LFTs/VLArb/SL2VL MADs in case of error
osm_qos.c: Fix possible seg fault
osm_node_info_rcv.c: Fix multiple switch discovery during a sweep
ib_types.h: Rename ib_switch_info_set_state_change function
osm_sw_info_rcv.c: Fix sending PortInfo during lightsweep
osm_sa_path_record.c: Search for requester port should be under lock
Fix crash during handover
osm_sm_state_mgr.c: Reconfigure the fabric when receiving
HANDOVER/POLLING_TIMEOUT in MASTER state
Revert "Reset client reregistration when receiving handover"
osm_state_mgr.c: Remove redundant unset to first_time_master_sweep
osm_sminfo_rcv.c: Send trap 144 to a newly found MASTER SM when in MASTER
state
osm_node_info_rcv.c: Update NodeInfo.SysImageGUID on heavy sweep
All SA queries should validate the requester port under lock
osm_sa_service_record.c: Improved locking
Add osm_congestion_control.c to per module logging support
Dan Ben Yosef (3):
osm_subnet.c: Fix resource leak guid2mkey parser (guid validation
function)
osm_subnet.c: Fix resource leak neighbor parser ("subn_validate_neighbor"
function)
osmtest: Add GRH tests for SA queries
Daniel Klein (4):
osm_node_info_rcv.c: Update local copy of node info for known nodes
osm_sa_mcmember_record.c: Fix removing members from existing mc group due
to invalid requests
Fix the creation of empty multicast groups from SADB
opensm.init.in: Fix return value checking in opensm.init script
Eli Dorfman (1):
osm_state_mgr.c: Clear first time sweep even after subnet error
Hal Rosenstock (46):
osm_port.c: Improve ERR 4108 log message
Handle bad SMP status
osm_ucast_mgr.c: Use LFT block of all port 0s to indicate resend
osm_state_mgr.c: Revert commit to "remove redundant unset to
first_time_master_sweep"
osm_sa_path_record.c: In osm_pr_rcv_process, release lock before log
message
man/opensm.8.in: Minor fixes to per module logging configuration
osm_sa_mcmember_record.c: Fix double locking in mcmr_rcv_join_mgrp
osm_sa_service_record.c: Fix locking issue in osm_sr_rcv_process
doc/performance-manager-HOWTO.txt: Fix typo
osm_sa_mad_ctrl.c: In sa_mad_ctrl_rcv_callback, improve 1A04 error log
message
osm_mcast_mgr.c: Fix endian of port GUID in ERR 0A06 log message
osm_mcast_mgr.c: Add missing new line at end of ERR 0A21 log message
osm_perfmgr.c: Add current PerfMgr sweep state to 54FF error log message
libvendor/osm_vendor_ibumad.c: Cosmetic change to umad_set_grh calls
libvendor/osm_vendor_ibumad.c: Support GRH (for GS classes)
osm_guid_info_rcv.c: Fix assert placement in osm_gi_rcv_process
osm_perfmgr.c: Cosmetic formatting changes
osm_perfmgr.c: Eliminate unneeded initialization in pc_recv_process
osm_perfmgr.c: Added Mellanox copyright
osm_perfmgr_db.c: Add missing clear of new xmit_wait counter in
clear_counters
Add support for additional Mellanox OUI
osm_subnet.c: Enhance opensm conf file documentation for PerfMgr options
osm_subnet.c: More cosmetic changes to opensm conf file PerfMgr
documentation
libvendor/osm_vendor_ibumad.c: Commentary change
doc/opensm_release_notes-3.3.txt: Updated repo location
osm_perfmgr.c: Minor code factoring in perfmgr_send_mad
osm_perfmgr.c: In perfmgr_send_mad, handle cl_event_wait_on return status
PerfMgr: Eliminate no longer used sig_sweep variable
osm_[sm sa]_mad_ctrl.c: Improve unsupported attribute error messages
osm_congestion_control.c: In cc_poller_send, handle cl_event_wait_on
return status
osm_mcast_mgr.c: In mcast_mgr_process_mlid, cosmetic change to log message
osm_trap_rcv.c: In shutup_noisy_port, improve ERR 3811 log message
osm_sm.c: In sm_sweeper, no need to check for timeout after
cl_event_wait_on EVENT_NO_TIMEOUT
osm_congestion_control.c: Fix error code
osm_vendor_ibumad.c: Better match table eviction strategy
osmtest/main.c: Output formatting change for case 'x'
osm_sa_path_record.c: Fix some commentary typos
osm_subnet.c: Cosmetic f
[for-next 2/2] xprtrdma: fix deallocation sequence of pd
xprtrdma tries to destroy pd after destruction of cm_id. However,
pd should be deallocated before destruction of cm_id.
Signed-off-by: Devesh Sharma
---
net/sunrpc/xprtrdma/verbs.c | 22 +-
1 files changed, 13 insertions(+), 9 deletions(-)
diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
index b00e55e..096e94b 100644
--- a/net/sunrpc/xprtrdma/verbs.c
+++ b/net/sunrpc/xprtrdma/verbs.c
@@ -544,7 +544,7 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr
*addr, int memreg)
if (rc) {
dprintk("RPC: %s: ib_query_device failed %d\n",
__func__, rc);
- goto out2;
+ goto out3;
}
if (devattr.device_cap_flags & IB_DEVICE_LOCAL_DMA_LKEY) {
@@ -602,14 +602,14 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct
sockaddr *addr, int memreg)
"phys register failed with %lX\n",
__func__, PTR_ERR(ia->ri_bind_mem));
rc = -ENOMEM;
- goto out2;
+ goto out3;
}
break;
default:
printk(KERN_ERR "RPC: Unsupported memory "
"registration mode: %d\n", memreg);
rc = -ENOMEM;
- goto out2;
+ goto out3;
}
dprintk("RPC: %s: memory registration strategy is %d\n",
__func__, memreg);
@@ -619,6 +619,9 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr
*addr, int memreg)
rwlock_init(&ia->ri_qplock);
return 0;
+out3:
+ ib_dealloc_pd(ia->ri_pd);
+ ia->ri_pd = NULL;
out2:
module_put(ia->ri_id->device->owner);
rdma_destroy_id(ia->ri_id);
@@ -643,19 +646,20 @@ rpcrdma_ia_close(struct rpcrdma_ia *ia)
dprintk("RPC: %s: ib_dereg_mr returned %i\n",
__func__, rc);
}
+
if (ia->ri_id != NULL && !IS_ERR(ia->ri_id)) {
if (ia->ri_id->qp)
rdma_destroy_qp(ia->ri_id);
+
+ if (ia->ri_pd != NULL && !IS_ERR(ia->ri_pd)) {
+ rc = ib_dealloc_pd(ia->ri_pd);
+ dprintk("RPC: %s: ib_dealloc_pd returned %i\n",
+ __func__, rc);
+ }
module_put(ia->ri_id->device->owner);
rdma_destroy_id(ia->ri_id);
ia->ri_id = NULL;
}
-
- if (ia->ri_pd != NULL && !IS_ERR(ia->ri_pd)) {
- rc = ib_dealloc_pd(ia->ri_pd);
- dprintk("RPC: %s: ib_dealloc_pd returned %i\n",
- __func__, rc);
- }
}
/*
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[for-next 1/2] xprtrdma: take reference of rdma provider module
If verndor driver is attempted for removal while xprtrdma still has an
active mount, the removal of driver may never complete and can cause
unseen races or in worst case system crash.
To solve this, xprtrdma module should get reference of struct ib_device
structure for every mount. Reference is taken after local device address
resolution is completed successfuly.
reference to the struct ib_device pointer is put just before cm_id destruction.
Signed-off-by: Devesh Sharma
---
net/sunrpc/xprtrdma/verbs.c | 17 +++--
net/sunrpc/xprtrdma/xprt_rdma.h |1 +
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
index 6ead5df..b00e55e 100644
--- a/net/sunrpc/xprtrdma/verbs.c
+++ b/net/sunrpc/xprtrdma/verbs.c
@@ -457,6 +457,11 @@ rpcrdma_create_id(struct rpcrdma_xprt *xprt,
}
wait_for_completion_interruptible_timeout(&ia->ri_done,
msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT) + 1);
+ if (!ia->ri_async_rc && !try_module_get(id->device->owner)) {
+ dprintk("RPC: %s: Failed to get device module\n",
+ __func__);
+ ia->ri_async_rc = -ENODEV;
+ }
rc = ia->ri_async_rc;
if (rc)
goto out;
@@ -466,16 +471,18 @@ rpcrdma_create_id(struct rpcrdma_xprt *xprt,
if (rc) {
dprintk("RPC: %s: rdma_resolve_route() failed %i\n",
__func__, rc);
- goto out;
+ goto out_put;
}
wait_for_completion_interruptible_timeout(&ia->ri_done,
msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT) + 1);
rc = ia->ri_async_rc;
if (rc)
- goto out;
+ goto out_put;
return id;
+out_put:
+ module_put(id->device->owner);
out:
rdma_destroy_id(id);
return ERR_PTR(rc);
@@ -613,6 +620,7 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr
*addr, int memreg)
rwlock_init(&ia->ri_qplock);
return 0;
out2:
+ module_put(ia->ri_id->device->owner);
rdma_destroy_id(ia->ri_id);
ia->ri_id = NULL;
out1:
@@ -638,9 +646,11 @@ rpcrdma_ia_close(struct rpcrdma_ia *ia)
if (ia->ri_id != NULL && !IS_ERR(ia->ri_id)) {
if (ia->ri_id->qp)
rdma_destroy_qp(ia->ri_id);
+ module_put(ia->ri_id->device->owner);
rdma_destroy_id(ia->ri_id);
ia->ri_id = NULL;
}
+
if (ia->ri_pd != NULL && !IS_ERR(ia->ri_pd)) {
rc = ib_dealloc_pd(ia->ri_pd);
dprintk("RPC: %s: ib_dealloc_pd returned %i\n",
@@ -886,6 +896,7 @@ retry:
if (ia->ri_id->device != id->device) {
printk("RPC: %s: can't reconnect on "
"different device!\n", __func__);
+ module_put(id->device->owner);
rdma_destroy_id(id);
rc = -ENETUNREACH;
goto out;
@@ -895,6 +906,7 @@ retry:
if (rc) {
dprintk("RPC: %s: rdma_create_qp failed %i\n",
__func__, rc);
+ module_put(id->device->owner);
rdma_destroy_id(id);
rc = -ENETUNREACH;
goto out;
@@ -906,6 +918,7 @@ retry:
write_unlock(&ia->ri_qplock);
rdma_destroy_qp(old);
+ module_put(old->device->owner);
rdma_destroy_id(old);
} else {
dprintk("RPC: %s: connecting...\n", __func__);
diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
index c419498..b35fa21 100644
--- a/net/sunrpc/xprtrdma/xprt_rdma.h
+++ b/net/sunrpc/xprtrdma/xprt_rdma.h
@@ -44,6 +44,7 @@
#include /* spinlock_t, etc */
#include /* atomic_t, etc */
#include/* struct work_struct */
+#include /* try_module_get()/module_put() */
#include /* RDMA connection api */
#include /* RDMA verbs api */
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[for-next 0/2] NFS/RDMA client fixes for 3.17
This patch series is test with 3.16-rc5 on top of chuck's series "[PATCH v3 00/21] NFS/RDMA client patches for 3.17" This series fixes hot removal of rdma provider driver and pd_deallocation sequence. Devesh Sharma (2): xprtrdma: take reference of rdma provider module xprtrdma: fix deallocation sequence of pd net/sunrpc/xprtrdma/verbs.c | 37 +++-- net/sunrpc/xprtrdma/xprt_rdma.h |1 + 2 files changed, 28 insertions(+), 10 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v3 00/21] NFS/RDMA client patches for 3.17
Yes, kindly do it. However, I have tested this only with ocrdma -Regards Devesh > -Original Message- > From: Chuck Lever [mailto:chuck.le...@oracle.com] > Sent: Thursday, July 17, 2014 7:46 PM > To: Devesh Sharma > Cc: linux-rdma; Linux NFS Mailing List > Subject: Re: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 > > > On Jul 17, 2014, at 10:12 AM, Devesh Sharma > wrote: > > > Hi Chuck, > > > > Tested the cable pull also. V3 is passing the cable pull test also. I have > > tried > following tests: > > > > Run iozone on nfs-rdma mount. > > Bring down the link from switch (to simulate cable pull). > > Wait for 10 secs. > > Bring back the link. > > This test passes, iozone resumes traffic. > > > > Run iozone on nfs-rdma mount. > > Bring down the link from switch (to simulate cable pull). > > Wait for 70 secs. > > Bring back the link. > > This test passes, iozone resumes traffic. > > Thanks Devesh! > > May I add "Tested-by: Devesh Sharma " ? > > > > >> -Original Message- > >> From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > >> ow...@vger.kernel.org] On Behalf Of Devesh Sharma > >> Sent: Thursday, July 17, 2014 10:37 AM > >> To: Chuck Lever; linux-rdma; Linux NFS Mailing List > >> Subject: RE: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 > >> > >> Hi Chuck, > >> > >> > >> Tested v3 with ocrdma (linux-3.16-rc5 inbox`ed ocrdma). Both Cthon > >> and iozone passes with and regressions. I will perform cable pull > >> test as well and get back to you. > >> > >> -Regards > >> Devesh > >> > >>> -Original Message- > >>> From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > >>> ow...@vger.kernel.org] On Behalf Of Chuck Lever > >>> Sent: Tuesday, July 15, 2014 7:54 PM > >>> To: linux-rdma; Linux NFS Mailing List > >>> Subject: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 > >>> > >>> The main purpose of this series is to address connection drop > >>> recovery issues by fixing FRMR re-use to make it less likely the > >>> client will deadlock due to a memory management operation error. > >>> > >>> Some clean-ups and other fixes are present as well. > >>> > >>> See topic branch nfs-rdma-for-3.17 in > >>> > >>> git://git.linux-nfs.org/projects/cel/cel-2.6.git > >>> > >>> I tested with NFSv3 and NFSv4 on all three supported memory > >>> registration modes. Used cthon04, iozone, and dbench with both > >>> Solaris and Linux NFS/RDMA servers. Used xfstests with Linux. > >>> > >>> v3: > >>> Only two substantive changes: > >>> > >>> - Patch 08/21 now uses generic IB helpers for managing FRMR rkeys > >>> > >>> - Add Tested-by: from Steve Wise > >>> > >>> > >>> v2: > >>> Many patches from v1 have been written or replaced. > >>> > >>> The MW ref counting approach in v1 is abandoned. Instead, I've > >>> eliminated signaling FAST_REG_MR and LOCAL_INV, and added > >> appropriate > >>> recovery mechanisms after a transport reconnect that should prevent > >>> rkey dis- synchrony entirely. > >>> > >>> A couple of optimizations have been added, including: > >>> > >>> - Allocating each MW separately rather than carving each out of a > >>> large piece of contiguous memory > >>> > >>> - Now that the receive CQ upcall handler dequeues a bundle of CQEs > >>> at once, fire off the reply handler tasklet just once per upcall to > >>> reduce context switches and how often hard IRQs are disabled > >>> > >>> Jury is still out on the latter. > >>> > >>> -- > >>> Chuck Lever > >>> chuck[dot]lever[at]oracle[dot]com > >>> > >>> > >>> > >>> -- > >>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" > >>> in the body of a message to majord...@vger.kernel.org More > >> majordomo > >>> info at http://vger.kernel.org/majordomo-info.html > >> -- > >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" > >> in the body of a message to majord...@vger.kernel.org More > majordomo > >> info at http://vger.kernel.org/majordomo-info.html > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-nfs" > > in the body of a message to majord...@vger.kernel.org More > majordomo > > info at http://vger.kernel.org/majordomo-info.html > > -- > Chuck Lever > chuck[dot]lever[at]oracle[dot]com > > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 00/21] NFS/RDMA client patches for 3.17
On Jul 17, 2014, at 10:12 AM, Devesh Sharma wrote: > Hi Chuck, > > Tested the cable pull also. V3 is passing the cable pull test also. I have > tried following tests: > > Run iozone on nfs-rdma mount. > Bring down the link from switch (to simulate cable pull). > Wait for 10 secs. > Bring back the link. > This test passes, iozone resumes traffic. > > Run iozone on nfs-rdma mount. > Bring down the link from switch (to simulate cable pull). > Wait for 70 secs. > Bring back the link. > This test passes, iozone resumes traffic. Thanks Devesh! May I add "Tested-by: Devesh Sharma ” ? > >> -Original Message- >> From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- >> ow...@vger.kernel.org] On Behalf Of Devesh Sharma >> Sent: Thursday, July 17, 2014 10:37 AM >> To: Chuck Lever; linux-rdma; Linux NFS Mailing List >> Subject: RE: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 >> >> Hi Chuck, >> >> >> Tested v3 with ocrdma (linux-3.16-rc5 inbox`ed ocrdma). Both Cthon and >> iozone passes with and regressions. I will perform cable pull test as well >> and >> get back to you. >> >> -Regards >> Devesh >> >>> -Original Message- >>> From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- >>> ow...@vger.kernel.org] On Behalf Of Chuck Lever >>> Sent: Tuesday, July 15, 2014 7:54 PM >>> To: linux-rdma; Linux NFS Mailing List >>> Subject: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 >>> >>> The main purpose of this series is to address connection drop recovery >>> issues by fixing FRMR re-use to make it less likely the client will >>> deadlock due to a memory management operation error. >>> >>> Some clean-ups and other fixes are present as well. >>> >>> See topic branch nfs-rdma-for-3.17 in >>> >>> git://git.linux-nfs.org/projects/cel/cel-2.6.git >>> >>> I tested with NFSv3 and NFSv4 on all three supported memory >>> registration modes. Used cthon04, iozone, and dbench with both Solaris >>> and Linux NFS/RDMA servers. Used xfstests with Linux. >>> >>> v3: >>> Only two substantive changes: >>> >>> - Patch 08/21 now uses generic IB helpers for managing FRMR >>> rkeys >>> >>> - Add Tested-by: from Steve Wise >>> >>> >>> v2: >>> Many patches from v1 have been written or replaced. >>> >>> The MW ref counting approach in v1 is abandoned. Instead, I've >>> eliminated signaling FAST_REG_MR and LOCAL_INV, and added >> appropriate >>> recovery mechanisms after a transport reconnect that should prevent >>> rkey dis- synchrony entirely. >>> >>> A couple of optimizations have been added, including: >>> >>> - Allocating each MW separately rather than carving each out of a >>> large piece of contiguous memory >>> >>> - Now that the receive CQ upcall handler dequeues a bundle of CQEs >>> at once, fire off the reply handler tasklet just once per upcall >>> to reduce context switches and how often hard IRQs are disabled >>> >>> Jury is still out on the latter. >>> >>> -- >>> Chuck Lever >>> chuck[dot]lever[at]oracle[dot]com >>> >>> >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" >>> in the body of a message to majord...@vger.kernel.org More >> majordomo >>> info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the >> body of a message to majord...@vger.kernel.org More majordomo info at >> http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Chuck Lever chuck[dot]lever[at]oracle[dot]com -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v3 00/21] NFS/RDMA client patches for 3.17
Hi Chuck, Tested the cable pull also. V3 is passing the cable pull test also. I have tried following tests: Run iozone on nfs-rdma mount. Bring down the link from switch (to simulate cable pull). Wait for 10 secs. Bring back the link. This test passes, iozone resumes traffic. Run iozone on nfs-rdma mount. Bring down the link from switch (to simulate cable pull). Wait for 70 secs. Bring back the link. This test passes, iozone resumes traffic. > -Original Message- > From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > ow...@vger.kernel.org] On Behalf Of Devesh Sharma > Sent: Thursday, July 17, 2014 10:37 AM > To: Chuck Lever; linux-rdma; Linux NFS Mailing List > Subject: RE: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 > > Hi Chuck, > > > Tested v3 with ocrdma (linux-3.16-rc5 inbox`ed ocrdma). Both Cthon and > iozone passes with and regressions. I will perform cable pull test as well and > get back to you. > > -Regards > Devesh > > > -Original Message- > > From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > > ow...@vger.kernel.org] On Behalf Of Chuck Lever > > Sent: Tuesday, July 15, 2014 7:54 PM > > To: linux-rdma; Linux NFS Mailing List > > Subject: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 > > > > The main purpose of this series is to address connection drop recovery > > issues by fixing FRMR re-use to make it less likely the client will > > deadlock due to a memory management operation error. > > > > Some clean-ups and other fixes are present as well. > > > > See topic branch nfs-rdma-for-3.17 in > > > > git://git.linux-nfs.org/projects/cel/cel-2.6.git > > > > I tested with NFSv3 and NFSv4 on all three supported memory > > registration modes. Used cthon04, iozone, and dbench with both Solaris > > and Linux NFS/RDMA servers. Used xfstests with Linux. > > > > v3: > > Only two substantive changes: > > > > - Patch 08/21 now uses generic IB helpers for managing FRMR > > rkeys > > > > - Add Tested-by: from Steve Wise > > > > > > v2: > > Many patches from v1 have been written or replaced. > > > > The MW ref counting approach in v1 is abandoned. Instead, I've > > eliminated signaling FAST_REG_MR and LOCAL_INV, and added > appropriate > > recovery mechanisms after a transport reconnect that should prevent > > rkey dis- synchrony entirely. > > > > A couple of optimizations have been added, including: > > > > - Allocating each MW separately rather than carving each out of a > > large piece of contiguous memory > > > > - Now that the receive CQ upcall handler dequeues a bundle of CQEs > > at once, fire off the reply handler tasklet just once per upcall > > to reduce context switches and how often hard IRQs are disabled > > > > Jury is still out on the latter. > > > > -- > > Chuck Lever > > chuck[dot]lever[at]oracle[dot]com > > > > > > > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-rdma" > > in the body of a message to majord...@vger.kernel.org More > majordomo > > info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the > body of a message to majord...@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] uapi/rdma_user_cm.h: include socket.h
Commit ee7aed4528f (RDMA/ucma: Support querying for AF_IB addresses) added struct sockaddr_storage to rdma_user_cm.h without also adding an include for linux/socket.h to make sure it is defined. As systemtap needs the header files to build standalone and not rely on the code files to pre-include other headers, add linux/socket.h to the list of includes in this file. This patch resolves Red Hat bugzilla #1072645. The above referenced commit was added during the 3.11 release cycle, so this fix is suitable for any stable kernels from that point on. Signed-off-by: Doug Ledford --- include/uapi/rdma/rdma_user_cm.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/rdma/rdma_user_cm.h b/include/uapi/rdma/rdma_user_cm.h index 99b80abf360..3066718eb12 100644 --- a/include/uapi/rdma/rdma_user_cm.h +++ b/include/uapi/rdma/rdma_user_cm.h @@ -34,6 +34,7 @@ #define RDMA_USER_CM_H #include +#include #include #include #include -- 1.9.3 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch] RDMA/amso1100: integer overflow in c2_alloc_cq_buf()
This is a static checker fix. The static checker says that q_size comes
from the user and can be any 32 bit value. The call tree is:
--> ib_uverbs_create_cq()
--> c2_create_cq()
--> c2_init_cq()
Signed-off-by: Dan Carpenter
diff --git a/drivers/infiniband/hw/amso1100/c2_cq.c
b/drivers/infiniband/hw/amso1100/c2_cq.c
index 49e0e85..1b63185 100644
--- a/drivers/infiniband/hw/amso1100/c2_cq.c
+++ b/drivers/infiniband/hw/amso1100/c2_cq.c
@@ -260,11 +260,14 @@ static void c2_free_cq_buf(struct c2_dev *c2dev, struct
c2_mq *mq)
mq->msg_pool.host, dma_unmap_addr(mq, mapping));
}
-static int c2_alloc_cq_buf(struct c2_dev *c2dev, struct c2_mq *mq, int q_size,
- int msg_size)
+static int c2_alloc_cq_buf(struct c2_dev *c2dev, struct c2_mq *mq,
+ size_t q_size, size_t msg_size)
{
u8 *pool_start;
+ if (q_size > SIZE_MAX / msg_size)
+ return -EINVAL;
+
pool_start = dma_alloc_coherent(&c2dev->pcidev->dev, q_size * msg_size,
&mq->host_dma, GFP_KERNEL);
if (!pool_start)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH net-next v2 4/4] cxgb4i: Add ipv6 code to driver, call into libcxgbi ipv6 api
Signed-off-by: Anish Bhatt
Signed-off-by: Karen Xie
Signed-off-by: Manoj Malviya
---
drivers/scsi/cxgbi/cxgb4i/cxgb4i.c | 353 +
1 file changed, 314 insertions(+), 39 deletions(-)
diff --git a/drivers/scsi/cxgbi/cxgb4i/cxgb4i.c
b/drivers/scsi/cxgbi/cxgb4i/cxgb4i.c
index e8ee5e5fe0ef..1041574edcfc 100644
--- a/drivers/scsi/cxgbi/cxgb4i/cxgb4i.c
+++ b/drivers/scsi/cxgbi/cxgb4i/cxgb4i.c
@@ -19,6 +19,7 @@
#include
#include
#include
+#include
#include "t4_regs.h"
#include "t4_msg.h"
@@ -150,6 +151,7 @@ static struct scsi_transport_template *cxgb4i_stt;
* The section below implments CPLs that related to iscsi tcp connection
* open/close/abort and data send/receive.
*/
+
#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
#define RCV_BUFSIZ_MASK0x3FFU
#define MAX_IMM_TX_PKT_LEN 128
@@ -179,6 +181,7 @@ static void send_act_open_req(struct cxgbi_sock *csk,
struct sk_buff *skb,
struct l2t_entry *e)
{
struct cxgb4_lld_info *lldi = cxgbi_cdev_priv(csk->cdev);
+ int t4 = is_t4(lldi->adapter_type);
int wscale = cxgbi_sock_compute_wscale(csk->mss_idx);
unsigned long long opt0;
unsigned int opt2;
@@ -248,6 +251,97 @@ static void send_act_open_req(struct cxgbi_sock *csk,
struct sk_buff *skb,
}
set_wr_txq(skb, CPL_PRIORITY_SETUP, csk->port_id);
+
+ pr_info_ipaddr("t%d csk 0x%p,%u,0x%lx,%u, rss_qid %u.\n",
+ (&csk->saddr), (&csk->daddr), t4 ? 4 : 5, csk,
+ csk->state, csk->flags, csk->atid, csk->rss_qid);
+
+ cxgb4_l2t_send(csk->cdev->ports[csk->port_id], skb, csk->l2t);
+}
+
+static void send_act_open_req6(struct cxgbi_sock *csk, struct sk_buff *skb,
+ struct l2t_entry *e)
+{
+ struct cxgb4_lld_info *lldi = cxgbi_cdev_priv(csk->cdev);
+ int t4 = is_t4(lldi->adapter_type);
+ int wscale = cxgbi_sock_compute_wscale(csk->mss_idx);
+ unsigned long long opt0;
+ unsigned int opt2;
+ unsigned int qid_atid = ((unsigned int)csk->atid) |
+(((unsigned int)csk->rss_qid) << 14);
+
+ opt0 = KEEP_ALIVE(1) |
+ WND_SCALE(wscale) |
+ MSS_IDX(csk->mss_idx) |
+ L2T_IDX(((struct l2t_entry *)csk->l2t)->idx) |
+ TX_CHAN(csk->tx_chan) |
+ SMAC_SEL(csk->smac_idx) |
+ ULP_MODE(ULP_MODE_ISCSI) |
+ RCV_BUFSIZ(cxgb4i_rcv_win >> 10);
+
+ opt2 = RX_CHANNEL(0) |
+ RSS_QUEUE_VALID |
+ RX_FC_DISABLE |
+ RSS_QUEUE(csk->rss_qid);
+
+ if (t4) {
+ struct cpl_act_open_req6 *req =
+ (struct cpl_act_open_req6 *)skb->head;
+
+ INIT_TP_WR(req, 0);
+ OPCODE_TID(req) = cpu_to_be32(MK_OPCODE_TID(CPL_ACT_OPEN_REQ6,
+ qid_atid));
+ req->local_port = csk->saddr6.sin6_port;
+ req->peer_port = csk->daddr6.sin6_port;
+
+ req->local_ip_hi = *(__be64 *)(csk->saddr6.sin6_addr.s6_addr);
+ req->local_ip_lo = *(__be64 *)(csk->saddr6.sin6_addr.s6_addr +
+ 8);
+ req->peer_ip_hi = *(__be64 *)(csk->daddr6.sin6_addr.s6_addr);
+ req->peer_ip_lo = *(__be64 *)(csk->daddr6.sin6_addr.s6_addr +
+ 8);
+
+ req->opt0 = cpu_to_be64(opt0);
+
+ opt2 |= RX_FC_VALID;
+ req->opt2 = cpu_to_be32(opt2);
+
+ req->params = cpu_to_be32(cxgb4_select_ntuple(
+ csk->cdev->ports[csk->port_id],
+ csk->l2t));
+ } else {
+ struct cpl_t5_act_open_req6 *req =
+ (struct cpl_t5_act_open_req6 *)skb->head;
+
+ INIT_TP_WR(req, 0);
+ OPCODE_TID(req) = cpu_to_be32(MK_OPCODE_TID(CPL_ACT_OPEN_REQ6,
+ qid_atid));
+ req->local_port = csk->saddr6.sin6_port;
+ req->peer_port = csk->daddr6.sin6_port;
+ req->local_ip_hi = *(__be64 *)(csk->saddr6.sin6_addr.s6_addr);
+ req->local_ip_lo = *(__be64 *)(csk->saddr6.sin6_addr.s6_addr +
+ 8);
+ req->peer_ip_hi = *(__be64 *)(csk->daddr6.sin6_addr.s6_addr);
+ req->peer_ip_lo = *(__be64 *)(csk->daddr6.sin6_addr.s6_addr +
+ 8);
+ req->opt0 = cpu_to_be64(opt0);
+
+ opt2 |= T5_OPT_2_VALID;
+ req->opt2 = cpu_to_be32(opt2);
+
+ req->params = cp
[PATCH net-next v2 0/4] cxgb4i: Add ipv6 support
Hi, The following patchset add ipv6 support for the cxgb4i(iscsi) driver. Patch 1 moves a define from the iw_cxgb4 to cxgb4 to prevent code duplication, as it is used by cxgb4i and iw_cxgb4 both. Patch 2 exports symbols needed by cxgb4i for ipv6 support from cxgb4. Patch 3 & 4 deal with actual ipv6 code in libcxgbi/cxgb4i. Submitting to net-next as changes were needed in cxgb4/iw_cxgb4, and submitting to scsi separately would break compilation, cc-ing scsi & rdma. -Anish v2 : Cleanup macro that prints ip addresses by using %pIS in patch 3 Anish Bhatt (4): cxgb4/iw_cxgb4: Move common defines to cxgb4 cxgb4: Export symbols required by cxgb4i for ipv6 support and required defines libcxgbi: Add ipv6 api to driver cxgb4i: Add ipv6 code to driver, call into libcxgbi ipv6 api drivers/infiniband/hw/cxgb4/t4fw_ri_api.h | 1 - drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 10 +- drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h | 4 + drivers/net/ethernet/chelsio/cxgb4/t4_msg.h | 3 + drivers/scsi/cxgbi/cxgb4i/cxgb4i.c | 353 +--- drivers/scsi/cxgbi/libcxgbi.c | 237 ++-- drivers/scsi/cxgbi/libcxgbi.h | 21 +- 7 files changed, 564 insertions(+), 65 deletions(-) -- 2.0.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH net-next v2 2/4] cxgb4: Export symbols required by cxgb4i for ipv6 support and required defines
Signed-off-by: Anish Bhatt
---
drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 10 ++
drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h | 4
drivers/net/ethernet/chelsio/cxgb4/t4_msg.h | 2 ++
3 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
index 9c7e4f0a7683..8b46534b06c1 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
@@ -3445,8 +3445,8 @@ static int tid_init(struct tid_info *t)
return 0;
}
-static int cxgb4_clip_get(const struct net_device *dev,
- const struct in6_addr *lip)
+int cxgb4_clip_get(const struct net_device *dev,
+ const struct in6_addr *lip)
{
struct adapter *adap;
struct fw_clip_cmd c;
@@ -3460,9 +3460,10 @@ static int cxgb4_clip_get(const struct net_device *dev,
c.ip_lo = *(__be64 *)(lip->s6_addr + 8);
return t4_wr_mbox_meat(adap, adap->mbox, &c, sizeof(c), &c, false);
}
+EXPORT_SYMBOL(cxgb4_clip_get);
-static int cxgb4_clip_release(const struct net_device *dev,
- const struct in6_addr *lip)
+int cxgb4_clip_release(const struct net_device *dev,
+ const struct in6_addr *lip)
{
struct adapter *adap;
struct fw_clip_cmd c;
@@ -3476,6 +3477,7 @@ static int cxgb4_clip_release(const struct net_device
*dev,
c.ip_lo = *(__be64 *)(lip->s6_addr + 8);
return t4_wr_mbox_meat(adap, adap->mbox, &c, sizeof(c), &c, false);
}
+EXPORT_SYMBOL(cxgb4_clip_release);
/**
* cxgb4_create_server - create an IP server
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
index 79a84de1d204..1366ba620c87 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
@@ -172,6 +172,10 @@ int cxgb4_create_server_filter(const struct net_device
*dev, unsigned int stid,
unsigned char port, unsigned char mask);
int cxgb4_remove_server_filter(const struct net_device *dev, unsigned int stid,
unsigned int queue, bool ipv6);
+int cxgb4_clip_get(const struct net_device *dev, const struct in6_addr *lip);
+int cxgb4_clip_release(const struct net_device *dev,
+ const struct in6_addr *lip);
+
static inline void set_wr_txq(struct sk_buff *skb, int prio, int queue)
{
skb_set_queue_mapping(skb, (queue << 1) | prio);
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
b/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
index 64006327df83..0259feeab1b3 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
@@ -270,6 +270,8 @@ struct cpl_pass_accept_rpl {
#define RX_COALESCE_VALID(x) ((x) << 11)
#define RX_COALESCE(x) ((x) << 12)
#define PACE(x) ((x) << 16)
+#define RX_FC_VALID ((1U) << 19)
+#define RX_FC_DISABLE ((1U) << 20)
#define TX_QUEUE(x) ((x) << 23)
#define RX_CHANNEL(x)((x) << 26)
#define CCTRL_ECN(x) ((x) << 27)
--
2.0.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH net-next v2 3/4] libcxgbi: Add ipv6 api to driver
Signed-off-by: Anish Bhatt
Signed-off-by: Karen Xie
Signed-off-by: Manoj Malviya
---
drivers/scsi/cxgbi/libcxgbi.c | 237 ++
drivers/scsi/cxgbi/libcxgbi.h | 21 +++-
2 files changed, 237 insertions(+), 21 deletions(-)
diff --git a/drivers/scsi/cxgbi/libcxgbi.c b/drivers/scsi/cxgbi/libcxgbi.c
index b44c1cff3114..d2fe507fc695 100644
--- a/drivers/scsi/cxgbi/libcxgbi.c
+++ b/drivers/scsi/cxgbi/libcxgbi.c
@@ -24,6 +24,10 @@
#include
#include
#include
+#include
+#include
+#include
+
#include /* ip_dev_find */
#include
#include
@@ -193,8 +197,8 @@ struct cxgbi_device *cxgbi_device_find_by_lldev(void *lldev)
}
EXPORT_SYMBOL_GPL(cxgbi_device_find_by_lldev);
-static struct cxgbi_device *cxgbi_device_find_by_netdev(struct net_device
*ndev,
- int *port)
+struct cxgbi_device *cxgbi_device_find_by_netdev(struct net_device *ndev,
+int *port)
{
struct net_device *vdev = NULL;
struct cxgbi_device *cdev, *tmp;
@@ -224,6 +228,40 @@ static struct cxgbi_device
*cxgbi_device_find_by_netdev(struct net_device *ndev,
"ndev 0x%p, %s, NO match found.\n", ndev, ndev->name);
return NULL;
}
+EXPORT_SYMBOL_GPL(cxgbi_device_find_by_netdev);
+
+static struct cxgbi_device *cxgbi_device_find_by_mac(struct net_device *ndev,
+int *port)
+{
+ struct net_device *vdev = NULL;
+ struct cxgbi_device *cdev, *tmp;
+ int i;
+
+ if (ndev->priv_flags & IFF_802_1Q_VLAN) {
+ vdev = ndev;
+ ndev = vlan_dev_real_dev(ndev);
+ pr_info("vlan dev %s -> %s.\n", vdev->name, ndev->name);
+ }
+
+ mutex_lock(&cdev_mutex);
+ list_for_each_entry_safe(cdev, tmp, &cdev_list, list_head) {
+ for (i = 0; i < cdev->nports; i++) {
+ if (!memcmp(ndev->dev_addr, cdev->ports[i]->dev_addr,
+ MAX_ADDR_LEN)) {
+ cdev->hbas[i]->vdev = vdev;
+ mutex_unlock(&cdev_mutex);
+ if (port)
+ *port = i;
+ return cdev;
+ }
+ }
+ }
+ mutex_unlock(&cdev_mutex);
+ log_debug(1 << CXGBI_DBG_DEV,
+ "ndev 0x%p, %s, NO match mac found.\n",
+ ndev, ndev->name);
+ return NULL;
+}
void cxgbi_hbas_remove(struct cxgbi_device *cdev)
{
@@ -320,6 +358,7 @@ static int sock_get_port(struct cxgbi_sock *csk)
struct cxgbi_ports_map *pmap = &cdev->pmap;
unsigned int start;
int idx;
+ __be16 *port;
if (!pmap->max_connect) {
pr_err("cdev 0x%p, p#%u %s, NO port map.\n",
@@ -327,9 +366,14 @@ static int sock_get_port(struct cxgbi_sock *csk)
return -EADDRNOTAVAIL;
}
- if (csk->saddr.sin_port) {
+ if (csk->csk_family == AF_INET)
+ port = &csk->saddr.sin_port;
+ else /* ipv6 */
+ port = &csk->saddr6.sin6_port;
+
+ if (*port) {
pr_err("source port NON-ZERO %u.\n",
- ntohs(csk->saddr.sin_port));
+ ntohs(*port));
return -EADDRINUSE;
}
@@ -347,8 +391,7 @@ static int sock_get_port(struct cxgbi_sock *csk)
idx = 0;
if (!pmap->port_csk[idx]) {
pmap->used++;
- csk->saddr.sin_port =
- htons(pmap->sport_base + idx);
+ *port = htons(pmap->sport_base + idx);
pmap->next = idx;
pmap->port_csk[idx] = csk;
spin_unlock_bh(&pmap->lock);
@@ -374,16 +417,22 @@ static void sock_put_port(struct cxgbi_sock *csk)
{
struct cxgbi_device *cdev = csk->cdev;
struct cxgbi_ports_map *pmap = &cdev->pmap;
+ __be16 *port;
- if (csk->saddr.sin_port) {
- int idx = ntohs(csk->saddr.sin_port) - pmap->sport_base;
+ if (csk->csk_family == AF_INET)
+ port = &csk->saddr.sin_port;
+ else /* ipv6 */
+ port = &csk->saddr6.sin6_port;
- csk->saddr.sin_port = 0;
+ if (*port) {
+ int idx = ntohs(*port) - pmap->sport_base;
+
+ *port = 0;
if (idx < 0 || idx >= pmap->max_connect) {
pr_err("cdev 0x%p, p#%u %s, port %u OOR.\n",
cdev, csk->port_id,
cdev->ports[csk->port_id]->name,
- ntohs(csk->saddr.sin_port));
+ ntohs(*port));
return;
}
@@ -479,17 +528,11 @@
[PATCH net-next v2 1/4] cxgb4/iw_cxgb4: Move common defines to cxgb4
This define is used by cxgb4i and iw_cxgb4, moving to avoid code duplication
Signed-off-by: Anish Bhatt
Acked-by: Steve Wise
---
drivers/infiniband/hw/cxgb4/t4fw_ri_api.h | 1 -
drivers/net/ethernet/chelsio/cxgb4/t4_msg.h | 1 +
2 files changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/cxgb4/t4fw_ri_api.h
b/drivers/infiniband/hw/cxgb4/t4fw_ri_api.h
index 91289a051af9..5709e77faf7c 100644
--- a/drivers/infiniband/hw/cxgb4/t4fw_ri_api.h
+++ b/drivers/infiniband/hw/cxgb4/t4fw_ri_api.h
@@ -849,6 +849,5 @@ enum { /* TCP congestion control
algorithms */
#define G_CONG_CNTRL(x) (((x) >> S_CONG_CNTRL) & M_CONG_CNTRL)
#define CONG_CNTRL_VALID (1 << 18)
-#define T5_OPT_2_VALID (1 << 31)
#endif /* _T4FW_RI_API_H_ */
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
b/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
index abb45809c0c8..64006327df83 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
@@ -276,6 +276,7 @@ struct cpl_pass_accept_rpl {
#define WND_SCALE_EN(x) ((x) << 28)
#define TSTAMPS_EN(x)((x) << 29)
#define SACK_EN(x) ((x) << 30)
+#define T5_OPT_2_VALID ((1U) << 31)
__be64 opt0;
};
--
2.0.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
