Re: [PATCH for-next V1 2/3] IB/core: RoCE GID management separate cleanup and release

2015-08-04 Thread Matan Barak
On Tue, Aug 4, 2015 at 6:10 AM, Jason Gunthorpe jguntho...@obsidianresearch.com wrote: On Mon, Aug 03, 2015 at 04:09:01PM +0300, Matan Barak wrote: The release function is called after the device was put. Although vendor drivers aren't expected to use IB cache in their removal process, we

[PATCH for-next V7 1/6] IB/uverbs: Fix reference counting usage of event files

2015-08-04 Thread Yishai Hadas
Fix the reference counting usage to be handled in the event file creation/destruction function, instead of being done by the caller. This is done for both async/non-async event files. Based on Jason Gunthorpe report at https://www.mail-archive.com/ linux-rdma@vger.kernel.org/msg24680.html: The

[PATCH for-next V7 3/6] IB/uverbs: Explicitly pass ib_dev to uverbs commands

2015-08-04 Thread Yishai Hadas
Done in preparation for deploying RCU for the device removal flow. Allows isolating the RCU handling to the uverb_main layer and keeping the uverbs_cmd code as is. Signed-off-by: Yishai Hadas yish...@mellanox.com Signed-off-by: Shachar Raindel rain...@mellanox.com ---

[PATCH for-next V7 0/6] HW Device hot-removal support

2015-08-04 Thread Yishai Hadas
Currently, if there is any user space application using an IB device, it is impossible to unload the HW device driver for this device. Similarly, if the device is hot-unplugged or reset, the device driver hardware removal flow blocks until all user contexts are destroyed. This patchset removes

[PATCH for-next V7 2/6] IB/uverbs: Fix race between ib_uverbs_open and remove_one

2015-08-04 Thread Yishai Hadas
Fixes: 2a72f212263701b927559f6850446421d5906c41 (IB/uverbs: Remove dev_table) Before this commit there was a device look-up table that was protected by a spin_lock used by ib_uverbs_open and by ib_uverbs_remove_one. When it was dropped and container_of was used instead, it enabled the race with

[PATCH for-next V7 4/6] IB/uverbs: Enable device removal when there are active user space applications

2015-08-04 Thread Yishai Hadas
Enables the uverbs_remove_one to succeed despite the fact that there are running IB applications working with the given ib device. This functionality enables a HW device to be unbind/reset despite the fact that there are running user space applications using it. It exposes a new IB kernel API

[PATCH for-next V7 5/6] IB/mlx4_ib: Disassociate support

2015-08-04 Thread Yishai Hadas
Implements the IB core disassociate_ucontext API. The driver detaches the HW resources for a given user context to prevent a dependency between application termination and device disconnecting. This is done by managing the VMAs that were mapped to the HW bars such as door bell and blueflame. When

[RFC] split struct ib_send_wr

2015-08-04 Thread Christoph Hellwig
Hi all, please take a look at my RFC patch here: http://git.infradead.org/users/hch/scsi.git/commitdiff/751774250b71da83a26ba8584cff70f5e7bb7b1e the commit contains my explanation, but apparently the patch is too large for the list limit and didn't make it through. -- To unsubscribe

Re: [PATCH v8 2/4] IB/core: Add rdma netlink helper functions

2015-08-04 Thread Jason Gunthorpe
On Tue, Aug 04, 2015 at 08:48:31PM -0400, ira.weiny wrote: We are using Netlink multicast. I believe that netlink_bind only allows root to bind to multicast. That is a good start... That said I have not tested the ability to change the timeout settings if one were to bind without multicast

Re: [PATCH v8 2/4] IB/core: Add rdma netlink helper functions

2015-08-04 Thread ira.weiny
On Mon, Aug 03, 2015 at 09:15:34PM -0600, Jason Gunthorpe wrote: On Thu, Jul 09, 2015 at 01:34:26PM -0400, kaike@intel.com wrote: From: Kaike Wan kaike@intel.com This patch adds a function to check if listeners for a netlink multicast group are present. It also adds a function to

Re: [PATCH for-next V7 2/6] IB/uverbs: Fix race between ib_uverbs_open and remove_one

2015-08-04 Thread Jason Gunthorpe
On Tue, Aug 04, 2015 at 05:03:24PM +0300, Yishai Hadas wrote: Signed-off-by: Yishai Hadas yish...@mellanox.com Signed-off-by: Shachar Raindel rain...@mellanox.com drivers/infiniband/core/uverbs.h |3 +- drivers/infiniband/core/uverbs_main.c | 43 +++--

Re: [PATCH 20/22] IB/iser: Support up to 8MB data transfer in a single command

2015-08-04 Thread Or Gerlitz
On Tue, Aug 4, 2015 at 8:10 PM, Sagi Grimberg sa...@dev.mellanox.co.il wrote: Why SIZE_4K and not PAGE_SIZE? Yes, I'll change that to PAGE_SIZE. Thanks. Would non-4KB pages (e.g. PPC 64KB) be an issue? Would this work between hosts with different page sizes? iser was always using 4K

Re: [PATCH for-next V7 6/6] IB/ucma: HW Device hot-removal support

2015-08-04 Thread Jason Gunthorpe
On Tue, Aug 04, 2015 at 05:03:28PM +0300, Yishai Hadas wrote: Currently, IB/cma remove_one flow blocks until all user descriptor managed by IB/ucma are released. This prevents hot-removal of IB devices. This patch allows IB/cma to remove devices regardless of user space activity. Upon getting

[PATCH] IB/ucma: Fix theoretical user triggered use-after-free

2015-08-04 Thread Jason Gunthorpe
Something like this: CPU A CPU B ucma_destroy_id() wait_for_completion() .. anything ucma_put_ctx() complete()

Re: [PATCH for-next V1 2/3] IB/core: RoCE GID management separate cleanup and release

2015-08-04 Thread Jason Gunthorpe
On Tue, Aug 04, 2015 at 09:55:03PM +0300, Matan Barak wrote: If it fails in ib_device_register_sysfs, the device release function isn't called and the device pointer isn't freed. Ugh, yes, the abuse in ib_dealloc_device needs to go too. Attached is a compile tested patch that fixes that up.

Re: [PATCH for-next V7 6/6] IB/ucma: HW Device hot-removal support

2015-08-04 Thread Jason Gunthorpe
On Tue, Aug 04, 2015 at 05:03:28PM +0300, Yishai Hadas wrote: Currently, IB/cma remove_one flow blocks until all user descriptor managed by IB/ucma are released. This prevents hot-removal of IB devices. This patch allows IB/cma to remove devices regardless of user space activity. Upon getting

Re: [RFC] split struct ib_send_wr

2015-08-04 Thread Christoph Hellwig
On Tue, Aug 04, 2015 at 04:07:42PM +, Hefty, Sean wrote: This looks like a reasonable start. It may help with feedback if you could just post the changes to ib_verbs.h. Not sure it's all that useful, but here we go: diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index

Re: [PATCH for-next V1 2/3] IB/core: RoCE GID management separate cleanup and release

2015-08-04 Thread Jason Gunthorpe
On Tue, Aug 04, 2015 at 03:09:39PM +0300, Matan Barak wrote: Correct, I'll change this comment to: The release function is called after the device was put. This is in order to avoid use-after-free errors if the vendor driver's teardown code uses IB cache. .. the vendor driver uses

Re: [RFC] split struct ib_send_wr

2015-08-04 Thread Bart Van Assche
On 08/04/2015 09:29 AM, Christoph Hellwig wrote: On Tue, Aug 04, 2015 at 04:07:42PM +, Hefty, Sean wrote: This looks like a reasonable start. It may help with feedback if you could just post the changes to ib_verbs.h. Not sure it's all that useful, but here we go: diff --git

RE: [PATCH] RDMA/amso1100: deprecate the amso1100 provider

2015-08-04 Thread Steve Wise
-Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Marciniszyn, Mike Sent: Tuesday, August 04, 2015 11:33 AM To: Doug Ledford; Steve Wise Cc: linux-rdma@vger.kernel.org; t...@opengridcomputing.com Subject: RE: [PATCH]

Re: [PATCH v2 00/12] IB: Replace safe uses for ib_get_dma_mr with pd-local_dma_lkey

2015-08-04 Thread Jason Gunthorpe
On Mon, Aug 03, 2015 at 11:33:51AM -0700, Bart Van Assche wrote: Bart, do you know what hardware this workaround is for? I hope the HW vendors can comment on this. Sorry but I'm not sure which HCA models and/or firmware versions do not support FMR mapping with a non-zero offset. Perhaps

RE: [RFC] split struct ib_send_wr

2015-08-04 Thread Hefty, Sean
please take a look at my RFC patch here: http://git.infradead.org/users/hch/scsi.git/commitdiff/751774250b71d a83a26ba8584cff70f5e7bb7b1e the commit contains my explanation, but apparently the patch is too large for the list limit and didn't make it through. This looks like a

[PATCH] IB/hfi1: Add Infiniband dependency to Kconfig

2015-08-04 Thread Mike Marciniszyn
From: Jubin John jubin.j...@intel.com The hfi1 driver depends on the infiniband core stack. Building a kernel without the infiniband core but with the hfi1 driver causes undefined reference errors to the ib_* functions. Reported by 0-day build:

[PATCH] RDMA/ipath: Do not build ipath driver without infiniband subsystem

2015-08-04 Thread dennis . dalessandro
From: Dennis Dalessandro dennis.dalessan...@intel.com Moving the ipath driver to staging now requires a guard on the INFINIBAND subsytem in order to prevent the driver from being built without infiniband enabled. This will lead to a broken build. Reviewed-by: Mike Marciniszyn

Re: [PATCH 20/22] IB/iser: Support up to 8MB data transfer in a single command

2015-08-04 Thread Sagi Grimberg
Why SIZE_4K and not PAGE_SIZE? Yes, I'll change that to PAGE_SIZE. Thanks. Would non-4KB pages (e.g. PPC 64KB) be an issue? Would this work between hosts with different page sizes? iser was always using 4K segments for reasons I don't perfectly understand. Maybe Or can comment on this?

Re: [PATCH, RFC] rdma: split struct ib_send_wr

2015-08-04 Thread Christoph Hellwig
On Tue, Aug 04, 2015 at 08:06:16PM +0300, Sagi Grimberg wrote: Question though, a ULP may want to keep a couple of WRs around instead of having each allocated in the stack and handled one by one. We need to provide it with a hint of what is the size it needs. Note that with the drastic shrink

Re: [PATCH, RFC] rdma: split struct ib_send_wr

2015-08-04 Thread Christoph Hellwig
On Tue, Aug 04, 2015 at 08:44:26PM +0300, Sagi Grimberg wrote: I do agree that the size on the stack is less of an issue now. What still can matter is handling each wr one by one vs. doing a collective post. But if structured correctly you can still do that with on-stack WRs. I can

RE: [PATCH] RDMA/amso1100: deprecate the amso1100 provider

2015-08-04 Thread Steve Wise
This might run into the same issue as in https://lists.01.org/pipermail/kbuild-all/2015-August/011216.html. Hey Mike, what is the issue exactly? The problem is when CONFIG_INFINIBAND=n and CONFIG_INFINIBAND_HFI1=y. Here is what I did for ipath which had the same issue:

Re: [PATCH, RFC] rdma: split struct ib_send_wr

2015-08-04 Thread Sagi Grimberg
On 8/4/2015 8:47 PM, Christoph Hellwig wrote: On Tue, Aug 04, 2015 at 08:44:26PM +0300, Sagi Grimberg wrote: I do agree that the size on the stack is less of an issue now. What still can matter is handling each wr one by one vs. doing a collective post. But if structured correctly you can

Re: [RFC] split struct ib_send_wr

2015-08-04 Thread Christoph Hellwig
On Tue, Aug 04, 2015 at 09:36:49AM -0700, Bart Van Assche wrote: diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h [ ... ] struct ib_recv_wr { +struct ib_send_wr wr; struct ib_recv_wr *next; u64 wr_id; struct ib_sge

Re: [PATCH] IB/core, cma: Make __attribute_const__ declarations sparse-friendly

2015-08-04 Thread Sagi Grimberg
On 8/3/2015 8:01 PM, Bart Van Assche wrote: Move the __attribute_const__ declarations such that sparse understands that these apply to the function itself and not to the return type. This avoids that sparse reports error messages like the following: drivers/infiniband/core/verbs.c:73:12: error:

Re: [PATCH, RFC] rdma: split struct ib_send_wr

2015-08-04 Thread Sagi Grimberg
On 8/4/2015 10:16 AM, Christoph Hellwig wrote: This patch split up struct ib_send_wr so that all non-trivial verbs use their own structure which embedds struct ib_send_wr. This dramaticly shrinks the size of a WR for most common operations. Hey Christoph, I think this looks good in general.

[PATCH] RDMA/amso1100: Do not build amso1100 without infiniband subsystem

2015-08-04 Thread Steve Wise
Moving the amso1100 driver to staging now requires a guard on the INFINIBAND subsytem in order to prevent the driver from being built without infiniband enabled. This will lead to a broken build. Signed-off-by: Steve Wise sw...@opengridcomputing.com --- drivers/staging/amso1100/Kconfig |2

RE: [PATCH] RDMA/amso1100: deprecate the amso1100 provider

2015-08-04 Thread Dalessandro, Dennis
-Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- ow...@vger.kernel.org] On Behalf Of Steve Wise Sent: Tuesday, August 4, 2015 1:13 PM To: Marciniszyn, Mike; 'Doug Ledford' Cc: linux-rdma@vger.kernel.org; t...@opengridcomputing.com Subject: RE: [PATCH]

RE: [PATCH] RDMA/amso1100: deprecate the amso1100 provider

2015-08-04 Thread Marciniszyn, Mike
Subject: Re: [PATCH] RDMA/amso1100: deprecate the amso1100 provider On 07/29/2015 10:44 AM, Steve Wise wrote: The HW hasn't been sold since 2005, and the SW has definite bit rot. Its time to remove it. So move it to staging for a few releases and then remove it after that.

Re: [PATCH, RFC] rdma: split struct ib_send_wr

2015-08-04 Thread Sagi Grimberg
On 8/4/2015 8:23 PM, Christoph Hellwig wrote: On Tue, Aug 04, 2015 at 08:06:16PM +0300, Sagi Grimberg wrote: Question though, a ULP may want to keep a couple of WRs around instead of having each allocated in the stack and handled one by one. We need to provide it with a hint of what is the size

RE: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-08-04 Thread Steve Wise
-Original Message- From: Sagi Grimberg [mailto:sa...@dev.mellanox.co.il] Sent: Tuesday, August 04, 2015 12:26 PM To: Steve Wise; dledf...@redhat.com Cc: infinip...@intel.com; sa...@mellanox.com; ogerl...@mellanox.com; r...@mellanox.com; linux-rdma@vger.kernel.org;

Re: [PATCH for-next V1 2/3] IB/core: RoCE GID management separate cleanup and release

2015-08-04 Thread Matan Barak
On Tue, Aug 4, 2015 at 7:46 PM, Jason Gunthorpe jguntho...@obsidianresearch.com wrote: On Tue, Aug 04, 2015 at 03:09:39PM +0300, Matan Barak wrote: Correct, I'll change this comment to: The release function is called after the device was put. This is in order to avoid use-after-free

[PATCH v3] IB/srp: Do not create an all physical insecure rkey by default

2015-08-04 Thread Jason Gunthorpe
The ULP only needs this if the insecure register_always performance optimization is enabled, or if FRWR/FMR is not supported in the driver. This is a WIP for this functionality, there are several WARN_ONs in this patch that can be hit under certain work loads. Additional patches will be needed to

Re: [PATCH for-next V7 4/6] IB/uverbs: Enable device removal when there are active user space applications

2015-08-04 Thread Jason Gunthorpe
On Tue, Aug 04, 2015 at 05:03:26PM +0300, Yishai Hadas wrote: Enables the uverbs_remove_one to succeed despite the fact that there are running IB applications working with the given ib device. This functionality enables a HW device to be unbind/reset despite the fact that there are running

Re: [PATCH v3 01/12] IB/core: Guarantee that a local_dma_lkey is available

2015-08-04 Thread ira.weiny
On Tue, Aug 04, 2015 at 04:21:30PM -0600, Jason Gunthorpe wrote: Every single ULP requires a local_dma_lkey to do anything with a QP, so let us ensure one exists for every PD created. If the driver can supply a global local_dma_lkey then use that, otherwise ask the driver to create a local