consignment
Contact us for your compensation consignment box -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
my subject
veenos and villa seafood pleace USA urgent recuritment going pls send update cv/resume at(veenosvilla2...@gmail.com) for immediate job in the U.S.A. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)
And yes this went through testing here and we want to run this as part of our prod kernels. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 12/26] xprtrdma: Port to new memory registration API
> On Sep 24, 2015, at 10:34 AM, Sagi Grimberg wrote:
>
> Instead of maintaining a fastreg page list, keep an sg table
> and convert an array of pages to a sg list. Then call ib_map_mr_sg
> and construct ib_reg_wr.
>
> Note that the next step would be to have NFS work with sg lists
> as it maps well to sk_frags (see comment from hch
> http://marc.info/?l=linux-rdma&m=143677002622296&w=2).
Fwiw, you would need to change tcp_sendpages() first.
One more comment below.
> Signed-off-by: Sagi Grimberg
> Acked-by: Christoph Hellwig
> ---
> net/sunrpc/xprtrdma/frwr_ops.c | 113 +++-
> net/sunrpc/xprtrdma/xprt_rdma.h | 3 +-
> 2 files changed, 68 insertions(+), 48 deletions(-)
>
> diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c
> index 0d2f46f600b6..b80a82149977 100644
> --- a/net/sunrpc/xprtrdma/frwr_ops.c
> +++ b/net/sunrpc/xprtrdma/frwr_ops.c
> @@ -151,9 +151,13 @@ __frwr_init(struct rpcrdma_mw *r, struct ib_pd *pd,
> struct ib_device *device,
> f->fr_mr = ib_alloc_mr(pd, IB_MR_TYPE_MEM_REG, depth);
> if (IS_ERR(f->fr_mr))
> goto out_mr_err;
> - f->fr_pgl = ib_alloc_fast_reg_page_list(device, depth);
> - if (IS_ERR(f->fr_pgl))
> +
> + f->sg = kcalloc(depth, sizeof(*f->sg), GFP_KERNEL);
> + if (!f->sg)
> goto out_list_err;
> +
> + sg_init_table(f->sg, depth);
> +
> return 0;
>
> out_mr_err:
> @@ -163,7 +167,7 @@ out_mr_err:
> return rc;
>
> out_list_err:
> - rc = PTR_ERR(f->fr_pgl);
> + rc = -ENOMEM;
> dprintk("RPC: %s: ib_alloc_fast_reg_page_list status %i\n",
> __func__, rc);
> ib_dereg_mr(f->fr_mr);
> @@ -179,7 +183,7 @@ __frwr_release(struct rpcrdma_mw *r)
> if (rc)
> dprintk("RPC: %s: ib_dereg_mr status %i\n",
> __func__, rc);
> - ib_free_fast_reg_page_list(r->r.frmr.fr_pgl);
> + kfree(r->r.frmr.sg);
> }
>
> static int
> @@ -312,14 +316,11 @@ frwr_op_map(struct rpcrdma_xprt *r_xprt, struct
> rpcrdma_mr_seg *seg,
> struct rpcrdma_mw *mw;
> struct rpcrdma_frmr *frmr;
> struct ib_mr *mr;
> - struct ib_fast_reg_wr fastreg_wr;
> + struct ib_reg_wr reg_wr;
> struct ib_send_wr *bad_wr;
> + unsigned int dma_nents;
> u8 key;
> - int len, pageoff;
> - int i, rc;
> - int seg_len;
> - u64 pa;
> - int page_no;
> + int i, rc, len, n;
>
> mw = seg1->rl_mw;
> seg1->rl_mw = NULL;
> @@ -332,64 +333,81 @@ frwr_op_map(struct rpcrdma_xprt *r_xprt, struct
> rpcrdma_mr_seg *seg,
> } while (mw->r.frmr.fr_state != FRMR_IS_INVALID);
> frmr = &mw->r.frmr;
> frmr->fr_state = FRMR_IS_VALID;
> + mr = frmr->fr_mr;
>
> - pageoff = offset_in_page(seg1->mr_offset);
> - seg1->mr_offset -= pageoff; /* start of page */
> - seg1->mr_len += pageoff;
> - len = -pageoff;
> if (nsegs > ia->ri_max_frmr_depth)
> nsegs = ia->ri_max_frmr_depth;
>
> - for (page_no = i = 0; i < nsegs;) {
> - rpcrdma_map_one(device, seg, direction);
> - pa = seg->mr_dma;
> - for (seg_len = seg->mr_len; seg_len > 0; seg_len -= PAGE_SIZE) {
> - frmr->fr_pgl->page_list[page_no++] = pa;
> - pa += PAGE_SIZE;
> - }
> + for (len = 0, i = 0; i < nsegs;) {
> + if (seg->mr_page)
> + sg_set_page(&frmr->sg[i],
> + seg->mr_page,
> + seg->mr_len,
> + offset_in_page(seg->mr_offset));
> + else
> + sg_set_buf(&frmr->sg[i], seg->mr_offset,
> +seg->mr_len);
> +
> len += seg->mr_len;
> ++seg;
> ++i;
> +
> /* Check for holes */
> if ((i < nsegs && offset_in_page(seg->mr_offset)) ||
> offset_in_page((seg-1)->mr_offset + (seg-1)->mr_len))
> break;
> }
> + frmr->sg_nents = i;
> +
> + dma_nents = ib_dma_map_sg(device, frmr->sg, frmr->sg_nents, direction);
> + if (!dma_nents) {
> + pr_err("RPC: %s: failed to dma map sg %p sg_nents %d\n",
> +__func__, frmr->sg, frmr->sg_nents);
> + return -ENOMEM;
> + }
> +
> + n = ib_map_mr_sg(mr, frmr->sg, frmr->sg_nents, PAGE_SIZE);
> + if (unlikely(n != frmr->sg_nents)) {
> + pr_err("RPC: %s: failed to map mr %p (%d/%d)\n",
> +__func__, frmr->fr_mr, n, frmr->sg_nents);
> + rc = n < 0 ? n : -EINVAL;
> + goto out_senderr;
> + }
> +
> dprintk("RPC: %s: Using frmr %p to map %d segments (%d bytes)\n",
> - __func__, mw, i, len);
> -
> - memset(&fastreg_wr, 0, sizeof(fastreg_wr));
> - fastreg_wr.wr.wr_id = (unsigned lo
Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)
On Fri, 25 Sep 2015, Or Gerlitz wrote: > On Thu, Sep 24, 2015 at 8:00 PM, Christoph Lameter wrote: > > Ok here is the fixed up and tested V2 of the patch. Can this go in with > > Doug's patch? > > > Repeating myself... do you find some over complexity in Erez's > implementation? what's the rational for not using his patch and yes > using yours? Erez and Co were very busy with some internal deadlines > and he's now OOO (it's a high Holiday season now) - will be able to > review your patch once he's back (Oct 6, I believe). It seems that the > patch does the job, but there are locking/contexts and such to > consider here, so I can't just ack it, have you passed it through > testing? Yes the patch introduces a new callback and creates workqueues that recheck conditions etc etc. Makes it difficult to review and potentially creates new race conditions. I'd rather have a straightforward solution. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 16/26] IB/srp: Convert to new registration API
On 09/24/2015 10:35 AM, Sagi Grimberg wrote: Remove srp_finish_mapping since no one is calling it. Please move the srp_finish_mapping() removal into a separate patch. Thanks, Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 15/26] IB/srp: Split srp_map_sg
On 09/24/2015 10:35 AM, Sagi Grimberg wrote:
This is a perparation patch for the new registration API
^ preparation ?
conversion. It splits srp_map_sg per registration strategy
(srp_map_sg[fmr|fr|dma]. On its own it adds some code duplication,
but it makes the API switch easier to comprehend.
Signed-off-by: Sagi Grimberg
---
drivers/infiniband/ulp/srp/ib_srp.c | 156
1 file changed, 105 insertions(+), 51 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c
b/drivers/infiniband/ulp/srp/ib_srp.c
index f8b9c18da03d..74748d1075fc 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -1286,6 +1286,17 @@ static int srp_map_finish_fmr(struct srp_map_state
*state,
if (WARN_ON_ONCE(state->fmr.next >= state->fmr.end))
return -ENOMEM;
+ WARN_ON_ONCE(!dev->use_fmr);
+
+ if (state->npages == 0)
+ return 0;
+
+ if (state->npages == 1 && target->global_mr) {
+ srp_map_desc(state, state->base_dma_addr, state->dma_len,
+target->global_mr->rkey);
+ return 0;
+ }
+
The above is not correct. The npages and dma_len variables have to be
reset before returning. How about changing the "return 0" statement into
"goto reset_state" and adding a "reset_state" label ?
fmr = ib_fmr_pool_map_phys(ch->fmr_pool, state->pages,
state->npages, io_addr);
if (IS_ERR(fmr))
@@ -1297,6 +1308,9 @@ static int srp_map_finish_fmr(struct srp_map_state *state,
srp_map_desc(state, state->base_dma_addr & ~dev->mr_page_mask,
state->dma_len, fmr->fmr->rkey);
+ state->npages = 0;
+ state->dma_len = 0;
+
return 0;
}
@@ -1309,10 +1323,23 @@ static int srp_map_finish_fr(struct srp_map_state
*state,
struct ib_fast_reg_wr wr;
struct srp_fr_desc *desc;
u32 rkey;
+ int err;
+
if (WARN_ON_ONCE(state->fr.next >= state->fr.end))
return -ENOMEM;
+ WARN_ON_ONCE(!dev->use_fast_reg);
+
+ if (state->npages == 0)
+ return 0;
+
+ if (state->npages == 1 && target->global_mr) {
+ srp_map_desc(state, state->base_dma_addr, state->dma_len,
+target->global_mr->rkey);
+ return 0;
+ }
+
Same comment here. npages and dma_len have to be reset before returning.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH for-next 0/2] RDMA/cxgb4: Add iWARP support for T6 adapter
Acked-by: Steve Wise -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)
On Thu, Sep 24, 2015 at 8:00 PM, Christoph Lameter wrote:
> Ok here is the fixed up and tested V2 of the patch. Can this go in with
> Doug's patch?
Repeating myself... do you find some over complexity in Erez's
implementation? what's the rational for not using his patch and yes
using yours? Erez and Co were very busy with some internal deadlines
and he's now OOO (it's a high Holiday season now) - will be able to
review your patch once he's back (Oct 6, I believe). It seems that the
patch does the job, but there are locking/contexts and such to
consider here, so I can't just ack it, have you passed it through
testing?
Or.
> Subject: ipoib: Expire sendonly multicast joins on neighbor expiration V2
>
> Add mcast_leave functionality to __ipoib_reap_neighbor.
>
> Signed-off-by: Christoph Lameter
>
> Index: linux/drivers/infiniband/ulp/ipoib/ipoib_main.c
> ===
> --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c2015-09-23
> 09:51:19.259274231 -0500
> +++ linux/drivers/infiniband/ulp/ipoib/ipoib_main.c 2015-09-23
> 09:59:59.803289023 -0500
> @@ -1149,6 +1149,9 @@
> unsigned long dt;
> unsigned long flags;
> int i;
> + LIST_HEAD(remove_list);
> + struct ipoib_mcast *mcast, *tmcast;
> + struct net_device *dev = priv->dev;
>
> if (test_bit(IPOIB_STOP_NEIGH_GC, &priv->flags))
> return;
> @@ -1176,6 +1179,19 @@
>
> lockdep_is_held(&priv->lock))) != NULL) {
> /* was the neigh idle for two GC periods */
> if (time_after(neigh_obsolete, neigh->alive)) {
> + u8 *mgid = neigh->daddr + 4;
> +
> + /* Is this multicast ? */
> + if (*mgid == 0xff) {
> + mcast = __ipoib_mcast_find(dev, mgid);
> +
> + if (mcast &&
> test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)) {
> + list_del(&mcast->list);
> + rb_erase(&mcast->rb_node,
> &priv->multicast_tree);
> + list_add_tail(&mcast->list,
> &remove_list);
> + }
> + }
> +
> rcu_assign_pointer(*np,
>
> rcu_dereference_protected(neigh->hnext,
>
> lockdep_is_held(&priv->lock)));
> @@ -1191,6 +1207,8 @@
>
> out_unlock:
> spin_unlock_irqrestore(&priv->lock, flags);
> + list_for_each_entry_safe(mcast, tmcast, &remove_list, list)
> + ipoib_mcast_leave(dev, mcast);
> }
>
> static void ipoib_reap_neigh(struct work_struct *work)
> Index: linux/drivers/infiniband/ulp/ipoib/ipoib.h
> ===
> --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2015-09-23
> 09:51:19.259274231 -0500
> +++ linux/drivers/infiniband/ulp/ipoib/ipoib.h 2015-09-23 09:51:19.255274231
> -0500
> @@ -548,6 +548,8 @@
>
> int ipoib_mcast_attach(struct net_device *dev, u16 mlid,
>union ib_gid *mgid, int set_qkey);
> +int ipoib_mcast_leave(struct net_device *dev, struct ipoib_mcast *mcast);
> +struct ipoib_mcast *__ipoib_mcast_find(struct net_device *dev, void *mgid);
>
> int ipoib_init_qp(struct net_device *dev);
> int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca);
> Index: linux/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
> ===
> --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2015-09-23
> 09:51:19.259274231 -0500
> +++ linux/drivers/infiniband/ulp/ipoib/ipoib_multicast.c2015-09-23
> 09:51:19.255274231 -0500
> @@ -158,7 +158,7 @@
> return mcast;
> }
>
> -static struct ipoib_mcast *__ipoib_mcast_find(struct net_device *dev, void
> *mgid)
> +struct ipoib_mcast *__ipoib_mcast_find(struct net_device *dev, void *mgid)
> {
> struct ipoib_dev_priv *priv = netdev_priv(dev);
> struct rb_node *n = priv->multicast_tree.rb_node;
> @@ -705,7 +705,7 @@
> return 0;
> }
>
> -static int ipoib_mcast_leave(struct net_device *dev, struct ipoib_mcast
> *mcast)
> +int ipoib_mcast_leave(struct net_device *dev, struct ipoib_mcast *mcast)
> {
> struct ipoib_dev_priv *priv = netdev_priv(dev);
> int ret = 0;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH rdma-rc v2 0/4] Remove mlx5 support for IB_DEVICE_LOCAL_DMA_LKEY
On 09/24/2015 03:34 AM, Sagi Grimberg wrote:
> The Connect-IB device has a specific issue with memory registration using
> the reserved lkey (device global_dma_lkey). This caused user-space memory
> registration which usually uses cached pre-registered memory keys to fail
> due to a device access error during registration. kernel-space memory
> registrations used an internal instance a physical memory key allocated with
> the private pd context, so this error didn't happen there.
>
> Since the reserved lkey is not fully functional, remove the support for
> it altogether. Jason's patches commit 96249d70dd70 ("IB/core: Guarantee
> that a local_dma_lkey is available") address consumers by allocating a
> physical
> lkey per PD in the core layer. It also allows mlx5 driver to get rid of it's
> private physical lkey (patch #2).
>
> The ConnectX-4 device should have this issue fixed so the capability will be
> restored depending on a FW query information.
>
> Also, fix NFS client to use the PD local_dma_lkey instead of the device
> local_dma_lkey (which requires a missing check of the device capability in
> frwr mode). And, fix iser initiator which encountered some issues when
> registering a signature capable memory region with an indirect dma_lkey
> area. The fix covers a larger ground as it does not allow using a global MR
> with remote access (long standing issue) but on the way makes the specific
> registration issue go away.
>
> Thanks to Haggai for catching this early enough.
>
> Many thanks for the reviews.
>
> Changes from v1:
> - Fixed changelog of patch #3 (comments from OrG)
>
> Changes from v0:
> - Replace xprtrdma patch to Chuck's one
> - Fixed typo in iser modparam description
>
> Chuck Lever (1):
> xprtrdma: Replace global lkey with lkey local to PD
>
> Sagi Grimberg (3):
> IB/iser: Add module parameter for always register memory
> IB/mlx5: Remove support for IB_DEVICE_LOCAL_DMA_LKEY
> IB/mlx5: Remove pa_lkey usages
>
> drivers/infiniband/hw/mlx5/main.c| 67
> +---
> drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 -
> drivers/infiniband/hw/mlx5/qp.c | 4 +-
> drivers/infiniband/ulp/iser/iscsi_iser.c | 5 +++
> drivers/infiniband/ulp/iser/iscsi_iser.h | 1 +
> drivers/infiniband/ulp/iser/iser_memory.c| 18 +---
> drivers/infiniband/ulp/iser/iser_verbs.c | 21 +
> drivers/net/ethernet/mellanox/mlx5/core/fw.c | 22 -
> include/linux/mlx5/device.h | 11 -
> include/linux/mlx5/driver.h | 1 -
> net/sunrpc/xprtrdma/fmr_ops.c| 19
> net/sunrpc/xprtrdma/frwr_ops.c | 5 ---
> net/sunrpc/xprtrdma/physical_ops.c | 10 +
> net/sunrpc/xprtrdma/verbs.c | 2 +-
> net/sunrpc/xprtrdma/xprt_rdma.h | 1 -
> 15 files changed, 35 insertions(+), 154 deletions(-)
>
Hi Sagi,
I've applied this for 4.3-rc. Thanks!
--
Doug Ledford
GPG KeyID: 0E572FDD
signature.asc
Description: OpenPGP digital signature
RE: [PATCH] staging/rdma: Kconfig change STAGING_RDMA to be tristate.
> Subject: [PATCH] staging/rdma: Kconfig change STAGING_RDMA to be > tristate. > > From: Ira Weiny > > STAGING_RDMA was failing to build when INFINIBAND was set to 'm' and > STAGING_RDMA was set to 'y'. > > Making this a tristate properly inherits the 'm' from the INFINIBAND setting. > This patch has been shown to fix the issue noted in http://marc.info/?l=linux-rdma&m=144311398317945&w=2. Tested-by: Mike Marciniszyn -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
