[ewg] [PATCH] IB/ipath - fix UC receive completion opcode
Vlad, please pull one small fix for OFED-1.3.1 that has been submitted and taken for 2.6.26. git://git.openfabrics.org/~ralphc/linux-2.6/.git ofed_kernel When I fixed the RC receive completion opcode, I forgot to fix UC which had the same problem for RDMA write with immediate returning the wrong opcode. Signed-off-by: Ralph Campbell <[EMAIL PROTECTED]> diff -up a/drivers/infiniband/hw/ipath/ipath_uc.c b/drivers/infiniband/hw/ipath/ipath_uc.c --- a/drivers/infiniband/hw/ipath/ipath_uc.c2008-05-15 16:07:53.0 -0700 +++ b/drivers/infiniband/hw/ipath/ipath_uc.c2008-05-15 16:12:23.0 -0700 @@ -408,12 +408,11 @@ void ipath_uc_rcv(struct ipath_ibdev *de dev->n_pkt_drops++; goto done; } - /* XXX Need to free SGEs */ + wc.opcode = IB_WC_RECV; last_imm: ipath_copy_sge(&qp->r_sge, data, tlen); wc.wr_id = qp->r_wr_id; wc.status = IB_WC_SUCCESS; - wc.opcode = IB_WC_RECV; wc.qp = &qp->ibqp; wc.src_qp = qp->remote_qpn; wc.slid = qp->remote_ah_attr.dlid; @@ -515,6 +514,7 @@ void ipath_uc_rcv(struct ipath_ibdev *de goto done; } wc.byte_len = qp->r_len; + wc.opcode = IB_WC_RECV_RDMA_WITH_IMM; goto last_imm; case OP(RDMA_WRITE_LAST): ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: OpenSM 3.1.11 for OFED 1.3.1
I just sent a patch with a HOWTO for the PerfMgr as well. That could be applied to 1.3 as well.[*] Ira [*] Sasha, sorry for all the last minute patches... :-( On Thu, 15 May 2008 04:55:26 -0700 Hal Rosenstock <[EMAIL PROTECTED]> wrote: > On Wed, 2008-05-14 at 18:34 +0300, Sasha Khapyorsky wrote: > > On 17:19 Wed 14 May , Tziporet Koren wrote: > > > There is the seg-fault bug that was fixed by Yevgeny that we need. > > > > This single commit was applied to 1.3 branch. > > I haven't seen any email yet to Vlad requesting it to be pulled into > OFED 1.3 so I assume that is pending. > > Also, the release notes would need some minor updating now (renumbering > and perhaps mention of additional bug fixes). > > > I think Hal was asking about OpenSM version bumping > > and tarball release for 1.3.x. > > If this is not done, then how is this identified as different from > 3.1.10 ? We're back to which OFED did the OpenSM come from and I thought > the agreed direction was to get away from this as much as possible with > tarballs. At least in this way they would be consistent. > > -- Hal > > > Sasha > > ___ > > ewg mailing list > > ewg@lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > > ___ > ewg mailing list > ewg@lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] version number of rpms on OFED-1.3.1
Hi, Some of the RPMs that will be included in OFED 1.3.1 will be different in content from the ones in OFED 1.3. (e.g. open-iscsi) Therefore, I think that those RPMs must have a different version number. Is there already a convention for that? Else, we need to define one. Thanks, Doron ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] IB/ipoib: Fix neigh destructor oops for kernels older than 2.6.21
On Thu, 2008-05-15 at 18:10 +0300, Or Gerlitz wrote: > 1 day is not enough time to review the patch, as of the importance, I > suggest you ask Roland and Alexey to review it and say what their > opinion on the solution, now once we all understand the problem. > I think I did ask, not explicitly but I guess it is understood that the patch was sent for review. Anyway now it's explicit: I am asking all the folks to review the patch and send comments. Alternate approaches towards solving the same problem are welcome too. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] IB/ipoib: Fix neigh destructor oops for kernels older than 2.6.21
Eli Cohen wrote: Good, so now we have some experience that it works and I don't see any comments on the patch itself. Unless there are objections I will push this to OFED as a backport for kernels <= 2.6.20. Eli, 1 day is not enough time to review the patch, as of the importance, I suggest you ask Roland and Alexey to review it and say what their opinion on the solution, now once we all understand the problem. Or. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] IB/ipoib: Fix neigh destructor oops for kernels older than 2.6.21
On Thu, 2008-05-15 at 16:50 +0300, Ron Livne wrote: > Hi, > I tested it on red hat 5 and it's working. > Good, so now we have some experience that it works and I don't see any comments on the patch itself. Unless there are objections I will push this to OFED as a backport for kernels <= 2.6.20. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] IB/ipoib: Fix neigh destructor oops for kernels older than 2.6.21
Hi,
I tested it on red hat 5 and it's working.
Ron
On Wed, May 14, 2008 at 2:37 PM, Eli Cohen <[EMAIL PROTECTED]> wrote:
> IB/ipoib: Fix neigh destructor oops
>
> For kernels 2.6.20 and older, it may happen that the pointer to
> ipoib_neigh_cleanup() is called after IPoIB has been unloades,
> causing a kernel oops. This problem has been fixed for 2.6.21 with
> the following commit: ecbb416939da77c0d107409976499724baddce7b
>
> The idea with this patch is to have a helper module which remains
> always loaded, and this modules provides the destructor for
> neighbours which calls IPoIB's destructor through a function poiner.
> When IPoIB is unloaded, the function pointer is cleared so subsequent
> calls to a neighbour destructor will be made to valid addresses but
> IPoIB's destructor won't get called.
>
> Signed-off-by: Eli Cohen <[EMAIL PROTECTED]>
> ---
>
> I know it's not the most elegant way to handle this but since patching
> distros' kernels is not relevant this is what I have. I checked this
> for 20 hours while originally the failure occurs after half an hour.
> Comments?
>
>
> Index: ofa_1_3_dev_kernel/drivers/infiniband/ulp/ipoib/ipoib_main.c
> ===
> --- ofa_1_3_dev_kernel.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c
> 2008-05-14 12:49:11.0 +0300
> +++ ofa_1_3_dev_kernel/drivers/infiniband/ulp/ipoib/ipoib_main.c
> 2008-05-14 12:49:32.0 +0300
> @@ -49,6 +49,7 @@
>
> #include
> #include
> +#include
>
> MODULE_AUTHOR("Roland Dreier");
> MODULE_DESCRIPTION("IP-over-InfiniBand net driver");
> @@ -916,7 +917,7 @@ void ipoib_neigh_free(struct net_device
>
> static int ipoib_neigh_setup_dev(struct net_device *dev, struct
> neigh_parms *parms)
> {
> - parms->neigh_cleanup = ipoib_neigh_cleanup;
> + parms->neigh_cleanup = ipoib_neigh_cleanup_container;
>
>return 0;
> }
> @@ -1383,9 +1384,13 @@ static int __init ipoib_init_module(void
>ipoib_max_conn_qp = min(ipoib_max_conn_qp, IPOIB_CM_MAX_CONN_QP);
> #endif
>
> +
> + ipoib_set_cleanup_function(ipoib_neigh_cleanup);
>ret = ipoib_register_debugfs();
> - if (ret)
> + if (ret) {
> + ipoib_set_cleanup_function(NULL);
>return ret;
> + }
>
>/*
> * We create our own workqueue mainly because we want to be
> @@ -1397,6 +1402,7 @@ static int __init ipoib_init_module(void
> */
>ipoib_workqueue = create_singlethread_workqueue("ipoib");
>if (!ipoib_workqueue) {
> + ipoib_set_cleanup_function(NULL);
>ret = -ENOMEM;
>goto err_fs;
>}
> @@ -1404,8 +1410,10 @@ static int __init ipoib_init_module(void
>ib_sa_register_client(&ipoib_sa_client);
>
>ret = ib_register_client(&ipoib_client);
> - if (ret)
> + if (ret) {
> + ipoib_set_cleanup_function(NULL);
>goto err_sa;
> + }
>
>return 0;
>
> @@ -1421,7 +1429,16 @@ err_fs:
>
> static void __exit ipoib_cleanup_module(void)
> {
> + int ret;
> +
>ib_unregister_client(&ipoib_client);
> +
> + do {
> + ret = ipoib_set_cleanup_function(NULL);
> + if (ret)
> + msleep(10);
> + } while(ret);
> +
>ib_sa_unregister_client(&ipoib_sa_client);
>ipoib_unregister_debugfs();
>destroy_workqueue(ipoib_workqueue);
> Index: ofa_1_3_dev_kernel/drivers/infiniband/ulp/ipoib/Makefile
> ===
> --- ofa_1_3_dev_kernel.orig/drivers/infiniband/ulp/ipoib/Makefile
> 2008-05-14 12:49:11.0 +0300
> +++ ofa_1_3_dev_kernel/drivers/infiniband/ulp/ipoib/Makefile2008-05-14
> 12:49:32.0 +0300
> @@ -1,4 +1,4 @@
> -obj-$(CONFIG_INFINIBAND_IPOIB) += ib_ipoib.o
> +obj-$(CONFIG_INFINIBAND_IPOIB) += ib_ipoib.o
> ipoib_helper.o
>
> ib_ipoib-y := ipoib_main.o \
> ipoib_ib.o \
> Index: ofa_1_3_dev_kernel/drivers/infiniband/ulp/ipoib/ipoib.h
> ===
> --- ofa_1_3_dev_kernel.orig/drivers/infiniband/ulp/ipoib/ipoib.h
> 2008-05-14 12:49:11.0 +0300
> +++ ofa_1_3_dev_kernel/drivers/infiniband/ulp/ipoib/ipoib.h 2008-05-14
> 12:49:32.0 +0300
> @@ -554,6 +554,9 @@ int ipoib_mcast_stop_thread(struct net_d
> void ipoib_mcast_dev_down(struct net_device *dev);
> void ipoib_mcast_dev_flush(struct net_device *dev);
>
> +int ipoib_set_cleanup_function(void (*func)(struct neighbour *n));
> +void ipoib_neigh_cleanup_container(struct neighbour *n);
> +
> #ifdef CONFIG_INFINIBAND_IPOIB_DEBUG
> struct ipoib_mcast_iter *ipoib_mcast_iter_init(struct net_device *dev);
> int ipoib_mcast_iter_next(struct ipoib_mcast_iter *iter);
> Index: ofa_1_3_dev_kernel/drivers/infiniband/ul
Re: [ewg] Re: OpenSM 3.1.11 for OFED 1.3.1
On Wed, 2008-05-14 at 18:34 +0300, Sasha Khapyorsky wrote: > On 17:19 Wed 14 May , Tziporet Koren wrote: > > There is the seg-fault bug that was fixed by Yevgeny that we need. > > This single commit was applied to 1.3 branch. I haven't seen any email yet to Vlad requesting it to be pulled into OFED 1.3 so I assume that is pending. Also, the release notes would need some minor updating now (renumbering and perhaps mention of additional bug fixes). > I think Hal was asking about OpenSM version bumping > and tarball release for 1.3.x. If this is not done, then how is this identified as different from 3.1.10 ? We're back to which OFED did the OpenSM come from and I thought the agreed direction was to get away from this as much as possible with tarballs. At least in this way they would be consistent. -- Hal > Sasha > ___ > ewg mailing list > ewg@lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCH] Add iSER bug fixes for OFED 1.3.1
Doron Shoham wrote: Hi Vlad, I have added all iSER bug fixes since 2.6.24. The backport patches are fixed accordingly. Please pull from git://git.openfabrics.org/~dorons/linux-2.6.git ofed_kernel Thanks, Doron ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg Done, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH] Add iSER bug fixes for OFED 1.3.1
Hi Vlad, I have added all iSER bug fixes since 2.6.24. The backport patches are fixed accordingly. Please pull from git://git.openfabrics.org/~dorons/linux-2.6.git ofed_kernel Thanks, Doron ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Compiling OFED 1.3 on Gentoo
Hi Olga, On 17:18 Mon 12 May , Olga Shern wrote: > > We are trying to compile OFED 1.3 on Gentoo and see the following error, > But if I install source RPM file and then running 'rpmbuild -ba > libibcommon.spec' then I can build RPM, so only rpmbuild --rebuild > command causing to problems. Basically Gentoo doesn't use RPM as package manager, but builds packages from sources using portage/emerge stuff. So it is nice that it builds somehow. As another workaround rpm2targz probably can be used too. Of course better would be to have native *.ebuild files. > Have someone succeeded to build OFED 1.3 on Gentoo? I'm using Gentoo as my workstation, don't do RPMs there however. Sasha ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH v2] IB/mlx4: Add send with invalidate support
>From 2fa86ee977039784f50de982e2f6bf197f00fbeb Mon Sep 17 00:00:00 2001
From: Eli Cohen <[EMAIL PROTECTED]>
Date: Sun, 11 May 2008 15:02:04 +0300
Subject: [PATCH] IB/mlx4: Add send with invalidate support
Add send with invalidate support to mlx4.
Signed-off-by: Eli Cohen <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/mlx4/cq.c |8
drivers/infiniband/hw/mlx4/main.c |4
drivers/infiniband/hw/mlx4/qp.c | 22 +-
drivers/net/mlx4/mr.c |6 --
4 files changed, 33 insertions(+), 7 deletions(-)
Changes since last post:
set IB_DEVICE_SEND_W_INV only if FW supports it.
diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index 4521319..291e856 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -637,6 +637,7 @@ repoll:
case MLX4_OPCODE_SEND_IMM:
wc->wc_flags |= IB_WC_WITH_IMM;
case MLX4_OPCODE_SEND:
+ case MLX4_OPCODE_SEND_INVAL:
wc->opcode= IB_WC_SEND;
break;
case MLX4_OPCODE_RDMA_READ:
@@ -676,6 +677,13 @@ repoll:
wc->wc_flags = IB_WC_WITH_IMM;
wc->imm_data = cqe->immed_rss_invalid;
break;
+ case MLX4_RECV_OPCODE_SEND_INVAL:
+ wc->opcode = IB_WC_RECV;
+ wc->wc_flags = IB_WC_WITH_INVALIDATE;
+ /*
+* TBD: maybe we should just call this ieth_val
+*/
+ wc->imm_data = be32_to_cpu(cqe->immed_rss_invalid);
}
wc->slid = be16_to_cpu(cqe->rlid);
diff --git a/drivers/infiniband/hw/mlx4/main.c
b/drivers/infiniband/hw/mlx4/main.c
index 4d61e32..b1e9505 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -56,6 +56,8 @@ static const char mlx4_ib_version[] =
DRV_NAME ": Mellanox ConnectX InfiniBand driver v"
DRV_VERSION " (" DRV_RELDATE ")\n";
+#define MLX4_FW_VER_LOCAL_SEND_INVL mlx4_fw_ver(2, 5, 0)
+
static void init_query_mad(struct ib_smp *mad)
{
mad->base_version = 1;
@@ -103,6 +105,8 @@ static int mlx4_ib_query_device(struct ib_device *ibdev,
props->device_cap_flags |= IB_DEVICE_UD_IP_CSUM;
if (dev->dev->caps.max_gso_sz)
props->device_cap_flags |= IB_DEVICE_UD_TSO;
+ if (dev->dev->caps.fw_ver >= MLX4_FW_VER_LOCAL_SEND_INVL)
+ props->device_cap_flags |= IB_DEVICE_SEND_W_INV;
props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data +
36)) &
0xff;
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index 8e02ecf..d0d5f77 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -78,6 +78,7 @@ static const __be32 mlx4_ib_opcode[] = {
[IB_WR_RDMA_READ] =
__constant_cpu_to_be32(MLX4_OPCODE_RDMA_READ),
[IB_WR_ATOMIC_CMP_AND_SWP] =
__constant_cpu_to_be32(MLX4_OPCODE_ATOMIC_CS),
[IB_WR_ATOMIC_FETCH_AND_ADD]=
__constant_cpu_to_be32(MLX4_OPCODE_ATOMIC_FA),
+ [IB_WR_SEND_WITH_INV] =
__constant_cpu_to_be32(MLX4_OPCODE_SEND_INVAL),
};
static struct mlx4_ib_sqp *to_msqp(struct mlx4_ib_qp *mqp)
@@ -1444,6 +1445,21 @@ static int build_lso_seg(struct mlx4_lso_seg *wqe,
struct ib_send_wr *wr,
return 0;
}
+static __be32 get_ieth(struct ib_send_wr *wr)
+{
+ switch (wr->opcode) {
+ case IB_WR_SEND_WITH_IMM:
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ return wr->ex.imm_data;
+
+ case IB_WR_SEND_WITH_INV:
+ return cpu_to_be32(wr->ex.invalidate_rkey);
+
+ default:
+ return 0;
+ }
+}
+
int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
struct ib_send_wr **bad_wr)
{
@@ -1490,11 +1506,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct
ib_send_wr *wr,
MLX4_WQE_CTRL_TCP_UDP_CSUM) : 0) |
qp->sq_signal_bits;
- if (wr->opcode == IB_WR_SEND_WITH_IMM ||
- wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM)
- ctrl->imm = wr->ex.imm_data;
- else
- ctrl->imm = 0;
+ ctrl->imm = get_ieth(wr);
wqe += sizeof *ctrl;
size = sizeof *ctrl / 16;
diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c
index 03a9abc..e78f53d 100644
--- a/drivers/net/mlx4/mr.c
+++ b/drivers/net/mlx4/mr.c
@@ -47,7 +47,7 @@ struct mlx4_mpt_entry {
__be32 flags;
__be32 qpn;
__be32 key;
- __be32 pd;
+ __be32 pd_flags;
__be64 start;
__be64 length;
__be32 lkey;
@@ -71,6 +71,8 @@ struct mlx4_mpt_entry {
#defin
