[PATCH v3 00/36] Add rdma verbs transport library

[Dennis Dalessandro]

The following series implements rdmavt. This is the rdma verbs transport
software library which will help to solve the problem of code duplication
between hardware drivers when it comes to a verbs implementation.

Rdmavt is basically just another verbs provider and lives in the Infiniband tree
in a new sw directory. It provides a software implementation of the Infiniband
verbs API. More details can be found in the following threads:

http://www.spinics.net/lists/linux-rdma/msg29064.html
http://www.spinics.net/lists/linux-rdma/msg29922.html

Changes since v1:
Removed driver specific version
Fixed license text to remove copyright and put on top
Return 0 in rvt_map_sg instead of BAD_DMA_AGGRESS
Remove #include of dma.h from dma.c
Update comment about protection domain limit
Remove comment on alternative design for private data
Rename CDR macro to CHECK_DRIVER_OVERRIDE
Change all the stubs to return EOPNOTSUPP
Fix comment style for rvt_query_port
Fix typo in subject
Rename rdi.lk_table to rdi.lkey_table
Rename rvt_sge.m => rvt_sge.cur_map (Sean)
Rename rvt_sge.n => rvt_sge.cur_seg (Sean)
Remove rvt_reg_phys_mr
Drop support for commit 38071a461f0a ("IB/qib: Support the new memory
   registration API")
Changes since v2:
Rebased onto k.o/for-4.5 commit: 59caaed7a72a0e3750dfb84636dae6b781559310
Add back in dma.h include in dma.c due to sparse error
Handle the removal of reg_phys_mr()

---

Dennis Dalessandro (34):
  IB/rdmavt: Create module framework and handle driver registration
  IB/rdmavt: Consolidate dma ops in rdmavt.
  IB/rdmavt: Add protection domain to rdmavt.
  IB/rdmavt: Add ib core device attributes to rvt driver params list
  IB/rdmavt: Macroize override checks during driver registration
  IB/rdmavt: Add query and modify device stubs
  IB/rdmavt: Add query and modify port stubs
  IB/rdmavt: Add pkey query stub
  IB/rdmavt: Add query gid stub
  IB/rdmavt: Alloc and dealloc ucontexts
  IB/rdmavt: Add queue pair function stubs
  IB/rdmavt: Add address handle stubs
  IB/rdmavt: Add memory region stubs
  IB/rdmavt: Add SRQ stubs
  IB/rdmavt: Add multicast stubs
  IB/rdmavt: Add process MAD stub
  IB/rdmavt: Add mmap stub
  IB/rdmavt: Add get port immutable stub
  IB/rdmavt: Add completion queue function stubs
  IB/rdmavt: Add post send and recv stubs
  IB/rdmavt: Move MR datastructures into rvt
  IB/rdmavt: Add queue pair data structure to rdmavt
  IB/rdmavt: Move driver helper functions to a common structure
  IB/rdmavt: Add device specific info prints
  IB/rdmavt: Add the start of capability flags
  IB/rdmavt: Move memory registration into rdmavt
  IB/rdmavt: Do not use rvt prints which rely on driver too early
  IB/rdmavt: Move SRQ data structure into rdmavt
  IB/rdmavt: Add an ibport data structure to rdmavt
  IB/rdmavt: Add driver notification for new AH
  IB/rdmavt: Break rdma_vt main include header file up
  IB/rdmavt: Initialize and teardown of qpn table
  IB/rdmavt: Add mmap related functions
  IB/rdmavt: Add pkey support

Kamal Heib (2):
  IB/rdmavt: Add common LID defines to rdmavt
  IB/rdmavt: Add AH to rdmavt


 MAINTAINERS   |6 
 drivers/infiniband/Kconfig|2 
 drivers/infiniband/Makefile   |1 
 drivers/infiniband/sw/Makefile|1 
 drivers/infiniband/sw/rdmavt/Kconfig  |6 
 drivers/infiniband/sw/rdmavt/Makefile |   10 
 drivers/infiniband/sw/rdmavt/ah.c |  169 +++
 drivers/infiniband/sw/rdmavt/ah.h |   59 ++
 drivers/infiniband/sw/rdmavt/cq.c |  113 +
 drivers/infiniband/sw/rdmavt/cq.h |   62 ++
 drivers/infiniband/sw/rdmavt/dma.c|  184 +++
 drivers/infiniband/sw/rdmavt/dma.h|   53 ++
 drivers/infiniband/sw/rdmavt/mad.c|   85 +++
 drivers/infiniband/sw/rdmavt/mad.h|   59 ++
 drivers/infiniband/sw/rdmavt/mcast.c  |   58 ++
 drivers/infiniband/sw/rdmavt/mcast.h  |   56 ++
 drivers/infiniband/sw/rdmavt/mmap.c   |  198 
 drivers/infiniband/sw/rdmavt/mmap.h   |   55 ++
 drivers/infiniband/sw/rdmavt/mr.c |  824 +
 drivers/infiniband/sw/rdmavt/mr.h |   92 
 drivers/infiniband/sw/rdmavt/pd.c |  103 
 drivers/infiniband/sw/rdmavt/pd.h |   58 ++
 drivers/infiniband/sw/rdmavt/qp.c |  379 +++
 drivers/infiniband/sw/rdmavt/qp.h |   69 +++
 drivers/infiniband/sw/rdmavt/srq.c|   86 +++
 drivers/infiniband/sw/rdmavt/srq.h|   61 ++
 drivers/infiniband/sw/rdmavt/vt.c |  377 +++
 drivers/infiniband/sw/rdmavt/vt.h |   90 
 include/rdma/ib_verbs.h   |1 
 include/rdma/rdma_vt.h|  333 +
 include/rdma/rdmavt_mr.h  |  130 +
 include/rdma/rdmavt_qp.h  |  295 
 32 files changed, 4075 insertions(+), 0 deletions

[PATCH v3 06/36] IB/rdmavt: Add query and modify device stubs

[Dennis Dalessandro]

Adds the stubs which will handle the query and modify device functions. At
this time the only intention is to support changing the node desc and the
guid via these calls.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/vt.c |   31 +++
 1 files changed, 31 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 8bd25c3..db14646 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -64,6 +64,33 @@ static void rvt_cleanup(void)
 }
 module_exit(rvt_cleanup);
 
+static int rvt_query_device(struct ib_device *ibdev,
+   struct ib_device_attr *props,
+   struct ib_udata *uhw)
+{
+   /*
+* Return rvt_dev_info.props contents
+*/
+   return -EOPNOTSUPP;
+}
+
+static int rvt_modify_device(struct ib_device *device,
+int device_modify_mask,
+struct ib_device_modify *device_modify)
+{
+   /*
+* Change dev props. Planned support is for node desc change and sys
+* guid change only. This matches hfi1 and qib behavior. Other drivers
+* that support existing modifications will need to add their support.
+*/
+
+   /*
+* VT-DRIVER-API: node_desc_change()
+* VT-DRIVER-API: sys_guid_change()
+*/
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -76,6 +103,10 @@ int rvt_register_device(struct rvt_dev_info *rdi)
if (!rdi)
return -EINVAL;
 
+   /* Dev Ops */
+   CHECK_DRIVER_OVERRIDE(rdi, query_device);
+   CHECK_DRIVER_OVERRIDE(rdi, modify_device);
+
/* DMA Operations */
rdi->ibdev.dma_ops =
rdi->ibdev.dma_ops ? : _default_dma_mapping_ops;

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 05/36] IB/rdmavt: Macroize override checks during driver registration

[Dennis Dalessandro]

Add a macro to cut down on writing the same lines over and over again for
what will be a large number of functions that will be supported.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change CDR macro to CHECK_DRIVER_OVERRIDE to be more clear

 drivers/infiniband/sw/rdmavt/vt.c |   22 +-
 1 files changed, 9 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index b65cde4..8bd25c3 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -64,29 +64,25 @@ static void rvt_cleanup(void)
 }
 module_exit(rvt_cleanup);
 
+/*
+ * Check driver override. If driver passes a value use it, otherwise we use our
+ * own value.
+ */
+#define CHECK_DRIVER_OVERRIDE(rdi, x) \
+   rdi->ibdev.x = rdi->ibdev.x ? : rvt_ ##x
+
 int rvt_register_device(struct rvt_dev_info *rdi)
 {
if (!rdi)
return -EINVAL;
 
-   /*
-* Drivers have the option to override anything in the ibdev that they
-* want to specifically handle. VT needs to check for things it supports
-* and if the driver wants to handle that functionality let it. We may
-* come up with a better mechanism that simplifies the code at some
-* point.
-*/
-
/* DMA Operations */
rdi->ibdev.dma_ops =
rdi->ibdev.dma_ops ? : _default_dma_mapping_ops;
 
/* Protection Domain */
-   rdi->ibdev.alloc_pd =
-   rdi->ibdev.alloc_pd ? : rvt_alloc_pd;
-   rdi->ibdev.dealloc_pd =
-   rdi->ibdev.dealloc_pd ? : rvt_dealloc_pd;
-
+   CHECK_DRIVER_OVERRIDE(rdi, alloc_pd);
+   CHECK_DRIVER_OVERRIDE(rdi, dealloc_pd);
spin_lock_init(>n_pds_lock);
rdi->n_pds_allocated = 0;
 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 04/36] IB/rdmavt: Add ib core device attributes to rvt driver params list

[Dennis Dalessandro]

Instead of trying to handle each parameter separately, add ib_device_attr
to rvt_driver_params. This means drivers will fill this in and pass to the
rvt registration function.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
remove comment on alternative design for private data

 drivers/infiniband/sw/rdmavt/pd.c |2 +
 include/rdma/rdma_vt.h|   57 -
 2 files changed, 56 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/pd.c 
b/drivers/infiniband/sw/rdmavt/pd.c
index e6e153f..f8dba88 100644
--- a/drivers/infiniband/sw/rdmavt/pd.c
+++ b/drivers/infiniband/sw/rdmavt/pd.c
@@ -69,7 +69,7 @@ struct ib_pd *rvt_alloc_pd(struct ib_device *ibdev,
 */
 
spin_lock(>n_pds_lock);
-   if (dev->n_pds_allocated == dev->dparms.max_pds) {
+   if (dev->n_pds_allocated == dev->dparms.props.max_pd) {
spin_unlock(>n_pds_lock);
kfree(pd);
ret = ERR_PTR(-ENOMEM);
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 6bf5fd4..2990e03 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -59,7 +59,45 @@
  * Things that are driver specific, module parameters in hfi1 and qib
  */
 struct rvt_driver_params {
-   int max_pds;
+   /*
+* driver required fields:
+*  node_guid
+*  phys_port_cnt
+*  dma_device
+*  owner
+* driver optional fields (rvt will provide generic value if blank):
+*  name
+*  node_desc
+* rvt fields, driver value ignored:
+*  uverbs_abi_ver
+*  node_type
+*  num_comp_vectors
+*  uverbs_cmd_mask
+*/
+   struct ib_device_attr props;
+
+   /*
+* Drivers will need to support a number of notifications to rvt in
+* accordance with certain events. This structure should contain a mask
+* of the supported events. Such events that the rvt may need to know
+* about include:
+* port errors
+* port active
+* lid change
+* sm change
+* client reregister
+* pkey change
+*
+* There may also be other events that the rvt layers needs to know
+* about this is not an exhaustive list. Some events though rvt does not
+* need to rely on the driver for such as completion queue error.
+*/
+int rvt_signal_supported;
+
+   /*
+* Anything driver specific that is not covered by props
+* For instance special module parameters. Goes here.
+*/
 };
 
 /* Protection domain */
@@ -69,10 +107,25 @@ struct rvt_pd {
 };
 
 struct rvt_dev_info {
+   /*
+* Prior to calling for registration the driver will be responsible for
+* allocating space for this structure.
+*
+* The driver will also be responsible for filling in certain members of
+* dparms.props
+*/
+
struct ib_device ibdev;
 
-   /* Driver specific */
+   /* Driver specific properties */
struct rvt_driver_params dparms;
+
+   /*
+* The work to create port files in /sys/class Infiniband is different
+* depending on the driver. This should not be extracted away and
+* instead drivers are responsible for setting the correct callback for
+* this.
+*/
int (*port_callback)(struct ib_device *, u8, struct kobject *);
 
/* Internal use */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 13/36] IB/rdmavt: Add memory region stubs

[Dennis Dalessandro]

Adds the function stubs for allocating, and registering memory regions, as
well as deregistering them.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

Changes since v2:
Remove rvt_reg_phys_mr

 drivers/infiniband/sw/rdmavt/Makefile |2 
 drivers/infiniband/sw/rdmavt/mr.c |  165 +
 drivers/infiniband/sw/rdmavt/mr.h |   69 ++
 drivers/infiniband/sw/rdmavt/vt.c |   10 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 246 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/mr.c
 create mode 100644 drivers/infiniband/sw/rdmavt/mr.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 628c684..084ee6a 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o pd.o qp.o
+rdmavt-y := vt.o ah.o dma.o mr.o pd.o qp.o
diff --git a/drivers/infiniband/sw/rdmavt/mr.c 
b/drivers/infiniband/sw/rdmavt/mr.c
new file mode 100644
index 000..c672043
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mr.c
@@ -0,0 +1,165 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include 
+#include "mr.h"
+
+/**
+ * rvt_get_dma_mr - get a DMA memory region
+ * @pd: protection domain for this memory region
+ * @acc: access flags
+ *
+ * Returns the memory region on success, otherwise returns an errno.
+ * Note that all DMA addresses should be created via the
+ * struct ib_dma_mapping_ops functions (see dma.c).
+ */
+struct ib_mr *rvt_get_dma_mr(struct ib_pd *pd, int acc)
+{
+   /*
+* Alloc mr and init it.
+* Alloc lkey.
+*/
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * rvt_reg_user_mr - register a userspace memory region
+ * @pd: protection domain for this memory region
+ * @start: starting userspace address
+ * @length: length of region to register
+ * @mr_access_flags: access flags for this memory region
+ * @udata: unused by the driver
+ *
+ * Returns the memory region on success, otherwise returns an errno.
+ */
+struct ib_mr *rvt_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
+ u64 virt_addr, int mr_access_flags,
+ struct ib_udata *udata)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * rvt_dereg_mr - unregister and free a memory region
+ * @ibmr: the memory region to free
+ *
+ * Returns 0 on success.
+ *
+ * Note that this is called to free MRs created by rvt_get_dma_mr()
+ * or rvt_reg_us

[PATCH v3 15/36] IB/rdmavt: Add multicast stubs

[Dennis Dalessandro]

Adds the function stubs for attach and detach multicast.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 +
 drivers/infiniband/sw/rdmavt/mcast.c  |   58 +
 drivers/infiniband/sw/rdmavt/mcast.h  |   56 
 drivers/infiniband/sw/rdmavt/vt.c |4 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 +
 5 files changed, 120 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/mcast.c
 create mode 100644 drivers/infiniband/sw/rdmavt/mcast.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 204be84..d2af114 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mr.o pd.o qp.o srq.o
+rdmavt-y := vt.o ah.o dma.o mcast.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/mcast.c 
b/drivers/infiniband/sw/rdmavt/mcast.c
new file mode 100644
index 000..5a78dc7
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mcast.c
@@ -0,0 +1,58 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "mcast.h"
+
+int rvt_attach_mcast(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_detach_mcast(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
+{
+   return -EOPNOTSUPP;
+}
diff --git a/drivers/infiniband/sw/rdmavt/mcast.h 
b/drivers/infiniband/sw/rdmavt/mcast.h
new file mode 100644
index 000..21647c3
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mcast.h
@@ -0,0 +1,56 @@
+#ifndef DEF_RVTMCAST_H
+#define DEF_RVTMCAST_H
+
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of sourc

[PATCH v3 14/36] IB/rdmavt: Add SRQ stubs

[Dennis Dalessandro]

Adds the stubs for create, modify, query, and destory for shared
request queues.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 -
 drivers/infiniband/sw/rdmavt/srq.c|   86 +
 drivers/infiniband/sw/rdmavt/srq.h|   61 +++
 drivers/infiniband/sw/rdmavt/vt.c |6 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 155 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/srq.c
 create mode 100644 drivers/infiniband/sw/rdmavt/srq.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 084ee6a..204be84 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mr.o pd.o qp.o
+rdmavt-y := vt.o ah.o dma.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/srq.c 
b/drivers/infiniband/sw/rdmavt/srq.c
new file mode 100644
index 000..bbb623a
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/srq.c
@@ -0,0 +1,86 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "srq.h"
+
+/**
+ * rvt_create_srq - create a shared receive queue
+ * @ibpd: the protection domain of the SRQ to create
+ * @srq_init_attr: the attributes of the SRQ
+ * @udata: data from libibverbs when creating a user SRQ
+ */
+struct ib_srq *rvt_create_srq(struct ib_pd *ibpd,
+ struct ib_srq_init_attr *srq_init_attr,
+ struct ib_udata *udata)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * rvt_modify_srq - modify a shared receive queue
+ * @ibsrq: the SRQ to modify
+ * @attr: the new attributes of the SRQ
+ * @attr_mask: indicates which attributes to modify
+ * @udata: user data for libibverbs.so
+ */
+int rvt_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,
+  enum ib_srq_attr_mask attr_mask,
+  struct ib_udata *udata)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_destroy_srq(struct ib_srq *ibsrq)
+{
+   return -EOPNOTSUPP;
+}
+
diff --git a/drivers/infiniband/sw/rdmavt/srq.h 
b/drivers/infiniband/sw/rdmavt/srq.h
new file mode 100644
index 000..0c3c5a7
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/srq.h
@@ -0,0 +1,61 @@
+#ifndef DEF_RVTSRQ_H
+#define DEF_RVTSRQ_H
+
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is p

[PATCH v3 09/10] IB/qib: Use address handle in rdmavt and remove from qib

[Dennis Dalessandro]

Original patch from Kamal Heib <kam...@mellanox.com>, split
apart from original.

Remove AH from qib and use rdmavt version.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Kamal Heib <kam...@mellanox.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_iba7322.c |2 
 drivers/infiniband/hw/qib/qib_mad.c |2 
 drivers/infiniband/hw/qib/qib_qp.c  |6 +
 drivers/infiniband/hw/qib/qib_ruc.c |2 
 drivers/infiniband/hw/qib/qib_ud.c  |4 -
 drivers/infiniband/hw/qib/qib_verbs.c   |  126 ++-
 drivers/infiniband/hw/qib/qib_verbs.h   |   20 -
 7 files changed, 19 insertions(+), 143 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_iba7322.c 
b/drivers/infiniband/hw/qib/qib_iba7322.c
index 6c8ff10..1fbe308 100644
--- a/drivers/infiniband/hw/qib/qib_iba7322.c
+++ b/drivers/infiniband/hw/qib/qib_iba7322.c
@@ -5515,7 +5515,7 @@ static void try_7322_ipg(struct qib_pportdata *ppd)
ret = PTR_ERR(ah);
else {
send_buf->ah = ah;
-   ibp->smi_ah = to_iah(ah);
+   ibp->smi_ah = ibah_to_rvtah(ah);
ret = 0;
}
} else {
diff --git a/drivers/infiniband/hw/qib/qib_mad.c 
b/drivers/infiniband/hw/qib/qib_mad.c
index 70fc1b2..43f8c49 100644
--- a/drivers/infiniband/hw/qib/qib_mad.c
+++ b/drivers/infiniband/hw/qib/qib_mad.c
@@ -109,7 +109,7 @@ static void qib_send_trap(struct qib_ibport *ibp, void 
*data, unsigned len)
ret = PTR_ERR(ah);
else {
send_buf->ah = ah;
-   ibp->sm_ah = to_iah(ah);
+   ibp->sm_ah = ibah_to_rvtah(ah);
ret = 0;
}
} else
diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index 143924d..f3964ef 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -438,7 +438,7 @@ static void clear_mr_refs(struct rvt_qp *qp, int clr_sends)
if (qp->ibqp.qp_type == IB_QPT_UD ||
qp->ibqp.qp_type == IB_QPT_SMI ||
qp->ibqp.qp_type == IB_QPT_GSI)
-   atomic_dec(_iah(wqe->ud_wr.ah)->refcount);
+   
atomic_dec(_to_rvtah(wqe->ud_wr.ah)->refcount);
if (++qp->s_last >= qp->s_size)
qp->s_last = 0;
}
@@ -595,7 +595,7 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
if (attr_mask & IB_QP_AV) {
if (attr->ah_attr.dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
goto inval;
-   if (qib_check_ah(qp->ibqp.device, >ah_attr))
+   if (rvt_check_ah(qp->ibqp.device, >ah_attr))
goto inval;
}
 
@@ -603,7 +603,7 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
if (attr->alt_ah_attr.dlid >=
be16_to_cpu(IB_MULTICAST_LID_BASE))
goto inval;
-   if (qib_check_ah(qp->ibqp.device, >alt_ah_attr))
+   if (rvt_check_ah(qp->ibqp.device, >alt_ah_attr))
goto inval;
if (attr->alt_pkey_index >= qib_get_npkeys(dd_from_dev(dev)))
goto inval;
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c 
b/drivers/infiniband/hw/qib/qib_ruc.c
index 02e79a8..3900459 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -789,7 +789,7 @@ void qib_send_complete(struct rvt_qp *qp, struct rvt_swqe 
*wqe,
if (qp->ibqp.qp_type == IB_QPT_UD ||
qp->ibqp.qp_type == IB_QPT_SMI ||
qp->ibqp.qp_type == IB_QPT_GSI)
-   atomic_dec(_iah(wqe->ud_wr.ah)->refcount);
+   atomic_dec(_to_rvtah(wqe->ud_wr.ah)->refcount);
 
/* See ch. 11.2.4.1 and 10.7.3.1 */
if (!(qp->s_flags & QIB_S_SIGNAL_REQ_WR) ||
diff --git a/drivers/infiniband/hw/qib/qib_ud.c 
b/drivers/infiniband/hw/qib/qib_ud.c
index 1d9d037..6dc20ca 100644
--- a/drivers/infiniband/hw/qib/qib_ud.c
+++ b/drivers/infiniband/hw/qib/qib_ud.c
@@ -77,7 +77,7 @@ static void qib_ud_loopback(struct rvt_qp *sqp, struct 
rvt_swqe *swqe)
goto drop;
}
 
-   ah_attr = _iah(swqe->ud_wr.ah)->attr;
+   ah_attr = _to_rvtah(swqe->ud_wr.ah)->attr;
ppd = ppd_from_ibp(ibp);
 
if (qp->ibqp.qp_num > 1) {
@@ -279,7 +279,7 @@ int qib_make_ud_req(struct rvt_qp *qp)
/* Construct the header. */
   

[PATCH v3 08/10] IB/qib: Remove qp and mr functionality from qib

[Dennis Dalessandro]

Remove qp and mr support from qib and use rdmavt. These two changes
cannot be reasonably be split apart into separate patches because they
depend on eachother in multple places. This paves the way to remove
even more functions in subsequent patches.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v2:
Handle removal of rvt_reg_phys_mr()

 drivers/infiniband/hw/qib/Makefile  |4 
 drivers/infiniband/hw/qib/qib.h |   14 -
 drivers/infiniband/hw/qib/qib_cq.c  |2 
 drivers/infiniband/hw/qib/qib_driver.c  |4 
 drivers/infiniband/hw/qib/qib_keys.c|  184 +-
 drivers/infiniband/hw/qib/qib_mmap.c|   18 -
 drivers/infiniband/hw/qib/qib_mr.c  |  490 ---
 drivers/infiniband/hw/qib/qib_qp.c  |   72 ++--
 drivers/infiniband/hw/qib/qib_rc.c  |   98 +++--
 drivers/infiniband/hw/qib/qib_ruc.c |   58 ++-
 drivers/infiniband/hw/qib/qib_sdma.c|8 
 drivers/infiniband/hw/qib/qib_srq.c |   28 +-
 drivers/infiniband/hw/qib/qib_uc.c  |8 
 drivers/infiniband/hw/qib/qib_ud.c  |   16 -
 drivers/infiniband/hw/qib/qib_verbs.c   |  176 --
 drivers/infiniband/hw/qib/qib_verbs.h   |  432 +++-
 drivers/infiniband/hw/qib/qib_verbs_mcast.c |8 
 17 files changed, 307 insertions(+), 1313 deletions(-)
 delete mode 100644 drivers/infiniband/hw/qib/qib_mr.c

diff --git a/drivers/infiniband/hw/qib/Makefile 
b/drivers/infiniband/hw/qib/Makefile
index 8a8f892..75140f5 100644
--- a/drivers/infiniband/hw/qib/Makefile
+++ b/drivers/infiniband/hw/qib/Makefile
@@ -1,8 +1,8 @@
 obj-$(CONFIG_INFINIBAND_QIB) += ib_qib.o
 
 ib_qib-y := qib_cq.o qib_diag.o qib_driver.o qib_eeprom.o \
-   qib_file_ops.o qib_fs.o qib_init.o qib_intr.o qib_keys.o \
-   qib_mad.o qib_mmap.o qib_mr.o qib_pcie.o qib_pio_copy.o \
+   qib_file_ops.o qib_fs.o qib_init.o qib_intr.o \
+   qib_mad.o qib_mmap.o qib_pcie.o qib_pio_copy.o \
qib_qp.o qib_qsfp.o qib_rc.o qib_ruc.o qib_sdma.o qib_srq.o \
qib_sysfs.o qib_twsi.o qib_tx.o qib_uc.o qib_ud.o \
qib_user_pages.o qib_user_sdma.o qib_verbs_mcast.o qib_iba7220.o \
diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index e610eaf..309b6f3 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -231,7 +231,7 @@ struct qib_ctxtdata {
/* ctxt rcvhdrq head offset */
u32 head;
/* lookaside fields */
-   struct qib_qp *lookaside_qp;
+   struct rvt_qp *lookaside_qp;
u32 lookaside_qpn;
/* QPs waiting for context processing */
struct list_head qp_wait_list;
@@ -241,7 +241,7 @@ struct qib_ctxtdata {
 #endif
 };
 
-struct qib_sge_state;
+struct rvt_sge_state;
 
 struct qib_sdma_txreq {
int flags;
@@ -259,14 +259,14 @@ struct qib_sdma_desc {
 
 struct qib_verbs_txreq {
struct qib_sdma_txreq   txreq;
-   struct qib_qp   *qp;
-   struct qib_swqe *wqe;
+   struct rvt_qp   *qp;
+   struct rvt_swqe *wqe;
u32 dwords;
u16 hdr_dwords;
u16 hdr_inx;
struct qib_pio_header   *align_buf;
-   struct qib_mregion  *mr;
-   struct qib_sge_state*ss;
+   struct rvt_mregion  *mr;
+   struct rvt_sge_state*ss;
 };
 
 #define QIB_SDMA_TXREQ_F_USELARGEBUF  0x1
@@ -1324,7 +1324,7 @@ void __qib_sdma_intr(struct qib_pportdata *);
 void qib_sdma_intr(struct qib_pportdata *);
 void qib_user_sdma_send_desc(struct qib_pportdata *dd,
struct list_head *pktlist);
-int qib_sdma_verbs_send(struct qib_pportdata *, struct qib_sge_state *,
+int qib_sdma_verbs_send(struct qib_pportdata *, struct rvt_sge_state *,
u32, struct qib_verbs_txreq *);
 /* ppd->sdma_lock should be locked before calling this. */
 int qib_sdma_make_progress(struct qib_pportdata *dd);
diff --git a/drivers/infiniband/hw/qib/qib_cq.c 
b/drivers/infiniband/hw/qib/qib_cq.c
index 2b45d0b..c1ea21e 100644
--- a/drivers/infiniband/hw/qib/qib_cq.c
+++ b/drivers/infiniband/hw/qib/qib_cq.c
@@ -466,7 +466,7 @@ int qib_resize_cq(struct ib_cq *ibcq, int cqe, struct 
ib_udata *udata)
 
if (cq->ip) {
struct qib_ibdev *dev = to_idev(ibcq->device);
-   struct qib_mmap_info *ip = cq->ip;
+   struct rvt_mmap_info *ip = cq->ip;
 
qib_update_mmap_info(dev, ip, sz, wc);
 
diff --git a/drivers/infiniband/hw/qib/qib_driver.c 
b/drivers/infiniband/hw/qib/qib_driver.c
index ae5a725..eafdee9 100644
--- a/drivers/infiniband/hw/qib/qib_driver.c
+++ b/drivers/infiniband/hw/qib/qib_driver.c
@@ -322,7 +322,7 @@

[PATCH v3 01/36] IB/rdmavt: Create module framework and handle driver registration

[Dennis Dalessandro]

This patch introduces the basics for a new module called rdma_vt. This new
driver is a software implementation of the InfiniBand verbs and aims to
replace the multiple implementations that exist and duplicate each others'
code.

While the call to actually register the device with the IB core happens in
rdma_vt, most of the work is still done in the drivers themselves. This
will be changing in a follow on patch this is just laying the groundwork
for this infrastructure.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
remove rdmavt specific driver version
move copyright above license text

 MAINTAINERS   |6 ++
 drivers/infiniband/Kconfig|2 +
 drivers/infiniband/Makefile   |1 
 drivers/infiniband/sw/Makefile|1 
 drivers/infiniband/sw/rdmavt/Kconfig  |6 ++
 drivers/infiniband/sw/rdmavt/Makefile |   10 
 drivers/infiniband/sw/rdmavt/vt.c |   83 +
 drivers/infiniband/sw/rdmavt/vt.h |   53 +
 include/rdma/rdma_vt.h|   70 
 9 files changed, 232 insertions(+), 0 deletions(-)
 create mode 100644 drivers/infiniband/sw/Makefile
 create mode 100644 drivers/infiniband/sw/rdmavt/Kconfig
 create mode 100644 drivers/infiniband/sw/rdmavt/Makefile
 create mode 100644 drivers/infiniband/sw/rdmavt/vt.c
 create mode 100644 drivers/infiniband/sw/rdmavt/vt.h
 create mode 100644 include/rdma/rdma_vt.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 978526c..6843301 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8907,6 +8907,12 @@ L:   rds-de...@oss.oracle.com (moderated for 
non-subscribers)
 S: Supported
 F: net/rds/
 
+RDMAVT - RDMA verbs software
+M: Dennis Dalessandro <dennis.dalessan...@intel.com>
+L: linux-rdma@vger.kernel.org
+S: Supported
+F: drivers/infiniband/sw/rdmavt
+
 READ-COPY UPDATE (RCU)
 M: "Paul E. McKenney" <paul...@linux.vnet.ibm.com>
 M: Josh Triplett <j...@joshtriplett.org>
diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index 8a8440c..d00d86d 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -82,4 +82,6 @@ source "drivers/infiniband/ulp/srpt/Kconfig"
 source "drivers/infiniband/ulp/iser/Kconfig"
 source "drivers/infiniband/ulp/isert/Kconfig"
 
+source "drivers/infiniband/sw/rdmavt/Kconfig"
+
 endif # INFINIBAND
diff --git a/drivers/infiniband/Makefile b/drivers/infiniband/Makefile
index dc21836..fad0b44 100644
--- a/drivers/infiniband/Makefile
+++ b/drivers/infiniband/Makefile
@@ -1,3 +1,4 @@
 obj-$(CONFIG_INFINIBAND)   += core/
 obj-$(CONFIG_INFINIBAND)   += hw/
 obj-$(CONFIG_INFINIBAND)   += ulp/
+obj-$(CONFIG_INFINIBAND)   += sw/
diff --git a/drivers/infiniband/sw/Makefile b/drivers/infiniband/sw/Makefile
new file mode 100644
index 000..988b6a0
--- /dev/null
+++ b/drivers/infiniband/sw/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_INFINIBAND_RDMAVT)+= rdmavt/
diff --git a/drivers/infiniband/sw/rdmavt/Kconfig 
b/drivers/infiniband/sw/rdmavt/Kconfig
new file mode 100644
index 000..11aa6a3
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/Kconfig
@@ -0,0 +1,6 @@
+config INFINIBAND_RDMAVT
+   tristate "RDMA verbs transport library"
+   depends on 64BIT
+   default m
+   ---help---
+   This is a common software verbs provider for RDMA networks.
diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
new file mode 100644
index 000..98a664d
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -0,0 +1,10 @@
+#
+# rdmavt driver
+#
+#
+#
+# Called from the kernel module build system.
+#
+obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
+
+rdmavt-y := vt.o
diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
new file mode 100644
index 000..aa325db
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -0,0 +1,83 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in sour

[PATCH v3 08/36] IB/rdmavt: Add pkey query stub

[Dennis Dalessandro]

The pkey table will reside in the rvt structure but it will be modified
only when the driver requests then rvt will simply read the value to return
in the query.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |   21 +
 include/rdma/rdma_vt.h|3 ++-
 2 files changed, 23 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 5ac241c..fc5372d 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -137,6 +137,26 @@ static int rvt_modify_port(struct ib_device *ibdev, u8 
port,
return -EOPNOTSUPP;
 }
 
+/**
+ * rvt_query_pkey - Return a pkey from the table at a given index
+ * @ibdev: Verbs IB dev
+ * @port: Port number
+ * @intex: Index into pkey table
+ *
+ * Returns 0 on failure pkey otherwise
+ */
+static int rvt_query_pkey(struct ib_device *ibdev, u8 port, u16 index,
+ u16 *pkey)
+{
+   /*
+* Driver will be responsible for keeping rvt_dev_info.pkey_table up to
+* date. This function will just return that value. There is no need to
+* lock, if a stale value is read and sent to the user so be it there is
+* no way to protect against that anyway.
+*/
+   return 0;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -154,6 +174,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, modify_device);
CHECK_DRIVER_OVERRIDE(rdi, query_port);
CHECK_DRIVER_OVERRIDE(rdi, modify_port);
+   CHECK_DRIVER_OVERRIDE(rdi, query_pkey);
 
/* DMA Operations */
rdi->ibdev.dma_ops =
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 2990e03..bf072a4 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -114,12 +114,13 @@ struct rvt_dev_info {
 * The driver will also be responsible for filling in certain members of
 * dparms.props
 */
-
struct ib_device ibdev;
 
/* Driver specific properties */
struct rvt_driver_params dparms;
 
+   /* PKey Table goes here */
+
/*
 * The work to create port files in /sys/class Infiniband is different
 * depending on the driver. This should not be extracted away and

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 07/36] IB/rdmavt: Add query and modify port stubs

[Dennis Dalessandro]

This patch adds the query and modify port stubs. The query will mostly
entail the driver returning everything in the ib_port_attr which will get
handed back to the verbs layer. The modify will need some API helpers in
the driver. The send_trap and post_mad_send are still issues to address.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Fix comment style for rvt_query_port
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/vt.c |   48 +
 1 files changed, 48 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index db14646..5ac241c 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -91,6 +91,52 @@ static int rvt_modify_device(struct ib_device *device,
return -EOPNOTSUPP;
 }
 
+/**
+ * rvt_query_port: Passes the query port call to the driver
+ * @ibdev: Verbs IB dev
+ * @port: port number
+ * @props: structure to hold returned properties
+ *
+ * Returns 0 on success
+ */
+static int rvt_query_port(struct ib_device *ibdev, u8 port,
+ struct ib_port_attr *props)
+{
+   /*
+* VT-DRIVER-API: query_port_state()
+* driver returns pretty much everything in ib_port_attr
+*/
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_modify_port
+ * @ibdev: Verbs IB dev
+ * @port: Port number
+ * @port_modify_mask: How to change the port
+ * @props: Structure to fill in
+ *
+ * Returns 0 on success
+ */
+static int rvt_modify_port(struct ib_device *ibdev, u8 port,
+  int port_modify_mask, struct ib_port_modify *props)
+{
+   /*
+* VT-DRIVER-API: set_link_state()
+* driver will set the link state using the IB enumeration
+*
+* VT-DRIVER-API: clear_qkey_violations()
+* clears driver private qkey counter
+*
+* VT-DRIVER-API: get_lid()
+* driver needs to return the LID
+*
+* TBD: send_trap() and post_mad_send() need examined to see where they
+* fit in.
+*/
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -106,6 +152,8 @@ int rvt_register_device(struct rvt_dev_info *rdi)
/* Dev Ops */
CHECK_DRIVER_OVERRIDE(rdi, query_device);
CHECK_DRIVER_OVERRIDE(rdi, modify_device);
+   CHECK_DRIVER_OVERRIDE(rdi, query_port);
+   CHECK_DRIVER_OVERRIDE(rdi, modify_port);
 
/* DMA Operations */
rdi->ibdev.dma_ops =

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 11/36] IB/rdmavt: Add queue pair function stubs

[Dennis Dalessandro]

Adds the stubs for create, modify, destroy and query functions for queue
pairs.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 -
 drivers/infiniband/sw/rdmavt/qp.c |  120 +
 drivers/infiniband/sw/rdmavt/qp.h |   67 ++
 drivers/infiniband/sw/rdmavt/vt.c |6 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 195 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/qp.c
 create mode 100644 drivers/infiniband/sw/rdmavt/qp.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index c6751bb..9a9a095 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,5 +7,5 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o dma.o pd.o
+rdmavt-y := vt.o dma.o pd.o qp.o
 
diff --git a/drivers/infiniband/sw/rdmavt/qp.c 
b/drivers/infiniband/sw/rdmavt/qp.c
new file mode 100644
index 000..a59f28d
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -0,0 +1,120 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "qp.h"
+
+/**
+ * rvt_create_qp - create a queue pair for a device
+ * @ibpd: the protection domain who's device we create the queue pair for
+ * @init_attr: the attributes of the queue pair
+ * @udata: user data for libibverbs.so
+ *
+ * Returns the queue pair on success, otherwise returns an errno.
+ *
+ * Called by the ib_create_qp() core verbs function.
+ */
+struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
+   struct ib_qp_init_attr *init_attr,
+   struct ib_udata *udata)
+{
+   /*
+* Queue pair creation is mostly an rvt issue. However, drivers have
+* their own unique idea of what queue pare numbers mean. For instance
+* there is a reserved range for PSM.
+*
+* VI-DRIVER-API: make_qpn()
+* Returns a valid QPN for verbs to use
+*/
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * qib_modify_qp - modify the attributes of a queue pair
+ * @ibqp: the queue pair who's attributes we're modifying
+ * @attr: the new attributes
+ * @attr_mask: the mask of attributes to modify
+ * @udata: user data for libibverbs.so
+ *
+ * Returns 0 on success, otherwise returns an errno.
+ */
+int rvt_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata)
+{
+   /*
+* VT-DRIVER-API: qp_mtu()
+* OPA devices have a per VL MTU t

[PATCH v3 19/36] IB/rdmavt: Add completion queue function stubs

[Dennis Dalessandro]

Create stubs for completion queue creation, polling,
resizing, calling for notification, and destroying.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 -
 drivers/infiniband/sw/rdmavt/cq.c |  113 +
 drivers/infiniband/sw/rdmavt/cq.h |   62 ++
 drivers/infiniband/sw/rdmavt/vt.c |7 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 184 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/cq.c
 create mode 100644 drivers/infiniband/sw/rdmavt/cq.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 6f530d1..00f0188 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o
+rdmavt-y := vt.o ah.o cq.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/cq.c 
b/drivers/infiniband/sw/rdmavt/cq.c
new file mode 100644
index 000..8d96194
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/cq.c
@@ -0,0 +1,113 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "cq.h"
+
+/**
+ * rvt_create_cq - create a completion queue
+ * @ibdev: the device this completion queue is attached to
+ * @attr: creation attributes
+ * @context: unused by the QLogic_IB driver
+ * @udata: user data for libibverbs.so
+ *
+ * Returns a pointer to the completion queue or negative errno values
+ * for failure.
+ *
+ * Called by ib_create_cq() in the generic verbs code.
+ */
+struct ib_cq *rvt_create_cq(struct ib_device *ibdev,
+   const struct ib_cq_init_attr *attr,
+   struct ib_ucontext *context,
+   struct ib_udata *udata)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * rvt_destroy_cq - destroy a completion queue
+ * @ibcq: the completion queue to destroy.
+ *
+ * Returns 0 for success.
+ *
+ * Called by ib_destroy_cq() in the generic verbs code.
+ */
+int rvt_destroy_cq(struct ib_cq *ibcq)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags notify_flags)
+{
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_resize_cq - change the size of the CQ
+ * @ibcq: the completion queue
+ *
+ * Returns 0 for success.
+ */
+int rvt_resize_cq(struct ib_cq *ibcq, int cqe, struct ib_udata *udata)
+{
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_poll

[PATCH v3 29/36] IB/rdmavt: Add AH to rdmavt

[Dennis Dalessandro]

From: Kamal Heib <kam...@mellanox.com>

Original patch is from Kamal Heib <kam...@mellanox.com>. It has
been split into three separate patches. This one for rdmavt,
a follow on for qib, and one for hfi1.

Create datastructure for address handle and implement the
create/destroy/modify/query of address handle for rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Kamal Heib <kam...@mellanox.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/ah.c |   98 +++--
 drivers/infiniband/sw/rdmavt/vt.c |5 ++
 include/rdma/rdma_vt.h|   17 ++
 3 files changed, 115 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/ah.c 
b/drivers/infiniband/sw/rdmavt/ah.c
index d368955..2519db9 100644
--- a/drivers/infiniband/sw/rdmavt/ah.c
+++ b/drivers/infiniband/sw/rdmavt/ah.c
@@ -45,7 +45,49 @@
  *
  */
 
+#include 
 #include "ah.h"
+#include "vt.h" /* for prints */
+
+/**
+ * rvt_check_ah - validate the attributes of AH
+ * @ibdev: the ib device
+ * @ah_attr: the attributes of the AH
+ */
+int rvt_check_ah(struct ib_device *ibdev,
+struct ib_ah_attr *ah_attr)
+{
+   int err;
+   struct ib_port_attr port_attr;
+   struct rvt_dev_info *rdi = ib_to_rvt(ibdev);
+   enum rdma_link_layer link = rdma_port_get_link_layer(ibdev,
+ah_attr->port_num);
+
+   err = ib_query_port(ibdev, ah_attr->port_num, _attr);
+   if (err)
+   return -EINVAL;
+   if (ah_attr->port_num < 1 ||
+   ah_attr->port_num > ibdev->phys_port_cnt)
+   return -EINVAL;
+   if (ah_attr->static_rate != IB_RATE_PORT_CURRENT &&
+   ib_rate_to_mbps(ah_attr->static_rate) < 0)
+   return -EINVAL;
+   if ((ah_attr->ah_flags & IB_AH_GRH) &&
+   ah_attr->grh.sgid_index >= port_attr.gid_tbl_len)
+   return -EINVAL;
+   if (link != IB_LINK_LAYER_ETHERNET) {
+   if (ah_attr->dlid == 0)
+   return -EINVAL;
+   if (ah_attr->dlid >= RVT_MULTICAST_LID_BASE &&
+   ah_attr->dlid != RVT_PERMISSIVE_LID &&
+   !(ah_attr->ah_flags & IB_AH_GRH))
+   return -EINVAL;
+   }
+   if (rdi->driver_f.check_ah(ibdev, ah_attr))
+   return -EINVAL;
+   return 0;
+}
+EXPORT_SYMBOL(rvt_check_ah);
 
 /**
  * rvt_create_ah - create an address handle
@@ -57,20 +99,68 @@
 struct ib_ah *rvt_create_ah(struct ib_pd *pd,
struct ib_ah_attr *ah_attr)
 {
-   return ERR_PTR(-EOPNOTSUPP);
+   struct rvt_ah *ah;
+   struct rvt_dev_info *dev = ib_to_rvt(pd->device);
+   unsigned long flags;
+
+   if (rvt_check_ah(pd->device, ah_attr))
+   return ERR_PTR(-EINVAL);
+
+   ah = kmalloc(sizeof(*ah), GFP_ATOMIC);
+   if (!ah)
+   return ERR_PTR(-ENOMEM);
+
+   spin_lock_irqsave(>n_ahs_lock, flags);
+   if (dev->n_ahs_allocated == dev->dparms.props.max_ah) {
+   spin_unlock(>n_ahs_lock);
+   kfree(ah);
+   return ERR_PTR(-ENOMEM);
+   }
+
+   dev->n_ahs_allocated++;
+   spin_unlock_irqrestore(>n_ahs_lock, flags);
+
+   ah->attr = *ah_attr;
+   atomic_set(>refcount, 0);
+
+   return >ibah;
 }
 
 int rvt_destroy_ah(struct ib_ah *ibah)
 {
-   return -EOPNOTSUPP;
+   struct rvt_dev_info *dev = ib_to_rvt(ibah->device);
+   struct rvt_ah *ah = ibah_to_rvtah(ibah);
+   unsigned long flags;
+
+   if (atomic_read(>refcount) != 0)
+   return -EBUSY;
+
+   spin_lock_irqsave(>n_ahs_lock, flags);
+   dev->n_ahs_allocated--;
+   spin_unlock_irqrestore(>n_ahs_lock, flags);
+
+   kfree(ah);
+
+   return 0;
 }
 
 int rvt_modify_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
 {
-   return -EOPNOTSUPP;
+   struct rvt_ah *ah = ibah_to_rvtah(ibah);
+
+   if (rvt_check_ah(ibah->device, ah_attr))
+   return -EINVAL;
+
+   ah->attr = *ah_attr;
+
+   return 0;
 }
 
 int rvt_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
 {
-   return -EOPNOTSUPP;
+   struct rvt_ah *ah = ibah_to_rvtah(ibah);
+
+   *ah_attr = ah->attr;
+
+   return 0;
 }
diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index e92af9c..7dab0ca 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -221,7 +221,8 @@ int rvt_register_device(struct rvt_dev_info *rdi)
 
if ((!rdi->driver_f.port_callback) ||
(!rdi->driver_f.get_card_name) ||
-   (!rdi->driver_f.

[PATCH v3 31/36] IB/rdmavt: Add an ibport data structure to rdmavt

[Dennis Dalessandro]

Converge the ibport data structures of qib and hfi1 into a common ib
port structure. Also provides a place to keep track of these ports
in case rdmavt needs it. Along with this goes an attach and detach
function for drivers to use to notify rdmavt of the ports.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Harish Chegondi <harish.chego...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |   24 +
 include/rdma/rdma_vt.h|   66 -
 2 files changed, 89 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 7dab0ca..44de280 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -300,6 +300,19 @@ int rvt_register_device(struct rvt_dev_info *rdi)
spin_lock_init(>n_pds_lock);
rdi->n_pds_allocated = 0;
 
+   if (rdi->dparms.nports) {
+   rdi->ports = kcalloc(rdi->dparms.nports,
+sizeof(struct rvt_ibport **),
+GFP_KERNEL);
+   if (!rdi->ports) {
+   rvt_pr_err(rdi, "Could not allocate port mem.\n");
+   ret = -ENOMEM;
+   goto bail_mr;
+   }
+   } else {
+   rvt_pr_warn(rdi, "Driver says it has no ports.\n");
+   }
+
/* We are now good to announce we exist */
ret =  ib_register_device(>ibdev, rdi->driver_f.port_callback);
if (ret) {
@@ -327,3 +340,14 @@ void rvt_unregister_device(struct rvt_dev_info *rdi)
rvt_mr_exit(rdi);
 }
 EXPORT_SYMBOL(rvt_unregister_device);
+
+/*
+ * Keep track of a list of ports. No need to have a detach port.
+ * They persist until the driver goes away.
+ */
+void rvt_attach_port(struct rvt_dev_info *rdi, struct rvt_ibport *port,
+int portnum)
+{
+   rdi->ports[portnum] = port;
+}
+EXPORT_SYMBOL(rvt_attach_port);
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index fcf3ec0..a3d6a5b 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -53,6 +53,8 @@
  * rdmavt layer.
  */
 
+#include 
+#include 
 #include "ib_verbs.h"
 
 #define RVT_MULTICAST_LID_BASE 0xC000
@@ -359,6 +361,65 @@ struct rvt_srq {
 
 /* End QP section */
 
+struct rvt_ibport {
+   struct rvt_qp __rcu *qp[2];
+   struct ib_mad_agent *send_agent;/* agent for SMI (traps) */
+   struct rb_root mcast_tree;
+   spinlock_t lock;/* protect changes in this struct */
+
+   /* non-zero when timer is set */
+   unsigned long mkey_lease_timeout;
+   unsigned long trap_timeout;
+   __be64 gid_prefix;  /* in network order */
+   __be64 mkey;
+   u64 tid;
+   u32 port_cap_flags;
+   u32 pma_sample_start;
+   u32 pma_sample_interval;
+   __be16 pma_counter_select[5];
+   u16 pma_tag;
+   u16 mkey_lease_period;
+   u16 sm_lid;
+   u8 sm_sl;
+   u8 mkeyprot;
+   u8 subnet_timeout;
+   u8 vl_high_limit;
+
+   /*
+* Driver is expected to keep these up to date. These
+* counters are informational only and not required to be
+* completely accurate.
+*/
+   u64 n_rc_resends;
+   u64 n_seq_naks;
+   u64 n_rdma_seq;
+   u64 n_rnr_naks;
+   u64 n_other_naks;
+   u64 n_loop_pkts;
+   u64 n_pkt_drops;
+   u64 n_vl15_dropped;
+   u64 n_rc_timeouts;
+   u64 n_dmawait;
+   u64 n_unaligned;
+   u64 n_rc_dupreq;
+   u64 n_rc_seqnak;
+   u16 pkey_violations;
+   u16 qkey_violations;
+   u16 mkey_violations;
+
+   /* Hot-path per CPU counters to avoid cacheline trading to update */
+   u64 z_rc_acks;
+   u64 z_rc_qacks;
+   u64 z_rc_delayed_comp;
+   u64 __percpu *rc_acks;
+   u64 __percpu *rc_qacks;
+   u64 __percpu *rc_delayed_comp;
+
+   void *priv; /* driver private data */
+
+   /* TODO: Move sm_ah and smi_ah into here as well*/
+};
+
 /*
  * Things that are driver specific, module parameters in hfi1 and qib
  */
@@ -403,6 +464,7 @@ struct rvt_driver_params {
 * For instance special module parameters. Goes here.
 */
unsigned int lkey_table_size;
+   int nports;
 };
 
 /*
@@ -465,6 +527,7 @@ struct rvt_dev_info {
spinlock_t n_ahs_lock; /* Protect ah allocated count */
 
int flags;
+   struct rvt_ibport **ports;
 };
 
 static inline struct rvt_pd *ibpd_to_rvtpd(struct ib_pd *ibpd)
@@ -501,9 +564,10 @@ static inline struct rvt_srq *ibsrq_to_rvtsrq(struct 
ib_srq *ibsrq)
 int rvt_register_device(struct rvt_dev_info *rvd);
 void rvt_unregister_device(struct rvt_dev_info *rvd);
 int rvt_check_ah(struct ib_device *ibdev, struct ib_ah_attr *ah_attr);
+void rvt_attach_port(

[PATCH v3 36/36] IB/rdmavt: Add pkey support

[Dennis Dalessandro]

Add pkey table in rdi per port data structure. Also bring in related pkey
functions. Drivers will still be responsible for allocating and
maintaining the pkey table. However they need to tell rdmavt where to find
the pkey table. We can not move the pkey table up into rdmavt because
drivers need to manipulate this long before registering with it.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |   46 -
 include/rdma/rdma_vt.h|   38 +++
 2 files changed, 64 insertions(+), 20 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index ab4105a..18b5f43 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -154,6 +154,17 @@ static int rvt_query_pkey(struct ib_device *ibdev, u8 
port, u16 index,
 * lock, if a stale value is read and sent to the user so be it there is
 * no way to protect against that anyway.
 */
+   struct rvt_dev_info *rdi = ib_to_rvt(ibdev);
+   int port_index;
+
+   if (index >= rvt_get_npkeys(rdi))
+   return -EINVAL;
+
+   port_index = port - 1; /* IB ports start at 1 our array at 0 */
+   if ((port_index < 0) || (port_index >= rdi->dparms.nports))
+   return -EINVAL;
+
+   *pkey = rvt_get_pkey(rdi, port_index, index);
return 0;
 }
 
@@ -227,19 +238,6 @@ int rvt_register_device(struct rvt_dev_info *rdi)
return -EINVAL;
}
 
-   if (!rdi->dparms.nports) {
-   rvt_pr_err(rdi, "Driver says it has no ports.\n");
-   return -EINVAL;
-   }
-
-   rdi->ports = kcalloc(rdi->dparms.nports,
-sizeof(struct rvt_ibport **),
-GFP_KERNEL);
-   if (!rdi->ports) {
-   rvt_pr_err(rdi, "Could not allocate port mem.\n");
-   return -ENOMEM;
-   }
-
/* Once we get past here we can use the rvt_pr macros */
rvt_mmap_init(rdi);
 
@@ -355,9 +353,25 @@ EXPORT_SYMBOL(rvt_unregister_device);
  * Keep track of a list of ports. No need to have a detach port.
  * They persist until the driver goes away.
  */
-void rvt_attach_port(struct rvt_dev_info *rdi, struct rvt_ibport *port,
-int portnum)
+int rvt_init_port(struct rvt_dev_info *rdi, struct rvt_ibport *port,
+ int portnum, u16 *pkey_table)
 {
+   if (!rdi->dparms.nports) {
+   rvt_pr_err(rdi, "Driver says it has no ports.\n");
+   return -EINVAL;
+   }
+
+   rdi->ports = kcalloc(rdi->dparms.nports,
+sizeof(struct rvt_ibport **),
+GFP_KERNEL);
+   if (!rdi->ports) {
+   rvt_pr_err(rdi, "Could not allocate port mem.\n");
+   return -ENOMEM;
+   }
+
rdi->ports[portnum] = port;
+   rdi->ports[portnum]->pkey_table = pkey_table;
+
+   return 0;
 }
-EXPORT_SYMBOL(rvt_attach_port);
+EXPORT_SYMBOL(rvt_init_port);
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index fd25d23..3a78f20 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -69,6 +69,8 @@
 #define RVT_FLAG_QP_INIT_DRIVER BIT(2)
 #define RVT_FLAG_CQ_INIT_DRIVER BIT(3)
 
+#define RVT_MAX_PKEY_VALUES 16
+
 struct rvt_ibport {
struct rvt_qp __rcu *qp[2];
struct ib_mad_agent *send_agent;/* agent for SMI (traps) */
@@ -125,6 +127,14 @@ struct rvt_ibport {
 
void *priv; /* driver private data */
 
+   /*
+* The pkey table is allocated and maintained by the driver. Drivers
+* need to have access to this before registering with rdmav. However
+* rdmavt will need access to it so drivers need to proviee this during
+* the attach port API call.
+*/
+   u16 *pkey_table;
+
/* TODO: Move sm_ah and smi_ah into here as well*/
 };
 
@@ -178,6 +188,7 @@ struct rvt_driver_params {
int qpn_res_start;
int qpn_res_end;
int nports;
+   int npkeys;
u8 qos_shift;
 };
 
@@ -238,8 +249,6 @@ struct rvt_dev_info {
struct rvt_mregion __rcu *dma_mr;
struct rvt_lkey_table lkey_table;
 
-   /* PKey Table goes here */
-
/* Driver specific helper functions */
struct rvt_driver_provided driver_f;
 
@@ -282,11 +291,32 @@ static inline struct rvt_srq *ibsrq_to_rvtsrq(struct 
ib_srq *ibsrq)
return container_of(ibsrq, struct rvt_srq, ibsrq);
 }
 
+static inline unsigned rvt_get_npkeys(struct rvt_dev_info *rdi)
+{
+   /*
+* All ports have same number of pkeys.
+*/
+   return rdi->dparms.npkeys;
+}
+
+/*
+ * Retur

[PATCH v3 32/36] IB/rdmavt: Add driver notification for new AH

[Dennis Dalessandro]

Drivers may need to do some work once an address handle has been
created. Add a driver function for this purpose.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Harish Chegondi <harish.chego...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/ah.c |7 +-
 include/rdma/rdma_vt.h|   41 +++--
 2 files changed, 30 insertions(+), 18 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/ah.c 
b/drivers/infiniband/sw/rdmavt/ah.c
index 2519db9..621afc3 100644
--- a/drivers/infiniband/sw/rdmavt/ah.c
+++ b/drivers/infiniband/sw/rdmavt/ah.c
@@ -83,8 +83,8 @@ int rvt_check_ah(struct ib_device *ibdev,
!(ah_attr->ah_flags & IB_AH_GRH))
return -EINVAL;
}
-   if (rdi->driver_f.check_ah(ibdev, ah_attr))
-   return -EINVAL;
+   if (rdi->driver_f.check_ah)
+   return rdi->driver_f.check_ah(ibdev, ah_attr);
return 0;
 }
 EXPORT_SYMBOL(rvt_check_ah);
@@ -123,6 +123,9 @@ struct ib_ah *rvt_create_ah(struct ib_pd *pd,
ah->attr = *ah_attr;
atomic_set(>refcount, 0);
 
+   if (dev->driver_f.notify_new_ah)
+   dev->driver_f.notify_new_ah(pd->device, ah_attr, ah);
+
return >ibah;
 }
 
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index a3d6a5b..ef66d2b 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -467,9 +467,21 @@ struct rvt_driver_params {
int nports;
 };
 
-/*
- * Functions that drivers are required to support
- */
+/* Protection domain */
+struct rvt_pd {
+   struct ib_pd ibpd;
+   int user;   /* non-zero if created from user space */
+};
+
+/* Address handle */
+struct rvt_ah {
+   struct ib_ah ibah;
+   struct ib_ah_attr attr;
+   atomic_t refcount;
+   u8 vl;
+   u8 log_pmtu;
+};
+
 struct rvt_dev_info;
 struct rvt_driver_provided {
/*
@@ -478,23 +490,20 @@ struct rvt_driver_provided {
 * instead drivers are responsible for setting the correct callback for
 * this.
 */
+
+   /* ---*/
+   /* Required functions */
+   /* ---*/
int (*port_callback)(struct ib_device *, u8, struct kobject *);
const char * (*get_card_name)(struct rvt_dev_info *rdi);
struct pci_dev * (*get_pci_dev)(struct rvt_dev_info *rdi);
-   int (*check_ah)(struct ib_device *, struct ib_ah_attr *);
-};
 
-/* Protection domain */
-struct rvt_pd {
-   struct ib_pd ibpd;
-   int user;   /* non-zero if created from user space */
-};
-
-/* Address handle */
-struct rvt_ah {
-   struct ib_ah ibah;
-   struct ib_ah_attr attr;
-   atomic_t refcount;
+   /**/
+   /* Optional functions */
+   /**/
+   int (*check_ah)(struct ib_device *, struct ib_ah_attr *);
+   void (*notify_new_ah)(struct ib_device *, struct ib_ah_attr *,
+ struct rvt_ah *);
 };
 
 struct rvt_dev_info {

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 28/36] IB/rdmavt: Add common LID defines to rdmavt

[Dennis Dalessandro]

From: Kamal Heib <kam...@mellanox.com>

Original patch is from Kamal Heib <kam...@mellanox.com>. It has
been split into separate patches.

This patch adds RVT_PERMISSIVE_LID and RVT_MULTICAST_LID_BASE
to rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Kamal Heib <kam...@mellanox.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 include/rdma/rdma_vt.h |3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 9a47957..dbb45bc 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -55,6 +55,9 @@
 
 #include "ib_verbs.h"
 
+#define RVT_MULTICAST_LID_BASE 0xC000
+#define RVT_PERMISSIVE_LID 0x
+
 /*
  * For some of the IBTA objects there will likely be some
  * initializations required. We need flags to determine whether it is OK

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 06/10] IB/qib: Remove driver specific members from qib qp type

[Dennis Dalessandro]

In preparation for moving the queue pair data structure to rdmavt the
members of the driver specific queue pairs which are not common need to be
pushed off to a private driver structure. This structure will be available
in the queue pair once moved to rdmavt as a void pointer. This patch while
not adding a lot of value in and of itself is a prerequisite to move the
queue pair out of the drivers and into rdmavt.

The driver specific, private queue pair data structure should condense as
more of the send side code moves to rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_qp.c|   75 -
 drivers/infiniband/hw/qib/qib_rc.c|7 ++-
 drivers/infiniband/hw/qib/qib_ruc.c   |   18 +---
 drivers/infiniband/hw/qib/qib_sdma.c  |   17 +--
 drivers/infiniband/hw/qib/qib_uc.c|7 ++-
 drivers/infiniband/hw/qib/qib_ud.c|   22 +-
 drivers/infiniband/hw/qib/qib_verbs.c |   61 +--
 drivers/infiniband/hw/qib/qib_verbs.h |   26 ---
 8 files changed, 145 insertions(+), 88 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index 116d323..7e570be 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -370,10 +370,11 @@ struct qib_qp *qib_lookup_qpn(struct qib_ibport *ibp, u32 
qpn)
  */
 static void qib_reset_qp(struct qib_qp *qp, enum ib_qp_type type)
 {
+   struct qib_qp_priv *priv = qp->priv;
qp->remote_qpn = 0;
qp->qkey = 0;
qp->qp_access_flags = 0;
-   atomic_set(>s_dma_busy, 0);
+   atomic_set(>s_dma_busy, 0);
qp->s_flags &= QIB_S_SIGNAL_REQ_WR;
qp->s_hdrwords = 0;
qp->s_wqe = NULL;
@@ -473,6 +474,7 @@ static void clear_mr_refs(struct qib_qp *qp, int clr_sends)
  */
 int qib_error_qp(struct qib_qp *qp, enum ib_wc_status err)
 {
+   struct qib_qp_priv *priv = qp->priv;
struct qib_ibdev *dev = to_idev(qp->ibqp.device);
struct ib_wc wc;
int ret = 0;
@@ -491,9 +493,9 @@ int qib_error_qp(struct qib_qp *qp, enum ib_wc_status err)
qp->s_flags &= ~QIB_S_ANY_WAIT_SEND;
 
spin_lock(>pending_lock);
-   if (!list_empty(>iowait) && !(qp->s_flags & QIB_S_BUSY)) {
+   if (!list_empty(>iowait) && !(qp->s_flags & QIB_S_BUSY)) {
qp->s_flags &= ~QIB_S_ANY_WAIT_IO;
-   list_del_init(>iowait);
+   list_del_init(>iowait);
}
spin_unlock(>pending_lock);
 
@@ -503,9 +505,9 @@ int qib_error_qp(struct qib_qp *qp, enum ib_wc_status err)
qib_put_mr(qp->s_rdma_mr);
qp->s_rdma_mr = NULL;
}
-   if (qp->s_tx) {
-   qib_put_txreq(qp->s_tx);
-   qp->s_tx = NULL;
+   if (priv->s_tx) {
+   qib_put_txreq(priv->s_tx);
+   priv->s_tx = NULL;
}
}
 
@@ -571,6 +573,7 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
 {
struct qib_ibdev *dev = to_idev(ibqp->device);
struct qib_qp *qp = to_iqp(ibqp);
+   struct qib_qp_priv *priv = qp->priv;
enum ib_qp_state cur_state, new_state;
struct ib_event ev;
int lastwqe = 0;
@@ -698,19 +701,20 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
if (qp->state != IB_QPS_RESET) {
qp->state = IB_QPS_RESET;
spin_lock(>pending_lock);
-   if (!list_empty(>iowait))
-   list_del_init(>iowait);
+   if (!list_empty(>iowait))
+   list_del_init(>iowait);
spin_unlock(>pending_lock);
qp->s_flags &= ~(QIB_S_TIMER | QIB_S_ANY_WAIT);
spin_unlock(>s_lock);
spin_unlock_irq(>r_lock);
/* Stop the sending work queue and retry timer */
-   cancel_work_sync(>s_work);
+   cancel_work_sync(>s_work);
del_timer_sync(>s_timer);
-   wait_event(qp->wait_dma, !atomic_read(>s_dma_busy));
-   if (qp->s_tx) {
-   qib_put_txreq(qp->s_tx);
-   qp->s_tx = NULL;
+   wait_event(priv->wait_dma,
+  !atomic_read(>s_dma_busy));
+   if (priv->s_tx) {
+ 

[PATCH v3 07/10] IB/qib: Add device specific info prints

[Dennis Dalessandro]

Implement get_card_name and get_pci_dev helper functions for rdmavt
for qib.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib.h|2 ++
 drivers/infiniband/hw/qib/qib_driver.c |   16 
 drivers/infiniband/hw/qib/qib_verbs.c  |2 ++
 3 files changed, 20 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index 0e68e1f..e610eaf 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -1455,6 +1455,8 @@ u64 qib_sps_ints(void);
 dma_addr_t qib_map_page(struct pci_dev *, struct page *, unsigned long,
  size_t, int);
 const char *qib_get_unit_name(int unit);
+const char *qib_get_card_name(struct rvt_dev_info *rdi);
+struct pci_dev *qib_get_pci_dev(struct rvt_dev_info *rdi);
 
 /*
  * Flush write combining store buffers (if present) and perform a write
diff --git a/drivers/infiniband/hw/qib/qib_driver.c 
b/drivers/infiniband/hw/qib/qib_driver.c
index 57b19ea..ae5a725 100644
--- a/drivers/infiniband/hw/qib/qib_driver.c
+++ b/drivers/infiniband/hw/qib/qib_driver.c
@@ -90,6 +90,22 @@ const char *qib_get_unit_name(int unit)
return iname;
 }
 
+const char *qib_get_card_name(struct rvt_dev_info *rdi)
+{
+   struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi);
+   struct qib_devdata *dd = container_of(ibdev,
+ struct qib_devdata, verbs_dev);
+   return qib_get_unit_name(dd->unit);
+}
+
+struct pci_dev *qib_get_pci_dev(struct rvt_dev_info *rdi)
+{
+   struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi);
+   struct qib_devdata *dd = container_of(ibdev,
+ struct qib_devdata, verbs_dev);
+   return dd->pcidev;
+}
+
 /*
  * Return count of units with at least one port ACTIVE.
  */
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index a912c92..bedc58a 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -2239,6 +2239,8 @@ int qib_register_ib_device(struct qib_devdata *dd)
 * Fill in rvt info object.
 */
dd->verbs_dev.rdi.driver_f.port_callback = qib_create_port_files;
+   dd->verbs_dev.rdi.driver_f.get_card_name = qib_get_card_name;
+   dd->verbs_dev.rdi.driver_f.get_pci_dev = qib_get_pci_dev;
dd->verbs_dev.rdi.dparms.props.max_pd = ib_qib_max_pds;
dd->verbs_dev.rdi.flags = (RVT_FLAG_MR_INIT_DRIVER |
   RVT_FLAG_QP_INIT_DRIVER |

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 05/10] IB/qib: Use rdmavt lid defines in qib

[Dennis Dalessandro]

Original patch for AH changes from Kamal Heib <kam...@mellanox.com>, split
apart from original. This patch also removes the qib specific multicast
lid base and permissive lid defines since they are no longer needed.

Use common LID defines in qib driver.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Kamal Heib <kam...@mellanox.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_common.h |2 --
 drivers/infiniband/hw/qib/qib_qp.c |1 +
 drivers/infiniband/hw/qib/qib_verbs.c  |4 ++--
 3 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_common.h 
b/drivers/infiniband/hw/qib/qib_common.h
index 4fb78ab..1d87ec0 100644
--- a/drivers/infiniband/hw/qib/qib_common.h
+++ b/drivers/infiniband/hw/qib/qib_common.h
@@ -742,14 +742,12 @@ struct qib_tid_session_member {
 #define SIZE_OF_CRC 1
 
 #define QIB_DEFAULT_P_KEY 0x
-#define QIB_PERMISSIVE_LID 0x
 #define QIB_AETH_CREDIT_SHIFT 24
 #define QIB_AETH_CREDIT_MASK 0x1F
 #define QIB_AETH_CREDIT_INVAL 0x1F
 #define QIB_PSN_MASK 0xFF
 #define QIB_MSN_MASK 0xFF
 #define QIB_QPN_MASK 0xFF
-#define QIB_MULTICAST_LID_BASE 0xC000
 #define QIB_EAGER_TID_ID QLOGIC_IB_I_TID_MASK
 #define QIB_MULTICAST_QPN 0xFF
 
diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index e6544dc..116d323 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -35,6 +35,7 @@
 #include 
 #include 
 #include 
+#include 
 #ifdef CONFIG_DEBUG_FS
 #include 
 #endif
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index 2578473..90d0f77 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -1754,8 +1754,8 @@ static int qib_query_gid(struct ib_device *ibdev, u8 port,
 int qib_check_ah(struct ib_device *ibdev, struct ib_ah_attr *ah_attr)
 {
/* A multicast address requires a GRH (see ch. 8.4.1). */
-   if (ah_attr->dlid >= QIB_MULTICAST_LID_BASE &&
-   ah_attr->dlid != QIB_PERMISSIVE_LID &&
+   if (ah_attr->dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE) &&
+   ah_attr->dlid != be16_to_cpu(IB_LID_PERMISSIVE) &&
!(ah_attr->ah_flags & IB_AH_GRH))
goto bail;
if ((ah_attr->ah_flags & IB_AH_GRH) &&

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 01/10] IB/qib: Begin to use rdmavt for verbs

[Dennis Dalessandro]

This patch begins to make use of rdmavt by registering with it and
providing access to the header files. This is just the beginning of
rdmavt support in qib.

Most functionality is still being done in the driver, set flags so that
rdmavt will let qib continue to handle mr, qp, and cq init.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/Kconfig |2 +-
 drivers/infiniband/hw/qib/qib.h   |1 +
 drivers/infiniband/hw/qib/qib_init.c  |5 +++--
 drivers/infiniband/hw/qib/qib_intr.c  |2 +-
 drivers/infiniband/hw/qib/qib_mad.c   |5 +++--
 drivers/infiniband/hw/qib/qib_sysfs.c |   20 ++--
 drivers/infiniband/hw/qib/qib_verbs.c |   19 ++-
 drivers/infiniband/hw/qib/qib_verbs.h |8 ++--
 8 files changed, 39 insertions(+), 23 deletions(-)

diff --git a/drivers/infiniband/hw/qib/Kconfig 
b/drivers/infiniband/hw/qib/Kconfig
index 495be09..e0fdb92 100644
--- a/drivers/infiniband/hw/qib/Kconfig
+++ b/drivers/infiniband/hw/qib/Kconfig
@@ -1,6 +1,6 @@
 config INFINIBAND_QIB
tristate "Intel PCIe HCA support"
-   depends on 64BIT
+   depends on 64BIT && INFINIBAND_RDMAVT
---help---
This is a low-level driver for Intel PCIe QLE InfiniBand host
channel adapters.  This driver does not support the Intel
diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index 7df16f7..0e68e1f 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -52,6 +52,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "qib_common.h"
 #include "qib_verbs.h"
diff --git a/drivers/infiniband/hw/qib/qib_init.c 
b/drivers/infiniband/hw/qib/qib_init.c
index 4ff340f..47190f1 100644
--- a/drivers/infiniband/hw/qib/qib_init.c
+++ b/drivers/infiniband/hw/qib/qib_init.c
@@ -42,6 +42,7 @@
 #ifdef CONFIG_INFINIBAND_QIB_DCA
 #include 
 #endif
+#include 
 
 #include "qib.h"
 #include "qib_common.h"
@@ -1081,7 +1082,7 @@ void qib_free_devdata(struct qib_devdata *dd)
qib_dbg_ibdev_exit(>verbs_dev);
 #endif
free_percpu(dd->int_counter);
-   ib_dealloc_device(>verbs_dev.ibdev);
+   ib_dealloc_device(>verbs_dev.rdi.ibdev);
 }
 
 u64 qib_int_counter(struct qib_devdata *dd)
@@ -1171,7 +1172,7 @@ struct qib_devdata *qib_alloc_devdata(struct pci_dev 
*pdev, size_t extra)
 bail:
if (!list_empty(>list))
list_del_init(>list);
-   ib_dealloc_device(>verbs_dev.ibdev);
+   ib_dealloc_device(>verbs_dev.rdi.ibdev);
return ERR_PTR(ret);
 }
 
diff --git a/drivers/infiniband/hw/qib/qib_intr.c 
b/drivers/infiniband/hw/qib/qib_intr.c
index 086616d..a014fd4 100644
--- a/drivers/infiniband/hw/qib/qib_intr.c
+++ b/drivers/infiniband/hw/qib/qib_intr.c
@@ -74,7 +74,7 @@ static void signal_ib_event(struct qib_pportdata *ppd, enum 
ib_event_type ev)
struct ib_event event;
struct qib_devdata *dd = ppd->dd;
 
-   event.device = >verbs_dev.ibdev;
+   event.device = >verbs_dev.rdi.ibdev;
event.element.port_num = ppd->port;
event.event = ev;
ib_dispatch_event();
diff --git a/drivers/infiniband/hw/qib/qib_mad.c 
b/drivers/infiniband/hw/qib/qib_mad.c
index 9625e7c..c65d3aa 100644
--- a/drivers/infiniband/hw/qib/qib_mad.c
+++ b/drivers/infiniband/hw/qib/qib_mad.c
@@ -1028,7 +1028,7 @@ static int set_pkeys(struct qib_devdata *dd, u8 port, u16 
*pkeys)
(void) dd->f_set_ib_cfg(ppd, QIB_IB_CFG_PKEYS, 0);
 
event.event = IB_EVENT_PKEY_CHANGE;
-   event.device = >verbs_dev.ibdev;
+   event.device = >verbs_dev.rdi.ibdev;
event.element.port_num = port;
ib_dispatch_event();
}
@@ -2483,7 +2483,8 @@ int qib_create_agents(struct qib_ibdev *dev)
 
for (p = 0; p < dd->num_pports; p++) {
ibp = >pport[p].ibport_data;
-   agent = ib_register_mad_agent(>ibdev, p + 1, IB_QPT_SMI,
+   agent = ib_register_mad_agent(>rdi.ibdev, p + 1,
+ IB_QPT_SMI,
  NULL, 0, send_handler,
  NULL, NULL, 0);
if (IS_ERR(agent)) {
diff --git a/drivers/infiniband/hw/qib/qib_sysfs.c 
b/drivers/infiniband/hw/qib/qib_sysfs.c
index 81f56cd..72a160e 100644
--- a/drivers/infiniband/hw/qib/qib_sysfs.c
+++ b/drivers/infiniband/hw/qib/qib_sysfs.c
@@ -502,7 +502,7 @@ static ssize_t show_rev(struct device *device, struct 
device_attribute *attr,
char *buf)
 {
struct qib_ibdev *dev =
-   container_of(device, struct qib_ibdev, ibdev.dev);
+   container_of(

[PATCH v3 02/10] IB/qib: Remove dma.c and use rdmavt version of dma functions

[Dennis Dalessandro]

This patch removes the qib_dma.c file and uses the version which has been
added to rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/Makefile|2 +-
 drivers/infiniband/hw/qib/qib_verbs.c |3 ++-
 drivers/infiniband/hw/qib/qib_verbs.h |2 --
 3 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/hw/qib/Makefile 
b/drivers/infiniband/hw/qib/Makefile
index 57f8103..8a8f892 100644
--- a/drivers/infiniband/hw/qib/Makefile
+++ b/drivers/infiniband/hw/qib/Makefile
@@ -1,6 +1,6 @@
 obj-$(CONFIG_INFINIBAND_QIB) += ib_qib.o
 
-ib_qib-y := qib_cq.o qib_diag.o qib_dma.o qib_driver.o qib_eeprom.o \
+ib_qib-y := qib_cq.o qib_diag.o qib_driver.o qib_eeprom.o \
qib_file_ops.o qib_fs.o qib_init.o qib_intr.o qib_keys.o \
qib_mad.o qib_mmap.o qib_mr.o qib_pcie.o qib_pio_copy.o \
qib_qp.o qib_qsfp.o qib_rc.o qib_ruc.o qib_sdma.o qib_srq.o \
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index c3af43d..fb6ef2c 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -41,6 +41,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "qib.h"
 #include "qib_common.h"
@@ -2268,7 +2269,7 @@ int qib_register_ib_device(struct qib_devdata *dd)
ibdev->detach_mcast = qib_multicast_detach;
ibdev->process_mad = qib_process_mad;
ibdev->mmap = qib_mmap;
-   ibdev->dma_ops = _dma_mapping_ops;
+   ibdev->dma_ops = NULL;
ibdev->get_port_immutable = qib_port_immutable;
 
snprintf(ibdev->node_desc, sizeof(ibdev->node_desc),
diff --git a/drivers/infiniband/hw/qib/qib_verbs.h 
b/drivers/infiniband/hw/qib/qib_verbs.h
index e175301..8ac0724 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.h
+++ b/drivers/infiniband/hw/qib/qib_verbs.h
@@ -1182,6 +1182,4 @@ extern unsigned int ib_qib_max_srq_wrs;
 
 extern const u32 ib_qib_rnr_table[];
 
-extern struct ib_dma_mapping_ops qib_dma_mapping_ops;
-
 #endif  /* QIB_VERBS_H */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/6] IB/rdmavt: Export reset_qp in rdmavt

[Dennis Dalessandro]

Until all queue pair functionality is moved to rdmavt we need to provide
access to the reset function. This is only temporary and will be reverted
back to a static, non exported function in the end.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Harish Chegondi <harish.chego...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/qp.c |7 ---
 include/rdma/rdma_vt.h|3 +++
 include/rdma/rdmavt_qp.h  |1 +
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.c 
b/drivers/infiniband/sw/rdmavt/qp.c
index 7d1f02e..44485ad 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -357,8 +357,8 @@ static void free_qpn(struct rvt_qpn_table *qpt, u32 qpn)
  * @qp: the QP to reset
  * @type: the QP type
  */
-static void reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,
-enum ib_qp_type type)
+void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,
+ enum ib_qp_type type)
 {
qp->remote_qpn = 0;
qp->qkey = 0;
@@ -409,6 +409,7 @@ static void reset_qp(struct rvt_dev_info *rdi, struct 
rvt_qp *qp,
}
qp->r_sge.num_sge = 0;
 }
+EXPORT_SYMBOL(rvt_reset_qp);
 
 /**
  * rvt_create_qp - create a queue pair for a device
@@ -543,7 +544,7 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
}
qp->ibqp.qp_num = err;
qp->port_num = init_attr->port_num;
-   reset_qp(rdi, qp, init_attr->qp_type);
+   rvt_reset_qp(rdi, qp, init_attr->qp_type);
break;
 
default:
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 3bdeac7..e412e67 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -338,4 +338,7 @@ struct rvt_mmap_info *rvt_create_mmap_info(struct 
rvt_dev_info *rdi,
 void rvt_update_mmap_info(struct rvt_dev_info *rdi, struct rvt_mmap_info *ip,
  u32 size, void *obj);
 
+/* Temporary export */
+void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,
+ enum ib_qp_type type);
 #endif  /* DEF_RDMA_VT_H */
diff --git a/include/rdma/rdmavt_qp.h b/include/rdma/rdmavt_qp.h
index 1aa8b5b..bce0a03 100644
--- a/include/rdma/rdmavt_qp.h
+++ b/include/rdma/rdmavt_qp.h
@@ -48,6 +48,7 @@
  *
  */
 
+#include 
 #include 
 /*
  * Atomic bit definitions for r_aflags.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/6] IB/rdmavt: Add IB user context allocation and de-alloction functions

[Dennis Dalessandro]

From: Harish Chegondi 

Adding IB user context alloc and dealloc functions to rdmavt so that the
drivers that use rdmavt can use these functions instead of defining their
own functions.

Reviewed-by: Ira Weiny 
Signed-off-by: Harish Chegondi 
---
 drivers/infiniband/sw/rdmavt/vt.c |   20 ++--
 1 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 18b5f43..df2df36 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -189,6 +189,16 @@ static int rvt_query_gid(struct ib_device *ibdev, u8 port,
return -EOPNOTSUPP;
 }
 
+struct rvt_ucontext {
+   struct ib_ucontext ibucontext;
+};
+
+static inline struct rvt_ucontext *to_iucontext(struct ib_ucontext
+   *ibucontext)
+{
+   return container_of(ibucontext, struct rvt_ucontext, ibucontext);
+}
+
 /**
  * rvt_alloc_ucontext - Allocate a user context
  * @ibdev: Vers IB dev
@@ -197,7 +207,12 @@ static int rvt_query_gid(struct ib_device *ibdev, u8 port,
 static struct ib_ucontext *rvt_alloc_ucontext(struct ib_device *ibdev,
  struct ib_udata *udata)
 {
-   return ERR_PTR(-EOPNOTSUPP);
+   struct rvt_ucontext *context;
+
+   context = kmalloc(sizeof(*context), GFP_KERNEL);
+   if (!context)
+   return ERR_PTR(-ENOMEM);
+   return >ibucontext;
 }
 
 /**
@@ -206,7 +221,8 @@ static struct ib_ucontext *rvt_alloc_ucontext(struct 
ib_device *ibdev,
  */
 static int rvt_dealloc_ucontext(struct ib_ucontext *context)
 {
-   return -EOPNOTSUPP;
+   kfree(to_iucontext(context));
+   return 0;
 }
 
 static int rvt_get_port_immutable(struct ib_device *ibdev, u8 port_num,

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 10/36] IB/rdmavt: Alloc and dealloc ucontexts

[Dennis Dalessandro]

Add the stubs to allocate and deallocate user contexts. This will be
handled completely by rvt.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/vt.c |   22 ++
 1 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index e95f197..cd19429 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -178,6 +178,26 @@ static int rvt_query_gid(struct ib_device *ibdev, u8 port,
return -EOPNOTSUPP;
 }
 
+/**
+ * rvt_alloc_ucontext - Allocate a user context
+ * @ibdev: Vers IB dev
+ * @data: User data allocated
+ */
+static struct ib_ucontext *rvt_alloc_ucontext(struct ib_device *ibdev,
+ struct ib_udata *udata)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ *rvt_dealloc_ucontext - Free a user context
+ *@context - Free this
+ */
+static int rvt_dealloc_ucontext(struct ib_ucontext *context)
+{
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -197,6 +217,8 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, modify_port);
CHECK_DRIVER_OVERRIDE(rdi, query_pkey);
CHECK_DRIVER_OVERRIDE(rdi, query_gid);
+   CHECK_DRIVER_OVERRIDE(rdi, alloc_ucontext);
+   CHECK_DRIVER_OVERRIDE(rdi, dealloc_ucontext);
 
/* DMA Operations */
rdi->ibdev.dma_ops =

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 12/36] IB/rdmavt: Add address handle stubs

[Dennis Dalessandro]

Adds the stubs for create, destroy, modify, and query of the
address handle.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |3 -
 drivers/infiniband/sw/rdmavt/ah.c |   76 +
 drivers/infiniband/sw/rdmavt/ah.h |   59 ++
 drivers/infiniband/sw/rdmavt/vt.c |6 +++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 143 insertions(+), 2 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/ah.c
 create mode 100644 drivers/infiniband/sw/rdmavt/ah.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 9a9a095..628c684 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,5 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o dma.o pd.o qp.o
-
+rdmavt-y := vt.o ah.o dma.o pd.o qp.o
diff --git a/drivers/infiniband/sw/rdmavt/ah.c 
b/drivers/infiniband/sw/rdmavt/ah.c
new file mode 100644
index 000..d368955
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/ah.c
@@ -0,0 +1,76 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "ah.h"
+
+/**
+ * rvt_create_ah - create an address handle
+ * @pd: the protection domain
+ * @ah_attr: the attributes of the AH
+ *
+ * This may be called from interrupt context.
+ */
+struct ib_ah *rvt_create_ah(struct ib_pd *pd,
+   struct ib_ah_attr *ah_attr)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+int rvt_destroy_ah(struct ib_ah *ibah)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_modify_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
+{
+   return -EOPNOTSUPP;
+}
diff --git a/drivers/infiniband/sw/rdmavt/ah.h 
b/drivers/infiniband/sw/rdmavt/ah.h
new file mode 100644
index 000..8cd7ea7
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/ah.h
@@ -0,0 +1,59 @@
+#ifndef DEF_RVTAH_H
+#define DEF_RVTAH_H
+
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the i

[PATCH v3 18/36] IB/rdmavt: Add get port immutable stub

[Dennis Dalessandro]

This adds the get port immutable verbs call.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/vt.c |7 +++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 41cfd84..a122ee8 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -198,6 +198,12 @@ static int rvt_dealloc_ucontext(struct ib_ucontext 
*context)
return -EOPNOTSUPP;
 }
 
+static int rvt_get_port_immutable(struct ib_device *ibdev, u8 port_num,
+ struct ib_port_immutable *immutable)
+{
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -219,6 +225,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, query_gid);
CHECK_DRIVER_OVERRIDE(rdi, alloc_ucontext);
CHECK_DRIVER_OVERRIDE(rdi, dealloc_ucontext);
+   CHECK_DRIVER_OVERRIDE(rdi, get_port_immutable);
 
/* Queue Pairs */
CHECK_DRIVER_OVERRIDE(rdi, create_qp);

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 17/36] IB/rdmavt: Add mmap stub

[Dennis Dalessandro]

Adds the stub for the mmap verbs call.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 +
 drivers/infiniband/sw/rdmavt/mmap.c   |   60 +
 drivers/infiniband/sw/rdmavt/mmap.h   |   55 ++
 drivers/infiniband/sw/rdmavt/vt.c |1 +
 drivers/infiniband/sw/rdmavt/vt.h |1 +
 5 files changed, 118 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/mmap.c
 create mode 100644 drivers/infiniband/sw/rdmavt/mmap.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index fe65410..6f530d1 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mad.o mcast.o mr.o pd.o qp.o srq.o
+rdmavt-y := vt.o ah.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/mmap.c 
b/drivers/infiniband/sw/rdmavt/mmap.c
new file mode 100644
index 000..d09f3a0
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mmap.c
@@ -0,0 +1,60 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include 
+#include "mmap.h"
+
+/**
+ * rvt_mmap - create a new mmap region
+ * @context: the IB user context of the process making the mmap() call
+ * @vma: the VMA to be initialized
+ * Return zero if the mmap is OK. Otherwise, return an errno.
+ */
+int rvt_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
+{
+   return -EOPNOTSUPP;
+}
diff --git a/drivers/infiniband/sw/rdmavt/mmap.h 
b/drivers/infiniband/sw/rdmavt/mmap.h
new file mode 100644
index 000..94f6377
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mmap.h
@@ -0,0 +1,55 @@
+#ifndef DEF_RDMAVTMMAP_H
+#define DEF_RDMAVTMMAP_H
+
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or w

[PATCH v3 26/36] IB/rdmavt: Move memory registration into rdmavt

[Dennis Dalessandro]

Use the memory registration routines in hfi1 and move them to rdmavt.
A follow on patch will address removing the duplicated code in the
hfi1 and qib drivers.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Remove rvt_reg_phys_mr

 drivers/infiniband/sw/rdmavt/mr.c |  683 -
 drivers/infiniband/sw/rdmavt/mr.h |   23 +
 drivers/infiniband/sw/rdmavt/vt.c |   24 +
 include/rdma/rdma_vt.h|   19 +
 4 files changed, 734 insertions(+), 15 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/mr.c 
b/drivers/infiniband/sw/rdmavt/mr.c
index c672043..f1dcaf4 100644
--- a/drivers/infiniband/sw/rdmavt/mr.c
+++ b/drivers/infiniband/sw/rdmavt/mr.c
@@ -46,8 +46,252 @@
  */
 
 #include 
+#include 
+#include 
+#include 
+#include "vt.h"
 #include "mr.h"
 
+/*
+ * Do any intilization needed when a driver registers with rdmavt.
+ */
+int rvt_driver_mr_init(struct rvt_dev_info *rdi)
+{
+   unsigned int lkey_table_size = rdi->dparms.lkey_table_size;
+   unsigned lk_tab_size;
+   int i;
+
+   if (rdi->flags & RVT_FLAG_MR_INIT_DRIVER) {
+   rvt_pr_info(rdi, "Driver is doing MR init.\n");
+   return 0;
+   }
+
+   /*
+* The top hfi1_lkey_table_size bits are used to index the
+* table.  The lower 8 bits can be owned by the user (copied from
+* the LKEY).  The remaining bits act as a generation number or tag.
+*/
+   if (!lkey_table_size)
+   return -EINVAL;
+
+   spin_lock_init(>lkey_table.lock);
+
+   rdi->lkey_table.max = 1 << lkey_table_size;
+
+   /* ensure generation is at least 4 bits */
+   if (lkey_table_size > RVT_MAX_LKEY_TABLE_BITS) {
+   rvt_pr_warn(rdi, "lkey bits %u too large, reduced to %u\n",
+   lkey_table_size, RVT_MAX_LKEY_TABLE_BITS);
+   rdi->dparms.lkey_table_size = RVT_MAX_LKEY_TABLE_BITS;
+   lkey_table_size = rdi->dparms.lkey_table_size;
+   }
+   lk_tab_size = rdi->lkey_table.max * sizeof(*rdi->lkey_table.table);
+   rdi->lkey_table.table = (struct rvt_mregion __rcu **)
+  vmalloc(lk_tab_size);
+   if (!rdi->lkey_table.table)
+   return -ENOMEM;
+
+   RCU_INIT_POINTER(rdi->dma_mr, NULL);
+   for (i = 0; i < rdi->lkey_table.max; i++)
+   RCU_INIT_POINTER(rdi->lkey_table.table[i], NULL);
+
+   return 0;
+}
+
+/*
+ * called when drivers have unregistered or perhaps failed to register with us
+ */
+void rvt_mr_exit(struct rvt_dev_info *rdi)
+{
+   if (rdi->dma_mr)
+   rvt_pr_err(rdi, "DMA MR not null!\n");
+
+   vfree(rdi->lkey_table.table);
+}
+
+static void rvt_deinit_mregion(struct rvt_mregion *mr)
+{
+   int i = mr->mapsz;
+
+   mr->mapsz = 0;
+   while (i)
+   kfree(mr->map[--i]);
+}
+
+static int rvt_init_mregion(struct rvt_mregion *mr, struct ib_pd *pd,
+   int count)
+{
+   int m, i = 0;
+
+   mr->mapsz = 0;
+   m = (count + RVT_SEGSZ - 1) / RVT_SEGSZ;
+   for (; i < m; i++) {
+   mr->map[i] = kzalloc(sizeof(*mr->map[0]), GFP_KERNEL);
+   if (!mr->map[i]) {
+   rvt_deinit_mregion(mr);
+   return -ENOMEM;
+   }
+   mr->mapsz++;
+   }
+   init_completion(>comp);
+   /* count returning the ptr to user */
+   atomic_set(>refcount, 1);
+   mr->pd = pd;
+   mr->max_segs = count;
+   return 0;
+}
+
+/**
+ * rvt_alloc_lkey - allocate an lkey
+ * @mr: memory region that this lkey protects
+ * @dma_region: 0->normal key, 1->restricted DMA key
+ *
+ * Returns 0 if successful, otherwise returns -errno.
+ *
+ * Increments mr reference count as required.
+ *
+ * Sets the lkey field mr for non-dma regions.
+ *
+ */
+static int rvt_alloc_lkey(struct rvt_mregion *mr, int dma_region)
+{
+   unsigned long flags;
+   u32 r;
+   u32 n;
+   int ret = 0;
+   struct rvt_dev_info *dev = ib_to_rvt(mr->pd->device);
+   struct rvt_lkey_table *rkt = >lkey_table;
+
+   rvt_get_mr(mr);
+   spin_lock_irqsave(>lock, flags);
+
+   /* special case for dma_mr lkey == 0 */
+   if (dma_region) {
+   struct rvt_mregion *tmr;
+
+   tmr = rcu_access_pointer(dev->dma_mr);
+   if (!tmr) {
+   rcu_assign_pointer(dev->dma_mr, mr);
+   mr->lkey_published = 1;
+   } else {
+   rvt_put_mr(mr);
+   }
+   goto success;
+   }
+
+   /* Find the next a

[PATCH v3 30/36] IB/rdmavt: Move SRQ data structure into rdmavt

[Dennis Dalessandro]

Patch moves the srq data structure into rdmavt in preparation for
removal from qib and hfi1 which will follow in subsequent patches.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Harish Chegondi <harish.chego...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 include/rdma/rdma_vt.h |   13 +
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 36cced6..fcf3ec0 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -349,6 +349,14 @@ struct rvt_qp {
cacheline_aligned_in_smp;
 };
 
+struct rvt_srq {
+   struct ib_srq ibsrq;
+   struct rvt_rq rq;
+   struct rvt_mmap_info *ip;
+   /* send signal when number of RWQEs < limit */
+   u32 limit;
+};
+
 /* End QP section */
 
 /*
@@ -485,6 +493,11 @@ static inline void rvt_get_mr(struct rvt_mregion *mr)
atomic_inc(>refcount);
 }
 
+static inline struct rvt_srq *ibsrq_to_rvtsrq(struct ib_srq *ibsrq)
+{
+   return container_of(ibsrq, struct rvt_srq, ibsrq);
+}
+
 int rvt_register_device(struct rvt_dev_info *rvd);
 void rvt_unregister_device(struct rvt_dev_info *rvd);
 int rvt_check_ah(struct ib_device *ibdev, struct ib_ah_attr *ah_attr);

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 35/36] IB/rdmavt: Add mmap related functions

[Dennis Dalessandro]

The mmap data structure was moved in a previous commit. This patch now
pulls in the related functions.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/mmap.c |  140 +++
 drivers/infiniband/sw/rdmavt/mmap.h |2 -
 drivers/infiniband/sw/rdmavt/vt.c   |1 
 include/rdma/rdma_vt.h  |   15 
 4 files changed, 156 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/mmap.c 
b/drivers/infiniband/sw/rdmavt/mmap.c
index d09f3a0..fc30ff7 100644
--- a/drivers/infiniband/sw/rdmavt/mmap.c
+++ b/drivers/infiniband/sw/rdmavt/mmap.c
@@ -46,8 +46,61 @@
  */
 
 #include 
+#include 
+#include 
+#include 
 #include "mmap.h"
 
+void rvt_mmap_init(struct rvt_dev_info *rdi)
+{
+   INIT_LIST_HEAD(>pending_mmaps);
+   spin_lock_init(>pending_lock);
+   rdi->mmap_offset = PAGE_SIZE;
+   spin_lock_init(>mmap_offset_lock);
+}
+
+/**
+ * rvt_release_mmap_info - free mmap info structure
+ * @ref: a pointer to the kref within struct rvt_mmap_info
+ */
+void rvt_release_mmap_info(struct kref *ref)
+{
+   struct rvt_mmap_info *ip =
+   container_of(ref, struct rvt_mmap_info, ref);
+   struct rvt_dev_info *rdi = ib_to_rvt(ip->context->device);
+
+   spin_lock_irq(>pending_lock);
+   list_del(>pending_mmaps);
+   spin_unlock_irq(>pending_lock);
+
+   vfree(ip->obj);
+   kfree(ip);
+}
+EXPORT_SYMBOL(rvt_release_mmap_info);
+
+/*
+ * open and close keep track of how many times the CQ is mapped,
+ * to avoid releasing it.
+ */
+static void rvt_vma_open(struct vm_area_struct *vma)
+{
+   struct rvt_mmap_info *ip = vma->vm_private_data;
+
+   kref_get(>ref);
+}
+
+static void rvt_vma_close(struct vm_area_struct *vma)
+{
+   struct rvt_mmap_info *ip = vma->vm_private_data;
+
+   kref_put(>ref, rvt_release_mmap_info);
+}
+
+static const struct vm_operations_struct rvt_vm_ops = {
+   .open = rvt_vma_open,
+   .close = rvt_vma_close,
+};
+
 /**
  * rvt_mmap - create a new mmap region
  * @context: the IB user context of the process making the mmap() call
@@ -56,5 +109,90 @@
  */
 int rvt_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
 {
-   return -EOPNOTSUPP;
+   struct rvt_dev_info *rdi = ib_to_rvt(context->device);
+   unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
+   unsigned long size = vma->vm_end - vma->vm_start;
+   struct rvt_mmap_info *ip, *pp;
+   int ret = -EINVAL;
+
+   /*
+* Search the device's list of objects waiting for a mmap call.
+* Normally, this list is very short since a call to create a
+* CQ, QP, or SRQ is soon followed by a call to mmap().
+*/
+   spin_lock_irq(>pending_lock);
+   list_for_each_entry_safe(ip, pp, >pending_mmaps,
+pending_mmaps) {
+   /* Only the creator is allowed to mmap the object */
+   if (context != ip->context || (__u64)offset != ip->offset)
+   continue;
+   /* Don't allow a mmap larger than the object. */
+   if (size > ip->size)
+   break;
+
+   list_del_init(>pending_mmaps);
+   spin_unlock_irq(>pending_lock);
+
+   ret = remap_vmalloc_range(vma, ip->obj, 0);
+   if (ret)
+   goto done;
+   vma->vm_ops = _vm_ops;
+   vma->vm_private_data = ip;
+   rvt_vma_open(vma);
+   goto done;
+   }
+   spin_unlock_irq(>pending_lock);
+done:
+   return ret;
+}
+EXPORT_SYMBOL(rvt_mmap);
+
+/*
+ * Allocate information for hfi1_mmap
+ */
+struct rvt_mmap_info *rvt_create_mmap_info(struct rvt_dev_info *rdi,
+  u32 size,
+  struct ib_ucontext *context,
+  void *obj)
+{
+   struct rvt_mmap_info *ip;
+
+   ip = kmalloc(sizeof(*ip), GFP_KERNEL);
+   if (!ip)
+   return ip;
+
+   size = PAGE_ALIGN(size);
+
+   spin_lock_irq(>mmap_offset_lock);
+   if (rdi->mmap_offset == 0)
+   rdi->mmap_offset = PAGE_SIZE;
+   ip->offset = rdi->mmap_offset;
+   rdi->mmap_offset += size;
+   spin_unlock_irq(>mmap_offset_lock);
+
+   INIT_LIST_HEAD(>pending_mmaps);
+   ip->size = size;
+   ip->context = context;
+   ip->obj = obj;
+   kref_init(>ref);
+
+   return ip;
+}
+EXPORT_SYMBOL(rvt_create_mmap_info);
+
+void rvt_update_mmap_info(struct rvt_dev_info *rdi, struct rvt_mmap_info *ip,
+ u32 size, void *obj)
+{
+   

[PATCH v3 34/36] IB/rdmavt: Initialize and teardown of qpn table

[Dennis Dalessandro]

Add table init as well as teardown for handling qpn maps. Drivers can still
provide this functionality by setting the QP_INIT_DRIVER bit.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/qp.c |  197 +
 drivers/infiniband/sw/rdmavt/qp.h |2 
 drivers/infiniband/sw/rdmavt/vt.c |   35 ---
 include/rdma/rdma_vt.h|9 ++
 include/rdma/rdmavt_qp.h  |   33 ++
 5 files changed, 263 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.c 
b/drivers/infiniband/sw/rdmavt/qp.c
index 23a5f68..17dd6ab 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -45,8 +45,205 @@
  *
  */
 
+#include 
+#include 
+#include "vt.h"
 #include "qp.h"
 
+static void get_map_page(struct rvt_qpn_table *qpt, struct rvt_qpn_map *map)
+{
+   unsigned long page = get_zeroed_page(GFP_KERNEL);
+
+   /*
+* Free the page if someone raced with us installing it.
+*/
+
+   spin_lock(>lock);
+   if (map->page)
+   free_page(page);
+   else
+   map->page = (void *)page;
+   spin_unlock(>lock);
+}
+
+/**
+ * init_qpn_table - initialize the QP number table for a device
+ * @qpt: the QPN table
+ */
+static int init_qpn_table(struct rvt_dev_info *rdi, struct rvt_qpn_table *qpt)
+{
+   u32 offset, i;
+   struct rvt_qpn_map *map;
+   int ret = 0;
+
+   if (!(rdi->dparms.qpn_res_end > rdi->dparms.qpn_res_start))
+   return -EINVAL;
+
+   spin_lock_init(>lock);
+
+   qpt->last = rdi->dparms.qpn_start;
+   qpt->incr = rdi->dparms.qpn_inc << rdi->dparms.qos_shift;
+
+   /*
+* Drivers may want some QPs beyond what we need for verbs let them use
+* our qpn table. No need for two. Lets go ahead and mark the bitmaps
+* for those. The reserved range must be *after* the range which verbs
+* will pick from.
+*/
+
+   /* Figure out number of bit maps needed before reserved range */
+   qpt->nmaps = rdi->dparms.qpn_res_start / RVT_BITS_PER_PAGE;
+
+   /* This should always be zero */
+   offset = rdi->dparms.qpn_res_start & RVT_BITS_PER_PAGE_MASK;
+
+   /* Starting with the first reserved bit map */
+   map = >map[qpt->nmaps];
+
+   rvt_pr_info(rdi, "Reserving QPNs from 0x%x to 0x%x for non-verbs use\n",
+   rdi->dparms.qpn_res_start, rdi->dparms.qpn_res_end);
+   for (i = rdi->dparms.qpn_res_start; i < rdi->dparms.qpn_res_end; i++) {
+   if (!map->page) {
+   get_map_page(qpt, map);
+   if (!map->page) {
+   ret = -ENOMEM;
+   break;
+   }
+   }
+   set_bit(offset, map->page);
+   offset++;
+   if (offset == RVT_BITS_PER_PAGE) {
+   /* next page */
+   qpt->nmaps++;
+   map++;
+   offset = 0;
+   }
+   }
+   return ret;
+}
+
+/**
+ * free_qpn_table - free the QP number table for a device
+ * @qpt: the QPN table
+ */
+static void free_qpn_table(struct rvt_qpn_table *qpt)
+{
+   int i;
+
+   for (i = 0; i < ARRAY_SIZE(qpt->map); i++)
+   free_page((unsigned long)qpt->map[i].page);
+}
+
+int rvt_driver_qp_init(struct rvt_dev_info *rdi)
+{
+   int i;
+   int ret = -ENOMEM;
+
+   if (rdi->flags & RVT_FLAG_QP_INIT_DRIVER) {
+   rvt_pr_info(rdi, "Driver is doing QP init.\n");
+   return 0;
+   }
+
+   if (!rdi->dparms.qp_table_size)
+   return -EINVAL;
+
+   /*
+* If driver is not doing any QP allocation then make sure it is
+* providing the necessary QP functions.
+*/
+   if (!rdi->driver_f.free_all_qps)
+   return -EINVAL;
+
+   /* allocate parent object */
+   rdi->qp_dev = kzalloc(sizeof(*rdi->qp_dev), GFP_KERNEL);
+   if (!rdi->qp_dev)
+   return -ENOMEM;
+
+   /* allocate hash table */
+   rdi->qp_dev->qp_table_size = rdi->dparms.qp_table_size;
+   rdi->qp_dev->qp_table_bits = ilog2(rdi->dparms.qp_table_size);
+   rdi->qp_dev->qp_table =
+   kmalloc(rdi->qp_dev->qp_table_size *
+   sizeof(*rdi->qp_dev->qp_table),
+   GFP_KERNEL);
+   if (!rdi->qp_dev->qp_table)
+   goto no_qp_table;
+
+   for (i = 0; i < rdi->qp_dev->qp_table_size; i++)
+   RCU_INIT_POINTER(rdi

[PATCH 0/6] Start to add qp support to rdmavt and other verbs

[Dennis Dalessandro]

This patch continues adding the support for queue pairs in rdmavt. It also adds
the create queue pair, user context alloc, dealloc, and query device verbs.

This patchset builds on the "Begin to use rdmavt for qib" patchset.

---

Dennis Dalessandro (3):
  IB/rdmavt: Add R and S flags for queue pairs
  IB/rdmavt: Add create queue pair functionality
  IB/rdmavt: Export reset_qp in rdmavt

Harish Chegondi (3):
  IB/rdmavt: Add IB user context allocation and de-alloction functions
  IB/rdmavt: Allow reserving just one qpn
  IB/rdmavt: Add support for rvt_query_device function


 drivers/infiniband/sw/rdmavt/qp.c |  430 +++--
 drivers/infiniband/sw/rdmavt/vt.c |   30 ++-
 include/rdma/rdma_vt.h|   18 +-
 include/rdma/rdmavt_qp.h  |   81 +++
 4 files changed, 529 insertions(+), 30 deletions(-)

-- 
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/6] IB/rdmavt: Add create queue pair functionality

[Dennis Dalessandro]

Add create queue pair verbs call as well as supporting functions.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Harish Chegondi <harish.chego...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/qp.c |  425 +++--
 drivers/infiniband/sw/rdmavt/vt.c |1 
 include/rdma/rdma_vt.h|   10 +
 3 files changed, 413 insertions(+), 23 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.c 
b/drivers/infiniband/sw/rdmavt/qp.c
index 17dd6ab..7d1f02e 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -47,8 +47,11 @@
 
 #include 
 #include 
-#include "vt.h"
+#include 
+#include 
+#include 
 #include "qp.h"
+#include "vt.h"
 
 static void get_map_page(struct rvt_qpn_table *qpt, struct rvt_qpn_map *map)
 {
@@ -151,7 +154,10 @@ int rvt_driver_qp_init(struct rvt_dev_info *rdi)
 * If driver is not doing any QP allocation then make sure it is
 * providing the necessary QP functions.
 */
-   if (!rdi->driver_f.free_all_qps)
+   if (!rdi->driver_f.free_all_qps ||
+   !rdi->driver_f.qp_priv_alloc ||
+   !rdi->driver_f.qp_priv_free ||
+   !rdi->driver_f.notify_qp_reset)
return -EINVAL;
 
/* allocate parent object */
@@ -178,7 +184,9 @@ int rvt_driver_qp_init(struct rvt_dev_info *rdi)
if (init_qpn_table(rdi, >qp_dev->qpn_table))
goto fail_table;
 
-   return ret;
+   spin_lock_init(>n_qps_lock);
+
+   return 0;
 
 fail_table:
kfree(rdi->qp_dev->qp_table);
@@ -197,31 +205,29 @@ no_qp_table:
  * There should not be any QPs still in use.
  * Free memory for table.
  */
-static unsigned free_all_qps(struct rvt_dev_info *rdi)
+static unsigned rvt_free_all_qps(struct rvt_dev_info *rdi)
 {
unsigned long flags;
struct rvt_qp *qp;
unsigned n, qp_inuse = 0;
spinlock_t *ql; /* work around too long line below */
 
-   rdi->driver_f.free_all_qps(rdi);
+   if (rdi->driver_f.free_all_qps)
+   qp_inuse = rdi->driver_f.free_all_qps(rdi);
 
if (!rdi->qp_dev)
-   return 0;
+   return qp_inuse;
 
ql = >qp_dev->qpt_lock;
-   spin_lock_irqsave(>qp_dev->qpt_lock, flags);
+   spin_lock_irqsave(ql, flags);
for (n = 0; n < rdi->qp_dev->qp_table_size; n++) {
qp = rcu_dereference_protected(rdi->qp_dev->qp_table[n],
   lockdep_is_held(ql));
RCU_INIT_POINTER(rdi->qp_dev->qp_table[n], NULL);
-   qp =  rcu_dereference_protected(qp->next,
-   lockdep_is_held(ql));
-   while (qp) {
+
+   for (; qp; qp = rcu_dereference_protected(qp->next,
+ lockdep_is_held(ql)))
qp_inuse++;
-   qp =  rcu_dereference_protected(qp->next,
-   lockdep_is_held(ql));
-   }
}
spin_unlock_irqrestore(ql, flags);
synchronize_rcu();
@@ -230,26 +236,190 @@ static unsigned free_all_qps(struct rvt_dev_info *rdi)
 
 void rvt_qp_exit(struct rvt_dev_info *rdi)
 {
-   u32 qps_inuse = free_all_qps(rdi);
+   u32 qps_inuse = rvt_free_all_qps(rdi);
 
-   qps_inuse = free_all_qps(rdi);
if (qps_inuse)
rvt_pr_err(rdi, "QP memory leak! %u still in use\n",
   qps_inuse);
if (!rdi->qp_dev)
return;
 
+   if (rdi->flags & RVT_FLAG_QP_INIT_DRIVER)
+   return; /* driver did the qp init so nothing else to do */
+
kfree(rdi->qp_dev->qp_table);
free_qpn_table(>qp_dev->qpn_table);
kfree(rdi->qp_dev);
 }
 
+static inline unsigned mk_qpn(struct rvt_qpn_table *qpt,
+ struct rvt_qpn_map *map, unsigned off)
+{
+   return (map - qpt->map) * RVT_BITS_PER_PAGE + off;
+}
+
+/*
+ * Allocate the next available QPN or
+ * zero/one for QP type IB_QPT_SMI/IB_QPT_GSI.
+ */
+static int alloc_qpn(struct rvt_dev_info *rdi, struct rvt_qpn_table *qpt,
+enum ib_qp_type type, u8 port)
+{
+   u32 i, offset, max_scan, qpn;
+   struct rvt_qpn_map *map;
+   u32 ret;
+
+   if (rdi->driver_f.alloc_qpn)
+   return rdi->driver_f.alloc_qpn(rdi, qpt, type, port);
+
+   if (type == IB_QPT_SMI || type == IB_QPT_GSI) {
+   unsigned n;
+
+   ret = type == IB_QPT_GSI;
+   n = 1 << (ret + 2 * (port - 1));
+   spin_lock(>lock);
+   if (qpt->flags & n)
+  

[PATCH 5/6] IB/rdmavt: Allow reserving just one qpn

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

qib needs to reserve only one qpn for non-verbs stuff. Also fixed the for
loop to reserve the end qpn.

Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/sw/rdmavt/qp.c |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.c 
b/drivers/infiniband/sw/rdmavt/qp.c
index 44485ad..ee19eae 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -79,7 +79,7 @@ static int init_qpn_table(struct rvt_dev_info *rdi, struct 
rvt_qpn_table *qpt)
struct rvt_qpn_map *map;
int ret = 0;
 
-   if (!(rdi->dparms.qpn_res_end > rdi->dparms.qpn_res_start))
+   if (!(rdi->dparms.qpn_res_end >= rdi->dparms.qpn_res_start))
return -EINVAL;
 
spin_lock_init(>lock);
@@ -105,7 +105,7 @@ static int init_qpn_table(struct rvt_dev_info *rdi, struct 
rvt_qpn_table *qpt)
 
rvt_pr_info(rdi, "Reserving QPNs from 0x%x to 0x%x for non-verbs use\n",
rdi->dparms.qpn_res_start, rdi->dparms.qpn_res_end);
-   for (i = rdi->dparms.qpn_res_start; i < rdi->dparms.qpn_res_end; i++) {
+   for (i = rdi->dparms.qpn_res_start; i <= rdi->dparms.qpn_res_end; i++) {
if (!map->page) {
get_map_page(qpt, map);
if (!map->page) {

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/6] IB/rdmavt: Add R and S flags for queue pairs

[Dennis Dalessandro]

Use the flags originally provided for hfi1 in the rdmavt driver. These will
be made available to drivers in the qp header file.

Reviewed-by: Harish Chegondi <harish.chego...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 include/rdma/rdmavt_qp.h |   80 ++
 1 files changed, 80 insertions(+), 0 deletions(-)

diff --git a/include/rdma/rdmavt_qp.h b/include/rdma/rdmavt_qp.h
index e6a7d17..1aa8b5b 100644
--- a/include/rdma/rdmavt_qp.h
+++ b/include/rdma/rdmavt_qp.h
@@ -48,6 +48,86 @@
  *
  */
 
+#include 
+/*
+ * Atomic bit definitions for r_aflags.
+ */
+#define RVT_R_WRID_VALID0
+#define RVT_R_REWIND_SGE1
+
+/*
+ * Bit definitions for r_flags.
+ */
+#define RVT_R_REUSE_SGE 0x01
+#define RVT_R_RDMAR_SEQ 0x02
+#define RVT_R_RSP_NAK   0x04
+#define RVT_R_RSP_SEND  0x08
+#define RVT_R_COMM_EST  0x10
+
+/*
+ * Bit definitions for s_flags.
+ *
+ * RVT_S_SIGNAL_REQ_WR - set if QP send WRs contain completion signaled
+ * RVT_S_BUSY - send tasklet is processing the QP
+ * RVT_S_TIMER - the RC retry timer is active
+ * RVT_S_ACK_PENDING - an ACK is waiting to be sent after RDMA read/atomics
+ * RVT_S_WAIT_FENCE - waiting for all prior RDMA read or atomic SWQEs
+ * before processing the next SWQE
+ * RVT_S_WAIT_RDMAR - waiting for a RDMA read or atomic SWQE to complete
+ * before processing the next SWQE
+ * RVT_S_WAIT_RNR - waiting for RNR timeout
+ * RVT_S_WAIT_SSN_CREDIT - waiting for RC credits to process next SWQE
+ * RVT_S_WAIT_DMA - waiting for send DMA queue to drain before generating
+ *  next send completion entry not via send DMA
+ * RVT_S_WAIT_PIO - waiting for a send buffer to be available
+ * RVT_S_WAIT_TX - waiting for a struct verbs_txreq to be available
+ * RVT_S_WAIT_DMA_DESC - waiting for DMA descriptors to be available
+ * RVT_S_WAIT_KMEM - waiting for kernel memory to be available
+ * RVT_S_WAIT_PSN - waiting for a packet to exit the send DMA queue
+ * RVT_S_WAIT_ACK - waiting for an ACK packet before sending more requests
+ * RVT_S_SEND_ONE - send one packet, request ACK, then wait for ACK
+ * RVT_S_ECN - a BECN was queued to the send engine
+ */
+#define RVT_S_SIGNAL_REQ_WR0x0001
+#define RVT_S_BUSY 0x0002
+#define RVT_S_TIMER0x0004
+#define RVT_S_RESP_PENDING 0x0008
+#define RVT_S_ACK_PENDING  0x0010
+#define RVT_S_WAIT_FENCE   0x0020
+#define RVT_S_WAIT_RDMAR   0x0040
+#define RVT_S_WAIT_RNR 0x0080
+#define RVT_S_WAIT_SSN_CREDIT  0x0100
+#define RVT_S_WAIT_DMA 0x0200
+#define RVT_S_WAIT_PIO 0x0400
+#define RVT_S_WAIT_TX  0x0800
+#define RVT_S_WAIT_DMA_DESC0x1000
+#define RVT_S_WAIT_KMEM0x2000
+#define RVT_S_WAIT_PSN 0x4000
+#define RVT_S_WAIT_ACK 0x8000
+#define RVT_S_SEND_ONE 0x1
+#define RVT_S_UNLIMITED_CREDIT 0x2
+#define RVT_S_AHG_VALID0x4
+#define RVT_S_AHG_CLEAR0x8
+#define RVT_S_ECN  0x10
+
+/*
+ * Wait flags that would prevent any packet type from being sent.
+ */
+#define RVT_S_ANY_WAIT_IO (RVT_S_WAIT_PIO | RVT_S_WAIT_TX | \
+   RVT_S_WAIT_DMA_DESC | RVT_S_WAIT_KMEM)
+
+/*
+ * Wait flags that would prevent send work requests from making progress.
+ */
+#define RVT_S_ANY_WAIT_SEND (RVT_S_WAIT_FENCE | RVT_S_WAIT_RDMAR | \
+   RVT_S_WAIT_RNR | RVT_S_WAIT_SSN_CREDIT | RVT_S_WAIT_DMA | \
+   RVT_S_WAIT_PSN | RVT_S_WAIT_ACK)
+
+#define RVT_S_ANY_WAIT (RVT_S_ANY_WAIT_IO | RVT_S_ANY_WAIT_SEND)
+
+/* Number of bits to pay attention to in the opcode for checking qp type */
+#define RVT_OPCODE_QP_MASK 0xE0
+
 /*
  * Send work request queue entry.
  * The size of the sg_list is determined when the QP is created and stored

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/6] IB/rdmavt: Add support for rvt_query_device function

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

With this commit, the drivers using rdmavt need not define query_device
function. But they should fill in the IB device attributes structure
rvt_dev_info.dparms.props

Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |9 +++--
 include/rdma/rdma_vt.h|5 -
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index e75eb3d..f2b6438 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -68,10 +68,15 @@ static int rvt_query_device(struct ib_device *ibdev,
struct ib_device_attr *props,
struct ib_udata *uhw)
 {
+   struct rvt_dev_info *rdi = ib_to_rvt(ibdev);
+
+   if (uhw->inlen || uhw->outlen)
+   return -EINVAL;
/*
-* Return rvt_dev_info.props contents
+* Return rvt_dev_info.dparms.props contents
 */
-   return -EOPNOTSUPP;
+   *props = rdi->dparms.props;
+   return 0;
 }
 
 static int rvt_modify_device(struct ib_device *device,
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index e412e67..725778a 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -245,7 +245,10 @@ struct rvt_dev_info {
 * allocating space for this structure.
 *
 * The driver will also be responsible for filling in certain members of
-* dparms.props
+* dparms.props. The driver needs to fill in dparms exactly as it would
+* want values reported to a ULP. This will be returned to the caller
+* in rdmavt's device. The driver should also therefore refrain from
+* modifying this directly after registration with rdmavt.
 */
 
/* Driver specific properties */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/9] Start to rely on rdmavt for qp support and other verbs

[Dennis Dalessandro]

This patch set removes the ibport from qib and starts to add in more support for
rdamvt provided verbs functionality. Included is AH notification, pkey support,
and queue pair creation.

This patchset builds on the "Start to add qp support to rdmavt and other verbs"
patchset.
---

Harish Chegondi (9):
  IB/qib: Remove ibport and use rdmavt version
  IB/qib: Implement qib support for AH notification
  IB/qib: Remove mmap from qib
  IB/qib: Use rdmavt pkey verbs function
  IB/qib: Remove qpn, qp tables and related variables from qib
  IB/qib: Delete QIB user context allocation and de-alloction functions
  IB/qib: Remove qib_query_device function
  IB/qib: Use rdmavt send and receive flags
  IB/qib: Remove create qp and create qp table functionality

 drivers/infiniband/hw/qib/Makefile  |2 
 drivers/infiniband/hw/qib/qib.h |3 
 drivers/infiniband/hw/qib/qib_cq.c  |   18 -
 drivers/infiniband/hw/qib/qib_driver.c  |   14 -
 drivers/infiniband/hw/qib/qib_iba6120.c |8 
 drivers/infiniband/hw/qib/qib_iba7322.c |2 
 drivers/infiniband/hw/qib/qib_init.c|   10 +
 drivers/infiniband/hw/qib/qib_mad.c |  240 ++---
 drivers/infiniband/hw/qib/qib_mmap.c|  174 -
 drivers/infiniband/hw/qib/qib_qp.c  |  505 ++-
 drivers/infiniband/hw/qib/qib_rc.c  |  182 +-
 drivers/infiniband/hw/qib/qib_ruc.c |   32 +-
 drivers/infiniband/hw/qib/qib_sdma.c|   10 -
 drivers/infiniband/hw/qib/qib_srq.c |   18 -
 drivers/infiniband/hw/qib/qib_sysfs.c   |   65 +++
 drivers/infiniband/hw/qib/qib_uc.c  |   18 -
 drivers/infiniband/hw/qib/qib_ud.c  |   36 +-
 drivers/infiniband/hw/qib/qib_verbs.c   |  371 +---
 drivers/infiniband/hw/qib/qib_verbs.h   |  187 +-
 drivers/infiniband/hw/qib/qib_verbs_mcast.c |   28 +
 20 files changed, 668 insertions(+), 1255 deletions(-)
 delete mode 100644 drivers/infiniband/hw/qib/qib_mmap.c

-- 
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/9] IB/qib: Remove mmap from qib

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

Since mmap functionality has been moved into rdmavt, its time for qib to
use that.

Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/hw/qib/Makefile|2 
 drivers/infiniband/hw/qib/qib_cq.c|   18 ++-
 drivers/infiniband/hw/qib/qib_mmap.c  |  174 -
 drivers/infiniband/hw/qib/qib_qp.c|   24 ++---
 drivers/infiniband/hw/qib/qib_sdma.c  |4 -
 drivers/infiniband/hw/qib/qib_srq.c   |   18 ++-
 drivers/infiniband/hw/qib/qib_verbs.c |   44 
 drivers/infiniband/hw/qib/qib_verbs.h |   15 ---
 8 files changed, 53 insertions(+), 246 deletions(-)
 delete mode 100644 drivers/infiniband/hw/qib/qib_mmap.c

diff --git a/drivers/infiniband/hw/qib/Makefile 
b/drivers/infiniband/hw/qib/Makefile
index 75140f5..45db4fc 100644
--- a/drivers/infiniband/hw/qib/Makefile
+++ b/drivers/infiniband/hw/qib/Makefile
@@ -2,7 +2,7 @@ obj-$(CONFIG_INFINIBAND_QIB) += ib_qib.o
 
 ib_qib-y := qib_cq.o qib_diag.o qib_driver.o qib_eeprom.o \
qib_file_ops.o qib_fs.o qib_init.o qib_intr.o \
-   qib_mad.o qib_mmap.o qib_pcie.o qib_pio_copy.o \
+   qib_mad.o qib_pcie.o qib_pio_copy.o \
qib_qp.o qib_qsfp.o qib_rc.o qib_ruc.o qib_sdma.o qib_srq.o \
qib_sysfs.o qib_twsi.o qib_tx.o qib_uc.o qib_ud.o \
qib_user_pages.o qib_user_sdma.o qib_verbs_mcast.o qib_iba7220.o \
diff --git a/drivers/infiniband/hw/qib/qib_cq.c 
b/drivers/infiniband/hw/qib/qib_cq.c
index c1ea21e..094f694 100644
--- a/drivers/infiniband/hw/qib/qib_cq.c
+++ b/drivers/infiniband/hw/qib/qib_cq.c
@@ -264,7 +264,7 @@ struct ib_cq *qib_create_cq(struct ib_device *ibdev,
if (udata && udata->outlen >= sizeof(__u64)) {
int err;
 
-   cq->ip = qib_create_mmap_info(dev, sz, context, wc);
+   cq->ip = rvt_create_mmap_info(>rdi, sz, context, wc);
if (!cq->ip) {
ret = ERR_PTR(-ENOMEM);
goto bail_wc;
@@ -290,9 +290,9 @@ struct ib_cq *qib_create_cq(struct ib_device *ibdev,
spin_unlock(>n_cqs_lock);
 
if (cq->ip) {
-   spin_lock_irq(>pending_lock);
-   list_add(>ip->pending_mmaps, >pending_mmaps);
-   spin_unlock_irq(>pending_lock);
+   spin_lock_irq(>rdi.pending_lock);
+   list_add(>ip->pending_mmaps, >rdi.pending_mmaps);
+   spin_unlock_irq(>rdi.pending_lock);
}
 
/*
@@ -342,7 +342,7 @@ int qib_destroy_cq(struct ib_cq *ibcq)
dev->n_cqs_allocated--;
spin_unlock(>n_cqs_lock);
if (cq->ip)
-   kref_put(>ip->ref, qib_release_mmap_info);
+   kref_put(>ip->ref, rvt_release_mmap_info);
else
vfree(cq->queue);
kfree(cq);
@@ -468,7 +468,7 @@ int qib_resize_cq(struct ib_cq *ibcq, int cqe, struct 
ib_udata *udata)
struct qib_ibdev *dev = to_idev(ibcq->device);
struct rvt_mmap_info *ip = cq->ip;
 
-   qib_update_mmap_info(dev, ip, sz, wc);
+   rvt_update_mmap_info(>rdi, ip, sz, wc);
 
/*
 * Return the offset to mmap.
@@ -481,10 +481,10 @@ int qib_resize_cq(struct ib_cq *ibcq, int cqe, struct 
ib_udata *udata)
goto bail;
}
 
-   spin_lock_irq(>pending_lock);
+   spin_lock_irq(>rdi.pending_lock);
if (list_empty(>pending_mmaps))
-   list_add(>pending_mmaps, >pending_mmaps);
-   spin_unlock_irq(>pending_lock);
+   list_add(>pending_mmaps, >rdi.pending_mmaps);
+   spin_unlock_irq(>rdi.pending_lock);
}
 
ret = 0;
diff --git a/drivers/infiniband/hw/qib/qib_mmap.c 
b/drivers/infiniband/hw/qib/qib_mmap.c
deleted file mode 100644
index c32078c..000
--- a/drivers/infiniband/hw/qib/qib_mmap.c
+++ /dev/null
@@ -1,174 +0,0 @@
-/*
- * Copyright (c) 2006, 2007, 2008, 2009 QLogic Corporation. All rights 
reserved.
- *
- * This software is available to you under a choice of one of two
- * licenses.  You may choose to be licensed under the terms of the GNU
- * General Public License (GPL) Version 2, available from the file
- * COPYING in the main directory of this source tree, or the
- * OpenIB.org BSD license below:
- *
- * Redistribution and use in source and binary forms, with or
- * without modification, are permitted provided that the following
- * conditions are met:
- *
- *  - Redistributions of source code must retain the above
- *copyright notice, this list of conditions and the following
- *disclaimer.
- *
- *  - Redistributions in binary form must reprod

[PATCH 4/9] IB/qib: Use rdmavt pkey verbs function

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

Remove qib query pkey function which is no longer needed as this is now
being done in rdmavt. The allocation and maintenance of the list still
resides in the driver.

Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/hw/qib/qib_verbs.c |   33 +
 1 files changed, 13 insertions(+), 20 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index 368b794..c923734 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -1836,24 +1836,6 @@ unsigned qib_get_pkey(struct qib_ibport *ibp, unsigned 
index)
return ret;
 }
 
-static int qib_query_pkey(struct ib_device *ibdev, u8 port, u16 index,
- u16 *pkey)
-{
-   struct qib_devdata *dd = dd_from_ibdev(ibdev);
-   int ret;
-
-   if (index >= qib_get_npkeys(dd)) {
-   ret = -EINVAL;
-   goto bail;
-   }
-
-   *pkey = qib_get_pkey(to_iport(ibdev, port), index);
-   ret = 0;
-
-bail:
-   return ret;
-}
-
 /**
  * qib_alloc_ucontext - allocate a ucontest
  * @ibdev: the infiniband device
@@ -1956,7 +1938,7 @@ int qib_register_ib_device(struct qib_devdata *dd)
struct qib_ibdev *dev = >verbs_dev;
struct ib_device *ibdev = >rdi.ibdev;
struct qib_pportdata *ppd = dd->pport;
-   unsigned i;
+   unsigned i, ctxt;
int ret;
 
dev->qp_table_size = ib_qib_qp_table_size;
@@ -2068,7 +2050,7 @@ int qib_register_ib_device(struct qib_devdata *dd)
ibdev->modify_device = qib_modify_device;
ibdev->query_port = qib_query_port;
ibdev->modify_port = qib_modify_port;
-   ibdev->query_pkey = qib_query_pkey;
+   ibdev->query_pkey = NULL;
ibdev->query_gid = qib_query_gid;
ibdev->alloc_ucontext = qib_alloc_ucontext;
ibdev->dealloc_ucontext = qib_dealloc_ucontext;
@@ -2126,6 +2108,17 @@ int qib_register_ib_device(struct qib_devdata *dd)
dd->verbs_dev.rdi.flags = (RVT_FLAG_QP_INIT_DRIVER |
   RVT_FLAG_CQ_INIT_DRIVER);
dd->verbs_dev.rdi.dparms.lkey_table_size = qib_lkey_table_size;
+   dd->verbs_dev.rdi.dparms.nports = dd->num_pports;
+   dd->verbs_dev.rdi.dparms.npkeys = qib_get_npkeys(dd);
+
+   ppd = dd->pport;
+   for (i = 0; i < dd->num_pports; i++, ppd++) {
+   ctxt = ppd->hw_pidx;
+   rvt_init_port(>verbs_dev.rdi,
+ >ibport_data.rvp,
+ i,
+ dd->rcd[ctxt]->pkeys);
+   }
 
ret = rvt_register_device(>verbs_dev.rdi);
if (ret)

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/9] IB/qib: Implement qib support for AH notification

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

Additional work is required to create an AH. This patch adds support to
set the VL correctly.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/hw/qib/qib_verbs.c |   19 +++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index f70..9583bc1 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -1772,6 +1772,24 @@ int qib_check_ah(struct ib_device *ibdev, struct 
ib_ah_attr *ah_attr)
return 0;
 }
 
+static void qib_notify_new_ah(struct ib_device *ibdev,
+ struct ib_ah_attr *ah_attr,
+ struct rvt_ah *ah)
+{
+   struct qib_ibport *ibp;
+   struct qib_pportdata *ppd;
+
+   /*
+* Do not trust reading anything from rvt_ah at this point as it is not
+* done being setup. We can however modify things which we need to set.
+*/
+
+   ibp = to_iport(ibdev, ah_attr->port_num);
+   ppd = ppd_from_ibp(ibp);
+   ah->vl = ibp->sl_to_vl[ah->attr.sl];
+   ah->log_pmtu = ilog2(ppd->ibmtu);
+}
+
 struct ib_ah *qib_create_qp0_ah(struct qib_ibport *ibp, u16 dlid)
 {
struct ib_ah_attr attr;
@@ -2106,6 +2124,7 @@ int qib_register_ib_device(struct qib_devdata *dd)
dd->verbs_dev.rdi.driver_f.get_card_name = qib_get_card_name;
dd->verbs_dev.rdi.driver_f.get_pci_dev = qib_get_pci_dev;
dd->verbs_dev.rdi.driver_f.check_ah = qib_check_ah;
+   dd->verbs_dev.rdi.driver_f.notify_new_ah = qib_notify_new_ah;
dd->verbs_dev.rdi.dparms.props.max_pd = ib_qib_max_pds;
dd->verbs_dev.rdi.dparms.props.max_ah = ib_qib_max_ahs;
dd->verbs_dev.rdi.flags = (RVT_FLAG_QP_INIT_DRIVER |

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 5/9] IB/qib: Remove qpn, qp tables and related variables from qib

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

This patch removes the private queue pair structure and the table which
holds the queue pair numbers in favor of using what is provided by rdmavt.

Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/hw/qib/qib_qp.c|  120 ++---
 drivers/infiniband/hw/qib/qib_verbs.c |   29 +---
 drivers/infiniband/hw/qib/qib_verbs.h |   33 +
 3 files changed, 84 insertions(+), 98 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index 89da1e1..5968eed 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -42,25 +42,31 @@
 
 #include "qib.h"
 
-#define BITS_PER_PAGE   (PAGE_SIZE*BITS_PER_BYTE)
-#define BITS_PER_PAGE_MASK  (BITS_PER_PAGE-1)
+/*
+ * mask field which was present in now deleted qib_qpn_table
+ * is not present in rvt_qpn_table. Defining the same field
+ * as qpt_mask here instead of adding the mask field to
+ * rvt_qpn_table.
+ */
+static u16 qpt_mask;
 
-static inline unsigned mk_qpn(struct qib_qpn_table *qpt,
- struct qpn_map *map, unsigned off)
+static inline unsigned mk_qpn(struct rvt_qpn_table *qpt,
+ struct rvt_qpn_map *map, unsigned off)
 {
-   return (map - qpt->map) * BITS_PER_PAGE + off;
+   return (map - qpt->map) * RVT_BITS_PER_PAGE + off;
 }
 
-static inline unsigned find_next_offset(struct qib_qpn_table *qpt,
-   struct qpn_map *map, unsigned off,
+static inline unsigned find_next_offset(struct rvt_qpn_table *qpt,
+   struct rvt_qpn_map *map, unsigned off,
unsigned n)
 {
-   if (qpt->mask) {
+   if (qpt_mask) {
off++;
-   if (((off & qpt->mask) >> 1) >= n)
-   off = (off | qpt->mask) + 2;
-   } else
-   off = find_next_zero_bit(map->page, BITS_PER_PAGE, off);
+   if (((off & qpt_mask) >> 1) >= n)
+   off = (off | qpt_mask) + 2;
+   } else {
+   off = find_next_zero_bit(map->page, RVT_BITS_PER_PAGE, off);
+   }
return off;
 }
 
@@ -101,7 +107,7 @@ static u32 credit_table[31] = {
32768   /* 1E */
 };
 
-static void get_map_page(struct qib_qpn_table *qpt, struct qpn_map *map)
+static void get_map_page(struct rvt_qpn_table *qpt, struct rvt_qpn_map *map)
 {
unsigned long page = get_zeroed_page(GFP_KERNEL);
 
@@ -121,11 +127,11 @@ static void get_map_page(struct qib_qpn_table *qpt, 
struct qpn_map *map)
  * Allocate the next available QPN or
  * zero/one for QP type IB_QPT_SMI/IB_QPT_GSI.
  */
-static int alloc_qpn(struct qib_devdata *dd, struct qib_qpn_table *qpt,
+static int alloc_qpn(struct qib_devdata *dd, struct rvt_qpn_table *qpt,
 enum ib_qp_type type, u8 port)
 {
u32 i, offset, max_scan, qpn;
-   struct qpn_map *map;
+   struct rvt_qpn_map *map;
u32 ret;
 
if (type == IB_QPT_SMI || type == IB_QPT_GSI) {
@@ -143,12 +149,12 @@ static int alloc_qpn(struct qib_devdata *dd, struct 
qib_qpn_table *qpt,
}
 
qpn = qpt->last + 2;
-   if (qpn >= QPN_MAX)
+   if (qpn >= RVT_QPN_MAX)
qpn = 2;
-   if (qpt->mask && ((qpn & qpt->mask) >> 1) >= dd->n_krcv_queues)
-   qpn = (qpn | qpt->mask) + 2;
-   offset = qpn & BITS_PER_PAGE_MASK;
-   map = >map[qpn / BITS_PER_PAGE];
+   if (qpt_mask && ((qpn & qpt_mask) >> 1) >= dd->n_krcv_queues)
+   qpn = (qpn | qpt_mask) + 2;
+   offset = qpn & RVT_BITS_PER_PAGE_MASK;
+   map = >map[qpn / RVT_BITS_PER_PAGE];
max_scan = qpt->nmaps - !offset;
for (i = 0;;) {
if (unlikely(!map->page)) {
@@ -173,14 +179,14 @@ static int alloc_qpn(struct qib_devdata *dd, struct 
qib_qpn_table *qpt,
 * We just need to be sure we don't loop
 * forever.
 */
-   } while (offset < BITS_PER_PAGE && qpn < QPN_MAX);
+   } while (offset < RVT_BITS_PER_PAGE && qpn < RVT_QPN_MAX);
/*
 * In order to keep the number of pages allocated to a
 * minimum, we scan the all existing pages before increasing
 * the size of the bitmap table.
 */
if (++i > max_scan) {
-   if (qpt->nmaps == QPNMAP_ENTRIES)
+   if (qpt->nmaps == RVT_QPNMAP_ENTRIES)
break;

[PATCH 6/9] IB/qib: Delete QIB user context allocation and de-alloction functions

[Dennis Dalessandro]

From: Harish Chegondi 

IB user context alloc and dealloc functions have been added to rdmavt.
Delete the QIB user context alloc/dealloc functions and use the ones in
rdmavt.

Reviewed-by: Ira Weiny 
Signed-off-by: Harish Chegondi 
---
 drivers/infiniband/hw/qib/qib_verbs.c |   44 ++---
 1 files changed, 2 insertions(+), 42 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index 059ebda..f785680 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -133,16 +133,6 @@ const int ib_qib_state_ops[IB_QPS_ERR + 1] = {
QIB_POST_SEND_OK | QIB_FLUSH_SEND,
 };
 
-struct qib_ucontext {
-   struct ib_ucontext ibucontext;
-};
-
-static inline struct qib_ucontext *to_iucontext(struct ib_ucontext
- *ibucontext)
-{
-   return container_of(ibucontext, struct qib_ucontext, ibucontext);
-}
-
 /*
  * Translate ib_wr_opcode into ib_wc_opcode.
  */
@@ -1836,36 +1826,6 @@ unsigned qib_get_pkey(struct qib_ibport *ibp, unsigned 
index)
return ret;
 }
 
-/**
- * qib_alloc_ucontext - allocate a ucontest
- * @ibdev: the infiniband device
- * @udata: not used by the QLogic_IB driver
- */
-
-static struct ib_ucontext *qib_alloc_ucontext(struct ib_device *ibdev,
- struct ib_udata *udata)
-{
-   struct qib_ucontext *context;
-   struct ib_ucontext *ret;
-
-   context = kmalloc(sizeof(*context), GFP_KERNEL);
-   if (!context) {
-   ret = ERR_PTR(-ENOMEM);
-   goto bail;
-   }
-
-   ret = >ibucontext;
-
-bail:
-   return ret;
-}
-
-static int qib_dealloc_ucontext(struct ib_ucontext *context)
-{
-   kfree(to_iucontext(context));
-   return 0;
-}
-
 static void init_ibport(struct qib_pportdata *ppd)
 {
struct qib_verbs_counters cntrs;
@@ -2057,8 +2017,8 @@ int qib_register_ib_device(struct qib_devdata *dd)
ibdev->modify_port = qib_modify_port;
ibdev->query_pkey = NULL;
ibdev->query_gid = qib_query_gid;
-   ibdev->alloc_ucontext = qib_alloc_ucontext;
-   ibdev->dealloc_ucontext = qib_dealloc_ucontext;
+   ibdev->alloc_ucontext = NULL;
+   ibdev->dealloc_ucontext = NULL;
ibdev->alloc_pd = NULL;
ibdev->dealloc_pd = NULL;
ibdev->create_ah = NULL;

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 9/9] IB/qib: Remove create qp and create qp table functionality

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

Rely on rdmavt functions for creation of qp and qp table.  Function to
allocate a qpn is still being provided by qib as the algorithm to allocate
a qpn in qib is different from that of the algorithm in rdmavt.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/hw/qib/qib.h   |3 
 drivers/infiniband/hw/qib/qib_qp.c|  345 +++--
 drivers/infiniband/hw/qib/qib_verbs.c |   48 ++---
 drivers/infiniband/hw/qib/qib_verbs.h |   19 +-
 4 files changed, 57 insertions(+), 358 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index 309b6f3..29cbe67 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -1136,8 +1136,9 @@ extern spinlock_t qib_devs_lock;
 extern struct qib_devdata *qib_lookup(int unit);
 extern u32 qib_cpulist_count;
 extern unsigned long *qib_cpulist;
-
+extern u16 qpt_mask;
 extern unsigned qib_cc_table_size;
+
 int qib_init(struct qib_devdata *, int);
 int init_chip_wc_pat(struct qib_devdata *dd, u32);
 int qib_enable_wc(struct qib_devdata *dd);
diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index a6588e0..79a28a6 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -48,7 +48,7 @@
  * as qpt_mask here instead of adding the mask field to
  * rvt_qpn_table.
  */
-static u16 qpt_mask;
+u16 qpt_mask;
 
 static inline unsigned mk_qpn(struct rvt_qpn_table *qpt,
  struct rvt_qpn_map *map, unsigned off)
@@ -127,12 +127,15 @@ static void get_map_page(struct rvt_qpn_table *qpt, 
struct rvt_qpn_map *map)
  * Allocate the next available QPN or
  * zero/one for QP type IB_QPT_SMI/IB_QPT_GSI.
  */
-static int alloc_qpn(struct qib_devdata *dd, struct rvt_qpn_table *qpt,
-enum ib_qp_type type, u8 port)
+int alloc_qpn(struct rvt_dev_info *rdi, struct rvt_qpn_table *qpt,
+ enum ib_qp_type type, u8 port)
 {
u32 i, offset, max_scan, qpn;
struct rvt_qpn_map *map;
u32 ret;
+   struct qib_ibdev *verbs_dev = container_of(rdi, struct qib_ibdev, rdi);
+   struct qib_devdata *dd = container_of(verbs_dev, struct qib_devdata,
+ verbs_dev);
 
if (type == IB_QPT_SMI || type == IB_QPT_GSI) {
unsigned n;
@@ -295,18 +298,13 @@ static void remove_qp(struct qib_ibdev *dev, struct 
rvt_qp *qp)
 
 /**
  * qib_free_all_qps - check for QPs still in use
- * @qpt: the QP table to empty
- *
- * There should not be any QPs still in use.
- * Free memory for table.
  */
-unsigned qib_free_all_qps(struct qib_devdata *dd)
+unsigned qib_free_all_qps(struct rvt_dev_info *rdi)
 {
-   struct qib_ibdev *dev = >verbs_dev;
-   unsigned long flags;
-   struct rvt_qp *qp;
+   struct qib_ibdev *verbs_dev = container_of(rdi, struct qib_ibdev, rdi);
+   struct qib_devdata *dd = container_of(verbs_dev, struct qib_devdata,
+ verbs_dev);
unsigned n, qp_inuse = 0;
-   spinlock_t *qpt_lock_ptr; /* Pointer to make checkpatch happy */
 
for (n = 0; n < dd->num_pports; n++) {
struct qib_ibport *ibp = >pport[n].ibport_data;
@@ -320,21 +318,6 @@ unsigned qib_free_all_qps(struct qib_devdata *dd)
qp_inuse++;
rcu_read_unlock();
}
-
-   spin_lock_irqsave(>rdi.qp_dev->qpt_lock, flags);
-   qpt_lock_ptr = >rdi.qp_dev->qpt_lock;
-   for (n = 0; n < dev->rdi.qp_dev->qp_table_size; n++) {
-   qp = rcu_dereference_protected(dev->rdi.qp_dev->qp_table[n],
-  lockdep_is_held(qpt_lock_ptr));
-   RCU_INIT_POINTER(dev->rdi.qp_dev->qp_table[n], NULL);
-
-   for (; qp; qp = rcu_dereference_protected(qp->next,
-   lockdep_is_held(qpt_lock_ptr)))
-   qp_inuse++;
-   }
-   spin_unlock_irqrestore(>rdi.qp_dev->qpt_lock, flags);
-   synchronize_rcu();
-
return qp_inuse;
 }
 
@@ -373,57 +356,11 @@ struct rvt_qp *qib_lookup_qpn(struct qib_ibport *ibp, u32 
qpn)
return qp;
 }
 
-/**
- * qib_reset_qp - initialize the QP state to the reset state
- * @qp: the QP to reset
- * @type: the QP type
- */
-static void qib_reset_qp(struct rvt_qp *qp, enum ib_qp_type type)
+void notify_qp_reset(struct rvt_qp *qp)
 {
struct qib_qp_priv *priv = qp->priv;
-   qp->remote_qpn = 0;
-   qp->qkey = 0;
-   qp->qp_access_flags = 0;
+
atomic_set(>s_dma_busy, 0);
-   qp->s_flag

[PATCH 8/9] IB/qib: Use rdmavt send and receive flags

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

Use the definitions of the s_flags and r_flags which are now in rdmavt.

Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/hw/qib/qib_driver.c |   10 +-
 drivers/infiniband/hw/qib/qib_qp.c |   44 +
 drivers/infiniband/hw/qib/qib_rc.c |  152 
 drivers/infiniband/hw/qib/qib_ruc.c|   18 ++--
 drivers/infiniband/hw/qib/qib_sdma.c   |4 -
 drivers/infiniband/hw/qib/qib_uc.c |   16 ++-
 drivers/infiniband/hw/qib/qib_ud.c |   22 ++---
 drivers/infiniband/hw/qib/qib_verbs.c  |   34 ---
 drivers/infiniband/hw/qib/qib_verbs.h  |   78 +---
 9 files changed, 153 insertions(+), 225 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_driver.c 
b/drivers/infiniband/hw/qib/qib_driver.c
index e8b239c..ad41df3 100644
--- a/drivers/infiniband/hw/qib/qib_driver.c
+++ b/drivers/infiniband/hw/qib/qib_driver.c
@@ -414,7 +414,7 @@ static u32 qib_rcv_hdrerr(struct qib_ctxtdata *rcd, struct 
qib_pportdata *ppd,
 */
if (list_empty(>rspwait)) {
qp->r_flags |=
-   QIB_R_RSP_NAK;
+   RVT_R_RSP_NAK;
atomic_inc(
>refcount);
list_add_tail(
@@ -583,14 +583,14 @@ move_along:
 */
list_for_each_entry_safe(qp, nqp, >qp_wait_list, rspwait) {
list_del_init(>rspwait);
-   if (qp->r_flags & QIB_R_RSP_NAK) {
-   qp->r_flags &= ~QIB_R_RSP_NAK;
+   if (qp->r_flags & RVT_R_RSP_NAK) {
+   qp->r_flags &= ~RVT_R_RSP_NAK;
qib_send_rc_ack(qp);
}
-   if (qp->r_flags & QIB_R_RSP_SEND) {
+   if (qp->r_flags & RVT_R_RSP_SEND) {
unsigned long flags;
 
-   qp->r_flags &= ~QIB_R_RSP_SEND;
+   qp->r_flags &= ~RVT_R_RSP_SEND;
spin_lock_irqsave(>s_lock, flags);
if (ib_qib_state_ops[qp->state] &
QIB_PROCESS_OR_FLUSH_SEND)
diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index 5968eed..a6588e0 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -385,7 +385,7 @@ static void qib_reset_qp(struct rvt_qp *qp, enum ib_qp_type 
type)
qp->qkey = 0;
qp->qp_access_flags = 0;
atomic_set(>s_dma_busy, 0);
-   qp->s_flags &= QIB_S_SIGNAL_REQ_WR;
+   qp->s_flags &= RVT_S_SIGNAL_REQ_WR;
qp->s_hdrwords = 0;
qp->s_wqe = NULL;
qp->s_draining = 0;
@@ -430,7 +430,7 @@ static void clear_mr_refs(struct rvt_qp *qp, int clr_sends)
 {
unsigned n;
 
-   if (test_and_clear_bit(QIB_R_REWIND_SGE, >r_aflags))
+   if (test_and_clear_bit(RVT_R_REWIND_SGE, >r_aflags))
qib_put_ss(>s_rdma_read_sge);
 
qib_put_ss(>r_sge);
@@ -494,22 +494,22 @@ int qib_error_qp(struct rvt_qp *qp, enum ib_wc_status err)
 
qp->state = IB_QPS_ERR;
 
-   if (qp->s_flags & (QIB_S_TIMER | QIB_S_WAIT_RNR)) {
-   qp->s_flags &= ~(QIB_S_TIMER | QIB_S_WAIT_RNR);
+   if (qp->s_flags & (RVT_S_TIMER | RVT_S_WAIT_RNR)) {
+   qp->s_flags &= ~(RVT_S_TIMER | RVT_S_WAIT_RNR);
del_timer(>s_timer);
}
 
-   if (qp->s_flags & QIB_S_ANY_WAIT_SEND)
-   qp->s_flags &= ~QIB_S_ANY_WAIT_SEND;
+   if (qp->s_flags & RVT_S_ANY_WAIT_SEND)
+   qp->s_flags &= ~RVT_S_ANY_WAIT_SEND;
 
spin_lock(>rdi.pending_lock);
-   if (!list_empty(>iowait) && !(qp->s_flags & QIB_S_BUSY)) {
-   qp->s_flags &= ~QIB_S_ANY_WAIT_IO;
+   if (!list_empty(>iowait) && !(qp->s_flags & RVT_S_BUSY)) {
+   qp->s_flags &= ~RVT_S_ANY_WAIT_IO;
list_del_init(>iowait);
}
spin_unlock(>rdi.pending_lock);
 
-   if (!(qp->s_flags & QIB_S_BUSY)) {
+   if (!(qp->s_flags & RVT_S_BUSY)) {
qp->s_hdrwords = 0;
if (qp->s_rdma_mr) {

[PATCH 7/9] IB/qib: Remove qib_query_device function

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

Removed qib_query_device function to use rdmavt rvt_query_device function
The device attributes still need to be filled in by the driver.

Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/hw/qib/qib_verbs.c |  102 -
 1 files changed, 50 insertions(+), 52 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index f785680..b20bc6f 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -1571,55 +1571,6 @@ full:
}
 }
 
-static int qib_query_device(struct ib_device *ibdev, struct ib_device_attr 
*props,
-   struct ib_udata *uhw)
-{
-   struct qib_devdata *dd = dd_from_ibdev(ibdev);
-   struct qib_ibdev *dev = to_idev(ibdev);
-
-   if (uhw->inlen || uhw->outlen)
-   return -EINVAL;
-   memset(props, 0, sizeof(*props));
-
-   props->device_cap_flags = IB_DEVICE_BAD_PKEY_CNTR |
-   IB_DEVICE_BAD_QKEY_CNTR | IB_DEVICE_SHUTDOWN_PORT |
-   IB_DEVICE_SYS_IMAGE_GUID | IB_DEVICE_RC_RNR_NAK_GEN |
-   IB_DEVICE_PORT_ACTIVE_EVENT | IB_DEVICE_SRQ_RESIZE;
-   props->page_size_cap = PAGE_SIZE;
-   props->vendor_id =
-   QIB_SRC_OUI_1 << 16 | QIB_SRC_OUI_2 << 8 | QIB_SRC_OUI_3;
-   props->vendor_part_id = dd->deviceid;
-   props->hw_ver = dd->minrev;
-   props->sys_image_guid = ib_qib_sys_image_guid;
-   props->max_mr_size = ~0ULL;
-   props->max_qp = ib_qib_max_qps;
-   props->max_qp_wr = ib_qib_max_qp_wrs;
-   props->max_sge = ib_qib_max_sges;
-   props->max_sge_rd = ib_qib_max_sges;
-   props->max_cq = ib_qib_max_cqs;
-   props->max_ah = ib_qib_max_ahs;
-   props->max_cqe = ib_qib_max_cqes;
-   props->max_mr = dev->rdi.lkey_table.max;
-   props->max_fmr = dev->rdi.lkey_table.max;
-   props->max_map_per_fmr = 32767;
-   props->max_pd = dev->rdi.dparms.props.max_pd;
-   props->max_qp_rd_atom = QIB_MAX_RDMA_ATOMIC;
-   props->max_qp_init_rd_atom = 255;
-   /* props->max_res_rd_atom */
-   props->max_srq = ib_qib_max_srqs;
-   props->max_srq_wr = ib_qib_max_srq_wrs;
-   props->max_srq_sge = ib_qib_max_srq_sges;
-   /* props->local_ca_ack_delay */
-   props->atomic_cap = IB_ATOMIC_GLOB;
-   props->max_pkeys = qib_get_npkeys(dd);
-   props->max_mcast_grp = ib_qib_max_mcast_grps;
-   props->max_mcast_qp_attach = ib_qib_max_mcast_qp_attached;
-   props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
-   props->max_mcast_grp;
-
-   return 0;
-}
-
 static int qib_query_port(struct ib_device *ibdev, u8 port,
  struct ib_port_attr *props)
 {
@@ -1889,6 +1840,53 @@ static int qib_port_immutable(struct ib_device *ibdev, 
u8 port_num,
 }
 
 /**
+ * qib_fill_device_attr - Fill in rvt dev info device attributes.
+ * @dd: the device data structure
+ */
+static void qib_fill_device_attr(struct qib_devdata *dd)
+{
+   struct rvt_dev_info *rdi = >verbs_dev.rdi;
+
+   memset(>dparms.props, 0, sizeof(rdi->dparms.props));
+
+   rdi->dparms.props.max_pd = ib_qib_max_pds;
+   rdi->dparms.props.max_ah = ib_qib_max_ahs;
+   rdi->dparms.props.device_cap_flags = IB_DEVICE_BAD_PKEY_CNTR |
+   IB_DEVICE_BAD_QKEY_CNTR | IB_DEVICE_SHUTDOWN_PORT |
+   IB_DEVICE_SYS_IMAGE_GUID | IB_DEVICE_RC_RNR_NAK_GEN |
+   IB_DEVICE_PORT_ACTIVE_EVENT | IB_DEVICE_SRQ_RESIZE;
+   rdi->dparms.props.page_size_cap = PAGE_SIZE;
+   rdi->dparms.props.vendor_id =
+   QIB_SRC_OUI_1 << 16 | QIB_SRC_OUI_2 << 8 | QIB_SRC_OUI_3;
+   rdi->dparms.props.vendor_part_id = dd->deviceid;
+   rdi->dparms.props.hw_ver = dd->minrev;
+   rdi->dparms.props.sys_image_guid = ib_qib_sys_image_guid;
+   rdi->dparms.props.max_mr_size = ~0ULL;
+   rdi->dparms.props.max_qp = ib_qib_max_qps;
+   rdi->dparms.props.max_qp_wr = ib_qib_max_qp_wrs;
+   rdi->dparms.props.max_sge = ib_qib_max_sges;
+   rdi->dparms.props.max_sge_rd = ib_qib_max_sges;
+   rdi->dparms.props.max_cq = ib_qib_max_cqs;
+   rdi->dparms.props.max_cqe = ib_qib_max_cqes;
+   rdi->dparms.props.max_ah = ib_qib_max_ahs;
+   rdi->dparms.props.max_mr = rdi->lkey_table.max;
+   rdi->dparms.props.max_fmr = rdi->lkey_table.max;
+   rdi->dparms.props.max_map_per_fmr = 32767;
+   rdi->dparms.props.max_qp_rd_atom = QIB_MAX_RDMA_ATOMIC;
+   rdi->dparms.props.max_qp_init_rd_atom = 255;
+   rdi->dparms.props.max_srq = i

[PATCH 1/9] IB/qib: Remove ibport and use rdmavt version

[Dennis Dalessandro]

From: Harish Chegondi <harish.chego...@intel.com>

Remove several ibport members from qib and use the rdmavt version. rc_acks,
rc_qacks, and rc_delayed_comp are defined as per CPU variables in rdmavt.
Add support for these rdmavt per CPU variables which were not per cpu
variables in qib ibport structure.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Harish Chegondi <harish.chego...@intel.com>
---
 drivers/infiniband/hw/qib/qib_driver.c  |4 
 drivers/infiniband/hw/qib/qib_iba6120.c |8 -
 drivers/infiniband/hw/qib/qib_iba7322.c |2 
 drivers/infiniband/hw/qib/qib_init.c|   10 +
 drivers/infiniband/hw/qib/qib_mad.c |  240 ++-
 drivers/infiniband/hw/qib/qib_qp.c  |   22 +-
 drivers/infiniband/hw/qib/qib_rc.c  |   30 ++-
 drivers/infiniband/hw/qib/qib_ruc.c |   14 +-
 drivers/infiniband/hw/qib/qib_sdma.c|2 
 drivers/infiniband/hw/qib/qib_sysfs.c   |   65 +++
 drivers/infiniband/hw/qib/qib_uc.c  |2 
 drivers/infiniband/hw/qib/qib_ud.c  |   14 +-
 drivers/infiniband/hw/qib/qib_verbs.c   |   52 +++---
 drivers/infiniband/hw/qib/qib_verbs.h   |   46 -
 drivers/infiniband/hw/qib/qib_verbs_mcast.c |   28 ++-
 15 files changed, 281 insertions(+), 258 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_driver.c 
b/drivers/infiniband/hw/qib/qib_driver.c
index eafdee9..e8b239c 100644
--- a/drivers/infiniband/hw/qib/qib_driver.c
+++ b/drivers/infiniband/hw/qib/qib_driver.c
@@ -379,7 +379,7 @@ static u32 qib_rcv_hdrerr(struct qib_ctxtdata *rcd, struct 
qib_pportdata *ppd,
/* Check for valid receive state. */
if (!(ib_qib_state_ops[qp->state] &
  QIB_PROCESS_RECV_OK)) {
-   ibp->n_pkt_drops++;
+   ibp->rvp.n_pkt_drops++;
goto unlock;
}
 
@@ -399,7 +399,7 @@ static u32 qib_rcv_hdrerr(struct qib_ctxtdata *rcd, struct 
qib_pportdata *ppd,
IB_OPCODE_RC_RDMA_READ_RESPONSE_FIRST) {
diff = qib_cmp24(psn, qp->r_psn);
if (!qp->r_nak_state && diff >= 0) {
-   ibp->n_rc_seqnak++;
+   ibp->rvp.n_rc_seqnak++;
qp->r_nak_state =
IB_NAK_PSN_ERROR;
/* Use the expected PSN. */
diff --git a/drivers/infiniband/hw/qib/qib_iba6120.c 
b/drivers/infiniband/hw/qib/qib_iba6120.c
index 4b92780..a3733f2 100644
--- a/drivers/infiniband/hw/qib/qib_iba6120.c
+++ b/drivers/infiniband/hw/qib/qib_iba6120.c
@@ -2956,13 +2956,13 @@ static void pma_6120_timer(unsigned long data)
struct qib_ibport *ibp = >ibport_data;
unsigned long flags;
 
-   spin_lock_irqsave(>lock, flags);
+   spin_lock_irqsave(>rvp.lock, flags);
if (cs->pma_sample_status == IB_PMA_SAMPLE_STATUS_STARTED) {
cs->pma_sample_status = IB_PMA_SAMPLE_STATUS_RUNNING;
qib_snapshot_counters(ppd, >sword, >rword,
  >spkts, >rpkts, >xmit_wait);
mod_timer(>pma_timer,
- jiffies + usecs_to_jiffies(ibp->pma_sample_interval));
+ jiffies + usecs_to_jiffies(ibp->rvp.pma_sample_interval));
} else if (cs->pma_sample_status == IB_PMA_SAMPLE_STATUS_RUNNING) {
u64 ta, tb, tc, td, te;
 
@@ -2975,11 +2975,11 @@ static void pma_6120_timer(unsigned long data)
cs->rpkts = td - cs->rpkts;
cs->xmit_wait = te - cs->xmit_wait;
}
-   spin_unlock_irqrestore(>lock, flags);
+   spin_unlock_irqrestore(>rvp.lock, flags);
 }
 
 /*
- * Note that the caller has the ibp->lock held.
+ * Note that the caller has the ibp->rvp.lock held.
  */
 static void qib_set_cntr_6120_sample(struct qib_pportdata *ppd, u32 intv,
 u32 start)
diff --git a/drivers/infiniband/hw/qib/qib_iba7322.c 
b/drivers/infiniband/hw/qib/qib_iba7322.c
index 1fbe308..ca28c19 100644
--- a/drivers/infiniband/hw/qib/qib_iba7322.c
+++ b/drivers/infiniband/hw/qib/qib_iba7322.c
@@ -5497,7 +5497,7 @@ static void try_7322_ipg(struct qib_pportdata *ppd)
unsigned delay;
int ret;
 
-   agent = ibp->send_agent;
+   agent = ibp->rvp.send_agent;
if (!agent)
goto retry;
 
diff --git a/drivers/infiniband/hw/qib/qib_init.c 
b/

[PATCH v3 03/36] IB/rdmavt: Add protection domain to rdmavt.

[Dennis Dalessandro]

Add datastructure for and allocation/deallocation of protection domains for
RDMAVT.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
update comment about protection domain limit

 drivers/infiniband/sw/rdmavt/Makefile |3 +
 drivers/infiniband/sw/rdmavt/pd.c |  103 +
 drivers/infiniband/sw/rdmavt/pd.h |   58 +++
 drivers/infiniband/sw/rdmavt/vt.c |   12 
 drivers/infiniband/sw/rdmavt/vt.h |1 
 include/rdma/rdma_vt.h|   34 ++-
 6 files changed, 206 insertions(+), 5 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/pd.c
 create mode 100644 drivers/infiniband/sw/rdmavt/pd.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 134d2d0..c6751bb 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,5 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o dma.o
+rdmavt-y := vt.o dma.o pd.o
+
diff --git a/drivers/infiniband/sw/rdmavt/pd.c 
b/drivers/infiniband/sw/rdmavt/pd.c
new file mode 100644
index 000..e6e153f
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/pd.c
@@ -0,0 +1,103 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include 
+#include "pd.h"
+
+struct ib_pd *rvt_alloc_pd(struct ib_device *ibdev,
+  struct ib_ucontext *context,
+  struct ib_udata *udata)
+{
+   struct rvt_dev_info *dev = ib_to_rvt(ibdev);
+   struct rvt_pd *pd;
+   struct ib_pd *ret;
+
+   pd = kmalloc(sizeof(*pd), GFP_KERNEL);
+   if (!pd) {
+   ret = ERR_PTR(-ENOMEM);
+   goto bail;
+   }
+   /*
+* While we could continue allocating protecetion domains, being
+* constrained only by system resources. The IBTA spec defines that
+* there is a max_pd limit that can be set and we need to check for
+* that.
+*/
+
+   spin_lock(>n_pds_lock);
+   if (dev->n_pds_allocated == dev->dparms.max_pds) {
+   spin_unlock(>n_pds_lock);
+   kfree(pd);
+   ret = ERR_PTR(-ENOMEM);
+   goto bail;
+   }
+
+   dev->n_pds_allocated++;
+   spin_unlock(>n_pds_lock);
+
+   /* ib_alloc_pd() will initialize pd->ibpd. */
+   pd->user = udata ? 1 : 0;
+
+   ret = >ibpd;
+
+bail:
+   return ret;
+}
+
+int rvt_dealloc_pd(struct ib_pd *ibpd)
+{
+   struct rvt_pd *pd = ibpd_to_rvtpd(ibpd);
+   struct rvt_dev_info *dev = ib_to_rvt(ibpd->dev

[PATCH v3 02/36] IB/rdmavt: Consolidate dma ops in rdmavt.

[Dennis Dalessandro]

This patch adds dma functions to rdmavt. The source is hfi1's version of
dma.c which will be removed by a subsequent hfi1 patch.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
return 0 in rvt_map_sg instead of BAD_DMA_ADDRESS
remove include of dma.h from dma.c

Changes since v2:
add back in the include of dma.h in dma.c to avoid sparse error

 drivers/infiniband/sw/rdmavt/Makefile |2 
 drivers/infiniband/sw/rdmavt/dma.c|  184 +
 drivers/infiniband/sw/rdmavt/dma.h|   53 ++
 drivers/infiniband/sw/rdmavt/vt.c |   10 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 249 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/dma.c
 create mode 100644 drivers/infiniband/sw/rdmavt/dma.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 98a664d..134d2d0 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o
+rdmavt-y := vt.o dma.o
diff --git a/drivers/infiniband/sw/rdmavt/dma.c 
b/drivers/infiniband/sw/rdmavt/dma.c
new file mode 100644
index 000..c070141
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/dma.c
@@ -0,0 +1,184 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+#include 
+#include 
+#include 
+
+#include "dma.h"
+
+#define BAD_DMA_ADDRESS ((u64)0)
+
+/*
+ * The following functions implement driver specific replacements
+ * for the ib_dma_*() functions.
+ *
+ * These functions return kernel virtual addresses instead of
+ * device bus addresses since the driver uses the CPU to copy
+ * data instead of using hardware DMA.
+ */
+
+static int rvt_mapping_error(struct ib_device *dev, u64 dma_addr)
+{
+   return dma_addr == BAD_DMA_ADDRESS;
+}
+
+static u64 rvt_dma_map_single(struct ib_device *dev, void *cpu_addr,
+ size_t size, enum dma_data_direction direction)
+{
+   if (WARN_ON(!valid_dma_direction(direction)))
+   return BAD_DMA_ADDRESS;
+
+   return (u64)cpu_addr;
+}
+
+static void rvt_dma_unmap_single(struct ib_device *dev, u64 addr, size_t size,
+enum dma_data_direction direction)
+{
+   /* This is a stub, nothing to be done here */
+}
+
+static u64 rvt_dma_map_page(struct ib_device *dev, struct page *page,
+   unsigned long offset, size_t size,
+   enum dma_data_direction direction)
+{
+   u64 addr;
+
+   if (WARN_ON(!valid_dma_direction(direction)))

[PATCH v3 09/36] IB/rdmavt: Add query gid stub

[Dennis Dalessandro]

The handler for query gid operates along the same lines as the query pkey
handler. The driver will take care to keep the guid table updated.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |   22 ++
 1 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index fc5372d..e95f197 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -157,6 +157,27 @@ static int rvt_query_pkey(struct ib_device *ibdev, u8 
port, u16 index,
return 0;
 }
 
+/**
+ * rvt_query_gid - Return a gid from the table
+ * @ibdev: Verbs IB dev
+ * @port: Port number
+ * @index: = Index in table
+ * @gid: Gid to return
+ *
+ * Returns 0 on success
+ */
+static int rvt_query_gid(struct ib_device *ibdev, u8 port,
+int index, union ib_gid *gid)
+{
+   /*
+* Driver is responsible for updating the guid table. Which will be used
+* to craft the return value. This will work similar to how query_pkey()
+* is being done.
+*/
+
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -175,6 +196,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, query_port);
CHECK_DRIVER_OVERRIDE(rdi, modify_port);
CHECK_DRIVER_OVERRIDE(rdi, query_pkey);
+   CHECK_DRIVER_OVERRIDE(rdi, query_gid);
 
/* DMA Operations */
rdi->ibdev.dma_ops =

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 16/36] IB/rdmavt: Add process MAD stub

[Dennis Dalessandro]

This adds the stub for process mad. More study is needed to determine the
final MAD interaction between the driver and rvt.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/Makefile |2 -
 drivers/infiniband/sw/rdmavt/mad.c|   85 +
 drivers/infiniband/sw/rdmavt/mad.h|   59 +++
 3 files changed, 145 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/mad.c
 create mode 100644 drivers/infiniband/sw/rdmavt/mad.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index d2af114..fe65410 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mcast.o mr.o pd.o qp.o srq.o
+rdmavt-y := vt.o ah.o dma.o mad.o mcast.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/mad.c 
b/drivers/infiniband/sw/rdmavt/mad.c
new file mode 100644
index 000..eef7029
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mad.c
@@ -0,0 +1,85 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "mad.h"
+
+/**
+ * rvt_process_mad - process an incoming MAD packet
+ * @ibdev: the infiniband device this packet came in on
+ * @mad_flags: MAD flags
+ * @port: the port number this packet came in on
+ * @in_wc: the work completion entry for this packet
+ * @in_grh: the global route header for this packet
+ * @in_mad: the incoming MAD
+ * @out_mad: any outgoing MAD reply
+ *
+ * Returns IB_MAD_RESULT_SUCCESS if this is a MAD that we are not
+ * interested in processing.
+ *
+ * Note that the verbs framework has already done the MAD sanity checks,
+ * and hop count/pointer updating for IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE
+ * MADs.
+ *
+ * This is called by the ib_mad module.
+ */
+int rvt_process_mad(struct ib_device *ibdev, int mad_flags, u8 port,
+   const struct ib_wc *in_wc, const struct ib_grh *in_grh,
+   const struct ib_mad_hdr *in, size_t in_mad_size,
+   struct ib_mad_hdr *out, size_t *out_mad_size,
+   u16 *out_mad_pkey_index)
+{
+   /*
+* Drivers will need to provide a number of things. For exmaple counters
+* will need to be maintained by the driver but shoud live in the rvt
+* structure. More study will be needed to finalize the interface
+* between drivers and rvt for mad packets.
+*
+*VT-DRIVER-API: 
+*
+*/
+   return IB_MAD_RESULT_FAILURE;
+}
diff --git a/drivers/infiniband/sw

[PATCH v3 27/36] IB/rdmavt: Do not use rvt prints which rely on driver too early

[Dennis Dalessandro]

Trying to print debug and error messages with the rdmavt helpers will not
work out so well if the drivers have not provided the get_card and get
pci functions. Use the normal pr_error instead until we can check this.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 516c810..e92af9c 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -266,7 +266,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
/* Mem Region */
ret = rvt_driver_mr_init(rdi);
if (ret) {
-   rvt_pr_err(rdi, "Error in driver MR init.\n");
+   pr_err("Error in driver MR init.\n");
goto bail_no_mr;
}
 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 20/36] IB/rdmavt: Add post send and recv stubs

[Dennis Dalessandro]

This adds the post sned and recv function stubs.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Fix typo in subject
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/qp.c |   62 +
 drivers/infiniband/sw/rdmavt/qp.h |7 
 drivers/infiniband/sw/rdmavt/vt.c |3 ++
 3 files changed, 71 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.c 
b/drivers/infiniband/sw/rdmavt/qp.c
index a59f28d..23a5f68 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -118,3 +118,65 @@ int rvt_query_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
 {
return -EOPNOTSUPP;
 }
+
+/**
+ * rvt_post_receive - post a receive on a QP
+ * @ibqp: the QP to post the receive on
+ * @wr: the WR to post
+ * @bad_wr: the first bad WR is put here
+ *
+ * This may be called from interrupt context.
+ */
+int rvt_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr)
+{
+   /*
+* When a packet arrives the driver needs to call up to rvt to process
+* the packet. The UD, RC, UC processing will be done in rvt, however
+* the driver should be able to override this if it so choses. Perhaps a
+* set of function pointers set up at registration time.
+*/
+
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_post_send - post a send on a QP
+ * @ibqp: the QP to post the send on
+ * @wr: the list of work requests to post
+ * @bad_wr: the first bad WR is put here
+ *
+ * This may be called from interrupt context.
+ */
+int rvt_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
+ struct ib_send_wr **bad_wr)
+{
+   /*
+* VT-DRIVER-API: do_send()
+* Driver needs to have a do_send() call which is a single entry point
+* to take an already formed packet and throw it out on the wire. Once
+* the packet is sent the driver needs to make an upcall to rvt so the
+* completion queue can be notified and/or any other outstanding
+* work/book keeping can be finished.
+*
+* Note that there should also be a way for rvt to protect itself
+* against hangs in the driver layer. If a send doesn't actually
+* complete in a timely manor rvt needs to return an error event.
+*/
+
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_post_srq_receive - post a receive on a shared receive queue
+ * @ibsrq: the SRQ to post the receive on
+ * @wr: the list of work requests to post
+ * @bad_wr: A pointer to the first WR to cause a problem is put here
+ *
+ * This may be called from interrupt context.
+ */
+int rvt_post_srq_recv(struct ib_srq *ibsrq, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr)
+{
+   return -EOPNOTSUPP;
+}
diff --git a/drivers/infiniband/sw/rdmavt/qp.h 
b/drivers/infiniband/sw/rdmavt/qp.h
index 43acba0..10bc636 100644
--- a/drivers/infiniband/sw/rdmavt/qp.h
+++ b/drivers/infiniband/sw/rdmavt/qp.h
@@ -63,5 +63,10 @@ int rvt_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
 int rvt_destroy_qp(struct ib_qp *ibqp);
 int rvt_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 int attr_mask, struct ib_qp_init_attr *init_attr);
-
+int rvt_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr);
+int rvt_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
+ struct ib_send_wr **bad_wr);
+int rvt_post_srq_recv(struct ib_srq *ibsrq, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr);
 #endif  /* DEF_RVTQP_H */
diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 7fd879f..367bc45 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -232,6 +232,9 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, modify_qp);
CHECK_DRIVER_OVERRIDE(rdi, destroy_qp);
CHECK_DRIVER_OVERRIDE(rdi, query_qp);
+   CHECK_DRIVER_OVERRIDE(rdi, post_send);
+   CHECK_DRIVER_OVERRIDE(rdi, post_recv);
+   CHECK_DRIVER_OVERRIDE(rdi, post_srq_recv);
 
/* Address Handle */
CHECK_DRIVER_OVERRIDE(rdi, create_ah);

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 23/36] IB/rdmavt: Move driver helper functions to a common structure

[Dennis Dalessandro]

Drivers are going to need to provide multiple functions for rdmavt to
call in to. We already have one, so go ahead and push this into a
data structure designated for driver supplied functions.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |6 +-
 include/rdma/rdma_vt.h|   22 +++---
 2 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 367bc45..b4ce97e 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -280,8 +280,12 @@ int rvt_register_device(struct rvt_dev_info *rdi)
spin_lock_init(>n_pds_lock);
rdi->n_pds_allocated = 0;
 
+   /* Validate that drivers have provided the right functions */
+   if (!rdi->driver_f.port_callback)
+   return -EINVAL;
+
/* We are now good to announce we exist */
-   return ib_register_device(>ibdev, rdi->port_callback);
+   return ib_register_device(>ibdev, rdi->driver_f.port_callback);
 }
 EXPORT_SYMBOL(rvt_register_device);
 
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 9baa7f0..e0beedc 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -383,6 +383,19 @@ struct rvt_driver_params {
 */
 };
 
+/*
+ * Functions that drivers are required to support
+ */
+struct rvt_driver_provided {
+   /*
+* The work to create port files in /sys/class Infiniband is different
+* depending on the driver. This should not be extracted away and
+* instead drivers are responsible for setting the correct callback for
+* this.
+*/
+   int (*port_callback)(struct ib_device *, u8, struct kobject *);
+};
+
 /* Protection domain */
 struct rvt_pd {
struct ib_pd ibpd;
@@ -407,13 +420,8 @@ struct rvt_dev_info {
 
/* PKey Table goes here */
 
-   /*
-* The work to create port files in /sys/class Infiniband is different
-* depending on the driver. This should not be extracted away and
-* instead drivers are responsible for setting the correct callback for
-* this.
-*/
-   int (*port_callback)(struct ib_device *, u8, struct kobject *);
+   /* Driver specific helper functions */
+   struct rvt_driver_provided driver_f;
 
/* Internal use */
int n_pds_allocated;

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 22/36] IB/rdmavt: Add queue pair data structure to rdmavt

[Dennis Dalessandro]

Add queue pair data structure as well as supporting structures to rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Rename rvt_sge.m => rvt_sge.cur_map
Rename rvt_sge.n => rvt_sge.cur

 drivers/infiniband/sw/rdmavt/qp.h |5 -
 include/rdma/rdma_vt.h|  233 +
 2 files changed, 233 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.h 
b/drivers/infiniband/sw/rdmavt/qp.h
index 10bc636..9c2999d 100644
--- a/drivers/infiniband/sw/rdmavt/qp.h
+++ b/drivers/infiniband/sw/rdmavt/qp.h
@@ -50,11 +50,6 @@
 
 #include 
 
-struct rvt_qp {
-   struct ib_qp *ibqp;
-   /* Other stuff */
-};
-
 struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
struct ib_qp_init_attr *init_attr,
struct ib_udata *udata);
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index f232e39..9baa7f0 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -106,6 +106,239 @@ struct rvt_lkey_table {
 /* End Memmory Region */
 
 /*
+ * Things needed for the Queue Pair definition. Like the MR stuff above the
+ * following should probably get moved to qp.h once drivers stop trying to make
+ * and manipulate thier own QPs. For the few instnaces where a driver may need
+ * to look into a queue pair there should be a pointer to a driver priavte data
+ * structure that they can look at.
+ */
+
+/*
+ * These keep track of the copy progress within a memory region.
+ * Used by the verbs layer.
+ */
+struct rvt_sge {
+   struct rvt_mregion *mr;
+   void *vaddr;/* kernel virtual address of segment */
+   u32 sge_length; /* length of the SGE */
+   u32 length; /* remaining length of the segment */
+   u16 m;  /* current index: mr->map[m] */
+   u16 n;  /* current index: mr->map[m]->segs[n] */
+};
+
+/*
+ * Send work request queue entry.
+ * The size of the sg_list is determined when the QP is created and stored
+ * in qp->s_max_sge.
+ */
+struct rvt_swqe {
+   union {
+   struct ib_send_wr wr;   /* don't use wr.sg_list */
+   struct ib_ud_wr ud_wr;
+   struct ib_reg_wr reg_wr;
+   struct ib_rdma_wr rdma_wr;
+   struct ib_atomic_wr atomic_wr;
+   };
+   u32 psn;/* first packet sequence number */
+   u32 lpsn;   /* last packet sequence number */
+   u32 ssn;/* send sequence number */
+   u32 length; /* total length of data in sg_list */
+   struct rvt_sge sg_list[0];
+};
+
+/*
+ * Receive work request queue entry.
+ * The size of the sg_list is determined when the QP (or SRQ) is created
+ * and stored in qp->r_rq.max_sge (or srq->rq.max_sge).
+ */
+struct rvt_rwqe {
+   u64 wr_id;
+   u8 num_sge;
+   struct ib_sge sg_list[0];
+};
+
+/*
+ * This structure is used to contain the head pointer, tail pointer,
+ * and receive work queue entries as a single memory allocation so
+ * it can be mmap'ed into user space.
+ * Note that the wq array elements are variable size so you can't
+ * just index into the array to get the N'th element;
+ * use get_rwqe_ptr() instead.
+ */
+struct rvt_rwq {
+   u32 head;   /* new work requests posted to the head */
+   u32 tail;   /* receives pull requests from here. */
+   struct rvt_rwqe wq[0];
+};
+
+struct rvt_rq {
+   struct rvt_rwq *wq;
+   u32 size;   /* size of RWQE array */
+   u8 max_sge;
+   /* protect changes in this struct */
+   spinlock_t lock cacheline_aligned_in_smp;
+};
+
+/*
+ * This structure is used by rvt_mmap() to validate an offset
+ * when an mmap() request is made.  The vm_area_struct then uses
+ * this as its vm_private_data.
+ */
+struct rvt_mmap_info {
+   struct list_head pending_mmaps;
+   struct ib_ucontext *context;
+   void *obj;
+   __u64 offset;
+   struct kref ref;
+   unsigned size;
+};
+
+#define RVT_MAX_RDMA_ATOMIC16
+
+/*
+ * This structure holds the information that the send tasklet needs
+ * to send a RDMA read response or atomic operation.
+ */
+struct rvt_ack_entry {
+   u8 opcode;
+   u8 sent;
+   u32 psn;
+   u32 lpsn;
+   union {
+   struct rvt_sge rdma_sge;
+   u64 atomic_data;
+   };
+};
+
+struct rvt_sge_state {
+   struct rvt_sge *sg_list;  /* next SGE to be used if any */
+   struct rvt_sge sge;   /* progress state for the current SGE */
+   u32 total_len;
+   u8 num_sge;
+};
+
+/*
+ * Variables prefixed with s_ are for the requester (sender).
+ * Variables prefixed with r_ are for the responder (receiver).
+ * Variables pr

[PATCH v3 25/36] IB/rdmavt: Add the start of capability flags

[Dennis Dalessandro]

Drivers will need a set of flags to dictate behavior to rdmavt. This patch
adds a placeholder and a spot for it to live, as well as a few flags
that will be used.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 include/rdma/rdma_vt.h |   12 
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 4b83770..b44ac17 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -56,6 +56,16 @@
 #include "ib_verbs.h"
 
 /*
+ * For some of the IBTA objects there will likely be some
+ * initializations required. We need flags to determine whether it is OK
+ * for rdmavt to do this or not. This does not imply any functions of a
+ * partiuclar IBTA object are overridden.
+ */
+#define RVT_FLAG_MR_INIT_DRIVER BIT(1)
+#define RVT_FLAG_QP_INIT_DRIVER BIT(2)
+#define RVT_FLAG_CQ_INIT_DRIVER BIT(3)
+
+/*
  * For Memory Regions. This stuff should probably be moved into rdmavt/mr.h 
once
  * drivers no longer need access to the MR directly.
  */
@@ -429,6 +439,8 @@ struct rvt_dev_info {
/* Internal use */
int n_pds_allocated;
spinlock_t n_pds_lock; /* Protect pd allocated count */
+
+   int flags;
 };
 
 static inline struct rvt_pd *ibpd_to_rvtpd(struct ib_pd *ibpd)

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 21/36] IB/rdmavt: Move MR datastructures into rvt

[Dennis Dalessandro]

This patch adds the MR datastructures based on hfi1 into rvt. For now the
data structures are defined in include/rdma/rdma_vt.h but once all MR
functionality has been moved from the drivers into rvt these should move to
rdmavt/mr.h

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Dean Luick <dean.lu...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Rename rdi.lk_table => rdi.lkey_table

 include/rdma/rdma_vt.h |   53 
 1 files changed, 53 insertions(+), 0 deletions(-)

diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index bf072a4..f232e39 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -56,6 +56,56 @@
 #include "ib_verbs.h"
 
 /*
+ * For Memory Regions. This stuff should probably be moved into rdmavt/mr.h 
once
+ * drivers no longer need access to the MR directly.
+ */
+
+/*
+ * A segment is a linear region of low physical memory.
+ * Used by the verbs layer.
+ */
+struct rvt_seg {
+   void *vaddr;
+   size_t length;
+};
+
+/* The number of rvt_segs that fit in a page. */
+#define RVT_SEGSZ (PAGE_SIZE / sizeof(struct rvt_seg))
+
+struct rvt_segarray {
+   struct rvt_seg segs[RVT_SEGSZ];
+};
+
+struct rvt_mregion {
+   struct ib_pd *pd;   /* shares refcnt of ibmr.pd */
+   u64 user_base;  /* User's address for this region */
+   u64 iova;   /* IB start address of this region */
+   size_t length;
+   u32 lkey;
+   u32 offset; /* offset (bytes) to start of region */
+   int access_flags;
+   u32 max_segs;   /* number of rvt_segs in all the arrays */
+   u32 mapsz;  /* size of the map array */
+   u8  page_shift; /* 0 - non unform/non powerof2 sizes */
+   u8  lkey_published; /* in global table */
+   struct completion comp; /* complete when refcount goes to zero */
+   atomic_t refcount;
+   struct rvt_segarray *map[0];/* the segments */
+};
+
+#define RVT_MAX_LKEY_TABLE_BITS 23
+
+struct rvt_lkey_table {
+   spinlock_t lock; /* protect changes in this struct */
+   u32 next;   /* next unused index (speeds search) */
+   u32 gen;/* generation count */
+   u32 max;/* size of the table */
+   struct rvt_mregion __rcu **table;
+};
+
+/* End Memmory Region */
+
+/*
  * Things that are driver specific, module parameters in hfi1 and qib
  */
 struct rvt_driver_params {
@@ -119,6 +169,9 @@ struct rvt_dev_info {
/* Driver specific properties */
struct rvt_driver_params dparms;
 
+   struct rvt_mregion __rcu *dma_mr;
+   struct rvt_lkey_table lkey_table;
+
/* PKey Table goes here */
 
/*

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 00/10] Begin to use rdmavt for qib

[Dennis Dalessandro]

The following series implements the beginning support of rdmavt in the qib
driver. The memory registration and protection domain functionality has been
moved out of the driver as well as critical qp data structures. More patches
will follow to incrementally remove code from qib and use the rdmavt
implementation.

This patchset builds on the v3 "Add rdma verbs transport library" patchset.

Changes since v1:
Rebased onto what appears to be the latest branch on Doug's kernel.org repo
 (k.o/for-4.4-rc). I can rebase onto something else if needed.

Changes since v2:
Rebased onto k.o/for-4.5 commit: 59caae
Handle removal of rvt_reg_phys_mr()

---

Dennis Dalessandro (10):
  IB/qib: Begin to use rdmavt for verbs
  IB/qib: Remove dma.c and use rdmavt version of dma functions
  IB/qib: Use rdmavt protection domain
  IB/qib: Remove most uses of QIB_PERMISSIVE_LID and QIB_MULTICAST_LID_BASE
  IB/qib: Use rdmavt lid defines in qib
  IB/qib: Remove driver specific members from qib qp type
  IB/qib: Add device specific info prints
  IB/qib: Remove qp and mr functionality from qib
  IB/qib: Use address handle in rdmavt and remove from qib
  IB/qib: Remove srq from qib


 drivers/infiniband/hw/qib/Kconfig   |2 
 drivers/infiniband/hw/qib/Makefile  |6 
 drivers/infiniband/hw/qib/qib.h |   17 +
 drivers/infiniband/hw/qib/qib_common.h  |2 
 drivers/infiniband/hw/qib/qib_cq.c  |2 
 drivers/infiniband/hw/qib/qib_driver.c  |   22 +
 drivers/infiniband/hw/qib/qib_iba7322.c |2 
 drivers/infiniband/hw/qib/qib_init.c|5 
 drivers/infiniband/hw/qib/qib_intr.c|2 
 drivers/infiniband/hw/qib/qib_keys.c|  186 +-
 drivers/infiniband/hw/qib/qib_mad.c |   11 -
 drivers/infiniband/hw/qib/qib_mmap.c|   18 -
 drivers/infiniband/hw/qib/qib_mr.c  |  490 ---
 drivers/infiniband/hw/qib/qib_qp.c  |  161 +
 drivers/infiniband/hw/qib/qib_rc.c  |  105 +++---
 drivers/infiniband/hw/qib/qib_ruc.c |   84 ++---
 drivers/infiniband/hw/qib/qib_sdma.c|   25 +
 drivers/infiniband/hw/qib/qib_srq.c |   38 +-
 drivers/infiniband/hw/qib/qib_sysfs.c   |   20 +
 drivers/infiniband/hw/qib/qib_uc.c  |   15 -
 drivers/infiniband/hw/qib/qib_ud.c  |   53 ++-
 drivers/infiniband/hw/qib/qib_verbs.c   |  449 -
 drivers/infiniband/hw/qib/qib_verbs.h   |  484 ---
 drivers/infiniband/hw/qib/qib_verbs_mcast.c |8 
 24 files changed, 545 insertions(+), 1662 deletions(-)
 delete mode 100644 drivers/infiniband/hw/qib/qib_mr.c

-- 
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 04/10] IB/qib: Remove most uses of QIB_PERMISSIVE_LID and QIB_MULTICAST_LID_BASE

[Dennis Dalessandro]

This patch removes most of the uses of QIB_PERMISSIBVE_LID and
QIB_MULTICAST_LID_BASE in favor of the recently added IB_* versions.
There are still minor uses in AH functions as well as the QIB_* defines
but those will be removed in a follow on patch.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_driver.c |2 +-
 drivers/infiniband/hw/qib/qib_mad.c|4 ++--
 drivers/infiniband/hw/qib/qib_qp.c |5 +++--
 drivers/infiniband/hw/qib/qib_ud.c |   11 ++-
 drivers/infiniband/hw/qib/qib_verbs.c  |2 +-
 5 files changed, 13 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_driver.c 
b/drivers/infiniband/hw/qib/qib_driver.c
index f58fdc3..57b19ea 100644
--- a/drivers/infiniband/hw/qib/qib_driver.c
+++ b/drivers/infiniband/hw/qib/qib_driver.c
@@ -319,7 +319,7 @@ static u32 qib_rcv_hdrerr(struct qib_ctxtdata *rcd, struct 
qib_pportdata *ppd,
if (tlen < 24)
goto drop;
 
-   if (lid < QIB_MULTICAST_LID_BASE) {
+   if (lid < be16_to_cpu(IB_MULTICAST_LID_BASE)) {
lid &= ~((1 << ppd->lmc) - 1);
if (unlikely(lid != ppd->lid))
goto drop;
diff --git a/drivers/infiniband/hw/qib/qib_mad.c 
b/drivers/infiniband/hw/qib/qib_mad.c
index c65d3aa..70fc1b2 100644
--- a/drivers/infiniband/hw/qib/qib_mad.c
+++ b/drivers/infiniband/hw/qib/qib_mad.c
@@ -691,7 +691,7 @@ static int subn_set_portinfo(struct ib_smp *smp, struct 
ib_device *ibdev,
 
lid = be16_to_cpu(pip->lid);
/* Must be a valid unicast LID address. */
-   if (lid == 0 || lid >= QIB_MULTICAST_LID_BASE)
+   if (lid == 0 || lid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
smp->status |= IB_SMP_INVALID_FIELD;
else if (ppd->lid != lid || ppd->lmc != (pip->mkeyprot_resv_lmc & 7)) {
if (ppd->lid != lid)
@@ -706,7 +706,7 @@ static int subn_set_portinfo(struct ib_smp *smp, struct 
ib_device *ibdev,
smlid = be16_to_cpu(pip->sm_lid);
msl = pip->neighbormtu_mastersmsl & 0xF;
/* Must be a valid unicast LID address. */
-   if (smlid == 0 || smlid >= QIB_MULTICAST_LID_BASE)
+   if (smlid == 0 || smlid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
smp->status |= IB_SMP_INVALID_FIELD;
else if (smlid != ibp->sm_lid || msl != ibp->sm_sl) {
spin_lock_irqsave(>lock, flags);
diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index 40f85bb..e6544dc 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -589,14 +589,15 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
goto inval;
 
if (attr_mask & IB_QP_AV) {
-   if (attr->ah_attr.dlid >= QIB_MULTICAST_LID_BASE)
+   if (attr->ah_attr.dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
goto inval;
if (qib_check_ah(qp->ibqp.device, >ah_attr))
goto inval;
}
 
if (attr_mask & IB_QP_ALT_PATH) {
-   if (attr->alt_ah_attr.dlid >= QIB_MULTICAST_LID_BASE)
+   if (attr->alt_ah_attr.dlid >=
+   be16_to_cpu(IB_MULTICAST_LID_BASE))
goto inval;
if (qib_check_ah(qp->ibqp.device, >alt_ah_attr))
goto inval;
diff --git a/drivers/infiniband/hw/qib/qib_ud.c 
b/drivers/infiniband/hw/qib/qib_ud.c
index 59193f6..32fc80c 100644
--- a/drivers/infiniband/hw/qib/qib_ud.c
+++ b/drivers/infiniband/hw/qib/qib_ud.c
@@ -32,6 +32,7 @@
  */
 
 #include 
+#include 
 
 #include "qib.h"
 #include "qib_mad.h"
@@ -278,8 +279,8 @@ int qib_make_ud_req(struct qib_qp *qp)
ibp = to_iport(qp->ibqp.device, qp->port_num);
ppd = ppd_from_ibp(ibp);
ah_attr = _iah(wqe->ud_wr.ah)->attr;
-   if (ah_attr->dlid >= QIB_MULTICAST_LID_BASE) {
-   if (ah_attr->dlid != QIB_PERMISSIVE_LID)
+   if (ah_attr->dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE)) {
+   if (ah_attr->dlid != be16_to_cpu(IB_LID_PERMISSIVE))
this_cpu_inc(ibp->pmastats->n_multicast_xmit);
else
this_cpu_inc(ibp->pmastats->n_unicast_xmit);
@@ -368,8 +369,8 @@ int qib_make_ud_req(struct qib_qp *qp)
/*
 * Use the multicast QP if the destination LID is a multicast LID.
 */
-   ohdr->bth[1] = ah_attr->dlid >= QIB_MULTICAST_LID_BASE &&
-   ah_attr->dlid != QIB_PERMISSIVE_LID ?

[PATCH v3 03/10] IB/qib: Use rdmavt protection domain

[Dennis Dalessandro]

Remove protection domain datastructure from qib and use rdmavts version.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_keys.c  |6 ++-
 drivers/infiniband/hw/qib/qib_mr.c|2 +
 drivers/infiniband/hw/qib/qib_ruc.c   |4 +-
 drivers/infiniband/hw/qib/qib_verbs.c |   66 +++--
 drivers/infiniband/hw/qib/qib_verbs.h |   13 +--
 5 files changed, 12 insertions(+), 79 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_keys.c 
b/drivers/infiniband/hw/qib/qib_keys.c
index d725c56..04fa272 100644
--- a/drivers/infiniband/hw/qib/qib_keys.c
+++ b/drivers/infiniband/hw/qib/qib_keys.c
@@ -152,7 +152,7 @@ out:
  * Check the IB SGE for validity and initialize our internal version
  * of it.
  */
-int qib_lkey_ok(struct qib_lkey_table *rkt, struct qib_pd *pd,
+int qib_lkey_ok(struct qib_lkey_table *rkt, struct rvt_pd *pd,
struct qib_sge *isge, struct ib_sge *sge, int acc)
 {
struct qib_mregion *mr;
@@ -263,7 +263,7 @@ int qib_rkey_ok(struct qib_qp *qp, struct qib_sge *sge,
 */
rcu_read_lock();
if (rkey == 0) {
-   struct qib_pd *pd = to_ipd(qp->ibqp.pd);
+   struct rvt_pd *pd = ibpd_to_rvtpd(qp->ibqp.pd);
struct qib_ibdev *dev = to_idev(pd->ibpd.device);
 
if (pd->user)
@@ -341,7 +341,7 @@ bail:
 int qib_reg_mr(struct qib_qp *qp, struct ib_reg_wr *wr)
 {
struct qib_lkey_table *rkt = _idev(qp->ibqp.device)->lk_table;
-   struct qib_pd *pd = to_ipd(qp->ibqp.pd);
+   struct rvt_pd *pd = ibpd_to_rvtpd(qp->ibqp.pd);
struct qib_mr *mr = to_imr(wr->mr);
struct qib_mregion *mrg;
u32 key = wr->key;
diff --git a/drivers/infiniband/hw/qib/qib_mr.c 
b/drivers/infiniband/hw/qib/qib_mr.c
index 5f53304..9d84e0d 100644
--- a/drivers/infiniband/hw/qib/qib_mr.c
+++ b/drivers/infiniband/hw/qib/qib_mr.c
@@ -99,7 +99,7 @@ struct ib_mr *qib_get_dma_mr(struct ib_pd *pd, int acc)
struct ib_mr *ret;
int rval;
 
-   if (to_ipd(pd)->user) {
+   if (ibpd_to_rvtpd(pd)->user) {
ret = ERR_PTR(-EPERM);
goto bail;
}
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c 
b/drivers/infiniband/hw/qib/qib_ruc.c
index b1aa21b..425c8c2 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -84,11 +84,11 @@ static int qib_init_sge(struct qib_qp *qp, struct qib_rwqe 
*wqe)
int i, j, ret;
struct ib_wc wc;
struct qib_lkey_table *rkt;
-   struct qib_pd *pd;
+   struct rvt_pd *pd;
struct qib_sge_state *ss;
 
rkt = _idev(qp->ibqp.device)->lk_table;
-   pd = to_ipd(qp->ibqp.srq ? qp->ibqp.srq->pd : qp->ibqp.pd);
+   pd = ibpd_to_rvtpd(qp->ibqp.srq ? qp->ibqp.srq->pd : qp->ibqp.pd);
ss = >r_sge;
ss->sg_list = qp->r_sg_list;
qp->r_len = 0;
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index fb6ef2c..1aaafea 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -346,7 +346,7 @@ static int qib_post_one_send(struct qib_qp *qp, struct 
ib_send_wr *wr,
int ret;
unsigned long flags;
struct qib_lkey_table *rkt;
-   struct qib_pd *pd;
+   struct rvt_pd *pd;
 
spin_lock_irqsave(>s_lock, flags);
 
@@ -396,7 +396,7 @@ static int qib_post_one_send(struct qib_qp *qp, struct 
ib_send_wr *wr,
}
 
rkt = _idev(qp->ibqp.device)->lk_table;
-   pd = to_ipd(qp->ibqp.pd);
+   pd = ibpd_to_rvtpd(qp->ibqp.pd);
wqe = get_swqe_ptr(qp, qp->s_head);
 
if (qp->ibqp.qp_type != IB_QPT_UC &&
@@ -1599,7 +1599,7 @@ static int qib_query_device(struct ib_device *ibdev, 
struct ib_device_attr *prop
props->max_mr = dev->lk_table.max;
props->max_fmr = dev->lk_table.max;
props->max_map_per_fmr = 32767;
-   props->max_pd = ib_qib_max_pds;
+   props->max_pd = dev->rdi.dparms.props.max_pd;
props->max_qp_rd_atom = QIB_MAX_RDMA_ATOMIC;
props->max_qp_init_rd_atom = 255;
/* props->max_res_rd_atom */
@@ -1751,61 +1751,6 @@ static int qib_query_gid(struct ib_device *ibdev, u8 
port,
return ret;
 }
 
-static struct ib_pd *qib_alloc_pd(struct ib_device *ibdev,
- struct ib_ucontext *context,
- struct ib_udata *udata)
-{
-   struct qib_ibdev *dev = to_idev(ibdev);
-   struct qib_pd *pd;
-   struct ib_pd *ret;
-
-   /*
-* This is actually totally arbitrary.  Some correctness tests
-* assume there's a maximum num

[PATCH v3 10/10] IB/qib: Remove srq from qib

[Dennis Dalessandro]

Remove srq from qib now that it has been moved into rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Harish Chegondi <harish.chego...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_qp.c|2 +-
 drivers/infiniband/hw/qib/qib_ruc.c   |4 ++--
 drivers/infiniband/hw/qib/qib_srq.c   |   10 +-
 drivers/infiniband/hw/qib/qib_verbs.h |   13 -
 4 files changed, 8 insertions(+), 21 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index f3964ef..79907d1 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -1036,7 +1036,7 @@ struct ib_qp *qib_create_qp(struct ib_pd *ibpd,
sz = sizeof(*qp);
sg_list_sz = 0;
if (init_attr->srq) {
-   struct qib_srq *srq = to_isrq(init_attr->srq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(init_attr->srq);
 
if (srq->rq.max_sge > 1)
sg_list_sz = sizeof(*qp->r_sg_list) *
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c 
b/drivers/infiniband/hw/qib/qib_ruc.c
index 3900459..682447e 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -141,14 +141,14 @@ int qib_get_rwqe(struct rvt_qp *qp, int wr_id_only)
unsigned long flags;
struct rvt_rq *rq;
struct rvt_rwq *wq;
-   struct qib_srq *srq;
+   struct rvt_srq *srq;
struct rvt_rwqe *wqe;
void (*handler)(struct ib_event *, void *);
u32 tail;
int ret;
 
if (qp->ibqp.srq) {
-   srq = to_isrq(qp->ibqp.srq);
+   srq = ibsrq_to_rvtsrq(qp->ibqp.srq);
handler = srq->ibsrq.event_handler;
rq = >rq;
} else {
diff --git a/drivers/infiniband/hw/qib/qib_srq.c 
b/drivers/infiniband/hw/qib/qib_srq.c
index 8547263..e9dfa30 100644
--- a/drivers/infiniband/hw/qib/qib_srq.c
+++ b/drivers/infiniband/hw/qib/qib_srq.c
@@ -48,7 +48,7 @@
 int qib_post_srq_receive(struct ib_srq *ibsrq, struct ib_recv_wr *wr,
 struct ib_recv_wr **bad_wr)
 {
-   struct qib_srq *srq = to_isrq(ibsrq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(ibsrq);
struct rvt_rwq *wq;
unsigned long flags;
int ret;
@@ -103,7 +103,7 @@ struct ib_srq *qib_create_srq(struct ib_pd *ibpd,
  struct ib_udata *udata)
 {
struct qib_ibdev *dev = to_idev(ibpd->device);
-   struct qib_srq *srq;
+   struct rvt_srq *srq;
u32 sz;
struct ib_srq *ret;
 
@@ -212,7 +212,7 @@ int qib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr 
*attr,
   enum ib_srq_attr_mask attr_mask,
   struct ib_udata *udata)
 {
-   struct qib_srq *srq = to_isrq(ibsrq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(ibsrq);
struct rvt_rwq *wq;
int ret = 0;
 
@@ -350,7 +350,7 @@ bail:
 
 int qib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr)
 {
-   struct qib_srq *srq = to_isrq(ibsrq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(ibsrq);
 
attr->max_wr = srq->rq.size - 1;
attr->max_sge = srq->rq.max_sge;
@@ -364,7 +364,7 @@ int qib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr 
*attr)
  */
 int qib_destroy_srq(struct ib_srq *ibsrq)
 {
-   struct qib_srq *srq = to_isrq(ibsrq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(ibsrq);
struct qib_ibdev *dev = to_idev(ibsrq->device);
 
spin_lock(>n_srqs_lock);
diff --git a/drivers/infiniband/hw/qib/qib_verbs.h 
b/drivers/infiniband/hw/qib/qib_verbs.h
index a9d7f0a..00dd2ad 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.h
+++ b/drivers/infiniband/hw/qib/qib_verbs.h
@@ -251,14 +251,6 @@ struct qib_cq {
struct rvt_mmap_info *ip;
 };
 
-struct qib_srq {
-   struct ib_srq ibsrq;
-   struct rvt_rq rq;
-   struct rvt_mmap_info *ip;
-   /* send signal when number of RWQEs < limit */
-   u32 limit;
-};
-
 /*
  * qib specific data structure that will be hidden from rvt after the queue 
pair
  * is made common.
@@ -539,11 +531,6 @@ static inline struct qib_cq *to_icq(struct ib_cq *ibcq)
return container_of(ibcq, struct qib_cq, ibcq);
 }
 
-static inline struct qib_srq *to_isrq(struct ib_srq *ibsrq)
-{
-   return container_of(ibsrq, struct qib_srq, ibsrq);
-}
-
 static inline struct rvt_qp *to_iqp(struct ib_qp *ibqp)
 {
return container_of(ibqp, struct rvt_qp, ibqp);

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Generic InfiniBand transport done in software

[Dennis Dalessandro]

On Tue, Dec 29, 2015 at 07:38:30PM +0200, Moni Shoua wrote:

This is not a question if I can hook Soft RoCE driver into this framework.
In fact, I can't think of an IB driver that can't use this framework. What 
this

framework offers is just another hop from ib_core the real driver.
Where is the removal of duplicated code? This is a list of functions
that for now
must be implemented in the low level driver.

create_cq
destroy_cq
poll_cq
req_notify_cq
resize_cq
create_srq
modify_srq
destroy_srq
query_srq
create_qp
query_device
query_gid
alloc_ucontext
modify_device
modify_qp
dealloc_ucontext
query_port
destroy_qp
get_port_immutable
modify_port
query_qp
post_send
post_recv
post_srq_recv

Most if not all of them have common part in all drivers.
What are the plans to get rid of them? When?
Don't you think that this should be known in advance?


We have patch sets that implement all of these which will be posted soon.  
With the holidays things have just been a bit slow to come out.


-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 00/36] Add rdma verbs transport library

[Dennis Dalessandro]

The following series implements rdmavt. This is the rdma verbs transport
software library which will help to solve the problem of code duplication
between hardware drivers when it comes to a verbs implementation.

Rdmavt is basically just another verbs provider and lives in the Infiniband tree
in a new sw directory. It provides a software implementation of the Infiniband
verbs API. More details can be found in the following threads:

http://www.spinics.net/lists/linux-rdma/msg29064.html
http://www.spinics.net/lists/linux-rdma/msg29922.html

This patch series is based on what appears to be the latest branch on Doug's
kernel.org repo (k.o/for-4.4-rc). I can rebase onto something else if needed.

Changes since v1:
Removed driver specific version
Fixed license text to remove copyright and put on top
Return 0 in rvt_map_sg instead of BAD_DMA_AGGRESS
Remove #include of dma.h from dma.c
Update comment about protection domain limit
Remove comment on alternative design for private data
Rename CDR macro to CHECK_DRIVER_OVERRIDE
Change all the stubs to return EOPNOTSUPP
Fix comment style for rvt_query_port
Fix typo in subject
Rename rdi.lk_table to rdi.lkey_table
Rename rvt_sge.m => rvt_sge.cur_map (Sean)
Rename rvt_sge.n => rvt_sge.cur_seg (Sean)
Remove rvt_reg_phys_mr
Drop support for commit 38071a461f0a ("IB/qib: Support the new memory
   registration API")

---

Dennis Dalessandro (34):
  IB/rdmavt: Create module framework and handle driver registration
  IB/rdmavt: Consolidate dma ops in rdmavt.
  IB/rdmavt: Add protection domain to rdmavt.
  IB/rdmavt: Add ib core device attributes to rvt driver params list
  IB/rdmavt: Macroize override checks during driver registration
  IB/rdmavt: Add query and modify device stubs
  IB/rdmavt: Add query and modify port stubs
  IB/rdmavt: Add pkey query stub
  IB/rdmavt: Add query gid stub
  IB/rdmavt: Alloc and dealloc ucontexts
  IB/rdmavt: Add queue pair function stubs
  IB/rdmavt: Add address handle stubs
  IB/rdmavt: Add memory region stubs
  IB/rdmavt: Add SRQ stubs
  IB/rdmavt: Add multicast stubs
  IB/rdmavt: Add process MAD stub
  IB/rdmavt: Add mmap stub
  IB/rdmavt: Add get port immutable stub
  IB/rdmavt: Add completion queue function stubs
  IB/rdmavt: Add post send and recv stubs
  IB/rdmavt: Move MR datastructures into rvt
  IB/rdmavt: Add queue pair data structure to rdmavt
  IB/rdmavt: Move driver helper functions to a common structure
  IB/rdmavt: Add device specific info prints
  IB/rdmavt: Add the start of capability flags
  IB/rdmavt: Move memory registration into rdmavt
  IB/rdmavt: Do not use rvt prints which rely on driver too early
  IB/rdmavt: Move SRQ data structure into rdmavt
  IB/rdmavt: Add an ibport data structure to rdmavt
  IB/rdmavt: Add driver notification for new AH
  IB/rdmavt: Break rdma_vt main include header file up
  IB/rdmavt: Initialize and teardown of qpn table
  IB/rdmavt: Add mmap related functions
  IB/rdmavt: Add pkey support

Kamal Heib (2):
  IB/rdmavt: Add common LID defines to rdmavt
  IB/rdmavt: Add AH to rdmavt


 MAINTAINERS   |6 
 drivers/infiniband/Kconfig|2 
 drivers/infiniband/Makefile   |1 
 drivers/infiniband/sw/Makefile|1 
 drivers/infiniband/sw/rdmavt/Kconfig  |6 
 drivers/infiniband/sw/rdmavt/Makefile |   10 
 drivers/infiniband/sw/rdmavt/ah.c |  169 +++
 drivers/infiniband/sw/rdmavt/ah.h |   59 ++
 drivers/infiniband/sw/rdmavt/cq.c |  113 +
 drivers/infiniband/sw/rdmavt/cq.h |   62 ++
 drivers/infiniband/sw/rdmavt/dma.c|  182 +++
 drivers/infiniband/sw/rdmavt/dma.h|   53 ++
 drivers/infiniband/sw/rdmavt/mad.c|   85 +++
 drivers/infiniband/sw/rdmavt/mad.h|   59 ++
 drivers/infiniband/sw/rdmavt/mcast.c  |   58 ++
 drivers/infiniband/sw/rdmavt/mcast.h  |   56 ++
 drivers/infiniband/sw/rdmavt/mmap.c   |  198 
 drivers/infiniband/sw/rdmavt/mmap.h   |   55 ++
 drivers/infiniband/sw/rdmavt/mr.c |  824 +
 drivers/infiniband/sw/rdmavt/mr.h |   95 
 drivers/infiniband/sw/rdmavt/pd.c |  103 
 drivers/infiniband/sw/rdmavt/pd.h |   58 ++
 drivers/infiniband/sw/rdmavt/qp.c |  379 +++
 drivers/infiniband/sw/rdmavt/qp.h |   69 +++
 drivers/infiniband/sw/rdmavt/srq.c|   86 +++
 drivers/infiniband/sw/rdmavt/srq.h|   61 ++
 drivers/infiniband/sw/rdmavt/vt.c |  377 +++
 drivers/infiniband/sw/rdmavt/vt.h |   90 
 include/rdma/ib_verbs.h   |1 
 include/rdma/rdma_vt.h|  333 +
 include/rdma/rdmavt_mr.h  |  130 +
 include/rdma/rdmavt_qp.h  |  295 
 32 files changed, 4076 insertions(+), 0 deletions(-)
 create mode 100644 drive

[PATCH v2 05/36] IB/rdmavt: Macroize override checks during driver registration

[Dennis Dalessandro]

Add a macro to cut down on writing the same lines over and over again for
what will be a large number of functions that will be supported.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change CDR macro to CHECK_DRIVER_OVERRIDE to be more clear

 drivers/infiniband/sw/rdmavt/vt.c |   22 +-
 1 files changed, 9 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index b65cde4..8bd25c3 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -64,29 +64,25 @@ static void rvt_cleanup(void)
 }
 module_exit(rvt_cleanup);
 
+/*
+ * Check driver override. If driver passes a value use it, otherwise we use our
+ * own value.
+ */
+#define CHECK_DRIVER_OVERRIDE(rdi, x) \
+   rdi->ibdev.x = rdi->ibdev.x ? : rvt_ ##x
+
 int rvt_register_device(struct rvt_dev_info *rdi)
 {
if (!rdi)
return -EINVAL;
 
-   /*
-* Drivers have the option to override anything in the ibdev that they
-* want to specifically handle. VT needs to check for things it supports
-* and if the driver wants to handle that functionality let it. We may
-* come up with a better mechanism that simplifies the code at some
-* point.
-*/
-
/* DMA Operations */
rdi->ibdev.dma_ops =
rdi->ibdev.dma_ops ? : _default_dma_mapping_ops;
 
/* Protection Domain */
-   rdi->ibdev.alloc_pd =
-   rdi->ibdev.alloc_pd ? : rvt_alloc_pd;
-   rdi->ibdev.dealloc_pd =
-   rdi->ibdev.dealloc_pd ? : rvt_dealloc_pd;
-
+   CHECK_DRIVER_OVERRIDE(rdi, alloc_pd);
+   CHECK_DRIVER_OVERRIDE(rdi, dealloc_pd);
spin_lock_init(>n_pds_lock);
rdi->n_pds_allocated = 0;
 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 10/36] IB/rdmavt: Alloc and dealloc ucontexts

[Dennis Dalessandro]

Add the stubs to allocate and deallocate user contexts. This will be
handled completely by rvt.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

drivers/infiniband/sw/rdmavt/vt.c |   22 ++
 1 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index e95f197..cd19429 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -178,6 +178,26 @@ static int rvt_query_gid(struct ib_device *ibdev, u8 port,
return -EOPNOTSUPP;
 }
 
+/**
+ * rvt_alloc_ucontext - Allocate a user context
+ * @ibdev: Vers IB dev
+ * @data: User data allocated
+ */
+static struct ib_ucontext *rvt_alloc_ucontext(struct ib_device *ibdev,
+ struct ib_udata *udata)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ *rvt_dealloc_ucontext - Free a user context
+ *@context - Free this
+ */
+static int rvt_dealloc_ucontext(struct ib_ucontext *context)
+{
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -197,6 +217,8 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, modify_port);
CHECK_DRIVER_OVERRIDE(rdi, query_pkey);
CHECK_DRIVER_OVERRIDE(rdi, query_gid);
+   CHECK_DRIVER_OVERRIDE(rdi, alloc_ucontext);
+   CHECK_DRIVER_OVERRIDE(rdi, dealloc_ucontext);
 
/* DMA Operations */
rdi->ibdev.dma_ops =

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 09/36] IB/rdmavt: Add query gid stub

[Dennis Dalessandro]

The handler for query gid operates along the same lines as the query pkey
handler. The driver will take care to keep the guid table updated.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |   22 ++
 1 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index fc5372d..e95f197 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -157,6 +157,27 @@ static int rvt_query_pkey(struct ib_device *ibdev, u8 
port, u16 index,
return 0;
 }
 
+/**
+ * rvt_query_gid - Return a gid from the table
+ * @ibdev: Verbs IB dev
+ * @port: Port number
+ * @index: = Index in table
+ * @gid: Gid to return
+ *
+ * Returns 0 on success
+ */
+static int rvt_query_gid(struct ib_device *ibdev, u8 port,
+int index, union ib_gid *gid)
+{
+   /*
+* Driver is responsible for updating the guid table. Which will be used
+* to craft the return value. This will work similar to how query_pkey()
+* is being done.
+*/
+
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -175,6 +196,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, query_port);
CHECK_DRIVER_OVERRIDE(rdi, modify_port);
CHECK_DRIVER_OVERRIDE(rdi, query_pkey);
+   CHECK_DRIVER_OVERRIDE(rdi, query_gid);
 
/* DMA Operations */
rdi->ibdev.dma_ops =

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 08/36] IB/rdmavt: Add pkey query stub

[Dennis Dalessandro]

The pkey table will reside in the rvt structure but it will be modified
only when the driver requests then rvt will simply read the value to return
in the query.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |   21 +
 include/rdma/rdma_vt.h|3 ++-
 2 files changed, 23 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 5ac241c..fc5372d 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -137,6 +137,26 @@ static int rvt_modify_port(struct ib_device *ibdev, u8 
port,
return -EOPNOTSUPP;
 }
 
+/**
+ * rvt_query_pkey - Return a pkey from the table at a given index
+ * @ibdev: Verbs IB dev
+ * @port: Port number
+ * @intex: Index into pkey table
+ *
+ * Returns 0 on failure pkey otherwise
+ */
+static int rvt_query_pkey(struct ib_device *ibdev, u8 port, u16 index,
+ u16 *pkey)
+{
+   /*
+* Driver will be responsible for keeping rvt_dev_info.pkey_table up to
+* date. This function will just return that value. There is no need to
+* lock, if a stale value is read and sent to the user so be it there is
+* no way to protect against that anyway.
+*/
+   return 0;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -154,6 +174,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, modify_device);
CHECK_DRIVER_OVERRIDE(rdi, query_port);
CHECK_DRIVER_OVERRIDE(rdi, modify_port);
+   CHECK_DRIVER_OVERRIDE(rdi, query_pkey);
 
/* DMA Operations */
rdi->ibdev.dma_ops =
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 2990e03..bf072a4 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -114,12 +114,13 @@ struct rvt_dev_info {
 * The driver will also be responsible for filling in certain members of
 * dparms.props
 */
-
struct ib_device ibdev;
 
/* Driver specific properties */
struct rvt_driver_params dparms;
 
+   /* PKey Table goes here */
+
/*
 * The work to create port files in /sys/class Infiniband is different
 * depending on the driver. This should not be extracted away and

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 13/36] IB/rdmavt: Add memory region stubs

[Dennis Dalessandro]

Adds the function stubs for allocating, and registering memory regions, as
well as deregistering them.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 
 drivers/infiniband/sw/rdmavt/mr.c |  182 +
 drivers/infiniband/sw/rdmavt/mr.h |   72 +
 drivers/infiniband/sw/rdmavt/vt.c |   11 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 267 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/mr.c
 create mode 100644 drivers/infiniband/sw/rdmavt/mr.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 628c684..084ee6a 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o pd.o qp.o
+rdmavt-y := vt.o ah.o dma.o mr.o pd.o qp.o
diff --git a/drivers/infiniband/sw/rdmavt/mr.c 
b/drivers/infiniband/sw/rdmavt/mr.c
new file mode 100644
index 000..c8e5cfe
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mr.c
@@ -0,0 +1,182 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include 
+#include "mr.h"
+
+/**
+ * rvt_get_dma_mr - get a DMA memory region
+ * @pd: protection domain for this memory region
+ * @acc: access flags
+ *
+ * Returns the memory region on success, otherwise returns an errno.
+ * Note that all DMA addresses should be created via the
+ * struct ib_dma_mapping_ops functions (see dma.c).
+ */
+struct ib_mr *rvt_get_dma_mr(struct ib_pd *pd, int acc)
+{
+   /*
+* Alloc mr and init it.
+* Alloc lkey.
+*/
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * rvt_reg_phys_mr - register a physical memory region
+ * @pd: protection domain for this memory region
+ * @buffer_list: pointer to the list of physical buffers to register
+ * @num_phys_buf: the number of physical buffers to register
+ * @iova_start: the starting address passed over IB which maps to this MR
+ *
+ * Returns the memory region on success, otherwise returns an errno.
+ */
+struct ib_mr *rvt_reg_phys_mr(struct ib_pd *pd,
+ struct ib_phys_buf *buffer_list,
+ int num_phys_buf, int acc, u64 *iova_start)
+{
+   /* Alloc mr and build it up */
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * rvt_reg_user_mr - register a userspace memory region
+ * @pd: protection domain for this memory region
+ * @start: starting userspace address
+ * @length: length of region to regis

[PATCH v2 12/36] IB/rdmavt: Add address handle stubs

[Dennis Dalessandro]

Adds the stubs for create, destroy, modify, and query of the
address handle.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

drivers/infiniband/sw/rdmavt/Makefile |3 -
 drivers/infiniband/sw/rdmavt/ah.c |   76 +
 drivers/infiniband/sw/rdmavt/ah.h |   59 ++
 drivers/infiniband/sw/rdmavt/vt.c |6 +++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 143 insertions(+), 2 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/ah.c
 create mode 100644 drivers/infiniband/sw/rdmavt/ah.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 9a9a095..628c684 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,5 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o dma.o pd.o qp.o
-
+rdmavt-y := vt.o ah.o dma.o pd.o qp.o
diff --git a/drivers/infiniband/sw/rdmavt/ah.c 
b/drivers/infiniband/sw/rdmavt/ah.c
new file mode 100644
index 000..d368955
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/ah.c
@@ -0,0 +1,76 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "ah.h"
+
+/**
+ * rvt_create_ah - create an address handle
+ * @pd: the protection domain
+ * @ah_attr: the attributes of the AH
+ *
+ * This may be called from interrupt context.
+ */
+struct ib_ah *rvt_create_ah(struct ib_pd *pd,
+   struct ib_ah_attr *ah_attr)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+int rvt_destroy_ah(struct ib_ah *ibah)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_modify_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
+{
+   return -EOPNOTSUPP;
+}
diff --git a/drivers/infiniband/sw/rdmavt/ah.h 
b/drivers/infiniband/sw/rdmavt/ah.h
new file mode 100644
index 000..8cd7ea7
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/ah.h
@@ -0,0 +1,59 @@
+#ifndef DEF_RVTAH_H
+#define DEF_RVTAH_H
+
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the i

[PATCH v2 20/36] IB/rdmavt: Add post send and recv stubs

[Dennis Dalessandro]

This adds the post sned and recv function stubs.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Fix typo in subject
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/qp.c |   62 +
 drivers/infiniband/sw/rdmavt/qp.h |7 
 drivers/infiniband/sw/rdmavt/vt.c |3 ++
 3 files changed, 71 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.c 
b/drivers/infiniband/sw/rdmavt/qp.c
index a59f28d..23a5f68 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -118,3 +118,65 @@ int rvt_query_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
 {
return -EOPNOTSUPP;
 }
+
+/**
+ * rvt_post_receive - post a receive on a QP
+ * @ibqp: the QP to post the receive on
+ * @wr: the WR to post
+ * @bad_wr: the first bad WR is put here
+ *
+ * This may be called from interrupt context.
+ */
+int rvt_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr)
+{
+   /*
+* When a packet arrives the driver needs to call up to rvt to process
+* the packet. The UD, RC, UC processing will be done in rvt, however
+* the driver should be able to override this if it so choses. Perhaps a
+* set of function pointers set up at registration time.
+*/
+
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_post_send - post a send on a QP
+ * @ibqp: the QP to post the send on
+ * @wr: the list of work requests to post
+ * @bad_wr: the first bad WR is put here
+ *
+ * This may be called from interrupt context.
+ */
+int rvt_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
+ struct ib_send_wr **bad_wr)
+{
+   /*
+* VT-DRIVER-API: do_send()
+* Driver needs to have a do_send() call which is a single entry point
+* to take an already formed packet and throw it out on the wire. Once
+* the packet is sent the driver needs to make an upcall to rvt so the
+* completion queue can be notified and/or any other outstanding
+* work/book keeping can be finished.
+*
+* Note that there should also be a way for rvt to protect itself
+* against hangs in the driver layer. If a send doesn't actually
+* complete in a timely manor rvt needs to return an error event.
+*/
+
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_post_srq_receive - post a receive on a shared receive queue
+ * @ibsrq: the SRQ to post the receive on
+ * @wr: the list of work requests to post
+ * @bad_wr: A pointer to the first WR to cause a problem is put here
+ *
+ * This may be called from interrupt context.
+ */
+int rvt_post_srq_recv(struct ib_srq *ibsrq, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr)
+{
+   return -EOPNOTSUPP;
+}
diff --git a/drivers/infiniband/sw/rdmavt/qp.h 
b/drivers/infiniband/sw/rdmavt/qp.h
index 43acba0..10bc636 100644
--- a/drivers/infiniband/sw/rdmavt/qp.h
+++ b/drivers/infiniband/sw/rdmavt/qp.h
@@ -63,5 +63,10 @@ int rvt_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
 int rvt_destroy_qp(struct ib_qp *ibqp);
 int rvt_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 int attr_mask, struct ib_qp_init_attr *init_attr);
-
+int rvt_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr);
+int rvt_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
+ struct ib_send_wr **bad_wr);
+int rvt_post_srq_recv(struct ib_srq *ibsrq, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr);
 #endif  /* DEF_RVTQP_H */
diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 33986e3..0cf80d1 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -232,6 +232,9 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, modify_qp);
CHECK_DRIVER_OVERRIDE(rdi, destroy_qp);
CHECK_DRIVER_OVERRIDE(rdi, query_qp);
+   CHECK_DRIVER_OVERRIDE(rdi, post_send);
+   CHECK_DRIVER_OVERRIDE(rdi, post_recv);
+   CHECK_DRIVER_OVERRIDE(rdi, post_srq_recv);
 
/* Address Handle */
CHECK_DRIVER_OVERRIDE(rdi, create_ah);

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 21/36] IB/rdmavt: Move MR datastructures into rvt

[Dennis Dalessandro]

This patch adds the MR datastructures based on hfi1 into rvt. For now the
data structures are defined in include/rdma/rdma_vt.h but once all MR
functionality has been moved from the drivers into rvt these should move to
rdmavt/mr.h

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Dean Luick <dean.lu...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Rename rdi.lk_table => rdi.lkey_table

 include/rdma/rdma_vt.h |   53 
 1 files changed, 53 insertions(+), 0 deletions(-)

diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index bf072a4..f232e39 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -56,6 +56,56 @@
 #include "ib_verbs.h"
 
 /*
+ * For Memory Regions. This stuff should probably be moved into rdmavt/mr.h 
once
+ * drivers no longer need access to the MR directly.
+ */
+
+/*
+ * A segment is a linear region of low physical memory.
+ * Used by the verbs layer.
+ */
+struct rvt_seg {
+   void *vaddr;
+   size_t length;
+};
+
+/* The number of rvt_segs that fit in a page. */
+#define RVT_SEGSZ (PAGE_SIZE / sizeof(struct rvt_seg))
+
+struct rvt_segarray {
+   struct rvt_seg segs[RVT_SEGSZ];
+};
+
+struct rvt_mregion {
+   struct ib_pd *pd;   /* shares refcnt of ibmr.pd */
+   u64 user_base;  /* User's address for this region */
+   u64 iova;   /* IB start address of this region */
+   size_t length;
+   u32 lkey;
+   u32 offset; /* offset (bytes) to start of region */
+   int access_flags;
+   u32 max_segs;   /* number of rvt_segs in all the arrays */
+   u32 mapsz;  /* size of the map array */
+   u8  page_shift; /* 0 - non unform/non powerof2 sizes */
+   u8  lkey_published; /* in global table */
+   struct completion comp; /* complete when refcount goes to zero */
+   atomic_t refcount;
+   struct rvt_segarray *map[0];/* the segments */
+};
+
+#define RVT_MAX_LKEY_TABLE_BITS 23
+
+struct rvt_lkey_table {
+   spinlock_t lock; /* protect changes in this struct */
+   u32 next;   /* next unused index (speeds search) */
+   u32 gen;/* generation count */
+   u32 max;/* size of the table */
+   struct rvt_mregion __rcu **table;
+};
+
+/* End Memmory Region */
+
+/*
  * Things that are driver specific, module parameters in hfi1 and qib
  */
 struct rvt_driver_params {
@@ -119,6 +169,9 @@ struct rvt_dev_info {
/* Driver specific properties */
struct rvt_driver_params dparms;
 
+   struct rvt_mregion __rcu *dma_mr;
+   struct rvt_lkey_table lkey_table;
+
/* PKey Table goes here */
 
/*

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 23/36] IB/rdmavt: Move driver helper functions to a common structure

[Dennis Dalessandro]

Drivers are going to need to provide multiple functions for rdmavt to
call in to. We already have one, so go ahead and push this into a
data structure designated for driver supplied functions.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |6 +-
 include/rdma/rdma_vt.h|   22 +++---
 2 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 0cf80d1..e4881ca 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -281,8 +281,12 @@ int rvt_register_device(struct rvt_dev_info *rdi)
spin_lock_init(>n_pds_lock);
rdi->n_pds_allocated = 0;
 
+   /* Validate that drivers have provided the right functions */
+   if (!rdi->driver_f.port_callback)
+   return -EINVAL;
+
/* We are now good to announce we exist */
-   return ib_register_device(>ibdev, rdi->port_callback);
+   return ib_register_device(>ibdev, rdi->driver_f.port_callback);
 }
 EXPORT_SYMBOL(rvt_register_device);
 
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 9baa7f0..e0beedc 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -383,6 +383,19 @@ struct rvt_driver_params {
 */
 };
 
+/*
+ * Functions that drivers are required to support
+ */
+struct rvt_driver_provided {
+   /*
+* The work to create port files in /sys/class Infiniband is different
+* depending on the driver. This should not be extracted away and
+* instead drivers are responsible for setting the correct callback for
+* this.
+*/
+   int (*port_callback)(struct ib_device *, u8, struct kobject *);
+};
+
 /* Protection domain */
 struct rvt_pd {
struct ib_pd ibpd;
@@ -407,13 +420,8 @@ struct rvt_dev_info {
 
/* PKey Table goes here */
 
-   /*
-* The work to create port files in /sys/class Infiniband is different
-* depending on the driver. This should not be extracted away and
-* instead drivers are responsible for setting the correct callback for
-* this.
-*/
-   int (*port_callback)(struct ib_device *, u8, struct kobject *);
+   /* Driver specific helper functions */
+   struct rvt_driver_provided driver_f;
 
/* Internal use */
int n_pds_allocated;

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 10/10] IB/qib: Remove srq from qib

[Dennis Dalessandro]

Remove srq from qib now that it has been moved into rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Harish Chegondi <harish.chego...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_qp.c|2 +-
 drivers/infiniband/hw/qib/qib_ruc.c   |4 ++--
 drivers/infiniband/hw/qib/qib_srq.c   |   10 +-
 drivers/infiniband/hw/qib/qib_verbs.h |   13 -
 4 files changed, 8 insertions(+), 21 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index f3964ef..79907d1 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -1036,7 +1036,7 @@ struct ib_qp *qib_create_qp(struct ib_pd *ibpd,
sz = sizeof(*qp);
sg_list_sz = 0;
if (init_attr->srq) {
-   struct qib_srq *srq = to_isrq(init_attr->srq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(init_attr->srq);
 
if (srq->rq.max_sge > 1)
sg_list_sz = sizeof(*qp->r_sg_list) *
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c 
b/drivers/infiniband/hw/qib/qib_ruc.c
index 3900459..682447e 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -141,14 +141,14 @@ int qib_get_rwqe(struct rvt_qp *qp, int wr_id_only)
unsigned long flags;
struct rvt_rq *rq;
struct rvt_rwq *wq;
-   struct qib_srq *srq;
+   struct rvt_srq *srq;
struct rvt_rwqe *wqe;
void (*handler)(struct ib_event *, void *);
u32 tail;
int ret;
 
if (qp->ibqp.srq) {
-   srq = to_isrq(qp->ibqp.srq);
+   srq = ibsrq_to_rvtsrq(qp->ibqp.srq);
handler = srq->ibsrq.event_handler;
rq = >rq;
} else {
diff --git a/drivers/infiniband/hw/qib/qib_srq.c 
b/drivers/infiniband/hw/qib/qib_srq.c
index 8547263..e9dfa30 100644
--- a/drivers/infiniband/hw/qib/qib_srq.c
+++ b/drivers/infiniband/hw/qib/qib_srq.c
@@ -48,7 +48,7 @@
 int qib_post_srq_receive(struct ib_srq *ibsrq, struct ib_recv_wr *wr,
 struct ib_recv_wr **bad_wr)
 {
-   struct qib_srq *srq = to_isrq(ibsrq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(ibsrq);
struct rvt_rwq *wq;
unsigned long flags;
int ret;
@@ -103,7 +103,7 @@ struct ib_srq *qib_create_srq(struct ib_pd *ibpd,
  struct ib_udata *udata)
 {
struct qib_ibdev *dev = to_idev(ibpd->device);
-   struct qib_srq *srq;
+   struct rvt_srq *srq;
u32 sz;
struct ib_srq *ret;
 
@@ -212,7 +212,7 @@ int qib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr 
*attr,
   enum ib_srq_attr_mask attr_mask,
   struct ib_udata *udata)
 {
-   struct qib_srq *srq = to_isrq(ibsrq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(ibsrq);
struct rvt_rwq *wq;
int ret = 0;
 
@@ -350,7 +350,7 @@ bail:
 
 int qib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr)
 {
-   struct qib_srq *srq = to_isrq(ibsrq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(ibsrq);
 
attr->max_wr = srq->rq.size - 1;
attr->max_sge = srq->rq.max_sge;
@@ -364,7 +364,7 @@ int qib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr 
*attr)
  */
 int qib_destroy_srq(struct ib_srq *ibsrq)
 {
-   struct qib_srq *srq = to_isrq(ibsrq);
+   struct rvt_srq *srq = ibsrq_to_rvtsrq(ibsrq);
struct qib_ibdev *dev = to_idev(ibsrq->device);
 
spin_lock(>n_srqs_lock);
diff --git a/drivers/infiniband/hw/qib/qib_verbs.h 
b/drivers/infiniband/hw/qib/qib_verbs.h
index a9d7f0a..00dd2ad 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.h
+++ b/drivers/infiniband/hw/qib/qib_verbs.h
@@ -251,14 +251,6 @@ struct qib_cq {
struct rvt_mmap_info *ip;
 };
 
-struct qib_srq {
-   struct ib_srq ibsrq;
-   struct rvt_rq rq;
-   struct rvt_mmap_info *ip;
-   /* send signal when number of RWQEs < limit */
-   u32 limit;
-};
-
 /*
  * qib specific data structure that will be hidden from rvt after the queue 
pair
  * is made common.
@@ -539,11 +531,6 @@ static inline struct qib_cq *to_icq(struct ib_cq *ibcq)
return container_of(ibcq, struct qib_cq, ibcq);
 }
 
-static inline struct qib_srq *to_isrq(struct ib_srq *ibsrq)
-{
-   return container_of(ibsrq, struct qib_srq, ibsrq);
-}
-
 static inline struct rvt_qp *to_iqp(struct ib_qp *ibqp)
 {
return container_of(ibqp, struct rvt_qp, ibqp);

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 09/10] IB/qib: Use address handle in rdmavt and remove from qib

[Dennis Dalessandro]

Original patch from Kamal Heib <kam...@mellanox.com>, split
apart from original.

Remove AH from qib and use rdmavt version.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Kamal Heib <kam...@mellanox.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_iba7322.c |2 
 drivers/infiniband/hw/qib/qib_mad.c |2 
 drivers/infiniband/hw/qib/qib_qp.c  |6 +
 drivers/infiniband/hw/qib/qib_ruc.c |2 
 drivers/infiniband/hw/qib/qib_ud.c  |4 -
 drivers/infiniband/hw/qib/qib_verbs.c   |  126 ++-
 drivers/infiniband/hw/qib/qib_verbs.h   |   20 -
 7 files changed, 19 insertions(+), 143 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_iba7322.c 
b/drivers/infiniband/hw/qib/qib_iba7322.c
index 6c8ff10..1fbe308 100644
--- a/drivers/infiniband/hw/qib/qib_iba7322.c
+++ b/drivers/infiniband/hw/qib/qib_iba7322.c
@@ -5515,7 +5515,7 @@ static void try_7322_ipg(struct qib_pportdata *ppd)
ret = PTR_ERR(ah);
else {
send_buf->ah = ah;
-   ibp->smi_ah = to_iah(ah);
+   ibp->smi_ah = ibah_to_rvtah(ah);
ret = 0;
}
} else {
diff --git a/drivers/infiniband/hw/qib/qib_mad.c 
b/drivers/infiniband/hw/qib/qib_mad.c
index 70fc1b2..43f8c49 100644
--- a/drivers/infiniband/hw/qib/qib_mad.c
+++ b/drivers/infiniband/hw/qib/qib_mad.c
@@ -109,7 +109,7 @@ static void qib_send_trap(struct qib_ibport *ibp, void 
*data, unsigned len)
ret = PTR_ERR(ah);
else {
send_buf->ah = ah;
-   ibp->sm_ah = to_iah(ah);
+   ibp->sm_ah = ibah_to_rvtah(ah);
ret = 0;
}
} else
diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index 143924d..f3964ef 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -438,7 +438,7 @@ static void clear_mr_refs(struct rvt_qp *qp, int clr_sends)
if (qp->ibqp.qp_type == IB_QPT_UD ||
qp->ibqp.qp_type == IB_QPT_SMI ||
qp->ibqp.qp_type == IB_QPT_GSI)
-   atomic_dec(_iah(wqe->ud_wr.ah)->refcount);
+   
atomic_dec(_to_rvtah(wqe->ud_wr.ah)->refcount);
if (++qp->s_last >= qp->s_size)
qp->s_last = 0;
}
@@ -595,7 +595,7 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
if (attr_mask & IB_QP_AV) {
if (attr->ah_attr.dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
goto inval;
-   if (qib_check_ah(qp->ibqp.device, >ah_attr))
+   if (rvt_check_ah(qp->ibqp.device, >ah_attr))
goto inval;
}
 
@@ -603,7 +603,7 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
if (attr->alt_ah_attr.dlid >=
be16_to_cpu(IB_MULTICAST_LID_BASE))
goto inval;
-   if (qib_check_ah(qp->ibqp.device, >alt_ah_attr))
+   if (rvt_check_ah(qp->ibqp.device, >alt_ah_attr))
goto inval;
if (attr->alt_pkey_index >= qib_get_npkeys(dd_from_dev(dev)))
goto inval;
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c 
b/drivers/infiniband/hw/qib/qib_ruc.c
index 02e79a8..3900459 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -789,7 +789,7 @@ void qib_send_complete(struct rvt_qp *qp, struct rvt_swqe 
*wqe,
if (qp->ibqp.qp_type == IB_QPT_UD ||
qp->ibqp.qp_type == IB_QPT_SMI ||
qp->ibqp.qp_type == IB_QPT_GSI)
-   atomic_dec(_iah(wqe->ud_wr.ah)->refcount);
+   atomic_dec(_to_rvtah(wqe->ud_wr.ah)->refcount);
 
/* See ch. 11.2.4.1 and 10.7.3.1 */
if (!(qp->s_flags & QIB_S_SIGNAL_REQ_WR) ||
diff --git a/drivers/infiniband/hw/qib/qib_ud.c 
b/drivers/infiniband/hw/qib/qib_ud.c
index 1d9d037..6dc20ca 100644
--- a/drivers/infiniband/hw/qib/qib_ud.c
+++ b/drivers/infiniband/hw/qib/qib_ud.c
@@ -77,7 +77,7 @@ static void qib_ud_loopback(struct rvt_qp *sqp, struct 
rvt_swqe *swqe)
goto drop;
}
 
-   ah_attr = _iah(swqe->ud_wr.ah)->attr;
+   ah_attr = _to_rvtah(swqe->ud_wr.ah)->attr;
ppd = ppd_from_ibp(ibp);
 
if (qp->ibqp.qp_num > 1) {
@@ -279,7 +279,7 @@ int qib_make_ud_req(struct rvt_qp *qp)
/* Construct the header. */
   

[PATCH v2 08/10] IB/qib: Remove qp and mr functionality from qib

[Dennis Dalessandro]

Remove qp and mr support from qib and use rdmavt. These two changes
cannot be reasonably be split apart into separate patches because they
depend on eachother in multple places. This paves the way to remove
even more functions in subsequent patches.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/Makefile  |4 
 drivers/infiniband/hw/qib/qib.h |   14 -
 drivers/infiniband/hw/qib/qib_cq.c  |2 
 drivers/infiniband/hw/qib/qib_driver.c  |4 
 drivers/infiniband/hw/qib/qib_keys.c|  184 +
 drivers/infiniband/hw/qib/qib_mmap.c|   18 -
 drivers/infiniband/hw/qib/qib_mr.c  |  539 ---
 drivers/infiniband/hw/qib/qib_qp.c  |   72 ++--
 drivers/infiniband/hw/qib/qib_rc.c  |   98 ++---
 drivers/infiniband/hw/qib/qib_ruc.c |   58 +--
 drivers/infiniband/hw/qib/qib_sdma.c|8 
 drivers/infiniband/hw/qib/qib_srq.c |   28 +
 drivers/infiniband/hw/qib/qib_uc.c  |8 
 drivers/infiniband/hw/qib/qib_ud.c  |   16 -
 drivers/infiniband/hw/qib/qib_verbs.c   |  175 -
 drivers/infiniband/hw/qib/qib_verbs.h   |  436 +++---
 drivers/infiniband/hw/qib/qib_verbs_mcast.c |8 
 17 files changed, 308 insertions(+), 1364 deletions(-)
 delete mode 100644 drivers/infiniband/hw/qib/qib_mr.c

diff --git a/drivers/infiniband/hw/qib/Makefile 
b/drivers/infiniband/hw/qib/Makefile
index 8a8f892..75140f5 100644
--- a/drivers/infiniband/hw/qib/Makefile
+++ b/drivers/infiniband/hw/qib/Makefile
@@ -1,8 +1,8 @@
 obj-$(CONFIG_INFINIBAND_QIB) += ib_qib.o
 
 ib_qib-y := qib_cq.o qib_diag.o qib_driver.o qib_eeprom.o \
-   qib_file_ops.o qib_fs.o qib_init.o qib_intr.o qib_keys.o \
-   qib_mad.o qib_mmap.o qib_mr.o qib_pcie.o qib_pio_copy.o \
+   qib_file_ops.o qib_fs.o qib_init.o qib_intr.o \
+   qib_mad.o qib_mmap.o qib_pcie.o qib_pio_copy.o \
qib_qp.o qib_qsfp.o qib_rc.o qib_ruc.o qib_sdma.o qib_srq.o \
qib_sysfs.o qib_twsi.o qib_tx.o qib_uc.o qib_ud.o \
qib_user_pages.o qib_user_sdma.o qib_verbs_mcast.o qib_iba7220.o \
diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index e610eaf..309b6f3 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -231,7 +231,7 @@ struct qib_ctxtdata {
/* ctxt rcvhdrq head offset */
u32 head;
/* lookaside fields */
-   struct qib_qp *lookaside_qp;
+   struct rvt_qp *lookaside_qp;
u32 lookaside_qpn;
/* QPs waiting for context processing */
struct list_head qp_wait_list;
@@ -241,7 +241,7 @@ struct qib_ctxtdata {
 #endif
 };
 
-struct qib_sge_state;
+struct rvt_sge_state;
 
 struct qib_sdma_txreq {
int flags;
@@ -259,14 +259,14 @@ struct qib_sdma_desc {
 
 struct qib_verbs_txreq {
struct qib_sdma_txreq   txreq;
-   struct qib_qp   *qp;
-   struct qib_swqe *wqe;
+   struct rvt_qp   *qp;
+   struct rvt_swqe *wqe;
u32 dwords;
u16 hdr_dwords;
u16 hdr_inx;
struct qib_pio_header   *align_buf;
-   struct qib_mregion  *mr;
-   struct qib_sge_state*ss;
+   struct rvt_mregion  *mr;
+   struct rvt_sge_state*ss;
 };
 
 #define QIB_SDMA_TXREQ_F_USELARGEBUF  0x1
@@ -1324,7 +1324,7 @@ void __qib_sdma_intr(struct qib_pportdata *);
 void qib_sdma_intr(struct qib_pportdata *);
 void qib_user_sdma_send_desc(struct qib_pportdata *dd,
struct list_head *pktlist);
-int qib_sdma_verbs_send(struct qib_pportdata *, struct qib_sge_state *,
+int qib_sdma_verbs_send(struct qib_pportdata *, struct rvt_sge_state *,
u32, struct qib_verbs_txreq *);
 /* ppd->sdma_lock should be locked before calling this. */
 int qib_sdma_make_progress(struct qib_pportdata *dd);
diff --git a/drivers/infiniband/hw/qib/qib_cq.c 
b/drivers/infiniband/hw/qib/qib_cq.c
index 2b45d0b..c1ea21e 100644
--- a/drivers/infiniband/hw/qib/qib_cq.c
+++ b/drivers/infiniband/hw/qib/qib_cq.c
@@ -466,7 +466,7 @@ int qib_resize_cq(struct ib_cq *ibcq, int cqe, struct 
ib_udata *udata)
 
if (cq->ip) {
struct qib_ibdev *dev = to_idev(ibcq->device);
-   struct qib_mmap_info *ip = cq->ip;
+   struct rvt_mmap_info *ip = cq->ip;
 
qib_update_mmap_info(dev, ip, sz, wc);
 
diff --git a/drivers/infiniband/hw/qib/qib_driver.c 
b/drivers/infiniband/hw/qib/qib_driver.c
index ae5a725..eafdee9 100644
--- a/drivers/infiniband/hw/qib/qib_driver.c
+++ b/drivers/infiniband/hw/qib/qib_driver.c
@@ -322,7 +322,7 @@ static u32 qib_rcv_hdrerr(struct qib_ctxtdata *rcd, struct 
qib_pport

[PATCH v2 06/10] IB/qib: Remove driver specific members from qib qp type

[Dennis Dalessandro]

In preparation for moving the queue pair data structure to rdmavt the
members of the driver specific queue pairs which are not common need to be
pushed off to a private driver structure. This structure will be available
in the queue pair once moved to rdmavt as a void pointer. This patch while
not adding a lot of value in and of itself is a prerequisite to move the
queue pair out of the drivers and into rdmavt.

The driver specific, private queue pair data structure should condense as
more of the send side code moves to rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_qp.c|   75 -
 drivers/infiniband/hw/qib/qib_rc.c|7 ++-
 drivers/infiniband/hw/qib/qib_ruc.c   |   18 +---
 drivers/infiniband/hw/qib/qib_sdma.c  |   17 +--
 drivers/infiniband/hw/qib/qib_uc.c|7 ++-
 drivers/infiniband/hw/qib/qib_ud.c|   22 +-
 drivers/infiniband/hw/qib/qib_verbs.c |   61 +--
 drivers/infiniband/hw/qib/qib_verbs.h |   26 ---
 8 files changed, 145 insertions(+), 88 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index 116d323..7e570be 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -370,10 +370,11 @@ struct qib_qp *qib_lookup_qpn(struct qib_ibport *ibp, u32 
qpn)
  */
 static void qib_reset_qp(struct qib_qp *qp, enum ib_qp_type type)
 {
+   struct qib_qp_priv *priv = qp->priv;
qp->remote_qpn = 0;
qp->qkey = 0;
qp->qp_access_flags = 0;
-   atomic_set(>s_dma_busy, 0);
+   atomic_set(>s_dma_busy, 0);
qp->s_flags &= QIB_S_SIGNAL_REQ_WR;
qp->s_hdrwords = 0;
qp->s_wqe = NULL;
@@ -473,6 +474,7 @@ static void clear_mr_refs(struct qib_qp *qp, int clr_sends)
  */
 int qib_error_qp(struct qib_qp *qp, enum ib_wc_status err)
 {
+   struct qib_qp_priv *priv = qp->priv;
struct qib_ibdev *dev = to_idev(qp->ibqp.device);
struct ib_wc wc;
int ret = 0;
@@ -491,9 +493,9 @@ int qib_error_qp(struct qib_qp *qp, enum ib_wc_status err)
qp->s_flags &= ~QIB_S_ANY_WAIT_SEND;
 
spin_lock(>pending_lock);
-   if (!list_empty(>iowait) && !(qp->s_flags & QIB_S_BUSY)) {
+   if (!list_empty(>iowait) && !(qp->s_flags & QIB_S_BUSY)) {
qp->s_flags &= ~QIB_S_ANY_WAIT_IO;
-   list_del_init(>iowait);
+   list_del_init(>iowait);
}
spin_unlock(>pending_lock);
 
@@ -503,9 +505,9 @@ int qib_error_qp(struct qib_qp *qp, enum ib_wc_status err)
qib_put_mr(qp->s_rdma_mr);
qp->s_rdma_mr = NULL;
}
-   if (qp->s_tx) {
-   qib_put_txreq(qp->s_tx);
-   qp->s_tx = NULL;
+   if (priv->s_tx) {
+   qib_put_txreq(priv->s_tx);
+   priv->s_tx = NULL;
}
}
 
@@ -571,6 +573,7 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
 {
struct qib_ibdev *dev = to_idev(ibqp->device);
struct qib_qp *qp = to_iqp(ibqp);
+   struct qib_qp_priv *priv = qp->priv;
enum ib_qp_state cur_state, new_state;
struct ib_event ev;
int lastwqe = 0;
@@ -698,19 +701,20 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
if (qp->state != IB_QPS_RESET) {
qp->state = IB_QPS_RESET;
spin_lock(>pending_lock);
-   if (!list_empty(>iowait))
-   list_del_init(>iowait);
+   if (!list_empty(>iowait))
+   list_del_init(>iowait);
spin_unlock(>pending_lock);
qp->s_flags &= ~(QIB_S_TIMER | QIB_S_ANY_WAIT);
spin_unlock(>s_lock);
spin_unlock_irq(>r_lock);
/* Stop the sending work queue and retry timer */
-   cancel_work_sync(>s_work);
+   cancel_work_sync(>s_work);
del_timer_sync(>s_timer);
-   wait_event(qp->wait_dma, !atomic_read(>s_dma_busy));
-   if (qp->s_tx) {
-   qib_put_txreq(qp->s_tx);
-   qp->s_tx = NULL;
+   wait_event(priv->wait_dma,
+  !atomic_read(>s_dma_busy));
+   if (priv->s_tx) {
+ 

[PATCH v2 04/10] IB/qib: Remove most uses of QIB_PERMISSIVE_LID and QIB_MULTICAST_LID_BASE

[Dennis Dalessandro]

This patch removes most of the uses of QIB_PERMISSIBVE_LID and
QIB_MULTICAST_LID_BASE in favor of the recently added IB_* versions.
There are still minor uses in AH functions as well as the QIB_* defines
but those will be removed in a follow on patch.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_driver.c |2 +-
 drivers/infiniband/hw/qib/qib_mad.c|4 ++--
 drivers/infiniband/hw/qib/qib_qp.c |5 +++--
 drivers/infiniband/hw/qib/qib_ud.c |   11 ++-
 drivers/infiniband/hw/qib/qib_verbs.c  |2 +-
 5 files changed, 13 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_driver.c 
b/drivers/infiniband/hw/qib/qib_driver.c
index f58fdc3..57b19ea 100644
--- a/drivers/infiniband/hw/qib/qib_driver.c
+++ b/drivers/infiniband/hw/qib/qib_driver.c
@@ -319,7 +319,7 @@ static u32 qib_rcv_hdrerr(struct qib_ctxtdata *rcd, struct 
qib_pportdata *ppd,
if (tlen < 24)
goto drop;
 
-   if (lid < QIB_MULTICAST_LID_BASE) {
+   if (lid < be16_to_cpu(IB_MULTICAST_LID_BASE)) {
lid &= ~((1 << ppd->lmc) - 1);
if (unlikely(lid != ppd->lid))
goto drop;
diff --git a/drivers/infiniband/hw/qib/qib_mad.c 
b/drivers/infiniband/hw/qib/qib_mad.c
index c65d3aa..70fc1b2 100644
--- a/drivers/infiniband/hw/qib/qib_mad.c
+++ b/drivers/infiniband/hw/qib/qib_mad.c
@@ -691,7 +691,7 @@ static int subn_set_portinfo(struct ib_smp *smp, struct 
ib_device *ibdev,
 
lid = be16_to_cpu(pip->lid);
/* Must be a valid unicast LID address. */
-   if (lid == 0 || lid >= QIB_MULTICAST_LID_BASE)
+   if (lid == 0 || lid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
smp->status |= IB_SMP_INVALID_FIELD;
else if (ppd->lid != lid || ppd->lmc != (pip->mkeyprot_resv_lmc & 7)) {
if (ppd->lid != lid)
@@ -706,7 +706,7 @@ static int subn_set_portinfo(struct ib_smp *smp, struct 
ib_device *ibdev,
smlid = be16_to_cpu(pip->sm_lid);
msl = pip->neighbormtu_mastersmsl & 0xF;
/* Must be a valid unicast LID address. */
-   if (smlid == 0 || smlid >= QIB_MULTICAST_LID_BASE)
+   if (smlid == 0 || smlid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
smp->status |= IB_SMP_INVALID_FIELD;
else if (smlid != ibp->sm_lid || msl != ibp->sm_sl) {
spin_lock_irqsave(>lock, flags);
diff --git a/drivers/infiniband/hw/qib/qib_qp.c 
b/drivers/infiniband/hw/qib/qib_qp.c
index 40f85bb..e6544dc 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -589,14 +589,15 @@ int qib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr 
*attr,
goto inval;
 
if (attr_mask & IB_QP_AV) {
-   if (attr->ah_attr.dlid >= QIB_MULTICAST_LID_BASE)
+   if (attr->ah_attr.dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
goto inval;
if (qib_check_ah(qp->ibqp.device, >ah_attr))
goto inval;
}
 
if (attr_mask & IB_QP_ALT_PATH) {
-   if (attr->alt_ah_attr.dlid >= QIB_MULTICAST_LID_BASE)
+   if (attr->alt_ah_attr.dlid >=
+   be16_to_cpu(IB_MULTICAST_LID_BASE))
goto inval;
if (qib_check_ah(qp->ibqp.device, >alt_ah_attr))
goto inval;
diff --git a/drivers/infiniband/hw/qib/qib_ud.c 
b/drivers/infiniband/hw/qib/qib_ud.c
index 59193f6..32fc80c 100644
--- a/drivers/infiniband/hw/qib/qib_ud.c
+++ b/drivers/infiniband/hw/qib/qib_ud.c
@@ -32,6 +32,7 @@
  */
 
 #include 
+#include 
 
 #include "qib.h"
 #include "qib_mad.h"
@@ -278,8 +279,8 @@ int qib_make_ud_req(struct qib_qp *qp)
ibp = to_iport(qp->ibqp.device, qp->port_num);
ppd = ppd_from_ibp(ibp);
ah_attr = _iah(wqe->ud_wr.ah)->attr;
-   if (ah_attr->dlid >= QIB_MULTICAST_LID_BASE) {
-   if (ah_attr->dlid != QIB_PERMISSIVE_LID)
+   if (ah_attr->dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE)) {
+   if (ah_attr->dlid != be16_to_cpu(IB_LID_PERMISSIVE))
this_cpu_inc(ibp->pmastats->n_multicast_xmit);
else
this_cpu_inc(ibp->pmastats->n_unicast_xmit);
@@ -368,8 +369,8 @@ int qib_make_ud_req(struct qib_qp *qp)
/*
 * Use the multicast QP if the destination LID is a multicast LID.
 */
-   ohdr->bth[1] = ah_attr->dlid >= QIB_MULTICAST_LID_BASE &&
-   ah_attr->dlid != QIB_PERMISSIVE_LID ?

[PATCH v2 01/10] IB/qib: Begin to use rdmavt for verbs

[Dennis Dalessandro]

This patch begins to make use of rdmavt by registering with it and
providing access to the header files. This is just the beginning of
rdmavt support in qib.

Most functionality is still being done in the driver, set flags so that
rdmavt will let qib continue to handle mr, qp, and cq init.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/Kconfig |2 +-
 drivers/infiniband/hw/qib/qib.h   |1 +
 drivers/infiniband/hw/qib/qib_init.c  |5 +++--
 drivers/infiniband/hw/qib/qib_intr.c  |2 +-
 drivers/infiniband/hw/qib/qib_mad.c   |5 +++--
 drivers/infiniband/hw/qib/qib_sysfs.c |   20 ++--
 drivers/infiniband/hw/qib/qib_verbs.c |   19 ++-
 drivers/infiniband/hw/qib/qib_verbs.h |8 ++--
 8 files changed, 39 insertions(+), 23 deletions(-)

diff --git a/drivers/infiniband/hw/qib/Kconfig 
b/drivers/infiniband/hw/qib/Kconfig
index 495be09..e0fdb92 100644
--- a/drivers/infiniband/hw/qib/Kconfig
+++ b/drivers/infiniband/hw/qib/Kconfig
@@ -1,6 +1,6 @@
 config INFINIBAND_QIB
tristate "Intel PCIe HCA support"
-   depends on 64BIT
+   depends on 64BIT && INFINIBAND_RDMAVT
---help---
This is a low-level driver for Intel PCIe QLE InfiniBand host
channel adapters.  This driver does not support the Intel
diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index 7df16f7..0e68e1f 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -52,6 +52,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "qib_common.h"
 #include "qib_verbs.h"
diff --git a/drivers/infiniband/hw/qib/qib_init.c 
b/drivers/infiniband/hw/qib/qib_init.c
index 4ff340f..47190f1 100644
--- a/drivers/infiniband/hw/qib/qib_init.c
+++ b/drivers/infiniband/hw/qib/qib_init.c
@@ -42,6 +42,7 @@
 #ifdef CONFIG_INFINIBAND_QIB_DCA
 #include 
 #endif
+#include 
 
 #include "qib.h"
 #include "qib_common.h"
@@ -1081,7 +1082,7 @@ void qib_free_devdata(struct qib_devdata *dd)
qib_dbg_ibdev_exit(>verbs_dev);
 #endif
free_percpu(dd->int_counter);
-   ib_dealloc_device(>verbs_dev.ibdev);
+   ib_dealloc_device(>verbs_dev.rdi.ibdev);
 }
 
 u64 qib_int_counter(struct qib_devdata *dd)
@@ -1171,7 +1172,7 @@ struct qib_devdata *qib_alloc_devdata(struct pci_dev 
*pdev, size_t extra)
 bail:
if (!list_empty(>list))
list_del_init(>list);
-   ib_dealloc_device(>verbs_dev.ibdev);
+   ib_dealloc_device(>verbs_dev.rdi.ibdev);
return ERR_PTR(ret);
 }
 
diff --git a/drivers/infiniband/hw/qib/qib_intr.c 
b/drivers/infiniband/hw/qib/qib_intr.c
index 086616d..a014fd4 100644
--- a/drivers/infiniband/hw/qib/qib_intr.c
+++ b/drivers/infiniband/hw/qib/qib_intr.c
@@ -74,7 +74,7 @@ static void signal_ib_event(struct qib_pportdata *ppd, enum 
ib_event_type ev)
struct ib_event event;
struct qib_devdata *dd = ppd->dd;
 
-   event.device = >verbs_dev.ibdev;
+   event.device = >verbs_dev.rdi.ibdev;
event.element.port_num = ppd->port;
event.event = ev;
ib_dispatch_event();
diff --git a/drivers/infiniband/hw/qib/qib_mad.c 
b/drivers/infiniband/hw/qib/qib_mad.c
index 9625e7c..c65d3aa 100644
--- a/drivers/infiniband/hw/qib/qib_mad.c
+++ b/drivers/infiniband/hw/qib/qib_mad.c
@@ -1028,7 +1028,7 @@ static int set_pkeys(struct qib_devdata *dd, u8 port, u16 
*pkeys)
(void) dd->f_set_ib_cfg(ppd, QIB_IB_CFG_PKEYS, 0);
 
event.event = IB_EVENT_PKEY_CHANGE;
-   event.device = >verbs_dev.ibdev;
+   event.device = >verbs_dev.rdi.ibdev;
event.element.port_num = port;
ib_dispatch_event();
}
@@ -2483,7 +2483,8 @@ int qib_create_agents(struct qib_ibdev *dev)
 
for (p = 0; p < dd->num_pports; p++) {
ibp = >pport[p].ibport_data;
-   agent = ib_register_mad_agent(>ibdev, p + 1, IB_QPT_SMI,
+   agent = ib_register_mad_agent(>rdi.ibdev, p + 1,
+ IB_QPT_SMI,
  NULL, 0, send_handler,
  NULL, NULL, 0);
if (IS_ERR(agent)) {
diff --git a/drivers/infiniband/hw/qib/qib_sysfs.c 
b/drivers/infiniband/hw/qib/qib_sysfs.c
index 81f56cd..72a160e 100644
--- a/drivers/infiniband/hw/qib/qib_sysfs.c
+++ b/drivers/infiniband/hw/qib/qib_sysfs.c
@@ -502,7 +502,7 @@ static ssize_t show_rev(struct device *device, struct 
device_attribute *attr,
char *buf)
 {
struct qib_ibdev *dev =
-   container_of(device, struct qib_ibdev, ibdev.dev);
+   container_of(

[PATCH v2 00/10] Begin to use rdmavt for qib

[Dennis Dalessandro]

The following series implements the beginning support of rdmavt in the qib
driver. The memory registration and protection domain functionality has been
moved out of the driver as well as critical qp data structures. More patches
will follow to incrementally remove code from qib and use the rdmavt
implementation.

This patch builds on the v2 "Add rdma verbs transport library" patch.

Changes since v1:
Rebased onto what appears to be the latest branch on Doug's kernel.org repo
 (k.o/for-4.4-rc). I can rebase onto something else if needed.

---

Dennis Dalessandro (10):
  IB/qib: Begin to use rdmavt for verbs
  IB/qib: Remove dma.c and use rdmavt version of dma functions
  IB/qib: Use rdmavt protection domain
  IB/qib: Remove most uses of QIB_PERMISSIVE_LID and QIB_MULTICAST_LID_BASE
  IB/qib: Use rdmavt lid defines in qib
  IB/qib: Remove driver specific members from qib qp type
  IB/qib: Add device specific info prints
  IB/qib: Remove qp and mr functionality from qib
  IB/qib: Use address handle in rdmavt and remove from qib
  IB/qib: Remove srq from qib


 drivers/infiniband/hw/qib/Kconfig   |2 
 drivers/infiniband/hw/qib/Makefile  |6 
 drivers/infiniband/hw/qib/qib.h |   17 +
 drivers/infiniband/hw/qib/qib_common.h  |2 
 drivers/infiniband/hw/qib/qib_cq.c  |2 
 drivers/infiniband/hw/qib/qib_driver.c  |   22 +
 drivers/infiniband/hw/qib/qib_iba7322.c |2 
 drivers/infiniband/hw/qib/qib_init.c|5 
 drivers/infiniband/hw/qib/qib_intr.c|2 
 drivers/infiniband/hw/qib/qib_keys.c|  186 +
 drivers/infiniband/hw/qib/qib_mad.c |   11 -
 drivers/infiniband/hw/qib/qib_mmap.c|   18 -
 drivers/infiniband/hw/qib/qib_mr.c  |  539 ---
 drivers/infiniband/hw/qib/qib_qp.c  |  161 +---
 drivers/infiniband/hw/qib/qib_rc.c  |  105 +++--
 drivers/infiniband/hw/qib/qib_ruc.c |   84 ++--
 drivers/infiniband/hw/qib/qib_sdma.c|   25 +
 drivers/infiniband/hw/qib/qib_srq.c |   38 +-
 drivers/infiniband/hw/qib/qib_sysfs.c   |   20 +
 drivers/infiniband/hw/qib/qib_uc.c  |   15 -
 drivers/infiniband/hw/qib/qib_ud.c  |   53 +--
 drivers/infiniband/hw/qib/qib_verbs.c   |  448 +++---
 drivers/infiniband/hw/qib/qib_verbs.h   |  488 +++-
 drivers/infiniband/hw/qib/qib_verbs_mcast.c |8 
 24 files changed, 546 insertions(+), 1713 deletions(-)
 delete mode 100644 drivers/infiniband/hw/qib/qib_mr.c

-- 
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 07/10] IB/qib: Add device specific info prints

[Dennis Dalessandro]

Implement get_card_name and get_pci_dev helper functions for rdmavt
for qib.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib.h|2 ++
 drivers/infiniband/hw/qib/qib_driver.c |   16 
 drivers/infiniband/hw/qib/qib_verbs.c  |2 ++
 3 files changed, 20 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index 0e68e1f..e610eaf 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -1455,6 +1455,8 @@ u64 qib_sps_ints(void);
 dma_addr_t qib_map_page(struct pci_dev *, struct page *, unsigned long,
  size_t, int);
 const char *qib_get_unit_name(int unit);
+const char *qib_get_card_name(struct rvt_dev_info *rdi);
+struct pci_dev *qib_get_pci_dev(struct rvt_dev_info *rdi);
 
 /*
  * Flush write combining store buffers (if present) and perform a write
diff --git a/drivers/infiniband/hw/qib/qib_driver.c 
b/drivers/infiniband/hw/qib/qib_driver.c
index 57b19ea..ae5a725 100644
--- a/drivers/infiniband/hw/qib/qib_driver.c
+++ b/drivers/infiniband/hw/qib/qib_driver.c
@@ -90,6 +90,22 @@ const char *qib_get_unit_name(int unit)
return iname;
 }
 
+const char *qib_get_card_name(struct rvt_dev_info *rdi)
+{
+   struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi);
+   struct qib_devdata *dd = container_of(ibdev,
+ struct qib_devdata, verbs_dev);
+   return qib_get_unit_name(dd->unit);
+}
+
+struct pci_dev *qib_get_pci_dev(struct rvt_dev_info *rdi)
+{
+   struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi);
+   struct qib_devdata *dd = container_of(ibdev,
+ struct qib_devdata, verbs_dev);
+   return dd->pcidev;
+}
+
 /*
  * Return count of units with at least one port ACTIVE.
  */
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index 32c2811..b7e03ae 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -2240,6 +2240,8 @@ int qib_register_ib_device(struct qib_devdata *dd)
 * Fill in rvt info object.
 */
dd->verbs_dev.rdi.driver_f.port_callback = qib_create_port_files;
+   dd->verbs_dev.rdi.driver_f.get_card_name = qib_get_card_name;
+   dd->verbs_dev.rdi.driver_f.get_pci_dev = qib_get_pci_dev;
dd->verbs_dev.rdi.dparms.props.max_pd = ib_qib_max_pds;
dd->verbs_dev.rdi.flags = (RVT_FLAG_MR_INIT_DRIVER |
   RVT_FLAG_QP_INIT_DRIVER |

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 03/10] IB/qib: Use rdmavt protection domain

[Dennis Dalessandro]

Remove protection domain datastructure from qib and use rdmavts version.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/qib_keys.c  |6 ++-
 drivers/infiniband/hw/qib/qib_mr.c|2 +
 drivers/infiniband/hw/qib/qib_ruc.c   |4 +-
 drivers/infiniband/hw/qib/qib_verbs.c |   66 +++--
 drivers/infiniband/hw/qib/qib_verbs.h |   13 +--
 5 files changed, 12 insertions(+), 79 deletions(-)

diff --git a/drivers/infiniband/hw/qib/qib_keys.c 
b/drivers/infiniband/hw/qib/qib_keys.c
index d725c56..04fa272 100644
--- a/drivers/infiniband/hw/qib/qib_keys.c
+++ b/drivers/infiniband/hw/qib/qib_keys.c
@@ -152,7 +152,7 @@ out:
  * Check the IB SGE for validity and initialize our internal version
  * of it.
  */
-int qib_lkey_ok(struct qib_lkey_table *rkt, struct qib_pd *pd,
+int qib_lkey_ok(struct qib_lkey_table *rkt, struct rvt_pd *pd,
struct qib_sge *isge, struct ib_sge *sge, int acc)
 {
struct qib_mregion *mr;
@@ -263,7 +263,7 @@ int qib_rkey_ok(struct qib_qp *qp, struct qib_sge *sge,
 */
rcu_read_lock();
if (rkey == 0) {
-   struct qib_pd *pd = to_ipd(qp->ibqp.pd);
+   struct rvt_pd *pd = ibpd_to_rvtpd(qp->ibqp.pd);
struct qib_ibdev *dev = to_idev(pd->ibpd.device);
 
if (pd->user)
@@ -341,7 +341,7 @@ bail:
 int qib_reg_mr(struct qib_qp *qp, struct ib_reg_wr *wr)
 {
struct qib_lkey_table *rkt = _idev(qp->ibqp.device)->lk_table;
-   struct qib_pd *pd = to_ipd(qp->ibqp.pd);
+   struct rvt_pd *pd = ibpd_to_rvtpd(qp->ibqp.pd);
struct qib_mr *mr = to_imr(wr->mr);
struct qib_mregion *mrg;
u32 key = wr->key;
diff --git a/drivers/infiniband/hw/qib/qib_mr.c 
b/drivers/infiniband/hw/qib/qib_mr.c
index 294f5c7..7fe4159 100644
--- a/drivers/infiniband/hw/qib/qib_mr.c
+++ b/drivers/infiniband/hw/qib/qib_mr.c
@@ -99,7 +99,7 @@ struct ib_mr *qib_get_dma_mr(struct ib_pd *pd, int acc)
struct ib_mr *ret;
int rval;
 
-   if (to_ipd(pd)->user) {
+   if (ibpd_to_rvtpd(pd)->user) {
ret = ERR_PTR(-EPERM);
goto bail;
}
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c 
b/drivers/infiniband/hw/qib/qib_ruc.c
index b1aa21b..425c8c2 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -84,11 +84,11 @@ static int qib_init_sge(struct qib_qp *qp, struct qib_rwqe 
*wqe)
int i, j, ret;
struct ib_wc wc;
struct qib_lkey_table *rkt;
-   struct qib_pd *pd;
+   struct rvt_pd *pd;
struct qib_sge_state *ss;
 
rkt = _idev(qp->ibqp.device)->lk_table;
-   pd = to_ipd(qp->ibqp.srq ? qp->ibqp.srq->pd : qp->ibqp.pd);
+   pd = ibpd_to_rvtpd(qp->ibqp.srq ? qp->ibqp.srq->pd : qp->ibqp.pd);
ss = >r_sge;
ss->sg_list = qp->r_sg_list;
qp->r_len = 0;
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index fb4f045..7711d64 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -346,7 +346,7 @@ static int qib_post_one_send(struct qib_qp *qp, struct 
ib_send_wr *wr,
int ret;
unsigned long flags;
struct qib_lkey_table *rkt;
-   struct qib_pd *pd;
+   struct rvt_pd *pd;
 
spin_lock_irqsave(>s_lock, flags);
 
@@ -396,7 +396,7 @@ static int qib_post_one_send(struct qib_qp *qp, struct 
ib_send_wr *wr,
}
 
rkt = _idev(qp->ibqp.device)->lk_table;
-   pd = to_ipd(qp->ibqp.pd);
+   pd = ibpd_to_rvtpd(qp->ibqp.pd);
wqe = get_swqe_ptr(qp, qp->s_head);
 
if (qp->ibqp.qp_type != IB_QPT_UC &&
@@ -1599,7 +1599,7 @@ static int qib_query_device(struct ib_device *ibdev, 
struct ib_device_attr *prop
props->max_mr = dev->lk_table.max;
props->max_fmr = dev->lk_table.max;
props->max_map_per_fmr = 32767;
-   props->max_pd = ib_qib_max_pds;
+   props->max_pd = dev->rdi.dparms.props.max_pd;
props->max_qp_rd_atom = QIB_MAX_RDMA_ATOMIC;
props->max_qp_init_rd_atom = 255;
/* props->max_res_rd_atom */
@@ -1751,61 +1751,6 @@ static int qib_query_gid(struct ib_device *ibdev, u8 
port,
return ret;
 }
 
-static struct ib_pd *qib_alloc_pd(struct ib_device *ibdev,
- struct ib_ucontext *context,
- struct ib_udata *udata)
-{
-   struct qib_ibdev *dev = to_idev(ibdev);
-   struct qib_pd *pd;
-   struct ib_pd *ret;
-
-   /*
-* This is actually totally arbitrary.  Some correctness tests
-* assume there's a maximum num

[PATCH v2 02/10] IB/qib: Remove dma.c and use rdmavt version of dma functions

[Dennis Dalessandro]

This patch removes the qib_dma.c file and uses the version which has been
added to rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/hw/qib/Makefile|2 +-
 drivers/infiniband/hw/qib/qib_verbs.c |3 ++-
 drivers/infiniband/hw/qib/qib_verbs.h |2 --
 3 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/hw/qib/Makefile 
b/drivers/infiniband/hw/qib/Makefile
index 57f8103..8a8f892 100644
--- a/drivers/infiniband/hw/qib/Makefile
+++ b/drivers/infiniband/hw/qib/Makefile
@@ -1,6 +1,6 @@
 obj-$(CONFIG_INFINIBAND_QIB) += ib_qib.o
 
-ib_qib-y := qib_cq.o qib_diag.o qib_dma.o qib_driver.o qib_eeprom.o \
+ib_qib-y := qib_cq.o qib_diag.o qib_driver.o qib_eeprom.o \
qib_file_ops.o qib_fs.o qib_init.o qib_intr.o qib_keys.o \
qib_mad.o qib_mmap.o qib_mr.o qib_pcie.o qib_pio_copy.o \
qib_qp.o qib_qsfp.o qib_rc.o qib_ruc.o qib_sdma.o qib_srq.o \
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c 
b/drivers/infiniband/hw/qib/qib_verbs.c
index 9c721aa..fb4f045 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -41,6 +41,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "qib.h"
 #include "qib_common.h"
@@ -2269,7 +2270,7 @@ int qib_register_ib_device(struct qib_devdata *dd)
ibdev->detach_mcast = qib_multicast_detach;
ibdev->process_mad = qib_process_mad;
ibdev->mmap = qib_mmap;
-   ibdev->dma_ops = _dma_mapping_ops;
+   ibdev->dma_ops = NULL;
ibdev->get_port_immutable = qib_port_immutable;
 
snprintf(ibdev->node_desc, sizeof(ibdev->node_desc),
diff --git a/drivers/infiniband/hw/qib/qib_verbs.h 
b/drivers/infiniband/hw/qib/qib_verbs.h
index 1e43fda..8487914 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.h
+++ b/drivers/infiniband/hw/qib/qib_verbs.h
@@ -1186,6 +1186,4 @@ extern unsigned int ib_qib_max_srq_wrs;
 
 extern const u32 ib_qib_rnr_table[];
 
-extern struct ib_dma_mapping_ops qib_dma_mapping_ops;
-
 #endif  /* QIB_VERBS_H */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 02/36] IB/rdmavt: Consolidate dma ops in rdmavt.

[Dennis Dalessandro]

This patch adds dma functions to rdmavt. The source is hfi1's version of
dma.c which will be removed by a subsequent hfi1 patch.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
return 0 in rvt_map_sg instead of BAD_DMA_ADDRESS
remove include of dma.h from dma.c

 drivers/infiniband/sw/rdmavt/Makefile |2 
 drivers/infiniband/sw/rdmavt/dma.c|  182 +
 drivers/infiniband/sw/rdmavt/dma.h|   53 ++
 drivers/infiniband/sw/rdmavt/vt.c |   10 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 247 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/dma.c
 create mode 100644 drivers/infiniband/sw/rdmavt/dma.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 98a664d..134d2d0 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o
+rdmavt-y := vt.o dma.o
diff --git a/drivers/infiniband/sw/rdmavt/dma.c 
b/drivers/infiniband/sw/rdmavt/dma.c
new file mode 100644
index 000..8fd7955
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/dma.c
@@ -0,0 +1,182 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+#include 
+#include 
+#include 
+
+#define BAD_DMA_ADDRESS ((u64)0)
+
+/*
+ * The following functions implement driver specific replacements
+ * for the ib_dma_*() functions.
+ *
+ * These functions return kernel virtual addresses instead of
+ * device bus addresses since the driver uses the CPU to copy
+ * data instead of using hardware DMA.
+ */
+
+static int rvt_mapping_error(struct ib_device *dev, u64 dma_addr)
+{
+   return dma_addr == BAD_DMA_ADDRESS;
+}
+
+static u64 rvt_dma_map_single(struct ib_device *dev, void *cpu_addr,
+ size_t size, enum dma_data_direction direction)
+{
+   if (WARN_ON(!valid_dma_direction(direction)))
+   return BAD_DMA_ADDRESS;
+
+   return (u64)cpu_addr;
+}
+
+static void rvt_dma_unmap_single(struct ib_device *dev, u64 addr, size_t size,
+enum dma_data_direction direction)
+{
+   /* This is a stub, nothing to be done here */
+}
+
+static u64 rvt_dma_map_page(struct ib_device *dev, struct page *page,
+   unsigned long offset, size_t size,
+   enum dma_data_direction direction)
+{
+   u64 addr;
+
+   if (WARN_ON(!valid_dma_direction(direction)))
+   return BAD_DMA_ADDRESS;
+
+   if (offset + size > PAGE_SIZE)
+   return BAD_DMA

[PATCH v2 01/36] IB/rdmavt: Create module framework and handle driver registration

[Dennis Dalessandro]

This patch introduces the basics for a new module called rdma_vt. This new
driver is a software implementation of the InfiniBand verbs and aims to
replace the multiple implementations that exist and duplicate each others'
code.

While the call to actually register the device with the IB core happens in
rdma_vt, most of the work is still done in the drivers themselves. This
will be changing in a follow on patch this is just laying the groundwork
for this infrastructure.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
remove rdmavt specific driver version
move copyright above license text

 MAINTAINERS   |6 ++
 drivers/infiniband/Kconfig|2 +
 drivers/infiniband/Makefile   |1 
 drivers/infiniband/sw/Makefile|1 
 drivers/infiniband/sw/rdmavt/Kconfig  |6 ++
 drivers/infiniband/sw/rdmavt/Makefile |   10 
 drivers/infiniband/sw/rdmavt/vt.c |   83 +
 drivers/infiniband/sw/rdmavt/vt.h |   53 +
 include/rdma/rdma_vt.h|   70 
 9 files changed, 232 insertions(+), 0 deletions(-)
 create mode 100644 drivers/infiniband/sw/Makefile
 create mode 100644 drivers/infiniband/sw/rdmavt/Kconfig
 create mode 100644 drivers/infiniband/sw/rdmavt/Makefile
 create mode 100644 drivers/infiniband/sw/rdmavt/vt.c
 create mode 100644 drivers/infiniband/sw/rdmavt/vt.h
 create mode 100644 include/rdma/rdma_vt.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 69c8a9c..37fc192 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8880,6 +8880,12 @@ L:   rds-de...@oss.oracle.com (moderated for 
non-subscribers)
 S: Supported
 F: net/rds/
 
+RDMAVT - RDMA verbs software
+M: Dennis Dalessandro <dennis.dalessan...@intel.com>
+L: linux-rdma@vger.kernel.org
+S: Supported
+F: drivers/infiniband/sw/rdmavt
+
 READ-COPY UPDATE (RCU)
 M: "Paul E. McKenney" <paul...@linux.vnet.ibm.com>
 M: Josh Triplett <j...@joshtriplett.org>
diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index aa26f3c..1b00bb8 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -72,4 +72,6 @@ source "drivers/infiniband/ulp/srpt/Kconfig"
 source "drivers/infiniband/ulp/iser/Kconfig"
 source "drivers/infiniband/ulp/isert/Kconfig"
 
+source "drivers/infiniband/sw/rdmavt/Kconfig"
+
 endif # INFINIBAND
diff --git a/drivers/infiniband/Makefile b/drivers/infiniband/Makefile
index dc21836..fad0b44 100644
--- a/drivers/infiniband/Makefile
+++ b/drivers/infiniband/Makefile
@@ -1,3 +1,4 @@
 obj-$(CONFIG_INFINIBAND)   += core/
 obj-$(CONFIG_INFINIBAND)   += hw/
 obj-$(CONFIG_INFINIBAND)   += ulp/
+obj-$(CONFIG_INFINIBAND)   += sw/
diff --git a/drivers/infiniband/sw/Makefile b/drivers/infiniband/sw/Makefile
new file mode 100644
index 000..988b6a0
--- /dev/null
+++ b/drivers/infiniband/sw/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_INFINIBAND_RDMAVT)+= rdmavt/
diff --git a/drivers/infiniband/sw/rdmavt/Kconfig 
b/drivers/infiniband/sw/rdmavt/Kconfig
new file mode 100644
index 000..11aa6a3
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/Kconfig
@@ -0,0 +1,6 @@
+config INFINIBAND_RDMAVT
+   tristate "RDMA verbs transport library"
+   depends on 64BIT
+   default m
+   ---help---
+   This is a common software verbs provider for RDMA networks.
diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
new file mode 100644
index 000..98a664d
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -0,0 +1,10 @@
+#
+# rdmavt driver
+#
+#
+#
+# Called from the kernel module build system.
+#
+obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
+
+rdmavt-y := vt.o
diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
new file mode 100644
index 000..aa325db
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -0,0 +1,83 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, wi

[PATCH v2 11/36] IB/rdmavt: Add queue pair function stubs

[Dennis Dalessandro]

Adds the stubs for create, modify, destroy and query functions for queue
pairs.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

drivers/infiniband/sw/rdmavt/Makefile |2 -
 drivers/infiniband/sw/rdmavt/qp.c |  120 +
 drivers/infiniband/sw/rdmavt/qp.h |   67 ++
 drivers/infiniband/sw/rdmavt/vt.c |6 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 195 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/qp.c
 create mode 100644 drivers/infiniband/sw/rdmavt/qp.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index c6751bb..9a9a095 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,5 +7,5 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o dma.o pd.o
+rdmavt-y := vt.o dma.o pd.o qp.o
 
diff --git a/drivers/infiniband/sw/rdmavt/qp.c 
b/drivers/infiniband/sw/rdmavt/qp.c
new file mode 100644
index 000..a59f28d
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -0,0 +1,120 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "qp.h"
+
+/**
+ * rvt_create_qp - create a queue pair for a device
+ * @ibpd: the protection domain who's device we create the queue pair for
+ * @init_attr: the attributes of the queue pair
+ * @udata: user data for libibverbs.so
+ *
+ * Returns the queue pair on success, otherwise returns an errno.
+ *
+ * Called by the ib_create_qp() core verbs function.
+ */
+struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
+   struct ib_qp_init_attr *init_attr,
+   struct ib_udata *udata)
+{
+   /*
+* Queue pair creation is mostly an rvt issue. However, drivers have
+* their own unique idea of what queue pare numbers mean. For instance
+* there is a reserved range for PSM.
+*
+* VI-DRIVER-API: make_qpn()
+* Returns a valid QPN for verbs to use
+*/
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * qib_modify_qp - modify the attributes of a queue pair
+ * @ibqp: the queue pair who's attributes we're modifying
+ * @attr: the new attributes
+ * @attr_mask: the mask of attributes to modify
+ * @udata: user data for libibverbs.so
+ *
+ * Returns 0 on success, otherwise returns an errno.
+ */
+int rvt_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata)
+{
+   /*
+* VT-DRIVER-API: qp_mtu()
+* OPA devices have a per VL MTU t

[PATCH v2 19/36] IB/rdmavt: Add completion queue function stubs

[Dennis Dalessandro]

Create stubs for completion queue creation, polling,
resizing, calling for notification, and destroying.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 -
 drivers/infiniband/sw/rdmavt/cq.c |  113 +
 drivers/infiniband/sw/rdmavt/cq.h |   62 ++
 drivers/infiniband/sw/rdmavt/vt.c |7 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 184 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/cq.c
 create mode 100644 drivers/infiniband/sw/rdmavt/cq.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 6f530d1..00f0188 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o
+rdmavt-y := vt.o ah.o cq.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/cq.c 
b/drivers/infiniband/sw/rdmavt/cq.c
new file mode 100644
index 000..8d96194
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/cq.c
@@ -0,0 +1,113 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "cq.h"
+
+/**
+ * rvt_create_cq - create a completion queue
+ * @ibdev: the device this completion queue is attached to
+ * @attr: creation attributes
+ * @context: unused by the QLogic_IB driver
+ * @udata: user data for libibverbs.so
+ *
+ * Returns a pointer to the completion queue or negative errno values
+ * for failure.
+ *
+ * Called by ib_create_cq() in the generic verbs code.
+ */
+struct ib_cq *rvt_create_cq(struct ib_device *ibdev,
+   const struct ib_cq_init_attr *attr,
+   struct ib_ucontext *context,
+   struct ib_udata *udata)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * rvt_destroy_cq - destroy a completion queue
+ * @ibcq: the completion queue to destroy.
+ *
+ * Returns 0 for success.
+ *
+ * Called by ib_destroy_cq() in the generic verbs code.
+ */
+int rvt_destroy_cq(struct ib_cq *ibcq)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags notify_flags)
+{
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_resize_cq - change the size of the CQ
+ * @ibcq: the completion queue
+ *
+ * Returns 0 for success.
+ */
+int rvt_resize_cq(struct ib_cq *ibcq, int cqe, struct ib_udata *udata)
+{
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_poll

[PATCH v2 29/36] IB/rdmavt: Add AH to rdmavt

[Dennis Dalessandro]

From: Kamal Heib <kam...@mellanox.com>

Original patch is from Kamal Heib <kam...@mellanox.com>. It has
been split into three separate patches. This one for rdmavt,
a follow on for qib, and one for hfi1.

Create datastructure for address handle and implement the
create/destroy/modify/query of address handle for rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Kamal Heib <kam...@mellanox.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/ah.c |   98 +++--
 drivers/infiniband/sw/rdmavt/vt.c |5 ++
 include/rdma/rdma_vt.h|   17 ++
 3 files changed, 115 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/ah.c 
b/drivers/infiniband/sw/rdmavt/ah.c
index d368955..2519db9 100644
--- a/drivers/infiniband/sw/rdmavt/ah.c
+++ b/drivers/infiniband/sw/rdmavt/ah.c
@@ -45,7 +45,49 @@
  *
  */
 
+#include 
 #include "ah.h"
+#include "vt.h" /* for prints */
+
+/**
+ * rvt_check_ah - validate the attributes of AH
+ * @ibdev: the ib device
+ * @ah_attr: the attributes of the AH
+ */
+int rvt_check_ah(struct ib_device *ibdev,
+struct ib_ah_attr *ah_attr)
+{
+   int err;
+   struct ib_port_attr port_attr;
+   struct rvt_dev_info *rdi = ib_to_rvt(ibdev);
+   enum rdma_link_layer link = rdma_port_get_link_layer(ibdev,
+ah_attr->port_num);
+
+   err = ib_query_port(ibdev, ah_attr->port_num, _attr);
+   if (err)
+   return -EINVAL;
+   if (ah_attr->port_num < 1 ||
+   ah_attr->port_num > ibdev->phys_port_cnt)
+   return -EINVAL;
+   if (ah_attr->static_rate != IB_RATE_PORT_CURRENT &&
+   ib_rate_to_mbps(ah_attr->static_rate) < 0)
+   return -EINVAL;
+   if ((ah_attr->ah_flags & IB_AH_GRH) &&
+   ah_attr->grh.sgid_index >= port_attr.gid_tbl_len)
+   return -EINVAL;
+   if (link != IB_LINK_LAYER_ETHERNET) {
+   if (ah_attr->dlid == 0)
+   return -EINVAL;
+   if (ah_attr->dlid >= RVT_MULTICAST_LID_BASE &&
+   ah_attr->dlid != RVT_PERMISSIVE_LID &&
+   !(ah_attr->ah_flags & IB_AH_GRH))
+   return -EINVAL;
+   }
+   if (rdi->driver_f.check_ah(ibdev, ah_attr))
+   return -EINVAL;
+   return 0;
+}
+EXPORT_SYMBOL(rvt_check_ah);
 
 /**
  * rvt_create_ah - create an address handle
@@ -57,20 +99,68 @@
 struct ib_ah *rvt_create_ah(struct ib_pd *pd,
struct ib_ah_attr *ah_attr)
 {
-   return ERR_PTR(-EOPNOTSUPP);
+   struct rvt_ah *ah;
+   struct rvt_dev_info *dev = ib_to_rvt(pd->device);
+   unsigned long flags;
+
+   if (rvt_check_ah(pd->device, ah_attr))
+   return ERR_PTR(-EINVAL);
+
+   ah = kmalloc(sizeof(*ah), GFP_ATOMIC);
+   if (!ah)
+   return ERR_PTR(-ENOMEM);
+
+   spin_lock_irqsave(>n_ahs_lock, flags);
+   if (dev->n_ahs_allocated == dev->dparms.props.max_ah) {
+   spin_unlock(>n_ahs_lock);
+   kfree(ah);
+   return ERR_PTR(-ENOMEM);
+   }
+
+   dev->n_ahs_allocated++;
+   spin_unlock_irqrestore(>n_ahs_lock, flags);
+
+   ah->attr = *ah_attr;
+   atomic_set(>refcount, 0);
+
+   return >ibah;
 }
 
 int rvt_destroy_ah(struct ib_ah *ibah)
 {
-   return -EOPNOTSUPP;
+   struct rvt_dev_info *dev = ib_to_rvt(ibah->device);
+   struct rvt_ah *ah = ibah_to_rvtah(ibah);
+   unsigned long flags;
+
+   if (atomic_read(>refcount) != 0)
+   return -EBUSY;
+
+   spin_lock_irqsave(>n_ahs_lock, flags);
+   dev->n_ahs_allocated--;
+   spin_unlock_irqrestore(>n_ahs_lock, flags);
+
+   kfree(ah);
+
+   return 0;
 }
 
 int rvt_modify_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
 {
-   return -EOPNOTSUPP;
+   struct rvt_ah *ah = ibah_to_rvtah(ibah);
+
+   if (rvt_check_ah(ibah->device, ah_attr))
+   return -EINVAL;
+
+   ah->attr = *ah_attr;
+
+   return 0;
 }
 
 int rvt_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
 {
-   return -EOPNOTSUPP;
+   struct rvt_ah *ah = ibah_to_rvtah(ibah);
+
+   *ah_attr = ah->attr;
+
+   return 0;
 }
diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index e92af9c..7dab0ca 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -221,7 +221,8 @@ int rvt_register_device(struct rvt_dev_info *rdi)
 
if ((!rdi->driver_f.port_callback) ||
(!rdi->driver_f.get_card_name) ||
-   (!rdi->driver_f.

[PATCH v2 27/36] IB/rdmavt: Do not use rvt prints which rely on driver too early

[Dennis Dalessandro]

Trying to print debug and error messages with the rdmavt helpers will not
work out so well if the drivers have not provided the get_card and get
pci functions. Use the normal pr_error instead until we can check this.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/vt.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 516c810..e92af9c 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -266,7 +266,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
/* Mem Region */
ret = rvt_driver_mr_init(rdi);
if (ret) {
-   rvt_pr_err(rdi, "Error in driver MR init.\n");
+   pr_err("Error in driver MR init.\n");
goto bail_no_mr;
}
 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 28/36] IB/rdmavt: Add common LID defines to rdmavt

[Dennis Dalessandro]

From: Kamal Heib <kam...@mellanox.com>

Original patch is from Kamal Heib <kam...@mellanox.com>. It has
been split into separate patches.

This patch adds RVT_PERMISSIVE_LID and RVT_MULTICAST_LID_BASE
to rdmavt.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Kamal Heib <kam...@mellanox.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 include/rdma/rdma_vt.h |3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 9a47957..dbb45bc 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -55,6 +55,9 @@
 
 #include "ib_verbs.h"
 
+#define RVT_MULTICAST_LID_BASE 0xC000
+#define RVT_PERMISSIVE_LID 0x
+
 /*
  * For some of the IBTA objects there will likely be some
  * initializations required. We need flags to determine whether it is OK

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 03/36] IB/rdmavt: Add protection domain to rdmavt.

[Dennis Dalessandro]

Add datastructure for and allocation/deallocation of protection domains for
RDMAVT.

Reviewed-by: Ira Weiny <ira.we...@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---

Changes since v1:
update comment about protection domain limit

 drivers/infiniband/sw/rdmavt/Makefile |3 +
 drivers/infiniband/sw/rdmavt/pd.c |  103 +
 drivers/infiniband/sw/rdmavt/pd.h |   58 +++
 drivers/infiniband/sw/rdmavt/vt.c |   12 
 drivers/infiniband/sw/rdmavt/vt.h |1 
 include/rdma/rdma_vt.h|   34 ++-
 6 files changed, 206 insertions(+), 5 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/pd.c
 create mode 100644 drivers/infiniband/sw/rdmavt/pd.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 134d2d0..c6751bb 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,5 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o dma.o
+rdmavt-y := vt.o dma.o pd.o
+
diff --git a/drivers/infiniband/sw/rdmavt/pd.c 
b/drivers/infiniband/sw/rdmavt/pd.c
new file mode 100644
index 000..e6e153f
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/pd.c
@@ -0,0 +1,103 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include 
+#include "pd.h"
+
+struct ib_pd *rvt_alloc_pd(struct ib_device *ibdev,
+  struct ib_ucontext *context,
+  struct ib_udata *udata)
+{
+   struct rvt_dev_info *dev = ib_to_rvt(ibdev);
+   struct rvt_pd *pd;
+   struct ib_pd *ret;
+
+   pd = kmalloc(sizeof(*pd), GFP_KERNEL);
+   if (!pd) {
+   ret = ERR_PTR(-ENOMEM);
+   goto bail;
+   }
+   /*
+* While we could continue allocating protecetion domains, being
+* constrained only by system resources. The IBTA spec defines that
+* there is a max_pd limit that can be set and we need to check for
+* that.
+*/
+
+   spin_lock(>n_pds_lock);
+   if (dev->n_pds_allocated == dev->dparms.max_pds) {
+   spin_unlock(>n_pds_lock);
+   kfree(pd);
+   ret = ERR_PTR(-ENOMEM);
+   goto bail;
+   }
+
+   dev->n_pds_allocated++;
+   spin_unlock(>n_pds_lock);
+
+   /* ib_alloc_pd() will initialize pd->ibpd. */
+   pd->user = udata ? 1 : 0;
+
+   ret = >ibpd;
+
+bail:
+   return ret;
+}
+
+int rvt_dealloc_pd(struct ib_pd *ibpd)
+{
+   struct rvt_pd *pd = ibpd_to_rvtpd(ibpd);
+   struct rvt_dev_info *dev = ib_to_rvt(ibpd->dev

[PATCH v2 04/36] IB/rdmavt: Add ib core device attributes to rvt driver params list

[Dennis Dalessandro]

Instead of trying to handle each parameter separately, add ib_device_attr
to rvt_driver_params. This means drivers will fill this in and pass to the
rvt registration function.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
remove comment on alternative design for private data

 drivers/infiniband/sw/rdmavt/pd.c |2 +
 include/rdma/rdma_vt.h|   57 -
 2 files changed, 56 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/pd.c 
b/drivers/infiniband/sw/rdmavt/pd.c
index e6e153f..f8dba88 100644
--- a/drivers/infiniband/sw/rdmavt/pd.c
+++ b/drivers/infiniband/sw/rdmavt/pd.c
@@ -69,7 +69,7 @@ struct ib_pd *rvt_alloc_pd(struct ib_device *ibdev,
 */
 
spin_lock(>n_pds_lock);
-   if (dev->n_pds_allocated == dev->dparms.max_pds) {
+   if (dev->n_pds_allocated == dev->dparms.props.max_pd) {
spin_unlock(>n_pds_lock);
kfree(pd);
ret = ERR_PTR(-ENOMEM);
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 6bf5fd4..2990e03 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -59,7 +59,45 @@
  * Things that are driver specific, module parameters in hfi1 and qib
  */
 struct rvt_driver_params {
-   int max_pds;
+   /*
+* driver required fields:
+*  node_guid
+*  phys_port_cnt
+*  dma_device
+*  owner
+* driver optional fields (rvt will provide generic value if blank):
+*  name
+*  node_desc
+* rvt fields, driver value ignored:
+*  uverbs_abi_ver
+*  node_type
+*  num_comp_vectors
+*  uverbs_cmd_mask
+*/
+   struct ib_device_attr props;
+
+   /*
+* Drivers will need to support a number of notifications to rvt in
+* accordance with certain events. This structure should contain a mask
+* of the supported events. Such events that the rvt may need to know
+* about include:
+* port errors
+* port active
+* lid change
+* sm change
+* client reregister
+* pkey change
+*
+* There may also be other events that the rvt layers needs to know
+* about this is not an exhaustive list. Some events though rvt does not
+* need to rely on the driver for such as completion queue error.
+*/
+int rvt_signal_supported;
+
+   /*
+* Anything driver specific that is not covered by props
+* For instance special module parameters. Goes here.
+*/
 };
 
 /* Protection domain */
@@ -69,10 +107,25 @@ struct rvt_pd {
 };
 
 struct rvt_dev_info {
+   /*
+* Prior to calling for registration the driver will be responsible for
+* allocating space for this structure.
+*
+* The driver will also be responsible for filling in certain members of
+* dparms.props
+*/
+
struct ib_device ibdev;
 
-   /* Driver specific */
+   /* Driver specific properties */
struct rvt_driver_params dparms;
+
+   /*
+* The work to create port files in /sys/class Infiniband is different
+* depending on the driver. This should not be extracted away and
+* instead drivers are responsible for setting the correct callback for
+* this.
+*/
int (*port_callback)(struct ib_device *, u8, struct kobject *);
 
/* Internal use */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 07/36] IB/rdmavt: Add query and modify port stubs

[Dennis Dalessandro]

This patch adds the query and modify port stubs. The query will mostly
entail the driver returning everything in the ib_port_attr which will get
handed back to the verbs layer. The modify will need some API helpers in
the driver. The send_trap and post_mad_send are still issues to address.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Fix comment style for rvt_query_port
Change all stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/vt.c |   48 +
 1 files changed, 48 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index db14646..5ac241c 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -91,6 +91,52 @@ static int rvt_modify_device(struct ib_device *device,
return -EOPNOTSUPP;
 }
 
+/**
+ * rvt_query_port: Passes the query port call to the driver
+ * @ibdev: Verbs IB dev
+ * @port: port number
+ * @props: structure to hold returned properties
+ *
+ * Returns 0 on success
+ */
+static int rvt_query_port(struct ib_device *ibdev, u8 port,
+ struct ib_port_attr *props)
+{
+   /*
+* VT-DRIVER-API: query_port_state()
+* driver returns pretty much everything in ib_port_attr
+*/
+   return -EOPNOTSUPP;
+}
+
+/**
+ * rvt_modify_port
+ * @ibdev: Verbs IB dev
+ * @port: Port number
+ * @port_modify_mask: How to change the port
+ * @props: Structure to fill in
+ *
+ * Returns 0 on success
+ */
+static int rvt_modify_port(struct ib_device *ibdev, u8 port,
+  int port_modify_mask, struct ib_port_modify *props)
+{
+   /*
+* VT-DRIVER-API: set_link_state()
+* driver will set the link state using the IB enumeration
+*
+* VT-DRIVER-API: clear_qkey_violations()
+* clears driver private qkey counter
+*
+* VT-DRIVER-API: get_lid()
+* driver needs to return the LID
+*
+* TBD: send_trap() and post_mad_send() need examined to see where they
+* fit in.
+*/
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -106,6 +152,8 @@ int rvt_register_device(struct rvt_dev_info *rdi)
/* Dev Ops */
CHECK_DRIVER_OVERRIDE(rdi, query_device);
CHECK_DRIVER_OVERRIDE(rdi, modify_device);
+   CHECK_DRIVER_OVERRIDE(rdi, query_port);
+   CHECK_DRIVER_OVERRIDE(rdi, modify_port);
 
/* DMA Operations */
rdi->ibdev.dma_ops =

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 06/36] IB/rdmavt: Add query and modify device stubs

[Dennis Dalessandro]

Adds the stubs which will handle the query and modify device functions. At
this time the only intention is to support changing the node desc and the
guid via these calls.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/vt.c |   31 +++
 1 files changed, 31 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index 8bd25c3..db14646 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -64,6 +64,33 @@ static void rvt_cleanup(void)
 }
 module_exit(rvt_cleanup);
 
+static int rvt_query_device(struct ib_device *ibdev,
+   struct ib_device_attr *props,
+   struct ib_udata *uhw)
+{
+   /*
+* Return rvt_dev_info.props contents
+*/
+   return -EOPNOTSUPP;
+}
+
+static int rvt_modify_device(struct ib_device *device,
+int device_modify_mask,
+struct ib_device_modify *device_modify)
+{
+   /*
+* Change dev props. Planned support is for node desc change and sys
+* guid change only. This matches hfi1 and qib behavior. Other drivers
+* that support existing modifications will need to add their support.
+*/
+
+   /*
+* VT-DRIVER-API: node_desc_change()
+* VT-DRIVER-API: sys_guid_change()
+*/
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -76,6 +103,10 @@ int rvt_register_device(struct rvt_dev_info *rdi)
if (!rdi)
return -EINVAL;
 
+   /* Dev Ops */
+   CHECK_DRIVER_OVERRIDE(rdi, query_device);
+   CHECK_DRIVER_OVERRIDE(rdi, modify_device);
+
/* DMA Operations */
rdi->ibdev.dma_ops =
rdi->ibdev.dma_ops ? : _default_dma_mapping_ops;

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 15/36] IB/rdmavt: Add multicast stubs

[Dennis Dalessandro]

Adds the function stubs for attach and detach multicast.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 +
 drivers/infiniband/sw/rdmavt/mcast.c  |   58 +
 drivers/infiniband/sw/rdmavt/mcast.h  |   56 
 drivers/infiniband/sw/rdmavt/vt.c |4 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 +
 5 files changed, 120 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/mcast.c
 create mode 100644 drivers/infiniband/sw/rdmavt/mcast.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 204be84..d2af114 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mr.o pd.o qp.o srq.o
+rdmavt-y := vt.o ah.o dma.o mcast.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/mcast.c 
b/drivers/infiniband/sw/rdmavt/mcast.c
new file mode 100644
index 000..5a78dc7
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mcast.c
@@ -0,0 +1,58 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "mcast.h"
+
+int rvt_attach_mcast(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_detach_mcast(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
+{
+   return -EOPNOTSUPP;
+}
diff --git a/drivers/infiniband/sw/rdmavt/mcast.h 
b/drivers/infiniband/sw/rdmavt/mcast.h
new file mode 100644
index 000..21647c3
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mcast.h
@@ -0,0 +1,56 @@
+#ifndef DEF_RVTMCAST_H
+#define DEF_RVTMCAST_H
+
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of sourc

[PATCH v2 16/36] IB/rdmavt: Add process MAD stub

[Dennis Dalessandro]

This adds the stub for process mad. More study is needed to determine the
final MAD interaction between the driver and rvt.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
 drivers/infiniband/sw/rdmavt/Makefile |2 -
 drivers/infiniband/sw/rdmavt/mad.c|   85 +
 drivers/infiniband/sw/rdmavt/mad.h|   59 +++
 3 files changed, 145 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/mad.c
 create mode 100644 drivers/infiniband/sw/rdmavt/mad.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index d2af114..fe65410 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mcast.o mr.o pd.o qp.o srq.o
+rdmavt-y := vt.o ah.o dma.o mad.o mcast.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/mad.c 
b/drivers/infiniband/sw/rdmavt/mad.c
new file mode 100644
index 000..eef7029
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mad.c
@@ -0,0 +1,85 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "mad.h"
+
+/**
+ * rvt_process_mad - process an incoming MAD packet
+ * @ibdev: the infiniband device this packet came in on
+ * @mad_flags: MAD flags
+ * @port: the port number this packet came in on
+ * @in_wc: the work completion entry for this packet
+ * @in_grh: the global route header for this packet
+ * @in_mad: the incoming MAD
+ * @out_mad: any outgoing MAD reply
+ *
+ * Returns IB_MAD_RESULT_SUCCESS if this is a MAD that we are not
+ * interested in processing.
+ *
+ * Note that the verbs framework has already done the MAD sanity checks,
+ * and hop count/pointer updating for IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE
+ * MADs.
+ *
+ * This is called by the ib_mad module.
+ */
+int rvt_process_mad(struct ib_device *ibdev, int mad_flags, u8 port,
+   const struct ib_wc *in_wc, const struct ib_grh *in_grh,
+   const struct ib_mad_hdr *in, size_t in_mad_size,
+   struct ib_mad_hdr *out, size_t *out_mad_size,
+   u16 *out_mad_pkey_index)
+{
+   /*
+* Drivers will need to provide a number of things. For exmaple counters
+* will need to be maintained by the driver but shoud live in the rvt
+* structure. More study will be needed to finalize the interface
+* between drivers and rvt for mad packets.
+*
+*VT-DRIVER-API: 
+*
+*/
+   return IB_MAD_RESULT_FAILURE;
+}
diff --git a/drivers/infiniband/sw

[PATCH v2 17/36] IB/rdmavt: Add mmap stub

[Dennis Dalessandro]

Adds the stub for the mmap verbs call.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 +
 drivers/infiniband/sw/rdmavt/mmap.c   |   60 +
 drivers/infiniband/sw/rdmavt/mmap.h   |   55 ++
 drivers/infiniband/sw/rdmavt/vt.c |1 +
 drivers/infiniband/sw/rdmavt/vt.h |1 +
 5 files changed, 118 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/mmap.c
 create mode 100644 drivers/infiniband/sw/rdmavt/mmap.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index fe65410..6f530d1 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mad.o mcast.o mr.o pd.o qp.o srq.o
+rdmavt-y := vt.o ah.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/mmap.c 
b/drivers/infiniband/sw/rdmavt/mmap.c
new file mode 100644
index 000..d09f3a0
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mmap.c
@@ -0,0 +1,60 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include 
+#include "mmap.h"
+
+/**
+ * rvt_mmap - create a new mmap region
+ * @context: the IB user context of the process making the mmap() call
+ * @vma: the VMA to be initialized
+ * Return zero if the mmap is OK. Otherwise, return an errno.
+ */
+int rvt_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
+{
+   return -EOPNOTSUPP;
+}
diff --git a/drivers/infiniband/sw/rdmavt/mmap.h 
b/drivers/infiniband/sw/rdmavt/mmap.h
new file mode 100644
index 000..94f6377
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/mmap.h
@@ -0,0 +1,55 @@
+#ifndef DEF_RDMAVTMMAP_H
+#define DEF_RDMAVTMMAP_H
+
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or w

[PATCH v2 14/36] IB/rdmavt: Add SRQ stubs

[Dennis Dalessandro]

Adds the stubs for create, modify, query, and destory for shared
request queues.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/Makefile |2 -
 drivers/infiniband/sw/rdmavt/srq.c|   86 +
 drivers/infiniband/sw/rdmavt/srq.h|   61 +++
 drivers/infiniband/sw/rdmavt/vt.c |6 ++
 drivers/infiniband/sw/rdmavt/vt.h |1 
 5 files changed, 155 insertions(+), 1 deletions(-)
 create mode 100644 drivers/infiniband/sw/rdmavt/srq.c
 create mode 100644 drivers/infiniband/sw/rdmavt/srq.h

diff --git a/drivers/infiniband/sw/rdmavt/Makefile 
b/drivers/infiniband/sw/rdmavt/Makefile
index 084ee6a..204be84 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,4 +7,4 @@
 #
 obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
 
-rdmavt-y := vt.o ah.o dma.o mr.o pd.o qp.o
+rdmavt-y := vt.o ah.o dma.o mr.o pd.o qp.o srq.o
diff --git a/drivers/infiniband/sw/rdmavt/srq.c 
b/drivers/infiniband/sw/rdmavt/srq.c
new file mode 100644
index 000..bbb623a
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/srq.c
@@ -0,0 +1,86 @@
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *  - Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ *  - Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in
+ *the documentation and/or other materials provided with the
+ *distribution.
+ *  - Neither the name of Intel Corporation nor the names of its
+ *contributors may be used to endorse or promote products derived
+ *from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#include "srq.h"
+
+/**
+ * rvt_create_srq - create a shared receive queue
+ * @ibpd: the protection domain of the SRQ to create
+ * @srq_init_attr: the attributes of the SRQ
+ * @udata: data from libibverbs when creating a user SRQ
+ */
+struct ib_srq *rvt_create_srq(struct ib_pd *ibpd,
+ struct ib_srq_init_attr *srq_init_attr,
+ struct ib_udata *udata)
+{
+   return ERR_PTR(-EOPNOTSUPP);
+}
+
+/**
+ * rvt_modify_srq - modify a shared receive queue
+ * @ibsrq: the SRQ to modify
+ * @attr: the new attributes of the SRQ
+ * @attr_mask: indicates which attributes to modify
+ * @udata: user data for libibverbs.so
+ */
+int rvt_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,
+  enum ib_srq_attr_mask attr_mask,
+  struct ib_udata *udata)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr)
+{
+   return -EOPNOTSUPP;
+}
+
+int rvt_destroy_srq(struct ib_srq *ibsrq)
+{
+   return -EOPNOTSUPP;
+}
+
diff --git a/drivers/infiniband/sw/rdmavt/srq.h 
b/drivers/infiniband/sw/rdmavt/srq.h
new file mode 100644
index 000..0c3c5a7
--- /dev/null
+++ b/drivers/infiniband/sw/rdmavt/srq.h
@@ -0,0 +1,61 @@
+#ifndef DEF_RVTSRQ_H
+#define DEF_RVTSRQ_H
+
+/*
+ * Copyright(c) 2015 Intel Corporation.
+ *
+ * This file is p

[PATCH v2 18/36] IB/rdmavt: Add get port immutable stub

[Dennis Dalessandro]

This adds the get port immutable verbs call.

Reviewed-by: Mike Marciniszyn <mike.marcinis...@intel.com>
Reviewed-by: Ira Weiny <ira.we...@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessan...@intel.com>
---
Changes since v1:
Change stubs to return EOPNOTSUPP

 drivers/infiniband/sw/rdmavt/vt.c |7 +++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c 
b/drivers/infiniband/sw/rdmavt/vt.c
index a889ec4..99fbe37 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -198,6 +198,12 @@ static int rvt_dealloc_ucontext(struct ib_ucontext 
*context)
return -EOPNOTSUPP;
 }
 
+static int rvt_get_port_immutable(struct ib_device *ibdev, u8 port_num,
+ struct ib_port_immutable *immutable)
+{
+   return -EOPNOTSUPP;
+}
+
 /*
  * Check driver override. If driver passes a value use it, otherwise we use our
  * own value.
@@ -219,6 +225,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
CHECK_DRIVER_OVERRIDE(rdi, query_gid);
CHECK_DRIVER_OVERRIDE(rdi, alloc_ucontext);
CHECK_DRIVER_OVERRIDE(rdi, dealloc_ucontext);
+   CHECK_DRIVER_OVERRIDE(rdi, get_port_immutable);
 
/* Queue Pairs */
CHECK_DRIVER_OVERRIDE(rdi, create_qp);

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html