date:20100401

Fork safe clarification

2010-04-01 Thread Matthew Small

I am trying to understand the behavior of the libibverbs after it has been
set into fork safe mode via a successful call to ibv_fork_init() or setting
the environmental variable IBV_FORK_SAFE.  For my purposes I would like to
know the following :

Are PDs, QPs and CQs created before a fork shared by the parent and child
after fork() has returned (ie. both can submit WRs, poll CQ, etc.)?


What about MRs registered before the fork?  Even though the child doesn't
have access to the parent's memory, can he sill submit WRs on a QP with an
MR created before the fork?


What if the MR pages in the above scenario are accessible in both parent and
child (shared memory)?  Are there complications with registering shared
memory?


In general, are pointers returned by libibverbs pointer to user/process
address space (as ibv_mr pointers must be) or kernel space (eg.  if an
unrelated process had another process's QP pointer, lkey, and a virtual
address could it post (almost certainly unsafely) a WR to the other
process's QP?


Sorry if the questions seem progressively more goofy and thanks in advance
for any clarification.

-Matt
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 26/26] rdma/ucm: allow user space to specify AF_IB when joining multicast

2010-04-01 Thread Sean Hefty

Allow user space applications to join multicast groups using MGIDs
directly.  MGIDs may be passed using AF_IB addresses.  Since the
current multicast join command only supports addresses as large as
sockaddr_in6, define a new structure for joining addresses specified
using sockaddr_ib.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c  |8 --
 drivers/infiniband/core/ucma.c |   55 
 include/rdma/rdma_user_cm.h|   12 -
 3 files changed, 60 insertions(+), 15 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index e0bdb80..4adad37 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -2928,14 +2928,16 @@ static void cma_set_mgid(struct rdma_id_private 
*id_priv,
 0xFF10A01B)) {
/* IPv6 address is an SA assigned MGID. */
memcpy(mgid, sin6-sin6_addr, sizeof *mgid);
+   } else if (addr-sa_family == AF_IB) {
+   memcpy(mgid, ((struct sockaddr_ib *) addr)-sib_addr, sizeof 
*mgid);
} else if ((addr-sa_family == AF_INET6)) {
ipv6_ib_mc_map(sin6-sin6_addr, dev_addr-broadcast, mc_map);
-   if (id_priv-id.ps == RDMA_PS_UDP)
+   if (id_priv-id.ps == RDMA_PS_UDP || id_priv-id.ps == 
RDMA_PS_IB)
mc_map[7] = 0x01;   /* Use RDMA CM signature */
*mgid = *(union ib_gid *) (mc_map + 4);
} else {
ip_ib_mc_map(sin-sin_addr.s_addr, dev_addr-broadcast, mc_map);
-   if (id_priv-id.ps == RDMA_PS_UDP)
+   if (id_priv-id.ps == RDMA_PS_UDP || id_priv-id.ps == 
RDMA_PS_IB)
mc_map[7] = 0x01;   /* Use RDMA CM signature */
*mgid = *(union ib_gid *) (mc_map + 4);
}
@@ -2956,7 +2958,7 @@ static int cma_join_ib_multicast(struct rdma_id_private 
*id_priv,
return ret;
 
cma_set_mgid(id_priv, (struct sockaddr *) mc-addr, rec.mgid);
-   if (id_priv-id.ps == RDMA_PS_UDP)
+   if (id_priv-id.ps == RDMA_PS_UDP || id_priv-id.ps == RDMA_PS_IB)
rec.qkey = cpu_to_be32(RDMA_UDP_QKEY);
rdma_addr_get_sgid(dev_addr, rec.port_gid);
rec.pkey = cpu_to_be16(ib_addr_get_pkey(dev_addr));
diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index f735150..8cf0d70 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -1161,23 +1161,23 @@ static ssize_t ucma_notify(struct ucma_file *file, 
const char __user *inbuf,
return ret;
 }
 
-static ssize_t ucma_join_ip_multicast(struct ucma_file *file,
- const char __user *inbuf,
- int in_len, int out_len)
+static ssize_t ucma_process_join(struct ucma_file *file,
+struct rdma_ucm_join_mcast *cmd,  int out_len)
 {
-   struct rdma_ucm_join_ip_mcast cmd;
struct rdma_ucm_create_id_resp resp;
struct ucma_context *ctx;
struct ucma_multicast *mc;
+   struct sockaddr *addr;
int ret;
 
if (out_len  sizeof(resp))
return -ENOSPC;
 
-   if (copy_from_user(cmd, inbuf, sizeof(cmd)))
-   return -EFAULT;
+   addr = (struct sockaddr *) cmd-addr;
+   if (cmd-reserved || !cmd-addr_size || (cmd-addr_size != 
rdma_addr_size(addr)))
+   return -EINVAL;
 
-   ctx = ucma_get_ctx(file, cmd.id);
+   ctx = ucma_get_ctx(file, cmd-id);
if (IS_ERR(ctx))
return PTR_ERR(ctx);
 
@@ -1188,14 +1188,14 @@ static ssize_t ucma_join_ip_multicast(struct ucma_file 
*file,
goto err1;
}
 
-   mc-uid = cmd.uid;
-   memcpy(mc-addr, cmd.addr, sizeof cmd.addr);
+   mc-uid = cmd-uid;
+   memcpy(mc-addr, addr, cmd-addr_size);
ret = rdma_join_multicast(ctx-cm_id, (struct sockaddr *) mc-addr, 
mc);
if (ret)
goto err2;
 
resp.id = mc-id;
-   if (copy_to_user((void __user *)(unsigned long)cmd.response,
+   if (copy_to_user((void __user *)(unsigned long) cmd-response,
 resp, sizeof(resp))) {
ret = -EFAULT;
goto err3;
@@ -1220,6 +1220,38 @@ err1:
return ret;
 }
 
+static ssize_t ucma_join_ip_multicast(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+   struct rdma_ucm_join_ip_mcast cmd;
+   struct rdma_ucm_join_mcast join_cmd;
+
+   if (copy_from_user(cmd, inbuf, sizeof(cmd)))
+   return -EFAULT;
+
+   join_cmd.response = cmd.response;
+   join_cmd.uid = cmd.uid;
+   join_cmd.id = cmd.id;
+   join_cmd.addr_size = rdma_addr_size((struct sockaddr *) cmd.addr);
+

[PATCH 25/26] rdma/ucm: allow user space to pass AF_IB into resolve

2010-04-01 Thread Sean Hefty

Allow user space applications to call resolve_addr using
AF_IB.  To support sockaddr_ib, we need to define a new
structure capable of handling the larger address size.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/ucma.c |   30 +-
 include/rdma/rdma_user_cm.h|   13 -
 2 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index aa50080..f735150 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -582,6 +582,33 @@ static ssize_t ucma_resolve_ip(struct ucma_file *file,
return ret;
 }
 
+static ssize_t ucma_resolve_addr(struct ucma_file *file,
+const char __user *inbuf,
+int in_len, int out_len)
+{
+   struct rdma_ucm_resolve_addr cmd;
+   struct sockaddr *src, *dst;
+   struct ucma_context *ctx;
+   int ret;
+
+   if (copy_from_user(cmd, inbuf, sizeof(cmd)))
+   return -EFAULT;
+
+   src = (struct sockaddr *) cmd.src_addr;
+   dst = (struct sockaddr *) cmd.dst_addr;
+   if (cmd.reserved || (cmd.src_size  (cmd.src_size != 
rdma_addr_size(src))) ||
+   !cmd.dst_size || (cmd.dst_size != rdma_addr_size(dst)))
+   return -EINVAL;
+
+   ctx = ucma_get_ctx(file, cmd.id);
+   if (IS_ERR(ctx))
+   return PTR_ERR(ctx);
+
+   ret = rdma_resolve_addr(ctx-cm_id, src, dst, cmd.timeout_ms);
+   ucma_put_ctx(ctx);
+   return ret;
+}
+
 static ssize_t ucma_resolve_route(struct ucma_file *file,
  const char __user *inbuf,
  int in_len, int out_len)
@@ -1355,7 +1382,8 @@ static ssize_t (*ucma_cmd_table[])(struct ucma_file *file,
[RDMA_USER_CM_CMD_LEAVE_MCAST]   = ucma_leave_multicast,
[RDMA_USER_CM_CMD_MIGRATE_ID]= ucma_migrate_id,
[RDMA_USER_CM_CMD_QUERY] = ucma_query,
-   [RDMA_USER_CM_CMD_BIND]  = ucma_bind
+   [RDMA_USER_CM_CMD_BIND]  = ucma_bind,
+   [RDMA_USER_CM_CMD_RESOLVE_ADDR]  = ucma_resolve_addr
 };
 
 static ssize_t ucma_write(struct file *filp, const char __user *buf,
diff --git a/include/rdma/rdma_user_cm.h b/include/rdma/rdma_user_cm.h
index 375f2a9..d488184 100644
--- a/include/rdma/rdma_user_cm.h
+++ b/include/rdma/rdma_user_cm.h
@@ -63,7 +63,8 @@ enum {
RDMA_USER_CM_CMD_LEAVE_MCAST,
RDMA_USER_CM_CMD_MIGRATE_ID,
RDMA_USER_CM_CMD_QUERY,
-   RDMA_USER_CM_CMD_BIND
+   RDMA_USER_CM_CMD_BIND,
+   RDMA_USER_CM_CMD_RESOLVE_ADDR
 };
 
 /*
@@ -117,6 +118,16 @@ struct rdma_ucm_resolve_ip {
__u32 timeout_ms;
 };
 
+struct rdma_ucm_resolve_addr {
+   __u32 id;
+   __u32 timeout_ms;
+   __u16 src_size;
+   __u16 dst_size;
+   __u32 reserved;
+   struct sockaddr_storage src_addr;
+   struct sockaddr_storage dst_addr;
+};
+
 struct rdma_ucm_resolve_route {
__u32 id;
__u32 timeout_ms;



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 24/26] rdma/ucm: allow user space to bind to AF_IB

2010-04-01 Thread Sean Hefty

Support user space binding to addresses using AF_IB.  Since
sockaddr_ib is larger than sockaddr_in6, we need to define
a larger structure when binding using AF_IB.  This time we
use sockaddr_storage to cover future cases.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/ucma.c |   27 ++-
 include/rdma/rdma_user_cm.h|   10 +-
 2 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index 6456502..aa50080 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -536,6 +536,30 @@ static ssize_t ucma_bind_ip(struct ucma_file *file, const 
char __user *inbuf,
return ret;
 }
 
+static ssize_t ucma_bind(struct ucma_file *file, const char __user *inbuf,
+int in_len, int out_len)
+{
+   struct rdma_ucm_bind cmd;
+   struct sockaddr *addr;
+   struct ucma_context *ctx;
+   int ret;
+
+   if (copy_from_user(cmd, inbuf, sizeof(cmd)))
+   return -EFAULT;
+
+   addr = (struct sockaddr *) cmd.addr;
+   if (cmd.reserved || !cmd.addr_size || (cmd.addr_size != 
rdma_addr_size(addr)))
+   return -EINVAL;
+
+   ctx = ucma_get_ctx(file, cmd.id);
+   if (IS_ERR(ctx))
+   return PTR_ERR(ctx);
+
+   ret = rdma_bind_addr(ctx-cm_id, addr);
+   ucma_put_ctx(ctx);
+   return ret;
+}
+
 static ssize_t ucma_resolve_ip(struct ucma_file *file,
   const char __user *inbuf,
   int in_len, int out_len)
@@ -1330,7 +1354,8 @@ static ssize_t (*ucma_cmd_table[])(struct ucma_file *file,
[RDMA_USER_CM_CMD_JOIN_IP_MCAST] = ucma_join_ip_multicast,
[RDMA_USER_CM_CMD_LEAVE_MCAST]   = ucma_leave_multicast,
[RDMA_USER_CM_CMD_MIGRATE_ID]= ucma_migrate_id,
-   [RDMA_USER_CM_CMD_QUERY] = ucma_query
+   [RDMA_USER_CM_CMD_QUERY] = ucma_query,
+   [RDMA_USER_CM_CMD_BIND]  = ucma_bind
 };
 
 static ssize_t ucma_write(struct file *filp, const char __user *buf,
diff --git a/include/rdma/rdma_user_cm.h b/include/rdma/rdma_user_cm.h
index c1e0ac9..375f2a9 100644
--- a/include/rdma/rdma_user_cm.h
+++ b/include/rdma/rdma_user_cm.h
@@ -62,7 +62,8 @@ enum {
RDMA_USER_CM_CMD_JOIN_IP_MCAST,
RDMA_USER_CM_CMD_LEAVE_MCAST,
RDMA_USER_CM_CMD_MIGRATE_ID,
-   RDMA_USER_CM_CMD_QUERY
+   RDMA_USER_CM_CMD_QUERY,
+   RDMA_USER_CM_CMD_BIND
 };
 
 /*
@@ -102,6 +103,13 @@ struct rdma_ucm_bind_ip {
__u32 id;
 };
 
+struct rdma_ucm_bind {
+   __u32 id;
+   __u16 addr_size;
+   __u16 reserved;
+   struct sockaddr_storage addr;
+};
+
 struct rdma_ucm_resolve_ip {
struct sockaddr_in6 src_addr;
struct sockaddr_in6 dst_addr;



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 23/26] rdma/ucm: name changes to indicate only IP addresses supported

2010-04-01 Thread Sean Hefty

Several commands into the RDMA CM from user space are
restricted to supporting addresses which fit into a sockaddr_in6
structure: bind address, resolve address, and join multicast.

With the addition of AF_IB, we need to support addresses
which are larger than sockaddr_in6.  This will be done by
adding new commands that exchange address information using
sockaddr_storage.  However, to support existing applications,
we maintain the current commands and structures, but rename
them to indicate that they only support IPv4 and v6 addresses.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/ucma.c |   60 
 include/rdma/rdma_user_cm.h|   12 
 2 files changed, 36 insertions(+), 36 deletions(-)

diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index 82eb1ac..6456502 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -517,10 +517,10 @@ static ssize_t ucma_destroy_id(struct ucma_file *file, 
const char __user *inbuf,
return ret;
 }
 
-static ssize_t ucma_bind_addr(struct ucma_file *file, const char __user *inbuf,
+static ssize_t ucma_bind_ip(struct ucma_file *file, const char __user *inbuf,
  int in_len, int out_len)
 {
-   struct rdma_ucm_bind_addr cmd;
+   struct rdma_ucm_bind_ip cmd;
struct ucma_context *ctx;
int ret;
 
@@ -536,11 +536,11 @@ static ssize_t ucma_bind_addr(struct ucma_file *file, 
const char __user *inbuf,
return ret;
 }
 
-static ssize_t ucma_resolve_addr(struct ucma_file *file,
-const char __user *inbuf,
-int in_len, int out_len)
+static ssize_t ucma_resolve_ip(struct ucma_file *file,
+  const char __user *inbuf,
+  int in_len, int out_len)
 {
-   struct rdma_ucm_resolve_addr cmd;
+   struct rdma_ucm_resolve_ip cmd;
struct ucma_context *ctx;
int ret;
 
@@ -1110,11 +1110,11 @@ static ssize_t ucma_notify(struct ucma_file *file, 
const char __user *inbuf,
return ret;
 }
 
-static ssize_t ucma_join_multicast(struct ucma_file *file,
-  const char __user *inbuf,
-  int in_len, int out_len)
+static ssize_t ucma_join_ip_multicast(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
 {
-   struct rdma_ucm_join_mcast cmd;
+   struct rdma_ucm_join_ip_mcast cmd;
struct rdma_ucm_create_id_resp resp;
struct ucma_context *ctx;
struct ucma_multicast *mc;
@@ -1311,26 +1311,26 @@ file_put:
 static ssize_t (*ucma_cmd_table[])(struct ucma_file *file,
   const char __user *inbuf,
   int in_len, int out_len) = {
-   [RDMA_USER_CM_CMD_CREATE_ID]= ucma_create_id,
-   [RDMA_USER_CM_CMD_DESTROY_ID]   = ucma_destroy_id,
-   [RDMA_USER_CM_CMD_BIND_ADDR]= ucma_bind_addr,
-   [RDMA_USER_CM_CMD_RESOLVE_ADDR] = ucma_resolve_addr,
-   [RDMA_USER_CM_CMD_RESOLVE_ROUTE]= ucma_resolve_route,
-   [RDMA_USER_CM_CMD_QUERY_ROUTE]  = ucma_query_route,
-   [RDMA_USER_CM_CMD_CONNECT]  = ucma_connect,
-   [RDMA_USER_CM_CMD_LISTEN]   = ucma_listen,
-   [RDMA_USER_CM_CMD_ACCEPT]   = ucma_accept,
-   [RDMA_USER_CM_CMD_REJECT]   = ucma_reject,
-   [RDMA_USER_CM_CMD_DISCONNECT]   = ucma_disconnect,
-   [RDMA_USER_CM_CMD_INIT_QP_ATTR] = ucma_init_qp_attr,
-   [RDMA_USER_CM_CMD_GET_EVENT]= ucma_get_event,
-   [RDMA_USER_CM_CMD_GET_OPTION]   = NULL,
-   [RDMA_USER_CM_CMD_SET_OPTION]   = ucma_set_option,
-   [RDMA_USER_CM_CMD_NOTIFY]   = ucma_notify,
-   [RDMA_USER_CM_CMD_JOIN_MCAST]   = ucma_join_multicast,
-   [RDMA_USER_CM_CMD_LEAVE_MCAST]  = ucma_leave_multicast,
-   [RDMA_USER_CM_CMD_MIGRATE_ID]   = ucma_migrate_id,
-   [RDMA_USER_CM_CMD_QUERY]= ucma_query
+   [RDMA_USER_CM_CMD_CREATE_ID] = ucma_create_id,
+   [RDMA_USER_CM_CMD_DESTROY_ID]= ucma_destroy_id,
+   [RDMA_USER_CM_CMD_BIND_IP]   = ucma_bind_ip,
+   [RDMA_USER_CM_CMD_RESOLVE_IP]= ucma_resolve_ip,
+   [RDMA_USER_CM_CMD_RESOLVE_ROUTE] = ucma_resolve_route,
+   [RDMA_USER_CM_CMD_QUERY_ROUTE]   = ucma_query_route,
+   [RDMA_USER_CM_CMD_CONNECT]   = ucma_connect,
+   [RDMA_USER_CM_CMD_LISTEN]= ucma_listen,
+   [RDMA_USER_CM_CMD_ACCEPT]= ucma_accept,
+   [RDMA_USER_CM_CMD_REJECT]= ucma_reject,
+   [RDMA_USER_CM_CMD_DISCONNECT]= ucma_disconnect,
+   [RDMA_USER_CM_CMD_INIT_QP_ATTR]  = ucma_init_qp_attr,
+   [RDMA_USER_CM_CMD_GET_EVENT] = ucma_get_event,
+   [RDMA_USER_CM_CMD_GET_OPTION]= NULL,
+   [RDMA_USER_CM_CMD_SET_OPTION]= ucma_set_option,
+

[PATCH 22/26] rdma/ucm: add ability to query GID addresses

2010-04-01 Thread Sean Hefty

Part of address resolution is mapping IP addresses to IB GIDs.
With the changes to support querying larger addresses and more
path records, also provide a way to query IB GIDs after
resolution completes.

This optimizes querying for IB GIDs

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/ucma.c |   50 
 include/rdma/rdma_user_cm.h|3 ++
 2 files changed, 52 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index 5d8d68e..82eb1ac 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -45,6 +45,7 @@
 #include rdma/rdma_cm.h
 #include rdma/rdma_cm_ib.h
 #include rdma/ib_addr.h
+#include rdma/ib.h
 
 MODULE_AUTHOR(Sean Hefty);
 MODULE_DESCRIPTION(RDMA Userspace Connection Manager Access);
@@ -728,6 +729,52 @@ static ssize_t ucma_query_path(struct ucma_context *ctx,
return ret;
 }
 
+static ssize_t ucma_query_gid(struct ucma_context *ctx,
+ void __user *response, int out_len)
+{
+   struct rdma_ucm_query_addr_resp resp;
+   struct sockaddr_ib *addr;
+   int ret = 0;
+
+   if (out_len  sizeof(resp))
+   return -ENOSPC;
+
+   memset(resp, 0, sizeof resp);
+
+   ucma_query_device_addr(ctx-cm_id, resp);
+
+   addr = (struct sockaddr_ib *) resp.src_addr;
+   resp.src_size = sizeof(*addr);
+   if (ctx-cm_id-route.addr.src_addr.ss_family == AF_IB) {
+   memcpy(addr, ctx-cm_id-route.addr.src_addr, resp.src_size);
+   } else {
+   addr-sib_family = AF_IB;
+   addr-sib_pkey = (__force __be16) resp.pkey;
+   rdma_addr_get_sgid(ctx-cm_id-route.addr.dev_addr,
+  (union ib_gid *) addr-sib_addr);
+   addr-sib_sid = rdma_get_service_id(ctx-cm_id, (struct 
sockaddr *)
+   
ctx-cm_id-route.addr.src_addr);
+   }
+
+   addr = (struct sockaddr_ib *) resp.dst_addr;
+   resp.dst_size = sizeof(*addr);
+   if (ctx-cm_id-route.addr.dst_addr.ss_family == AF_IB) {
+   memcpy(addr, ctx-cm_id-route.addr.dst_addr, resp.dst_size);
+   } else {
+   addr-sib_family = AF_IB;
+   addr-sib_pkey = (__force __be16) resp.pkey;
+   rdma_addr_get_dgid(ctx-cm_id-route.addr.dev_addr,
+  (union ib_gid *) addr-sib_addr);
+   addr-sib_sid = rdma_get_service_id(ctx-cm_id, (struct 
sockaddr *)
+   
ctx-cm_id-route.addr.dst_addr);
+   }
+
+   if (copy_to_user(response, resp, sizeof(resp)))
+   ret = -EFAULT;
+
+   return ret;
+}
+
 static ssize_t ucma_query(struct ucma_file *file,
  const char __user *inbuf,
  int in_len, int out_len)
@@ -752,6 +799,9 @@ static ssize_t ucma_query(struct ucma_file *file,
case RDMA_USER_CM_QUERY_PATH:
ret = ucma_query_path(ctx, response, out_len);
break;
+   case RDMA_USER_CM_QUERY_GID:
+   ret = ucma_query_gid(ctx, response, out_len);
+   break;
default:
ret = -ENOSYS;
break;
diff --git a/include/rdma/rdma_user_cm.h b/include/rdma/rdma_user_cm.h
index 7784203..95bb515 100644
--- a/include/rdma/rdma_user_cm.h
+++ b/include/rdma/rdma_user_cm.h
@@ -116,7 +116,8 @@ struct rdma_ucm_resolve_route {
 
 enum {
RDMA_USER_CM_QUERY_ADDR,
-   RDMA_USER_CM_QUERY_PATH
+   RDMA_USER_CM_QUERY_PATH,
+   RDMA_USER_CM_QUERY_GID
 };
 
 struct rdma_ucm_query {



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 21/26] rdma/cm: export cma_get_service_id

2010-04-01 Thread Sean Hefty

Allow the rdma_ucm to query the IB service ID formed or
allocated by the rdma_cm by exporting the cma_get_service_id
functionality.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |   19 ++-
 include/rdma/rdma_cm.h|7 +++
 2 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 8e22119..e0bdb80 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -1255,13 +1255,14 @@ out:
return ret;
 }
 
-static __be64 cma_get_service_id(enum rdma_port_space ps, struct sockaddr 
*addr)
+__be64 rdma_get_service_id(struct rdma_cm_id *id, struct sockaddr *addr)
 {
if (addr-sa_family == AF_IB)
return ((struct sockaddr_ib *) addr)-sib_sid;
 
-   return cpu_to_be64(((u64)ps  16) + be16_to_cpu(cma_port(addr)));
+   return cpu_to_be64(((u64)id-ps  16) + be16_to_cpu(cma_port(addr)));
 }
+EXPORT_SYMBOL(rdma_get_service_id);
 
 static void cma_set_compare_data(enum rdma_port_space ps, struct sockaddr 
*addr,
 struct ib_cm_compare_data *compare)
@@ -1477,7 +1478,7 @@ static int cma_ib_listen(struct rdma_id_private *id_priv)
return PTR_ERR(id_priv-cm_id.ib);
 
addr = (struct sockaddr *) id_priv-id.route.addr.src_addr;
-   svc_id = cma_get_service_id(id_priv-id.ps, addr);
+   svc_id = rdma_get_service_id(id_priv-id, addr);
if (cma_any_addr(addr))
ret = ib_cm_listen(id_priv-cm_id.ib, svc_id, 0, NULL);
else {
@@ -1659,8 +1660,8 @@ static int cma_query_ib_route(struct rdma_id_private 
*id_priv, int timeout_ms,
path_rec.pkey = cpu_to_be16(ib_addr_get_pkey(addr-dev_addr));
path_rec.numb_path = 1;
path_rec.reversible = 1;
-   path_rec.service_id = cma_get_service_id(id_priv-id.ps,
-(struct sockaddr *) 
addr-dst_addr);
+   path_rec.service_id = rdma_get_service_id(id_priv-id,
+ (struct sockaddr *) 
addr-dst_addr);
 
comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID |
IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH |
@@ -2509,8 +2510,8 @@ static int cma_resolve_ib_udp(struct rdma_id_private 
*id_priv,
}
 
req.path = route-path_rec;
-   req.service_id = cma_get_service_id(id_priv-id.ps,
-   (struct sockaddr *) 
route-addr.dst_addr);
+   req.service_id = rdma_get_service_id(id_priv-id,
+(struct sockaddr *) 
route-addr.dst_addr);
req.timeout_ms = 1  (CMA_CM_RESPONSE_TIMEOUT - 8);
req.max_cm_retries = CMA_MAX_CM_RETRIES;
 
@@ -2560,8 +2561,8 @@ static int cma_connect_ib(struct rdma_id_private *id_priv,
if (route-num_paths == 2)
req.alternate_path = route-path_rec[1];
 
-   req.service_id = cma_get_service_id(id_priv-id.ps,
-   (struct sockaddr *) 
route-addr.dst_addr);
+   req.service_id = rdma_get_service_id(id_priv-id,
+(struct sockaddr *) 
route-addr.dst_addr);
req.qp_num = id_priv-qp_num;
req.qp_type = IB_QPT_RC;
req.starting_psn = id_priv-seq_num;
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index 68e73d1..e74de96 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -339,4 +339,11 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct 
sockaddr *addr);
  */
 void rdma_set_service_type(struct rdma_cm_id *id, int tos);
 
+/**
+ * rdma_get_service_id - Return the IB service ID for a specified address.
+ * @id: Communication identifier associated with the address.
+ * @addr: Address for the service ID.
+ */
+__be64 rdma_get_service_id(struct rdma_cm_id *id, struct sockaddr *addr);
+
 #endif /* RDMA_CM_H */



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 20/26] rdma/ucm: support querying when IB paths are not reversible

2010-04-01 Thread Sean Hefty

The current query_route call can return up to two path records.
The assumption being that one is the primary path, with optional
support for an alternate path.  In both cases, the paths are
assumed to be reversible and are used to send CM MADs.

With the ability to manually set IB path data, the rdma cm
can eventually be capable of using up to 6 paths per
connection:

forward primary, reverse primary,
forward alternate, reverse alternate,
reversible primary path for CM MADs
reversible alternate path for CM MADs.

(It is unclear at this time if IB routing will complicate this.)
In order to handle more flexible routing topologies, add a new 
command to report any number of paths.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/ucma.c |   35 +++
 include/rdma/rdma_user_cm.h|9 -
 2 files changed, 43 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index 2b6b51a..5d8d68e 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -696,6 +696,38 @@ static ssize_t ucma_query_addr(struct ucma_context *ctx,
return ret;
 }
 
+static ssize_t ucma_query_path(struct ucma_context *ctx,
+  void __user *response, int out_len)
+{
+   struct rdma_ucm_query_path_resp *resp;
+   int i, ret = 0;
+
+   if (out_len  sizeof(*resp))
+   return -ENOSPC;
+
+   resp = kzalloc(out_len, GFP_KERNEL);
+   if (!resp)
+   return -ENOMEM;
+
+   resp-num_paths = ctx-cm_id-route.num_paths;
+   for (i = 0, out_len -= sizeof(*resp);
+i  resp-num_paths  out_len  sizeof(struct ib_path_rec_data);
+i++, out_len -= sizeof(struct ib_path_rec_data)) {
+
+   resp-path_data[i].flags = IB_PATH_GMP | IB_PATH_PRIMARY |
+  IB_PATH_BIDIRECTIONAL;
+   ib_sa_pack_path(ctx-cm_id-route.path_rec[i],
+   resp-path_data[i].path_rec);
+   }
+
+   if (copy_to_user(response, resp,
+sizeof(*resp) + (i * sizeof(struct ib_path_rec_data
+   ret = -EFAULT;
+
+   kfree(resp);
+   return ret;
+}
+
 static ssize_t ucma_query(struct ucma_file *file,
  const char __user *inbuf,
  int in_len, int out_len)
@@ -717,6 +749,9 @@ static ssize_t ucma_query(struct ucma_file *file,
case RDMA_USER_CM_QUERY_ADDR:
ret = ucma_query_addr(ctx, response, out_len);
break;
+   case RDMA_USER_CM_QUERY_PATH:
+   ret = ucma_query_path(ctx, response, out_len);
+   break;
default:
ret = -ENOSYS;
break;
diff --git a/include/rdma/rdma_user_cm.h b/include/rdma/rdma_user_cm.h
index 9687b2f..7784203 100644
--- a/include/rdma/rdma_user_cm.h
+++ b/include/rdma/rdma_user_cm.h
@@ -115,7 +115,8 @@ struct rdma_ucm_resolve_route {
 };
 
 enum {
-   RDMA_USER_CM_QUERY_ADDR
+   RDMA_USER_CM_QUERY_ADDR,
+   RDMA_USER_CM_QUERY_PATH
 };
 
 struct rdma_ucm_query {
@@ -145,6 +146,12 @@ struct rdma_ucm_query_addr_resp {
struct sockaddr_storage dst_addr;
 };
 
+struct rdma_ucm_query_path_resp {
+   __u32 num_paths;
+   __u32 reserved;
+   struct ib_path_rec_data path_data[0];
+};
+
 struct rdma_ucm_conn_param {
__u32 qp_num;
__u32 reserved;



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 19/26] ib/sa: export function to pack a path record into wire format

2010-04-01 Thread Sean Hefty

Allow converting from struct ib_sa_path_rec to the IB defined
SA path record wire format.  This will be used to report path
data from the rdma cm into user space.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/sa_query.c |6 ++
 include/rdma/ib_sa.h   |6 ++
 2 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/core/sa_query.c 
b/drivers/infiniband/core/sa_query.c
index 7e1ffd8..54ec971 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -610,6 +610,12 @@ void ib_sa_unpack_path(void *attribute, struct 
ib_sa_path_rec *rec)
 }
 EXPORT_SYMBOL(ib_sa_unpack_path);
 
+void ib_sa_pack_path(struct ib_sa_path_rec *rec, void *attribute)
+{
+   ib_pack(path_rec_table, ARRAY_SIZE(path_rec_table), rec, attribute);
+}
+EXPORT_SYMBOL(ib_sa_pack_path);
+
 static void ib_sa_path_rec_callback(struct ib_sa_query *sa_query,
int status,
struct ib_sa_mad *mad)
diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h
index 1082afa..86aa772 100644
--- a/include/rdma/ib_sa.h
+++ b/include/rdma/ib_sa.h
@@ -385,4 +385,10 @@ int ib_init_ah_from_path(struct ib_device *device, u8 
port_num,
  */
 void ib_sa_unpack_path(void *attribute, struct ib_sa_path_rec *rec);
 
+/**
+ * ib_sa_pack_path - Conert a path record from struct ib_sa_path_rec
+ * to IB MAD wire format.
+ */
+void ib_sa_pack_path(struct ib_sa_path_rec *rec, void *attribute);
+
 #endif /* IB_SA_H */



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 13/26] rdma/cm: add support for AF_IB to rdma_resolve_addr

2010-04-01 Thread Sean Hefty

Allow the user to specify the remote address using AF_IB format.
When AF_IB is used, the remote address simply needs to be recorded,
and no resolution using ARP is done.  The local address may still
need to be resolved however.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |  107 +++--
 1 files changed, 101 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 7179409..5c956be 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -346,6 +346,61 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv)
return ret;
 }
 
+/*
+ * Select the source IB device and address to reach the destination IB address.
+ */
+static int cma_resolve_ib_dev(struct rdma_id_private *id_priv)
+{
+   struct cma_device *cma_dev, *cur_dev;
+   struct sockaddr_ib *addr;
+   union ib_gid gid, sgid, *dgid;
+   u16 pkey, index;
+   u8 port, p;
+   int i;
+
+   cma_dev = NULL;
+   addr = (struct sockaddr_ib *) id_priv-id.route.addr.dst_addr;
+   dgid = (union ib_gid *) addr-sib_addr;
+   pkey = ntohs(addr-sib_pkey);
+
+   list_for_each_entry(cur_dev, dev_list, list) {
+   if (rdma_node_get_transport(cur_dev-device-node_type) != 
RDMA_TRANSPORT_IB)
+   continue;
+
+   for (p = 1; p = cur_dev-device-phys_port_cnt; ++p) {
+   if (ib_find_cached_pkey(cur_dev-device, p, pkey, 
index))
+   continue;
+
+   for (i = 0; !ib_get_cached_gid(cur_dev-device, p, i, 
gid); i++) {
+   if (!memcmp(gid, dgid, sizeof(gid))) {
+   cma_dev = cur_dev;
+   sgid = gid;
+   port = p;
+   goto found;
+   }
+
+   if (!cma_dev  (gid.global.subnet_prefix ==
+dgid-global.subnet_prefix)) {
+   cma_dev = cur_dev;
+   sgid = gid;
+   port = p;
+   }
+   }
+   }
+   }
+
+   if (!cma_dev)
+   return -ENODEV;
+
+found:
+   cma_attach_to_dev(id_priv, cma_dev);
+   id_priv-id.port_num = port;
+   addr = (struct sockaddr_ib *) id_priv-id.route.addr.src_addr;
+   memcpy(addr-sib_addr, sgid, sizeof sgid);
+   cma_translate_ib(addr, id_priv-id.route.addr.dev_addr);
+   return 0;
+}
+
 static void cma_deref_id(struct rdma_id_private *id_priv)
 {
if (atomic_dec_and_test(id_priv-refcount))
@@ -1929,14 +1984,48 @@ err:
return ret;
 }
 
+static int cma_resolve_ib_addr(struct rdma_id_private *id_priv)
+{
+   struct cma_work *work;
+   int ret;
+
+   work = kzalloc(sizeof *work, GFP_KERNEL);
+   if (!work)
+   return -ENOMEM;
+
+   if (!id_priv-cma_dev) {
+   ret = cma_resolve_ib_dev(id_priv);
+   if (ret)
+   goto err;
+   }
+
+   rdma_addr_set_dgid(id_priv-id.route.addr.dev_addr, (union ib_gid *)
+   (((struct sockaddr_ib *) 
id_priv-id.route.addr.dst_addr)-sib_addr));
+
+   work-id = id_priv;
+   INIT_WORK(work-work, cma_work_handler);
+   work-old_state = CMA_ADDR_QUERY;
+   work-new_state = CMA_ADDR_RESOLVED;
+   work-event.event = RDMA_CM_EVENT_ADDR_RESOLVED;
+   queue_work(cma_wq, work-work);
+   return 0;
+err:
+   kfree(work);
+   return ret;
+}
+
 static int cma_bind_addr(struct rdma_cm_id *id, struct sockaddr *src_addr,
 struct sockaddr *dst_addr)
 {
if (!src_addr || !src_addr-sa_family) {
src_addr = (struct sockaddr *) id-route.addr.src_addr;
-   if ((src_addr-sa_family = dst_addr-sa_family) == AF_INET6) {
+   src_addr-sa_family = dst_addr-sa_family;
+   if (dst_addr-sa_family == AF_INET6) {
((struct sockaddr_in6 *) src_addr)-sin6_scope_id =
((struct sockaddr_in6 *) 
dst_addr)-sin6_scope_id;
+   } else if (dst_addr-sa_family == AF_IB) {
+   ((struct sockaddr_ib *) src_addr)-sib_pkey =
+   ((struct sockaddr_ib *) dst_addr)-sib_pkey;
}
}
return rdma_bind_addr(id, src_addr);
@@ -1960,12 +2049,18 @@ int rdma_resolve_addr(struct rdma_cm_id *id, struct 
sockaddr *src_addr,
 
atomic_inc(id_priv-refcount);
memcpy(id-route.addr.dst_addr, dst_addr, rdma_addr_size(dst_addr));
-   if (cma_any_addr(dst_addr))
+   if (cma_any_addr(dst_addr)) {
ret =

[PATCH 12/26] rdma/cm: restrict AF_IB loopback to binding to IB devices only

2010-04-01 Thread Sean Hefty

If a user specifies AF_IB as the source address for a loopback
connection, limit the resolution to IB devices only.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |   30 ++
 1 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 224b817..7179409 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -1794,26 +1794,40 @@ static void cma_set_loopback(struct sockaddr *addr)
 
 static int cma_bind_loopback(struct rdma_id_private *id_priv)
 {
-   struct cma_device *cma_dev;
+   struct cma_device *cma_dev, *cur_dev;
+   struct sockaddr *addr;
struct ib_port_attr port_attr;
union ib_gid gid;
u16 pkey;
int ret;
u8 p;
 
+   cma_dev = NULL;
+   addr = (struct sockaddr *) id_priv-id.route.addr.src_addr;
mutex_lock(lock);
-   if (list_empty(dev_list)) {
+   list_for_each_entry(cur_dev, dev_list, list) {
+   if (addr-sa_family == AF_IB 
+   rdma_node_get_transport(cur_dev-device-node_type) != 
RDMA_TRANSPORT_IB)
+   continue;
+
+   if (!cma_dev)
+   cma_dev = cur_dev;
+
+   for (p = 1; p = cur_dev-device-phys_port_cnt; ++p) {
+   if (!ib_query_port(cur_dev-device, p, port_attr) 
+   port_attr.state == IB_PORT_ACTIVE) {
+   cma_dev = cur_dev;
+   goto port_found;
+   }
+   }
+   }
+
+   if (!cma_dev) {
ret = -ENODEV;
goto out;
}
-   list_for_each_entry(cma_dev, dev_list, list)
-   for (p = 1; p = cma_dev-device-phys_port_cnt; ++p)
-   if (!ib_query_port(cma_dev-device, p, port_attr) 
-   port_attr.state == IB_PORT_ACTIVE)
-   goto port_found;
 
p = 1;
-   cma_dev = list_entry(dev_list.next, struct cma_device, list);
 
 port_found:
ret = ib_get_cached_gid(cma_dev-device, p, 0, gid);



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 11/26] rdma/cm: do not modify sa_family when setting loopback address

2010-04-01 Thread Sean Hefty

cma_resolve_loopback is called after an rdma_cm_id has been
bound to a specific sa_family and port.  Once the
source sa_family for the id has been set, do not modify it.
Only the actual IP address portion of the source address
needs to be set.

As part of this fix, we can simplify setting the source address
by moving the loopback address assignment from cma_resolve_loopback
to cma_bind_loopback.  cma_bind_loopback is only invoked when
the source address is the loopback address.

Finally, add loopback support for AF_IB as part of the change.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |   31 ++-
 1 files changed, 18 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index b724eac..224b817 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -1775,6 +1775,23 @@ err:
 }
 EXPORT_SYMBOL(rdma_resolve_route);
 
+static void cma_set_loopback(struct sockaddr *addr)
+{
+   switch (addr-sa_family) {
+   case AF_INET:
+   ((struct sockaddr_in *) addr)-sin_addr.s_addr = 
htonl(INADDR_LOOPBACK);
+   break;
+   case AF_INET6:
+   ipv6_addr_set(((struct sockaddr_in6 *) addr)-sin6_addr,
+ 0, 0, 0, htonl(1));
+   break;
+   default:
+   ib_addr_set(((struct sockaddr_ib *) addr)-sib_addr,
+   0, 0, 0, htonl(1));
+   break;
+   }
+}
+
 static int cma_bind_loopback(struct rdma_id_private *id_priv)
 {
struct cma_device *cma_dev;
@@ -1815,6 +1832,7 @@ port_found:
ib_addr_set_pkey(id_priv-id.route.addr.dev_addr, pkey);
id_priv-id.port_num = p;
cma_attach_to_dev(id_priv, cma_dev);
+   cma_set_loopback((struct sockaddr *) id_priv-id.route.addr.src_addr);
 out:
mutex_unlock(lock);
return ret;
@@ -1869,7 +1887,6 @@ out:
 static int cma_resolve_loopback(struct rdma_id_private *id_priv)
 {
struct cma_work *work;
-   struct sockaddr *src, *dst;
union ib_gid gid;
int ret;
 
@@ -1886,18 +1903,6 @@ static int cma_resolve_loopback(struct rdma_id_private 
*id_priv)
rdma_addr_get_sgid(id_priv-id.route.addr.dev_addr, gid);
rdma_addr_set_dgid(id_priv-id.route.addr.dev_addr, gid);
 
-   src = (struct sockaddr *) id_priv-id.route.addr.src_addr;
-   if (cma_zero_addr(src)) {
-   dst = (struct sockaddr *) id_priv-id.route.addr.dst_addr;
-   if ((src-sa_family = dst-sa_family) == AF_INET) {
-   ((struct sockaddr_in *) src)-sin_addr.s_addr =
-   ((struct sockaddr_in *) dst)-sin_addr.s_addr;
-   } else {
-   ipv6_addr_copy(((struct sockaddr_in6 *) 
src)-sin6_addr,
-  ((struct sockaddr_in6 *) 
dst)-sin6_addr);
-   }
-   }
-
work-id = id_priv;
INIT_WORK(work-work, cma_work_handler);
work-old_state = CMA_ADDR_QUERY;



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 10/26] rdma/cm: Allow user to specify AF_IB when binding

2010-04-01 Thread Sean Hefty

Modify rdma_bind_addr to allow the user to specify AF_IB when
binding to a device.  AF_IB indicates that the user is not
mapping an IP address to the native IB addressing.  (The mapping
may have already been done, or is not needed.)

AF_IB is restricted to the RDMA_PS_IB port space.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |   22 ++
 1 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 84909b9..b724eac 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -320,6 +320,13 @@ static int cma_set_qkey(struct rdma_id_private *id_priv)
return ret;
 }
 
+static void cma_translate_ib(struct sockaddr_ib *addr, struct rdma_dev_addr 
*dev_addr)
+{
+   dev_addr-dev_type = ARPHRD_INFINIBAND;
+   rdma_addr_set_sgid(dev_addr, (union ib_gid *) addr-sib_addr);
+   ib_addr_set_pkey(dev_addr, ntohs(addr-sib_pkey));
+}
+
 static int cma_acquire_dev(struct rdma_id_private *id_priv)
 {
struct rdma_dev_addr *dev_addr = id_priv-id.route.addr.dev_addr;
@@ -2177,7 +2184,9 @@ int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr 
*addr)
struct rdma_id_private *id_priv;
int ret;
 
-   if (addr-sa_family != AF_INET  addr-sa_family != AF_INET6)
+   if ((addr-sa_family != AF_INET  addr-sa_family != AF_INET6 
+   addr-sa_family != AF_IB) ||
+   (addr-sa_family == AF_IB  id-ps != RDMA_PS_IB))
return -EAFNOSUPPORT;
 
id_priv = container_of(id, struct rdma_id_private, id);
@@ -2189,9 +2198,14 @@ int rdma_bind_addr(struct rdma_cm_id *id, struct 
sockaddr *addr)
goto err1;
 
if (!cma_any_addr(addr)) {
-   ret = rdma_translate_ip(addr, id-route.addr.dev_addr);
-   if (ret)
-   goto err1;
+   if (addr-sa_family == AF_IB) {
+   cma_translate_ib((struct sockaddr_ib *) addr,
+id-route.addr.dev_addr);
+   } else {
+   ret = rdma_translate_ip(addr, id-route.addr.dev_addr);
+   if (ret)
+   goto err1;
+   }
 
mutex_lock(lock);
ret = cma_acquire_dev(id_priv);



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 9/26] rdma/cm: set qkey for port space IB

2010-04-01 Thread Sean Hefty

Include RDMA_PS_IB checks when setting the qkey for UD QPs.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index ccf47c6..84909b9 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -303,6 +303,7 @@ static int cma_set_qkey(struct rdma_id_private *id_priv)
 
switch (id_priv-id.ps) {
case RDMA_PS_UDP:
+   case RDMA_PS_IB:
id_priv-qkey = RDMA_UDP_QKEY;
break;
case RDMA_PS_IPOIB:



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 8/26] rdma/ucm: change event handling based on port space

2010-04-01 Thread Sean Hefty

The rdma_ucm handles events differently based on the port space
of the associated rdma_cm_id.  In order to support RDMA_PS_IB,
fix the handling based on the qp type.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/ucma.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index 7a59039..723fe83 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -258,7 +258,7 @@ static int ucma_event_handler(struct rdma_cm_id *cm_id,
ucma_set_event_context(ctx, event, uevent);
uevent-resp.event = event-event;
uevent-resp.status = event-status;
-   if (cm_id-ps == RDMA_PS_UDP || cm_id-ps == RDMA_PS_IPOIB)
+   if (cm_id-qp_type == IB_QPT_UD)
ucma_copy_ud_event(uevent-resp.param.ud, event-param.ud);
else
ucma_copy_conn_event(uevent-resp.param.conn,



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/26] rdma/cm: specify QP type when creating an ID

2010-04-01 Thread Sean Hefty

rdma/cm: specify QP type when creating an ID

From: Sean Hefty sean.he...@intel.com

The rdma_cm infers the QP type from the port space selected by the
user.  In order to support RDMA_PS_IB, we need to know which
QP type the user intends to support in order to perform the
correction operation, such as sending a REQ versus SIDR.

Modify rdma_create_id to allow the user to specify the QP type,
and use it to make our selections of datagram versus connected
mode.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c|   35 ++
 drivers/infiniband/core/ucma.c   |   27 ---
 drivers/infiniband/ulp/iser/iser_verbs.c |2 +-
 include/rdma/rdma_cm.h   |5 +++-
 net/9p/trans_rdma.c  |3 ++-
 net/rds/ib.c |2 +-
 net/rds/ib_cm.c  |2 +-
 net/rds/iw.c |2 +-
 net/rds/iw_cm.c  |2 +-
 net/rds/rdma_transport.c |3 ++-
 net/sunrpc/xprtrdma/svc_rdma_transport.c |3 ++-
 net/sunrpc/xprtrdma/verbs.c  |2 +-
 12 files changed, 55 insertions(+), 33 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 1328a8e..ccf47c6 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -271,11 +271,6 @@ static inline void sdp_set_ip_ver(struct sdp_hh *hh, u8 
ip_ver)
hh-ip_version = (ip_ver  4) | (hh-ip_version  0xF);
 }
 
-static inline int cma_is_ud_ps(enum rdma_port_space ps)
-{
-   return (ps == RDMA_PS_UDP || ps == RDMA_PS_IPOIB);
-}
-
 static void cma_attach_to_dev(struct rdma_id_private *id_priv,
  struct cma_device *cma_dev)
 {
@@ -366,7 +361,8 @@ static int cma_has_cm_dev(struct rdma_id_private *id_priv)
 }
 
 struct rdma_cm_id *rdma_create_id(rdma_cm_event_handler event_handler,
- void *context, enum rdma_port_space ps)
+ void *context, enum rdma_port_space ps,
+ enum ib_qp_type qp_type)
 {
struct rdma_id_private *id_priv;
 
@@ -378,6 +374,7 @@ struct rdma_cm_id *rdma_create_id(rdma_cm_event_handler 
event_handler,
id_priv-id.context = context;
id_priv-id.event_handler = event_handler;
id_priv-id.ps = ps;
+   id_priv-id.qp_type = qp_type;
spin_lock_init(id_priv-lock);
mutex_init(id_priv-qp_mutex);
init_completion(id_priv-comp);
@@ -445,7 +442,7 @@ int rdma_create_qp(struct rdma_cm_id *id, struct ib_pd *pd,
if (IS_ERR(qp))
return PTR_ERR(qp);
 
-   if (cma_is_ud_ps(id_priv-id.ps))
+   if (id-qp_type == IB_QPT_UD)
ret = cma_init_ud_qp(id_priv, qp);
else
ret = cma_init_conn_qp(id_priv, qp);
@@ -567,7 +564,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private 
*id_priv,
qp_attr-port_num = id_priv-id.port_num;
*qp_attr_mask = IB_QP_STATE | IB_QP_PKEY_INDEX | IB_QP_PORT;
 
-   if (cma_is_ud_ps(id_priv-id.ps)) {
+   if (id_priv-id.qp_type == IB_QPT_UD) {
ret = cma_set_qkey(id_priv);
if (ret)
return ret;
@@ -590,7 +587,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct 
ib_qp_attr *qp_attr,
id_priv = container_of(id, struct rdma_id_private, id);
switch (rdma_node_get_transport(id_priv-id.device-node_type)) {
case RDMA_TRANSPORT_IB:
-   if (!id_priv-cm_id.ib || cma_is_ud_ps(id_priv-id.ps))
+   if (!id_priv-cm_id.ib || (id_priv-id.qp_type == IB_QPT_UD))
ret = cma_ib_init_qp_attr(id_priv, qp_attr, 
qp_attr_mask);
else
ret = ib_cm_init_qp_attr(id_priv-cm_id.ib, qp_attr,
@@ -1035,7 +1032,7 @@ static struct rdma_id_private *cma_new_conn_id(struct 
rdma_cm_id *listen_id,
goto err;
 
id = rdma_create_id(listen_id-event_handler, listen_id-context,
-   listen_id-ps);
+   listen_id-ps, ib_event-param.req_rcvd.qp_type);
if (IS_ERR(id))
goto err;
 
@@ -1086,7 +1083,7 @@ static struct rdma_id_private *cma_new_udp_id(struct 
rdma_cm_id *listen_id,
int ret;
 
id = rdma_create_id(listen_id-event_handler, listen_id-context,
-   listen_id-ps);
+   listen_id-ps, IB_QPT_UD);
if (IS_ERR(id))
return NULL;
 
@@ -1141,7 +1138,7 @@ static int cma_req_handler(struct ib_cm_id *cm_id, struct 
ib_cm_event *ib_event)
memset(event, 0, sizeof event);
offset = cma_user_data_offset(listen_id-id.ps);
event.event = RDMA_CM_EVENT_CONNECT_REQUEST;
-   if (cma_is_ud_ps(listen_id-id.ps)) {
+   if (listen_id-id.qp_type == IB_QPT_UD) {

[PATCH 5/26] rdma/cm: update port reservation to support AF_IB

2010-04-01 Thread Sean Hefty

Update the port reservation code path to support AF_IB addresses.
Define a new port space, RDMA_PS_IB, for AF_IB addressing.

The AF_IB port space is a 64-bit service id (SID), which the
user can control through the use of a mask.  The rdma_cm
will assign values to the unmasked portions of the SID
when an rdma_cm_id is bound to an AF_IB address.

Because the IB spec divides the SID range into several regions,
a SID/mask combination may fall into one of the existing
port space ranges as defined by the RDMA CM IP Annex.  We need
to coordinate port reservation between RDMA_PS_IB and 
RDMA_PS_TCP/RDMA_PS_UDP.
 
Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |   69 +
 include/rdma/rdma_cm.h|6 
 2 files changed, 68 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index ac57155..1328a8e 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -657,18 +657,29 @@ static int cma_addr_cmp(struct sockaddr *src, struct 
sockaddr *dst)
case AF_INET:
return ((struct sockaddr_in *) src)-sin_addr.s_addr !=
   ((struct sockaddr_in *) dst)-sin_addr.s_addr;
-   default:
+   case AF_INET6:
return ipv6_addr_cmp(((struct sockaddr_in6 *) src)-sin6_addr,
 ((struct sockaddr_in6 *) dst)-sin6_addr);
+   default:
+   return ib_addr_cmp(((struct sockaddr_ib *) src)-sib_addr,
+  ((struct sockaddr_ib *) dst)-sib_addr);
}
 }
 
-static inline __be16 cma_port(struct sockaddr *addr)
+/* AF_IB must be using the RDMA CM IP Annex */
+static __be16 cma_port(struct sockaddr *addr)
 {
-   if (addr-sa_family == AF_INET)
+   switch (addr-sa_family) {
+   case AF_INET:
return ((struct sockaddr_in *) addr)-sin_port;
-   else
+   case AF_INET6:
return ((struct sockaddr_in6 *) addr)-sin6_port;
+   case AF_IB:
+   return htons((u16) (be64_to_cpu(((struct sockaddr_ib *) 
addr)-sib_sid) 
+be64_to_cpu(((struct sockaddr_ib *) 
addr)-sib_sid_mask)));
+   default:
+   return 0;
+   }
 }
 
 static inline int cma_any_port(struct sockaddr *addr)
@@ -1945,10 +1956,29 @@ EXPORT_SYMBOL(rdma_resolve_addr);
 static void cma_bind_port(struct rdma_bind_list *bind_list,
  struct rdma_id_private *id_priv)
 {
-   struct sockaddr_in *sin;
+   struct sockaddr *addr;
+   struct sockaddr_ib *sib;
+   u64 sid, mask;
+   __be16 port;
 
-   sin = (struct sockaddr_in *) id_priv-id.route.addr.src_addr;
-   sin-sin_port = htons(bind_list-port);
+   addr = (struct sockaddr *) id_priv-id.route.addr.src_addr;
+   port = htons(bind_list-port);
+
+   switch (addr-sa_family) {
+   case AF_INET:
+   ((struct sockaddr_in *) addr)-sin_port = port;
+   break;
+   case AF_INET6:
+   ((struct sockaddr_in6 *) addr)-sin6_port = port;
+   break;
+   case AF_IB:
+   sib = (struct sockaddr_ib *) addr;
+   sid = be64_to_cpu(sib-sib_sid);
+   mask = be64_to_cpu(sib-sib_sid_mask);
+   sib-sib_sid = cpu_to_be64((sid  mask) | (u64) ntohs(port));
+   sib-sib_sid_mask = cpu_to_be64(~0ULL);
+   break;
+   }
id_priv-bind_list = bind_list;
hlist_add_head(id_priv-node, bind_list-owners);
 }
@@ -2068,6 +2098,26 @@ static int cma_use_port(struct idr *ps, struct 
rdma_id_private *id_priv)
return 0;
 }
 
+static struct idr *cma_select_ib_ps(struct rdma_id_private *id_priv)
+{
+   struct sockaddr_ib *sib;
+   u64 mask, sid;
+
+   sib = (struct sockaddr_ib *) id_priv-id.route.addr.src_addr;
+   mask = be64_to_cpu(sib-sib_sid_mask);
+   sid = be64_to_cpu(sib-sib_sid);
+
+   if (mask != RDMA_IB_IP_PS_MASK  mask != ~0ULL)
+   return NULL;
+
+   if ((sid  RDMA_IB_IP_PS_MASK) == RDMA_IB_IP_PS_TCP)
+   return tcp_ps;
+   else if ((sid  RDMA_IB_IP_PS_MASK) == RDMA_IB_IP_PS_UDP)
+   return udp_ps;
+   else
+   return NULL;
+}
+
 static int cma_get_port(struct rdma_id_private *id_priv)
 {
struct idr *ps;
@@ -2086,6 +2136,11 @@ static int cma_get_port(struct rdma_id_private *id_priv)
case RDMA_PS_IPOIB:
ps = ipoib_ps;
break;
+   case RDMA_PS_IB:
+   ps = cma_select_ib_ps(id_priv);
+   if (!ps)
+   return -EINVAL;
+   break;
default:
return -EPROTONOSUPPORT;
}
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index c6b2962..8c27b6e 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -65,11 +65,17 @@ enum

[PATCH 4/26] ib/addr: add AF_IB support to ip_addr_size

2010-04-01 Thread Sean Hefty

Add support for AF_IB to ip_addr_size, and rename the function
to account for the change.  Give the compiler more control over
whether the call should be inline or not by moving the definition
into the .c file, removing the static inline, and exporting it.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/addr.c |   20 ++--
 drivers/infiniband/core/cma.c  |   12 ++--
 include/rdma/ib_addr.h |6 +-
 3 files changed, 25 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c
index abbb069..f73c5de 100644
--- a/drivers/infiniband/core/addr.c
+++ b/drivers/infiniband/core/addr.c
@@ -43,6 +43,7 @@
 #include net/addrconf.h
 #include net/ip6_route.h
 #include rdma/ib_addr.h
+#include rdma/ib.h
 
 MODULE_AUTHOR(Sean Hefty);
 MODULE_DESCRIPTION(IB Address Translation);
@@ -68,6 +69,21 @@ static LIST_HEAD(req_list);
 static DECLARE_DELAYED_WORK(work, process_req);
 static struct workqueue_struct *addr_wq;
 
+int rdma_addr_size(struct sockaddr *addr)
+{
+   switch (addr-sa_family) {
+   case AF_INET:
+   return sizeof(struct sockaddr_in);
+   case AF_INET6:
+   return sizeof(struct sockaddr_in6);
+   case AF_IB:
+   return sizeof(struct sockaddr_ib);
+   default:
+   return 0;
+   }
+}
+EXPORT_SYMBOL(rdma_addr_size);
+
 void rdma_addr_register_client(struct rdma_addr_client *client)
 {
atomic_set(client-refcount, 1);
@@ -366,12 +382,12 @@ int rdma_resolve_ip(struct rdma_addr_client *client,
goto err;
}
 
-   memcpy(src_in, src_addr, ip_addr_size(src_addr));
+   memcpy(src_in, src_addr, rdma_addr_size(src_addr));
} else {
src_in-sa_family = dst_addr-sa_family;
}
 
-   memcpy(dst_in, dst_addr, ip_addr_size(dst_addr));
+   memcpy(dst_in, dst_addr, rdma_addr_size(dst_addr));
req-addr = addr;
req-callback = callback;
req-context = context;
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 6460fbf..ac57155 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -1468,7 +1468,7 @@ static void cma_listen_on_dev(struct rdma_id_private 
*id_priv,
 
dev_id_priv-state = CMA_ADDR_BOUND;
memcpy(id-route.addr.src_addr, id_priv-id.route.addr.src_addr,
-  ip_addr_size((struct sockaddr *) 
id_priv-id.route.addr.src_addr));
+  rdma_addr_size((struct sockaddr *) 
id_priv-id.route.addr.src_addr));
 
cma_attach_to_dev(dev_id_priv, cma_dev);
list_add_tail(dev_id_priv-listen_list, id_priv-listen_list);
@@ -1834,7 +1834,7 @@ static void addr_handler(int status, struct sockaddr 
*src_addr,
event.status = status;
} else {
memcpy(id_priv-id.route.addr.src_addr, src_addr,
-  ip_addr_size(src_addr));
+  rdma_addr_size(src_addr));
event.event = RDMA_CM_EVENT_ADDR_RESOLVED;
}
 
@@ -1924,7 +1924,7 @@ int rdma_resolve_addr(struct rdma_cm_id *id, struct 
sockaddr *src_addr,
return -EINVAL;
 
atomic_inc(id_priv-refcount);
-   memcpy(id-route.addr.dst_addr, dst_addr, ip_addr_size(dst_addr));
+   memcpy(id-route.addr.dst_addr, dst_addr, rdma_addr_size(dst_addr));
if (cma_any_addr(dst_addr))
ret = cma_resolve_loopback(id_priv);
else
@@ -2147,7 +2147,7 @@ int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr 
*addr)
goto err1;
}
 
-   memcpy(id-route.addr.src_addr, addr, ip_addr_size(addr));
+   memcpy(id-route.addr.src_addr, addr, rdma_addr_size(addr));
ret = cma_get_port(id_priv);
if (ret)
goto err2;
@@ -2807,7 +2807,7 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct 
sockaddr *addr,
if (!mc)
return -ENOMEM;
 
-   memcpy(mc-addr, addr, ip_addr_size(addr));
+   memcpy(mc-addr, addr, rdma_addr_size(addr));
mc-context = context;
mc-id_priv = id_priv;
 
@@ -2842,7 +2842,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct 
sockaddr *addr)
id_priv = container_of(id, struct rdma_id_private, id);
spin_lock_irq(id_priv-lock);
list_for_each_entry(mc, id_priv-mc_list, list) {
-   if (!memcmp(mc-addr, addr, ip_addr_size(addr))) {
+   if (!memcmp(mc-addr, addr, rdma_addr_size(addr))) {
list_del(mc-list);
spin_unlock_irq(id_priv-lock);
 
diff --git a/include/rdma/ib_addr.h b/include/rdma/ib_addr.h
index fa0d52b..9e06769 100644
--- a/include/rdma/ib_addr.h
+++ b/include/rdma/ib_addr.h
@@ -99,11 +99,7 @@ void rdma_addr_cancel(struct rdma_dev_addr *addr);
 int rdma_copy_addr(struct rdma_dev_addr *dev_addr, struct net_device *dev,

[PATCH 3/26] rdma/cm: include AF_IB in loopback and any address checks

2010-04-01 Thread Sean Hefty

Enhance checks for loopback and any address to support AF_IB
in addition to AF_INET and AF_INT6.  This will allow future
patches to use AF_IB when binding and resolving addresses.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |   35 ---
 1 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 9041a2b..6460fbf 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -46,6 +46,7 @@
 
 #include rdma/rdma_cm.h
 #include rdma/rdma_cm_ib.h
+#include rdma/ib.h
 #include rdma/ib_cache.h
 #include rdma/ib_cm.h
 #include rdma/ib_sa.h
@@ -616,26 +617,30 @@ EXPORT_SYMBOL(rdma_init_qp_attr);
 
 static inline int cma_zero_addr(struct sockaddr *addr)
 {
-   struct in6_addr *ip6;
-
-   if (addr-sa_family == AF_INET)
-   return ipv4_is_zeronet(
-   ((struct sockaddr_in *)addr)-sin_addr.s_addr);
-   else {
-   ip6 = ((struct sockaddr_in6 *) addr)-sin6_addr;
-   return (ip6-s6_addr32[0] | ip6-s6_addr32[1] |
-   ip6-s6_addr32[2] | ip6-s6_addr32[3]) == 0;
+   switch (addr-sa_family) {
+   case AF_INET:
+   return ipv4_is_zeronet(((struct sockaddr_in 
*)addr)-sin_addr.s_addr);
+   case AF_INET6:
+   return ipv6_addr_any(((struct sockaddr_in6 *) 
addr)-sin6_addr);
+   case AF_IB:
+   return ib_addr_any(((struct sockaddr_ib *) addr)-sib_addr);
+   default:
+   return 0;
}
 }
 
 static inline int cma_loopback_addr(struct sockaddr *addr)
 {
-   if (addr-sa_family == AF_INET)
-   return ipv4_is_loopback(
-   ((struct sockaddr_in *) addr)-sin_addr.s_addr);
-   else
-   return ipv6_addr_loopback(
-   ((struct sockaddr_in6 *) addr)-sin6_addr);
+   switch (addr-sa_family) {
+   case AF_INET:
+   return ipv4_is_loopback(((struct sockaddr_in *) 
addr)-sin_addr.s_addr);
+   case AF_INET6:
+   return ipv6_addr_loopback(((struct sockaddr_in6 *) 
addr)-sin6_addr);
+   case AF_IB:
+   return ib_addr_loopback(((struct sockaddr_ib *) 
addr)-sib_addr);
+   default:
+   return 0;
+   }
 }
 
 static inline int cma_any_addr(struct sockaddr *addr)



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/26] rdma/cm: fix handling of ipv6 addressing in cma_use_port

2010-04-01 Thread Sean Hefty

cma_use_port is coded assuming that the sockaddr is an ipv4 address.
Since ipv6 addressing is supported, and also to support other address
families, make the code more generic in its address handling.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 drivers/infiniband/core/cma.c |   29 ++---
 1 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 875e34e..9041a2b 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -643,6 +643,21 @@ static inline int cma_any_addr(struct sockaddr *addr)
return cma_zero_addr(addr) || cma_loopback_addr(addr);
 }
 
+static int cma_addr_cmp(struct sockaddr *src, struct sockaddr *dst)
+{
+   if (src-sa_family != dst-sa_family)
+   return -1;
+
+   switch (src-sa_family) {
+   case AF_INET:
+   return ((struct sockaddr_in *) src)-sin_addr.s_addr !=
+  ((struct sockaddr_in *) dst)-sin_addr.s_addr;
+   default:
+   return ipv6_addr_cmp(((struct sockaddr_in6 *) src)-sin6_addr,
+((struct sockaddr_in6 *) dst)-sin6_addr);
+   }
+}
+
 static inline __be16 cma_port(struct sockaddr *addr)
 {
if (addr-sa_family == AF_INET)
@@ -2014,13 +2029,13 @@ err1:
 static int cma_use_port(struct idr *ps, struct rdma_id_private *id_priv)
 {
struct rdma_id_private *cur_id;
-   struct sockaddr_in *sin, *cur_sin;
+   struct sockaddr *addr, *cur_addr;
struct rdma_bind_list *bind_list;
struct hlist_node *node;
unsigned short snum;
 
-   sin = (struct sockaddr_in *) id_priv-id.route.addr.src_addr;
-   snum = ntohs(sin-sin_port);
+   addr = (struct sockaddr *) id_priv-id.route.addr.src_addr;
+   snum = ntohs(cma_port(addr));
if (snum  PROT_SOCK  !capable(CAP_NET_BIND_SERVICE))
return -EACCES;
 
@@ -2032,15 +2047,15 @@ static int cma_use_port(struct idr *ps, struct 
rdma_id_private *id_priv)
 * We don't support binding to any address if anyone is bound to
 * a specific address on the same port.
 */
-   if (cma_any_addr((struct sockaddr *) id_priv-id.route.addr.src_addr))
+   if (cma_any_addr(addr))
return -EADDRNOTAVAIL;
 
hlist_for_each_entry(cur_id, node, bind_list-owners, node) {
-   if (cma_any_addr((struct sockaddr *) 
cur_id-id.route.addr.src_addr))
+   cur_addr = (struct sockaddr *) cur_id-id.route.addr.src_addr;
+   if (cma_any_addr(cur_addr))
return -EADDRNOTAVAIL;
 
-   cur_sin = (struct sockaddr_in *) 
cur_id-id.route.addr.src_addr;
-   if (sin-sin_addr.s_addr == cur_sin-sin_addr.s_addr)
+   if (!cma_addr_cmp(addr, cur_addr))
return -EADDRINUSE;
}
 



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/26] rdma/cm: define native IB address

2010-04-01 Thread Sean Hefty

Define AF_IB and sockaddr_ib to allow the rdma_cm to use native IB
addressing.

Signed-off-by: Sean Hefty sean.he...@intel.com
---

 include/linux/socket.h |2 +
 include/rdma/ib.h  |   89 
 2 files changed, 91 insertions(+), 0 deletions(-)

diff --git a/include/linux/socket.h b/include/linux/socket.h
index 7b3aae2..966e268 100644
--- a/include/linux/socket.h
+++ b/include/linux/socket.h
@@ -181,6 +181,7 @@ struct ucred {
 #define AF_PPPOX   24  /* PPPoX sockets*/
 #define AF_WANPIPE 25  /* Wanpipe API Sockets */
 #define AF_LLC 26  /* Linux LLC*/
+#define AF_IB  27  /* Native InfiniBand address*/
 #define AF_CAN 29  /* Controller Area Network  */
 #define AF_TIPC30  /* TIPC sockets */
 #define AF_BLUETOOTH   31  /* Bluetooth sockets*/
@@ -221,6 +222,7 @@ struct ucred {
 #define PF_PPPOX   AF_PPPOX
 #define PF_WANPIPE AF_WANPIPE
 #define PF_LLC AF_LLC
+#define PF_IB  AF_IB
 #define PF_CAN AF_CAN
 #define PF_TIPCAF_TIPC
 #define PF_BLUETOOTH   AF_BLUETOOTH
diff --git a/include/rdma/ib.h b/include/rdma/ib.h
new file mode 100644
index 000..cf8f9e7
--- /dev/null
+++ b/include/rdma/ib.h
@@ -0,0 +1,89 @@
+/*
+ * Copyright (c) 2010 Intel Corporation.  All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ *  - Redistributions of source code must retain the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer.
+ *
+ *  - Redistributions in binary form must reproduce the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer in the documentation and/or other materials
+ *provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#if !defined(_RDMA_IB_H)
+#define _RDMA_IB_H
+
+#include linux/types.h
+
+struct ib_addr {
+   union {
+   __u8uib_addr8[16];
+   __be16  uib_addr16[8];
+   __be32  uib_addr32[4];
+   __be64  uib_addr64[2];
+   } ib_u;
+#define sib_addr8  ib_u.uib_addr8
+#define sib_addr16 ib_u.uib_addr16
+#define sib_addr32 ib_u.uib_addr32
+#define sib_addr64 ib_u.uib_addr64
+#define sib_rawib_u.uib_addr8
+#define sib_subnet_prefix  ib_u.uib_addr64[0]
+#define sib_interface_id   ib_u.uib_addr64[1]
+};
+
+static inline int ib_addr_any(const struct ib_addr *a)
+{
+   return ((a-sib_addr64[0] | a-sib_addr64[1]) == 0);
+}
+
+static inline int ib_addr_loopback(const struct ib_addr *a)
+{
+   return ((a-sib_addr32[0] | a-sib_addr32[1] |
+a-sib_addr32[2] | (a-sib_addr32[3] ^ htonl(1))) == 0);
+}
+
+static inline void ib_addr_set(struct ib_addr *addr,
+  __be32 w1, __be32 w2, __be32 w3, __be32 w4)
+{
+   addr-sib_addr32[0] = w1;
+   addr-sib_addr32[1] = w2;
+   addr-sib_addr32[2] = w3;
+   addr-sib_addr32[3] = w4;
+}
+
+static inline int ib_addr_cmp(const struct ib_addr *a1, const struct ib_addr 
*a2)
+{
+   return memcmp(a1, a2, sizeof(struct ib_addr));
+}
+
+struct sockaddr_ib {
+   unsigned short int  sib_family; /* AF_IB */
+   __be16  sib_pkey;
+   __be32  sib_flowinfo;
+   struct ib_addr  sib_addr;
+   __be64  sib_sid;
+   __be64  sib_sid_mask;
+   __u64   sib_scope_id;
+};
+
+#endif /* _RDMA_IB_H */



--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/26] rdma/cm: add support for native IB addressing

2010-04-01 Thread Sean Hefty

The following patch series extends the rdma_cm to support native
Infiniband addressing through the use of a new AF_IB address family
and RDMA_PS_IB port space.

It defines a new struct sockaddr_ib that may be used to specify an
IB GID, along with other IB address attributes, such as the pkey and
service ID.

The higher level intent is to support a user space call, rdma_getaddrinfo,
which can return AF_IB addresses to an application.  This allows the
rdma_cm to support transport specific features, such as failover and
non-reversible paths, and eliminates the need for the kernel to perform
address and route resolution.  (An implementation of rdma_getaddrinfo is 
included in a separate patch set to the librdmacm.)

This patch series is also available from:

git://git.openfabrics.org/~shefty/rdma-dev.git af_ib

Signed-off-by: Sean Hefty sean.he...@intel.com
---
Changes from RFC patch series:
Added RDMA_PS_IB for use with AF_IB

 drivers/infiniband/core/addr.c   |   20 +
 drivers/infiniband/core/cma.c|  442 +++
 drivers/infiniband/core/sa_query.c   |6
 drivers/infiniband/core/ucma.c   |  365 ++---
 drivers/infiniband/ulp/iser/iser_verbs.c |2
 include/linux/socket.h   |2
 include/rdma/ib.h|   89 ++
 include/rdma/ib_addr.h   |6
 include/rdma/ib_sa.h |6
 include/rdma/rdma_cm.h   |   18 +
 include/rdma/rdma_user_cm.h  |   84 -
 net/9p/trans_rdma.c  |3
 net/rds/ib.c |2
 net/rds/ib_cm.c  |2
 net/rds/iw.c |2
 net/rds/iw_cm.c  |2
 net/rds/rdma_transport.c |3
 net/sunrpc/xprtrdma/svc_rdma_transport.c |3
 net/sunrpc/xprtrdma/verbs.c  |2
 19 files changed, 863 insertions(+), 196 deletions(-)

--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 9/26] rdma/cm: set qkey for port space IB

2010-04-01 Thread Jason Gunthorpe

On Thu, Apr 01, 2010 at 10:08:41AM -0700, Sean Hefty wrote:
 Include RDMA_PS_IB checks when setting the qkey for UD QPs.

I know we talked about this, but seeing this patch makes me ask again,
should the QKEY be part of sockaddr_ib? Or at least be settable somehow?

Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 26/26] rdma/ucm: allow user space to specify AF_IB when joining multicast

2010-04-01 Thread Sean Hefty

  } else if ((addr-sa_family == AF_INET6)) {
  ipv6_ib_mc_map(sin6-sin6_addr, dev_addr-broadcast, mc_map);
 -if (id_priv-id.ps == RDMA_PS_UDP)
 +if (id_priv-id.ps == RDMA_PS_UDP || id_priv-id.ps ==
RDMA_PS_IB)
  mc_map[7] = 0x01;   /* Use RDMA CM signature */
  *mgid = *(union ib_gid *) (mc_map + 4);

What does the combination of AF_INET/INET6 and RDMA_PS_IB mean?

This indicates that the rdma_cm_id is bound to AF_IB/RDMA_PS_IB, but that the
multicast join address was provided using AF_INET6.  The 'else' code that
follows the above code:

} else {
ip_ib_mc_map(sin-sin_addr.s_addr, dev_addr-broadcast, mc_map);
-   if (id_priv-id.ps == RDMA_PS_UDP)
+   if (id_priv-id.ps == RDMA_PS_UDP || id_priv-id.ps ==
RDMA_PS_IB)
mc_map[7] = 0x01;   /* Use RDMA CM signature */
*mgid = *(union ib_gid *) (mc_map + 4);
}

is similar, except that the multicast join address was specified using AF_INET.

- Sean

--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 9/26] rdma/cm: set qkey for port space IB

2010-04-01 Thread Sean Hefty

I know we talked about this, but seeing this patch makes me ask again,
should the QKEY be part of sockaddr_ib? Or at least be settable somehow?

I think so.  I'm not sure of the best approach.  With these patches, the qkey
seems to be the one thing blocking the rdma_cm from supporting any IB UD
application.

Another item that I wonder about putting into sockaddr_ib is the SL.  There's a
way to set it today, but I wonder if it should be part of sockaddr_ib anyway.

- Sean

--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nfsrdma broken on 2.6.34-rc1?

2010-04-01 Thread Tom Tucker


Sean Hefty wrote:

Sean, will you add this to the rdma_cm?



Not immediately because I lack the time to do it.

It would be really nice to share the kernel's port space code and remove the
port code in the rdma_cm.

  


LOL. Yes...yes it would. There is of course a Dragon to be slain. Roland?


--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
  


--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 9/26] rdma/cm: set qkey for port space IB

2010-04-01 Thread Jason Gunthorpe

On Thu, Apr 01, 2010 at 11:00:44AM -0700, Sean Hefty wrote:
 I know we talked about this, but seeing this patch makes me ask again,
 should the QKEY be part of sockaddr_ib? Or at least be settable somehow?
 
 I think so.  I'm not sure of the best approach.  With these patches,
 the qkey seems to be the one thing blocking the rdma_cm from
 supporting any IB UD application.

Right.. qkey is kinda a funny thing, and pkey too I suppose.

I'm not sure, I don't really know the usage model for qkey..

 Another item that I wonder about putting into sockaddr_ib is the SL.
 There's a way to set it today, but I wonder if it should be part of
 sockaddr_ib anyway.

SL is the LRH form of TClass/FlowLabel, like sockaddr_in6 sockaddr_ib
should have those two. If you want to control the SL then I think it
is reasonable that you'd have to use the APIs you added to set the
entire path record.

Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 26/26] rdma/ucm: allow user space to specify AF_IB when joining multicast

2010-04-01 Thread Jason Gunthorpe

On Thu, Apr 01, 2010 at 10:46:06AM -0700, Sean Hefty wrote:
 } else if ((addr-sa_family == AF_INET6)) {
 ipv6_ib_mc_map(sin6-sin6_addr, dev_addr-broadcast, mc_map);
  -  if (id_priv-id.ps == RDMA_PS_UDP)
  +  if (id_priv-id.ps == RDMA_PS_UDP || id_priv-id.ps ==
 RDMA_PS_IB)
 mc_map[7] = 0x01;   /* Use RDMA CM signature */
 *mgid = *(union ib_gid *) (mc_map + 4);
 
 What does the combination of AF_INET/INET6 and RDMA_PS_IB mean?
 
 This indicates that the rdma_cm_id is bound to AF_IB/RDMA_PS_IB, but that the
 multicast join address was provided using AF_INET6.  The 'else' code that
 follows the above code:

Humm, interesting.. So you propose to accept addresses for families
other than the one used to create the cm_id?

Just from a simplicity of the future perspective I'd say not to bother
...

Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH,RFC] nfsd: Make INET6 transport creation failure an informational message

2010-04-01 Thread Tom Tucker


If this looks right to everyone, I'll post this to linux-nfs.

Tom

nfsd: Make INET6 transport creation failure an informational message

The write_ports code will fail both the IENT4 and INET6 transport creation if
the transport returns an error when PF_INET6 is specified. Some transports
that do not support INET6 return an error other than EAFNOSUPPORT. We should
allow communication on INET4 even if INET6 is not yet supported or fails
for some reason.

Signed-off-by: Tom Tucker t...@opengridcomputing.com
---

fs/nfsd/nfsctl.c |6 --
1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 0f0e77f..019a89e 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -1008,8 +1008,10 @@ static ssize_t __write_ports_addxprt(char *buf)

err = svc_create_xprt(nfsd_serv, transport,
PF_INET6, port, SVC_SOCK_ANONYMOUS);
-   if (err  0  err != -EAFNOSUPPORT)
-   goto out_close;
+   if (err  0)
+   dprintk(nfsd: Error creating PF_INET6 listener for transport 
'%s'\n,
+transport);
+
return 0;
out_close:
xprt = svc_find_xprt(nfsd_serv, transport, PF_INET, port);

--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH,RFC] nfsd: Make INET6 transport creation failure an informational message

2010-04-01 Thread Steve Wise


Looks good.


Tom Tucker wrote:

If this looks right to everyone, I'll post this to linux-nfs.

Tom

nfsd: Make INET6 transport creation failure an informational message

The write_ports code will fail both the IENT4 and INET6 transport 
creation if
the transport returns an error when PF_INET6 is specified. Some 
transports
that do not support INET6 return an error other than EAFNOSUPPORT. We 
should

allow communication on INET4 even if INET6 is not yet supported or fails
for some reason.

Signed-off-by: Tom Tucker t...@opengridcomputing.com
---

fs/nfsd/nfsctl.c |6 --
1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 0f0e77f..019a89e 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -1008,8 +1008,10 @@ static ssize_t __write_ports_addxprt(char *buf)

err = svc_create_xprt(nfsd_serv, transport,
PF_INET6, port, SVC_SOCK_ANONYMOUS);
-if (err  0  err != -EAFNOSUPPORT)
-goto out_close;
+if (err  0)
+dprintk(nfsd: Error creating PF_INET6 listener for transport 
'%s'\n,

+ transport);
+
return 0;
out_close:
xprt = svc_find_xprt(nfsd_serv, transport, PF_INET, port);


--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH,RFC] nfsd: Make INET6 transport creation failure an informational message

2010-04-01 Thread Tom Tucker


Hi Bruce/Chuck,

RDMA Transports are currently broken in 2.6.34 because they don't have a 
V4ONLY setsockopt. So what happens is that when write_ports attempts to 
create the PF_INET6 transport it fails because the port is already in 
use. There is discussion on linux-rdma about how to fix this, but in the 
interim and perhaps indefinitely, I propose the following:


Tom

nfsd: Make INET6 transport creation failure an informational message

The write_ports code will fail both the INET4 and INET6 transport creation if
the transport returns an error when PF_INET6 is specified. Some transports
that do not support INET6 return an error other than EAFNOSUPPORT. We should
allow communication on INET4 even if INET6 is not yet supported or fails
for some reason.

Signed-off-by: Tom Tucker t...@opengridcomputing.com
---

fs/nfsd/nfsctl.c |6 --
1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 0f0e77f..934b624 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -1008,8 +1008,10 @@ static ssize_t __write_ports_addxprt(char *buf)

err = svc_create_xprt(nfsd_serv, transport,
PF_INET6, port, SVC_SOCK_ANONYMOUS);
-   if (err  0  err != -EAFNOSUPPORT)
-   goto out_close;
+   if (err  0)
+   printk(KERN_INFO nfsd: Error creating PF_INET6 listener 
+  for transport '%s'\n, transport);
+
return 0;
out_close:
xprt = svc_find_xprt(nfsd_serv, transport, PF_INET, port);

--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] RDMA/cxgb3: use the dma state API instead of the pci equivalents

2010-04-01 Thread FUJITA Tomonori

The DMA API is preferred.

No functional change.

Signed-off-by: FUJITA Tomonori fujita.tomon...@lab.ntt.co.jp
---
 drivers/infiniband/hw/cxgb3/cxio_hal.c |   12 ++--
 drivers/infiniband/hw/cxgb3/cxio_hal.h |2 +-
 drivers/infiniband/hw/cxgb3/cxio_wr.h  |4 ++--
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.c 
b/drivers/infiniband/hw/cxgb3/cxio_hal.c
index a28e862..619bae2 100644
--- a/drivers/infiniband/hw/cxgb3/cxio_hal.c
+++ b/drivers/infiniband/hw/cxgb3/cxio_hal.c
@@ -173,7 +173,7 @@ int cxio_create_cq(struct cxio_rdev *rdev_p, struct t3_cq 
*cq, int kernel)
kfree(cq-sw_queue);
return -ENOMEM;
}
-   pci_unmap_addr_set(cq, mapping, cq-dma_addr);
+   dma_unmap_addr_set(cq, mapping, cq-dma_addr);
memset(cq-queue, 0, size);
setup.id = cq-cqid;
setup.base_addr = (u64) (cq-dma_addr);
@@ -296,7 +296,7 @@ int cxio_create_qp(struct cxio_rdev *rdev_p, u32 
kernel_domain,
goto err4;
 
memset(wq-queue, 0, depth * sizeof(union t3_wr));
-   pci_unmap_addr_set(wq, mapping, wq-dma_addr);
+   dma_unmap_addr_set(wq, mapping, wq-dma_addr);
wq-doorbell = (void __iomem *)rdev_p-rnic_info.kdb_addr;
if (!kernel_domain)
wq-udb = (u64)rdev_p-rnic_info.udbell_physbase +
@@ -324,7 +324,7 @@ int cxio_destroy_cq(struct cxio_rdev *rdev_p, struct t3_cq 
*cq)
dma_free_coherent((rdev_p-rnic_info.pdev-dev),
  (1UL  (cq-size_log2))
  * sizeof(struct t3_cqe), cq-queue,
- pci_unmap_addr(cq, mapping));
+ dma_unmap_addr(cq, mapping));
cxio_hal_put_cqid(rdev_p-rscp, cq-cqid);
return err;
 }
@@ -335,7 +335,7 @@ int cxio_destroy_qp(struct cxio_rdev *rdev_p, struct t3_wq 
*wq,
dma_free_coherent((rdev_p-rnic_info.pdev-dev),
  (1UL  (wq-size_log2))
  * sizeof(union t3_wr), wq-queue,
- pci_unmap_addr(wq, mapping));
+ dma_unmap_addr(wq, mapping));
kfree(wq-sq);
cxio_hal_rqtpool_free(rdev_p, wq-rq_addr, (1UL  wq-rq_size_log2));
kfree(wq-rq);
@@ -536,7 +536,7 @@ static int cxio_hal_init_ctrl_qp(struct cxio_rdev *rdev_p)
err = -ENOMEM;
goto err;
}
-   pci_unmap_addr_set(rdev_p-ctrl_qp, mapping,
+   dma_unmap_addr_set(rdev_p-ctrl_qp, mapping,
   rdev_p-ctrl_qp.dma_addr);
rdev_p-ctrl_qp.doorbell = (void __iomem *)rdev_p-rnic_info.kdb_addr;
memset(rdev_p-ctrl_qp.workq, 0,
@@ -582,7 +582,7 @@ static int cxio_hal_destroy_ctrl_qp(struct cxio_rdev 
*rdev_p)
dma_free_coherent((rdev_p-rnic_info.pdev-dev),
  (1UL  T3_CTRL_QP_SIZE_LOG2)
  * sizeof(union t3_wr), rdev_p-ctrl_qp.workq,
- pci_unmap_addr(rdev_p-ctrl_qp, mapping));
+ dma_unmap_addr(rdev_p-ctrl_qp, mapping));
return cxio_hal_clear_qp_ctx(rdev_p, T3_CTRL_QP_ID);
 }
 
diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.h 
b/drivers/infiniband/hw/cxgb3/cxio_hal.h
index 073373c..8f0caf7 100644
--- a/drivers/infiniband/hw/cxgb3/cxio_hal.h
+++ b/drivers/infiniband/hw/cxgb3/cxio_hal.h
@@ -71,7 +71,7 @@ struct cxio_hal_ctrl_qp {
wait_queue_head_t waitq;/* wait for RspQ/CQE msg */
union t3_wr *workq; /* the work request queue */
dma_addr_t dma_addr;/* pci bus address of the workq */
-   DECLARE_PCI_UNMAP_ADDR(mapping)
+   DEFINE_DMA_UNMAP_ADDR(mapping);
void __iomem *doorbell;
 };
 
diff --git a/drivers/infiniband/hw/cxgb3/cxio_wr.h 
b/drivers/infiniband/hw/cxgb3/cxio_wr.h
index 15073b2..e5ddb63 100644
--- a/drivers/infiniband/hw/cxgb3/cxio_wr.h
+++ b/drivers/infiniband/hw/cxgb3/cxio_wr.h
@@ -691,7 +691,7 @@ struct t3_swrq {
 struct t3_wq {
union t3_wr *queue; /* DMA accessable memory */
dma_addr_t dma_addr;/* DMA address for HW */
-   DECLARE_PCI_UNMAP_ADDR(mapping) /* unmap kruft */
+   DEFINE_DMA_UNMAP_ADDR(mapping); /* unmap kruft */
u32 error;  /* 1 once we go to ERROR */
u32 qpid;
u32 wptr;   /* idx to next available WR slot */
@@ -718,7 +718,7 @@ struct t3_cq {
u32 wptr;
u32 size_log2;
dma_addr_t dma_addr;
-   DECLARE_PCI_UNMAP_ADDR(mapping)
+   DEFINE_DMA_UNMAP_ADDR(mapping);
struct t3_cqe *queue;
struct t3_cqe *sw_queue;
u32 sw_rptr;
-- 
1.7.0

--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] IB/mthca: use the dma state API instead of the pci equivalents

2010-04-01 Thread FUJITA Tomonori

The DMA API is preferred.

No functional change.

Signed-off-by: FUJITA Tomonori fujita.tomon...@lab.ntt.co.jp
---
 drivers/infiniband/hw/mthca/mthca_allocator.c |8 
 drivers/infiniband/hw/mthca/mthca_eq.c|6 +++---
 drivers/infiniband/hw/mthca/mthca_provider.h  |2 +-
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/mthca/mthca_allocator.c 
b/drivers/infiniband/hw/mthca/mthca_allocator.c
index c5ccc2d..b4e0cf4 100644
--- a/drivers/infiniband/hw/mthca/mthca_allocator.c
+++ b/drivers/infiniband/hw/mthca/mthca_allocator.c
@@ -211,7 +211,7 @@ int mthca_buf_alloc(struct mthca_dev *dev, int size, int 
max_direct,
if (!buf-direct.buf)
return -ENOMEM;
 
-   pci_unmap_addr_set(buf-direct, mapping, t);
+   dma_unmap_addr_set(buf-direct, mapping, t);
 
memset(buf-direct.buf, 0, size);
 
@@ -251,7 +251,7 @@ int mthca_buf_alloc(struct mthca_dev *dev, int size, int 
max_direct,
goto err_free;
 
dma_list[i] = t;
-   pci_unmap_addr_set(buf-page_list[i], mapping, t);
+   dma_unmap_addr_set(buf-page_list[i], mapping, t);
 
clear_page(buf-page_list[i].buf);
}
@@ -289,12 +289,12 @@ void mthca_buf_free(struct mthca_dev *dev, int size, 
union mthca_buf *buf,
 
if (is_direct)
dma_free_coherent(dev-pdev-dev, size, buf-direct.buf,
- pci_unmap_addr(buf-direct, mapping));
+ dma_unmap_addr(buf-direct, mapping));
else {
for (i = 0; i  (size + PAGE_SIZE - 1) / PAGE_SIZE; ++i)
dma_free_coherent(dev-pdev-dev, PAGE_SIZE,
  buf-page_list[i].buf,
- pci_unmap_addr(buf-page_list[i],
+ dma_unmap_addr(buf-page_list[i],
 mapping));
kfree(buf-page_list);
}
diff --git a/drivers/infiniband/hw/mthca/mthca_eq.c 
b/drivers/infiniband/hw/mthca/mthca_eq.c
index 8c31fa3..4bb25bc 100644
--- a/drivers/infiniband/hw/mthca/mthca_eq.c
+++ b/drivers/infiniband/hw/mthca/mthca_eq.c
@@ -503,7 +503,7 @@ static int mthca_create_eq(struct mthca_dev *dev,
goto err_out_free_pages;
 
dma_list[i] = t;
-   pci_unmap_addr_set(eq-page_list[i], mapping, t);
+   dma_unmap_addr_set(eq-page_list[i], mapping, t);
 
clear_page(eq-page_list[i].buf);
}
@@ -578,7 +578,7 @@ static int mthca_create_eq(struct mthca_dev *dev,
if (eq-page_list[i].buf)
dma_free_coherent(dev-pdev-dev, PAGE_SIZE,
  eq-page_list[i].buf,
- pci_unmap_addr(eq-page_list[i],
+ dma_unmap_addr(eq-page_list[i],
 mapping));
 
mthca_free_mailbox(dev, mailbox);
@@ -628,7 +628,7 @@ static void mthca_free_eq(struct mthca_dev *dev,
for (i = 0; i  npages; ++i)
pci_free_consistent(dev-pdev, PAGE_SIZE,
eq-page_list[i].buf,
-   pci_unmap_addr(eq-page_list[i], mapping));
+   dma_unmap_addr(eq-page_list[i], mapping));
 
kfree(eq-page_list);
mthca_free_mailbox(dev, mailbox);
diff --git a/drivers/infiniband/hw/mthca/mthca_provider.h 
b/drivers/infiniband/hw/mthca/mthca_provider.h
index 90f4c4d..596acc4 100644
--- a/drivers/infiniband/hw/mthca/mthca_provider.h
+++ b/drivers/infiniband/hw/mthca/mthca_provider.h
@@ -46,7 +46,7 @@
 
 struct mthca_buf_list {
void *buf;
-   DECLARE_PCI_UNMAP_ADDR(mapping)
+   DEFINE_DMA_UNMAP_ADDR(mapping);
 };
 
 union mthca_buf {
-- 
1.7.0

--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] RDMA/amso1100: use the dma state API instead of the pci equivalents

2010-04-01 Thread FUJITA Tomonori

The DMA API is preferred.

No functional change.

Signed-off-by: FUJITA Tomonori fujita.tomon...@lab.ntt.co.jp
---
 drivers/infiniband/hw/amso1100/c2.h  |2 +-
 drivers/infiniband/hw/amso1100/c2_alloc.c|4 ++--
 drivers/infiniband/hw/amso1100/c2_cq.c   |4 ++--
 drivers/infiniband/hw/amso1100/c2_mq.h   |2 +-
 drivers/infiniband/hw/amso1100/c2_provider.h |2 +-
 drivers/infiniband/hw/amso1100/c2_rnic.c |   12 ++--
 6 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/hw/amso1100/c2.h 
b/drivers/infiniband/hw/amso1100/c2.h
index f7ff66f..6ae698e 100644
--- a/drivers/infiniband/hw/amso1100/c2.h
+++ b/drivers/infiniband/hw/amso1100/c2.h
@@ -250,7 +250,7 @@ struct c2_array {
 struct sp_chunk {
struct sp_chunk *next;
dma_addr_t dma_addr;
-   DECLARE_PCI_UNMAP_ADDR(mapping);
+   DEFINE_DMA_UNMAP_ADDR(mapping);
u16 head;
u16 shared_ptr[0];
 };
diff --git a/drivers/infiniband/hw/amso1100/c2_alloc.c 
b/drivers/infiniband/hw/amso1100/c2_alloc.c
index e911016..31b5a7b 100644
--- a/drivers/infiniband/hw/amso1100/c2_alloc.c
+++ b/drivers/infiniband/hw/amso1100/c2_alloc.c
@@ -50,7 +50,7 @@ static int c2_alloc_mqsp_chunk(struct c2_dev *c2dev, gfp_t 
gfp_mask,
return -ENOMEM;
 
new_head-dma_addr = dma_addr;
-   pci_unmap_addr_set(new_head, mapping, new_head-dma_addr);
+   dma_unmap_addr_set(new_head, mapping, new_head-dma_addr);
 
new_head-next = NULL;
new_head-head = 0;
@@ -82,7 +82,7 @@ void c2_free_mqsp_pool(struct c2_dev *c2dev, struct sp_chunk 
*root)
while (root) {
next = root-next;
dma_free_coherent(c2dev-pcidev-dev, PAGE_SIZE, root,
- pci_unmap_addr(root, mapping));
+ dma_unmap_addr(root, mapping));
root = next;
}
 }
diff --git a/drivers/infiniband/hw/amso1100/c2_cq.c 
b/drivers/infiniband/hw/amso1100/c2_cq.c
index f5c45b1..17b9663 100644
--- a/drivers/infiniband/hw/amso1100/c2_cq.c
+++ b/drivers/infiniband/hw/amso1100/c2_cq.c
@@ -255,7 +255,7 @@ int c2_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags 
notify_flags)
 static void c2_free_cq_buf(struct c2_dev *c2dev, struct c2_mq *mq)
 {
dma_free_coherent(c2dev-pcidev-dev, mq-q_size * mq-msg_size,
- mq-msg_pool.host, pci_unmap_addr(mq, mapping));
+ mq-msg_pool.host, dma_unmap_addr(mq, mapping));
 }
 
 static int c2_alloc_cq_buf(struct c2_dev *c2dev, struct c2_mq *mq, int q_size,
@@ -276,7 +276,7 @@ static int c2_alloc_cq_buf(struct c2_dev *c2dev, struct 
c2_mq *mq, int q_size,
   NULL,/* peer (currently unknown) */
   C2_MQ_HOST_TARGET);
 
-   pci_unmap_addr_set(mq, mapping, mq-host_dma);
+   dma_unmap_addr_set(mq, mapping, mq-host_dma);
 
return 0;
 }
diff --git a/drivers/infiniband/hw/amso1100/c2_mq.h 
b/drivers/infiniband/hw/amso1100/c2_mq.h
index acede00..fc1b9a7 100644
--- a/drivers/infiniband/hw/amso1100/c2_mq.h
+++ b/drivers/infiniband/hw/amso1100/c2_mq.h
@@ -71,7 +71,7 @@ struct c2_mq {
u8 __iomem *adapter;
} msg_pool;
dma_addr_t host_dma;
-   DECLARE_PCI_UNMAP_ADDR(mapping);
+   DEFINE_DMA_UNMAP_ADDR(mapping);
u16 hint_count;
u16 priv;
struct c2_mq_shared __iomem *peer;
diff --git a/drivers/infiniband/hw/amso1100/c2_provider.h 
b/drivers/infiniband/hw/amso1100/c2_provider.h
index 1076df2..bf18998 100644
--- a/drivers/infiniband/hw/amso1100/c2_provider.h
+++ b/drivers/infiniband/hw/amso1100/c2_provider.h
@@ -50,7 +50,7 @@
 
 struct c2_buf_list {
void *buf;
-DECLARE_PCI_UNMAP_ADDR(mapping)
+   DEFINE_DMA_UNMAP_ADDR(mapping);
 };
 
 
diff --git a/drivers/infiniband/hw/amso1100/c2_rnic.c 
b/drivers/infiniband/hw/amso1100/c2_rnic.c
index dd05c48..a2e863c 100644
--- a/drivers/infiniband/hw/amso1100/c2_rnic.c
+++ b/drivers/infiniband/hw/amso1100/c2_rnic.c
@@ -523,7 +523,7 @@ int __devinit c2_rnic_init(struct c2_dev *c2dev)
err = -ENOMEM;
goto bail1;
}
-   pci_unmap_addr_set(c2dev-rep_vq, mapping, c2dev-rep_vq.host_dma);
+   dma_unmap_addr_set(c2dev-rep_vq, mapping, c2dev-rep_vq.host_dma);
pr_debug(%s rep_vq va %p dma %llx\n, __func__, q1_pages,
 (unsigned long long) c2dev-rep_vq.host_dma);
c2_mq_rep_init(c2dev-rep_vq,
@@ -544,7 +544,7 @@ int __devinit c2_rnic_init(struct c2_dev *c2dev)
err = -ENOMEM;
goto bail2;
}
-   pci_unmap_addr_set(c2dev-aeq, mapping, c2dev-aeq.host_dma);
+   dma_unmap_addr_set(c2dev-aeq, mapping, c2dev-aeq.host_dma);
pr_debug(%s aeq va %p dma %llx\n, __func__, q2_pages,
 (unsigned long long) c2dev-aeq.host_dma);
c2_mq_rep_init(c2dev-aeq,
@@ -595,11 +595,11 @@ int __devinit c2_rnic_init(struct

Fork safe clarification

[PATCH 26/26] rdma/ucm: allow user space to specify AF_IB when joining multicast

[PATCH 25/26] rdma/ucm: allow user space to pass AF_IB into resolve

[PATCH 24/26] rdma/ucm: allow user space to bind to AF_IB

[PATCH 23/26] rdma/ucm: name changes to indicate only IP addresses supported

[PATCH 22/26] rdma/ucm: add ability to query GID addresses

[PATCH 21/26] rdma/cm: export cma_get_service_id

[PATCH 20/26] rdma/ucm: support querying when IB paths are not reversible

[PATCH 19/26] ib/sa: export function to pack a path record into wire format

[PATCH 13/26] rdma/cm: add support for AF_IB to rdma_resolve_addr

[PATCH 12/26] rdma/cm: restrict AF_IB loopback to binding to IB devices only

[PATCH 11/26] rdma/cm: do not modify sa_family when setting loopback address

[PATCH 10/26] rdma/cm: Allow user to specify AF_IB when binding

[PATCH 9/26] rdma/cm: set qkey for port space IB

[PATCH 8/26] rdma/ucm: change event handling based on port space

[PATCH 6/26] rdma/cm: specify QP type when creating an ID

[PATCH 5/26] rdma/cm: update port reservation to support AF_IB

[PATCH 4/26] ib/addr: add AF_IB support to ip_addr_size

[PATCH 3/26] rdma/cm: include AF_IB in loopback and any address checks

[PATCH 2/26] rdma/cm: fix handling of ipv6 addressing in cma_use_port

[PATCH 1/26] rdma/cm: define native IB address

[PATCH 0/26] rdma/cm: add support for native IB addressing

Re: [PATCH 9/26] rdma/cm: set qkey for port space IB

RE: [PATCH 26/26] rdma/ucm: allow user space to specify AF_IB when joining multicast

RE: [PATCH 9/26] rdma/cm: set qkey for port space IB

Re: nfsrdma broken on 2.6.34-rc1?

Re: [PATCH 9/26] rdma/cm: set qkey for port space IB

Re: [PATCH 26/26] rdma/ucm: allow user space to specify AF_IB when joining multicast

[PATCH,RFC] nfsd: Make INET6 transport creation failure an informational message

Re: [PATCH,RFC] nfsd: Make INET6 transport creation failure an informational message

[PATCH,RFC] nfsd: Make INET6 transport creation failure an informational message

[PATCH] RDMA/cxgb3: use the dma state API instead of the pci equivalents

[PATCH] IB/mthca: use the dma state API instead of the pci equivalents

[PATCH] RDMA/amso1100: use the dma state API instead of the pci equivalents

34 matches

Site Navigation

Mail list logo

Footer information