Re: [ewg] Re: [PATCH] uDAPL release note update for OFED 1.5
On 12/16/2009 11:52 AM, Vladimir Sokolovsky wrote: Davis, Arlin R wrote: Vlad/Tziporet, Please apply patch for OFED 1.5 docs. --- Update uDAPL OFED 1.5 release notes to include new features and bug fixes. Add explanation about each provider (cma, scm, ucm) and the pros/cons. Signed-off-by: Arlin Davis arlin.r.da...@intel.com --- Applied, Regards, Vladimir What about Sean's comments? Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] uDAPL release note update for OFED 1.5
Davis, Arlin R wrote: Vlad/Tziporet, Please apply patch for OFED 1.5 docs. --- Update uDAPL OFED 1.5 release notes to include new features and bug fixes. Add explanation about each provider (cma, scm, ucm) and the pros/cons. Signed-off-by: Arlin Davis arlin.r.da...@intel.com --- Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] uDAPL release note update for OFED 1.5
+ 2. SCM - uses sockets to exchange QP information. IPoIB, ARP, and SA queries NOT required. This is only true if all nodes are connected with a second networking fabric. + 3. UCM - use's IB UD QP to exchange QP info. Sockets, ARP, IPoIB, and SA queries NOT required. Same as above. Requires out of band (i.e. socket) connections to exchange UD QP information. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
On Thu, Dec 10, 2009 at 02:49:45PM -0700, Jason Gunthorpe wrote:
- Change the library API for ibv_port_attr to include a link_layer
- Change the kernel API to retrieve link_layer
- Add ibv_cmd_get_mac and other stuff to support RDMAoE
- Update verbs examples to support RDMAoE
- Update man pages (you missed these)
Will be addressed in the next patch set.
--- a/include/infiniband/kern-abi.h
+++ b/include/infiniband/kern-abi.h
@@ -46,7 +46,7 @@
* The minimum and maximum kernel ABI that we can handle.
*/
#define IB_USER_VERBS_MIN_ABI_VERSION 1
-#define IB_USER_VERBS_MAX_ABI_VERSION 6
+#define IB_USER_VERBS_MAX_ABI_VERSION 7
Whats this about? That seems like it needs a much bigger review,
changing the kernel ABI version instantly breaks every existing
libibverbs, shouldn't be done without alot of discussion!!
I think we can do without chagning the ABI version so I am going to
ommit it in the next patch set.
Extra include?
Yes, thanks.
@@ -86,6 +86,7 @@ default_symver(__ibv_query_device, ibv_query_device);
int __ibv_query_port(struct ibv_context *context, uint8_t port_num,
struct ibv_port_attr *port_attr)
{
+ port_attr-link_layer = IBV_LINK_LAYER_UNSPECIFIED;
return context-ops.query_port(context, port_num, port_attr);
}
Seems like this should be just
return ___ibv_query_port(context,port_num,port_attr);
A leftover, thanks.
diff --git a/drivers/infiniband/core/uverbs_cmd.c
b/drivers/infiniband/core/uverbs_cmd.c
index 012aadf..d592bd2 100644
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -452,7 +452,8 @@ ssize_t ib_uverbs_query_port(struct ib_uverbs_file
*file,
resp.active_width= attr.active_width;
resp.active_speed= attr.active_speed;
resp.phys_state = attr.phys_state;
- resp.transport = attr.transport;
+ resp.transport = attr.transport == RDMA_TRANSPORT_RDMAOE ?
+ IB_LINK_LAYER_ETHERNET : IB_LINK_LAYER_INFINIBAND;
Are you going to change the kernel patches to use the new link_layer
name?
Yes, in the next patch set.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
On Thu, Dec 10, 2009 at 01:50:00PM -0800, Sean Hefty wrote:
@@ -306,7 +314,7 @@ static struct pingpong_context *pp_init_ctx(struct
ibv_device *ib_dev, int size,
return NULL;
}
-memset(ctx-buf, 0, size);
+memset(ctx-buf, 0x7b + is_server, size);
ctx-context = ibv_open_device(ib_dev);
if (!ctx-context) {
@@ -481,6 +489,7 @@ static void usage(const char *argv0)
printf( -n, --iters=itersnumber of exchanges (default 1000)\n);
printf( -l, --sl=sl service level value\n);
printf( -e, --events sleep on CQ events (default poll)\n);
+printf( -g, --gid=remote gid gid of the other port\n);
It seems easier for the user if the tests discovered its local GID and
exchanged
it with the remote side like it does with the LID and QPN.
Will be changed in the next patch set -- the user will specify the
index to the GID table of the local port.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
On Mon, Dec 14, 2009 at 10:41:27AM +0200, Eli Cohen wrote:
On Thu, Dec 10, 2009 at 01:50:00PM -0800, Sean Hefty wrote:
@@ -306,7 +314,7 @@ static struct pingpong_context *pp_init_ctx(struct
ibv_device *ib_dev, int size,
return NULL;
}
- memset(ctx-buf, 0, size);
+ memset(ctx-buf, 0x7b + is_server, size);
ctx-context = ibv_open_device(ib_dev);
if (!ctx-context) {
@@ -481,6 +489,7 @@ static void usage(const char *argv0)
printf( -n, --iters=itersnumber of exchanges (default 1000)\n);
printf( -l, --sl=sl service level value\n);
printf( -e, --events sleep on CQ events (default poll)\n);
+ printf( -g, --gid=remote gid gid of the other port\n);
It seems easier for the user if the tests discovered its local GID and
exchanged
it with the remote side like it does with the LID and QPN.
Will be changed in the next patch set -- the user will specify the
index to the GID table of the local port.
It would be nice if the tools would consistently let you choose the
source device based on gid.. I've been meaning to make a patch for
verbs to add a ibv_open_device_by_gid() function or something like
that.
Using gid idx is pretty unfriendly ...
But it is just a test prorgam, no worries.
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
On Thu, Dec 10, 2009 at 07:05:36PM +0200, Eli Cohen wrote:
here is the patch I prepared based on the discussions we had. The patch is
based on the last
rdmaoe/libibverbs patch I sent. libmlx4 was modified too, a trivial
change that changes name. Both fixes were push to OFED. I will send a
new patch set in a few days.
Could you prepare this based on Roland's tree? This patch won't apply.
Why are things still going into OFED so quickly? Shouldn't Roland
accept this stuff before it gets to OFED - or at least review it once..
+static inline int ___ibv_query_port(struct ibv_context *context,
+ uint8_t port_num,
+ struct ibv_port_attr *port_attr)
+{
+ uint8_t *padp;
+ int padsize;
+
+ port_attr-link_layer = IBV_LINK_LAYER_UNSPECIFIED;
+ padp = port_attr-link_layer + sizeof port_attr-link_layer;
+ padsize = sizeof(int) - ((unsigned long)padp (sizeof(int) - 1));
+ memset(padp, 0, padsize);
+
+ return context-ops.query_port(context, port_num, port_attr);
+}
You should explicity declare the padding, like ibv_query_port_resp
does, the above is very fragile.
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
On Thu, Dec 10, 2009 at 10:33:53AM -0700, Jason Gunthorpe wrote:
Could you prepare this based on Roland's tree? This patch won't apply.
I quote two patches, one for libibverbs based on 74638ac, and the
other for libmlx4 based on 444f634. I changed the padding handling as
you requested for libibverbs. You also need to apply a patch to the
kernel driver to match the new values for link_layer. I put it here
too.
libibverbs:
diff --git a/examples/devinfo.c b/examples/devinfo.c
index 84f95c7..393ec04 100644
--- a/examples/devinfo.c
+++ b/examples/devinfo.c
@@ -184,6 +184,19 @@ static int print_all_port_gids(struct ibv_context *ctx,
uint8_t port_num, int tb
return rc;
}
+static const char *link_layer_str(uint8_t link_layer)
+{
+ switch (link_layer) {
+ case IBV_LINK_LAYER_UNSPECIFIED:
+ case IBV_LINK_LAYER_INFINIBAND:
+ return IB;
+ case IBV_LINK_LAYER_ETHERNET:
+ return Ethernet;
+ default:
+ return Unknown;
+ }
+}
+
static int print_hca_cap(struct ibv_device *ib_dev, uint8_t ib_port)
{
struct ibv_context *ctx;
@@ -284,6 +297,7 @@ static int print_hca_cap(struct ibv_device *ib_dev, uint8_t
ib_port)
printf(\t\t\tsm_lid:\t\t\t%d\n, port_attr.sm_lid);
printf(\t\t\tport_lid:\t\t%d\n, port_attr.lid);
printf(\t\t\tport_lmc:\t\t0x%02x\n, port_attr.lmc);
+ printf(\t\t\tlink_layer:\t\t%s\n,
link_layer_str(port_attr.link_layer));
if (verbose) {
printf(\t\t\tmax_msg_sz:\t\t0x%x\n,
port_attr.max_msg_sz);
diff --git a/examples/pingpong.c b/examples/pingpong.c
index b916f59..d4a46e4 100644
--- a/examples/pingpong.c
+++ b/examples/pingpong.c
@@ -31,6 +31,8 @@
*/
#include pingpong.h
+#include arpa/inet.h
+#include stdlib.h
enum ibv_mtu pp_mtu_to_enum(int mtu)
{
@@ -53,3 +55,10 @@ uint16_t pp_get_local_lid(struct ibv_context *context, int
port)
return attr.lid;
}
+
+int pp_get_port_info(struct ibv_context *context, int port,
+struct ibv_port_attr *attr)
+{
+ return ibv_query_port(context, port, attr);
+}
+
diff --git a/examples/pingpong.h b/examples/pingpong.h
index 71d7c3f..16d3466 100644
--- a/examples/pingpong.h
+++ b/examples/pingpong.h
@@ -37,5 +37,7 @@
enum ibv_mtu pp_mtu_to_enum(int mtu);
uint16_t pp_get_local_lid(struct ibv_context *context, int port);
+int pp_get_port_info(struct ibv_context *context, int port,
+struct ibv_port_attr *attr);
#endif /* IBV_PINGPONG_H */
diff --git a/examples/rc_pingpong.c b/examples/rc_pingpong.c
index fa969e0..4d0bd0d 100644
--- a/examples/rc_pingpong.c
+++ b/examples/rc_pingpong.c
@@ -67,6 +67,8 @@ struct pingpong_context {
int size;
int rx_depth;
int pending;
+ struct ibv_port_attr portinfo;
+ union ibv_giddgid;
};
struct pingpong_dest {
@@ -94,6 +96,12 @@ static int pp_connect_ctx(struct pingpong_context *ctx, int
port, int my_psn,
.port_num = port
}
};
+
+ if (ctx-dgid.global.interface_id) {
+ attr.ah_attr.is_global = 1;
+ attr.ah_attr.grh.hop_limit = 1;
+ attr.ah_attr.grh.dgid = ctx-dgid;
+ }
if (ibv_modify_qp(ctx-qp, attr,
IBV_QP_STATE |
IBV_QP_AV |
@@ -289,11 +297,11 @@ out:
static struct pingpong_context *pp_init_ctx(struct ibv_device *ib_dev, int
size,
int rx_depth, int port,
- int use_event)
+ int use_event, int is_server)
{
struct pingpong_context *ctx;
- ctx = malloc(sizeof *ctx);
+ ctx = calloc(1, sizeof *ctx);
if (!ctx)
return NULL;
@@ -306,7 +314,7 @@ static struct pingpong_context *pp_init_ctx(struct
ibv_device *ib_dev, int size,
return NULL;
}
- memset(ctx-buf, 0, size);
+ memset(ctx-buf, 0x7b + is_server, size);
ctx-context = ibv_open_device(ib_dev);
if (!ctx-context) {
@@ -481,6 +489,7 @@ static void usage(const char *argv0)
printf( -n, --iters=itersnumber of exchanges (default 1000)\n);
printf( -l, --sl=sl service level value\n);
printf( -e, --events sleep on CQ events (default poll)\n);
+ printf( -g, --gid=remote gid gid of the other port\n);
}
int main(int argc, char *argv[])
@@ -504,6 +513,7 @@ int main(int argc, char *argv[])
int rcnt, scnt;
int num_cq_events = 0;
int sl = 0;
+ char*grh = NULL;
srand48(getpid() * time(NULL));
@@ -520,10 +530,11 @@ int
Re: [ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
On Thu, Dec 10, 2009 at 11:14:55PM +0200, Eli Cohen wrote:
On Thu, Dec 10, 2009 at 10:33:53AM -0700, Jason Gunthorpe wrote:
Could you prepare this based on Roland's tree? This patch won't apply.
I quote two patches, one for libibverbs based on 74638ac, and the
other for libmlx4 based on 444f634. I changed the padding handling as
you requested for libibverbs. You also need to apply a patch to the
kernel driver to match the new values for link_layer. I put it here
too.
Seems like this will work to me. I think you need to split this patch
if you want Roland to apply it. I'd suggest
- Change the library API for ibv_port_attr to include a link_layer
- Change the kernel API to retrieve link_layer
- Add ibv_cmd_get_mac and other stuff to support RDMAoE
- Update verbs examples to support RDMAoE
- Update man pages (you missed these)
--- a/include/infiniband/kern-abi.h
+++ b/include/infiniband/kern-abi.h
@@ -46,7 +46,7 @@
* The minimum and maximum kernel ABI that we can handle.
*/
#define IB_USER_VERBS_MIN_ABI_VERSION 1
-#define IB_USER_VERBS_MAX_ABI_VERSION 6
+#define IB_USER_VERBS_MAX_ABI_VERSION 7
Whats this about? That seems like it needs a much bigger review,
changing the kernel ABI version instantly breaks every existing
libibverbs, shouldn't be done without alot of discussion!!
+enum {
+ IBV_LINK_LAYER_UNSPECIFIED,
+ IBV_LINK_LAYER_INFINIBAND,
+ IBV_LINK_LAYER_ETHERNET,
+};
Why do you have IBV_LINK_LAYER_UNSPECIFIED ? That seems pointless. Why
should existing kernels return UNSPECIFIED when we know they are
always IB. I'd say drop IBV_LINK_LAYER_UNSPECIFIED and set
IBV_LINK_LAYER_INFINIBAND to 0.
What are iWarp devices going to return? Seems like the verbs library
should probably force link_layer to ethernet for iwarp devices, for
compatability.
diff --git a/src/cmd.c b/src/cmd.c
index cbd5288..5183d59 100644
+++ b/src/cmd.c
@@ -162,6 +162,7 @@ int ibv_cmd_query_device(struct ibv_context *context,
return 0;
}
+#include stdio.h
int ibv_cmd_query_port(struct ibv_context *context, uint8_t port_num,
struct ibv_port_attr *port_attr,
struct ibv_query_port *cmd, size_t cmd_size)
Extra include?
@@ -86,6 +86,7 @@ default_symver(__ibv_query_device, ibv_query_device);
int __ibv_query_port(struct ibv_context *context, uint8_t port_num,
struct ibv_port_attr *port_attr)
{
+ port_attr-link_layer = IBV_LINK_LAYER_UNSPECIFIED;
return context-ops.query_port(context, port_num, port_attr);
}
Seems like this should be just
return ___ibv_query_port(context,port_num,port_attr);
diff --git a/drivers/infiniband/core/uverbs_cmd.c
b/drivers/infiniband/core/uverbs_cmd.c
index 012aadf..d592bd2 100644
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -452,7 +452,8 @@ ssize_t ib_uverbs_query_port(struct ib_uverbs_file *file,
resp.active_width= attr.active_width;
resp.active_speed= attr.active_speed;
resp.phys_state = attr.phys_state;
- resp.transport = attr.transport;
+ resp.transport = attr.transport == RDMA_TRANSPORT_RDMAOE ?
+ IB_LINK_LAYER_ETHERNET : IB_LINK_LAYER_INFINIBAND;
Are you going to change the kernel patches to use the new link_layer
name?
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
RE: [ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
@@ -306,7 +314,7 @@ static struct pingpong_context *pp_init_ctx(struct
ibv_device *ib_dev, int size,
return NULL;
}
- memset(ctx-buf, 0, size);
+ memset(ctx-buf, 0x7b + is_server, size);
ctx-context = ibv_open_device(ib_dev);
if (!ctx-context) {
@@ -481,6 +489,7 @@ static void usage(const char *argv0)
printf( -n, --iters=itersnumber of exchanges (default 1000)\n);
printf( -l, --sl=sl service level value\n);
printf( -e, --events sleep on CQ events (default poll)\n);
+ printf( -g, --gid=remote gid gid of the other port\n);
It seems easier for the user if the tests discovered its local GID and exchanged
it with the remote side like it does with the LID and QPN.
diff --git a/include/infiniband/kern-abi.h b/include/infiniband/kern-abi.h
index 0db083a..8ef8844 100644
/*
@@ -223,7 +224,8 @@ struct ibv_query_port_resp {
__u8 active_width;
__u8 active_speed;
__u8 phys_state;
- __u8 reserved[3];
+ __u8 link_layer;
+ __u8 reserved[2];
};
Should we define the network layer too? Right now we have IB transport, which
assumes IB network and link; iWarp transport, which almost assumes IP network
and Ethernet; and RDMAoE, which may or may not have a network (but requires the
L3 address) and Ethernet (or is it DCB) link.
- Sean
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
#define IB_USER_VERBS_MIN_ABI_VERSION 1 -#define IB_USER_VERBS_MAX_ABI_VERSION 6 +#define IB_USER_VERBS_MAX_ABI_VERSION 7 Whats this about? That seems like it needs a much bigger review, changing the kernel ABI version instantly breaks every existing libibverbs, shouldn't be done without alot of discussion!! Yes, I've been meaning to bring this up earlier. There was a time, long ago, when changing this ABI was maybe OK. I think even when I first designed the uverbs ABI, having this version instead of capability bits was a goof. But I don't have a time machine. In any case I think at this point we need to stick with ABI version 6 will full backwards compat unless we really really can't. And I don't think the changes here are nearly drastic enough that we can't figure out a way forward without breaking things. Why do you have IBV_LINK_LAYER_UNSPECIFIED ? That seems pointless. Why should existing kernels return UNSPECIFIED when we know they are always IB. I'd say drop IBV_LINK_LAYER_UNSPECIFIED and set IBV_LINK_LAYER_INFINIBAND to 0. I prefer having the UNSPECIFIED. Not a strong preference but my reasoning is that if you have an old kernel that doesn't answer anything, you're better off letting the app see that. And there's no reason why iWARP has to run over ethernet in principle. Maybe I'm wrong but I don't like setting don't know magically to IB behind the scenes. - R. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] rdmaoe/libibverbs: handle binary compatibility
On Thu, Dec 10, 2009 at 01:57:11PM -0800, Roland Dreier wrote: Maybe I'm wrong but I don't like setting don't know magically to IB behind the scenes. Well, it isn't just don't know it also means the kernel doesn't support the link_layer query. The kernels that don't support link_layer also only support link_layer == IB for IBV_TRANSPORT_IB. One proves the other.. So IBV_TRANSPORT_IB ports should always return IBV_LINK_LAYER_INFINIBAND unless the kernel supports the new API and says otherwise. You can still have IBV_LINK_LAYER_UNSPECIFIED, but it can't be 0. If this isn't done then again compatability with apps is compromised, what should an app do when it sees IBV_LINK_LAYER_UNSPECIFIED due to an older kernel and newer libibverbs? And there's no reason why iWARP has to run over ethernet in principle. Right.. so existing kernels with the new library should return IBV_LINK_LAYER_INFINIBAND for ports on IBV_TRANSPORT_IB devices and IBV_LINK_LAYER_UNSPECIFIED for ports on everything else (IBV_TRANSPORT_IWARP, IBV_TRANSPORT_UNKNOWN) Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] docs: Update NFSRDMA SLES10 comment
Jon Mason wrote: Update the NFS SLES10 comment to reflect the rnfs-utils rpm updating the nfsserver scripts. Signed-off-by: Jon Mason j...@opengridcomputing.com --- nfs-rdma.release-notes.txt |5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/nfs-rdma.release-notes.txt b/nfs-rdma.release-notes.txt index a4369e4..f876908 100644 --- a/nfs-rdma.release-notes.txt +++ b/nfs-rdma.release-notes.txt @@ -187,8 +187,9 @@ NFS/RDMA Setup rpc.statd by default. However, the in-kernel lockd that was in SLES10 has been removed in the new kernels. Since OFED is back-porting the new code to the older distro's, there is no in-kernel lockd in SLES10 and the SLES10 -nfsserver scripts do not know the need to start it. Therefore, the user -MUST start rpc.statd. +nfsserver scripts do not know the need to start it. Therefore, the +nfsserver scripts will be modified when the rnfs-utils rpm is installed to +start/stop rpc.statd. - On the client system Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] docs: Document NFSRDMA mlx4 firmware requirement
Jon Mason wrote: Per Bug 1815, Document the NFSRDMA requirement for ConnectX 2.7.0 firmware. Signed-off-by: Jon Mason j...@opengridcomputing.com --- nfs-rdma.release-notes.txt |5 + 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/nfs-rdma.release-notes.txt b/nfs-rdma.release-notes.txt index f876908..0e966e3 100644 --- a/nfs-rdma.release-notes.txt +++ b/nfs-rdma.release-notes.txt @@ -219,3 +219,8 @@ a 64KB page size (like PPC64 and IA64 systems) and your server is using a 4KB page size (like i386 and X86_64), then you need to mount the server using rsize=32768,wsize=32768 to avoid overrunning the Chelsio RNIC fast register limits. This is a known firmware limitation in the Chelsio RNIC. + +Running NFSRDMA over Mellanox's ConnectX HCA requires that the adapter firmware +be 2.7.0 or greater on all NFS clients and servers. Firmware 2.6.0 has known +issues that prevent the RDMA connection from being established. Firmware 2.7.0 +has resolved these issues. Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] IB/qib: fix IPoIB device stop deadlock
Ralph Campbell wrote: Vlad, please pull 4 more fixes to the qib driver from: git://git.openfabrics.org/~ralphc/linux-2.6/.git ofed_kernel_1_5 commit 70f277ec61809c15b3352aa6b650882ebef05235 Author: Ralph Campbell (QLogic) ral...@hosting.openfabrics.org Date: Tue Dec 8 17:14:43 2009 -0800 IB/qib: fix IPoIB device stop deadlock We create our own workqueue mainly because we want to be able to flush it when devices are being removed. We can't use schedule_work()/flush_scheduled_work() because both unregister_netdev() and linkwatch_event take the rtnl lock, so flush_scheduled_work() can deadlock during device removal. Signed-off-by: Mitko Haralanov mi...@qlogic.com commit 9e829f8742501afefc49c1b16945322cd6577dfd Author: Ralph Campbell (QLogic) ral...@hosting.openfabrics.org Date: Tue Dec 8 17:12:02 2009 -0800 IB/qib: serdes changes for QME7342 serdes Because we now have different values of H1 all over, I redid the h1_vals code that we weren't really using, changed it to a single h1_val, and initialized for the board types that need different values. Signed-off-by: Dave Olson dave.ol...@qlogic.com commit 52cb89d80698f6cfdf7d58f0b0bbef85cd70dc69 Author: Ralph Campbell (QLogic) ral...@hosting.openfabrics.org Date: Tue Dec 8 17:11:22 2009 -0800 IB/qib: added missing code to report if 7322 memory BIST failed Don't clear the memory built-in-self-test error bit so it gets reported. Signed-off-by: Dave Olson dave.ol...@qlogic.com commit 56d291199ea1479a441af974ab3f311b4703c897 Author: Ralph Campbell (QLogic) ral...@hosting.openfabrics.org Date: Tue Dec 8 17:10:35 2009 -0800 IB/qib: improve twsi error messages for human beings Some people were confused by TWSI, so make messages somewhat clearer. Signed-off-by: Dave Olson dave.ol...@qlogic.com Done, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] mlx4_core: Cleanup bug in __mlx4_init_one()
On Wed, Dec 09, 2009 at 02:35:18PM -0800, Roland Dreier wrote: Looks like a valid fix but your patch doesn't apply. I don't seem to have mlx4_cleanup_counters_table() in my tree. I noticed that already. I'll send a new patch in my next working day. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] docs: update nes release notes for OFED 1.5
Chien Tung wrote: Signed-off-by: Chien Tung chien.tin.t...@intel.com --- nes_release_notes.txt | 100 ++--- 1 files changed, 53 insertions(+), 47 deletions(-) Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] docs: update ehca release notes for OFED-1.5
Alexander Schmidt wrote: Hi Vlad, Signed-off-by: Alexander Schmidt al...@linux.vnet.ibm.com --- ehca_release_notes.txt | 33 + 1 file changed, 17 insertions(+), 16 deletions(-) Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
RE: [ewg] Re: [PATCH 7/9] rdma/cm: fix loopback address support
On Tue, 2009-11-24 at 09:28 -0800, Sean Hefty wrote: Pradeep, earlier you wanted to hear the rdma-cm maintainer opinion on inclusion of this 2.6.33-rc1 patch set (which isn't finalized by-the-way, Jason still has fixes to submit), any reason to skip it now? I agree with Or lets have Sean's blessing before we take them to OFED 1.5 I have no issue with someone adding it to OFED. I will create a new librdmacm package that corresponds with the changes. - Sean I have ported the patches series that Sean sent earlier to rc3. I am currently testing them and hope to post them later today. Sean, is the patch series (9 patches) you sent earlier still current? Jason, do you have any updates? Dave ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
RE: [ewg] Re: [PATCH 7/9] rdma/cm: fix loopback address support
Sean, is the patch series (9 patches) you sent earlier still current? Jason, do you have any updates? Roland added at least one change to fix the build when ipv6 is not enabled. I would check his for-next branch to see what he actually has queued. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH 7/9] rdma/cm: fix loopback address support
Sean Hefty sean.he...@intel.com wrote: I will create a new librdmacm package that corresponds with the changes I made all my testing of the patch set with librdmacm 1.0.10 and patched 2.6.32-rc5 kernel, where as I wrote you, I was focusing on AF_INET/PS_TCP and AF_INET/PS_IPOIB. I understand that Dave was covering AF_INET6/PS_TCP with plenty of the ipv6 variations. So what will this new librdmacm package will let cover which wasn't possible so far? do you refer to ipv6 support in mckey? anything else? Or. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH 7/9] rdma/cm: fix loopback address support
So what will this new librdmacm package will let cover which wasn't possible so far? do you refer to ipv6 support in mckey? anything else? Changes were your changes to mckey, plus changes Dave added to cmatose to support IPv6. The actual library itself hasn't been modified. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH 7/9] rdma/cm: fix loopback address support
Changes were your changes to mckey, plus changes Dave added to cmatose to support IPv6. The actual library itself hasn't been modified. okay, got it. I was under the impression that mckey still misses an option to get from the user an ipv6 multicast address which isn't all zeros nor unmapped, correct? or the -m option will work with both ipv4 and ipv6 addresses? Or. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH 7/9] rdma/cm: fix loopback address support
okay, got it. I was under the impression that mckey still misses an option to get from the user an ipv6 multicast address which isn't all zeros nor unmapped, correct? or the -m option will work with both ipv4 and ipv6 addresses? The -m and -b options work with IPv4 and IPv6 addresses. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH 7/9] rdma/cm: fix loopback address support
ewg-boun...@lists.openfabrics.org wrote on 11/22/2009 02:36:32 AM: Pradeep Satyanarayana wrote: Roland Dreier wrote: Thanks... in any case I applied all 9 of the patches in this series. Thanks for pulling all this together. Sean, Thanks a lot for pulling it all together. Can we consider including this into OFED-1.5 too? Pradeep Pradeep If someone will send a patch to Vlad we can add it to OFED 1.5 Tziporet, Sure, we will do it. Thanks Pradeep prad...@us.ibm.com___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH 7/9] rdma/cm: fix loopback address support
Pradeep Satyanarayana wrote: Roland Dreier wrote: Thanks... in any case I applied all 9 of the patches in this series. Thanks for pulling all this together. Sean, Thanks a lot for pulling it all together. Can we consider including this into OFED-1.5 too? Pradeep Pradeep If someone will send a patch to Vlad we can add it to OFED 1.5 Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH 7/9] rdma/cm: fix loopback address support
Roland Dreier wrote: Thanks... in any case I applied all 9 of the patches in this series. Thanks for pulling all this together. Sean, Thanks a lot for pulling it all together. Can we consider including this into OFED-1.5 too? Pradeep ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] IB/perftest: increase CQ size to avoid overflow
Ralph Campbell wrote:
When running the ib_send_bw test bidirectional, there can be
ctx-rx_depth + ctx-tx_depth/2 number of outstanding posted work
queue entries but the (shared) completion queue is only size
ctx-rx_depth. This can lead to completions being lost and the test
hanging. Since memory isn't limited in the microbenchmark test and
I think the ctx-tx_depth/2 send limit should probably be ctx-tx_depth,
make the CQ size be ctx-rx_depth + ctx-tx_depth.
Signed-off-by: Ralph Campbell ralph.campb...@qlogic.com
diff --git a/send_bw.c b/send_bw.c
index f842fb9..9624cfe 100644
--- a/send_bw.c
+++ b/send_bw.c
@@ -489,7 +489,8 @@ static struct pingpong_context *pp_init_ctx(struct
ibv_device *ib_dev,
}
}
- ctx-cq = ibv_create_cq(ctx-context, ctx-rx_depth, NULL, ctx-channel, 0);
+ ctx-cq = ibv_create_cq(ctx-context, ctx-rx_depth + ctx-tx_depth,
+ NULL, ctx-channel, 0);
if (!ctx-cq) {
fprintf(stderr, Couldn't create CQ\n);
return NULL;
Applied .
Thanks Ralph.
Regards,
Ido
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] link-local address fix for rdma_resolve_addr
On Thu, 2009-10-22 at 15:02 -0700, Sean Hefty wrote:
For ipv6 I ran what I described previously. What I do need to do is add
the option to rping to specify a source address and run it with various
address. Any help you can give defining what exactly needs to be tested
would be appreciated.
You can also test with ucmatose to verify ipv4 still works. Use the -b option
to bind to a specific address.
- Sean
Sean
This patch adds ipv6 support to ucmatose.
Signed-off-by: David Wilder dwil...@us.ibm.com
--
examples/cmatose.c | 31 +++
1 files changed, 19 insertions(+), 12 deletions(-)
diff --git a/examples/cmatose.c b/examples/cmatose.c
index 8c12347..481a6d0 100644
--- a/examples/cmatose.c
+++ b/examples/cmatose.c
@@ -516,15 +516,15 @@ static int get_addr(char *dst, struct sockaddr_in *addr)
return ret;
}
- if (res-ai_family != PF_INET) {
+ if (res-ai_family == PF_INET)
+ memcpy(addr, res-ai_addr, sizeof(struct sockaddr_in));
+ else if (res-ai_family == PF_INET6)
+ memcpy(addr, res-ai_addr, sizeof(struct sockaddr_in6));
+ else
ret = -1;
- goto out;
- }
- *addr = *(struct sockaddr_in *) res-ai_addr;
-out:
- freeaddrinfo(res);
- return ret;
+freeaddrinfo(res);
+return ret;
}
static int run_server(void)
@@ -543,11 +543,18 @@ static int run_server(void)
ret = get_addr(src_addr, test.src_in);
if (ret)
goto out;
- } else
+ if(test.src_in.sin_family == AF_INET)
+ ((struct sockaddr_in *) test.src_in)-sin_port = port;
+ else
+ ((struct sockaddr_in6 *) test.src_in)-sin6_port=port;
+
+ } else {
test.src_in.sin_family = PF_INET;
+ test.src_in.sin_port = port;
+ }
+
+ ret = rdma_bind_addr(listen_id, (struct sockaddr *)test.src_in);
- test.src_in.sin_port = port;
- ret = rdma_bind_addr(listen_id, test.src_addr);
if (ret) {
perror(cmatose: bind address failed);
goto out;
@@ -628,8 +635,8 @@ static int run_client(void)
printf(cmatose: connecting\n);
for (i = 0; i connections; i++) {
ret = rdma_resolve_addr(test.nodes[i].cma_id,
- src_addr ? test.src_addr : NULL,
- test.dst_addr, 2000);
+ src_addr ? (struct sockaddr *)test.src_in : NULL,
+ (struct sockaddr *)test.dst_in, 2000);
if (ret) {
perror(cmatose: failure getting addr);
connect_error();
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] re: [PATCH] librdmacm/cmatose: add support for ipv6
This patch adds ipv6 support to ucmatose. Signed-off-by: David Wilder dwil...@us.ibm.com Thanks. I pulled this patch into my local git tree with just a couple of minor cleanups. What other patches, if any, did you use to test with it? - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] librdmacm/cmatose: add support for ipv6
From: David J. Wilder [dwil...@us.ibm.com]
Signed-off-by: David Wilder dwil...@us.ibm.com
Signed-off-by: Sean Hefty sean.he...@intel.com
---
We need to update struct cmatest to allow storing an IPv6 address,
or we can overrun the buffer.
Running with this patch, the client causes a kernel bug on 2.6.31 in
addr_resolve_local, so we'll still fixes for that. It also meant that
I couldn't test this patch.
examples/cmatose.c | 53
1 files changed, 29 insertions(+), 24 deletions(-)
diff --git a/examples/cmatose.c b/examples/cmatose.c
index 8c12347..84831ec 100644
--- a/examples/cmatose.c
+++ b/examples/cmatose.c
@@ -75,10 +75,7 @@ struct cmatest {
int connects_left;
int disconnects_left;
- struct sockaddr_in dst_in;
- struct sockaddr *dst_addr;
- struct sockaddr_in src_in;
- struct sockaddr *src_addr;
+ struct rdma_addraddr;
};
static struct cmatest test;
@@ -505,7 +502,7 @@ static int migrate_channel(struct rdma_cm_id *listen_id)
return ret;
}
-static int get_addr(char *dst, struct sockaddr_in *addr)
+static int get_addr(char *dst, struct sockaddr *addr)
{
struct addrinfo *res;
int ret;
@@ -516,15 +513,15 @@ static int get_addr(char *dst, struct sockaddr_in *addr)
return ret;
}
- if (res-ai_family != PF_INET) {
+ if (res-ai_family == PF_INET)
+ memcpy(addr, res-ai_addr, sizeof(struct sockaddr_in));
+ else if (res-ai_family == PF_INET6)
+ memcpy(addr, res-ai_addr, sizeof(struct sockaddr_in6));
+ else
ret = -1;
- goto out;
- }
- *addr = *(struct sockaddr_in *) res-ai_addr;
-out:
- freeaddrinfo(res);
- return ret;
+freeaddrinfo(res);
+return ret;
}
static int run_server(void)
@@ -540,14 +537,21 @@ static int run_server(void)
}
if (src_addr) {
- ret = get_addr(src_addr, test.src_in);
+ ret = get_addr(src_addr, test.addr.src_addr);
if (ret)
goto out;
- } else
- test.src_in.sin_family = PF_INET;
+ if (test.addr.src_addr.sa_family == AF_INET)
+ ((struct sockaddr_in *) test.addr.src_addr)-sin_port
= port;
+ else
+ ((struct sockaddr_in6 *)
test.addr.src_addr)-sin6_port = port;
+
+ } else {
+ test.addr.src_addr.sa_family = PF_INET;
+ ((struct sockaddr_in *) test.addr.src_addr)-sin_port = port;
+ }
+
+ ret = rdma_bind_addr(listen_id, test.addr.src_addr);
- test.src_in.sin_port = port;
- ret = rdma_bind_addr(listen_id, test.src_addr);
if (ret) {
perror(cmatose: bind address failed);
goto out;
@@ -614,22 +618,25 @@ static int run_client(void)
printf(cmatose: starting client\n);
if (src_addr) {
- ret = get_addr(src_addr, test.src_in);
+ ret = get_addr(src_addr, test.addr.src_addr);
if (ret)
return ret;
}
- ret = get_addr(dst_addr, test.dst_in);
+ ret = get_addr(dst_addr, test.addr.dst_addr);
if (ret)
return ret;
- test.dst_in.sin_port = port;
+ if (test.addr.dst_addr.sa_family == AF_INET)
+ ((struct sockaddr_in *) test.addr.dst_addr)-sin_port = port;
+ else
+ ((struct sockaddr_in6 *) test.addr.dst_addr)-sin6_port = port;
printf(cmatose: connecting\n);
for (i = 0; i connections; i++) {
ret = rdma_resolve_addr(test.nodes[i].cma_id,
- src_addr ? test.src_addr : NULL,
- test.dst_addr, 2000);
+ src_addr ? test.addr.src_addr : NULL,
+ test.addr.dst_addr, 2000);
if (ret) {
perror(cmatose: failure getting addr);
connect_error();
@@ -717,8 +724,6 @@ int main(int argc, char **argv)
}
}
- test.dst_addr = (struct sockaddr *) test.dst_in;
- test.src_addr = (struct sockaddr *) test.src_in;
test.connects_left = connections;
test.disconnects_left = connections;
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] re: [PATCH] librdmacm/cmatose: add support for ipv6
On Tue, 2009-10-27 at 11:58 -0700, Sean Hefty wrote:
This patch adds ipv6 support to ucmatose.
Signed-off-by: David Wilder dwil...@us.ibm.com
Thanks.
I pulled this patch into my local git tree with just a couple of minor
cleanups.
What other patches, if any, did you use to test with it?
- Sean
I have been testing ucmatose against today's ofed 1.5 build, with the
patch I posted earlier with fixes to resolve_local() and
ipv6_resolve_remote().
This patch is still missing the binding changes Jason suggested it just
fixes the existing code. I did not see the panic you saw on 2.6.32, but
I have run on that kernel yet.
Dave..
Patch (posted earlier)
rdma_resolve_addr() returns an error when attempting to resolve ipv6
link-local address. This patch fixes the handling of link-local address.
Signed-off-by: David Wilder dwil...@us.ibm.com
--
drivers/infiniband/core/addr.c | 25 ++---
1 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c
index bd07803..3442256 100644
--- a/drivers/infiniband/core/addr.c
+++ b/drivers/infiniband/core/addr.c
@@ -278,6 +278,21 @@ static int addr6_resolve_remote(struct sockaddr_in6
*src_in,
fl.nl_u.ip6_u.daddr = dst_in-sin6_addr;
fl.nl_u.ip6_u.saddr = src_in-sin6_addr;
+ if (ipv6_addr_type(src_in-sin6_addr) IPV6_ADDR_LINKLOCAL) {
+ if (!src_in-sin6_scope_id)
+ return -EINVAL;
+ fl.oif = src_in-sin6_scope_id;
+ }
+ if (ipv6_addr_type(dst_in-sin6_addr) IPV6_ADDR_LINKLOCAL) {
+ if (dst_in-sin6_scope_id) {
+ if (fl.oif fl.oif != dst_in-sin6_scope_id)
+ return -EINVAL;
+ fl.oif = dst_in-sin6_scope_id;
+ }
+ if (!fl.oif)
+ return -EINVAL;
+ }
+
dst = ip6_route_output(init_net, NULL, fl);
if (!dst)
return ret;
@@ -390,14 +405,16 @@ static int addr_resolve_local(struct sockaddr *src_in,
case AF_INET6:
{
struct in6_addr *a;
+ int found = 0;
for_each_netdev(init_net, dev)
if (ipv6_chk_addr(init_net,
((struct sockaddr_in6 *)
dst_in)-sin6_addr,
- dev, 1))
+ dev, 1)) {
+ found = 1;
break;
-
- if (!dev)
+ }
+ if (!found)
return -EADDRNOTAVAIL;
a = ((struct sockaddr_in6 *) src_in)-sin6_addr;
@@ -406,6 +423,8 @@ static int addr_resolve_local(struct sockaddr *src_in,
src_in-sa_family = dst_in-sa_family;
((struct sockaddr_in6 *) src_in)-sin6_addr =
((struct sockaddr_in6 *) dst_in)-sin6_addr;
+ ((struct sockaddr_in6 *) src_in)-sin6_scope_id =
+ ((struct sockaddr_in6 *) dst_in)-sin6_scope_id;
ret = rdma_copy_addr(addr, dev, dev-dev_addr);
} else if (ipv6_addr_loopback(a)) {
ret = rdma_translate_ip(dst_in, addr);
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, 2009-10-21 at 17:08 -0600, Jason Gunthorpe wrote: This looks exactly like what I was thinking of - have you tested this? Yes I did do some testing, but that brings up a good question. I am not sure I know what all should be tested? I am running rping with different destination address (and scoping). On the ipv4 side: rping -c -a ip-of-my-ib0-interface rping -c -a ip-of-remote-nodes-ib0-interface For ipv6 I ran what I described previously. What I do need to do is add the option to rping to specify a source address and run it with various address. Any help you can give defining what exactly needs to be tested would be appreciated. If it is OK, I'd make it the first in the series. There were two things I was not sure about in my example. 1) Is 'init_net.loopback_dev' the correct reference for the loop device? Or is it something like dev_net(rt-idev-dev)-loopback_dev ? I'm sensing it may be the latter, but can't investigate right now Donno much about this new namespace stuff really I think you may be correct I will look at that closer. I did explicitly verify the test worked in both cases. 2) Was rt-idev-dev the right choice for the ipv4 case? Or is it rt-u.dst.dev ? The TCP case kinda looks like int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tmp = ip_route_connect(rt, nexthop, inet-saddr, RT_CONN_FLAGS(sk), sk-sk_bound_dev_if, IPPROTO_TCP, inet-sport, usin-sin_port, sk, 1); sk_setup_caps(sk, rt-u.dst); void sk_setup_caps(struct sock *sk, struct dst_entry *dst) __sk_dst_set(sk, dst); And all later things key off the sk_get_dst. So I'm thinking that u.dst.dev might be correct. I have no idea what the difference is though (can't look too hard right now) The main other fixup I see is to remove ret = cma_bind_addr(id, src_addr, dst_addr); From rdma_resolve_addr and rely on the routing lookup in addr_resolve_remote called by addr_resolve_ip to setup the bind device from the routing lookup. (This is what I mentioned in my last email) Which then lets you fixup the checking and handling of the sin6_scopeid on the source address - and fixes the main other routing difference against the TCP stack. Thanks for working on this! Jason Lots of discussion :) I will go through the mails, address the comments and post the entire series of patches. Thanks for all your input. Dave. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] link-local address fix for rdma_resolve_addr
For ipv6 I ran what I described previously. What I do need to do is add the option to rping to specify a source address and run it with various address. Any help you can give defining what exactly needs to be tested would be appreciated. You can also test with ucmatose to verify ipv4 still works. Use the -b option to bind to a specific address. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, Oct 21, 2009 at 03:30:29PM -0700, David J. Wilder wrote: addr6_resolve_remote(). The second patch fixes the bugs in addr_resolve_local(). The 3d patch I am posting now for discussion. This patch, as Jason's suggested, moves the function of addr_resolve_local() into addr4_resolve_remote() and addr6_resolve_remote(). It eliminates the need for addr_resolve_local(). This looks exactly like what I was thinking of - have you tested this? If it is OK, I'd make it the first in the series. There were two things I was not sure about in my example. 1) Is 'init_net.loopback_dev' the correct reference for the loop device? Or is it something like dev_net(rt-idev-dev)-loopback_dev ? I'm sensing it may be the latter, but can't investigate right now Donno much about this new namespace stuff really 2) Was rt-idev-dev the right choice for the ipv4 case? Or is it rt-u.dst.dev ? The TCP case kinda looks like int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tmp = ip_route_connect(rt, nexthop, inet-saddr, RT_CONN_FLAGS(sk), sk-sk_bound_dev_if, IPPROTO_TCP, inet-sport, usin-sin_port, sk, 1); sk_setup_caps(sk, rt-u.dst); void sk_setup_caps(struct sock *sk, struct dst_entry *dst) __sk_dst_set(sk, dst); And all later things key off the sk_get_dst. So I'm thinking that u.dst.dev might be correct. I have no idea what the difference is though (can't look too hard right now) The main other fixup I see is to remove ret = cma_bind_addr(id, src_addr, dst_addr); From rdma_resolve_addr and rely on the routing lookup in addr_resolve_remote called by addr_resolve_ip to setup the bind device from the routing lookup. (This is what I mentioned in my last email) Which then lets you fixup the checking and handling of the sin6_scopeid on the source address - and fixes the main other routing difference against the TCP stack. Thanks for working on this! Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] link-local address fix for rdma_resolve_addr
This patch, as Jason's suggested, moves the function of addr_resolve_local()
into addr4_resolve_remote()
and addr6_resolve_remote(). It eliminates the need for addr_resolve_local().
One quick comment, remove '_remote' from function names:
addr4_resolve_remote, addr6_resolve_remote, and addr_resolve_remote
drivers/infiniband/core/addr.c | 99 +++
1 files changed, 18 insertions(+), 81 deletions(-)
diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c
index bd07803..f7a5861 100644
--- a/drivers/infiniband/core/addr.c
+++ b/drivers/infiniband/core/addr.c
@@ -233,6 +233,14 @@ static int addr4_resolve_remote(struct sockaddr_in
*src_in,
if (ret)
goto out;
+ if (rt-idev-dev == init_net.loopback_dev){
+ ret = rdma_translate_ip((struct sockaddr *)dst_in, addr);
+ if (!ret)
+ memcpy(addr-dst_dev_addr, addr-src_dev_addr,
+ MAX_ADDR_LEN);
+ goto put;
+ }
This doesn't end up doing the same thing as what resolve_local did. It only
matches up with the 'else if' portion below:
-static int addr_resolve_local(struct sockaddr *src_in,
-struct sockaddr *dst_in,
-struct rdma_dev_addr *addr)
-{
- struct net_device *dev;
- int ret;
-
- switch (dst_in-sa_family) {
- case AF_INET:
- {
- __be32 src_ip = ((struct sockaddr_in *)
src_in)-sin_addr.s_addr;
- __be32 dst_ip = ((struct sockaddr_in *)
dst_in)-sin_addr.s_addr;
-
- dev = ip_dev_find(init_net, dst_ip);
- if (!dev)
- return -EADDRNOTAVAIL;
-
- if (ipv4_is_zeronet(src_ip)) {
- src_in-sa_family = dst_in-sa_family;
- ((struct sockaddr_in *) src_in)-sin_addr.s_addr =
dst_ip;
- ret = rdma_copy_addr(addr, dev, dev-dev_addr);
- } else if (ipv4_is_loopback(src_ip)) {
- ret = rdma_translate_ip(dst_in, addr);
- if (!ret)
- memcpy(addr-dst_dev_addr, dev-dev_addr,
MAX_ADDR_LEN);
- } else {
- ret = rdma_translate_ip(src_in, addr);
- if (!ret)
- memcpy(addr-dst_dev_addr, dev-dev_addr,
MAX_ADDR_LEN);
- }
We need to handle the case where the source address is not given and provide
one. We also need to handle the case where the source address is given, but may
not match the destination address, or even use the same RDMA device.
I didn't look at the ipv6 changes yet.
- Sean
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, Oct 21, 2009 at 04:17:01PM -0700, Sean Hefty wrote:
This doesn't end up doing the same thing as what resolve_local did. It only
matches up with the 'else if' portion below:
It does all the behaviors, it passes the input into a routing lookup
- though it is true the source needs to be passed into the routing
lookup, which I think David had in another patch. Dave; I guess you
need to include that here now.
So, first case:
-if (ipv4_is_zeronet(src_ip)) {
Looks like:
$ ip route get 10.0.0.11
local 10.0.0.11 dev lo src 10.0.0.11
cache local mtu 16436 advmss 16396 hoplimit 64
trips through the if == loopback and does
'rdma_translate_ip(10.0.0.11)'
Same as the old code (little different path, but seems to give the
same result)
Second case:
-} else if (ipv4_is_loopback(src_ip)) {
Looks like:
$ ip route get 10.0.0.11 from 127.0.0.1
local 10.0.0.11 from 127.0.0.1 dev lo
cache local mtu 16436 advmss 16396 hoplimit 64
so trips through the if == loopback and does rdma_translate_ip(10.0.0.11)
Same as the old code
Third case:
-} else {
$ ip route get 10.0.0.11 from 10.0.0.11
local 10.0.0.11 from 10.0.0.11 dev lo
cache local mtu 16436 advmss 16396 hoplimit 64
Again, does rdma_translate_ip(10.0.0.11)
And the weird case is different:
$ ip route get 10.0.0.11 from 192.168.122.1
local 10.0.0.11 from 192.168.122.1 dev lo
cache local mtu 16436 advmss 16396 hoplimit 64
(192.168.122.1 is bound to a different device on my system than
10.0.0.11)
The new case trips the if == loopback and does
rdma_translate_ip(10.0.0.11)
The old case does rdma_translate_ip(192.168.122.1)
I don't think this is a significant difference, both behaviors are
reasonable choices and the code/complexity savings are worth it, IMHO.
[Alternatively, I suppose one could call rdma_translate_ip(rt-rt_src)
and if that fails then do rdma_translate_ip(dst_in) but why bother?]
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, Oct 21, 2009 at 05:36:39PM -0600, Jason Gunthorpe wrote:
- if (ipv4_is_zeronet(src_ip)) {
Looks like:
$ ip route get 10.0.0.11
local 10.0.0.11 dev lo src 10.0.0.11
cache local mtu 16436 advmss 16396 hoplimit 64
trips through the if == loopback and does
'rdma_translate_ip(10.0.0.11)'
Same as the old code (little different path, but seems to give the
same result)
Oops, there is a little woopsie here:
Dave:
+ if (rt-idev-dev == init_net.loopback_dev){
+ ret = rdma_translate_ip((struct sockaddr *)dst_in, addr);
+ if (!ret)
+ memcpy(addr-dst_dev_addr, addr-src_dev_addr,
+ MAX_ADDR_LEN);
+ goto put;
+ }
+
The 'goto put' will skip over the source address assignment step.
Maybe move this:
if (!src_ip) {
src_in-sin_family = dst_in-sin_family;
src_in-sin_addr.s_addr = rt-rt_src;
}
Up above your if (rt-idev)
And similarly for v6.
Also add a src_in = rt-rt_src assignment for v6.
I'd also remove the test for 0 address, just do it unconditionally
(IIRC routing table always returns src if src is not 0)
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] link-local address fix for rdma_resolve_addr
$ ip route get 10.0.0.11 from 192.168.122.1 local 10.0.0.11 from 192.168.122.1 dev lo cache local mtu 16436 advmss 16396 hoplimit 64 (192.168.122.1 is bound to a different device on my system than 10.0.0.11) The new case trips the if == loopback and does rdma_translate_ip(10.0.0.11) The old case does rdma_translate_ip(192.168.122.1) I don't think this is a significant difference, both behaviors are reasonable choices and the code/complexity savings are worth it, IMHO. [Alternatively, I suppose one could call rdma_translate_ip(rt-rt_src) and if that fails then do rdma_translate_ip(dst_in) but why bother?] I agree this usage case is weird, but I'm still not sure about the patch. If the destination service is listening on 10.0.0.11, then the listen is bound to a different gid than the source gid that we're trying to connect from. The src_dev_addr and dst_dev_addr need to reflect this, so that a PR gives us a path that routes the CM REQ correctly. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] link-local address fix for rdma_resolve_addr
The local loopback case uses PRs? Yes - the rdma cm makes no distinction when resolve route is called. It does a PR query. Even so, it still seems OK to me: Path: addr4_resolve_remote $ ip route get 10.0.0.11 from 192.168.122.1 local 10.0.0.11 from 192.168.122.1 dev lo srcIP = 192.168.122.1 rdma_translate_ip(dst_ip = 10.0.0.11) rdma_copy_addr(eth0); src_dev_addr = eth0.dev_addr (ie GID of 10.0.0.11) memcpy(dst_dev_addr = src_dev_addr) (ie GID of 10.0.0.11) So everthing is bound to the GID of 10.0.0.11 which matches the listen of 10.0.0.11, which seems OK. The source could have called rdma_bind_addr(192.168.122.1) prior to calling rdma_resolve_addr(). (DAPL does this.) This would have returned a different RDMA device than binding to 10.0.0.11. The client app could have allocated resources on that device, but the CM REQ will carry the gid/lid of the other device. The endpoints won't be able to communicate. Yes, it's weird, and may not be optimal, but if a source address is explicitly given, then its mapping to a specific RDMA device should be honored. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, Oct 21, 2009 at 05:40:30PM -0700, Sean Hefty wrote: Even so, it still seems OK to me: Path: addr4_resolve_remote $ ip route get 10.0.0.11 from 192.168.122.1 local 10.0.0.11 from 192.168.122.1 dev lo srcIP = 192.168.122.1 rdma_translate_ip(dst_ip = 10.0.0.11) rdma_copy_addr(eth0); src_dev_addr = eth0.dev_addr (ie GID of 10.0.0.11) memcpy(dst_dev_addr = src_dev_addr) (ie GID of 10.0.0.11) So everthing is bound to the GID of 10.0.0.11 which matches the listen of 10.0.0.11, which seems OK. The source could have called rdma_bind_addr(192.168.122.1) prior to calling rdma_resolve_addr(). (DAPL does this.) This would have returned a different RDMA device than binding to 10.0.0.11. The client app could have allocated resources on that device, but the CM REQ will carry the gid/lid of the other device. The endpoints won't be able to communicate. That is very difficult to fit into the semantics the IP routing model uses :( And it looks like an API problem in DAPL :( So, I see now, you are proposing that in this case the connection attempt to be routed through the network and not looped back.. I actually have a big problem with that, ignoring a 'lo' entry in a routing table is very much not IP like and not a good idea. That should be respected.. I guess I'd much rather see that one situation return EHOSTUNREACH or something. But, I suppose you are going to tell me that Intel MPI uses DAPL to loopback connect to other processes on the same node, and relies on this? :( :( :( Sigh. Anyhow, lets not get side tracked. It seems to me, the easy way out for David's approach is to simply check if the device is already bound via rdma_bind() and if so force it to that device no matter what the routing table lookup returns. Can you suggest a reliable way to make that check? [What happens now if I do this: rdma_bind(10.0.0.11) rdma_resolve_addr(src = 192.168.122.1 dst = 10.0.0.11) Does the cma_bind path check that it is already bound and give out an error? too late for me to check] Once the cma_bind for rdma_resolve_addr is moved into the addr_resolve_remote function then people using the API without calling bind on the client path will get sane IP-like behavior. Yes, it's weird, and may not be optimal, but if a source address is explicitly given, then its mapping to a specific RDMA device should be honored. Remember, on Linux the IP is *not* attached to a device, it is part of the host itself. So the idea that a source address somehow specifies a RDMA device does not fit into the Linux IP networking model. Unfortunately the definition of rdma_bind kinda bakes this mismatched model into the API :( Truth be told, to fit the Linux IP model, the RDMA CM should have provided exactly only two ways to bind a cm_id to a specific device - rdma_accept and rdma_resolve_addr. Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] link-local address fix for rdma_resolve_addr
That is very difficult to fit into the semantics the IP routing model uses :( And it looks like an API problem in DAPL :( This depends on if your view is that the rdma cm is trying to match IP routing, trying to use IP addresses as convenient names for RDMA ports, or something in between that may lean more one way or the other depending on the device type. IMO, the primary reason for using IP addressing over IB is more for convenience, than compliance. So, I see now, you are proposing that in this case the connection attempt to be routed through the network and not looped back.. I actually have a big problem with that, ignoring a 'lo' entry in a routing table is very much not IP like and not a good idea. That should be respected.. I hesitate comparing RDMA versus IP too closely. Sigh. Anyhow, lets not get side tracked. It seems to me, the easy way out for David's approach is to simply check if the device is already bound via rdma_bind() and if so force it to that device no matter what the routing table lookup returns. Can you suggest a reliable way to make that check? I'm not entirely sure of the best way to test this. Checking the src_dev_addr is my first thought. [What happens now if I do this: rdma_bind(10.0.0.11) rdma_resolve_addr(src = 192.168.122.1 dst = 10.0.0.11) Does the cma_bind path check that it is already bound and give out an error? too late for me to check] rdma_resolve_addr only calls bind if the rdma id is not already bound. The src_addr simply gets ignored in this case, and the bound address is used instead. Once the cma_bind for rdma_resolve_addr is moved into the addr_resolve_remote function then people using the API without calling bind on the client path will get sane IP-like behavior. I like the approach, I'm just concerned about the case where the app has requested a specific source address, which today means that a specific RDMA device should be used. Remember, on Linux the IP is *not* attached to a device, it is part of the host itself. So the idea that a source address somehow specifies a RDMA device does not fit into the Linux IP networking model. I would say we're extending the linux networking model to incorporate this concept, while staying in our imposed little RDMA sand box world. Even iWarp has these issues. Truth be told, to fit the Linux IP model, the RDMA CM should have provided exactly only two ways to bind a cm_id to a specific device - rdma_accept and rdma_resolve_addr. I think this is more restrictive than things need to be. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Mon, Oct 19, 2009 at 03:47:09PM -0700, David J. Wilder wrote:
+++ b/drivers/infiniband/core/addr.c
@@ -278,6 +278,21 @@ static int addr6_resolve_remote(struct sockaddr_in6
*src_in,
fl.nl_u.ip6_u.daddr = dst_in-sin6_addr;
fl.nl_u.ip6_u.saddr = src_in-sin6_addr;
+ if (ipv6_addr_type(src_in-sin6_addr) IPV6_ADDR_LINKLOCAL) {
+ if (!src_in-sin6_scope_id)
+ return -EINVAL;
+ fl.oif = src_in-sin6_scope_id;
+ }
Seeing it all together like this make it clear this test needs to move
up the call chain and test the sockaddr passed from userspace, not
the one created by addr_resolve_local. Probably somewhere along the
rdma_resolve_addr - cma_bind_addr - rmda_bind_addr -
rdma_translate_ip path. Maybe rdma_translate_ip should use and check
the scope as a temporary hack?
BTW, while researching the above comment, I'm not certain your last
patch is at all correct:
commit 85f20b39fd44310a163a9b33708fea57f08a4e40
RDMA/addr: Fix resolution of local IPv6 addresses
This patch allows a local IPv6 address to be resolved by rdma_cm.
To reproduce the problem:
$ rping -s -v -a ::0
$ rping -c -v -a IPv6 address local to this system
--- a/drivers/infiniband/core/addr.c
+++ b/drivers/infiniband/core/addr.c
@@ -393,7 +393,7 @@ static int addr_resolve_local(struct sockaddr *src_in,
for_each_netdev(init_net, dev)
if (ipv6_chk_addr(init_net,
- ((struct sockaddr_in6 *)
addr)-sin6_addr,
+ ((struct sockaddr_in6 *)
dst_in)-sin6_addr,
dev, 1))
break;
I can believe it fixes the case you describe (ie loopback) but
matching the *dest* IP against the local interface's IP list cannot
possibly be right.
The primary problem is that for_each_netdev/ipv6_chk_addr is NOT the
same as ip_dev_find. ip_dev_find is a routing lookup, ipv6_chk_addr
compares the local address list. Not at all the same. I don't see a
route lookup helper for ipv6, so you have to code full flowi lookup.
With your change I expect ipv6 is 100% broken now for non loop cases?
Regards,
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] RE: [PATCH] link-local address fix for rdma_resolve_addr
@@ -393,7 +393,7 @@ static int addr_resolve_local(struct sockaddr *src_in, for_each_netdev(init_net, dev) if (ipv6_chk_addr(init_net, - ((struct sockaddr_in6 *) addr)- sin6_addr, + ((struct sockaddr_in6 *) dst_in)- sin6_addr, dev, 1)) break; I can believe it fixes the case you describe (ie loopback) but matching the *dest* IP against the local interface's IP list cannot possibly be right. The intent is to see if the destination address is local. A source address may not be given. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Mon, Oct 19, 2009 at 04:47:59PM -0700, Sean Hefty wrote: @@ -393,7 +393,7 @@ static int addr_resolve_local(struct sockaddr *src_in, for_each_netdev(init_net, dev) if (ipv6_chk_addr(init_net, - ((struct sockaddr_in6 *) addr)- sin6_addr, + ((struct sockaddr_in6 *) dst_in)- sin6_addr, dev, 1)) break; I can believe it fixes the case you describe (ie loopback) but matching the *dest* IP against the local interface's IP list cannot possibly be right. The intent is to see if the destination address is local. A source address may not be given. Well, that makes more sense, but it still pretty strange to match the IP list like that, the proper thing is to query RT6_TABLE_LOCAL, like the IPv4 case does. Anyhow, couldn't the whole addr_resolve_local routine be replaced with something like this in addr_resolve_remote: if (rt-idev == init_net-loopback_dev) rdma_translate_ip(rt-rt_src, dev_addr, NULL); for IPv4 and similar for IPv6? That does query the proper RT_TABLEs to determine if the IP is local and then we get the searching and ip_dev_find only for the case where the address is definitely looped back. Much closer to how the IP stack works normally. Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Thu, 2009-10-15 at 15:32 -0600, Jason Gunthorpe wrote: The actual fixing to the code is not hard, remove rdma_translate_ip, addr_resolve_local, split addr_resolve_remote into a part to resolve the route and a part that does the arp/nd. Make the route resolve part work almost exactly like addr4_resolve_remote (noting that the v6 version is wrong, since is doesn't respect unset source addres, another bug). Call rdma_copy_addr based on the rt-idev-dev (or should it be odev??). Do the ARP. The pain is in retesting everything :| Jason There is the wrinkle in this plan. If you pass ip6_route_output() your own link-local address (scoped or not) it returns a neighbor entry bound to the loop back device. $ ip route get fe80::202:c903:1:28ed oif ib0 local fe80::202:c903:1:28ed from :: via :: dev lo table local proto none src fe80::202:c903:1:28ed metric 0 mtu 16436 advmss 16376 hoplimit 4294967295 Thats not going to work as lo has no RDMA support. I suspect this is why addr_resolve_local() was added in the first place :) I will post a patch with our first solution and include the fix to addr_resolve_local(). Dave. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Fri, Oct 16, 2009 at 11:54:39AM -0700, David J. Wilder wrote: There is the wrinkle in this plan. If you pass ip6_route_output() your own link-local address (scoped or not) it returns a neighbor entry bound to the loop back device. Well, that really is correct. The RDMA CM could try to do something smart if lo is returned from the route lookup - ie choose the first RDMA device bound to that IP or something. There are other problems with the loop back case, like the Linux stack doesn't generate neighbor entries for its own IP addresses. The loopback case is far less important than the other policy routing features anyhow.. I will post a patch with our first solution and include the fix to addr_resolve_local(). May as well, a little bit of progress is better than none at all. Also note that I observed that the IPv6 code path never sets the source address from the routing table if IN6_ADDR_ANY is used as source, this is certainly wrong and could cause other problems down the line, probably more so for iwarp than IB though.. Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, 2009-10-14 at 13:09 Jason Gunthorpe wrote:
So, it tries to match the source addr to the addrs bound to the
device, which is wrong - that isn't how the ip stack works.
You can patch this up a little bit by fixing up addr_resolve_local to
set sin6_scope_ip.
I found the bug in addr_resolve_local(). (more comments below)
--- addr.c.1759 2009-10-13 15:57:48.0 -0500
+++ addr.c.ip_local 2009-10-15 14:03:50.0 -0500
@@ -390,14 +390,17 @@ static int addr_resolve_local(struct soc
case AF_INET6:
{
struct in6_addr *a;
+ int found = 0;
for_each_netdev(init_net, dev)
if (ipv6_chk_addr(init_net,
((struct sockaddr_in6 *)
dst_in)-sin6_addr,
- dev, 1))
+ dev, 1)){
+ found = 1;
break;
+ }
- if (!dev)
+ if (!found)
return -EADDRNOTAVAIL;
a = ((struct sockaddr_in6 *) src_in)-sin6_addr;
@@ -406,6 +409,8 @@ static int addr_resolve_local(struct soc
src_in-sa_family = dst_in-sa_family;
((struct sockaddr_in6 *) src_in)-sin6_addr =
((struct sockaddr_in6 *) dst_in)-sin6_addr;
+ ((struct sockaddr_in6 *) src_in)-sin6_scope_id =
+((struct sockaddr_in6 *)
dst_in)-sin6_scope_id;
ret = rdma_copy_addr(addr, dev, dev-dev_addr);
} else if (ipv6_addr_loopback(a)) {
ret = rdma_translate_ip(dst_in, addr);
But really the correct thing to do is to remove addr_resolve_local and
place the source address into the struct flowi and use the result of
the route lookup to bind to the source device, and set the source
address if it is unset.
Sorry I don't get it..
Are you saying that ip6_route_output() will resolve the address even if
it is a link-local address bound to my own interface? Therefor
addr_resolve_local() is not needed.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Thu, Oct 15, 2009 at 12:27:21PM -0700, David J. Wilder wrote: On Wed, 2009-10-14 at 13:09 Jason Gunthorpe wrote: So, it tries to match the source addr to the addrs bound to the device, which is wrong - that isn't how the ip stack works. You can patch this up a little bit by fixing up addr_resolve_local to set sin6_scope_ip. I found the bug in addr_resolve_local(). (more comments below) Yes, that is the hacky work around I was mentioning.. But really the correct thing to do is to remove addr_resolve_local and place the source address into the struct flowi and use the result of the route lookup to bind to the source device, and set the source address if it is unset. Sorry I don't get it.. Are you saying that ip6_route_output() will resolve the address even if it is a link-local address bound to my own interface? Therefor addr_resolve_local() is not needed. Yes, and more. In Linux the routing table takes as input the source (optional), device (optional) and destination address and returns as output the device to use. To determine the device to bind to you ask the routing table what device to use for all the route information you have. For example: $ ip route get fe80::c2d from fe80::213:72ff:fe29:e65d oif eth0 fe80::c2d via fe80::c2d dev eth0 src fe80::213:72ff:fe29:e65d metric 0 cache mtu 1500 advmss 1440 hoplimit 4294967295 $ ip route get fe80::c2d oif eth0 fe80::c2d via fe80::c2d dev eth0 src fe80::213:72ff:fe29:e65d metric 0 cache mtu 1500 advmss 1440 hoplimit 4294967295 You can see in both cases the routing table returns a 'src' entry. 'src' is the address to bind to if no bind address was specified. When doing link local addresess the sin6_scope_id should sets the 'oif' key in the routing lookup, which will result in the correct src address and output device being selected by the routing algorithm. For instance on my machine here, I have two interfaces: $ ip route get fe80::c2d oif virbr0 fe80::c2d via fe80::c2d dev virbr0 src fe80::2c5d:c4ff:feb8:1ce5 metric 0 cache mtu 1500 advmss 1440 hoplimit 4294967295 As you can see it is returning the link local address for virbr0 as the source. So the algorithm in RDMA CM should look like this: - If src is specified then set the bind local address to src [if src is link local then it must specify sin6_scope_id, and sin6_scope_id becomes the oif input to the route lookup] - If dst is link local then its sin6_scope_id is the oif to the route lookup (and must match src, as we did last go round) - Src (or 'any'), dst and device (or 'any') are passed to the route lookup - The RDMA CM ID is bound to the device returned by the route lookup - If the src address was not specified then the connection source IP is set to the 'src' value from the route lookup. This is why addr_resolve_local/rdma_translate_ip is not needed, that entire entire function is done by the routing table code. You can see why this becomes important when it is combined with policy routing, for instance consider this example: $ ip rule 32765: from 10.0.0.4 lookup dnat $ ip route show table dnat default via 10.0.0.1 dev eth1 $ ip route get 10.0.0.100 10.0.0.100 dev eth0 src 10.0.0.2 $ ip route get 10.0.0.100 from 10.0.0.4 10.0.0.100 from 10.0.0.4 via 10.0.0.1 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 The two results are radically different and dependant on the source address. (10.0.0.4 could be attached to eth0, and eth1!) The actual fixing to the code is not hard, remove rdma_translate_ip, addr_resolve_local, split addr_resolve_remote into a part to resolve the route and a part that does the arp/nd. Make the route resolve part work almost exactly like addr4_resolve_remote (noting that the v6 version is wrong, since is doesn't respect unset source addres, another bug). Call rdma_copy_addr based on the rt-idev-dev (or should it be odev??). Do the ARP. The pain is in retesting everything :| Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
Sean-
This patch should fix the behavior of rdma_resolve_addr when using
link-local addressing.
On Tue, 2009-10-13 at 17:12 -0600, Jason Gunthorpe wrote:
On Tue, Oct 13, 2009 at 03:09:40PM -0700, David J. Wilder wrote:
Here is a patch to addr6_resolve_remote() to correctly handle link-local
address.
It should cover all the conditions Jason described.
Looks pretty good to me, definitely on the right track.
Hmm..
Actually, upon comparing to tcp_ipv6.c, I'd say one more behavior is
necessary. The code in tcp_ipv6 allows the destination to not specify
a scope id if an interface has already been set. Looks like the two
ways to set an interface ID are to use bind() or SO_BINDTODEVICE..
Specifying a source address to RDMA CM is similar to bind(), so if the
source address is link local it must specify a sin6_scope_id and the
dest address can specify 0, or the same value.
Jason -
This new patch should closely emulate tcp_ipv6.c. when both source and
destination scope_ids are given with link-local address.
With this patch rping works as expected.
Link-local with scope:
# /usr/bin/rping -c -a fe80::202:c903:1:1925%ib0
Link-local w/out scope:
# /usr/bin/rping -c -a fe80::202:c903:1:1925
rdma_resolve_addr error -1
Other ipv6 address:
# /usr/bin/rping -c -a 2001:db8:1234::2
(server side started with rping -s -P -v -a ::0)
Signed-off-by: David Wilder dwil...@us.ibm.com
-
--- drivers/infiniband/core/addr.c.1759 2009-10-13 15:57:48.0 -0500
+++ drivers/infiniband/core/addr.c 2009-10-14 10:47:56.0 -0500
@@ -278,6 +278,21 @@ static int addr6_resolve_remote(struct s
fl.nl_u.ip6_u.daddr = dst_in-sin6_addr;
fl.nl_u.ip6_u.saddr = src_in-sin6_addr;
+ if (ipv6_addr_type(dst_in-sin6_addr) IPV6_ADDR_LINKLOCAL){
+ // Link-local address require an interface to be specified.
+ if (!(dst_in-sin6_scope_id||src_in-sin6_scope_id))
+ return -EINVAL;
+
+ // If src and dst interfaces are supplied they must match.
+ if ( (dst_in-sin6_scope_id src_in-sin6_scope_id)
+ (src_in-sin6_scope_id != dst_in-sin6_scope_id) )
+ return -EINVAL;
+ if ( dst_in-sin6_scope_id )
+ fl.oif = dst_in-sin6_scope_id;
+ else
+ fl.oif = src_in-sin6_scope_id;
+ }
+
dst = ip6_route_output(init_net, NULL, fl);
if (!dst)
return ret;
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, Oct 14, 2009 at 09:23:57AM -0700, David J. Wilder wrote:
This new patch should closely emulate tcp_ipv6.c. when both source and
destination scope_ids are given with link-local address.
Maybe like this:
fl.oif = 0;
if (ipv6_addr_type(src_in-sin6_addr) IPV6_ADDR_LINKLOCAL) {
if (!src_in-sin6_scope_id)
return -EINVAL;
fl.oif = src_in-sin6scope_id;
}
if (ipv6_addr_type(dst_in-sin6_addr) IPV6_ADDR_LINKLOCAL){
if (dst_in.sin6_scope_id) {
if (fl.oif fl.oif != dst_in.sin6_scope_id)
return -EINVAL;
fl.oif = dst_in.sin6_scope_id;
}
if (!fl.oif)
return -EINVAL;
}
src and dest have to be tested seperately, the TCPv6 versions don't
require them to be both link local.
Don't forget to run checkpatch ;)
Regards,
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, 2009-10-14 at 11:01 -0600, Jason Gunthorpe wrote:
On Wed, Oct 14, 2009 at 09:23:57AM -0700, David J. Wilder wrote:
This new patch should closely emulate tcp_ipv6.c. when both source and
destination scope_ids are given with link-local address.
Maybe like this:
fl.oif = 0;
if (ipv6_addr_type(src_in-sin6_addr) IPV6_ADDR_LINKLOCAL) {
if (!src_in-sin6_scope_id)
return -EINVAL;
fl.oif = src_in-sin6scope_id;
}
if (ipv6_addr_type(dst_in-sin6_addr) IPV6_ADDR_LINKLOCAL){
if (dst_in.sin6_scope_id) {
if (fl.oif fl.oif != dst_in.sin6_scope_id)
return -EINVAL;
fl.oif = dst_in.sin6_scope_id;
}
if (!fl.oif)
return -EINVAL;
}
src and dest have to be tested seperately, the TCPv6 versions don't
require them to be both link local.
Don't forget to run checkpatch ;)
This looks good. Once concern, it may be obtuse, but if both the src and
dst are link-local addresses should only one need to be scoped? This
patch will required the src to always be scoped when using link local.
Dave...
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, Oct 14, 2009 at 10:30:05AM -0700, David J. Wilder wrote: This looks good. Once concern, it may be obtuse, but if both the src and dst are link-local addresses should only one need to be scoped? This patch will required the src to always be scoped when using link local. The TCPv6 code requires the src to be scoped (or SO_BINDTODEVICE to be used prior). If src is scoped then it is optional to scope dest, but if dest is scoped then it must match. I intended to capture that logic.. Can you give a quick test and send a git format-patch/checkpatch'd patch to Roland, with your signed off line, etc. You can put a Reviewed-by line from me as well if you like. Thanks, Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, 2009-10-14 at 11:01 -0600, Jason Gunthorpe wrote:
On Wed, Oct 14, 2009 at 09:23:57AM -0700, David J. Wilder wrote:
This new patch should closely emulate tcp_ipv6.c. when both source and
destination scope_ids are given with link-local address.
Maybe like this:
fl.oif = 0;
if (ipv6_addr_type(src_in-sin6_addr) IPV6_ADDR_LINKLOCAL) {
if (!src_in-sin6_scope_id)
return -EINVAL;
fl.oif = src_in-sin6scope_id;
}
if (ipv6_addr_type(dst_in-sin6_addr) IPV6_ADDR_LINKLOCAL){
if (dst_in.sin6_scope_id) {
if (fl.oif fl.oif != dst_in.sin6_scope_id)
return -EINVAL;
fl.oif = dst_in.sin6_scope_id;
}
if (!fl.oif)
return -EINVAL;
}
src and dest have to be tested seperately, the TCPv6 versions don't
require them to be both link local.
Hum.. this change is not working with rping. The src address has been
filled in already with a link-local address and no scope_id so it is
returning -EINVAL. Another bug I guess I am doing some more
debug.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Wed, Oct 14, 2009 at 12:33:17PM -0700, David J. Wilder wrote:
On Wed, 2009-10-14 at 11:01 -0600, Jason Gunthorpe wrote:
On Wed, Oct 14, 2009 at 09:23:57AM -0700, David J. Wilder wrote:
This new patch should closely emulate tcp_ipv6.c. when both source and
destination scope_ids are given with link-local address.
Maybe like this:
fl.oif = 0;
if (ipv6_addr_type(src_in-sin6_addr) IPV6_ADDR_LINKLOCAL) {
if (!src_in-sin6_scope_id)
return -EINVAL;
fl.oif = src_in-sin6scope_id;
}
if (ipv6_addr_type(dst_in-sin6_addr) IPV6_ADDR_LINKLOCAL){
if (dst_in.sin6_scope_id) {
if (fl.oif fl.oif != dst_in.sin6_scope_id)
return -EINVAL;
fl.oif = dst_in.sin6_scope_id;
}
if (!fl.oif)
return -EINVAL;
}
src and dest have to be tested seperately, the TCPv6 versions don't
require them to be both link local.
Hum.. this change is not working with rping. The src address has been
filled in already with a link-local address and no scope_id so it is
returning -EINVAL. Another bug I guess I am doing some more
debug.
Ah feh, you are running into that other bug..
http://lists.openfabrics.org/pipermail/general/2009-July/060612.html
Basically it does this:
int rdma_resolve_addr(struct rdma_cm_id *id, struct sockaddr *src_addr,
struct sockaddr *dst_addr, int timeout_ms)
ret = rdma_resolve_ip(addr_client, (struct sockaddr *)
id-route.addr.src_addr,
dst_addr, id-route.addr.dev_addr,
timeout_ms, addr_handler,
id_priv);
int rdma_resolve_ip(struct rdma_addr_client *client,
struct sockaddr *src_addr, struct sockaddr *dst_addr,
struct rdma_dev_addr *addr, int timeout_ms,
void (*callback)(int status, struct sockaddr *src_addr,
struct rdma_dev_addr *addr, void *context),
void *context)
req-status = addr_resolve_local(src_in, dst_in, addr);
static int addr_resolve_local(struct sockaddr *src_in,
struct sockaddr *dst_in,
struct rdma_dev_addr *addr)
dev = ip_dev_find(init_net, dst_ip);
[..]
for_each_netdev(init_net, dev)
if (ipv6_chk_addr(init_net,
((struct sockaddr_in6 *)
addr)-sin6_addr,
dev, 1))
break;
So, it tries to match the source addr to the addrs bound to the
device, which is wrong - that isn't how the ip stack works.
You can patch this up a little bit by fixing up addr_resolve_local to
set sin6_scope_ip.
But really the correct thing to do is to remove addr_resolve_local and
place the source address into the struct flowi and use the result of
the route lookup to bind to the source device, and set the source
address if it is unset.
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] link-local address fix for rdma_resolve_addr
On Tue, Oct 13, 2009 at 03:09:40PM -0700, David J. Wilder wrote:
Here is a patch to addr6_resolve_remote() to correctly handle link-local
address.
It should cover all the conditions Jason described.
Looks pretty good to me, definitely on the right track.
Hmm..
Actually, upon comparing to tcp_ipv6.c, I'd say one more behavior is
necessary. The code in tcp_ipv6 allows the destination to not specify
a scope id if an interface has already been set. Looks like the two
ways to set an interface ID are to use bind() or SO_BINDTODEVICE..
Specifying a source address to RDMA CM is similar to bind(), so if the
source address is link local it must specify a sin6_scope_id and the
dest address can specify 0, or the same value.
How af_inet6 handles bind():
if (addr_type IPV6_ADDR_LINKLOCAL) {
if (addr_len = sizeof(struct sockaddr_in6)
addr-sin6_scope_id) {
/* Override any existing binding, if
another one
* is supplied by user.
*/
sk-sk_bound_dev_if =
addr-sin6_scope_id;
}
/* Binding to link-local address requires an
interface */
if (!sk-sk_bound_dev_if) {
err = -EINVAL;
goto out;
}
dev = dev_get_by_index(net,
sk-sk_bound_dev_if);
if (!dev) {
err = -ENODEV;
goto out;
}
}
How tcp_ipv6 checks the destination address:
if (addr_typeIPV6_ADDR_LINKLOCAL) {
if (addr_len = sizeof(struct sockaddr_in6)
usin-sin6_scope_id) {
/* If interface is set while binding, indices
* must coincide.
*/
if (sk-sk_bound_dev_if
sk-sk_bound_dev_if != usin-sin6_scope_id)
return -EINVAL;
sk-sk_bound_dev_if = usin-sin6_scope_id;
}
/* Connect to link-local address requires an interface */
if (!sk-sk_bound_dev_if)
return -EINVAL;
}
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] perftest: Make rdma_lat, rdma_bw, and clock_test executable names rdma neutral
Hey Hal,
Just applied the 2 patches ( version 2 of the executable_names and
executable permisions)
Regards,
Ido Shaay
Hal Rosenstock wrote:
Hi Ido,
On Sun, Oct 4, 2009 at 6:29 AM, Ido Shamai i...@dev.mellanox.co.il wrote:
Hey Hal ,
Sorry about the delay on response , I've waited for instructions for the
patch.
All is good except the change - ib_clock_test - to - rdma_clock_test,
my manager says clock_test is not a rdma benchmark. so If you can just
remove this change I'll be greatfull
I just submitted v2 of this as requested. Also, there's a minor
outstanding patch on changing the file permissions in the perftest git
repo.
Thanks.
-- Hal
Regards
Ido
Hal Rosenstock wrote:
Since rdma_lat and rdma_bw use RDMA CM, they can be used with both IB and
iWARP so make their executable names neutral (by removing ib_)
IB only tests only require linking with libibverbs
Also, spec file change for executable name changes
Signed-off-by: Hal Rosenstock hal.rosenst...@gmail.com
---
diff --git a/Makefile b/Makefile
index 8042531..83c22c3 100755
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,8 @@
-TESTS = write_bw_postlist rdma_lat rdma_bw send_lat send_bw write_lat
write_bw read_lat read_bw
+RDMACM_TESTS = rdma_lat rdma_bw
+TESTS = write_bw_postlist send_lat send_bw write_lat write_bw read_lat
read_bw
UTILS = clock_test
-all: ${TESTS} ${UTILS}
+all: ${RDMACM_TESTS} ${TESTS} ${UTILS}
CFLAGS += -Wall -g -D_GNU_SOURCE -O2
EXTRA_FILES = get_clock.c
@@ -10,11 +11,18 @@ EXTRA_HEADERS = get_clock.h
LOADLIBES += LDFLAGS +=
-${TESTS}: LOADLIBES += -libverbs -lrdmacm
+${RDMACM_TESTS} ${UTILS}: LOADLIBES += -libverbs -lrdmacm
+${TESTS}: LOADLIBES += -libverbs
-${TESTS} ${UTILS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+${RDMACM_TESTS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+ $(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES}
$(LOADLIBES) $(LDLIBS) -o $@
+${TESTS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES}
$(LOADLIBES) $(LDLIBS) -o ib_$@
+${UTILS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+ $(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES}
$(LOADLIBES) $(LDLIBS) -o rdma_$@
+
clean:
- $(foreach fname,${TESTS} ${UTILS}, rm -f ib_${fname})
+ $(foreach fname,${RDMACM_TESTS} ${UTILS}, rm -f ${fname})
+ $(foreach fname,${TESTS}, rm -f ib_${fname})
.DELETE_ON_ERROR:
.PHONY: all clean
diff --git a/perftest.spec b/perftest.spec
index bd234e1..81ca90a 100755
--- a/perftest.spec
+++ b/perftest.spec
@@ -23,8 +23,8 @@ export CFLAGS=$RPM_OPT_FLAGS
chmod -x runme
%install
-install -D -m 0755 ib_rdma_lat $RPM_BUILD_ROOT%{_bindir}/ib_rdma_lat
-install -D -m 0755 ib_rdma_bw $RPM_BUILD_ROOT%{_bindir}/ib_rdma_bw
+install -D -m 0755 rdma_lat $RPM_BUILD_ROOT%{_bindir}/rdma_lat
+install -D -m 0755 rdma_bw $RPM_BUILD_ROOT%{_bindir}/rdma_bw
install -D -m 0755 ib_write_lat $RPM_BUILD_ROOT%{_bindir}/ib_write_lat
install -D -m 0755 ib_write_bw $RPM_BUILD_ROOT%{_bindir}/ib_write_bw
install -D -m 0755 ib_send_lat $RPM_BUILD_ROOT%{_bindir}/ib_send_lat
@@ -32,7 +32,7 @@ install -D -m 0755 ib_send_bw
$RPM_BUILD_ROOT%{_bindir}/ib_send_bw
install -D -m 0755 ib_read_lat $RPM_BUILD_ROOT%{_bindir}/ib_read_lat
install -D -m 0755 ib_read_bw $RPM_BUILD_ROOT%{_bindir}/ib_read_bw
install -D -m 0755 ib_write_bw_postlist
$RPM_BUILD_ROOT%{_bindir}/ib_write_bw_postlist
-install -D -m 0755 ib_clock_test $RPM_BUILD_ROOT%{_bindir}/ib_clock_test
+install -D -m 0755 rdma_clock_test
$RPM_BUILD_ROOT%{_bindir}/rdma_clock_test
%clean
rm -rf ${RPM_BUILD_ROOT}
@@ -43,6 +43,8 @@ rm -rf ${RPM_BUILD_ROOT}
%_bindir/*
%changelog
+* Sat Apr 18 2009 - hal.rosenst...@gmail.com
+- Change executable names for rdma_lat, rdma_bw, and clock_test
* Mon Jul 09 2007 - hvo...@suse.de
- Use correct version
* Wed Jul 04 2007 - hvo...@suse.de
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH] perftest: Make rdma_lat, rdma_bw, and clock_test executable names rdma neutral
Hi again Ido,
On Mon, Oct 5, 2009 at 8:20 AM, Ido Shamai i...@dev.mellanox.co.il wrote:
Hey Hal,
Just applied the 2 patches ( version 2 of the executable_names and
executable permisions)
Thanks. What's the plan to apply the ofed_1_5 branch changes to the master ?
-- Hal
Regards,
Ido Shaay
Hal Rosenstock wrote:
Hi Ido,
On Sun, Oct 4, 2009 at 6:29 AM, Ido Shamai i...@dev.mellanox.co.il
wrote:
Hey Hal ,
Sorry about the delay on response , I've waited for instructions for the
patch.
All is good except the change - ib_clock_test - to - rdma_clock_test,
my manager says clock_test is not a rdma benchmark. so If you can just
remove this change I'll be greatfull
I just submitted v2 of this as requested. Also, there's a minor
outstanding patch on changing the file permissions in the perftest git
repo.
Thanks.
-- Hal
Regards
Ido
Hal Rosenstock wrote:
Since rdma_lat and rdma_bw use RDMA CM, they can be used with both IB
and
iWARP so make their executable names neutral (by removing ib_)
IB only tests only require linking with libibverbs
Also, spec file change for executable name changes
Signed-off-by: Hal Rosenstock hal.rosenst...@gmail.com
---
diff --git a/Makefile b/Makefile
index 8042531..83c22c3 100755
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,8 @@
-TESTS = write_bw_postlist rdma_lat rdma_bw send_lat send_bw write_lat
write_bw read_lat read_bw
+RDMACM_TESTS = rdma_lat rdma_bw
+TESTS = write_bw_postlist send_lat send_bw write_lat write_bw read_lat
read_bw
UTILS = clock_test
-all: ${TESTS} ${UTILS}
+all: ${RDMACM_TESTS} ${TESTS} ${UTILS}
CFLAGS += -Wall -g -D_GNU_SOURCE -O2
EXTRA_FILES = get_clock.c
@@ -10,11 +11,18 @@ EXTRA_HEADERS = get_clock.h
LOADLIBES += LDFLAGS +=
-${TESTS}: LOADLIBES += -libverbs -lrdmacm
+${RDMACM_TESTS} ${UTILS}: LOADLIBES += -libverbs -lrdmacm
+${TESTS}: LOADLIBES += -libverbs
-${TESTS} ${UTILS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+${RDMACM_TESTS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+ $(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES}
$(LOADLIBES) $(LDLIBS) -o $@
+${TESTS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES}
$(LOADLIBES) $(LDLIBS) -o ib_$@
+${UTILS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+ $(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES}
$(LOADLIBES) $(LDLIBS) -o rdma_$@
+
clean:
- $(foreach fname,${TESTS} ${UTILS}, rm -f ib_${fname})
+ $(foreach fname,${RDMACM_TESTS} ${UTILS}, rm -f ${fname})
+ $(foreach fname,${TESTS}, rm -f ib_${fname})
.DELETE_ON_ERROR:
.PHONY: all clean
diff --git a/perftest.spec b/perftest.spec
index bd234e1..81ca90a 100755
--- a/perftest.spec
+++ b/perftest.spec
@@ -23,8 +23,8 @@ export CFLAGS=$RPM_OPT_FLAGS
chmod -x runme
%install
-install -D -m 0755 ib_rdma_lat $RPM_BUILD_ROOT%{_bindir}/ib_rdma_lat
-install -D -m 0755 ib_rdma_bw $RPM_BUILD_ROOT%{_bindir}/ib_rdma_bw
+install -D -m 0755 rdma_lat $RPM_BUILD_ROOT%{_bindir}/rdma_lat
+install -D -m 0755 rdma_bw $RPM_BUILD_ROOT%{_bindir}/rdma_bw
install -D -m 0755 ib_write_lat $RPM_BUILD_ROOT%{_bindir}/ib_write_lat
install -D -m 0755 ib_write_bw $RPM_BUILD_ROOT%{_bindir}/ib_write_bw
install -D -m 0755 ib_send_lat $RPM_BUILD_ROOT%{_bindir}/ib_send_lat
@@ -32,7 +32,7 @@ install -D -m 0755 ib_send_bw
$RPM_BUILD_ROOT%{_bindir}/ib_send_bw
install -D -m 0755 ib_read_lat $RPM_BUILD_ROOT%{_bindir}/ib_read_lat
install -D -m 0755 ib_read_bw $RPM_BUILD_ROOT%{_bindir}/ib_read_bw
install -D -m 0755 ib_write_bw_postlist
$RPM_BUILD_ROOT%{_bindir}/ib_write_bw_postlist
-install -D -m 0755 ib_clock_test
$RPM_BUILD_ROOT%{_bindir}/ib_clock_test
+install -D -m 0755 rdma_clock_test
$RPM_BUILD_ROOT%{_bindir}/rdma_clock_test
%clean
rm -rf ${RPM_BUILD_ROOT}
@@ -43,6 +43,8 @@ rm -rf ${RPM_BUILD_ROOT}
%_bindir/*
%changelog
+* Sat Apr 18 2009 - hal.rosenst...@gmail.com
+- Change executable names for rdma_lat, rdma_bw, and clock_test
* Mon Jul 09 2007 - hvo...@suse.de
- Use correct version
* Wed Jul 04 2007 - hvo...@suse.de
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] krping: fix qp signalling type (resend)
applied both.
Thanks.
Jon Mason wrote:
Per the IB spec for create qp(spec 1.2.1 section 11.2.4.11)
The Signaling Type must be specified for the Send Queue on this QP. The valid
types are:
* Non-selectable: All Work Requests submitted to the Send Queue always generate
a completion entry.
* Selectable: Consumer must specify on each Work Request submitted to the Send
Queue whether to generate a completion entry for successful completions.
The default for IB is non-selectable (.sq_sig_type set to 0). In the
qp_init_attr structure the field .sq_sig_type needs to be set to
IB_SIGNAL_REQ_WR in order to set it as selecetable (ie controlled from the WR).
Signed-off-by: Jon Mason j...@opengridcomputing.com
---
krping.c |7 +--
1 files changed, 1 insertions(+), 6 deletions(-)
diff --git a/krping.c b/krping.c
index 095b556..85cb9f9 100644
--- a/krping.c
+++ b/krping.c
@@ -429,12 +429,6 @@ static void krping_cq_event_handler(struct ib_cq *cq, void
*ctx)
wake_up_interruptible(cb-sem);
break;
- case IB_WC_LOCAL_INV:
- case IB_WC_FAST_REG_MR:
- printk(KERN_ERR PFX
- Unexpected opcode %d, most likely
unsignalled\n,
- __func__, __LINE__, wc.opcode);
- break;
default:
printk(KERN_ERR PFX
Unexpected opcode %d, Shutting down\n,
@@ -779,6 +773,7 @@ static int krping_create_qp(struct krping_cb *cb)
init_attr.qp_type = IB_QPT_RC;
init_attr.send_cq = cb-cq;
init_attr.recv_cq = cb-cq;
+ init_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
if (cb-server) {
ret = rdma_create_qp(cb-child_cm_id, cb-pd, init_attr);
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] perftest: Make rdma_lat, rdma_bw, and clock_test executable names rdma neutral
Hey Hal ,
Sorry about the delay on response , I've waited for instructions for the
patch.
All is good except the change - ib_clock_test - to - rdma_clock_test,
my manager says clock_test is not a rdma benchmark.
so If you can just remove this change I'll be greatfull
Regards
Ido
Hal Rosenstock wrote:
Since rdma_lat and rdma_bw use RDMA CM, they can be used with both IB and
iWARP so make their executable names neutral (by removing ib_)
IB only tests only require linking with libibverbs
Also, spec file change for executable name changes
Signed-off-by: Hal Rosenstock hal.rosenst...@gmail.com
---
diff --git a/Makefile b/Makefile
index 8042531..83c22c3 100755
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,8 @@
-TESTS = write_bw_postlist rdma_lat rdma_bw send_lat send_bw write_lat write_bw
read_lat read_bw
+RDMACM_TESTS = rdma_lat rdma_bw
+TESTS = write_bw_postlist send_lat send_bw write_lat write_bw read_lat read_bw
UTILS = clock_test
-all: ${TESTS} ${UTILS}
+all: ${RDMACM_TESTS} ${TESTS} ${UTILS}
CFLAGS += -Wall -g -D_GNU_SOURCE -O2
EXTRA_FILES = get_clock.c
@@ -10,11 +11,18 @@ EXTRA_HEADERS = get_clock.h
LOADLIBES +=
LDFLAGS +=
-${TESTS}: LOADLIBES += -libverbs -lrdmacm
+${RDMACM_TESTS} ${UTILS}: LOADLIBES += -libverbs -lrdmacm
+${TESTS}: LOADLIBES += -libverbs
-${TESTS} ${UTILS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+${RDMACM_TESTS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+ $(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES} $(LOADLIBES)
$(LDLIBS) -o $@
+${TESTS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES} $(LOADLIBES)
$(LDLIBS) -o ib_$@
+${UTILS}: %: %.c ${EXTRA_FILES} ${EXTRA_HEADERS}
+ $(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $ ${EXTRA_FILES} $(LOADLIBES)
$(LDLIBS) -o rdma_$@
+
clean:
- $(foreach fname,${TESTS} ${UTILS}, rm -f ib_${fname})
+ $(foreach fname,${RDMACM_TESTS} ${UTILS}, rm -f ${fname})
+ $(foreach fname,${TESTS}, rm -f ib_${fname})
.DELETE_ON_ERROR:
.PHONY: all clean
diff --git a/perftest.spec b/perftest.spec
index bd234e1..81ca90a 100755
--- a/perftest.spec
+++ b/perftest.spec
@@ -23,8 +23,8 @@ export CFLAGS=$RPM_OPT_FLAGS
chmod -x runme
%install
-install -D -m 0755 ib_rdma_lat $RPM_BUILD_ROOT%{_bindir}/ib_rdma_lat
-install -D -m 0755 ib_rdma_bw $RPM_BUILD_ROOT%{_bindir}/ib_rdma_bw
+install -D -m 0755 rdma_lat $RPM_BUILD_ROOT%{_bindir}/rdma_lat
+install -D -m 0755 rdma_bw $RPM_BUILD_ROOT%{_bindir}/rdma_bw
install -D -m 0755 ib_write_lat $RPM_BUILD_ROOT%{_bindir}/ib_write_lat
install -D -m 0755 ib_write_bw $RPM_BUILD_ROOT%{_bindir}/ib_write_bw
install -D -m 0755 ib_send_lat $RPM_BUILD_ROOT%{_bindir}/ib_send_lat
@@ -32,7 +32,7 @@ install -D -m 0755 ib_send_bw
$RPM_BUILD_ROOT%{_bindir}/ib_send_bw
install -D -m 0755 ib_read_lat $RPM_BUILD_ROOT%{_bindir}/ib_read_lat
install -D -m 0755 ib_read_bw $RPM_BUILD_ROOT%{_bindir}/ib_read_bw
install -D -m 0755 ib_write_bw_postlist
$RPM_BUILD_ROOT%{_bindir}/ib_write_bw_postlist
-install -D -m 0755 ib_clock_test $RPM_BUILD_ROOT%{_bindir}/ib_clock_test
+install -D -m 0755 rdma_clock_test $RPM_BUILD_ROOT%{_bindir}/rdma_clock_test
%clean
rm -rf ${RPM_BUILD_ROOT}
@@ -43,6 +43,8 @@ rm -rf ${RPM_BUILD_ROOT}
%_bindir/*
%changelog
+* Sat Apr 18 2009 - hal.rosenst...@gmail.com
+- Change executable names for rdma_lat, rdma_bw, and clock_test
* Mon Jul 09 2007 - hvo...@suse.de
- Use correct version
* Wed Jul 04 2007 - hvo...@suse.de
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] ehca: backports for RHEL5.3
Alexander Schmidt wrote: Hi Vlad, please apply the following ehca backports for RHEL5.3. Thanks! Signed-off-by: Alexander Schmidt al...@linux.vnet.ibm.com Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] libehca supported kernel versions
Alexander Schmidt wrote:
Hi Vlad,
please apply the following patch for install.pl.
Signed-off-by: Alexander Schmidt al...@linux.vnet.ibm.com
Index: OFED-1.5-20090915-0844/install.pl
===
--- OFED-1.5-20090915-0844.orig/install.pl
+++ OFED-1.5-20090915-0844/install.pl
@@ -1646,10 +1646,8 @@ sub set_availability
set_compilers();
# Ehca
-# if ($arch =~ m/ppc64|powerpc/ and
-# $kernel =~ m/2.6.1[6-9]|2.6.2[0-9]/) {
if ($arch =~ m/ppc64|powerpc/ and
-$kernel =~ m/2.6.30/) {
+$kernel =~ m/2.6.1[6-9]|2.6.2[0-9]|2.6.30/) {
$kernel_modules_info{'ehca'}{'available'} = 1;
$packages_info{'libehca'}{'available'} = 1;
$packages_info{'libehca-devel-static'}{'available'} = 1;
Applied,
Regards,
Vladimir
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] ehca: backports for 2.6.27
Alexander Schmidt wrote: Hi Vlad, please apply the following ehca backports for 2.6.27. Thanks! Signed-off-by: Alexander Schmidt al...@linux.vnet.ibm.com Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] ehca: fixes for OFED-1.5
Alexander Schmidt wrote: Hi Vlad, please apply the following fixes for OFED-1.5, thanks! Index: ofa_kernel-1.5/kernel_patches/fixes/ehca-0080-mad_redirection.patch Done, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] RDMA/nes: add backport for RHEL5U4
Chien Tung wrote: Copy iw_nes backport patches from RHEL5U3 to RHEL5U4. Signed-off-by: Chien Tung chien.tin.t...@intel.com --- Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] IB/qib: update driver with latest fixes
Ralph Campbell wrote: Vlad, Please pull for OFED-1.5 git://git.openfabrics.org/~ralphc/ofed-1.4-linux-2.6/.git ofed_kernel_1_5 commit 4fc3bd26b6bca72cdc1c06c326ead7aa61aa240d Author: Ralph Campbell (QLogic) ral...@hosting.openfabrics.org Date: Fri Sep 11 10:54:29 2009 -0700 IB/qib: update driver with latest fixes This patch updates the QIB driver with the latest bug fixes and updates the backport patches to match. Signed-off-by: Ralph Campbell ralph.campb...@qlogic.com Done. I used git://git.openfabrics.org/~ralphc/linux-2.6/.git ofed_kernel_1_5 Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] IB/ehca: Fix CQE flags reporting
applied, thanks ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] IB/ehca: Construct MAD redirect replies from request MAD
this seems reasonable to me, applied, thanks. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] mlx4_core: map sufficient ICM memory for EQs
Thanks, applied with a few cleanups: ilog2(roundup_pow_of_two()) - order_base_2() xxx * (1 yy) - xxx yy ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] backports: OFED 1.4.2 rds fix for double-def of assert_spin_locked in RHEL4_U4/5/6/7
Acked-by: Andy Grover andy.gro...@oracle.com
Thanks! -- Regards -- Andy
Jack Morgenstein wrote:
rds backport: fix for double-def of assert_spin_locked in RHEL4_U4/5/6/7
Any RHEL 4 infiniband app which includes jbd.h will get a compilation
error of double-definition of macro assert_spin_locked.
This fix was suggested by Brian Murrell of Sun.
This fixes bugzilla 1655.
Signed_off_by: Jack Morgenstein ja...@dev.mellanox.co.il
---
Andy,
There is a requirement for Lustre to fix bug 1655 in ofed 1.4.2.
https://bugs.openfabrics.org/show_bug.cgi?id=1655
In keeping with Brian Murrell's suggestion in comment #5
(to move the assert_spin_locked definition out of the spinlock.h
backport addons (for RHEL U4,5,6,7) and put it closer to the
C code that actually uses it), I propose moving the assert_spin_locked
definition from the spinlock.h addons files to file net/rds/rdma.h
(just after including rds.h) (done via an addition to patch file
rds_to_2_6_9.patch).
The patch is below.
PLEASE VERIFY AND OK THIS PATCH ASAP, so that I may check it in to the OFED
1.4.2
git before release. (I have checked only compilation on RHEL_U4/7 on X86_64).
Thanks!
diff --git a/kernel_addons/backport/2.6.9_U4/include/linux/spinlock.h
b/kernel_addons/backport/2.6.9_U4/include/linux/spinlock.h
index 0d24ba3..8b2b892 100644
--- a/kernel_addons/backport/2.6.9_U4/include/linux/spinlock.h
+++ b/kernel_addons/backport/2.6.9_U4/include/linux/spinlock.h
@@ -6,6 +6,4 @@
#define spin_lock_nested(lock, subclass) spin_lock(lock)
-#define assert_spin_locked(lock) do { (void)(lock); } while(0)
-
#endif
diff --git a/kernel_addons/backport/2.6.9_U5/include/linux/spinlock.h
b/kernel_addons/backport/2.6.9_U5/include/linux/spinlock.h
index 0d24ba3..8b2b892 100644
--- a/kernel_addons/backport/2.6.9_U5/include/linux/spinlock.h
+++ b/kernel_addons/backport/2.6.9_U5/include/linux/spinlock.h
@@ -6,6 +6,4 @@
#define spin_lock_nested(lock, subclass) spin_lock(lock)
-#define assert_spin_locked(lock) do { (void)(lock); } while(0)
-
#endif
diff --git a/kernel_addons/backport/2.6.9_U6/include/linux/spinlock.h
b/kernel_addons/backport/2.6.9_U6/include/linux/spinlock.h
index 665f011..4e22280 100644
--- a/kernel_addons/backport/2.6.9_U6/include/linux/spinlock.h
+++ b/kernel_addons/backport/2.6.9_U6/include/linux/spinlock.h
@@ -5,6 +5,4 @@
#define spin_lock_nested(lock, subclass) spin_lock(lock)
-#define assert_spin_locked(lock) do { (void)(lock); } while(0)
-
#endif
diff --git a/kernel_addons/backport/2.6.9_U7/include/linux/spinlock.h
b/kernel_addons/backport/2.6.9_U7/include/linux/spinlock.h
index 665f011..4e22280 100644
--- a/kernel_addons/backport/2.6.9_U7/include/linux/spinlock.h
+++ b/kernel_addons/backport/2.6.9_U7/include/linux/spinlock.h
@@ -5,6 +5,4 @@
#define spin_lock_nested(lock, subclass) spin_lock(lock)
-#define assert_spin_locked(lock) do { (void)(lock); } while(0)
-
#endif
diff --git a/kernel_patches/backport/2.6.9_U4/rds_to_2_6_9.patch
b/kernel_patches/backport/2.6.9_U4/rds_to_2_6_9.patch
index d3e2cb9..e260242 100644
--- a/kernel_patches/backport/2.6.9_U4/rds_to_2_6_9.patch
+++ b/kernel_patches/backport/2.6.9_U4/rds_to_2_6_9.patch
@@ -32,3 +32,18 @@ Index: ofed_kernel/net/rds/rds.h
#ifdef ATOMIC64_INIT
#define KERNEL_HAS_ATOMIC64
+diff --git a/net/rds/rdma.h b/net/rds/rdma.h
+index 4255120..8c35e7f 100644
+--- a/net/rds/rdma.h
b/net/rds/rdma.h
+@@ -7,6 +7,10 @@
+
+ #include rds.h
+
++#ifndef assert_spin_locked
++#define assert_spin_locked(lock) do { (void)(lock); } while(0)
++#endif
++
+ struct rds_mr {
+ struct rb_node r_rb_node;
+ atomic_tr_refcount;
diff --git a/kernel_patches/backport/2.6.9_U5/rds_to_2_6_9.patch
b/kernel_patches/backport/2.6.9_U5/rds_to_2_6_9.patch
index d3e2cb9..e260242 100644
--- a/kernel_patches/backport/2.6.9_U5/rds_to_2_6_9.patch
+++ b/kernel_patches/backport/2.6.9_U5/rds_to_2_6_9.patch
@@ -32,3 +32,18 @@ Index: ofed_kernel/net/rds/rds.h
#ifdef ATOMIC64_INIT
#define KERNEL_HAS_ATOMIC64
+diff --git a/net/rds/rdma.h b/net/rds/rdma.h
+index 4255120..8c35e7f 100644
+--- a/net/rds/rdma.h
b/net/rds/rdma.h
+@@ -7,6 +7,10 @@
+
+ #include rds.h
+
++#ifndef assert_spin_locked
++#define assert_spin_locked(lock) do { (void)(lock); } while(0)
++#endif
++
+ struct rds_mr {
+ struct rb_node r_rb_node;
+ atomic_tr_refcount;
diff --git a/kernel_patches/backport/2.6.9_U6/rds_to_2_6_9.patch
b/kernel_patches/backport/2.6.9_U6/rds_to_2_6_9.patch
index d3e2cb9..e260242 100644
--- a/kernel_patches/backport/2.6.9_U6/rds_to_2_6_9.patch
+++ b/kernel_patches/backport/2.6.9_U6/rds_to_2_6_9.patch
@@ -32,3 +32,18 @@ Index: ofed_kernel/net/rds/rds.h
#ifdef ATOMIC64_INIT
#define KERNEL_HAS_ATOMIC64
+diff --git a/net/rds/rdma.h b/net/rds/rdma.h
+index 4255120..8c35e7f 100644
[ewg] Re: [PATCH OFED-1.5] eHCA: SLES11 backports
Alexander Schmidt wrote: Hi Vlad, please apply the following patch to get our SLES11 backports. Thanks, Alex Done, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH]
Ralph Campbell wrote: Vlad, Please use git://git.openfabrics.org/~ralphc/libipathverbs/.git ofed_1_5 for OFED-1.5. The only change from OFED-1.4 is the following commit: commit 3978e32b233c0f5a8977082a33df50ffba96dc26 Author: Ralph Campbell (QLogic) ral...@hosting.openfabrics.org Date: Fri Jul 17 16:30:49 2009 -0700 IB/libipathverbs: change PathScale to QLogic The spec file used to construct RPMs is updated to change PathScale to QLogic Signed-off-by: Ralph Campbell ralph.campb...@qlogic.com Done Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v4] libibmad: Handle MAD redirection
Hal Rosenstock hal.rosenst...@gmail.com wrote on 08.07.2009 18:48:29: This patch should make its way into OFED 1.5... so who should pull it? You? Vlad? Someone not on CC? Whoever, please apply for OFED 1.5 -- thanks! Sasha is the management maintainer. Userspace trees for OFED 1.5 haven't been created and I think this aspect is in transition. Sasha, can you apply this patch? Thanks! Here's a link to the patch: http://lists.openfabrics.org/pipermail/ewg/2009-July/013519.html Cheers, Joachim ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] IB/qib: fix compiler errors for 2.6.30
Ralph Campbell wrote: Vlad, Please pull: git://git.openfabrics.org/~ralphc/linux-2.6/.git ofed_kernel_1_5 This should fix the 2.6.30 build errors. commit 4047cc3120a7c9b36fbddc3158ae0c01ec6636d9 Author: Ralph Campbell (QLogic) ral...@hosting.openfabrics.org Date: Thu Jul 9 15:47:02 2009 -0700 IB/qib: backport QIB from 2.6.30 to 2.6.27 Signed-off-by: Ralph Campbell ralph.campb...@qlogic.com commit 7251f2dbf99d0a97ea936f277d85baecbeafa1fd Author: Ralph Campbell (QLogic) ral...@hosting.openfabrics.org Date: Thu Jul 9 15:43:52 2009 -0700 IB/qib: fix compiler errors for 2.6.30 Signed-off-by: Ralph Campbell ralph.campb...@qlogic.com Done, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] install.pl: Install QIB driver instead of Ipath
Ralph Campbell wrote: Vlad, Please apply this to your ~vlad/ofed_scripts.git ofed_1_5 repo. This patch installs the qib driver which replaces the ipath driver in OFED-1.5. Signed-off-by: Ralph Campbell ralph.campb...@qlogic.com Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v4] libibmad: Handle MAD redirection
Hal Rosenstock hal.rosenst...@gmail.com wrote on 07.07.2009 17:23:18:
+static int redirect_port(ib_portid_t *port, uint8_t *mad)
+{
+ port-lid = mad_get_field(mad, 64, IB_CPI_REDIRECT_LID_F);
+ if (!port-lid) {
+ IBWARN(GID-based redirection is not supported);
+ return -1;
+ }
I hate to keep beating this horse but the lack of a LID certainly
means GID based redirection when the GID is not 0, IMO this LID check
is insufficient in general.
If the LID is given, my code does the right thing by redirecting
regardless
of any GID, as the spec requires. If no LID is given, but a GID is, my
code
bails with an error stating that GID-based redirection is not supported.
If
both GID and LID are 0, that's an error and my code bails with an error
message (which may or may not be misleading depending on your perspective,
but frankly I couldn't care less about broken agents).
Which of those three reactions do you think is insufficient?
I suppose this can be fixed down the road.
Is that an ACK? ;)
Cheers,
Joachim
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v4] libibmad: Handle MAD redirection
On 7/8/09, Joachim Fenkes fen...@de.ibm.com wrote:
Hal Rosenstock hal.rosenst...@gmail.com wrote on 07.07.2009 17:23:18:
+static int redirect_port(ib_portid_t *port, uint8_t *mad)
+{
+ port-lid = mad_get_field(mad, 64, IB_CPI_REDIRECT_LID_F);
+ if (!port-lid) {
+ IBWARN(GID-based redirection is not supported);
+ return -1;
+ }
I hate to keep beating this horse but the lack of a LID certainly
means GID based redirection when the GID is not 0, IMO this LID check
is insufficient in general.
If the LID is given, my code does the right thing by redirecting
regardless
of any GID, as the spec requires. If no LID is given, but a GID is, my
code
bails with an error stating that GID-based redirection is not supported.
If
both GID and LID are 0, that's an error and my code bails with an error
message (which may or may not be misleading depending on your perspective,
but frankly I couldn't care less about broken agents).
Which of those three reactions do you think is insufficient?
It looks to me like both GID and LID are allowed to be specified in
the redirect and if so, there is the possibility of GID based
redirection there (as well as when LID is 0) and it is the requester
which decides on GRH inclusion or not.
I suppose this can be fixed down the road.
Is that an ACK? ;)
Indeed, a qualified ACK :-) This case that concerns me ends up
entangled in the to be specified multiple IB subnet case (router
spec).
-- Hal
Cheers,
Joachim
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] Add QIB driver to OFED builds
Ralph Campbell wrote:
Vlad,
This should be applied to your ofabuild.git tree after pulling
the changes to add ib_qib to OFED-1.5.
Signed-off-by: Ralph Campbell ralph.campb...@qlogic.com
diff --git a/build_ofa_kernel.sh b/build_ofa_kernel.sh
index 88bf94e..2f5b690 100755
--- a/build_ofa_kernel.sh
+++ b/build_ofa_kernel.sh
@@ -150,6 +150,17 @@ set_packages()
;;
esac
+# QIB
+case ${arch} in
+x86_64)
+case ${kern} in
+
*2.6.9-67*|*2.6.9-78*|*2.6.16.60-0.21*|*2.6.18-93*|*2.6.18-128*|*2.6.27*)
+WITH_PACKAGES=${WITH_PACKAGES} --with-qib-mod
+;;
+esac
+;;
+esac
+
# srp target
case ${kern} in
2.6.16.*-*-*|2.6.18-*fc[56]*|2.6.*.el5|2.6.2[0-9]*|2.6.30)
Hi Ralph,
This patch is applied,
Please send me the patch for the install.pl script.
Thanks,
Vladimir
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v4] libibmad: Handle MAD redirection
Hal Rosenstock hal.rosenst...@gmail.com wrote on 08.07.2009 15:24:53: I suppose this can be fixed down the road. Is that an ACK? ;) Indeed, a qualified ACK :-) Cool, thanks! This patch should make its way into OFED 1.5... so who should pull it? You? Vlad? Someone not on CC? Whoever, please apply for OFED 1.5 -- thanks! Cheers Joachim ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v4] libibmad: Handle MAD redirection
On 7/8/09, Joachim Fenkes fen...@de.ibm.com wrote: Hal Rosenstock hal.rosenst...@gmail.com wrote on 08.07.2009 15:24:53: I suppose this can be fixed down the road. Is that an ACK? ;) Indeed, a qualified ACK :-) Cool, thanks! This patch should make its way into OFED 1.5... so who should pull it? You? Vlad? Someone not on CC? Whoever, please apply for OFED 1.5 -- thanks! Sasha is the management maintainer. Userspace trees for OFED 1.5 haven't been created and I think this aspect is in transition. -- Hal Cheers Joachim ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] ehca: use port autodetect mode as default
looks good, applied ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v4] libibmad: Handle MAD redirection
On Tue, Jul 7, 2009 at 10:20 AM, Joachim Fenkesfen...@de.ibm.com wrote:
Previously, libibmad reacted to GSI MAD responses with a redirect status
by throwing an error. IBM eHCA adapters use redirection, so most
infiniband_diags tools didn't work against eHCA.
Fix: Modify mad_rpc() so that it resends the request to the redirection
target if a redirect GS response is received. This is repeated until no
redirect response is received, allowing for multiple levels of
indirection.
The dport argument is updated with the redirection target, so subsequent
MADs will not go through the redirection process again but reach the target
directly.
Tested using perfquery between ehca, mlx4 and mthca in all possible
combinations.
Signed-off-by: Joachim Fenkes fen...@de.ibm.com
---
After all has been said and done, here's the hopefully last iteration of the
patch, with the hex display of the redirect LID replaced by decimal.
Any objections against this patch?
See below for comment.
Regards,
Joachim
libibmad/include/infiniband/mad.h | 9 +
libibmad/src/gs.c | 6 ++-
libibmad/src/rpc.c | 65
3 files changed, 63 insertions(+), 17 deletions(-)
diff --git a/libibmad/include/infiniband/mad.h
b/libibmad/include/infiniband/mad.h
index aa27eb5..bdf5158 100644
--- a/libibmad/include/infiniband/mad.h
+++ b/libibmad/include/infiniband/mad.h
@@ -115,6 +115,8 @@ enum MAD_ATTR_ID {
enum MAD_STATUS {
IB_MAD_STS_OK = (0 2),
+ IB_MAD_STS_BUSY = (1 0),
+ IB_MAD_STS_REDIRECT = (1 1),
IB_MAD_STS_BAD_BASE_VER_OR_CLASS = (1 2),
IB_MAD_STS_METHOD_NOT_SUPPORTED = (2 2),
IB_MAD_STS_METHOD_ATTR_NOT_SUPPORTED = (3 2),
@@ -783,8 +785,15 @@ MAD_EXPORT int madrpc_set_timeout(int timeout);
MAD_EXPORT struct ibmad_port *mad_rpc_open_port(char *dev_name, int dev_port,
int *mgmt_classes, int num_classes);
MAD_EXPORT void mad_rpc_close_port(struct ibmad_port *srcport);
+
+/*
+ * On redirection, the dport argument is updated with the redirection target,
+ * so subsequent MADs will not go through the redirection process again but
+ * reach the target directly.
+ */
MAD_EXPORT void *mad_rpc(const struct ibmad_port *srcport, ib_rpc_t * rpc,
ib_portid_t * dport, void *payload, void *rcvdata);
+
MAD_EXPORT void *mad_rpc_rmpp(const struct ibmad_port *srcport, ib_rpc_t *
rpc,
ib_portid_t * dport, ib_rmpp_hdr_t * rmpp,
void *data);
diff --git a/libibmad/src/gs.c b/libibmad/src/gs.c
index f3d245e..c7e4ff6 100644
--- a/libibmad/src/gs.c
+++ b/libibmad/src/gs.c
@@ -70,7 +70,8 @@ uint8_t *pma_query_via(void *rcvbuf, ib_portid_t * dest,
int port,
rpc.datasz = IB_PC_DATA_SZ;
rpc.dataoffs = IB_PC_DATA_OFFS;
- dest-qp = 1;
+ if (!dest-qp)
+ dest-qp = 1;
if (!dest-qkey)
dest-qkey = IB_DEFAULT_QP1_QKEY;
@@ -109,7 +110,8 @@ uint8_t *performance_reset_via(void *rcvbuf, ib_portid_t
* dest,
rpc.timeout = timeout;
rpc.datasz = IB_PC_DATA_SZ;
rpc.dataoffs = IB_PC_DATA_OFFS;
- dest-qp = 1;
+ if (!dest-qp)
+ dest-qp = 1;
if (!dest-qkey)
dest-qkey = IB_DEFAULT_QP1_QKEY;
diff --git a/libibmad/src/rpc.c b/libibmad/src/rpc.c
index 07b623d..efea1d3 100644
--- a/libibmad/src/rpc.c
+++ b/libibmad/src/rpc.c
@@ -183,33 +183,68 @@ _do_madrpc(int port_id, void *sndbuf, void *rcvbuf, int
agentid, int len,
return -1;
}
+static int redirect_port(ib_portid_t *port, uint8_t *mad)
+{
+ port-lid = mad_get_field(mad, 64, IB_CPI_REDIRECT_LID_F);
+ if (!port-lid) {
+ IBWARN(GID-based redirection is not supported);
+ return -1;
+ }
I hate to keep beating this horse but the lack of a LID certainly
means GID based redirection when the GID is not 0, IMO this LID check
is insufficient in general. I suppose this can be fixed down the road.
-- Hal
+
+ port-qp = mad_get_field(mad, 64, IB_CPI_REDIRECT_QP_F);
+ port-qkey = mad_get_field(mad, 64, IB_CPI_REDIRECT_QKEY_F);
+ port-sl = mad_get_field(mad, 64, IB_CPI_REDIRECT_SL_F);
+
+ /* TODO: Reverse map redirection P_Key to P_Key index */
+
+ if (ibdebug)
+ IBWARN(redirected to lid %d, qp 0x%x, qkey 0x%x, sl 0x%x,
+ port-lid, port-qp, port-qkey, port-sl);
+
+ return 0;
+}
+
void *mad_rpc(const struct ibmad_port *port, ib_rpc_t * rpc,
ib_portid_t * dport, void *payload, void *rcvdata)
{
int status, len;
uint8_t sndbuf[1024], rcvbuf[1024], *mad;
int timeout, retries;
+ int redirect = 1;
- len = 0;
- memset(sndbuf, 0, umad_size() +
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
On Wed, Jul 1, 2009 at 9:34 AM, Joachim Fenkesfen...@de.ibm.com wrote:
Previously, libibmad reacted to GSI MAD responses with a redirect status
by throwing an error. IBM eHCA adapters use redirection, so most
infiniband_diags tools didn't work against eHCA.
Fix: Modify mad_rpc() so that it resends the request to the redirection
target if a redirect GS response is received. This is repeated until no
redirect response is received, allowing for multiple levels of
indirection.
The dport argument is updated with the redirection target, so subsequent
MADs will not go through the redirection process again but reach the target
directly.
Tested using perfquery between ehca, mlx4 and mthca in all possible
combinations.
Signed-off-by: Joachim Fenkes fen...@de.ibm.com
---
Hi, Hal and Jason,
here's an updated patch that will bail on GID-routed redirection. Also, I
moved the redirection itself into its own function so it can easily be
included into RMPP as well.
Of course, I tested this again using ehca, mthca and mlx4.
If you have nothing to add to this patch, please queue it for OFED 1.5.
Thanks and regards,
Joachim
libibmad/include/infiniband/mad.h | 9 +
libibmad/src/gs.c | 6 ++-
libibmad/src/rpc.c | 65
3 files changed, 63 insertions(+), 17 deletions(-)
diff --git a/libibmad/include/infiniband/mad.h
b/libibmad/include/infiniband/mad.h
index aa27eb5..bdf5158 100644
--- a/libibmad/include/infiniband/mad.h
+++ b/libibmad/include/infiniband/mad.h
@@ -115,6 +115,8 @@ enum MAD_ATTR_ID {
enum MAD_STATUS {
IB_MAD_STS_OK = (0 2),
+ IB_MAD_STS_BUSY = (1 0),
+ IB_MAD_STS_REDIRECT = (1 1),
IB_MAD_STS_BAD_BASE_VER_OR_CLASS = (1 2),
IB_MAD_STS_METHOD_NOT_SUPPORTED = (2 2),
IB_MAD_STS_METHOD_ATTR_NOT_SUPPORTED = (3 2),
@@ -783,8 +785,15 @@ MAD_EXPORT int madrpc_set_timeout(int timeout);
MAD_EXPORT struct ibmad_port *mad_rpc_open_port(char *dev_name, int dev_port,
int *mgmt_classes, int num_classes);
MAD_EXPORT void mad_rpc_close_port(struct ibmad_port *srcport);
+
+/*
+ * On redirection, the dport argument is updated with the redirection target,
+ * so subsequent MADs will not go through the redirection process again but
+ * reach the target directly.
+ */
MAD_EXPORT void *mad_rpc(const struct ibmad_port *srcport, ib_rpc_t * rpc,
ib_portid_t * dport, void *payload, void *rcvdata);
+
MAD_EXPORT void *mad_rpc_rmpp(const struct ibmad_port *srcport, ib_rpc_t *
rpc,
ib_portid_t * dport, ib_rmpp_hdr_t * rmpp,
void *data);
diff --git a/libibmad/src/gs.c b/libibmad/src/gs.c
index f3d245e..c7e4ff6 100644
--- a/libibmad/src/gs.c
+++ b/libibmad/src/gs.c
@@ -70,7 +70,8 @@ uint8_t *pma_query_via(void *rcvbuf, ib_portid_t * dest,
int port,
rpc.datasz = IB_PC_DATA_SZ;
rpc.dataoffs = IB_PC_DATA_OFFS;
- dest-qp = 1;
+ if (!dest-qp)
+ dest-qp = 1;
if (!dest-qkey)
dest-qkey = IB_DEFAULT_QP1_QKEY;
@@ -109,7 +110,8 @@ uint8_t *performance_reset_via(void *rcvbuf, ib_portid_t
* dest,
rpc.timeout = timeout;
rpc.datasz = IB_PC_DATA_SZ;
rpc.dataoffs = IB_PC_DATA_OFFS;
- dest-qp = 1;
+ if (!dest-qp)
+ dest-qp = 1;
if (!dest-qkey)
dest-qkey = IB_DEFAULT_QP1_QKEY;
diff --git a/libibmad/src/rpc.c b/libibmad/src/rpc.c
index 07b623d..7364940 100644
--- a/libibmad/src/rpc.c
+++ b/libibmad/src/rpc.c
@@ -183,33 +183,68 @@ _do_madrpc(int port_id, void *sndbuf, void *rcvbuf, int
agentid, int len,
return -1;
}
+static int redirect_port(ib_portid_t *port, uint8_t *mad)
+{
+ port-lid = mad_get_field(mad, 64, IB_CPI_REDIRECT_LID_F);
+ if (!port-lid) {
+ IBWARN(GID-based redirection is not supported);
+ return -1;
+ }
Sorry for the confusion: determination of GID redirection should be
based on a comparison of the RedirectGID to 0. It's valid to supply
both a non zero RedirectGID and RedirectLID.
+
+ port-qp = mad_get_field(mad, 64, IB_CPI_REDIRECT_QP_F);
+ port-qkey = mad_get_field(mad, 64, IB_CPI_REDIRECT_QKEY_F);
+ port-sl = mad_get_field(mad, 64, IB_CPI_REDIRECT_SL_F);
+
+ /* TODO: Reverse map redirection P_Key to P_Key index */
+
+ if (ibdebug)
+ IBWARN(redirected to lid 0x%x, qp 0x%x, qkey 0x%x, sl 0x%x,
+ port-lid, port-qp, port-qkey, port-sl);
Unicast LIDs should be displayed in decimal rather than hex.
-- Hal
+
+ return 0;
+}
+
void *mad_rpc(const struct ibmad_port *port, ib_rpc_t * rpc,
ib_portid_t * dport, void *payload, void *rcvdata)
{
int status,
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
On Wed, Jul 01, 2009 at 09:59:41AM -0400, Hal Rosenstock wrote:
+static int redirect_port(ib_portid_t *port, uint8_t *mad)
+{
+ ?? ?? ?? port-lid = mad_get_field(mad, 64, IB_CPI_REDIRECT_LID_F);
+ ?? ?? ?? if (!port-lid) {
+ ?? ?? ?? ?? ?? ?? ?? IBWARN(GID-based redirection is not supported);
+ ?? ?? ?? ?? ?? ?? ?? return -1;
+ ?? ?? ?? }
Sorry for the confusion: determination of GID redirection should be
based on a comparison of the RedirectGID to 0. It's valid to supply
both a non zero RedirectGID and RedirectLID.
?? The above is correct. As I said, RedirectGID is not allowed to be 0.
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
Hal Rosenstock hal.rosenst...@gmail.com wrote on 01.07.2009 15:59:41:
+static int redirect_port(ib_portid_t *port, uint8_t *mad)
+{
+ port-lid = mad_get_field(mad, 64, IB_CPI_REDIRECT_LID_F);
+ if (!port-lid) {
+ IBWARN(GID-based redirection is not supported);
+ return -1;
+ }
Sorry for the confusion: determination of GID redirection should be
based on a comparison of the RedirectGID to 0. It's valid to supply
both a non zero RedirectGID and RedirectLID.
Are you sure?
About the Redirection GID, the spec says If redirection is not being
performed, this shall be set to zero, so if redirection _is_ being
performed, the GID may or may not be zero without any explicit
implication.
For the LID, it says If this value is zero, the redirect requires the
requester to use the supplied RedirectGID to request further path
resolution
from subnet administration. To me, this explicitly states that a zero LID
means that the GID must be used.
If both LID and GID are non-zero, it is not specified whether the
requester
should use the LID or the GID, so I choose to always use the LID as long
as it's non-zero, because that's what the code supports.
Am I talking crazy or does this make sense to you?
+
+ port-qp = mad_get_field(mad, 64, IB_CPI_REDIRECT_QP_F);
+ port-qkey = mad_get_field(mad, 64, IB_CPI_REDIRECT_QKEY_F);
+ port-sl = mad_get_field(mad, 64, IB_CPI_REDIRECT_SL_F);
+
+ /* TODO: Reverse map redirection P_Key to P_Key index */
+
+ if (ibdebug)
+ IBWARN(redirected to lid 0x%x, qp 0x%x, qkey 0x%x, sl
0x%x,
+ port-lid, port-qp, port-qkey, port-sl);
Unicast LIDs should be displayed in decimal rather than hex.
Couldn't you have noticed this in the first patch? ;) I'll change it.
Cheers,
Joachim
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH OFED-1.5] NFSRDMA: NFS backport for RHEL5.2
On Friday 19 June 2009 19:12, Jon Mason wrote: This patch provides the NFS backport for RHEL5.2. It passes Connectathon as a client and server over TCP and RDMA. Signed-Off-By: Jon Mason j...@opengridcomputing.com Thanks. I fixed the set_cpus_allowed issue as discussed in the thread, and applied the patch (and compiled it successfully). -Jack ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH OFED-1.5] NFSRDMA: NFS backport for 2.6.22
Any progress on the 2.6.25 and 2.6.22 backports?
-Jack
On Thursday 25 June 2009 19:08, Jon Mason wrote:
On Thu, Jun 25, 2009 at 10:43:02AM -0500, Jon Mason wrote:
On Thu, Jun 25, 2009 at 04:23:13PM +0300, Jack Morgenstein wrote:
On Friday 19 June 2009 19:14, Jon Mason wrote:
This patch provides the NFS backport for 2.6.22.
It passes Connectathon as a client and server over TCP and RDMA.
Signed-Off-By: Jon Mason j...@opengridcomputing.com
This patch fails compilation on the 2.6.22 kernel, as follows:
ja...@mtldesk006:/local/of15/ofed_kernel-2.6.22 make -k all /dev/null
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:132: error: expected ‘=’,
‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__initconst’
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c: In function
‘root_nfs_parse’:
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: ‘tokens’
undeclared (first use in this function)
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: (Each
undeclared identifier is reported only once
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: for each
function it appears in.)
make[3]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.o] Error 1
make[3]: Target `__build' not remade because of errors.
make[2]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs] Error 2
make[2]: Target `__build' not remade because of errors.
make[1]: *** [_module_/local/of15/ofed_kernel-2.6.22] Error 2
make[1]: Target `modules' not remade because of errors.
make: *** [kernel] Error 2
make: Target `all' not remade because of errors.
Note that in file fs/nfs/nfsroot.c, we have (after I applied the patch):
static const match_table_t tokens __initconst = {
{Opt_port, port=%u},
{Opt_rsize, rsize=%u},
It should probably be:
static match_table_t tokens = {
(which is consistent with changes for super.c in your patch:
+-static const match_table_t nfs_mount_option_tokens = {
++static match_table_t nfs_mount_option_tokens = {
except that you also need to get rid of the __initconst keyword).
Please resolve this, and re-send the patch (and also check this for the
2.6.25 patch)
Did you actually compile the patch on kernel 2.6.22?
Yes, I even ran Connectathon to verify that it works...
Funny, I do not see this error on my setup. I'll re-clone the git tree
and apply the patches to verify I see the same error you do.
I still don't see it. Maybe you can tell me what I'm doing wrong.
Here are the steps I did:
I saved the patch into mbox named foo
# git pull
# git checkout -b sanity ofed_kernel_1_5
# git am -3 ~/foo
# git log
It shows the patch in there..
# git clean -f
# git reset --hard HEAD
# git clean -f
The tree is now super clean
# ./ofed_scripts/ofed_makedist.sh
# scp ofed_kernel-2.6.22.tgz r...@vic12:
The kernel source is now on my victim system
On the victim:
# tar -xzvf ofed_kernel-2.6.22.tgz
# cd ofed_kernel-2.6.22
# ./configure --with-core-mod --with-user_mad-mod --with-user_access-mod
--with-addr_trans-mod --with-mthca-mod --with-mlx4-mod --with-mlx4_en-mod
--with-ipoib-mod --with-sdp-mod --with-nfsrdma-mod
# make install /dev/null
The last one ouputs:
/root/ofed_kernel-2.6.22/drivers/infiniband/ulp/sdp/sdp_bcopy.c: In function
‘sdp_post_sends’:
/root/ofed_kernel-2.6.22/drivers/infiniband/ulp/sdp/sdp_bcopy.c:214: warning:
unused variable ‘wc_processed’
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/core/ib_addr.ko
needs unknown symbol ip6_route_output
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/core/ib_addr.ko
needs unknown symbol ipv6_chk_addr
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/ulp/ipoib/ib_ipoib.ko
needs unknown symbol icmpv6_send
Do you see anything I did incorrectly?
Thanks,
Jon
thanks,
Jon
-Jack
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH OFED-1.5] NFSRDMA: NFS backport for 2.6.22
On Wed, Jul 01, 2009 at 06:22:45PM +0300, Jack Morgenstein wrote:
Any progress on the 2.6.25 and 2.6.22 backports?
I recently acquired an account on sofa.openfabrics.org. With
build_ofa_kernel.sh, I can duplicate the issue you see below (after
removing the other modules that break the build). I am working on a
solution.
I will run build_ofa_kernel.sh on all patches I send to you prior to the
request to pull. Hopefully this will prevent any more bad patches from
being sentout.
Thanks,
Jon
-Jack
On Thursday 25 June 2009 19:08, Jon Mason wrote:
On Thu, Jun 25, 2009 at 10:43:02AM -0500, Jon Mason wrote:
On Thu, Jun 25, 2009 at 04:23:13PM +0300, Jack Morgenstein wrote:
On Friday 19 June 2009 19:14, Jon Mason wrote:
This patch provides the NFS backport for 2.6.22.
It passes Connectathon as a client and server over TCP and RDMA.
Signed-Off-By: Jon Mason j...@opengridcomputing.com
This patch fails compilation on the 2.6.22 kernel, as follows:
ja...@mtldesk006:/local/of15/ofed_kernel-2.6.22 make -k all /dev/null
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:132: error: expected
‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__initconst’
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c: In function
‘root_nfs_parse’:
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: ‘tokens’
undeclared (first use in this function)
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: (Each
undeclared identifier is reported only once
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: for each
function it appears in.)
make[3]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.o] Error 1
make[3]: Target `__build' not remade because of errors.
make[2]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs] Error 2
make[2]: Target `__build' not remade because of errors.
make[1]: *** [_module_/local/of15/ofed_kernel-2.6.22] Error 2
make[1]: Target `modules' not remade because of errors.
make: *** [kernel] Error 2
make: Target `all' not remade because of errors.
Note that in file fs/nfs/nfsroot.c, we have (after I applied the patch):
static const match_table_t tokens __initconst = {
{Opt_port, port=%u},
{Opt_rsize, rsize=%u},
It should probably be:
static match_table_t tokens = {
(which is consistent with changes for super.c in your patch:
+-static const match_table_t nfs_mount_option_tokens = {
++static match_table_t nfs_mount_option_tokens = {
except that you also need to get rid of the __initconst keyword).
Please resolve this, and re-send the patch (and also check this for the
2.6.25 patch)
Did you actually compile the patch on kernel 2.6.22?
Yes, I even ran Connectathon to verify that it works...
Funny, I do not see this error on my setup. I'll re-clone the git tree
and apply the patches to verify I see the same error you do.
I still don't see it. Maybe you can tell me what I'm doing wrong.
Here are the steps I did:
I saved the patch into mbox named foo
# git pull
# git checkout -b sanity ofed_kernel_1_5
# git am -3 ~/foo
# git log
It shows the patch in there..
# git clean -f
# git reset --hard HEAD
# git clean -f
The tree is now super clean
# ./ofed_scripts/ofed_makedist.sh
# scp ofed_kernel-2.6.22.tgz r...@vic12:
The kernel source is now on my victim system
On the victim:
# tar -xzvf ofed_kernel-2.6.22.tgz
# cd ofed_kernel-2.6.22
# ./configure --with-core-mod --with-user_mad-mod --with-user_access-mod
--with-addr_trans-mod --with-mthca-mod --with-mlx4-mod --with-mlx4_en-mod
--with-ipoib-mod --with-sdp-mod --with-nfsrdma-mod
# make install /dev/null
The last one ouputs:
/root/ofed_kernel-2.6.22/drivers/infiniband/ulp/sdp/sdp_bcopy.c: In
function ‘sdp_post_sends’:
/root/ofed_kernel-2.6.22/drivers/infiniband/ulp/sdp/sdp_bcopy.c:214:
warning: unused variable ‘wc_processed’
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/core/ib_addr.ko
needs unknown symbol ip6_route_output
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/core/ib_addr.ko
needs unknown symbol ipv6_chk_addr
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/ulp/ipoib/ib_ipoib.ko
needs unknown symbol icmpv6_send
Do you see anything I did incorrectly?
Thanks,
Jon
thanks,
Jon
-Jack
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
On Wed, Jul 1, 2009 at 11:14 AM, Jason
Gunthorpejguntho...@obsidianresearch.com wrote:
On Wed, Jul 01, 2009 at 09:59:41AM -0400, Hal Rosenstock wrote:
+static int redirect_port(ib_portid_t *port, uint8_t *mad)
+{
+ ?? ?? ?? port-lid = mad_get_field(mad, 64, IB_CPI_REDIRECT_LID_F);
+ ?? ?? ?? if (!port-lid) {
+ ?? ?? ?? ?? ?? ?? ?? IBWARN(GID-based redirection is not supported);
+ ?? ?? ?? ?? ?? ?? ?? return -1;
+ ?? ?? ?? }
Sorry for the confusion: determination of GID redirection should be
based on a comparison of the RedirectGID to 0. It's valid to supply
both a non zero RedirectGID and RedirectLID.
?? The above is correct. As I said, RedirectGID is not allowed to be 0.
I think it depends on the interpretation of If redirection is not being
performed, this shall be set to zero. in the RedirectGID description
as to whether it is referring to redirection in general or just GID
redirection.
Futhermore, RedirectLID can be non zero but GID redirection is still
being used as indicated by the RedirectLID description indicating that
a non zero RedirectLID will in general not be valid.
-- Hal
Jason
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
On Wed, Jul 1, 2009 at 11:17 AM, Joachim Fenkesfen...@de.ibm.com wrote:
Hal Rosenstock hal.rosenst...@gmail.com wrote on 01.07.2009 15:59:41:
+static int redirect_port(ib_portid_t *port, uint8_t *mad)
+{
+ port-lid = mad_get_field(mad, 64, IB_CPI_REDIRECT_LID_F);
+ if (!port-lid) {
+ IBWARN(GID-based redirection is not supported);
+ return -1;
+ }
Sorry for the confusion: determination of GID redirection should be
based on a comparison of the RedirectGID to 0. It's valid to supply
both a non zero RedirectGID and RedirectLID.
Are you sure?
No; I'm not sure. See previous post to Jason.
About the Redirection GID, the spec says If redirection is not being
performed, this shall be set to zero, so if redirection _is_ being
performed, the GID may or may not be zero without any explicit
implication.
Agreed but Jason doesn't appear to agree.
For the LID, it says If this value is zero, the redirect requires the
requester to use the supplied RedirectGID to request further path
resolution
from subnet administration. To me, this explicitly states that a zero LID
means that the GID must be used.
If both LID and GID are non-zero, it is not specified whether the
requester
should use the LID or the GID, so I choose to always use the LID as long
as it's non-zero, because that's what the code supports.
It does say that the LID might not be valid even though non-zero. I'm
thinking of the more general case (future) rather than just IBM eHCA
usage.
Am I talking crazy or does this make sense to you?
+
+ port-qp = mad_get_field(mad, 64, IB_CPI_REDIRECT_QP_F);
+ port-qkey = mad_get_field(mad, 64, IB_CPI_REDIRECT_QKEY_F);
+ port-sl = mad_get_field(mad, 64, IB_CPI_REDIRECT_SL_F);
+
+ /* TODO: Reverse map redirection P_Key to P_Key index */
+
+ if (ibdebug)
+ IBWARN(redirected to lid 0x%x, qp 0x%x, qkey 0x%x, sl
0x%x,
+ port-lid, port-qp, port-qkey, port-sl);
Unicast LIDs should be displayed in decimal rather than hex.
Couldn't you have noticed this in the first patch? ;)
Somehow I missed it :-( Sorry.
I'll change it.
Thanks.
-- Hal
Cheers,
Joachim
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
On Wed, Jul 01, 2009 at 11:54:13AM -0400, Hal Rosenstock wrote: I think it depends on the interpretation of If redirection is not being performed, this shall be set to zero. in the RedirectGID description as to whether it is referring to redirection in general or just GID redirection. ClassPortInfo is used for alot of things, I take that to mean that when it is used in non-redirection contexts that RedirectGID can be 0. Clearly the only sane way this can work is if the GID is always filled in for the redirection case. Futhermore, RedirectLID can be non zero but GID redirection is still being used as indicated by the RedirectLID description indicating that a non zero RedirectLID will in general not be valid. The spec says if it is not zero the requester shall use it. I don't see an ambiguity here. Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
On Wed, Jul 1, 2009 at 12:20 PM, Joachim Fenkesfen...@de.ibm.com wrote: Hal Rosenstock hal.rosenst...@gmail.com wrote on 01.07.2009 17:54:41: It does say that the LID might not be valid even though non-zero. Can you elaborate on that? I can't seem to find that in the spec. What I do find, though, is this: If this value is non-zero, it is the DLID a requester _shall_ use to access the class services. -- v1.2, p736, line 7 Yes, that's same as v1.2.1 p.743 line 14. Which sounds to me like: If it's non-zero, it's valid and you must use it. v1.2.1 p.743 line 17 goes on to say: The RedirectGID, the RedirectQP and RedirectP_Key from this redirect response are all valid, but the RedirectSL, RedirectFL, RedirectTC, and RedirectLID will in general not be valid; they must be replaced using a Path- Record obtained from the SA. Doesn't look like that was a change from v1.2 as there are no change bars. -- Hal I'm thinking of the more general case (future) rather than just IBM eHCA usage. I stopped thinking eHCA only two patches ago, don't worry ;) BTW, I'm going to be out of the office starting now and returning next Tuesday, so let's continue this discussion next week ;) Cheers, Joachim ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
On Wed, Jul 1, 2009 at 12:50 PM, Jason Gunthorpejguntho...@obsidianresearch.com wrote: On Wed, Jul 01, 2009 at 11:54:13AM -0400, Hal Rosenstock wrote: I think it depends on the interpretation of If redirection is not being performed, this shall be set to zero. in the RedirectGID description as to whether it is referring to redirection in general or just GID redirection. ClassPortInfo is used for alot of things, Of course. I take that to mean that when it is used in non-redirection contexts that RedirectGID can be 0. I took it to mean differently as there's some conflicting text in RedirectLID. Clearly the only sane way this can work is if the GID is always filled in for the redirection case. Why is that ? Why must the redirector provide GRH info when it's not required for subnet local cases ? Futhermore, RedirectLID can be non zero but GID redirection is still being used as indicated by the RedirectLID description indicating that a non zero RedirectLID will in general not be valid. The spec says if it is not zero the requester shall use it. I don't see an ambiguity here. To me, the ambiguity is several lines below it where it states that the RedirectLID might not be valid and says to obtain a PathRecord when RedirectGID is supplied rather than relying on the RedirectLID is non zero. -- Hal Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH v3] libibmad: Handle MAD redirection
On Wed, Jul 1, 2009 at 4:00 PM, Jason Gunthorpejguntho...@obsidianresearch.com wrote: On Wed, Jul 01, 2009 at 03:39:01PM -0400, Hal Rosenstock wrote: Clearly the only sane way this can work is if the GID is always filled in for the redirection case. Why is that ? Why must the redirector provide GRH info when it's not required for subnet local cases ? Because the redirector doesn't know what the initiator will do. It could include a GRH, or maybe not. It must include the GID to cover both cases. It could restrict what the initiator can do by doing this. Nothing wrong with that AFAIT. I agree that this is not what you'd want if the requester were not subnet local. I'm only talking about the subnet local case. Futhermore, RedirectLID can be non zero but GID redirection is still being used as indicated by the RedirectLID description indicating that a non zero RedirectLID will in general not be valid. The spec says if it is not zero the requester shall use it. I don't see an ambiguity here. To me, the ambiguity is several lines below it where it states that the RedirectLID might not be valid and says to obtain a PathRecord when RedirectGID is supplied rather than relying on the RedirectLID is non zero. Whoever authored this should not have mixed 'will in general not be valid' and 'they must be replaced' in the same sentance - but I think the meaning is still clear. With a 0 RedirectLID only the RedirectGID, QP and P_Key are to be used by the receiver. When RedirectLID is not 0 then all of the Redirect fields must contain correct data and should be used as necessary by the receiver. It never says to obtain a Path Record when a GID is supplied. It says to obtain a path record with RedirectLID is 0. In looking at this some more, I agree with you on this part now since all that text is part of the RedirectLID 0 paragraph. -- Hal Jason ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] perftest: Add command line SL support
Oren, On Sun, Jun 28, 2009 at 9:11 AM, orenmeronorenme...@dev.mellanox.co.il wrote: Signed-off-by: Oren Meron orenme...@dev.mellanox.co.il Shouldn't this patch add your signoff to mine ? Thanks. -- Hal ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH OFED-1.5] NFSRDMA: NFS backport for 2.6.22
On Thursday 25 June 2009 19:08, Jon Mason wrote:
# ./ofed_scripts/ofed_makedist.sh
# scp ofed_kernel-2.6.22.tgz r...@vic12:
The kernel source is now on my victim system
On the victim:
# tar -xzvf ofed_kernel-2.6.22.tgz
# cd ofed_kernel-2.6.22
# ./configure --with-core-mod --with-user_mad-mod --with-user_access-mod
--with-addr_trans-mod --with-mthca-mod --with-mlx4-mod --with-mlx4_en-mod
--with-ipoib-mod --with-sdp-mod --with-nfsrdma-mod
# make install /dev/null
What are you running on the victim system?
I am compiling on a SuSe 10.0 system using gcc version: (GCC) 4.1.0 (SUSE Linux)
When I test out your patches, I do the following:
mkdir /tmp/of15
cd /tmp/of15
# Prepare and configure a copy of linux kernel 2.6.22 from kernel.org
tar xfj linux-2.6.22.tar.bz2
cd linux-2.6.22
./configure
make defconfig
make prepare
make scripts
# get and configure ofed-linux-2.6.22.tgz
cd ..
tar xfz /wherever/ofed_kernel-2.6.22.tgz
cd ofed_kernel-2.6.22
export WITH_BASIC_PACKAGES= \
--with-ipoib-mod \
--with-user_mad-mod \
--with-user_access-mod \
--with-mthca-mod \
--with-mlx4-mod \
--with-mlx4_en-mod \
--with-core-mod \
--with-addr_trans-mod \
--with-sdp-mod \
--with-nfsrdma-mod \
export WITH_ALLPACKAGES=$WITH_BASIC_PACKAGES
./configure ${WITH_ALLPACKAGES} --kernel-sources=/tmp/of15/linux-2.6.22
--kernel-version=2.6.22
# Apply the patch manually (2 steps), using quilt
quilt import -f -p1 your 2.6.22 patch.patch
quilt push # this gets the addons installed, and the backport patch file in
place
quilt import -f -p1 kernel_patches/backport/rnfs_fs.patch # apply your
backport patch
quilt push
# make
make -k all /dev/null
Note that in the ofed-linux-2.6.22 ./configure, I specify taking the kernel
sources from the
linux-2.6.22 directory that I just un-tarred and configured.
-Jack
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH OFED-1.5] NFSRDMA: NFS backport for 2.6.22
On Friday 19 June 2009 19:14, Jon Mason wrote:
This patch provides the NFS backport for 2.6.22.
It passes Connectathon as a client and server over TCP and RDMA.
Signed-Off-By: Jon Mason j...@opengridcomputing.com
This patch fails compilation on the 2.6.22 kernel, as follows:
ja...@mtldesk006:/local/of15/ofed_kernel-2.6.22 make -k all /dev/null
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:132: error: expected ‘=’, ‘,’,
‘;’, ‘asm’ or ‘__attribute__’ before ‘__initconst’
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c: In function ‘root_nfs_parse’:
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: ‘tokens’ undeclared
(first use in this function)
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: (Each undeclared
identifier is reported only once
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: for each function
it appears in.)
make[3]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.o] Error 1
make[3]: Target `__build' not remade because of errors.
make[2]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs] Error 2
make[2]: Target `__build' not remade because of errors.
make[1]: *** [_module_/local/of15/ofed_kernel-2.6.22] Error 2
make[1]: Target `modules' not remade because of errors.
make: *** [kernel] Error 2
make: Target `all' not remade because of errors.
Note that in file fs/nfs/nfsroot.c, we have (after I applied the patch):
static const match_table_t tokens __initconst = {
{Opt_port, port=%u},
{Opt_rsize, rsize=%u},
It should probably be:
static match_table_t tokens = {
(which is consistent with changes for super.c in your patch:
+-static const match_table_t nfs_mount_option_tokens = {
++static match_table_t nfs_mount_option_tokens = {
except that you also need to get rid of the __initconst keyword).
Please resolve this, and re-send the patch (and also check this for the 2.6.25
patch)
Did you actually compile the patch on kernel 2.6.22?
-Jack
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH OFED-1.5] NFSRDMA: NFS backport for 2.6.22
On Thu, Jun 25, 2009 at 04:23:13PM +0300, Jack Morgenstein wrote:
On Friday 19 June 2009 19:14, Jon Mason wrote:
This patch provides the NFS backport for 2.6.22.
It passes Connectathon as a client and server over TCP and RDMA.
Signed-Off-By: Jon Mason j...@opengridcomputing.com
This patch fails compilation on the 2.6.22 kernel, as follows:
ja...@mtldesk006:/local/of15/ofed_kernel-2.6.22 make -k all /dev/null
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:132: error: expected ‘=’,
‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__initconst’
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c: In function ‘root_nfs_parse’:
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: ‘tokens’
undeclared (first use in this function)
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: (Each undeclared
identifier is reported only once
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: for each function
it appears in.)
make[3]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.o] Error 1
make[3]: Target `__build' not remade because of errors.
make[2]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs] Error 2
make[2]: Target `__build' not remade because of errors.
make[1]: *** [_module_/local/of15/ofed_kernel-2.6.22] Error 2
make[1]: Target `modules' not remade because of errors.
make: *** [kernel] Error 2
make: Target `all' not remade because of errors.
Note that in file fs/nfs/nfsroot.c, we have (after I applied the patch):
static const match_table_t tokens __initconst = {
{Opt_port, port=%u},
{Opt_rsize, rsize=%u},
It should probably be:
static match_table_t tokens = {
(which is consistent with changes for super.c in your patch:
+-static const match_table_t nfs_mount_option_tokens = {
++static match_table_t nfs_mount_option_tokens = {
except that you also need to get rid of the __initconst keyword).
Please resolve this, and re-send the patch (and also check this for the
2.6.25 patch)
Did you actually compile the patch on kernel 2.6.22?
Yes, I even ran Connectathon to verify that it works...
Funny, I do not see this error on my setup. I'll re-clone the git tree
and apply the patches to verify I see the same error you do.
thanks,
Jon
-Jack
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH OFED-1.5] NFSRDMA: NFS backport for 2.6.22
On Thu, Jun 25, 2009 at 10:43:02AM -0500, Jon Mason wrote:
On Thu, Jun 25, 2009 at 04:23:13PM +0300, Jack Morgenstein wrote:
On Friday 19 June 2009 19:14, Jon Mason wrote:
This patch provides the NFS backport for 2.6.22.
It passes Connectathon as a client and server over TCP and RDMA.
Signed-Off-By: Jon Mason j...@opengridcomputing.com
This patch fails compilation on the 2.6.22 kernel, as follows:
ja...@mtldesk006:/local/of15/ofed_kernel-2.6.22 make -k all /dev/null
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:132: error: expected ‘=’,
‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__initconst’
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c: In function
‘root_nfs_parse’:
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: ‘tokens’
undeclared (first use in this function)
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: (Each
undeclared identifier is reported only once
/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.c:191: error: for each
function it appears in.)
make[3]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs/nfsroot.o] Error 1
make[3]: Target `__build' not remade because of errors.
make[2]: *** [/local/of15/ofed_kernel-2.6.22/fs/nfs] Error 2
make[2]: Target `__build' not remade because of errors.
make[1]: *** [_module_/local/of15/ofed_kernel-2.6.22] Error 2
make[1]: Target `modules' not remade because of errors.
make: *** [kernel] Error 2
make: Target `all' not remade because of errors.
Note that in file fs/nfs/nfsroot.c, we have (after I applied the patch):
static const match_table_t tokens __initconst = {
{Opt_port, port=%u},
{Opt_rsize, rsize=%u},
It should probably be:
static match_table_t tokens = {
(which is consistent with changes for super.c in your patch:
+-static const match_table_t nfs_mount_option_tokens = {
++static match_table_t nfs_mount_option_tokens = {
except that you also need to get rid of the __initconst keyword).
Please resolve this, and re-send the patch (and also check this for the
2.6.25 patch)
Did you actually compile the patch on kernel 2.6.22?
Yes, I even ran Connectathon to verify that it works...
Funny, I do not see this error on my setup. I'll re-clone the git tree
and apply the patches to verify I see the same error you do.
I still don't see it. Maybe you can tell me what I'm doing wrong.
Here are the steps I did:
I saved the patch into mbox named foo
# git pull
# git checkout -b sanity ofed_kernel_1_5
# git am -3 ~/foo
# git log
It shows the patch in there..
# git clean -f
# git reset --hard HEAD
# git clean -f
The tree is now super clean
# ./ofed_scripts/ofed_makedist.sh
# scp ofed_kernel-2.6.22.tgz r...@vic12:
The kernel source is now on my victim system
On the victim:
# tar -xzvf ofed_kernel-2.6.22.tgz
# cd ofed_kernel-2.6.22
# ./configure --with-core-mod --with-user_mad-mod --with-user_access-mod
--with-addr_trans-mod --with-mthca-mod --with-mlx4-mod --with-mlx4_en-mod
--with-ipoib-mod --with-sdp-mod --with-nfsrdma-mod
# make install /dev/null
The last one ouputs:
/root/ofed_kernel-2.6.22/drivers/infiniband/ulp/sdp/sdp_bcopy.c: In function
‘sdp_post_sends’:
/root/ofed_kernel-2.6.22/drivers/infiniband/ulp/sdp/sdp_bcopy.c:214: warning:
unused variable ‘wc_processed’
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/core/ib_addr.ko
needs unknown symbol ip6_route_output
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/core/ib_addr.ko
needs unknown symbol ipv6_chk_addr
WARNING:
/root/ofed_kernel-2.6.22/lib/modules/2.6.22.19-smp/extra/drivers/infiniband/ulp/ipoib/ib_ipoib.ko
needs unknown symbol icmpv6_send
Do you see anything I did incorrectly?
Thanks,
Jon
thanks,
Jon
-Jack
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [PATCH 2.6.31 try 2] ehca: Tolerate dynamic memory operations and huge pages
On Mon, 22 Jun 2009 22:19:21 -0700 Roland Dreier rdre...@cisco.com wrote: thanks, applied. -#define HCAD_VERSION 0026 +#define HCAD_VERSION 0027 the driver version was already 0027 (since bde2cfaf), so I dropped this chunk. thank you for applying, we would like to increase the version number for this patch, so please also apply the following: ehca: Increment version number for DMEM toleration Signed-off-by: Alexander Schmidt al...@linux.vnet.ibm.com --- drivers/infiniband/hw/ehca/ehca_main.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_main.c +++ infiniband.git/drivers/infiniband/hw/ehca/ehca_main.c @@ -52,7 +52,7 @@ #include ehca_tools.h #include hcp_if.h -#define HCAD_VERSION 0027 +#define HCAD_VERSION 0028 MODULE_LICENSE(Dual BSD/GPL); MODULE_AUTHOR(Christoph Raisch rai...@de.ibm.com); ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
