RDMA over Ethernet (RDMAoE) allows running the IB transport protocol over Ethernet, providing IB capabilities for Ethernet fabrics. The packets are standard Ethernet frames with an Ethertype, an IB GRH, unmodified IB transport headers and payload. HCA RDMAoE ports are no different than regular IB ports from the RDMA stack perspective. IB subnet management and SA services are not required for RDMAoE operation; Ethernet management practices are used instead. In Ethernet, nodes are commonly referred to by applications by means of an IP address. RDMAoE treats IP addresses that were assigned to the corresponding Ethernet port as GIDs, and makes use of the IP stack to bind a destination address to the corresponding netdevice (just as the CMA does today for IB and iWARP) and to obtain its L2 MAC addresses. The RDMA Verbs API is syntactically unmodified. When referring to RDMAoE ports, Address handles are required to contain GIDs and the L2 address fields in the API are ignored. The Ethernet L2 information is then obtained by the vendor-specific driver (both in kernel- and user-space) while modifying QPs to RTR and creating address handles. In order to maintain application compatibility, RDMAoE implements a SA_Query API that locally returns path records with the corresponding GIDs and the other relevant parameters . Consequently, any CMA or native Verbs application, in kernel or user-space, that uses path queries to obtain its address information, will run transparently over RDMAoE with no changes. We have successfully tested MPI, SDP, RDS, and native Verbs applications over RDMAoE without *any* changes. In the mlx4 driver stack, mlx4_en must be loaded and the corresponding eth Ethernet (with support for RDMAoE) or IB, as it was already the case.
Following is a series of 9 patches based on version 2.6.30 of the Linux kernel. Signed-off-by: Eli Cohen <e...@mellanox.co.il> drivers/infiniband/core/addr.c | 20 ++- drivers/infiniband/core/agent.c | 16 +- drivers/infiniband/core/cma.c | 39 ++++- drivers/infiniband/core/mad.c | 48 ++++-- drivers/infiniband/core/multicast.c | 153 ++++++++++++++-- drivers/infiniband/core/sa_query.c | 167 ++++++++++++++---- drivers/infiniband/core/ud_header.c | 111 ++++++++++++ drivers/infiniband/core/uverbs.h | 1 + drivers/infiniband/core/uverbs_cmd.c | 34 ++++ drivers/infiniband/core/uverbs_main.c | 1 + drivers/infiniband/core/verbs.c | 18 ++ drivers/infiniband/hw/mlx4/ah.c | 228 ++++++++++++++++++++---- drivers/infiniband/hw/mlx4/main.c | 276 ++++++++++++++++++++++++++--- drivers/infiniband/hw/mlx4/mlx4_ib.h | 30 +++- drivers/infiniband/hw/mlx4/qp.c | 253 +++++++++++++++++++++----- drivers/infiniband/ulp/ipoib/ipoib_main.c | 3 + drivers/net/mlx4/cmd.c | 6 + drivers/net/mlx4/en_main.c | 15 ++- drivers/net/mlx4/en_port.c | 4 +- drivers/net/mlx4/en_port.h | 3 +- drivers/net/mlx4/intf.c | 20 ++ drivers/net/mlx4/main.c | 6 + drivers/net/mlx4/mlx4.h | 1 + include/linux/mlx4/cmd.h | 1 + include/linux/mlx4/device.h | 31 +++- include/linux/mlx4/driver.h | 16 ++- include/linux/mlx4/qp.h | 8 +- include/rdma/ib_addr.h | 51 ++++++ include/rdma/ib_pack.h | 26 +++ include/rdma/ib_user_verbs.h | 21 ++- include/rdma/ib_verbs.h | 22 +++ 31 files changed, 1419 insertions(+), 210 deletions(-) _______________________________________________ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg