RE: BUG: RDMA/ocrdma calls invalid vlan_dev_real_dev()
I'll provide you fix in short while.
Parav
-Original Message-
From: Fengguang Wu [mailto:fengguang...@intel.com]
Sent: Friday, August 10, 2012 5:39 AM
To: Roland Dreier
Cc: linux-rdma@vger.kernel.org; Pandit, Parav; Sean Hefty; linux-
ker...@vger.kernel.org
Subject: Re: BUG: RDMA/ocrdma calls invalid vlan_dev_real_dev()
On Thu, Aug 09, 2012 at 04:54:37PM -0700, Roland Dreier wrote:
thanks for the report. I assume the system doesn't actually have ocrdma
hw?
Yeah, it's a test boot inside KVM.
Thanks,
Fengguang
- R.
On Aug 9, 2012 3:00 AM, Fengguang Wu fengguang...@intel.com
wrote:
Hi Parav,
commit fe2caefcdf (RDMA/ocrdma: Add driver for Emulex OneConnect
IBoE RDMA adapter) triggers the below kernel BUG for the attached
config.
[ 280.861196] kernel BUG at
/c/kernel-tests/src/stable/include/linux/if_vlan.h:113!
[ 280.861196] invalid opcode: [#1] PREEMPT [ 280.861196] CPU
0 [ 280.861196] Pid: 304, comm: ip Not tainted 3.6.0-rc1 #1 Bochs
Bochs [ 280.861196] RIP: 0010:[816df084]
[816df084]
ocrdma_inet6addr_event+0x4/0x6
[ 280.861196] RSP: 0018:8800066a1548 EFLAGS: 0202 [
280.861196] RAX: 0001 RBX: RCX:
[ 280.861196] RDX: 880006b6b400 RSI: 0001 RDI:
8207ecc0
[ 280.861196] RBP: 8800066a1548 R08: R09:
8109b657
[ 280.861196] R10: R11: 81e0f318 R12:
[ 280.861196] R13: 8207ecc0 R14: R15:
[ 280.861196] FS: 7f025c12f700() GS:81dfb000()
knlGS:
[ 280.861196] CS: 0010 DS: ES: CR0: 8005003b [
280.861196] CR2: 7fe4e1d3 CR3: 06b65000 CR4:
06b0
[ 280.861196] DR0: DR1: DR2:
[ 280.861196] DR3: DR6: DR7:
[ 280.861196] Process ip (pid: 304, threadinfo 8800066a,
task
880006a7c2c0)
[ 280.861196] Stack:
[ 280.861196] 8800066a1598 8109b5a2 880006b6b400
0001
[ 280.861196] 8800066a1598 0001 880006b6b400
[ 280.861196] 820ad9b0
8800066a15f8
8109b6b7
[ 280.861196] Call Trace:
[ 280.861196] [8109b5a2] notifier_call_chain+0x60/0x90 [
280.861196] [8109b6b7]
__atomic_notifier_call_chain+0x60/0x92
[ 280.861196] [8109b657] ?
atomic_notifier_chain_unregister+0x46/0x46
[ 280.861196] [8109b6f8]
atomic_notifier_call_chain+0xf/0x11
[ 280.861196] [81898dbc] ipv6_add_addr+0x333/0x388 [
280.861196] [81898ade] ? ipv6_add_addr+0x55/0x388 [
280.861196] [8189be09] add_addr+0x12/0x5c [ 280.861196]
[8189c0ff] init_loopback+0x7b/0x7f [ 280.861196]
[8189d25a] addrconf_notify+0x178/0x2d4 [ 280.861196]
[8109b5a2] notifier_call_chain+0x60/0x90 [ 280.861196]
[8109b87a] __raw_notifier_call_chain+0x9/0xb [
280.861196] [8109b88b] raw_notifier_call_chain+0xf/0x11 [
280.861196] [817dfac6] call_netdevice_notifiers+0x45/0x4a
[ 280.861196] [817e2bbd] __dev_notify_flags+0x32/0x56 [
280.861196] [817e2c24] dev_change_flags+0x43/0x4e [
280.861196] [817ef770] do_setlink+0x2da/0x7f6 [
280.861196] [81033fba] ? native_sched_clock+0x38/0x68 [
280.861196] [81033ff3] ? sched_clock+0x9/0xd [
280.861196] [810a0326] ?
sched_clock_local.constprop.2+0xd/0x78
[ 280.861196] [810a043a] ? sched_clock_cpu+0x7b/0x89 [
280.861196] [817f02cf] rtnl_newlink+0x264/0x438 [
280.861196] [817f0125] ? rtnl_newlink+0xba/0x438 [
280.861196] [8125a98a] ? avc_has_perm_noaudit+0xd1/0xe3 [
280.861196] [8125a8db] ? avc_has_perm_noaudit+0x22/0xe3 [
280.861196] [817ee988] rtnetlink_rcv_msg+0x22c/0x23b [
280.861196] [817ee720] ? rtnl_lock+0x12/0x14 [
280.861196] [817ee75c] ? __rtnl_unlock+0x12/0x12 [
280.861196] [8181e535] netlink_rcv_skb+0x3d/0x8a [
280.861196] [817ee743] rtnetlink_rcv+0x21/0x28 [
280.861196] [8181d2a4] netlink_unicast+0x12c/0x1b8 [
280.861196] [8181d8f5] netlink_sendmsg+0x212/0x29a [
280.861196] [817cf525] sock_sendmsg+0x9e/0xbf [
280.861196] [817cf78e] __sys_sendmsg+0x248/0x2d5 [
280.861196] [819c2107] ? _raw_spin_unlock_irq+0x34/0x50 [
280.861196] [8109dc1d] ?
finish_task_switch.constprop.48+0x72/0xd9
[ 280.861196] [8109dbdf] ?
finish_task_switch.constprop.48+0x34/0xd9
[ 280.861196] [819c0ed9] ? __schedule+0x501/0x607 [
280.861196]
RE: [Q] How to tranfer a file which is over 2GB(2^31) size in RDMA network?
-Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- ow...@vger.kernel.org] On Behalf Of Hiroyuki Sato Sent: Monday, July 02, 2012 7:20 PM To: linux-rdma Subject: [Q] How to tranfer a file which is over 2GB(2^31) size in RDMA network? Dear developers. I'm writing simple file transfer program. I would like to know about the following. Q: How to tranfer a file which is over 2GB(2^31) size in RDMA network? Please imagine to transfer whole DVD(4.7GB) file via RDMA networok. The maximum message size of RC is 2^31. so I can't transfer it with one RDMA message. Maybe It must split multiple parts. When I sent first part with the following sequence, how to transfer second part? Do I have to call qp ibv_destroy_qp and recreate new qp? Or can I reuse current qp? You can use the same QP to send multiple messages of same or different size. Receive side needs to have sufficient memory to receive 2GB of data, which you should have posted using post_recv(). Since you are going to divide a file/consumer application data to multiple RDMA messages, make sure sender and receiver agrees to a message size of interest. This is the simplest way to do via RDMA_SEND and RECV buffers. You can further do it via sharing the stag before invoking post_send/recv and use RDMA Write or READ operations and don't need to synchronize the message size. Sender can send/split data into one or more WRITE messages followed by ending it with write_immidiate or send_immidiate to notify the peer of file transfer done. In a third way, you can void writing the application and use SDP protocol to send your file via FTP/SFTP, in which FTP server and client application sockets use the SDP sockets instead of TCP sockets. I'm looking for similar example but I can't find it. Any information is welcome. Thank you for your advice. Sincerely -- Hiroyuki Sato. File transfer sequence (1st part) ibv_open_device ibv_alloc_pd ibv_reg_mr ibv_create_cq ibv_create_qp ibv_modify_qp(RESET-INIT) ibv_post_recv exchange lid, sid,qpn ibv_connect ibv_modify_qp(INIT-RTR) ibv_modify_qp(RTR-RTS) ibv_post_send ... Environment OS Scientific Linux 6.2 OFED: 1.5.4.1 -- Hiroyuki Sato -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
fmr pool and remap doubt
Hi, I am trying to understand remapping functionality and fmr_pool.c. Looking back at old thread: http://lists.openfabrics.org/pipermail/general/2006-February/017198.html Can you please confirm whether my understanding is correct or not. 1. max_map_per_fmr indicates that - different memory pages can be remapped (again) without invoking unmap_fmr(). Basically previous mapping is over-written on every map_phys_fmr() call with new mapping without doing unmapping. 2. adapter indicates above limit - how many times old mapping can be overwritten (remapped) before invoking unmap_fmr(). 3. Remapping allows faster operation compare to map(), unmap() sequence, due to which unmap_fmr() is mostly done in worker threads in SDP, RDS etc consumers. Is above understanding correct? Regards, Parav Pandit -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
be2net: when can I expect roce support patch will be merged?
Hi, Did you get chance to merge below be2net patch for supporing RoCE driver? http://marc.info/?l=linux-rdmam=133279326217836w=2 Once this is done, Roland can merge ocrdma RoCE driver addition to his tree. This NIC driver patch is required to merge ocrdma patch. When can I expect this patch to be merged to net-next git tree? Let me know if there any issue with the patch that I got to resolve. Regards, Parav Pandit -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message-
From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com]
Sent: Friday, March 23, 2012 4:14 AM
To: Pandit, Parav
Cc: david.lai...@aculab.com; rol...@purestorage.com; linux-
r...@vger.kernel.org; net...@vger.kernel.org
Subject: Re: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
On Thu, Mar 22, 2012 at 02:20:28PM -0700, parav.pan...@emulex.com
wrote:
I got a question here lately.
aligned directive will ensure that it will fall on boundary. Say
aligned(4) ensures that structure is aligned to 4 byte boundary.
Compiler can (at least theoretically) still have 4 byte structure
aligned to 8 byte boundary on 64-bit platform (which is 4 byte aligned
too).
There are very specific rules defined in the platform's ABI for how C
structures are layed out in memory, each ABI (ie CPU) has its own specific
quirks, but broadly in Linux land you can boil it down to:
1) The alignment of a structure is the greatest alignment of all the
members
2) Each member is aligned to its alignment.
The alignment of the structure drives the total size of the structure, and
specifically the padding added at the end to reach that alignment.
So, no, a compiler that increased the alignment of a struct with one
u32 to 8 would violate the various ABIs and not be usuable for Linux. It is
important to bear in mind that Linux targets a particular set of ABI
conventions, and it is not 'anything goes'.
struct {
u32 field;
};
So in this case: the u32 is aligned to 4, the structure is aligned to
4 and the total size of the structure is 4 on everything linux supports.
struct {
u64 fielda
u32 field;
};
In this case: On 64 bit: the u64 is aligned to 8 and the u32 is aligned to 4.
So
the structure is aligned to 8. A pad is inserted at the end of the struct to
bring
it out. On 32 bit, the u64 is aligned to 4, so the struct is aligned to 4, so
no pad
is added.
struct {
__float128 fielda
u32 field;
};
In this case the float128 is aligned to 16 and thus the structure is aligned
to 16
and 12 pad bytes are added.
However requirement is to have this structure only 4 byte size(
because adapter excepts it to be 4B sise) and therefor packed is used.
I don't know the way to ensure size of 4 byte and alignment too. Or I
am misunderstanding?
Yes, you are mis-understanding the rules for padding.. Structures are only
padded out to their alignment, which depends on their constituent types.
This is so arrays of structures have each array element starting on its
natural
alignment.
The aligned attribute overrides the automatic determination of the
alignment based on the contents and just forces it.
So, as an example, if you have this hardware layout:
struct {
u32 fielda;
u64 fieldb;
} attribute ((aligned(4));
David is saying you will get a 12 byte struct and fieldb will be unaligned.
Since
12 is aligned to 4 no padding is added.
So I decided to experiment above example before implementing in driver. However
I find structure of 16 bytes (instead of 12) with padding after fielda in below
example.
Am I missing some compiler option or syntax error in attribute? Sorry to ask
this silly question.
I tried __attribute__((__aligned__(4))); too based on usage in other kernel
code.
#include linux/module.h
struct foo {
u32 a;
u64 b;
} __attribute__((aligned(4)));
static int __init main_init(void)
{
printk(1 sizeof foo=%ld\n, sizeof(struct foo));
printk(1 offset of a=%ld\n, offsetof(struct foo, a));
printk(1 offset of b=%ld\n, offsetof(struct foo, b));
return 0;
}
static void __exit main_exit(void)
{
}
module_init(main_init);
module_exit(main_exit);
Output:
Mar 24 05:44:10 parav-sles11-sp1-64 kernel: [2006356.094931] sizeof foo=16
Mar 24 05:44:10 parav-sles11-sp1-64 kernel: [2006356.094934] offset of a=0
Mar 24 05:44:10 parav-sles11-sp1-64 kernel: [2006356.094936] offset of b=8
Gcc version is below.
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info
--mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64
--enable-languages=c,c++,objc,fortran,obj-c++,java,ada
--enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3
--enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/
--with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap
--with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit
--enable-libstdcxx-allocator=new --disable-libstdcxx-pch
--enable-version-specific-runtime-libs --program-suffix=-4.3
--enable-linux-futex --without-system-libunwind --with-cpu=generic
--build=x86_64-suse-linux
Thread model: posix
gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)
For hardware facing structures I'd combine this with a static assert to verify
structure size at compile time.
RE: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
Got it. You did mention about typedef in email chain, but I understood as
different way to achieve same.
I reviewed my code and found that most of the fields between driver-adapter
doesn't need attribute.
So far (a) removing packed and (b) BUILD_BUG_ON looks sufficient for current
set of structures. Initial test suffice the need.
Thanks.
Parav
-Original Message-
From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com]
Sent: Friday, March 23, 2012 10:24 PM
To: Pandit, Parav
Cc: david.lai...@aculab.com; rol...@purestorage.com; linux-
r...@vger.kernel.org; net...@vger.kernel.org
Subject: Re: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
On Fri, Mar 23, 2012 at 07:03:37AM -0700, parav.pan...@emulex.com wrote:
David is saying you will get a 12 byte struct and fieldb will be
unaligned. Since
12 is aligned to 4 no padding is added.
So I decided to experiment above example before implementing in
driver. However I find structure of 16 bytes (instead of 12) with
padding after fielda in below example. Am I missing some compiler
option or syntax error in attribute? Sorry to ask this silly question.
I tried __attribute__((__aligned__(4))); too based on usage in other
kernel code.
I got the syntax wrong for that specific case (it is a little unintuitive..
IMHO,
capping the alignment of a container should cap the alignment of all
members, otherwise it is nonsense!):
typedef uint64_t u64_unaligned_8 __attribute__((__aligned__(4)));
struct foo {
uint32_t fielda;
u64_unaligned_8 fieldb;
};
struct foo2 {
uint32_t fielda;
uint64_t fieldb;
};
int main(int argc,const char *argv[])
{
printf(sizeof(foo) = %zu, fieldb = %zu\n,sizeof(struct foo),
offsetof(struct foo,fieldb));
printf(sizeof(foo2) = %zu, fieldb = %zu\n,sizeof(struct foo2),
offsetof(struct foo2,fieldb));
return 0;
}
sizeof(foo) = 12, fieldb = 4
sizeof(foo2) = 16, fieldb = 8
gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)
Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 4/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
Inline.
-Original Message-
From: Hefty, Sean [mailto:sean.he...@intel.com]
Sent: Thursday, March 22, 2012 5:50 AM
To: Pandit, Parav; linux-rdma@vger.kernel.org
Subject: RE: [PATCH 4/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
+static inline void *ocrdma_get_eqe(struct ocrdma_eq *eq) {
+ return (u8 *)eq-q.va + (eq-q.tail * sizeof(struct ocrdma_eqe)); }
casts from (void *) to (u8 *) are not needed. This occurs in multiple places.
Removed.
+enum ib_qp_state get_ibqp_state(enum ocrdma_qp_state qps) {
+ switch (qps) {
+ case OCRDMA_QPS_RST:
+ return IB_QPS_RESET;
+ case OCRDMA_QPS_INIT:
+ return IB_QPS_INIT;
+ case OCRDMA_QPS_RTR:
+ return IB_QPS_RTR;
+ case OCRDMA_QPS_RTS:
+ return IB_QPS_RTS;
+ case OCRDMA_QPS_SQD:
+ case OCRDMA_QPS_SQ_DRAINING:
+ return IB_QPS_SQD;
+ case OCRDMA_QPS_SQE:
+ return IB_QPS_SQE;
+ case OCRDMA_QPS_ERR:
+ return IB_QPS_ERR;
+ };
+ return IB_QPS_ERR;
+}
+
+enum ocrdma_qp_state get_ocrdma_qp_state(enum ib_qp_state qps) {
+ switch (qps) {
+ case IB_QPS_RESET:
+ return OCRDMA_QPS_RST;
+ case IB_QPS_INIT:
+ return OCRDMA_QPS_INIT;
+ case IB_QPS_RTR:
+ return OCRDMA_QPS_RTR;
+ case IB_QPS_RTS:
+ return OCRDMA_QPS_RTS;
+ case IB_QPS_SQD:
+ return OCRDMA_QPS_SQD;
+ case IB_QPS_SQE:
+ return OCRDMA_QPS_SQE;
+ case IB_QPS_ERR:
+ return OCRDMA_QPS_ERR;
+ };
+ return OCRDMA_QPS_ERR;
+}
+
+static int ocrdma_get_mbx_errno(u32 status) {
+ int err_num = -EFAULT;
no need to initialize, since it's set in all cases
Removed.
+ u8 mbox_status = (status OCRDMA_MBX_RSP_STATUS_MASK)
+ OCRDMA_MBX_RSP_STATUS_SHIFT;
+ u8 add_status = (status OCRDMA_MBX_RSP_ASTATUS_MASK)
+
OCRDMA_MBX_RSP_ASTATUS_SHIFT;
+
+ switch (mbox_status) {
+ case OCRDMA_MBX_STATUS_OOR:
+ case OCRDMA_MBX_STATUS_MAX_QP_EXCEEDS:
+ err_num = -EAGAIN;
+ break;
+
+ case OCRDMA_MBX_STATUS_INVALID_PD:
+ case OCRDMA_MBX_STATUS_INVALID_CQ:
+ case OCRDMA_MBX_STATUS_INVALID_SRQ_ID:
+ case OCRDMA_MBX_STATUS_INVALID_QP:
+ case OCRDMA_MBX_STATUS_INVALID_CHANGE:
+ case OCRDMA_MBX_STATUS_MTU_EXCEEDS:
+ case OCRDMA_MBX_STATUS_INVALID_RNR_NAK_TIMER:
+ case OCRDMA_MBX_STATUS_PKEY_INDEX_INVALID:
+ case OCRDMA_MBX_STATUS_PKEY_INDEX_EXCEEDS:
+ case OCRDMA_MBX_STATUS_ILLEGAL_FIELD:
+ case OCRDMA_MBX_STATUS_INVALID_PBL_ENTRY:
+ case OCRDMA_MBX_STATUS_INVALID_LKEY:
+ case OCRDMA_MBX_STATUS_INVALID_VA:
+ case OCRDMA_MBX_STATUS_INVALID_LENGTH:
+ case OCRDMA_MBX_STATUS_INVALID_FBO:
+ case OCRDMA_MBX_STATUS_INVALID_ACC_RIGHTS:
+ case OCRDMA_MBX_STATUS_INVALID_PBE_SIZE:
+ case OCRDMA_MBX_STATUS_ATOMIC_OPS_UNSUP:
+ case OCRDMA_MBX_STATUS_SRQ_ERROR:
+ case OCRDMA_MBX_STATUS_SRQ_SIZE_UNDERUNS:
+ err_num = -EINVAL;
+ break;
+
+ case OCRDMA_MBX_STATUS_PD_INUSE:
+ case OCRDMA_MBX_STATUS_QP_BOUND:
+ case OCRDMA_MBX_STATUS_MW_STILL_BOUND:
+ case OCRDMA_MBX_STATUS_MW_BOUND:
+ err_num = -EBUSY;
+ break;
+
+ case OCRDMA_MBX_STATUS_RECVQ_RQE_EXCEEDS:
+ case OCRDMA_MBX_STATUS_SGE_RECV_EXCEEDS:
+ case OCRDMA_MBX_STATUS_RQE_EXCEEDS:
+ case OCRDMA_MBX_STATUS_SRQ_LIMIT_EXCEEDS:
+ case OCRDMA_MBX_STATUS_ORD_EXCEEDS:
+ case OCRDMA_MBX_STATUS_IRD_EXCEEDS:
+ case OCRDMA_MBX_STATUS_SENDQ_WQE_EXCEEDS:
+ case OCRDMA_MBX_STATUS_SGE_SEND_EXCEEDS:
+ case OCRDMA_MBX_STATUS_SGE_WRITE_EXCEEDS:
+ err_num = -ENOBUFS;
+ break;
+
+ case OCRDMA_MBX_STATUS_FAILED:
+ switch (add_status) {
+ case
OCRDMA_MBX_ADDI_STATUS_INSUFFICIENT_RESOURCES:
+ err_num = -EAGAIN;
+ break;
+ }
+ default:
+ err_num = -EFAULT;
+ }
+ return err_num;
+}
+
+static int ocrdma_get_mbx_cqe_errno(u16 cqe_status) {
+ int err_num = -EINVAL;
no need to initialize
+ switch (cqe_status) {
+ case OCRDMA_MBX_CQE_STATUS_INSUFFICIENT_PRIVILEDGES:
+ err_num = -EPERM;
+ break;
+ case OCRDMA_MBX_CQE_STATUS_INVALID_PARAMETER:
+ err_num = -EINVAL;
+ break;
+ case OCRDMA_MBX_CQE_STATUS_INSUFFICIENT_RESOURCES:
+ case OCRDMA_MBX_CQE_STATUS_QUEUE_FLUSHING:
+ err_num = -EAGAIN;
+ break;
+ case OCRDMA_MBX_CQE_STATUS_DMA_FAILED:
+ err_num = -EIO;
+ break;
+ }
+ return err_num;
+}
+static int ocrdma_mbx_create_eq(struct ocrdma_dev *dev, struct
+ocrdma_eq *eq) {
+ int status;
+ struct ocrdma_create_eq_req *cmd = dev-mbx_cmd;
+ struct ocrdma_create_eq_rsp *rsp = dev-mbx_cmd;
+
+
RE: [PATCH 5/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message-
From: Hefty, Sean [mailto:sean.he...@intel.com]
Sent: Thursday, March 22, 2012 6:14 AM
To: Pandit, Parav; linux-rdma@vger.kernel.org
Subject: RE: [PATCH 5/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
+static int ocrdma_inet6addr_event(struct notifier_block *,
+ unsigned long, void *);
+
+static struct notifier_block ocrdma_inet6addr_notifier = {
+ .notifier_call = ocrdma_inet6addr_event };
+
+int ocrdma_get_instance(void)
+{
+ int instance = 0;
+
+ /* Assign an unused number */
+ if (!idr_pre_get(ocrdma_dev_id, GFP_KERNEL))
+ return -1;
+ if (idr_get_new(ocrdma_dev_id, NULL, instance))
+ return -1;
+ return instance;
+}
+
+void ocrdma_get_guid(struct ocrdma_dev *dev, u8 *guid) {
+ u8 mac_addr[6];
+
+ memcpy(mac_addr[0], dev-nic_info.mac_addr[0], ETH_ALEN);
+ guid[0] = mac_addr[0] ^ 2;
+ guid[1] = mac_addr[1];
+ guid[2] = mac_addr[2];
+ guid[3] = 0xff;
+ guid[4] = 0xfe;
+ guid[5] = mac_addr[3];
+ guid[6] = mac_addr[4];
+ guid[7] = mac_addr[5];
+}
+
+static void ocrdma_build_sgid_mac(union ib_gid *sgid, unsigned char
*mac_addr,
+ bool is_vlan, u16 vlan_id)
+{
+ sgid-global.subnet_prefix = cpu_to_be64(0xfe80LL);
+ sgid-raw[8] = mac_addr[0] ^ 2;
+ sgid-raw[9] = mac_addr[1];
+ sgid-raw[10] = mac_addr[2];
+ if (is_vlan) {
+ sgid-raw[11] = vlan_id 8;
+ sgid-raw[12] = vlan_id 0xff;
+ } else {
+ sgid-raw[11] = 0xff;
+ sgid-raw[12] = 0xfe;
+ }
+ sgid-raw[13] = mac_addr[3];
+ sgid-raw[14] = mac_addr[4];
+ sgid-raw[15] = mac_addr[5];
+}
+
+static void ocrdma_add_sgid(struct ocrdma_dev *dev, unsigned char
*mac_addr,
+ bool is_vlan, u16 vlan_id)
+{
+ int i;
+ bool found = false;
+ union ib_gid new_sgid;
+ int free_idx = OCRDMA_MAX_SGID;
+ unsigned long flags;
+
+ memset(ocrdma_zero_sgid, 0, sizeof(union ib_gid));
ocrdma_zero_sgid should already be zeroed
Yes. Removed memset().
Actually, can't we either use in6addr_any instead? I see that the mlx4 driver
also has a zero gid. So, if we don't want to overload the use of in6addr_any,
we can define gid_zero in ib_core.
o.k. once its defined in ib_core, will remove this and provide a subsequent
patch. It will be easier to handle it.
+
+ ocrdma_build_sgid_mac(new_sgid, mac_addr, is_vlan, vlan_id);
+
+ spin_lock_irqsave(dev-sgid_lock, flags);
+ for (i = 0; i OCRDMA_MAX_SGID; i++) {
+ if (!memcmp(dev-sgid_tbl[i], ocrdma_zero_sgid,
+ sizeof(union ib_gid))) {
+ /* found free entry */
+ if (!found) {
+ free_idx = i;
+ found = true;
+ break;
+ }
+ } else if (!memcmp(dev-sgid_tbl[i], new_sgid,
+ sizeof(union ib_gid))) {
+ /* entry already present, no addition is required. */
+ spin_unlock_irqrestore(dev-sgid_lock, flags);
+ return;
+ }
+ }
+ /* if entry doesn't exist and if table has some space, add entry */
+ if (found)
+ memcpy(dev-sgid_tbl[free_idx], new_sgid,
+ sizeof(union ib_gid));
can move memcpy into for loop and remove found flag
Yes, removed.
+ spin_unlock_irqrestore(dev-sgid_lock, flags); }
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 1/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message-
From: Hefty, Sean [mailto:sean.he...@intel.com]
Sent: Wednesday, March 21, 2012 10:49 PM
To: Pandit, Parav; linux-rdma@vger.kernel.org
Subject: RE: [PATCH 1/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
+struct ocrdma_cq {
+ struct ib_cq ibcq;
+ struct ocrdma_dev *dev;
nit: There are several structures where you store ocrdma_dev *. You can
remove these and use the struct ib_* to reach it as well.
Removed from all the ocrdma_xxx structures except ocrdma_eq and ocrdma_qp.
ocrdma_eq is anyway necessary.
Removing it from ocrdma_qp will require a lot bigger test cycle at my end.
I prefer to have it in this first patch and once the internal test cycle is
finish (with removed code) in few days, will submit the separate patch?
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 1/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message- From: Roland Dreier [mailto:rol...@purestorage.com] Sent: Thursday, March 22, 2012 9:30 PM To: Pandit, Parav Cc: sean.he...@intel.com; linux-rdma@vger.kernel.org Subject: Re: [PATCH 1/9] ocrdma: Driver for Emulex OneConnect RDMA adapter On Thu, Mar 22, 2012 at 7:27 AM, parav.pan...@emulex.com wrote: I prefer to have it in this first patch and once the internal test cycle is finish (with removed code) in few days, will submit the separate patch? It's fine to leave some of these items for future cleanups post-merge. Just please try to get to the cleanups someday unlike some other drivers... Sure. Item is list and tracking the same. Mostly likely it's going to be 2nd patch after this first release before adding any new features. (*cough* nes *cough*) - R. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message-
From: David Laight [mailto:david.lai...@aculab.com]
Sent: Wednesday, March 21, 2012 10:02 PM
To: Roland Dreier; Pandit, Parav
Cc: linux-rdma@vger.kernel.org; net...@vger.kernel.org
Subject: RE: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
- Header file for userspace library and kernel driver interface.
+struct ocrdma_alloc_ucontext_resp {
+ u32 dev_id;
+ u32 wqe_size;
+ u32 max_inline_data;
+ u32 dpp_wqe_size;
+ u64 ah_tbl_page;
+ u32 ah_tbl_len;
+ u32 rsvd;
+ u8 fw_ver[32];
+ u32 rqe_size;
+ u64 rsvd1;
+} __packed;
If I'm reading this correctly, you have the 8-byte rsvd1 member at an
offset only aligned to 4 bytes, because of the __packed directive. It
would be much better to have these structures laid out so they are
naturally the same on both 32-bit and 64-bit ABIs, and get rid of the
__packed directive, which wrecks gcc code generation in some cases.
gcc also supports defining types that have non-standard alignment
constraints that can be used to force the same alignment for 64bit fields
between i386 and amd64.
Probably __attribute__((aligned,n)) or similar.
This can be used to force 32bit alignment in amd64 code in order to match
definitions in 32bit userspace.
For new things it would make sense to force 64bit alignment of 64bit fields
for 32bit code.
o.k. so I'll use aligned attribute to align user-kernel interface data
structure to 8 byte boundary.
That should work for 32-bit and 64-bit user and kernel space and does't hurt
performance either?
Driver-adapter structures will be aligned to 4 byte boundary using aligned
attribute instead of packed.
Adding __packed (rather than 32bit alignment) forces the compiler to
generate byte by byte accesses for all the fields on systems that can't do
misaligned accesses in hardware (eg sparc).
David
David
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message- From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com] Sent: Friday, March 23, 2012 2:28 AM To: Pandit, Parav Cc: david.lai...@aculab.com; rol...@purestorage.com; linux- r...@vger.kernel.org; net...@vger.kernel.org Subject: Re: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA adapter On Thu, Mar 22, 2012 at 01:52:30PM -0700, parav.pan...@emulex.com wrote: This can be used to force 32bit alignment in amd64 code in order to match definitions in 32bit userspace. For new things it would make sense to force 64bit alignment of 64bit fields for 32bit code. o.k. so I'll use aligned attribute to align user-kernel interface data structure to 8 byte boundary. That should work for 32-bit and 64-bit user and kernel space and does't hurt performance either? If the structure is only for user/kernel interfacing then it is much better to add explicit padding fields to naturally place 64 bit quantities on an 8 byte alignment than to mess with gcc specific attributes (user space has a much wide choice of compilers). This was David's second suggestion. Better to do this now before the driver is accepted :) o.k. I'll align them to naturally 8 byte boundary. Jason -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
I got a question here lately.
aligned directive will ensure that it will fall on boundary.
Say aligned(4) ensures that structure is aligned to 4 byte boundary.
Compiler can (at least theoretically) still have 4 byte structure aligned to 8
byte boundary on 64-bit platform (which is 4 byte aligned too).
struct {
u32 field;
} attribute ((aligned(4));
However requirement is to have this structure only 4 byte size( because adapter
excepts it to be 4B sise) and therefor packed is used.
I don't know the way to ensure size of 4 byte and alignment too.
Or I am misunderstanding?
Parav
-Original Message-
From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-
ow...@vger.kernel.org] On Behalf Of parav.pan...@emulex.com
Sent: Friday, March 23, 2012 2:41 AM
To: jguntho...@obsidianresearch.com
Cc: david.lai...@aculab.com; rol...@purestorage.com; linux-
r...@vger.kernel.org; net...@vger.kernel.org
Subject: RE: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
-Original Message-
From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com]
Sent: Friday, March 23, 2012 2:28 AM
To: Pandit, Parav
Cc: david.lai...@aculab.com; rol...@purestorage.com; linux-
r...@vger.kernel.org; net...@vger.kernel.org
Subject: Re: [PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
On Thu, Mar 22, 2012 at 01:52:30PM -0700, parav.pan...@emulex.com
wrote:
This can be used to force 32bit alignment in amd64 code in order
to match definitions in 32bit userspace.
For new things it would make sense to force 64bit alignment of
64bit fields for 32bit code.
o.k. so I'll use aligned attribute to align user-kernel interface
data structure to 8 byte boundary. That should work for 32-bit and
64-bit user and kernel space and does't hurt performance either?
If the structure is only for user/kernel interfacing then it is much
better to add explicit padding fields to naturally place 64 bit
quantities on an 8 byte alignment than to mess with gcc specific
attributes (user space has a much wide choice of compilers).
This was David's second suggestion. Better to do this now before the
driver is accepted :)
o.k. I'll align them to naturally 8 byte boundary.
Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in the
body of a message to majord...@vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 6/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
Inline.
-Original Message-
From: Roland Dreier [mailto:rol...@purestorage.com]
Sent: Wednesday, March 21, 2012 11:27 PM
To: frank zago
Cc: Pandit, Parav; linux-rdma@vger.kernel.org
Subject: Re: [PATCH 6/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
On Wed, Mar 21, 2012 at 10:42 AM, frank zago
fz...@systemfabricworks.com wrote:
On 03/20/2012 05:39 PM, parav.pan...@emulex.com wrote:
+struct ib_mr *ocrdma_get_dma_mr(struct ib_pd *ibpd, int acc) {
+ struct ocrdma_mr *mr;
+
+ mr = ocrdma_alloc_lkey(ibpd, acc, 0,
+ OCRDMA_ADDR_CHECK_DISABLE);
+ if (!mr)
+ return ERR_PTR(-ENOMEM);
ocrdma_alloc_lkey does not return NULL on error.
I'll fix this part.
Good catch! Even more to the point, ocrdma_alloc_lkey() doesn't return a
struct ocrdma_mr* on success. So this function is totally broken.
It does returns ocrdma_mr* on success. Why do you think it doesn't return?
Below is the snippet.
status = ocrdma_mbx_alloc_lkey(dev, mr-hwmr, pd-id, addr_check);
if (status) {
kfree(mr);
return ERR_PTR(-ENOMEM);
}
mr-pd = pd;
atomic_inc(pd-use_cnt);
mr-ibmr.lkey = mr-hwmr.lkey;
if (mr-hwmr.remote_wr || mr-hwmr.remote_rd)
mr-ibmr.rkey = mr-hwmr.lkey;
return mr;
- R.
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 4/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message-
From: Roland Dreier [mailto:rol...@purestorage.com]
Sent: Wednesday, March 21, 2012 10:04 PM
To: Pandit, Parav
Cc: linux-rdma@vger.kernel.org; net...@vger.kernel.org
Subject: Re: [PATCH 4/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
+int ocrdma_qp_state_machine(struct ocrdma_qp *qp, enum ib_qp_state
+new_ib_state,
+ enum ib_qp_state *old_ib_state) {
+ unsigned long flags;
+ int status = 0;
+ enum ocrdma_qp_state new_state;
+ new_state = get_ocrdma_qp_state(new_ib_state);
+
+ /* sync with wqe and rqe posting */
+ spin_lock_irqsave(qp-q_lock, flags);
+
+ if (old_ib_state)
+ *old_ib_state = get_ibqp_state(qp-state);
+ if (new_state == qp-state) {
+ spin_unlock_irqrestore(qp-q_lock, flags);
+ return 1;
+ }
+
+ switch (qp-state) {
+ case OCRDMA_QPS_RST:
+ switch (new_state) {
+ case OCRDMA_QPS_RST:
+ case OCRDMA_QPS_INIT:
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ };
+ break;
+ case OCRDMA_QPS_INIT:
+ /* qps: INIT-XXX */
+ switch (new_state) {
+ case OCRDMA_QPS_INIT:
+ case OCRDMA_QPS_RTR:
+ break;
+ case OCRDMA_QPS_ERR:
+ ocrdma_flush_qp(qp);
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ };
+ break;
+ case OCRDMA_QPS_RTR:
+ /* qps: RTS-XXX */
+ switch (new_state) {
+ case OCRDMA_QPS_RTS:
+ break;
+ case OCRDMA_QPS_ERR:
+ ocrdma_flush_qp(qp);
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ };
+ break;
+ case OCRDMA_QPS_RTS:
+ /* qps: RTS-XXX */
+ switch (new_state) {
+ case OCRDMA_QPS_SQD:
+ case OCRDMA_QPS_SQE:
+ break;
+ case OCRDMA_QPS_ERR:
+ ocrdma_flush_qp(qp);
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ };
+ break;
+ case OCRDMA_QPS_SQD:
+ /* qps: SQD-XXX */
+ switch (new_state) {
+ case OCRDMA_QPS_RTS:
+ case OCRDMA_QPS_SQE:
+ case OCRDMA_QPS_ERR:
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ };
+ break;
+ case OCRDMA_QPS_SQE:
+ switch (new_state) {
+ case OCRDMA_QPS_RTS:
+ case OCRDMA_QPS_ERR:
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ };
+ break;
+ case OCRDMA_QPS_ERR:
+ /* qps: ERR-XXX */
+ switch (new_state) {
+ case OCRDMA_QPS_RST:
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ };
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ };
+ if (!status)
+ qp-state = new_state;
+
+ spin_unlock_irqrestore(qp-q_lock, flags);
+ return status;
+}
The switch statement here seems to largely reimpliment
ib_modify_qp_is_ok() (which is exported from the rdma midlayer). Is there
some reason that doesn't work for your driver? I'd rather fix / generalize
the
core helper function instead of having something mostly duplicate in a
hardware driver.
Yes. Driver needs to put QP to flush state. So that appropriate CQEs can be
returned during poll_cq() phase.
So state machine is implemented above.
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 6/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message-
From: Roland Dreier [mailto:rol...@purestorage.com]
Sent: Wednesday, March 21, 2012 10:13 PM
To: Pandit, Parav
Cc: linux-rdma@vger.kernel.org; net...@vger.kernel.org
Subject: Re: [PATCH 6/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
+struct ib_pd *ocrdma_alloc_pd(struct ib_device *ibdev,
+ struct ib_ucontext *context,
+ struct ib_udata *udata) {
+ struct ocrdma_dev *dev = get_ocrdma_dev(ibdev);
+ struct ocrdma_pd *pd;
+ int status;
+
+ pd = kzalloc(sizeof(*pd), GFP_KERNEL);
+ if (!pd)
+ return ERR_PTR(-ENOMEM);
+ pd-dev = dev;
+ if (udata context) {
+ pd-dpp_enabled = (dev-nic_info.dev_family ==
+ OCRDMA_GEN2_FAMILY) ? true :
+ false;
Writing
(bool expr) ? true : false
is pretty silly, since it's just an obfuscated way of writing
bool expr
IOW, you can just write
pd-dpp_enabled = (dev-nic_info.dev_family ==
OCRDMA_GEN2_FAMILY);
+int ocrdma_dealloc_pd(struct ib_pd *ibpd) {
+ struct ocrdma_pd *pd = get_ocrdma_pd(ibpd);
+ struct ocrdma_dev *dev = pd-dev;
+ int status;
+ u64 usr_db;
+
+ if (atomic_read(pd-use_cnt)) {
+ ocrdma_err(%s(%d) pd=0x%x is in use.\n,
+ __func__, dev-id, pd-id);
+ status = -EFAULT;
+ goto dealloc_err;
+ }
all of the use_cnt tracking in this driver seems to duplicate what the rdma
midlayer already does... is there any reason we need that in the low-level
hardware driver too, or can we just get rid of the various use_cnt members?
This use_cnt can be removed from low-level hardware driver. I'll remove it.
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 4/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
-Original Message-
From: Roland Dreier [mailto:rol...@purestorage.com]
Sent: Thursday, March 22, 2012 1:02 AM
To: Pandit, Parav
Cc: linux-rdma@vger.kernel.org; net...@vger.kernel.org
Subject: Re: [PATCH 4/9] ocrdma: Driver for Emulex OneConnect RDMA
adapter
On Wed, Mar 21, 2012 at 12:09 PM, parav.pan...@emulex.com wrote:
Yes. Driver needs to put QP to flush state. So that appropriate CQEs can be
returned during poll_cq() phase.
So state machine is implemented above.
Couldn't you just write
if (ib_modify_qp_is_ok(...)) {
if (new_state == OCRDMA_QPS_ERR)
ocrdma_flush_qp(qp);
} else {
status = -EINVAL;
}
and save about 100 lines of code?
Yes, this can be done. This is one path.
Another path is async_event coming from adapter. So I still need
qp_state_machine function but as you suggested, I'll remove the states and will
have invoke flush_qp() on error.
- R.
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/2] ocrdma: Driver for Emulex OneConnect RDMA device.
From: Parav Pandit parav.pan...@emulex.com Emulex One Connect Adapter is RDMA (RoCE) capable multi-function PCI Express device. This driver patch enables RoCE support on such adapter. This ocrdma driver depends on be2net NIC driver. This patch depends on the previously submitted be2net NIC driver patch. Code organization: - ocrdma.h : driver header file. - ocrdma_main.c : driver registration with stack. - ocrdma_sli.h : driver-adapter interface definitions. - ocrdma_hw.*: hardware specific initialization, mailbox cmds. - ocrdma_verbs.* : verbs interface functionality. - ocrdma_ah.*: address handle related functionaliy. - ocrdma_abi.h : user space library interaction definitions. This patch is made against the current git tree. Please provide your review comments. Thank you. Parav Pandit (2): ocrdma: Driver for Emulex OneConnect RDMA device. ocrdma: Driver for Emulex OneConnect RDMA device. drivers/infiniband/Kconfig |1 + drivers/infiniband/Makefile |1 + drivers/infiniband/hw/ocrdma/Kconfig|8 + drivers/infiniband/hw/ocrdma/Makefile |5 + drivers/infiniband/hw/ocrdma/ocrdma.h | 392 drivers/infiniband/hw/ocrdma/ocrdma_abi.h | 134 ++ drivers/infiniband/hw/ocrdma/ocrdma_ah.c| 172 ++ drivers/infiniband/hw/ocrdma/ocrdma_ah.h| 42 + drivers/infiniband/hw/ocrdma/ocrdma_hw.c| 2640 +++ drivers/infiniband/hw/ocrdma/ocrdma_hw.h| 132 ++ drivers/infiniband/hw/ocrdma/ocrdma_main.c | 558 ++ drivers/infiniband/hw/ocrdma/ocrdma_sli.h | 1672 + drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 2542 ++ drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 94 + 14 files changed, 8393 insertions(+), 0 deletions(-) create mode 100644 drivers/infiniband/hw/ocrdma/Kconfig create mode 100644 drivers/infiniband/hw/ocrdma/Makefile create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma.h create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_abi.h create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_ah.c create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_ah.h create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_hw.c create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_hw.h create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_main.c create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_sli.h create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_verbs.h -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] ocrdma: Driver for Emulex OneConnect RDMA device.
From: Parav Pandit parav.pan...@emulex.com - Added entries to build ocrdma driver. Signed-off-by: Parav Pandit parav.pan...@emulex.com --- drivers/infiniband/Kconfig |1 + drivers/infiniband/Makefile |1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig index eb0add3..a0f29c1 100644 --- a/drivers/infiniband/Kconfig +++ b/drivers/infiniband/Kconfig @@ -51,6 +51,7 @@ source drivers/infiniband/hw/cxgb3/Kconfig source drivers/infiniband/hw/cxgb4/Kconfig source drivers/infiniband/hw/mlx4/Kconfig source drivers/infiniband/hw/nes/Kconfig +source drivers/infiniband/hw/ocrdma/Kconfig source drivers/infiniband/ulp/ipoib/Kconfig diff --git a/drivers/infiniband/Makefile b/drivers/infiniband/Makefile index a3b2d8e..bf846a1 100644 --- a/drivers/infiniband/Makefile +++ b/drivers/infiniband/Makefile @@ -8,6 +8,7 @@ obj-$(CONFIG_INFINIBAND_CXGB3) += hw/cxgb3/ obj-$(CONFIG_INFINIBAND_CXGB4) += hw/cxgb4/ obj-$(CONFIG_MLX4_INFINIBAND) += hw/mlx4/ obj-$(CONFIG_INFINIBAND_NES) += hw/nes/ +obj-$(CONFIG_INFINIBAND_OCRDMA)+= hw/ocrdma/ obj-$(CONFIG_INFINIBAND_IPOIB) += ulp/ipoib/ obj-$(CONFIG_INFINIBAND_SRP) += ulp/srp/ obj-$(CONFIG_INFINIBAND_SRPT) += ulp/srpt/ -- 1.6.0.2 -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/2] ocrdma: Driver for Emulex OneConnect RDMA device.
Ok. I'll resend it with smaller patches. Parav -Original Message- From: Roland Dreier [mailto:rol...@purestorage.com] Sent: Tuesday, March 20, 2012 10:21 PM To: Or Gerlitz Cc: Pandit, Parav; linux-rdma@vger.kernel.org Subject: Re: [PATCH 0/2] ocrdma: Driver for Emulex OneConnect RDMA device. On Tue, Mar 20, 2012 at 9:42 AM, Or Gerlitz ogerl...@mellanox.com wrote: patch 1/2 didn't reach the list, if it happens to be 100KB vger.kernel.org probably dropped it Parav, can you split the patch into a few pieces and resend? Thanks, Roland -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
From: Parav Pandit parav.pan...@emulex.com
- Header file for userspace library and kernel driver interface.
Signed-off-by: Parav Pandit parav.pan...@emulex.com
---
drivers/infiniband/hw/ocrdma/ocrdma_abi.h | 134 +
1 files changed, 134 insertions(+), 0 deletions(-)
create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_abi.h
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_abi.h
b/drivers/infiniband/hw/ocrdma/ocrdma_abi.h
new file mode 100644
index 000..a411a4e
--- /dev/null
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_abi.h
@@ -0,0 +1,134 @@
+/***
+ * This file is part of the Emulex RoCE Device Driver for *
+ * RoCE (RDMA over Converged Ethernet) adapters. *
+ * Copyright (C) 2008-2012 Emulex. All rights reserved.*
+ * EMULEX and SLI are trademarks of Emulex.*
+ * www.emulex.com *
+ * *
+ * This program is free software; you can redistribute it and/or *
+ * modify it under the terms of version 2 of the GNU General *
+ * Public License as published by the Free Software Foundation.*
+ * This program is distributed in the hope that it will be useful. *
+ * ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND *
+ * WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, *
+ * FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT, ARE *
+ * DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD *
+ * TO BE LEGALLY INVALID. See the GNU General Public License for *
+ * more details, a copy of which can be found in the file COPYING *
+ * included with this package. *
+ *
+ * Contact Information:
+ * linux-driv...@emulex.com
+ *
+ * Emulex
+ * Susan Street
+ * Costa Mesa, CA 92626
+ ***/
+
+#ifndef __OCRDMA_ABI_H__
+#define __OCRDMA_ABI_H__
+
+struct ocrdma_alloc_ucontext_resp {
+ u32 dev_id;
+ u32 wqe_size;
+ u32 max_inline_data;
+ u32 dpp_wqe_size;
+ u64 ah_tbl_page;
+ u32 ah_tbl_len;
+ u32 rsvd;
+ u8 fw_ver[32];
+ u32 rqe_size;
+ u64 rsvd1;
+} __packed;
+
+/* user kernel communication data structures. */
+struct ocrdma_alloc_pd_ureq {
+ u64 rsvd1;
+} __packed;
+
+struct ocrdma_alloc_pd_uresp {
+ u32 id;
+ u32 dpp_enabled;
+ u32 dpp_page_addr_hi;
+ u32 dpp_page_addr_lo;
+ u64 rsvd1;
+} __packed;
+
+struct ocrdma_create_cq_ureq {
+ u32 dpp_cq;
+ u32 rsvd;
+} __packed;
+
+#define MAX_CQ_PAGES 8
+struct ocrdma_create_cq_uresp {
+ u32 cq_id;
+ u32 page_size;
+ u32 num_pages;
+ u32 max_hw_cqe;
+ u64 page_addr[MAX_CQ_PAGES];
+ u64 db_page_addr;
+ u32 db_page_size;
+ u32 phase_change;
+ u64 rsvd1;
+ u64 rsvd2;
+} __packed;
+
+#define MAX_QP_PAGES 8
+#define MAX_UD_AV_PAGES 8
+
+struct ocrdma_create_qp_ureq {
+ u8 enable_dpp_cq;
+ u8 rsvd;
+ u16 dpp_cq_id;
+ u32 rsvd1;
+};
+
+struct ocrdma_create_qp_uresp {
+ u16 qp_id;
+ u16 sq_dbid;
+ u16 rq_dbid;
+ u16 resv0;
+ u32 sq_page_size;
+ u32 rq_page_size;
+ u32 num_sq_pages;
+ u32 num_rq_pages;
+ u64 sq_page_addr[MAX_QP_PAGES];
+ u64 rq_page_addr[MAX_QP_PAGES];
+ u64 db_page_addr;
+ u32 db_page_size;
+ u32 dpp_credit;
+ u32 dpp_offset;
+ u32 rsvd1;
+ u32 num_wqe_allocated;
+ u32 num_rqe_allocated;
+ u32 free_wqe_delta;
+ u32 free_rqe_delta;
+ u32 db_sq_offset;
+ u32 db_rq_offset;
+ u32 db_shift;
+ u64 rsvd2;
+ u64 rsvd3;
+} __packed;
+
+struct ocrdma_create_srq_uresp {
+ u16 rq_dbid;
+ u16 resv0;
+ u32 resv1;
+
+ u32 rq_page_size;
+ u32 num_rq_pages;
+
+ u64 rq_page_addr[MAX_QP_PAGES];
+ u64 db_page_addr;
+
+ u32 db_page_size;
+ u32 num_rqe_allocated;
+ u32 db_rq_offset;
+ u32 db_shift;
+
+ u32 free_rqe_delta;
+ u32 rsvd2;
+ u64 rsvd3;
+} __packed;
+
+#endif /* __OCRDMA_ABI_H__ */
--
1.6.0.2
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
From: Parav Pandit parav.pan...@emulex.com
- Header file for driver-adapter interface.
Signed-off-by: Parav Pandit parav.pan...@emulex.com
---
drivers/infiniband/hw/ocrdma/ocrdma_sli.h | 1672 +
1 files changed, 1672 insertions(+), 0 deletions(-)
create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_sli.h
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
b/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
new file mode 100644
index 000..7fd80cc
--- /dev/null
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_sli.h
@@ -0,0 +1,1672 @@
+/***
+ * This file is part of the Emulex RoCE Device Driver for *
+ * RoCE (RDMA over Converged Ethernet) adapters. *
+ * Copyright (C) 2008-2012 Emulex. All rights reserved.*
+ * EMULEX and SLI are trademarks of Emulex.*
+ * www.emulex.com *
+ * *
+ * This program is free software; you can redistribute it and/or *
+ * modify it under the terms of version 2 of the GNU General *
+ * Public License as published by the Free Software Foundation.*
+ * This program is distributed in the hope that it will be useful. *
+ * ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND *
+ * WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, *
+ * FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT, ARE *
+ * DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD *
+ * TO BE LEGALLY INVALID. See the GNU General Public License for *
+ * more details, a copy of which can be found in the file COPYING *
+ * included with this package. *
+ *
+ * Contact Information:
+ * linux-driv...@emulex.com
+ *
+ * Emulex
+ * Susan Street
+ * Costa Mesa, CA 92626
+ ***/
+
+#ifndef __OCRDMA_SLI_H__
+#define __OCRDMA_SLI_H__
+
+#define Bit(_b) (1 (_b))
+
+#define OCRDMA_GEN1_FAMILY 0xB
+#define OCRDMA_GEN2_FAMILY 0x2
+
+#define OCRDMA_SUBSYS_ROCE 10
+enum {
+ OCRDMA_CMD_QUERY_CONFIG = 1,
+ OCRDMA_CMD_ALLOC_PD,
+ OCRDMA_CMD_DEALLOC_PD,
+
+ OCRDMA_CMD_CREATE_AH_TBL,
+ OCRDMA_CMD_DELETE_AH_TBL,
+
+ OCRDMA_CMD_CREATE_QP,
+ OCRDMA_CMD_QUERY_QP,
+ OCRDMA_CMD_MODIFY_QP,
+ OCRDMA_CMD_DELETE_QP,
+
+ OCRDMA_CMD_RSVD1,
+ OCRDMA_CMD_ALLOC_LKEY,
+ OCRDMA_CMD_DEALLOC_LKEY,
+ OCRDMA_CMD_REGISTER_NSMR,
+ OCRDMA_CMD_REREGISTER_NSMR,
+ OCRDMA_CMD_REGISTER_NSMR_CONT,
+ OCRDMA_CMD_QUERY_NSMR,
+ OCRDMA_CMD_ALLOC_MW,
+ OCRDMA_CMD_QUERY_MW,
+
+ OCRDMA_CMD_CREATE_SRQ,
+ OCRDMA_CMD_QUERY_SRQ,
+ OCRDMA_CMD_MODIFY_SRQ,
+ OCRDMA_CMD_DELETE_SRQ,
+
+ OCRDMA_CMD_ATTACH_MCAST,
+ OCRDMA_CMD_DETACH_MCAST,
+
+ OCRDMA_CMD_MAX
+};
+
+#define OCRDMA_SUBSYS_COMMON 1
+enum {
+ OCRDMA_CMD_CREATE_CQ= 12,
+ OCRDMA_CMD_CREATE_EQ= 13,
+ OCRDMA_CMD_CREATE_MQ= 21,
+ OCRDMA_CMD_GET_FW_VER = 35,
+ OCRDMA_CMD_DELETE_MQ= 53,
+ OCRDMA_CMD_DELETE_CQ= 54,
+ OCRDMA_CMD_DELETE_EQ= 55,
+ OCRDMA_CMD_GET_FW_CONFIG= 58,
+ OCRDMA_CMD_CREATE_MQ_EXT= 90
+};
+
+enum {
+ QTYPE_EQ= 1,
+ QTYPE_CQ= 2,
+ QTYPE_MCCQ = 3
+};
+
+#define OCRDMA_MAX_SGID (8)
+
+#define OCRDMA_MAX_QP2048
+#define OCRDMA_MAX_CQ2048
+
+enum {
+ OCRDMA_DB_RQ_OFFSET = 0xE0,
+ OCRDMA_DB_GEN2_RQ1_OFFSET = 0x100,
+ OCRDMA_DB_GEN2_RQ2_OFFSET = 0xC0,
+ OCRDMA_DB_SQ_OFFSET = 0x60,
+ OCRDMA_DB_GEN2_SQ_OFFSET= 0x1C0,
+ OCRDMA_DB_SRQ_OFFSET= OCRDMA_DB_RQ_OFFSET,
+ OCRDMA_DB_GEN2_SRQ_OFFSET = OCRDMA_DB_GEN2_RQ1_OFFSET,
+ OCRDMA_DB_CQ_OFFSET = 0x120,
+ OCRDMA_DB_EQ_OFFSET = OCRDMA_DB_CQ_OFFSET,
+ OCRDMA_DB_MQ_OFFSET = 0x140
+};
+
+#define OCRDMA_DB_CQ_RING_ID_MASK 0x3FF /* bits 0 - 9 */
+#define OCRDMA_DB_CQ_RING_ID_EXT_MASK 0x0C00 /* bits 10-11 of qid at 12-11 */
+/* qid #2 msbits at 12-11 */
+#define OCRDMA_DB_CQ_RING_ID_EXT_MASK_SHIFT 0x1
+#define OCRDMA_DB_CQ_NUM_POPPED_SHIFT (16) /* bits 16 - 28 */
+/* Rearm bit */
+#define OCRDMA_DB_CQ_REARM_SHIFT(29) /* bit 29 */
+/* solicited bit */
+#define OCRDMA_DB_CQ_SOLICIT_SHIFT (31) /* bit 31 */
+
+#define OCRDMA_EQ_ID_MASK 0x1FF /* bits 0 - 8 */
+#define OCRDMA_EQ_ID_EXT_MASK 0x3e00 /* bits 9-13 */
+#define OCRDMA_EQ_ID_EXT_MASK_SHIFT(2) /* qid bits 9-13 at 11-15 */
+
+/* Clear the interrupt for this eq */
+#define OCRDMA_EQ_CLR_SHIFT(9) /* bit 9 */
[PATCH 5/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
From: Parav Pandit parav.pan...@emulex.com
Signed-off-by: Parav Pandit parav.pan...@emulex.com
---
drivers/infiniband/hw/ocrdma/ocrdma_main.c | 558
1 files changed, 558 insertions(+), 0 deletions(-)
create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_main.c
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_main.c
b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
new file mode 100644
index 000..8aa3416
--- /dev/null
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
@@ -0,0 +1,558 @@
+/***
+ * This file is part of the Emulex RoCE Device Driver for *
+ * RoCE (RDMA over Converged Ethernet) adapters. *
+ * Copyright (C) 2008-2012 Emulex. All rights reserved.*
+ * EMULEX and SLI are trademarks of Emulex.*
+ * www.emulex.com *
+ * *
+ * This program is free software; you can redistribute it and/or *
+ * modify it under the terms of version 2 of the GNU General *
+ * Public License as published by the Free Software Foundation.*
+ * This program is distributed in the hope that it will be useful. *
+ * ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND *
+ * WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, *
+ * FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT, ARE *
+ * DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD *
+ * TO BE LEGALLY INVALID. See the GNU General Public License for *
+ * more details, a copy of which can be found in the file COPYING *
+ * included with this package. *
+ *
+ * Contact Information:
+ * linux-driv...@emulex.com
+ *
+ * Emulex
+ * Susan Street
+ * Costa Mesa, CA 92626
+ ***/
+
+#include linux/module.h
+#include linux/version.h
+#include linux/idr.h
+#include rdma/ib_verbs.h
+#include rdma/ib_user_verbs.h
+#include rdma/ib_addr.h
+
+#include linux/netdevice.h
+#include net/addrconf.h
+
+#include ocrdma.h
+#include ocrdma_verbs.h
+#include ocrdma_ah.h
+#include be_roce.h
+#include ocrdma_hw.h
+
+MODULE_VERSION(OCRDMA_ROCE_DEV_VERSION);
+MODULE_DESCRIPTION(Emulex RoCE HCA Driver);
+MODULE_AUTHOR(Emulex Corporation);
+MODULE_LICENSE(GPL);
+
+static LIST_HEAD(ocrdma_dev_list);
+static DEFINE_MUTEX(ocrdma_devlist_lock);
+static DEFINE_IDR(ocrdma_dev_id);
+
+static union ib_gid ocrdma_zero_sgid;
+static int ocrdma_inet6addr_event(struct notifier_block *,
+ unsigned long, void *);
+
+static struct notifier_block ocrdma_inet6addr_notifier = {
+ .notifier_call = ocrdma_inet6addr_event
+};
+
+int ocrdma_get_instance(void)
+{
+ int instance = 0;
+
+ /* Assign an unused number */
+ if (!idr_pre_get(ocrdma_dev_id, GFP_KERNEL))
+ return -1;
+ if (idr_get_new(ocrdma_dev_id, NULL, instance))
+ return -1;
+ return instance;
+}
+
+void ocrdma_get_guid(struct ocrdma_dev *dev, u8 *guid)
+{
+ u8 mac_addr[6];
+
+ memcpy(mac_addr[0], dev-nic_info.mac_addr[0], ETH_ALEN);
+ guid[0] = mac_addr[0] ^ 2;
+ guid[1] = mac_addr[1];
+ guid[2] = mac_addr[2];
+ guid[3] = 0xff;
+ guid[4] = 0xfe;
+ guid[5] = mac_addr[3];
+ guid[6] = mac_addr[4];
+ guid[7] = mac_addr[5];
+}
+
+static void ocrdma_build_sgid_mac(union ib_gid *sgid, unsigned char *mac_addr,
+ bool is_vlan, u16 vlan_id)
+{
+ sgid-global.subnet_prefix = cpu_to_be64(0xfe80LL);
+ sgid-raw[8] = mac_addr[0] ^ 2;
+ sgid-raw[9] = mac_addr[1];
+ sgid-raw[10] = mac_addr[2];
+ if (is_vlan) {
+ sgid-raw[11] = vlan_id 8;
+ sgid-raw[12] = vlan_id 0xff;
+ } else {
+ sgid-raw[11] = 0xff;
+ sgid-raw[12] = 0xfe;
+ }
+ sgid-raw[13] = mac_addr[3];
+ sgid-raw[14] = mac_addr[4];
+ sgid-raw[15] = mac_addr[5];
+}
+
+static void ocrdma_add_sgid(struct ocrdma_dev *dev, unsigned char *mac_addr,
+ bool is_vlan, u16 vlan_id)
+{
+ int i;
+ bool found = false;
+ union ib_gid new_sgid;
+ int free_idx = OCRDMA_MAX_SGID;
+ unsigned long flags;
+
+ memset(ocrdma_zero_sgid, 0, sizeof(union ib_gid));
+
+ ocrdma_build_sgid_mac(new_sgid, mac_addr, is_vlan, vlan_id);
+
+ spin_lock_irqsave(dev-sgid_lock, flags);
+ for (i = 0; i OCRDMA_MAX_SGID; i++) {
+ if (!memcmp(dev-sgid_tbl[i], ocrdma_zero_sgid,
+ sizeof(union ib_gid))) {
+ /* found free entry */
+ if (!found) {
+ free_idx = i;
+ found = true;
+
[PATCH 7/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
From: Parav Pandit parav.pan...@emulex.com
- address handle specific handling.
Signed-off-by: Parav Pandit parav.pan...@emulex.com
---
drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 172 ++
drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 42 +++
2 files changed, 214 insertions(+), 0 deletions(-)
create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_ah.c
create mode 100644 drivers/infiniband/hw/ocrdma/ocrdma_ah.h
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
new file mode 100644
index 000..a877a8e
--- /dev/null
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
@@ -0,0 +1,172 @@
+/***
+ * This file is part of the Emulex RoCE Device Driver for *
+ * RoCE (RDMA over Converged Ethernet) adapters. *
+ * Copyright (C) 2008-2012 Emulex. All rights reserved.*
+ * EMULEX and SLI are trademarks of Emulex.*
+ * www.emulex.com *
+ * *
+ * This program is free software; you can redistribute it and/or *
+ * modify it under the terms of version 2 of the GNU General *
+ * Public License as published by the Free Software Foundation.*
+ * This program is distributed in the hope that it will be useful. *
+ * ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND *
+ * WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, *
+ * FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT, ARE *
+ * DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD *
+ * TO BE LEGALLY INVALID. See the GNU General Public License for *
+ * more details, a copy of which can be found in the file COPYING *
+ * included with this package. *
+ *
+ * Contact Information:
+ * linux-driv...@emulex.com
+ *
+ * Emulex
+ * Susan Street
+ * Costa Mesa, CA 92626
+ ***/
+
+#include net/neighbour.h
+#include net/netevent.h
+
+#include rdma/ib_addr.h
+#include rdma/ib_cache.h
+
+#include ocrdma.h
+#include ocrdma_verbs.h
+#include ocrdma_ah.h
+#include ocrdma_hw.h
+
+static inline int set_av_attr(struct ocrdma_ah *ah,
+ struct ib_ah_attr *attr, int pdid)
+{
+ int status = 0;
+ u16 vlan_tag; bool vlan_enabled = false;
+ struct ocrdma_dev *dev = ah-dev;
+ struct ocrdma_eth_vlan eth;
+ struct ocrdma_grh grh;
+ int eth_sz;
+
+ memset(eth, 0, sizeof(eth));
+ memset(grh, 0, sizeof(grh));
+
+ ah-sgid_index = attr-grh.sgid_index;
+
+ vlan_tag = rdma_get_vlan_id(attr-grh.dgid);
+ if (vlan_tag (vlan_tag 0x1000)) {
+ eth.eth_type = cpu_to_be16(0x8100);
+ eth.roce_eth_type = cpu_to_be16(OCRDMA_ROCE_ETH_TYPE);
+ vlan_tag |= (attr-sl 7) 13;
+ eth.vlan_tag = cpu_to_be16(vlan_tag);
+ eth_sz = sizeof(struct ocrdma_eth_vlan);
+ vlan_enabled = true;
+ } else {
+ eth.eth_type = cpu_to_be16(OCRDMA_ROCE_ETH_TYPE);
+ eth_sz = sizeof(struct ocrdma_eth_basic);
+ }
+ memcpy(eth.smac[0], dev-nic_info.mac_addr[0], ETH_ALEN);
+ status = ocrdma_resolve_dgid(dev, attr-grh.dgid, eth.dmac[0]);
+ if (status)
+ return status;
+ status = ocrdma_query_gid(dev-ibdev, 1, attr-grh.sgid_index,
+ (union ib_gid *)grh.sgid[0]);
+ if (status)
+ return status;
+
+ grh.tclass_flow = cpu_to_be32((6 28) |
+ (attr-grh.traffic_class 24) |
+ attr-grh.flow_label);
+ /* 0x1b is next header value in GRH */
+ grh.pdid_hoplimit = cpu_to_be32((pdid 16) |
+ (0x1b 8) | attr-grh.hop_limit);
+
+ memcpy(grh.dgid[0], attr-grh.dgid.raw, sizeof(attr-grh.dgid.raw));
+ memcpy(ah-av-eth_hdr, eth, eth_sz);
+ memcpy((u8 *)ah-av + eth_sz, grh, sizeof(struct ocrdma_grh));
+ if (vlan_enabled)
+ ah-av-valid |= OCRDMA_AV_VLAN_VALID;
+ return status;
+}
+
+struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr)
+{
+ u32 *ahid_addr;
+ int status;
+ struct ocrdma_ah *ah;
+ struct ocrdma_pd *pd = get_ocrdma_pd(ibpd);
+ struct ocrdma_dev *dev = pd-dev;
+
+ if (!(attr-ah_flags IB_AH_GRH))
+ return ERR_PTR(-EINVAL);
+
+ ah = kzalloc(sizeof *ah, GFP_ATOMIC);
+ if (!ah)
+ return ERR_PTR(-ENOMEM);
+ ah-dev = pd-dev;
+
+ status = ocrdma_alloc_av(dev, ah);
+ if (status)
+ goto av_err;
+ status = set_av_attr(ah, attr, pd-id);
+ if (status)
+ goto av_conf_err;
+
+ /* if pd is for the user
[PATCH 8/9] ocrdma: Driver for Emulex OneConnect RDMA adapter
From: Parav Pandit parav.pan...@emulex.com - build files for building ocrdma driver Signed-off-by: Parav Pandit parav.pan...@emulex.com --- drivers/infiniband/hw/ocrdma/Kconfig |8 drivers/infiniband/hw/ocrdma/Makefile |5 + 2 files changed, 13 insertions(+), 0 deletions(-) create mode 100644 drivers/infiniband/hw/ocrdma/Kconfig create mode 100644 drivers/infiniband/hw/ocrdma/Makefile diff --git a/drivers/infiniband/hw/ocrdma/Kconfig b/drivers/infiniband/hw/ocrdma/Kconfig new file mode 100644 index 000..cf99342 --- /dev/null +++ b/drivers/infiniband/hw/ocrdma/Kconfig @@ -0,0 +1,8 @@ +config INFINIBAND_OCRDMA + tristate Emulex One Connect HCA support + depends on ETHERNET NETDEVICES PCI + select NET_VENDOR_EMULEX + select BE2NET + ---help--- + This driver provides low-level InfiniBand over Ethernet + support for Emulex One Connect host channel adapters (HCAs). diff --git a/drivers/infiniband/hw/ocrdma/Makefile b/drivers/infiniband/hw/ocrdma/Makefile new file mode 100644 index 000..06a5bed --- /dev/null +++ b/drivers/infiniband/hw/ocrdma/Makefile @@ -0,0 +1,5 @@ +ccflags-y := -Idrivers/net/ethernet/emulex/benet + +obj-$(CONFIG_INFINIBAND_OCRDMA)+= ocrdma.o + +ocrdma-y :=ocrdma_main.o ocrdma_verbs.o ocrdma_hw.o ocrdma_ah.o -- 1.6.0.2 -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V1 0/2] be2net: Added functionality to support RoCE driver
From: Parav Pandit parav.pan...@emulex.com This patch addresses all the review comments given by Francois Romieu David Laight for past patch except be_roce_supported() macro as it breaks the modularity of be_cmds.x This patch adds functionality to support RoCE (RDMA over Ethernet) driver. - Detecting RoCE supported adapters and creating linked list of them. - Enabling 5 more MSIX vectors for RoCE functionality. - Calling registered callback functions of the RoCE driver whenever new RoCE capable device is added/removed. - Notifying events to RoCE driver when interface is up or down. - Provides device specific details to RoCE driver for each roce device. - Provides low level mailbox command to be issued by RoCE driver before it can have it own MQ. Parav Pandit (2): be2net: Added function to issue mailbox cmd on MQ. be2net: Added functionality to support RoCE driver drivers/net/ethernet/emulex/benet/Makefile |2 +- drivers/net/ethernet/emulex/benet/be.h | 38 ++- drivers/net/ethernet/emulex/benet/be_cmds.c | 39 ++ drivers/net/ethernet/emulex/benet/be_cmds.h |1 + drivers/net/ethernet/emulex/benet/be_hw.h |4 +- drivers/net/ethernet/emulex/benet/be_main.c | 83 ++-- drivers/net/ethernet/emulex/benet/be_roce.c | 183 +++ drivers/net/ethernet/emulex/benet/be_roce.h | 75 +++ 8 files changed, 409 insertions(+), 16 deletions(-) create mode 100644 drivers/net/ethernet/emulex/benet/be_roce.c create mode 100644 drivers/net/ethernet/emulex/benet/be_roce.h -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V1 2/2] be2net: Added functionality to support RoCE driver
From: Parav Pandit parav.pan...@emulex.com
- Increased MSIX vectors by 5 for RoCE traffic.
- Added macro to check roce support on a device.
- Added device specific doorbell, msix vector fields shared with nic
functionality.
- Provides RoCE driver registration and deregistration functions.
- Added support functions which will be invoked on adapter
add/remove and port up/down events.
- Traverses through the list of adapters for invoking callback functions.
Signed-off-by: Parav Pandit parav.pan...@emulex.com
---
drivers/net/ethernet/emulex/benet/Makefile |2 +-
drivers/net/ethernet/emulex/benet/be.h | 38 ++-
drivers/net/ethernet/emulex/benet/be_cmds.h |1 +
drivers/net/ethernet/emulex/benet/be_hw.h |4 +-
drivers/net/ethernet/emulex/benet/be_main.c | 83 ++--
drivers/net/ethernet/emulex/benet/be_roce.c | 183 +++
drivers/net/ethernet/emulex/benet/be_roce.h | 75 +++
7 files changed, 370 insertions(+), 16 deletions(-)
create mode 100644 drivers/net/ethernet/emulex/benet/be_roce.c
create mode 100644 drivers/net/ethernet/emulex/benet/be_roce.h
diff --git a/drivers/net/ethernet/emulex/benet/Makefile
b/drivers/net/ethernet/emulex/benet/Makefile
index a60cd80..1a91b27 100644
--- a/drivers/net/ethernet/emulex/benet/Makefile
+++ b/drivers/net/ethernet/emulex/benet/Makefile
@@ -4,4 +4,4 @@
obj-$(CONFIG_BE2NET) += be2net.o
-be2net-y := be_main.o be_cmds.o be_ethtool.o
+be2net-y := be_main.o be_cmds.o be_ethtool.o be_roce.o
diff --git a/drivers/net/ethernet/emulex/benet/be.h
b/drivers/net/ethernet/emulex/benet/be.h
index cbdec25..fc094d9 100644
--- a/drivers/net/ethernet/emulex/benet/be.h
+++ b/drivers/net/ethernet/emulex/benet/be.h
@@ -32,6 +32,7 @@
#include linux/u64_stats_sync.h
#include be_hw.h
+#include be_roce.h
#define DRV_VER4.0.100u
#define DRV_NAME be2net
@@ -92,7 +93,7 @@ static inline char *nic_name(struct pci_dev *pdev)
#define MAX_RSS_QS 4 /* BE limit is 4 queues/port */
#define MAX_RX_QS (MAX_RSS_QS + 1) /* RSS qs + 1 def Rx */
#define MAX_TX_QS 8
-#define BE_MAX_MSIX_VECTORS(MAX_RX_QS + 1)/* RX + TX */
+#define BE_MAX_MSIX_VECTORS(MAX_RX_QS + 1 + 5)/* RX + TX + 5 RoCE */
#define BE_NAPI_WEIGHT 64
#define MAX_RX_POSTBE_NAPI_WEIGHT /* Frags posted at a time */
#define RX_FRAGS_REFILL_WM (RX_Q_LEN - MAX_RX_POST)
@@ -320,6 +321,7 @@ struct be_adapter {
struct msix_entry msix_entries[BE_MAX_MSIX_VECTORS];
u32 num_msix_vec;
+ u32 num_eqs;
bool isr_registered;
/* TX Rings */
@@ -372,6 +374,16 @@ struct be_adapter {
u8 transceiver;
u8 autoneg;
u8 generation; /* BladeEngine ASIC generation */
+ u32 if_type;
+ struct {
+ u8 __iomem *base; /* Door Bell */
+ u32 size;
+ u32 total_size;
+ u64 io_addr;
+ } roce_db;
+ struct ocrdma_dev *ocrdma_dev;
+ struct list_head entry;
+
u32 flash_status;
struct completion flash_compl;
@@ -399,6 +411,10 @@ struct be_adapter {
#define lancer_chip(adapter) ((adapter-pdev-device == OC_DEVICE_ID3) || \
(adapter-pdev-device == OC_DEVICE_ID4))
+#define be_roce_supported(adapter) ((adapter-if_type == SLI_INTF_TYPE_3 || \
+ adapter-sli_family == SKYHAWK_SLI_FAMILY) \
+ (adapter-function_mode RDMA_ENABLED))
+
extern const struct ethtool_ops be_ethtool_ops;
#define msix_enabled(adapter) (adapter-num_msix_vec 0)
@@ -539,9 +555,29 @@ static inline bool be_error(struct be_adapter *adapter)
return adapter-eeh_err || adapter-ue_detected || adapter-fw_timeout;
}
+
+static inline bool be_type_2_3(struct be_adapter *adapter)
+{
+ return (adapter-if_type == SLI_INTF_TYPE_2 ||
+ adapter-if_type == SLI_INTF_TYPE_3) ? true : false;
+}
+
extern void be_cq_notify(struct be_adapter *adapter, u16 qid, bool arm,
u16 num_popped);
extern void be_link_status_update(struct be_adapter *adapter, u8 link_status);
extern void be_parse_stats(struct be_adapter *adapter);
extern int be_load_fw(struct be_adapter *adapter, u8 *func);
+
+/*
+ * internal function to initialize-cleanup roce device.
+ */
+extern void be_roce_dev_add(struct be_adapter *);
+extern void be_roce_dev_remove(struct be_adapter *);
+
+/*
+ * internal function to open-close roce device during ifup-ifdown.
+ */
+extern void be_roce_dev_open(struct be_adapter *);
+extern void be_roce_dev_close(struct be_adapter *);
+
#endif /* BE_H */
diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.h
b/drivers/net/ethernet/emulex/benet/be_cmds.h
index dca8924..d7ad03b 100644
--- a/drivers/net/ethernet/emulex/benet/be_cmds.h
+++ b/drivers/net/ethernet/emulex/benet/be_cmds.h
@@
[PATCH V1 1/2] be2net: Added function to issue mailbox cmd on MQ.
From: Parav Pandit parav.pan...@emulex.com
- Added generic function to issue mailbox cmd on MQ as export function.
- RoCE driver will use this before it setups its own MQ.
Signed-off-by: Parav Pandit parav.pan...@emulex.com
---
drivers/net/ethernet/emulex/benet/be_cmds.c | 39 +++
1 files changed, 39 insertions(+), 0 deletions(-)
diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.c
b/drivers/net/ethernet/emulex/benet/be_cmds.c
index 0fcb456..19037b0 100644
--- a/drivers/net/ethernet/emulex/benet/be_cmds.c
+++ b/drivers/net/ethernet/emulex/benet/be_cmds.c
@@ -15,6 +15,7 @@
* Costa Mesa, CA 92626
*/
+#include linux/module.h
#include be.h
#include be_cmds.h
@@ -2391,3 +2392,41 @@ err:
spin_unlock_bh(adapter-mcc_lock);
return status;
}
+
+int be_roce_mcc_cmd(void *netdev_handle, void *wrb_payload,
+ int wrb_payload_size, u16 *cmd_status, u16 *ext_status)
+{
+ struct be_adapter *adapter = netdev_priv(netdev_handle);
+ struct be_mcc_wrb *wrb;
+ struct be_cmd_req_hdr *hdr = (struct be_cmd_req_hdr *) wrb_payload;
+ struct be_cmd_req_hdr *req;
+ struct be_cmd_resp_hdr *resp;
+ int status;
+
+ spin_lock_bh(adapter-mcc_lock);
+
+ wrb = wrb_from_mccq(adapter);
+ if (!wrb) {
+ status = -EBUSY;
+ goto err;
+ }
+ req = embedded_payload(wrb);
+ resp = embedded_payload(wrb);
+
+ be_wrb_cmd_hdr_prepare(req, hdr-subsystem,
+ hdr-opcode, wrb_payload_size, wrb, NULL);
+ memcpy(req, wrb_payload, wrb_payload_size);
+ be_dws_cpu_to_le(req, wrb_payload_size);
+
+ status = be_mcc_notify_wait(adapter);
+ if (cmd_status)
+ *cmd_status = (status 0x);
+ if (ext_status)
+ *ext_status = 0;
+ memcpy(wrb_payload, resp, sizeof(*resp) + resp-response_length);
+ be_dws_le_to_cpu(wrb_payload, sizeof(*resp) + resp-response_length);
+err:
+ spin_unlock_bh(adapter-mcc_lock);
+ return status;
+}
+EXPORT_SYMBOL(be_roce_mcc_cmd);
--
1.6.0.2
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] be2net: Added function to issue mailbox cmd on MQ.
From: Parav Pandit parav.pan...@emulex.com
- Added generic function to issue mailbox cmd on MQ as export function.
- RoCE driver will use this before it setups its own MQ.
Signed-off-by: Parav Pandit parav.pan...@emulex.com
---
drivers/net/ethernet/emulex/benet/be_cmds.c | 39 +++
1 files changed, 39 insertions(+), 0 deletions(-)
diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.c
b/drivers/net/ethernet/emulex/benet/be_cmds.c
index 0fcb456..19037b0 100644
--- a/drivers/net/ethernet/emulex/benet/be_cmds.c
+++ b/drivers/net/ethernet/emulex/benet/be_cmds.c
@@ -15,6 +15,7 @@
* Costa Mesa, CA 92626
*/
+#include linux/module.h
#include be.h
#include be_cmds.h
@@ -2391,3 +2392,41 @@ err:
spin_unlock_bh(adapter-mcc_lock);
return status;
}
+
+int be_roce_mcc_cmd(void *netdev_handle, void *wrb_payload,
+ int wrb_payload_size, u16 *cmd_status, u16 *ext_status)
+{
+ struct be_adapter *adapter = netdev_priv(netdev_handle);
+ struct be_mcc_wrb *wrb;
+ struct be_cmd_req_hdr *hdr = (struct be_cmd_req_hdr *) wrb_payload;
+ struct be_cmd_req_hdr *req;
+ struct be_cmd_resp_hdr *resp;
+ int status;
+
+ spin_lock_bh(adapter-mcc_lock);
+
+ wrb = wrb_from_mccq(adapter);
+ if (!wrb) {
+ status = -EBUSY;
+ goto err;
+ }
+ req = embedded_payload(wrb);
+ resp = embedded_payload(wrb);
+
+ be_wrb_cmd_hdr_prepare(req, hdr-subsystem,
+ hdr-opcode, wrb_payload_size, wrb, NULL);
+ memcpy(req, wrb_payload, wrb_payload_size);
+ be_dws_cpu_to_le(req, wrb_payload_size);
+
+ status = be_mcc_notify_wait(adapter);
+ if (cmd_status)
+ *cmd_status = (status 0x);
+ if (ext_status)
+ *ext_status = 0;
+ memcpy(wrb_payload, resp, sizeof(*resp) + resp-response_length);
+ be_dws_le_to_cpu(wrb_payload, sizeof(*resp) + resp-response_length);
+err:
+ spin_unlock_bh(adapter-mcc_lock);
+ return status;
+}
+EXPORT_SYMBOL(be_roce_mcc_cmd);
--
1.6.0.2
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/2] be2net: Added functionality to support RoCE driver
From: Parav Pandit parav.pan...@emulex.com This patch adds functionality to support RoCE (RDMA over Ethernet) driver. - Detecting RoCE supported adapters and creating linked list of them. - Enabling 5 more MSIX vectors for RoCE functionality. - Calling registered callback functions of the RoCE driver whenever new RoCE capable device is added/removed. - Notifying events to RoCE driver when interface is up or down. - Provides device specific details to RoCE driver for each roce device. - Provides low level mailbox command to be issued by RoCE driver before it can have it own MQ. Parav Pandit (2): be2net: Added function to issue mailbox cmd on MQ. be2net: Added functionality to support RoCE driver drivers/net/ethernet/emulex/benet/Makefile |2 +- drivers/net/ethernet/emulex/benet/be.h | 28 - drivers/net/ethernet/emulex/benet/be_cmds.c | 39 ++ drivers/net/ethernet/emulex/benet/be_cmds.h |1 + drivers/net/ethernet/emulex/benet/be_hw.h |4 +- drivers/net/ethernet/emulex/benet/be_main.c | 83 +++-- drivers/net/ethernet/emulex/benet/be_roce.c | 185 +++ drivers/net/ethernet/emulex/benet/be_roce.h | 75 +++ 8 files changed, 404 insertions(+), 13 deletions(-) create mode 100644 drivers/net/ethernet/emulex/benet/be_roce.c create mode 100644 drivers/net/ethernet/emulex/benet/be_roce.h -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] be2net: Added functionality to support RoCE driver
From: Parav Pandit parav.pan...@emulex.com
- Increased MSIX vectors by 5 for RoCE traffic.
- Added macro to check roce support on a device.
- Added device specific doorbell, msix vector fields shared with nic
functionality.
- Provides RoCE driver registration and deregistration functions.
- Added support functions which will be invoked on adapter
add/remove and port up/down events.
- Traverses through the list of adapters for invoking callback functions.
Signed-off-by: Parav Pandit parav.pan...@emulex.com
---
drivers/net/ethernet/emulex/benet/Makefile |2 +-
drivers/net/ethernet/emulex/benet/be.h | 28 -
drivers/net/ethernet/emulex/benet/be_cmds.h |1 +
drivers/net/ethernet/emulex/benet/be_hw.h |4 +-
drivers/net/ethernet/emulex/benet/be_main.c | 83 +++--
drivers/net/ethernet/emulex/benet/be_roce.c | 185 +++
drivers/net/ethernet/emulex/benet/be_roce.h | 75 +++
7 files changed, 365 insertions(+), 13 deletions(-)
create mode 100644 drivers/net/ethernet/emulex/benet/be_roce.c
create mode 100644 drivers/net/ethernet/emulex/benet/be_roce.h
diff --git a/drivers/net/ethernet/emulex/benet/Makefile
b/drivers/net/ethernet/emulex/benet/Makefile
index a60cd80..1a91b27 100644
--- a/drivers/net/ethernet/emulex/benet/Makefile
+++ b/drivers/net/ethernet/emulex/benet/Makefile
@@ -4,4 +4,4 @@
obj-$(CONFIG_BE2NET) += be2net.o
-be2net-y := be_main.o be_cmds.o be_ethtool.o
+be2net-y := be_main.o be_cmds.o be_ethtool.o be_roce.o
diff --git a/drivers/net/ethernet/emulex/benet/be.h
b/drivers/net/ethernet/emulex/benet/be.h
index cbdec25..767f41f 100644
--- a/drivers/net/ethernet/emulex/benet/be.h
+++ b/drivers/net/ethernet/emulex/benet/be.h
@@ -32,6 +32,7 @@
#include linux/u64_stats_sync.h
#include be_hw.h
+#include be_roce.h
#define DRV_VER4.0.100u
#define DRV_NAME be2net
@@ -92,7 +93,7 @@ static inline char *nic_name(struct pci_dev *pdev)
#define MAX_RSS_QS 4 /* BE limit is 4 queues/port */
#define MAX_RX_QS (MAX_RSS_QS + 1) /* RSS qs + 1 def Rx */
#define MAX_TX_QS 8
-#define BE_MAX_MSIX_VECTORS(MAX_RX_QS + 1)/* RX + TX */
+#define BE_MAX_MSIX_VECTORS(MAX_RX_QS + 1 + 5)/* RX + TX + 5 RoCE */
#define BE_NAPI_WEIGHT 64
#define MAX_RX_POSTBE_NAPI_WEIGHT /* Frags posted at a time */
#define RX_FRAGS_REFILL_WM (RX_Q_LEN - MAX_RX_POST)
@@ -320,6 +321,7 @@ struct be_adapter {
struct msix_entry msix_entries[BE_MAX_MSIX_VECTORS];
u32 num_msix_vec;
+ u32 num_eqs;
bool isr_registered;
/* TX Rings */
@@ -372,6 +374,14 @@ struct be_adapter {
u8 transceiver;
u8 autoneg;
u8 generation; /* BladeEngine ASIC generation */
+ u32 if_type;
+ u8 __iomem *roce_db;/* Door Bell */
+ u32 roce_db_size;
+ u32 roce_db_total_size;
+ u64 roce_db_io_addr;
+ void *ocrdma_dev;
+ struct list_head entry;
+
u32 flash_status;
struct completion flash_compl;
@@ -398,6 +408,9 @@ struct be_adapter {
#define OFF0
#define lancer_chip(adapter) ((adapter-pdev-device == OC_DEVICE_ID3) || \
(adapter-pdev-device == OC_DEVICE_ID4))
+#define be_roce_supported(adapter) ((adapter-if_type == SLI_INTF_TYPE_3 || \
+ adapter-sli_family == SKYHAWK_SLI_FAMILY) \
+ (adapter-function_mode RDMA_ENABLED))
extern const struct ethtool_ops be_ethtool_ops;
@@ -544,4 +557,17 @@ extern void be_cq_notify(struct be_adapter *adapter, u16
qid, bool arm,
extern void be_link_status_update(struct be_adapter *adapter, u8 link_status);
extern void be_parse_stats(struct be_adapter *adapter);
extern int be_load_fw(struct be_adapter *adapter, u8 *func);
+
+/*
+ * internal function to initialize-cleanup roce device.
+ */
+extern void be_roce_dev_add(struct be_adapter *adapter);
+extern void be_roce_dev_remove(struct be_adapter *adapter);
+
+/*
+ * internal function to open-close roce device during ifup-ifdown.
+ */
+extern void be_roce_dev_open(struct be_adapter *adapter);
+extern void be_roce_dev_close(struct be_adapter *adapter);
+
#endif /* BE_H */
diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.h
b/drivers/net/ethernet/emulex/benet/be_cmds.h
index dca8924..d7ad03b 100644
--- a/drivers/net/ethernet/emulex/benet/be_cmds.h
+++ b/drivers/net/ethernet/emulex/benet/be_cmds.h
@@ -1057,6 +1057,7 @@ struct be_cmd_resp_modify_eq_delay {
#define FLEX10_MODE0x400
#define VNIC_MODE 0x2
#define UMC_ENABLED0x100
+#define RDMA_ENABLED 0x4
struct be_cmd_req_query_fw_cfg {
struct be_cmd_req_hdr hdr;
u32 rsvd[31];
diff --git
RE: [RFC 0/2] be2net: Added functionality to support RoCE driver
-Original Message- From: Roland Dreier [mailto:rol...@purestorage.com] Sent: Thursday, March 08, 2012 12:32 AM To: Pandit, Parav; David Miller Cc: net...@vger.kernel.org; linux-rdma@vger.kernel.org Subject: Re: [RFC 0/2] be2net: Added functionality to support RoCE driver On Wed, Mar 7, 2012 at 9:47 AM, parav.pan...@emulex.com wrote: From: Parav Pandit parav.pan...@emulex.com This patch is against roland's below Infiniband tree. http://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git Simiilar patch against netdev net-next tree will be made based on this patch. This patch adds functionality to support RoCE (RDMA over Ethernet) driver. - Detecting RoCE supported adapters and creating linked list of them. - Enabling 5 more MSIX vectors for RoCE functionality. - Calling registered callback functions of the RoCE driver whenever new RoCE capable device is added/removed. - Notifying events to RoCE driver when interface is up or down. - Provides device specific details to RoCE driver for each roce device. - Provides low level mailbox command to be issued by RoCE driver before it can have it own MQ. Parav Pandit (2): be2net: Added function to issue mailbox cmd on MQ. be2net: Added functionality to support RoCE driver Great, I'm really excited to be getting another upstream driver. I would suggest that when the actual RoCE driver is posted, we merge these 2 prerequisite patches + the driver through my tree, to avoid the hassle of a dependency between my tree and Dave's net-next tree. [PP] There are minor differences in be2net driver in your tree and net-next tree. So if we are fine with the interface between RoCE driver and NIC driver which is already available in this patch, then you can just merge RoCE driver patch from your tree to net-next tree. There will be separate patch against net-next tree for NIC driver which will be almost similar to this one. (Assuming these patches are OK with Dave and everyone else... they look pretty reasonable to me after a quick read) When do you expect to post the actual RoCE driver? If that's going to be a while, another possibility is to just merge these patches through net-next for 3.4, although I'm not sure about how Dave feels about merging all this stuff without seeing what will actually use it. [PP] I was just waiting for comments on this RFC/PATCH to submit separate patch for RoCE driver. But surely if RoCE driver patch helps review, I can send it right away. I'll prepare the patch for Roland's tree and send in few hours. Thanks, Roland -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] RDMA/cxgb4: Unblock reads on comp_channel
You are right, cq_lock will result into dead lock. Should there be a additional compl_handler spin_lock? I was measuring performance impact for adding it, and, irq_save() and irq_restore() variant showed additional 200 cycles, which I believe should be o.k.? Parav -Original Message- From: Steve Wise [mailto:sw...@opengridcomputing.com] Sent: Thursday, October 20, 2011 9:52 PM To: Roland Dreier Cc: Pandit, Parav; kuma...@chelsio.com; linux-rdma@vger.kernel.org; d...@chelsio.com Subject: Re: [PATCH] RDMA/cxgb4: Unblock reads on comp_channel On 10/20/2011 11:06 AM, Roland Dreier wrote: On Thu, Oct 20, 2011 at 2:11 AM,parav.pan...@emulex.com wrote: http://lxr.linux.no/#linux+v3.0.4/Documentation/infiniband/core_locking.txt Line no 66 to 97 states that - at a given point of time, there should be only one callback per CQ should be active. Is this ensured? compl_handler() is invoked from multiple places flush_qp() and post_qp_event() to my mind. Yes, does look like an issue with this patch :( And I think this bug exists in the T3 driver too. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
create new library project and few queries to open source
Hi, 1. I would like to submit user space library for a new RDMA adapter. Can you please do needful for creating empty project under http://git.openfabrics.org/git/projects/~ppandit/libocrdma.git? 2. For adding hardware driver for a new RDMA adapter, to which git tree(s) do I need to submit patches? openfabrics.org as well as kernel.org or just openfabrics is sufficient? git://git.openfabrics.org/ofed_1_5/linux-2.6.git, branch ofed_kernel_1_5? and git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git, branch for-next? 4. I am little confused. pub/scm/linux/kernel/git/roland/infiniband.git is not showing up in the list at http://git.kernel.org. Am I looking at wrong place? Or? 5. my_new_rdma_hw_driver that will be located in drivers/infiniband/hw/my_new_rdma_hw_driver directory, does it have to compile for all backports or only my desired kernel versions are sufficient for OFED 1.5.x, as I don't plan to support 2.6.22 and older kernel version? 6. If I have to support all backport kernel versions, do I need to submit patch for all the required kernel versions? Can this backport patches come at later stage? Regards, Parav Pandit -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
