Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-29 Thread Jason Gunthorpe
On Sat, Jun 27, 2015 at 12:12:11PM +0300, Sagi Grimberg wrote: Also since we have the new rdma_cap_read_multi_sge() helper, I thought I should use it. :) I think that reading the exact device caps max_sge, max_sge_is better and more straight forward... Right, rdma_cap_read_multi_sge was

Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-27 Thread Sagi Grimberg
Resending - somehow this didn't make it to the lists... On 6/25/2015 7:51 PM, Sagi Grimberg wrote: On 6/25/2015 6:39 PM, Steve Wise wrote: Memory regions that are the target of an iWARP RDMA READ RESPONSE need REMOTE_WRITE access rights. So enable REMOTE_WRITE for iWARP devices. iWARP RDMA

Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-27 Thread Sagi Grimberg
-devel Subject: Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport On 6/25/2015 6:39 PM, Steve Wise wrote: Memory regions that are the target of an iWARP RDMA READ RESPONSE need REMOTE_WRITE access rights. So enable REMOTE_WRITE for iWARP devices. iWARP RDMA READ target sge depth is 1. So

Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-27 Thread Sagi Grimberg
On 6/25/2015 10:29 PM, Jason Gunthorpe wrote: On Thu, Jun 25, 2015 at 02:25:49PM -0500, Steve Wise wrote: To stage the changes we could introduce a new function that returns the needed ib_access_flags value given the desired opcodes. Then have a series that changes all the existing ULPs to

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Steve Wise
Dayan' Subject: Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport On Thu, Jun 25, 2015 at 02:25:49PM -0500, Steve Wise wrote: To stage the changes we could introduce a new function that returns the needed ib_access_flags value given the desired opcodes. Then have a series

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Steve Wise
I don't understand why iWarp HW choose to ignore the verbs spec and not use IB_ACCESS_LOCAL_WRITE to cover RDMA READ responses... The iWARP verbs spec mandates this. Steve. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to

Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Sagi Grimberg
On 6/25/2015 6:39 PM, Steve Wise wrote: Memory regions that are the target of an iWARP RDMA READ RESPONSE need REMOTE_WRITE access rights. So enable REMOTE_WRITE for iWARP devices. iWARP RDMA READ target sge depth is 1. So save the max_read_sge in the target device structure and use that when

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Steve Wise
] RDMA/isert: Support iWARP transport On 6/25/2015 6:39 PM, Steve Wise wrote: Memory regions that are the target of an iWARP RDMA READ RESPONSE need REMOTE_WRITE access rights. So enable REMOTE_WRITE for iWARP devices. iWARP RDMA READ target sge depth is 1. So save the max_read_sge

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Hefty, Sean
+* IWARP transports need REMOTE_WRITE for MRs used as the target of +* an RDMA_READ. Since the DMA MR is used for all ports, then if +* any port is running IWARP, add REMOTE_WRITE. +*/ + if (any_port_is_iwarp(device)) It would be nice to have a new-style cap test

Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Jason Gunthorpe
On Thu, Jun 25, 2015 at 10:39:23AM -0500, Steve Wise wrote: + /* + * IWARP transports need REMOTE_WRITE for MRs used as the target of + * an RDMA_READ. Since the DMA MR is used for all ports, then if + * any port is running IWARP, add REMOTE_WRITE. + */ + if

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Steve Wise
-Original Message- From: Jason Gunthorpe [mailto:jguntho...@obsidianresearch.com] Sent: Thursday, June 25, 2015 1:25 PM To: Steve Wise Cc: linux-rdma@vger.kernel.org; sa...@mellanox.com; orgerl...@mellanox.com; r...@mellanox.com Subject: Re: [PATCH RFC 2/2] RDMA/isert: Support

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Steve Wise
...@mellanox.com Subject: RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport + * IWARP transports need REMOTE_WRITE for MRs used as the target of + * an RDMA_READ. Since the DMA MR is used for all ports, then if + * any port is running IWARP, add REMOTE_WRITE

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Hefty, Sean
How would you envision doing this? At the time a MR is registered the device driver doesn't know if the application will be doing RDMA reads or not on that MR. I was thinking of checking for REMOTE_READ, but that doesn't work on the initiator side. I guess you could a READ_DEST(SOURCE?

Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Jason Gunthorpe
On Thu, Jun 25, 2015 at 06:45:56PM +, Hefty, Sean wrote: How would you envision doing this? At the time a MR is registered the device driver doesn't know if the application will be doing RDMA reads or not on that MR. I was thinking of checking for REMOTE_READ, but that doesn't work

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Hefty, Sean
What about moving to something more specific? Encode the allowed verbs in the access flag? This makes more sense to me. Something like: SEND, RECV, INIT READ, INIT WRITE, READ TARGET, WRITE TARGET, etc. We're close to this, but it's not clear, for example, what flags are needed for a

RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Steve Wise
Dayan Subject: RE: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport What about moving to something more specific? Encode the allowed verbs in the access flag? This makes more sense to me. Something like: SEND, RECV, INIT READ, INIT WRITE, READ TARGET, WRITE TARGET, etc. We're close

Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

2015-06-25 Thread Jason Gunthorpe
On Thu, Jun 25, 2015 at 02:25:49PM -0500, Steve Wise wrote: To stage the changes we could introduce a new function that returns the needed ib_access_flags value given the desired opcodes. Then have a series that changes all the existing ULPs to make use of this new function. I wouldn't be