Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Sagi Grimberg
On 10/6/2015 9:49 PM, Bart Van Assche wrote: On 10/06/2015 01:37 AM, Sagi Grimberg wrote: I see now the error you are referring to. The issue is that the device requires the MR page array to have an alignment (0x40 for mlx4 and 0x400 for mlx5). When I modified the page array allocation to be

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Christoph Hellwig
On Tue, Oct 06, 2015 at 11:37:40AM +0300, Sagi Grimberg wrote: > The issue is that the device requires the MR page array to have > an alignment (0x40 for mlx4 and 0x400 for mlx5). When I modified the > page array allocation to be non-coherent I didn't take care of > alignment. Just curious: why

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Christoph Hellwig
On Wed, Oct 07, 2015 at 12:25:25PM +0300, Sagi Grimberg wrote: > Bart suggested that having to sync once for the entire page list might > perform better than coherent memory. I'll settle either way since using > non-coherent memory might cause higher-order allocations due to > alignment, so it's

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Sagi Grimberg
I don't really care either way, it just seemed like an odd change hiding in here that I missed when reviewing earlier. OK, so I'm sticking with it until someone suggests otherwise. Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Sagi Grimberg
On 10/7/2015 12:20 PM, Christoph Hellwig wrote: On Tue, Oct 06, 2015 at 11:37:40AM +0300, Sagi Grimberg wrote: The issue is that the device requires the MR page array to have an alignment (0x40 for mlx4 and 0x400 for mlx5). When I modified the page array allocation to be non-coherent I didn't

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Sagi Grimberg
On 10/7/2015 6:46 PM, Bart Van Assche wrote: On 10/06/2015 11:42 PM, Sagi Grimberg wrote: On 10/6/2015 9:49 PM, Bart Van Assche wrote: On 10/06/2015 01:37 AM, Sagi Grimberg wrote: I see now the error you are referring to. The issue is that the device requires the MR page array to have an

Re: [PATCH v1 00/24] New fast registration API

2015-10-07 Thread Bart Van Assche
On 10/07/2015 02:20 AM, Christoph Hellwig wrote: On Tue, Oct 06, 2015 at 11:37:40AM +0300, Sagi Grimberg wrote: The issue is that the device requires the MR page array to have an alignment (0x40 for mlx4 and 0x400 for mlx5). When I modified the page array allocation to be non-coherent I didn't

Re: [PATCH v1 00/24] New fast registration API

2015-10-06 Thread Bart Van Assche
On 10/06/2015 01:37 AM, Sagi Grimberg wrote: > I see now the error you are referring to. > > The issue is that the device requires the MR page array to have > an alignment (0x40 for mlx4 and 0x400 for mlx5). When I modified the > page array allocation to be non-coherent I didn't take care of >

Re: [PATCH v1 00/24] New fast registration API

2015-10-02 Thread Bart Van Assche
On 10/01/2015 11:14 PM, Sagi Grimberg wrote: Would you mind sending me your .config? Hello Sagi, I just sent this .config file to you off-list. Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo

Re: [PATCH v1 00/24] New fast registration API

2015-10-01 Thread Bart Van Assche
On 10/01/2015 12:16 AM, Sagi Grimberg wrote: Just this morning (my morning) I tested the v2 set on iser, srp, nfs. I placed that in branch reg_api.5. Would you mind running reg_api.5 and see if this issue persist (I would be surprised because I haven't seen any sign of it)? Sorry but I still

Re: [PATCH v1 00/24] New fast registration API

2015-10-01 Thread Bart Van Assche
On 10/01/2015 10:53 AM, Bart Van Assche wrote: On 10/01/2015 12:16 AM, Sagi Grimberg wrote: I wander what is the difference between our test environments? I can't look into this if I'm not able to reproduce. Hello Sagi, At the target side I see "Sep 30 12:56:06 ibdev1 kernel: [178664.300296]

Re: [PATCH v1 00/24] New fast registration API

2015-10-01 Thread Sagi Grimberg
Just this morning (my morning) I tested the v2 set on iser, srp, nfs. I placed that in branch reg_api.5. Would you mind running reg_api.5 and see if this issue persist (I would be surprised because I haven't seen any sign of it)? Sorry but I still see these messages with the reg_api.5 branch.

Re: [PATCH v1 00/24] New fast registration API

2015-09-30 Thread Sagi Grimberg
Hi Bart, How has the converted SRP initiator driver been tested ? With the kernel tree that is available on branch reg_api.4 That's odd. Although I haven't formally submitted reg_api.4 yet, I did test ib_srp initiator against upstream srpt over CX3 (mlx4) and CX4 (mlx5). I ran connect,

Re: [PATCH v1 00/24] New fast registration API

2015-09-29 Thread Sagi Grimberg
On 9/29/2015 10:03 PM, Bart Van Assche wrote: On 09/17/2015 02:42 AM, Sagi Grimberg wrote: - Converted SRP initiator and RDS iwarp ULPs to the new API Hello Sagi, Hi Bart, How has the converted SRP initiator driver been tested ? With the kernel tree that is available on branch reg_api.4

Re: [PATCH v1 00/24] New fast registration API

2015-09-29 Thread Bart Van Assche
On 09/17/2015 02:42 AM, Sagi Grimberg wrote: - Converted SRP initiator and RDS iwarp ULPs to the new API Hello Sagi, How has the converted SRP initiator driver been tested ? With the kernel tree that is available on branch reg_api.4 (427def03e9fa9801efbb27f6c3c6bf7fc0d012e1) I see on the

Re: [PATCH v1 00/24] New fast registration API

2015-09-22 Thread santosh shilimkar
On 9/22/2015 12:56 AM, Sagi Grimberg wrote: On 9/22/2015 10:19 AM, Sagi Grimberg wrote: As mentioned earlier, I have a WIP RDS fastreg branch [3] which is functional (at least I can RDMA messages across nodes ;-)). Nice! So merging [2] and [3], I created [4] and applied a delta change

Re: [PATCH v1 00/24] New fast registration API

2015-09-22 Thread Bart Van Assche
On 09/17/2015 02:42 AM, Sagi Grimberg wrote: came from Bart Van Assache which pointed out that some applications Most people appreciate it if their name is spelled correctly :-) Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to

Re: [PATCH v1 00/24] New fast registration API

2015-09-22 Thread Sagi Grimberg
On 9/22/2015 10:19 AM, Sagi Grimberg wrote: As mentioned earlier, I have a WIP RDS fastreg branch [3] which is functional (at least I can RDMA messages across nodes ;-)). Nice! So merging [2] and [3], I created [4] and applied a delta change based on your other patches. I saw ib_post_send

Re: [PATCH v1 00/24] New fast registration API

2015-09-22 Thread Sagi Grimberg
As mentioned earlier, I have a WIP RDS fastreg branch [3] which is functional (at least I can RDMA messages across nodes ;-)). Nice! So merging [2] and [3], I created [4] and applied a delta change based on your other patches. I saw ib_post_send failure with my HCA driver returning

Re: [PATCH v1 00/24] New fast registration API

2015-09-21 Thread santosh.shilim...@oracle.com
Hi Sagi, On 9/20/15 2:36 AM, Sagi Grimberg wrote: Hi Santosh, Nice to see this consolidaton happening. I too don't have access to iWARP hardware for RDS test but will use this series and convert our WIP IB fastreg code and see how it goes. I'm very pleased to hear about this WIP. Please

Re: [PATCH v1 00/24] New fast registration API

2015-09-19 Thread Christoph Hellwig
Hi Sagi, I've converted the driver I'm developing to your API and it works great. I think this is an important step towards making the RDMA more usable! -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo

Re: [PATCH v1 00/24] New fast registration API

2015-09-19 Thread santosh.shilim...@oracle.com
On 9/17/15 5:42 AM, Sagi Grimberg wrote: Hi all, As discussed on the linux-rdma list, there is plenty of room for improvement in our memory registration APIs. We keep finding ULPs that are duplicating code, sometimes use wrong strategies and mis-use our current API. As a first step, this patch

[PATCH v1 00/24] New fast registration API

2015-09-17 Thread Sagi Grimberg
Hi all, As discussed on the linux-rdma list, there is plenty of room for improvement in our memory registration APIs. We keep finding ULPs that are duplicating code, sometimes use wrong strategies and mis-use our current API. As a first step, this patch set replaces the fast registration API to