Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Christoph Lameter
On Mon, 28 Sep 2015, Or Gerlitz wrote: > Personally, up to few weeks ago, I was under the misimpression that > not only IPoIB joins as full member also on the sendonly flow, but > also that such group can be actually opened under that flow, and it > turns out they don't. Later you said that your p

Re: [PATCH 1/3] xprtrdma: disconnect and flush cqs before freeing buffers

2015-09-28 Thread Steve Wise
On 9/21/2015 12:24 PM, Steve Wise wrote: Otherwise a FRMR completion can cause a touch-after-free crash. In xprt_rdma_destroy(), call rpcrdma_buffer_destroy() only after calling rpcrdma_ep_destroy(). In rpcrdma_ep_destroy(), disconnect the cm_id first which should flush the qp, then drain the c

Re: [PATCH 2/3] svcrdma: handle rdma read with a non-zero initial page offset

2015-09-28 Thread Steve Wise
On 9/21/2015 12:24 PM, Steve Wise wrote: The server rdma_read_chunk_lcl() and rdma_read_chunk_frmr() functions were not taking into account the initial page_offset when determining the rdma read length. This resulted in a read who's starting address and length exceeded the base/bounds of the frm

Re: [PATCH 1/3] xprtrdma: disconnect and flush cqs before freeing buffers

2015-09-28 Thread Anna Schumaker
Hi Steve, On 09/28/2015 10:30 AM, Steve Wise wrote: > On 9/21/2015 12:24 PM, Steve Wise wrote: >> Otherwise a FRMR completion can cause a touch-after-free crash. >> >> In xprt_rdma_destroy(), call rpcrdma_buffer_destroy() only after calling >> rpcrdma_ep_destroy(). >> >> In rpcrdma_ep_destroy(), d

RE: [PATCH 1/3] xprtrdma: disconnect and flush cqs before freeing buffers

2015-09-28 Thread Steve Wise
> -Original Message- > From: Anna Schumaker [mailto:anna.schuma...@netapp.com] > Sent: Monday, September 28, 2015 9:45 AM > To: Steve Wise; trond.mykleb...@primarydata.com; bfie...@fieldses.org > Cc: linux-...@vger.kernel.org; linux-rdma@vger.kernel.org > Subject: Re: [PATCH 1/3] xprtrdma

Re: [PATCH 1/3] xprtrdma: disconnect and flush cqs before freeing buffers

2015-09-28 Thread Anna Schumaker
On 09/28/2015 10:50 AM, Steve Wise wrote: > > >> -Original Message- >> From: Anna Schumaker [mailto:anna.schuma...@netapp.com] >> Sent: Monday, September 28, 2015 9:45 AM >> To: Steve Wise; trond.mykleb...@primarydata.com; bfie...@fieldses.org >> Cc: linux-...@vger.kernel.org; linux-rdma@

Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Doug Ledford
On 09/27/2015 10:28 PM, Christoph Lameter wrote: > On Sun, 27 Sep 2015, Doug Ledford wrote: > >> Currently I'm testing your patch with a couple other patches. I dropped >> the patch of mine that added a module option, and added two different >> patches. However, I'm still waffling on this patch

libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Christoph Lameter
Where are these trees and who is maintaining them? I see that there is a libibverbs on kernel.org that is updated by Doug. There are some mlx4/5 trees around but those have no recent commits. There is a libmlx4 on kernel.org as well but the last merge there was by Roland in May 2014. -- To unsubs

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Doug Ledford
On 09/28/2015 11:42 AM, Christoph Lameter wrote: > Where are these trees and who is maintaining them? I see that there is a > libibverbs on kernel.org that is updated by Doug. > > There are some mlx4/5 trees around but those have no recent commits. There > is a libmlx4 on kernel.org as well but th

Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Christoph Lameter
On Mon, 28 Sep 2015, Doug Ledford wrote: > > We would like to keep > > irrelevant traffic off the fabric as much as possible. An a reception > > event that requires traffic to be thrown out will cause jitter in the > > processing of inbound traffic that we also would like to avoid. > > That may no

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Christoph Lameter
On Mon, 28 Sep 2015, Doug Ledford wrote: > On 09/28/2015 11:42 AM, Christoph Lameter wrote: > > Where are these trees and who is maintaining them? I see that there is a > > libibverbs on kernel.org that is updated by Doug. > > > > There are some mlx4/5 trees around but those have no recent commits

RE: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Woodruff, Robert J
Doug Ledford wrote: >git://git.openfabrics.org/~yishaih/libmlx4.git >git://git.openfabrics.org/~eli/libmlx5.git The OFA maintainers list contains this for the maintainers of these trees. ibmlx4 (upstream): Yishai Hadas http://www.openfabrics.org/downloads/mlx4 git://git.openfabrics.org/~yishai

Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Doug Ledford
On 09/28/2015 11:51 AM, Christoph Lameter wrote: > On Mon, 28 Sep 2015, Doug Ledford wrote: > >>> We would like to keep >>> irrelevant traffic off the fabric as much as possible. An a reception >>> event that requires traffic to be thrown out will cause jitter in the >>> processing of inbound traf

Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Christoph Lameter
On Mon, 28 Sep 2015, Doug Ledford wrote: > No, I was referring to using this on top of your patch and my other two > patches, which change the ipoib driver to create sendonly groups and > then expire them when the neighbor expires. Ok under which conditions could the joining be deferred and packe

Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Jason Gunthorpe
On Mon, Sep 28, 2015 at 11:36:11AM -0400, Doug Ledford wrote: > > Also broadcast could cause a unecessary reception event on the NICs of > > machines that have no interest in this traffic. > > This is true. However, I'm trying to balance between several competing > issues. You also stated the r

Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Christoph Lameter
Ok I refactored the whole thing to make it less invasive and keep more functionality in ipoib_multicast.c. Since you are working on it it would be best for you to have the newest version. I split this into two patches: One preparatory and one that implements the actual logic. Both attached. The pat

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Jason Gunthorpe
On Mon, Sep 28, 2015 at 11:43:47AM -0400, Doug Ledford wrote: > On 09/28/2015 11:42 AM, Christoph Lameter wrote: > > Where are these trees and who is maintaining them? I see that there is a > > libibverbs on kernel.org that is updated by Doug. > > > > There are some mlx4/5 trees around but those h

Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Christoph Lameter
On Mon, 28 Sep 2015, Jason Gunthorpe wrote: > I think your original idea of broadcast immediately and deferred > optimal mlid lookup is the best *functionally* for every case - only > when you enter the very edge world of caring about timing does it make > any difference. Infiniband is about the

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Christoph Lameter
On Mon, 28 Sep 2015, Jason Gunthorpe wrote: > Should we combine the user side of the kapi 'core' stack (libverbs, > all open source providers, libumad, libcm) into one source > package? Many projects have been working in that model lately with > some success, IMHO. Yes please. > Right now we eve

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Christoph Hellwig
On Mon, Sep 28, 2015 at 11:17:24AM -0600, Jason Gunthorpe wrote: > Should we combine the user side of the kapi 'core' stack (libverbs, > all open source providers, libumad, libcm) into one source > package? Many projects have been working in that model lately with > some success, IMHO. > > Many of

RE: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Woodruff, Robert J
On Mon, 28 Sep 2015, Christoph Lameter wrote: > Right. Its really nasty when you are trying to add features that require > libibverbs and libmlx? changes. Plus it may depend on kernel changes. On the other hand, combining everything into one package limits the ability of the maintainer of the i

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Christoph Hellwig
Hi Robert, getting a package out should not be an issue. master should always be in releasable state, and cutting a release should be a simple shell script doing all the tagging and uploading. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to ma

Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications)

2015-09-28 Thread Jason Gunthorpe
On Mon, Sep 28, 2015 at 12:19:04PM -0500, Christoph Lameter wrote: > > Christoph's needs would probably be better served by giving some API > > to control the mlid cache (ie the neightbour table is already 99% of > > the way there). This would let some userspace component pre-load and > > fix all

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Jason Gunthorpe
On Mon, Sep 28, 2015 at 05:28:20PM +, Woodruff, Robert J wrote: > On Mon, 28 Sep 2015, Christoph Lameter wrote: > > > Right. Its really nasty when you are trying to add features that require > > libibverbs and libmlx? changes. Plus it may depend on kernel changes. > > On the other hand, comb

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Jason Gunthorpe
On Mon, Sep 28, 2015 at 10:32:08AM -0700, Christoph Hellwig wrote: > > Hi Robert, > > getting a package out should not be an issue. master should always be > in releasable state, and cutting a release should be a simple shell > script doing all the tagging and uploading. +1 We are well past th

[PATCH RESEND] staging/rdma: Kconfig change STAGING_RDMA to be tristate.

2015-09-28 Thread ira . weiny
From: Ira Weiny STAGING_RDMA was failing to build when INFINIBAND was set to 'm' and STAGING_RDMA was set to 'y'. Making this a tristate properly inherits the 'm' from the INFINIBAND setting. Reviewed-by: Dalessandro, Dennis Reviewed-by: John, Jubin Reviewed-by: Mike Marciniszyn Signed-off-b

RE: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Woodruff, Robert J
On Mon, Sep 28, 2015 at 10:42:08AM -0700, Jason Gunthorpe wrote: >We are well past the point of doing experimental breaking stuff in the core >uapi libraries. If it is in the master git it should be shippable by a distro, >and it is so easy >to slap a version number on the HEAD if a distro/ofed/e

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Jason Gunthorpe
On Mon, Sep 28, 2015 at 06:18:52PM +, Woodruff, Robert J wrote: > On Mon, Sep 28, 2015 at 10:42:08AM -0700, Jason Gunthorpe wrote: > >We are well past the point of doing experimental breaking stuff in the core > >uapi libraries. If it is in the master git it should be shippable by a > >distro

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Christoph Lameter
On Mon, 28 Sep 2015, Jason Gunthorpe wrote: > I would guess all-together is actually less work for the distro side > because the whole candidate patch stream for backporting is right > there, easy to see, not sprinkled across all manner of git trees. This is true not only for the distro but also

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Steve Wise
On 9/28/2015 12:39 PM, Jason Gunthorpe wrote: On Mon, Sep 28, 2015 at 05:28:20PM +, Woodruff, Robert J wrote: On Mon, 28 Sep 2015, Christoph Lameter wrote: Right. Its really nasty when you are trying to add features that require libibverbs and libmlx? changes. Plus it may depend on kernel

Re: [PATCH v2 00/26] New fast registration API

2015-09-28 Thread Steve Wise
On 9/24/2015 12:34 PM, Sagi Grimberg wrote: I don't have access to other HW devices (qib, nes) nor iwarp devices so RDS is compile tested only. Santosh reports I'm targeting this to 4.4 so I'll appreciate more feedback and a bigger testing coverage. The code is available at: https://github.com/

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Doug Ledford
On 09/28/2015 03:00 PM, Jason Gunthorpe wrote: > On Mon, Sep 28, 2015 at 06:18:52PM +, Woodruff, Robert J wrote: >> On Mon, Sep 28, 2015 at 10:42:08AM -0700, Jason Gunthorpe wrote: >>> We are well past the point of doing experimental breaking stuff in the core >>> uapi libraries. If it is in t

RE: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Hefty, Sean
> Anyway, I had intended, and I've started on, making a change in how > libibverbs is done anyway. The idea that a new release is just a script > that throws a version on and we go is naive. I *will* be doing > pre-release rc tarballs and there will be testing and there will be a > release proces

RE: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Woodruff, Robert J
Sean wrote, >The issue is that basically no one tests the releases of the packages. We assume that the maintainer of the package tests it before they release it :) The OFED testing then tests all the currently released packages as a bundle with a released kernel version backported to a distro

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Doug Ledford
On 09/28/2015 04:40 PM, Woodruff, Robert J wrote: > Sean wrote, >> The issue is that basically no one tests the releases of the packages. > > We assume that the maintainer of the package tests it before they release it > :) Every package maintainer I know of does. But there isn't a single pac

RE: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Hefty, Sean
> >The issue is that basically no one tests the releases of the packages. > > We assume that the maintainer of the package tests it before they release > it :) No one has access to that many pieces of hardware and system configurations. The "community" can't dump the responsibility for testing

Re: [PATCH v1 01/24] IB/core: Introduce new fast registration API

2015-09-28 Thread Bart Van Assche
On 09/24/2015 12:37 AM, Sagi Grimberg wrote: On 9/23/2015 12:21 AM, Bart Van Assche wrote: On 09/17/2015 02:42 AM, Sagi Grimberg wrote: +} else if (last_page_off + dma_len < mr->page_size) { +/* chunk this fragment with the last */ +last_end_dma_addr

Re: [PATCH 2/3] svcrdma: handle rdma read with a non-zero initial page offset

2015-09-28 Thread J. Bruce Fields
On Mon, Sep 28, 2015 at 09:31:25AM -0500, Steve Wise wrote: > On 9/21/2015 12:24 PM, Steve Wise wrote: > >The server rdma_read_chunk_lcl() and rdma_read_chunk_frmr() functions > >were not taking into account the initial page_offset when determining > >the rdma read length. This resulted in a read

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Jason Gunthorpe
On Mon, Sep 28, 2015 at 04:22:50PM -0400, Doug Ledford wrote: > >> just a bug fix. Further, that new stuff might even require new > >> kernel code, so it could not just be replaced as a new user-space > >> package by a distro w/o updating the kernel. > > > > We are not going to make a change like

[PATCH RESEND] svcrdma: handle rdma read with a non-zero initial page offset

2015-09-28 Thread Steve Wise
The server rdma_read_chunk_lcl() and rdma_read_chunk_frmr() functions were not taking into account the initial page_offset when determining the rdma read length. This resulted in a read who's starting address and length exceeded the base/bounds of the frmr. Most work loads don't tickle this bug a

RE: [PATCH 2/3] svcrdma: handle rdma read with a non-zero initial page offset

2015-09-28 Thread Steve Wise
> -Original Message- > From: J. Bruce Fields [mailto:bfie...@fieldses.org] > Sent: Monday, September 28, 2015 4:05 PM > To: Steve Wise > Cc: trond.mykleb...@primarydata.com; linux-...@vger.kernel.org; > linux-rdma@vger.kernel.org > Subject: Re: [PATCH 2/3] svcrdma: handle rdma read with

RE: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Woodruff, Robert J
Jason wrote, >I was commenting specifically on the idea that we'd ever release a libverbs >that forced a kernel upgrade. I hope we all agree that is not acceptable. This means that user-space and kernel space packages can be released on separate schedules, so there is no need to tie the two toge

Re: libmlx4 and libmlx5 git trees? Who is handling those?

2015-09-28 Thread Doug Ledford
On 09/28/2015 06:08 PM, Woodruff, Robert J wrote: > Jason wrote, >> IMHO - we should be talking about getting to the point where we can >> deliver the kernel rc and uapi rc together to someplace like UNH >> and vendor internal labs and >expect them to test that pair. >> Regularly, ideally on the ke

Re: [PATCH v1 01/24] IB/core: Introduce new fast registration API

2015-09-28 Thread Christoph Hellwig
On Mon, Sep 28, 2015 at 01:57:52PM -0700, Bart Van Assche wrote: > >Actually I think it doesn't since it is only relevant for the else > >statement where we are passing the page_size boundary. > > Hello Sagi, > > Suppose that the following sg-list is passed to this function as { offset, > length

Re: [PATCH v1 01/24] IB/core: Introduce new fast registration API

2015-09-28 Thread Sagi Grimberg
On 9/29/2015 9:47 AM, Sagi Grimberg wrote: Shouldn't higher layers take care of this? Trying to implement the same coalescing algorithm at various layers isn't very optimal, although we need to decide and document which one is responsible. The block layer can take care of it, but I'm not sure

Re: [PATCH v1 01/24] IB/core: Introduce new fast registration API

2015-09-28 Thread Sagi Grimberg
On 9/28/2015 11:57 PM, Bart Van Assche wrote: On 09/24/2015 12:37 AM, Sagi Grimberg wrote: On 9/23/2015 12:21 AM, Bart Van Assche wrote: On 09/17/2015 02:42 AM, Sagi Grimberg wrote: +} else if (last_page_off + dma_len < mr->page_size) { +/* chunk this fragment with