Re: device attr cleanup (was: Handle mlx4 max_sge_rd correctly)

On Thu, Dec 10, 2015 at 07:49:59PM -0500, Chuck Lever wrote: > > > On Dec 10, 2015, at 6:30 PM, Christoph Hellwig wrote: > > > > On Thu, Dec 10, 2015 at 11:07:03AM -0700, Jason Gunthorpe wrote: > >> The ARM folks do this sort of stuff on a regular basis.. Very early on > >>

Re: [PATCH v4 01/11] svcrdma: Do not send XDR roundup bytes for a write chunk

On Mon, Dec 21, 2015 at 05:11:56PM -0500, Chuck Lever wrote: > > > On Dec 21, 2015, at 4:29 PM, J. Bruce Fields <bfie...@fieldses.org> wrote: > > > > On Mon, Dec 21, 2015 at 04:15:23PM -0500, Chuck Lever wrote: > >> > >>> On Dec 21, 2015, at 4:07

Re: [PATCH v4 01/11] svcrdma: Do not send XDR roundup bytes for a write chunk

On Mon, Dec 14, 2015 at 04:30:09PM -0500, Chuck Lever wrote: > Minor optimization: when dealing with write chunk XDR roundup, do > not post a Write WR for the zero bytes in the pad. Simply update > the write segment in the RPC-over-RDMA header to reflect the extra > pad bytes. > > The Reply chunk

Re: [PATCH v4 01/11] svcrdma: Do not send XDR roundup bytes for a write chunk

On Mon, Dec 21, 2015 at 04:15:23PM -0500, Chuck Lever wrote: > > > On Dec 21, 2015, at 4:07 PM, J. Bruce Fields <bfie...@fieldses.org> wrote: > > > > On Mon, Dec 14, 2015 at 04:30:09PM -0500, Chuck Lever wrote: > >> Minor optimization: when dealing with write

Re: warning in ext4 with nfs/rdma server

On Tue, Dec 08, 2015 at 07:31:56AM -0600, Steve Wise wrote: > > > > -Original Message- > > From: Chuck Lever [mailto:chuck.le...@oracle.com] > > Sent: Monday, December 07, 2015 9:45 AM > > To: Steve Wise > > Cc: linux-rdma@vger.kernel.org; Veeresh U. Kokatnur; Linux NFS Mailing List > >

Re: [PATCH] svcrdma: Do not send XDR roundup bytes for a write chunk

On Wed, Nov 11, 2015 at 03:20:33PM -0500, Chuck Lever wrote: > > > On Nov 3, 2015, at 3:10 PM, Chuck Lever wrote: > > > > > >> On Oct 16, 2015, at 9:30 AM, Chuck Lever wrote: > >> > >> Minor optimization: when dealing with write chunk XDR

Re: Fwd: [PATCH] svcrdma: Do not send XDR roundup bytes for a write chunk

On Thu, Nov 12, 2015 at 09:47:13AM -0500, Chuck Lever wrote: > I wish git send-mail had an address book. I’m really tired > of misspelling the to/cc addresses on patches. Hah. Yeah, my hack is a text file of addresses that I cut-and-paste from. --b. > > Resending so there is one thread for

Re: [PATCH] svcrdma: Fix NFS server crash triggered by 1MB NFS WRITE

On Mon, Oct 12, 2015 at 10:53:39AM -0400, Chuck Lever wrote: > Now that the NFS server advertises a maximum payload size of 1MB > for RPC/RDMA again, it crashes in svc_process_common() when NFS > client sends a 1MB NFS WRITE on an NFS/RDMA mount. > > The server has set up a 259 element array of

Re: [PATCH RESEND] svcrdma: handle rdma read with a non-zero initial page offset

On Wed, Oct 07, 2015 at 01:33:05PM -0400, Doug Ledford wrote: > On 10/07/2015 01:01 PM, J. Bruce Fields wrote: > > On Tue, Oct 06, 2015 at 01:44:25PM -0400, Doug Ledford wrote: > >> On 09/28/2015 05:46 PM, Steve Wise wrote: > >>> The server rdma_read_chunk_

Re: [PATCH RESEND] svcrdma: handle rdma read with a non-zero initial page offset

On Tue, Oct 06, 2015 at 01:44:25PM -0400, Doug Ledford wrote: > On 09/28/2015 05:46 PM, Steve Wise wrote: > > The server rdma_read_chunk_lcl() and rdma_read_chunk_frmr() functions > > were not taking into account the initial page_offset when determining > > the rdma read length. This resulted in

Re: [PATCH 2/3] svcrdma: handle rdma read with a non-zero initial page offset

On Mon, Sep 28, 2015 at 04:49:21PM -0500, Steve Wise wrote: > > > > -Original Message- > > From: J. Bruce Fields [mailto:bfie...@fieldses.org] > > Sent: Monday, September 28, 2015 4:05 PM > > To: Steve Wise > > Cc: trond.mykleb...@primarydata.com; l

Re: [PATCH 2/3] svcrdma: handle rdma read with a non-zero initial page offset

On Mon, Sep 28, 2015 at 09:31:25AM -0500, Steve Wise wrote: > On 9/21/2015 12:24 PM, Steve Wise wrote: > >The server rdma_read_chunk_lcl() and rdma_read_chunk_frmr() functions > >were not taking into account the initial page_offset when determining > >the rdma read length. This resulted in a read

Re: [PATCH] svcrdma: limit FRMR page list lengths to device max

(), which shouldn't conflict. Acked-by: J. Bruce Fields bfie...@redhat.com --b. --- net/sunrpc/xprtrdma/svc_rdma_transport.c |6 -- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma

Re: [PATCH RESEND 0/4] Use max_sge_rd device capability

to those files in my for-next as well, but they don't appear to conflict. Feel free to add my Acked-by: J. Bruce Fields bfie...@redhat.com to patch 3/4 if it helps. --b. --- Sagi Grimberg (1): mlx4, mlx5, mthca: Expose max_sge_rd correctly Steve Wise (3): RDMA/Core

Re: [PATCH v1 5/5] svcrdma: Boost NFS READ/WRITE payload size maximum

On Fri, Jul 10, 2015 at 10:59:48AM -0400, Chuck Lever wrote: On Jul 10, 2015, at 10:18 AM, bfie...@fieldses.org wrote: On Thu, Jul 09, 2015 at 04:45:47PM -0400, Chuck Lever wrote: Increased to 1 megabyte. Why not more or less? Why do we even have this constant, why shouldn't we

Re: [PATCH v1 5/5] svcrdma: Boost NFS READ/WRITE payload size maximum

On Fri, Jul 10, 2015 at 10:18:14AM -0400, J. Bruce Fields wrote: On Thu, Jul 09, 2015 at 04:45:47PM -0400, Chuck Lever wrote: Increased to 1 megabyte. Why not more or less? Why do we even have this constant, why shouldn't we just use RPCSVC_MAXPAYLOAD? (That one question aside

Re: [PATCH v1 5/5] svcrdma: Boost NFS READ/WRITE payload size maximum

On Thu, Jul 09, 2015 at 04:45:47PM -0400, Chuck Lever wrote: Increased to 1 megabyte. Why not more or less? Why do we even have this constant, why shouldn't we just use RPCSVC_MAXPAYLOAD? --b. Signed-off-by: Chuck Lever chuck.le...@oracle.com --- include/linux/sunrpc/svc_rdma.h |2

Re: [PATCH v1 5/5] svcrdma: Boost NFS READ/WRITE payload size maximum

On Fri, Jul 10, 2015 at 11:59:20AM -0400, Chuck Lever wrote: On Jul 10, 2015, at 11:54 AM, J. Bruce Fields bfie...@fieldses.org wrote: On Fri, Jul 10, 2015 at 10:59:48AM -0400, Chuck Lever wrote: On Jul 10, 2015, at 10:18 AM, bfie...@fieldses.org wrote: On Thu, Jul 09, 2015 at 04

Re: NFSoRDMA bi-weekly developers meeting minutes (5/28/15)

On Thu, May 28, 2015 at 11:11:30AM -0700, Shirley Ma wrote: NFSD workqueue mode vs. threading mode performance impacts: (Shirley) - Shirley has spent sometime test/fix/evaluating NFSD workqueue mode originally implemented by

Re: [PATCH v2 00/10] NFS/RDMA server for 3.20

On Tue, Jan 13, 2015 at 11:31:16AM -0600, Steve Wise wrote: Reviewed-by: Steve Wise sw...@opengridcomputing.com Thanks, applying for 3.20. --b. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [PATCH v1 09/10] Move read list XDR round-up logic

On Fri, Jan 09, 2015 at 02:23:11PM -0500, Chuck Lever wrote: Simpler approach, and this needs to be done before the tail is copied to the end of the read list (pre-requisite for a subsequent patch). Fixes: e560e3b510d2 (svcrdma: Add zero padding if the client... ) Wait, does this actually

Re: [PATCH v1 00/10] NFS/RDMA server for 3.20

On Fri, Jan 09, 2015 at 02:21:54PM -0500, Chuck Lever wrote: In addition to miscellaneous clean up, the following series of patches for the Linux NFS server introduces support in the server’s RPC/RDMA transport implementation for RDMA_NOMSG type messages, and fixes a bug that prevents the

Re: [PATCH v1 00/10] NFS/RDMA server for 3.20

On Fri, Jan 09, 2015 at 03:40:51PM -0500, Chuck Lever wrote: On Jan 9, 2015, at 3:39 PM, J. Bruce Fields bfie...@fieldses.org wrote: On Fri, Jan 09, 2015 at 02:21:54PM -0500, Chuck Lever wrote: In addition to miscellaneous clean up, the following series of patches for the Linux NFS

Re: [PATCH v2 06/10] xprtrdma: Enable pad optimization

On Mon, Nov 10, 2014 at 08:54:27AM -0600, Chuck Lever wrote: On Nov 10, 2014, at 8:36 AM, Anna Schumaker anna.schuma...@netapp.com wrote: Hey Chuck, On 11/08/2014 08:14 PM, Chuck Lever wrote: The Linux NFS/RDMA server used to reject NFSv3 WRITE requests when pad optimization was

Re: [PATCH V3] svcrdma: advertise the correct max payload

On Mon, Sep 29, 2014 at 11:07:25AM -0500, Steve Wise wrote: Hey Bruce, is this version acceptable for 3.18? Yes, applying, thanks for the reminder.--b. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo

Re: [PATCH V2] svcrdma: Advertise the correct max payload

On Tue, Sep 23, 2014 at 02:42:34PM -0500, Steve Wise wrote: diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index c419498..a9cf5c3 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -392,4 +392,8 @@ extern

Re: [PATCH V2] svcrdma: Advertise the correct max payload

On Tue, Sep 23, 2014 at 02:53:28PM -0500, Steve Wise wrote: -Original Message- From: 'J. Bruce Fields' [mailto:bfie...@fieldses.org] Sent: Tuesday, September 23, 2014 2:48 PM To: Steve Wise Cc: 'Chuck Lever'; linux-...@vger.kernel.org; linux-rdma@vger.kernel.org Subject: Re

Re: [PATCH] svcrdma: Advertise the correct max payload

On Mon, Sep 22, 2014 at 01:36:53PM -0500, Steve Wise wrote: Svcrdma currently advertises 1MB, which is too large. The correct value is the max scatter-gather allowed in an NFSRDMA IO chunk * the host page size. This bug is usually benign because the Linux X64 NFSRDMA client correctly limits

Re: [PATCH] svcrdma: remove rdma_create_qp() failure recovery logic

Thanks, applying for 3.17.--b. On Thu, Jul 31, 2014 at 03:26:07PM -0500, Steve Wise wrote: In svc_rdma_accept(), if rdma_create_qp() fails, there is useless logic to try and call rdma_create_qp() again with reduced sge depths. The assumption, I guess, was that perhaps the initial sge depths

Re: [PATCH 2/2] svcrdma: Remove extra writeargs sanity check for NFSv2/3

On Thu, Jul 10, 2014 at 04:43:54PM -0400, Chuck Lever wrote: I think the sanity check you pointed out is strictly satisfied by testing against the unaligned number of bytes. Is there a strong reason to do the extra math for that check during each WRITE? I can't think of any good reason why the

Re: [PATCH 2/2] svcrdma: Remove extra writeargs sanity check for NFSv2/3

On Thu, Jul 10, 2014 at 01:44:35PM -0400, Chuck Lever wrote: The server should comply with RFC 5667, Where's the relevant language? (I took a quick look but didn't see it.) and handle WRITE payloads that may not have a zero pad on the end (XDR length round-up). Fixes: f34b9568 (The

Re: [PATCH 2/2] svcrdma: Remove extra writeargs sanity check for NFSv2/3

On Thu, Jul 10, 2014 at 02:24:57PM -0400, Chuck Lever wrote: On Jul 10, 2014, at 2:19 PM, J. Bruce Fields bfie...@fieldses.org wrote: On Thu, Jul 10, 2014 at 01:44:35PM -0400, Chuck Lever wrote: The server should comply with RFC 5667, Where's the relevant language? (I took a quick

Re: [PATCH 2/2] svcrdma: Remove extra writeargs sanity check for NFSv2/3

On Thu, Jul 10, 2014 at 03:07:34PM -0400, Chuck Lever wrote: Hi Bruce- On Jul 10, 2014, at 2:49 PM, J. Bruce Fields bfie...@fieldses.org wrote: On Thu, Jul 10, 2014 at 02:24:57PM -0400, Chuck Lever wrote: On Jul 10, 2014, at 2:19 PM, J. Bruce Fields bfie...@fieldses.org wrote

Re: [PATCH] svcrdma: send_write() must not overflow the device's max sge

On Wed, Jul 09, 2014 at 01:49:15PM -0500, Steve Wise wrote: Function send_write() must stop creating sges when it reaches the device max and return the amount sent in the RDMA Write to the caller. Queueing up for 3.17, thanks.--b. Signed-off-by: Steve Wise sw...@opengridcomputing.com ---

Re: [PATCH V2] svcrdma: Fence LOCAL_INV work requests

On Thu, Jun 05, 2014 at 09:54:31AM -0500, Steve Wise wrote: This applies on top of: commit e5a070216356dbcb03607cb264cc3104e17339b3 Author: Steve Wise sw...@opengridcomputing.com Date: Wed May 28 15:12:01 2014 -0500 svcrdma: refactor marshalling logic Sorry,

Re: [PATCH V3] svcrdma: refactor marshalling logic

On Mon, Jun 02, 2014 at 11:47:58AM -0500, Steve Wise wrote: Steve I have not checked the code because I am away from my laptop But I assume mlx and mthca is using fmr and cxgb4 is using rdma-read-with-invalidate which has implicit fence. If invalidate is issued before read

Re: [PATCH V3] svcrdma: refactor marshalling logic

On Mon, Jun 02, 2014 at 11:52:47AM -0500, Steve Wise wrote: You're correct. And this bug appears to be in the current upstream code as well. If an IB_WR_LOCAL_INV wr is used, it must include IB_SEND_FENCE to fence it until the prior read completes. Good catch! I'll

Re: [PATCH V3] svcrdma: refactor marshalling logic

On Mon, Jun 02, 2014 at 12:06:39PM -0500, Steve Wise wrote: On Mon, Jun 02, 2014 at 11:52:47AM -0500, Steve Wise wrote: You're correct. And this bug appears to be in the current upstream code as well. If an IB_WR_LOCAL_INV wr is used, it must include IB_SEND_FENCE to fence

Re: [PATCH] NFSD: Ignore client's source port on RDMA transports

On Mon, May 19, 2014 at 01:40:22PM -0400, Chuck Lever wrote: An NFS/RDMA client's source port is meaningless for RDMA transports. The transport layer typically sets the source port value on the connection to a random ephemeral port. Currently, NFS server administrators must specify the

Re: [PATCH] NFSD: Ignore client's source port on RDMA transports

On Thu, May 22, 2014 at 03:55:17PM -0400, bfields wrote: On Mon, May 19, 2014 at 01:40:22PM -0400, Chuck Lever wrote: An NFS/RDMA client's source port is meaningless for RDMA transports. The transport layer typically sets the source port value on the connection to a random ephemeral port.

Re: [PATCH V2 RFC 1/3] svcrdma: Transport and header file changes

On Tue, May 06, 2014 at 12:46:27PM -0500, Steve Wise wrote: From: Tom Tucker t...@opengridcomputing.com Change poll logic to grab up to 6 completions at a time. RDMA write and send completions no longer deal with fastreg objects. Set SVCRDMA_DEVCAP_FAST_REG and allocate a dma_mr based on

Re: [PATCH V2 RFC 0/3] svcrdma: refactor marshalling logic

On Tue, May 06, 2014 at 12:46:21PM -0500, Steve Wise wrote: This patch series refactors the NFSRDMA server marshalling logic to remove the intermediary map structures. It also fixes an existing bug where the NFSRDMA server was not minding the device fast register page list length limitations.

Re: [PATCH V2 RFC 1/3] svcrdma: Transport and header file changes

On Tue, May 06, 2014 at 04:02:41PM -0500, Steve Wise wrote: On 5/6/2014 2:21 PM, J. Bruce Fields wrote: On Tue, May 06, 2014 at 12:46:27PM -0500, Steve Wise wrote: From: Tom Tucker t...@opengridcomputing.com Change poll logic to grab up to 6 completions at a time. RDMA write and send

Re: [PATCH] svcrdma: refactor marshalling logic

On Thu, Apr 10, 2014 at 01:37:23PM -0500, Steve Wise wrote: From: Tom Tucker t...@ogc.us This patch refactors the marshalling logic to remove the intermediary map structures. It also fixes an existing bug where the NFSRDMA server was not minding the device fast register page list length

Re: NFS over RDMA benchmark

On Sun, Apr 28, 2013 at 10:42:48AM -0400, J. Bruce Fields wrote: On Sun, Apr 28, 2013 at 06:28:16AM +, Yan Burman wrote: On Wed, Apr 17, 2013 at 7:36 AM, Yan Burman y...@mellanox.com I've been trying to do some benchmarks for NFS over RDMA and I seem

Re: NFS over RDMA benchmark

On Sun, Apr 28, 2013 at 06:28:16AM +, Yan Burman wrote: On Wed, Apr 17, 2013 at 7:36 AM, Yan Burman y...@mellanox.com I've been trying to do some benchmarks for NFS over RDMA and I seem to only get about half of the bandwidth that the HW can give me.

Re: NFS over RDMA benchmark

On Wed, Apr 24, 2013 at 12:35:03PM +, Yan Burman wrote: -Original Message- From: J. Bruce Fields [mailto:bfie...@fieldses.org] Sent: Wednesday, April 24, 2013 00:06 To: Yan Burman Cc: Wendy Cheng; Atchley, Scott; Tom Tucker; linux-rdma@vger.kernel.org; linux

Re: NFS over RDMA benchmark

On Wed, Apr 24, 2013 at 11:05:40AM -0400, J. Bruce Fields wrote: On Wed, Apr 24, 2013 at 12:35:03PM +, Yan Burman wrote: -Original Message- From: J. Bruce Fields [mailto:bfie...@fieldses.org] Sent: Wednesday, April 24, 2013 00:06 To: Yan Burman Cc: Wendy Cheng

Re: NFS over RDMA benchmark

On Thu, Apr 18, 2013 at 12:47:09PM +, Yan Burman wrote: -Original Message- From: Wendy Cheng [mailto:s.wendy.ch...@gmail.com] Sent: Wednesday, April 17, 2013 21:06 To: Atchley, Scott Cc: Yan Burman; J. Bruce Fields; Tom Tucker; linux-rdma@vger.kernel.org; linux

Re: NFS over RDMA crashing

On Mon, Feb 11, 2013 at 03:19:42PM +, Yan Burman wrote: -Original Message- From: J. Bruce Fields [mailto:bfie...@fieldses.org] Sent: Thursday, February 07, 2013 18:42 To: Yan Burman Cc: linux-...@vger.kernel.org; sw...@opengridcomputing.com; linux- r...@vger.kernel.org

Re: NFS over RDMA crashing

On Mon, Feb 11, 2013 at 03:19:42PM +, Yan Burman wrote: -Original Message- From: J. Bruce Fields [mailto:bfie...@fieldses.org] Sent: Thursday, February 07, 2013 18:42 To: Yan Burman Cc: linux-...@vger.kernel.org; sw...@opengridcomputing.com; linux- r...@vger.kernel.org

Re: NFS over RDMA crashing

On Wed, Feb 06, 2013 at 05:24:35PM -0500, J. Bruce Fields wrote: On Wed, Feb 06, 2013 at 05:48:15PM +0200, Yan Burman wrote: When killing mount command that got stuck: --- BUG: unable to handle kernel paging request at 880324dc7ff8 IP

Re: NFS over RDMA crashing

On Wed, Feb 06, 2013 at 05:48:15PM +0200, Yan Burman wrote: When killing mount command that got stuck: --- BUG: unable to handle kernel paging request at 880324dc7ff8 IP: [a05f3dfb] rdma_read_xdr+0x8bb/0xd40 [svcrdma] PGD 1a0c063 PUD

Re: [Q] I tried to mount nfs_server:/mnt volume, but It mounted another volume.

On Sun, Nov 21, 2010 at 03:41:18PM +0200, Boaz Harrosh wrote: On 11/21/2010 02:23 PM, Hiroyuki Sato wrote: Hello lists OS: CentOS 5.5 kernel: 2.6.36 rebuild myself. I'm trying to test NFS/RDMA. I tried to mount nfs_server:/mnt volume on on nfs_client but It mounted

Re: [PATCH 0/2] svcrdma: NFSRDMA Server fixes for 2.6.37

On Tue, Oct 12, 2010 at 03:33:46PM -0500, Tom Tucker wrote: Hi Bruce, These fixes are ready for 2.6.37. They fix two bugs in the server-side NFSRDMA transport. Both applied and pushed out, thanks. --b. Thanks, Tom --- Tom Tucker (2): svcrdma: Cleanup DMA unmapping in error

Re: [PATCH] svcrdma: RDMA support not yet compatible with RPC6

On Mon, Apr 05, 2010 at 12:16:18PM -0400, J. Bruce Fields wrote: On Mon, Apr 05, 2010 at 10:50:16AM -0500, Tom Tucker wrote: J. Bruce Fields wrote: On Mon, Apr 05, 2010 at 10:55:12AM -0400, Chuck Lever wrote: On 04/03/2010 09:27 AM, Tom Tucker wrote: RPC6 requires