Re: [PATCH net-next 3/4] libcxgbi: Add ipv6 api to driver
From: Anish Bhatt Date: Tue, 15 Jul 2014 19:55:52 -0700 > +#define pr_info_ipaddr(fmt_trail,\ > + addr1, addr2, args_trail...)\ Barf... use "%pIS" instead, which takes a pointer to a sockaddr of family AF_INET or AF_INET6. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v3 00/21] NFS/RDMA client patches for 3.17
Hi Chuck, Tested v3 with ocrdma (linux-3.16-rc5 inbox`ed ocrdma). Both Cthon and iozone passes with and regressions. I will perform cable pull test as well and get back to you. -Regards Devesh > -Original Message- > From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > ow...@vger.kernel.org] On Behalf Of Chuck Lever > Sent: Tuesday, July 15, 2014 7:54 PM > To: linux-rdma; Linux NFS Mailing List > Subject: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 > > The main purpose of this series is to address connection drop recovery issues > by fixing FRMR re-use to make it less likely the client will deadlock due to a > memory management operation error. > > Some clean-ups and other fixes are present as well. > > See topic branch nfs-rdma-for-3.17 in > > git://git.linux-nfs.org/projects/cel/cel-2.6.git > > I tested with NFSv3 and NFSv4 on all three supported memory registration > modes. Used cthon04, iozone, and dbench with both Solaris and Linux > NFS/RDMA servers. Used xfstests with Linux. > > v3: > Only two substantive changes: > > - Patch 08/21 now uses generic IB helpers for managing FRMR > rkeys > > - Add Tested-by: from Steve Wise > > > v2: > Many patches from v1 have been written or replaced. > > The MW ref counting approach in v1 is abandoned. Instead, I've eliminated > signaling FAST_REG_MR and LOCAL_INV, and added appropriate recovery > mechanisms after a transport reconnect that should prevent rkey dis- > synchrony entirely. > > A couple of optimizations have been added, including: > > - Allocating each MW separately rather than carving each out of a > large piece of contiguous memory > > - Now that the receive CQ upcall handler dequeues a bundle of CQEs > at once, fire off the reply handler tasklet just once per upcall > to reduce context switches and how often hard IRQs are disabled > > Jury is still out on the latter. > > -- > Chuck Lever > chuck[dot]lever[at]oracle[dot]com > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the > body of a message to majord...@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 00/21] NFS/RDMA client patches for 3.17
On 07/16/2014 10:57 AM, Chuck Lever wrote: > >> On Jul 16, 2014, at 11:48 AM, Shirley Ma wrote: >> >> These two patches have been significantly reduced interrupt rate by around 4 >> times. >> >> xprtrdma: Disable completions for FAST_REG_MR Work Requests >> xprtrdma: Disable completions for LOCAL_INV Work Requests > > Thanks Shirley! This is result applies only to FRMR, correct? Also, i'd > imagine the savings would be even greater for adapters that have short page > list depth? Yes, only tested FRMR with mlx4. I can hack the code to test short page page list depth to check the savings. When looking the difference between irq and softirq, it is much closer now. > >> >> Same NFS read/write workload, here are interrupts rate irq/per sec report >> based upon /proc/interrupts: >> >> w/o patches: >> --- >> PCI-MSI-edge mlx4-ib (204): 105176 >> PCI-MSI-edge mlx4-ib (204): 123650 >> PCI-MSI-edge mlx4-ib (204): 123690 >> PCI-MSI-edge mlx4-ib (204): 116554 >> PCI-MSI-edge mlx4-ib (204): 122864 >> >> And perf stat irq report: >> Performance counter stats for 'system wide': >> >> 2,131,870 irq:irq_handler_entry >>[100.00%] >> 2,131,870 irq:irq_handler_exit >>[100.00%] >> 635,587 irq:softirq_entry >>[100.00%] >> 635,597 irq:softirq_exit >>[100.00%] >> 636,155 irq:softirq_raise >> >> 25.422821792 seconds time elapsed >> >> w/i patches: >> --- >> PCI-MSI-edge mlx4-ib (204): 31131 >> PCI-MSI-edge mlx4-ib (204): 32958 >> PCI-MSI-edge mlx4-ib (204): 31068 >> PCI-MSI-edge mlx4-ib (204): 30236 >> PCI-MSI-edge mlx4-ib (204): 33041 >> >> And perf stat irq report: >> >> Performance counter stats for 'system wide': >> >> 653,548 irq:irq_handler_entry >>[100.00%] >> 653,548 irq:irq_handler_exit >>[100.00%] >> 568,138 irq:softirq_entry >>[100.00%] >> 568,148 irq:softirq_exit >>[100.00%] >> 568,690 irq:softirq_raise >> >> >> 21.675597062 seconds time elapsed >> >> Shirley >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 00/21] NFS/RDMA client patches for 3.17
> On Jul 16, 2014, at 11:48 AM, Shirley Ma wrote: > > These two patches have been significant reduced interrupt rate by around 4 > times. > > xprtrdma: Disable completions for FAST_REG_MR Work Requests > xprtrdma: Disable completions for LOCAL_INV Work Requests Thanks Shirley! This is result applies only to FRMR, correct? Also, i'd imagine the savings would be even greater for adapters that have short page list depth? > > Same NFS read/write workload, here are interrupts rate irq/per sec report > based upon /proc/interrupts: > > w/o patches: > --- > PCI-MSI-edge mlx4-ib (204): 105176 > PCI-MSI-edge mlx4-ib (204): 123650 > PCI-MSI-edge mlx4-ib (204): 123690 > PCI-MSI-edge mlx4-ib (204): 116554 > PCI-MSI-edge mlx4-ib (204): 122864 > > And perf stat irq report: > Performance counter stats for 'system wide': > > 2,131,870 irq:irq_handler_entry > [100.00%] > 2,131,870 irq:irq_handler_exit > [100.00%] > 635,587 irq:softirq_entry > [100.00%] > 635,597 irq:softirq_exit > [100.00%] > 636,155 irq:softirq_raise > > 25.422821792 seconds time elapsed > > w/i patches: > --- > PCI-MSI-edge mlx4-ib (204): 31131 > PCI-MSI-edge mlx4-ib (204): 32958 > PCI-MSI-edge mlx4-ib (204): 31068 > PCI-MSI-edge mlx4-ib (204): 30236 > PCI-MSI-edge mlx4-ib (204): 33041 > > And perf stat irq report: > > Performance counter stats for 'system wide': > > 653,548 irq:irq_handler_entry > [100.00%] > 653,548 irq:irq_handler_exit > [100.00%] > 568,138 irq:softirq_entry > [100.00%] > 568,148 irq:softirq_exit > [100.00%] > 568,690 irq:softirq_raise > > > 21.675597062 seconds time elapsed > > Shirley > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 00/21] NFS/RDMA client patches for 3.17
These two patches have been significant reduced interrupt rate by around 4 times. xprtrdma: Disable completions for FAST_REG_MR Work Requests xprtrdma: Disable completions for LOCAL_INV Work Requests Same NFS read/write workload, here are interrupts rate irq/per sec report based upon /proc/interrupts: w/o patches: --- PCI-MSI-edge mlx4-ib (204): 105176 PCI-MSI-edge mlx4-ib (204): 123650 PCI-MSI-edge mlx4-ib (204): 123690 PCI-MSI-edge mlx4-ib (204): 116554 PCI-MSI-edge mlx4-ib (204): 122864 And perf stat irq report: Performance counter stats for 'system wide': 2,131,870 irq:irq_handler_entry [100.00%] 2,131,870 irq:irq_handler_exit [100.00%] 635,587 irq:softirq_entry [100.00%] 635,597 irq:softirq_exit [100.00%] 636,155 irq:softirq_raise 25.422821792 seconds time elapsed w/i patches: --- PCI-MSI-edge mlx4-ib (204): 31131 PCI-MSI-edge mlx4-ib (204): 32958 PCI-MSI-edge mlx4-ib (204): 31068 PCI-MSI-edge mlx4-ib (204): 30236 PCI-MSI-edge mlx4-ib (204): 33041 And perf stat irq report: Performance counter stats for 'system wide': 653,548 irq:irq_handler_entry [100.00%] 653,548 irq:irq_handler_exit [100.00%] 568,138 irq:softirq_entry [100.00%] 568,148 irq:softirq_exit [100.00%] 568,690 irq:softirq_raise 21.675597062 seconds time elapsed Shirley -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH net-next 0/4] cxgb4i: Add ipv6 support
On Wed, Jul 16, 2014 at 08:32:59AM +, Anish Bhatt wrote: > We don't have any specific changes queued for 3.17, but the next update > requires a similar change to cxgb4. I'm okay submitting that to net-next if it > is okay with you. That sounds fine to me. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH net-next 0/4] cxgb4i: Add ipv6 support
We don't have any specific changes queued for 3.17, but the next update requires a similar change to cxgb4. I'm okay submitting that to net-next if it is okay with you. Plus one of the patches requires cxgb4 dcb code that is not yet in the scsi tree. -Anish From: Christoph Hellwig [h...@infradead.org] Sent: Wednesday, July 16, 2014 1:08 AM To: Anish Bhatt Cc: net...@vger.kernel.org; linux-rdma@vger.kernel.org; linux-s...@vger.kernel.org; da...@davemloft.net; rol...@purestorage.com; h...@infradead.org; micha...@cs.wisc.edu; jbottom...@parallels.com; Karen Xie; Casey Leedom; Hariprasad S; SWise OGC Subject: Re: [PATCH net-next 0/4] cxgb4i: Add ipv6 support Hi Anish, the scsi side changes looks fine to me. Unless you have major updates to the scsi driver for 3.17 that could cause conflicts I'm fine with merging the whole series in through the networking tree. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH net-next 0/4] cxgb4i: Add ipv6 support
Hi Anish, the scsi side changes looks fine to me. Unless you have major updates to the scsi driver for 3.17 that could cause conflicts I'm fine with merging the whole series in through the networking tree. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
