Re: Performance regressions in networking & storage benchmarks in Linux kernel 5.8
Hello Thomas, Thank you very much for your comments. Since the performance regressions were fixed when we tested version 5.9-rc4, we were not reporting it as an issue and our intention was just to share this as an information only. Thanks, Abdul Anshad A From: Thomas Gleixner Sent: Tuesday, September 22, 2020 04:55 PM To: Abdul Anshad Azeez ; linux-kernel@vger.kernel.org ; x...@kernel.org ; net...@vger.kernel.org ; linux-fsde...@vger.kernel.org Cc: rost...@goodmis.org Subject: Re: Performance regressions in networking & storage benchmarks in Linux kernel 5.8 Abdul, On Tue, Sep 22 2020 at 08:51, Abdul Anshad Azeez wrote: > Part of VMware's performance regression testing for Linux Kernel upstream rele > ases we compared Linux kernel 5.8 against 5.7. Our evaluation revealed perform > ance regressions mostly in networking latency/response-time benchmarks up to 6 > 0%. Storage throughput & latency benchmarks were also up by 8%. > In order to find the fix commit, we bisected again between 5.8 and 5.9-rc4 and > identified that regressions were fixed from a commit made by the same author > Thomas Gleixner, which unbreaks the interrupt affinity settings - "e027f79 > 9cdd70400c5485b1a54f482255985(x86/irq: Unbreak interrupt affinity setting)". > > We believe these findings would be useful to the Linux community and wanted to > document the same. thanks for letting us know, but the issue is known already and the fix has been backported to the stable kernel version 5.8.6 as of Sept. 3rd. Please always check the latest stable version. Thanks, tglx
Re: Performance regressions in networking & storage benchmarks in Linux kernel 5.8
Abdul, On Tue, Sep 22 2020 at 08:51, Abdul Anshad Azeez wrote: > Part of VMware's performance regression testing for Linux Kernel upstream rele > ases we compared Linux kernel 5.8 against 5.7. Our evaluation revealed perform > ance regressions mostly in networking latency/response-time benchmarks up to 6 > 0%. Storage throughput & latency benchmarks were also up by 8%. > In order to find the fix commit, we bisected again between 5.8 and 5.9-rc4 and > identified that regressions were fixed from a commit made by the same author > Thomas Gleixner, which unbreaks the interrupt affinity settings - "e027f79 > 9cdd70400c5485b1a54f482255985(x86/irq: Unbreak interrupt affinity setting)". > > We believe these findings would be useful to the Linux community and wanted to > document the same. thanks for letting us know, but the issue is known already and the fix has been backported to the stable kernel version 5.8.6 as of Sept. 3rd. Please always check the latest stable version. Thanks, tglx
Performance regressions in networking & storage benchmarks in Linux kernel 5.8
Part of VMware's performance regression testing for Linux Kernel upstream rele ases we compared Linux kernel 5.8 against 5.7. Our evaluation revealed perform ance regressions mostly in networking latency/response-time benchmarks up to 6 0%. Storage throughput & latency benchmarks were also up by 8%. After performing the bisect between kernel 5.8 and 5.7, we identified the root cause behaviour to be an interrupt related change from Thomas Gleixner's "633 260fa143bbed05e65dc557a492667dfdc45bb(x86/irq: Convey vector as argument and n ot in ptregs)" commit. To confirm this, we backed out the commit from 5.8 & re ran our tests and found that the performance was similar to 5.7 kernel. Impacted test cases: Networking: - Netperf TCP_RR & TCP_CRR - Response time - Ping - Response time - Memcache - Response time - Netperf TCP_STREAM small(8K socket & 256B message)(TCP_NODELAY set) pack ets - Throughput & CPU utilization(CPU/Gbits) Storage: - FIO: - 4K (rand|seq)_(read|write) local-NVMe MultiVM tests - Throughput & l atency >From our testing, overall results indicate that above-mentioned commit has int roduced performance regressions in latency-sensitive workloads for networking. For storage, it affected both throughput & latency workloads. Also, since Linux 5.9-rc4 kernel was released recently, we repeated the same e xperiments on 5.9-rc4. We observed all regressions were fixed and the performa nce numbers between 5.7 and 5.9-rc4 were similar. In order to find the fix commit, we bisected again between 5.8 and 5.9-rc4 and identified that regressions were fixed from a commit made by the same author Thomas Gleixner, which unbreaks the interrupt affinity settings - "e027f79 9cdd70400c5485b1a54f482255985(x86/irq: Unbreak interrupt affinity setting)". We believe these findings would be useful to the Linux community and wanted to document the same. Abdul Anshad Azeez Performance Engineering VMware, Inc.
Re: Benchmarks for Linux kernel
On Fri, Jun 01, 2001 at 09:12:14PM -0700, Jaswinder Singh wrote: > Can please point me some nice benchmarks for linux kernel . Linux Benchmark Suite http://lbs.sf.net/ -- Nate Straz [EMAIL PROTECTED] sgi, inc http://www.sgi.com/ Linux Test Project http://ltp.sf.net/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Benchmarks for Linux kernel
On Fri, Jun 01, 2001 at 09:12:14PM -0700, Jaswinder Singh wrote: Can please point me some nice benchmarks for linux kernel . Linux Benchmark Suite http://lbs.sf.net/ -- Nate Straz [EMAIL PROTECTED] sgi, inc http://www.sgi.com/ Linux Test Project http://ltp.sf.net/ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Benchmarks for Linux kernel
Can please point me some nice benchmarks for linux kernel . Which tells the performance of following , under Linux Kernel :- 1. CPU 2. Bus 3. Cache 4. DMA 5. Interrupts and Exceptions 6. File Systems 7. FPU 8. forking and pthread (Process Management) 9. IDE 10. Ethernet 11. Memory Management 12. PCI 13. USB 14. Serial 15. Clocks and Timers 16. Sound 17. SMP 18. Virtual Memory Management 19. Networking 20. PCMCIA 21. RAID Thank you, Jaswinder. -- These are my opinions not 3Di. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Benchmarks for Linux kernel
Can please point me some nice benchmarks for linux kernel . Which tells the performance of following , under Linux Kernel :- 1. CPU 2. Bus 3. Cache 4. DMA 5. Interrupts and Exceptions 6. File Systems 7. FPU 8. forking and pthread (Process Management) 9. IDE 10. Ethernet 11. Memory Management 12. PCI 13. USB 14. Serial 15. Clocks and Timers 16. Sound 17. SMP 18. Virtual Memory Management 19. Networking 20. PCMCIA 21. RAID Thank you, Jaswinder. -- These are my opinions not 3Di. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
