[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-07-21 Thread Zahid Bukhari
This bug is also affecting us. We're on Xenial but use this kernel for HWE. Basically for our monitoring, the leader to all this shows as dirty memory and or filesystem writeback. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-07-08 Thread Leon van Heerden
I have also been running 4.15.0-144-generic for three weeks so far without the issue occurring. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1928744 Title: Disk IO very slow on kernel

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-07-05 Thread Mariusz
John - thanks for finding ==> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1926081 There is a great testing work done there. For this thread I have been running kernel 4.15.0-144-generic for the last 3 weeks and haven't seen the issue. -- You received this bug notification because you

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-07-05 Thread John McDonnell
I note it is possible this is a duplicate of https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1926081 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1928744 Title: Disk IO very slow on kernel

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-06-29 Thread Paul Krizak
We were directed to this bug by our Portworx support team, and believe we're seeing the same thing. We are running Ubuntu 16.04 with the hardware enablement kernel. Most of our nodes are running 4.15.0-140, and like Peter notes above, the I/O performance problem does not show up right away, but

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-06-23 Thread Peter Jönsson
Hi! We are running some large (100+) kubernetes clusters on bare metal machines also running a LSI MegaRAID controller and we are experiencing these exact problems since about a week (cluster being in production since early june). We run with this controller: # lspci | grep -i mega 59:00.0 RAID

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-06-21 Thread Mariusz
Hey, We experienced the performance issue on different hardware servers: LENOVO System x3650 M5 Lenovo ThinkSystem SR650 with the following kernels: 4.15.0-137-generic: disk performance issue 4.15.0-139-generic: disk performance issue 4.15.0-140-generic: disk performance issue

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-06-18 Thread Leon van Heerden
Hi, We think we are affected by the same bug. We were on version 4.15.0-139 and upgraded to 144 before finding this post. Is there any confirmation that this issue also occurs in 4.15.0-144? I have have installed version 4.15.0-136 in the meantime to allow us to reboot and select that kernel,

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-06-18 Thread Launchpad Bug Tracker
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: linux (Ubuntu Bionic) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1928744

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-06-08 Thread Mariusz
Hi Khaled, I have finished testing the kernels - it took more time than I expected but the results are: 4.15.0-136-generic: no issues 4.15.0-137-generic: disk performance issue 4.15.0-139-generic: disk performance issue 4.15.0-140-generic: disk performance issue 4.15.0-142-generic: disk

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-06-08 Thread Khaled El Mously
Thanks for the extensive testing, Mariusz. (I assume -137 doesn't have the performance problem?) So, for comment #9 - the difference in loaded modules comes down to 20 additional modules that were loaded when performance was bad: -ip6_tables 28672 0 -iptable_filter 16384 0 -ip_tables 28672 1

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-30 Thread Mariusz
Hi, I have done longer testing of kernel 140 and it has performance degradation issue as well. It took a couple of days to fully max out disk IOs on that kernel with the same load. Therefore, I came back to testing the other kernels in longer time-window. At the moment, I'm running kernel

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-23 Thread Mariusz
Hi, In the last 3 days, I have been testing the other kernels. The steps which I took: 1. Boot to kernel-4.15.0-136. 2. Install kernel X, where X = 136,137,139,140: sudo apt-get install linux-image-4.15.0-X-generic linux-headers-4.15.0-X-generic linux-modules-4.15.0-X-generic

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-20 Thread Mariusz
Cool cool, I will try to do what you have suggested - that make sense. Will get back once I have some details. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1928744 Title: Disk IO very slow on

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-20 Thread Khaled El Mously
Hi @Mariusz - thank you for the bug report. That certainly sounds like a kernel bug based on your description. Doing a kernel bisect is probably the fastest approach to identifying the offending change. There are 392 changes between -136 and -142 so it could take up to 9 tests for a full bisect.

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-20 Thread Juerg Haefliger
** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1928744 Title: Disk IO very slow on kernel 4.15.0-142-generic To

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-19 Thread Mariusz
Hi, I have done another test and compared the same server running on 4.15.0-136-generic and kernel 4.15.0-142-generic. Collected zabbix CPU utilization graph which is showing the problem from the performance point of view. On 4.15.0-136-generic, the server works normal and there are no processes

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-18 Thread Mariusz
Unable to run apport-collect 1928744 command. ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1928744 Title: Disk IO very slow on

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-18 Thread Chris Guiver
changed HWE to linux as I see no reference to a HWE (5.4) kernel ** Package changed: linux-signed-hwe (Ubuntu) => linux (Ubuntu) ** Tags added: bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-18 Thread Mariusz
Thanks a lot for coming back to me. I couldn't run apport-collect as we don't have python-launchpadlib installed. I did check (apt install python3-launchpadlib) and in my case it would install quite a lot of dependent packages - we would like to avoid that due to the nature of our server. But I

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-18 Thread Chris Guiver
Thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command only once, as it will automatically gather debugging information, in a terminal: apport-collect 1928744 When reporting bugs in the future please use apport by using 'ubuntu-

[Bug 1928744] Re: Disk IO very slow on kernel 4.15.0-142-generic

2021-05-18 Thread Mariusz
** Package changed: ubuntu => linux-signed-hwe (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1928744 Title: Disk IO very slow on kernel 4.15.0-142-generic To manage notifications about