This bug is awaiting verification that the linux/5.19.0-44.45 kernel in
-proposed solves the problem. Please test the kernel and update this bug
with the results. If the problem is solved, change the tag
'verification-needed-kinetic' to 'verification-done-kinetic'. If the
problem still exists, change the tag 'verification-needed-kinetic' to
'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will
be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how
to enable and use -proposed. Thank you!


** Tags added: kernel-spammed-kinetic-linux verification-needed-kinetic

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2016186

Title:
  5.19 not reporting cgroups v1 blkio.throttle.io_serviced

Status in linux package in Ubuntu:
  Incomplete
Status in linux source package in Kinetic:
  Fix Committed
Status in linux source package in Lunar:
  Fix Committed
Status in linux source package in Mantic:
  Incomplete

Bug description:
  [Impact]

  Commit f382fb0bcef4 ("block: remove legacy IO schedulers") introduced
  a behavior change in the blkio throttle cgroup subsystem: IO
  statistics are not reported anymore unless a throttling rule is
  explicitly defined, because the current code only counts bios that are
  actually throttled.

  This behavior change is potentially breaking some user-space 
  applications that are relying on the old behavior (see original bug 
  report below).

  [Test case]

   - mount cgroup v1
   - create a blkio cgroup
   - move a task into the blkio cgroup
   - perform some I/O (i.e., dd)
   - read the IO stats for the cgroup (blkio.throttle.io_serviced and 
blkio.throttle.io_service_bytes in cgroupfs)
   - IO stats are all 0, unless a throttle rule is defined

  Previous behavior (kernel 5.15) was showing I/O statistics even
  without throttling rules defined.

  [Fix]

  Apply / backport this fix:

  
https://lore.kernel.org/lkml/20230507170631.89607-1-hanjinke....@bytedance.com/t/

  [Regression potential]

  The fix is affecting the block IO cgroup subsystem, we may see
  potential regressions in this particular cgroup subsystem with this
  fix applied.

  [Original bug report]

  Hi,

  I'm still investigating but, am a bit stuck. Here's what I've found so
  far.

  Today I've upgraded some nodes in AWS EC2 from the previous v5.15
  linux-aws package to the recently pusblished v5.19 package and
  rebooted. It seems that even when there's disk activity, the files:

  /sys/fs/cgroup/blkio/blkio.throttle.io_serviced
  /sys/fs/cgroup/blkio/blkio.throttle.io_service_bytes

  Are only ever populated with 0's. Prior on v5.15 these would reflect
  the actual disk usage. No other system configuration changes were
  applied just the kernel upgrade and reboot. I've also verified that
  simply rebooting a v5.15 where this does work doesn't break the
  reporting. These EC2 instances are running with cgroups v1 due to
  other compatability issues and I suspect that might be the issue. So
  far, I cannot find any differences. mtab shows the same v1 mount
  setup, the kernel options match betwen v5.15 and v5.19.

  I'm more than happy to fetch whatever info would help out here. I'd
  love to get 5.19 working for us but, we really need the data from
  these files.

  Info:
  Prior version that works: Linux ip-10-128-168-154 5.15.0-1031-aws #35-Ubuntu 
SMP Fri Feb 10 02:07:18 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
  Upgraded version that's broken: Linux ip-10-128-166-219 5.19.0-1022-aws 
#23~22.04.1-Ubuntu SMP Fri Mar 17 15:38:24 UTC 2023 x86_64 x86_64 x86_64 
GNU/Linux

  EC2 instances built off of the published 22.04 LTS AMI in us-east-1.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2016186/+subscriptions


-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to