On Thu, May 9, 2019 at 5:54 AM Justin Piszcz <jpis...@lucidpixels.com> wrote: > > Hello, > > Kernel: 5.1 (self-compiled, no modules) > Arch: x86_64 > Distro: Debian Testing > > Issue: I was performing a dump of ext3 and ext4 filesystems and then > restoring them to a separate volume (testing)-- afterwards I noticed that > khugepaged is stuck at 100% CPU. It is currently still stuck at 100% CPU, is > there anything I can do to debug what is happening prior to a reboot to work > around the issue? I have never seen this behavior prior to 5.1. > > $ cat /proc/cmdline > auto BOOT_IMAGE=5.1.0-2 ro root=901 3w-sas.use_msi=1 nohugeiomap > page_poison=1 pcie_aspm=off pti=on rootfstype=ext4 slub_debug=P > zram.enabled=1 zram.num_devices=12 zswap.enabled=1 zswap.compressor=lz4 > zswap.zpool=z3fold > > $ 5.1 .config attached: config.txt.gz > > $ attachment: graphic.JPG -> graph of the processes, dark green -> > khugepaged > > $ top > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 77 root 39 19 0 0 0 R 100.0 0.0 23:29.27 > khugepaged > 1 root 20 0 171800 7832 4948 S 0.0 0.0 1:25.59 systemd > 2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 > kthreadd > 3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gp > 4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 > rcu_par_gp > 6 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 > kworker/0+ > 8 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 > mm_percpu+ >
Will disable zcache/zswap for now, but FWIW >From perf top: PerfTop: 1181 irqs/sec kernel:83.1% exact: 96.9% lost: 73/137 drop: 0/0 [4000Hz cycles], (all, 12 CPUs) ------------------------------------------------------------------------------- 54.56% [kernel] [k] compaction_alloc 29.46% [kernel] [k] __pageblock_pfn_to_page 10.66% [kernel] [k] nmi 0.61% [kernel] [k] _cond_resched 0.32% [kernel] [k] format_decode 0.25% [kernel] [k] __lock_text_start 0.20% [kernel] [k] __vma_adjust 0.16% perf [.] __symbols__insert 0.13% perf [.] rb_insert_color_cached I tried dropping caches, etc, I will disable huge pages for now. Some other posts: https://bugzilla.redhat.com/show_bug.cgi?id=879801 https://lkml.org/lkml/2012/6/27/565 https://lkml.org/lkml/2011/11/9/252