I collected some stats for interrupts in 3 scenarios. We do get more timer interrupts even if the VM is not running with just the kvm module loaded. I collected the hrtimer_expire_entry tracing which aligns with the interrupt data and your earlier findings from gdb. None of the timer events related to KVM when VM is not running. So we can rule out vcpu/kvm timer issues.
Periodic interrupt stats
------------------------------
Case 1: inner VM is running:
root@debian:/mnt/host/timer_trace_logs# ./interrupt_stat_watch.sh
===== Δ /proc/interrupts @ 03:31:47 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +6395 | RISC-V INTC 5 Edge riscv-timer
IRQ 14 Δ = +22 | SiFive PLIC 35 Edge virtio3
IRQ 16 Δ = +5 | SiFive PLIC 8 Edge virtio0
===== Δ /proc/interrupts @ 03:31:52 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +7484 | RISC-V INTC 5 Edge riscv-timer
IRQ 14 Δ = +6 | SiFive PLIC 35 Edge virtio3
IRQ 16 Δ = +6 | SiFive PLIC 8 Edge virtio0
===== Δ /proc/interrupts @ 03:31:59 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +5992 | RISC-V INTC 5 Edge riscv-timer
IRQ 14 Δ = +14 | SiFive PLIC 35 Edge virtio3
IRQ 16 Δ = +6 | SiFive PLIC 8 Edge virtio0
Case 2: inner VM shutdown with virtualization on
root@debian:/mnt/host/timer_trace_logs# ./interrupt_stat_watch.sh
===== Δ /proc/interrupts @ 03:31:01 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +4825 | RISC-V INTC 5 Edge riscv-timer
IRQ 14 Δ = +9 | SiFive PLIC 35 Edge virtio3
IRQ 16 Δ = +5 | SiFive PLIC 8 Edge virtio0
===== Δ /proc/interrupts @ 03:31:06 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +5454 | RISC-V INTC 5 Edge riscv-timer
IRQ 14 Δ = +30 | SiFive PLIC 35 Edge virtio3
IRQ 16 Δ = +6 | SiFive PLIC 8 Edge virtio0
===== Δ /proc/interrupts @ 03:31:12 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +4823 | RISC-V INTC 5 Edge riscv-timer
IRQ 14 Δ = +137 | SiFive PLIC 35 Edge virtio3
IRQ 16 Δ = +6 | SiFive PLIC 8 Edge virtio0
Case 3: inner VM shutdown with virtualization off (kvm module is removed)
root@debian:/mnt/host/timer_trace_logs# ./interrupt_stat_watch.sh
===== Δ /proc/interrupts @ 03:32:18 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +1578 | RISC-V INTC 5 Edge riscv-timer
IRQ 14 Δ = +8 | SiFive PLIC 35 Edge virtio3
IRQ 16 Δ = +6 | SiFive PLIC 8 Edge virtio0
===== Δ /proc/interrupts @ 03:32:19 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +1525 | RISC-V INTC 5 Edge riscv-timer
IRQ 16 Δ = +6 | SiFive PLIC 8 Edge virtio0
===== Δ /proc/interrupts @ 03:32:21 =====
IRQ 12 Δ = +36 | SiFive PLIC 33 Edge virtio1
IRQ 10 Δ = +1524 | RISC-V INTC 5 Edge riscv-timer
IRQ 14 Δ = +12 | SiFive PLIC 35 Edge virtio3
IRQ 16 Δ = +5 | SiFive PLIC 8 Edge virtio0
=========================================
hrtimer_expire_entry trace enable
=========================================
Case 1: inner VM is running:
./group_trace_by_process_and_function.sh timer_trace_with_vm_vm_running | head
-n 10
2980 <idle> tick_nohz_handler
890 mosh tick_nohz_handler
690 systemd tick_nohz_handler
566 <idle> dl_task_timer
300 rcu_sched inactive_task_timer
167 <idle> inactive_task_timer
126 sshd tick_nohz_handler
100 kworker/u75:2 tick_nohz_handler
80 sshd inactive_task_timer
70 dbus tick_nohz_handler
Case 2: inner VM shutdown with virtualization on
./group_trace_by_process_and_function.sh timer_trace_with_vm_virt_on | head -n
10
3086 <idle> tick_nohz_handler
1332 mosh tick_nohz_handler
792 systemd tick_nohz_handler
601 <idle> dl_task_timer
369 rcu_sched inactive_task_timer
148 <idle> inactive_task_timer
142 sshd tick_nohz_handler
130 utempter tick_nohz_handler
110 kworker/u71:2 tick_nohz_handler
78 sshd inactive_task_timer
Case 3: inner VM shutdown with virtualization off (kvm module is removed)
./group_trace_by_process_and_function.sh timer_trace_with_vm_virt_off | head -n
10
1632 <idle> tick_nohz_handler
382 <idle> dl_task_timer
272 systemd tick_nohz_handler
250 mosh tick_nohz_handler
146 rcu_sched inactive_task_timer
136 <idle> inactive_task_timer
58 sshd inactive_task_timer
54 systemd inactive_task_timer
44 kworker/u80:1 tick_nohz_handler
38 systemd dl_task_timer
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2112578
Title:
riscv64: KVM does not release harts
Status in linux package in Ubuntu:
New
Status in qemu package in Ubuntu:
Invalid
Bug description:
In the 6.14.0-13-generic kernel there seems to be a problem with KVM.
On my amd64 machine I am running a riscv64 machine using QEMU 10 with
16 cores. Inside the VM I spin up a further machine with 2 cores. Once
KVM emulation is started the load of my amd64 machine rises to > 14/16
cores for qemu-system-riscv64. KVM seems not to release cores that
were ever used. Top in 1st VM shows 0.25 cores used for 2nd VM.
The usage on my amd64 machine only drops when unloading the kvm module in the
first VM.
---
ProblemType: Bug
ApportVersion: 2.32.0-0ubuntu5
Architecture: riscv64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: zfsdt 2105 F.... wireplumber
/dev/snd/seq: zfsdt 2103 F.... pipewire
CasperMD5json:
{
"result": "skip"
}
CurrentDesktop: XFCE
DistroRelease: Ubuntu 25.10
InstallationDate: Installed on 2025-03-17 (80 days ago)
InstallationMedia: Ubuntu-Server 25.04 "Plucky Puffin" - Daily riscv64
(20250316)
Lsusb:
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU Tablet
Bus 001 Device 003: ID 0627:0001 Adomax Technology Co., Ltd QEMU Tablet
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
MachineType: QEMU QEMU Virtual Machine
Package: linux (not installed)
ProcEnviron:
LANG=en_US.UTF-8
PATH=(custom, no user)
SHELL=/bin/bash
TERM=xterm-256color
ProcFB: 0 virtio_gpudrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-6.14.0-13-generic
root=UUID=af336ba2-ea81-4dd5-bada-b3e8e7fb1c58 ro
crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M
ProcVersionSignature: Ubuntu 6.14.0-13.13.2-generic 6.14.0
PulseList: Error: command ['pacmd', 'list'] failed with exit code 1: No
PulseAudio daemon running, or not running as session daemon.
RelatedPackageVersions:
linux-restricted-modules-6.14.0-13-generic N/A
linux-backports-modules-6.14.0-13-generic N/A
linux-firmware 20250317.git1d4c88ee-0ubuntu1
RfKill:
Tags: questing
Uname: Linux 6.14.0-13-generic riscv64
UpgradeStatus: Upgraded to questing on 2025-05-17 (19 days ago)
UserGroups: N/A
_MarkForUpload: True
acpidump:
dmi.bios.date: 05/13/2025
dmi.bios.release: 0.0
dmi.bios.vendor: Ubuntu distribution of EDK II
dmi.bios.version: 2025.02-8
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: virt
dmi.modalias:
dmi:bvnUbuntudistributionofEDKII:bvr2025.02-8:bd05/13/2025:br0.0:svnQEMU:pnQEMUVirtualMachine:pvrvirt:cvnQEMU:ct1:cvrvirt:sku:
dmi.product.name: QEMU Virtual Machine
dmi.product.version: virt
dmi.sys.vendor: QEMU
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2112578/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : [email protected]
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
