Re: KVM: x86: fix pvclock guest stopped flag reporting (v2)
On 2012-08-03 14:57, Marcelo Tosatti wrote: kvm_guest_time_update unconditionally clears hv_clock.flags field, so the notification never reaches the guest. Fix it by allowing PVCLOCK_GUEST_STOPPED to passthrough. Signed-off-by: Marcelo Tosatti Reviewed-by: Eric B Munson -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at arch/x86/kernel/kvmclock.c:127
On Mon, 11 Jun 2012 19:07:19 -0300, Marcelo Tosatti wrote:
On Mon, Jun 11, 2012 at 05:47:00PM -0400, Eric B Munson wrote:
On Sun, 10 Jun 2012 17:47:24 +0200, Sedat Dilek wrote:
>Hi,
>
>I see the same warning especially when resuming from a suspend (see
>timestamps >=30-35sec) between Linux v3.5-rc1..v3.5-rc2.
>
>$ sudo grep kvmclock.c /var/log/kern.log
>Jun 4 21:11:32 fambox kernel: [ 784.037237] WARNING: at
>arch/x86/kernel/kvmclock.c:127
>kvm_check_and_clear_guest_paused+0x52/0x60()
>Jun 5 20:35:57 fambox kernel: [ 1928.458060] WARNING: at
>arch/x86/kernel/kvmclock.c:127
>kvm_check_and_clear_guest_paused+0x52/0x60()
>Jun 8 09:35:52 fambox kernel: [ 3290.134637] WARNING: at
>arch/x86/kernel/kvmclock.c:127
>kvm_check_and_clear_guest_paused+0x52/0x60()
>Jun 8 09:35:52 fambox kernel: [ 3290.238592] WARNING: at
>arch/x86/kernel/kvmclock.c:127
>kvm_check_and_clear_guest_paused+0x52/0x60()
>Jun 8 12:11:20 fambox kernel: [ 5777.023571] WARNING: at
>arch/x86/kernel/kvmclock.c:127
>kvm_check_and_clear_guest_paused+0x52/0x60()
>Jun 9 13:32:50 fambox kernel: [ 2778.842695] WARNING: at
>arch/x86/kernel/kvmclock.c:127
>kvm_check_and_clear_guest_paused+0x52/0x60()
>
>From [1]:
>
>"...The warning itself is not required for the check_and_clear
>function and can be removed as far as I am concerned."
>
>From [2] commit 3b5d56b9317fa7b5407dff1aa7b115bf6cdbd494
("kvmclock:
>Add functions to check if the host has stopped the vm")
>...
>+bool kvm_check_and_clear_guest_paused(void)
>+{
>+ bool ret = false;
>+ struct pvclock_vcpu_time_info *src;
>+
>+ /*
>+* per_cpu() is safe here because this function is only
>called from
>+* timer functions where preemption is already disabled.
>+*/
>+ WARN_ON(!in_atomic());
>+ src = &__get_cpu_var(hv_clock);
>+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
>+ __this_cpu_and(hv_clock.flags,
>~PVCLOCK_GUEST_STOPPED);
>+ ret = true;
>+ }
>+
>+ return ret;
>+}
>+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
>+
>...
>( The export macro was dropped in a followup commit. )
>
>So you mean "WARN_ON(!in_atomic());" can be deleted?
Yes.
When I wrote the patch originally, I was under the (incorrect)
assumption that the watch dog was only ever reset in an atomic
context. Given that this is not the case, the warning can be
removed. Though before that happens, I have a question: if this is
called outside of an atomic context, is the use of __get_cpu_Var()
and __this_cpu_and() invalid?
It remains valid because its called with interrupts
disabled (see migrate_hrtimers).
Thank you, that was my assumption but I wanted to confirm. I do not
have access to my working kernel tree and won't for 2 days. I can't get
to the patch until late Wednesday so if someone else wants to work that
up I'd appreciate it.
Thanks,
Eric
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at arch/x86/kernel/kvmclock.c:127
On Sun, 10 Jun 2012 17:47:24 +0200, Sedat Dilek wrote:
Hi,
I see the same warning especially when resuming from a suspend (see
timestamps >=30-35sec) between Linux v3.5-rc1..v3.5-rc2.
$ sudo grep kvmclock.c /var/log/kern.log
Jun 4 21:11:32 fambox kernel: [ 784.037237] WARNING: at
arch/x86/kernel/kvmclock.c:127
kvm_check_and_clear_guest_paused+0x52/0x60()
Jun 5 20:35:57 fambox kernel: [ 1928.458060] WARNING: at
arch/x86/kernel/kvmclock.c:127
kvm_check_and_clear_guest_paused+0x52/0x60()
Jun 8 09:35:52 fambox kernel: [ 3290.134637] WARNING: at
arch/x86/kernel/kvmclock.c:127
kvm_check_and_clear_guest_paused+0x52/0x60()
Jun 8 09:35:52 fambox kernel: [ 3290.238592] WARNING: at
arch/x86/kernel/kvmclock.c:127
kvm_check_and_clear_guest_paused+0x52/0x60()
Jun 8 12:11:20 fambox kernel: [ 5777.023571] WARNING: at
arch/x86/kernel/kvmclock.c:127
kvm_check_and_clear_guest_paused+0x52/0x60()
Jun 9 13:32:50 fambox kernel: [ 2778.842695] WARNING: at
arch/x86/kernel/kvmclock.c:127
kvm_check_and_clear_guest_paused+0x52/0x60()
From [1]:
"...The warning itself is not required for the check_and_clear
function and can be removed as far as I am concerned."
From [2] commit 3b5d56b9317fa7b5407dff1aa7b115bf6cdbd494 ("kvmclock:
Add functions to check if the host has stopped the vm")
...
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only
called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags,
~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
...
( The export macro was dropped in a followup commit. )
So you mean "WARN_ON(!in_atomic());" can be deleted?
When I wrote the patch originally, I was under the (incorrect)
assumption that the watch dog was only ever reset in an atomic context.
Given that this is not the case, the warning can be removed. Though
before that happens, I have a question: if this is called outside of an
atomic context, is the use of __get_cpu_Var() and __this_cpu_and()
invalid?
Thanks,
Eric
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at arch/x86/kernel/kvmclock.c:127
On Wed, 06 Jun 2012 11:52:23 -0400, Eric B Munson wrote: On Tue, 5 Jun 2012 20:20:59 +0200, Frank Arnold wrote: Hi all, I observed the following warning during testing of 3.5-rc1, involving some intensive and random CPU online state toggling through sysfs. It only shows up about 1 out of 500 times. Also, there was no KVM guest running on the system. This warning kicks off if the watchdog_timer_fn function is called outside of an atomic context, the warning itself is not re Grr, keyboard shortcut... The warning itself is not required for the check_and_clear function and can be removed as far as I am concerned. Eric [ cut here ] WARNING: at arch/x86/kernel/kvmclock.c:127 kvm_check_and_clear_guest_paused+0x52/0x60() Hardware name: Dinar Modules linked in: nfs lockd fscache auth_rpcgss nfs_acl sunrpc ipv6 ext4 jbd2 dm_mod powernow_k8 freq_table mperf kvm_amd kvm crc32c_intel ghash_clmulni_intel aesni_intel cryptd aes_x86_64 aes_generic microcode serio_raw pcspkr k10temp amd64_edac_mod edac_core edac_mce_amd i2c_piix4 sg bnx2 ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif ata_generic pata_acpi pata_atiixp ahci libahci usb_storage radeon ttm drm_kms_helper drm i2c_algo_bit i2c_core [last unloaded: scsi_wait_scan] Pid: 3593, comm: py_topology Not tainted 3.5.0-rc1+ #1 Call Trace: [] warn_slowpath_common+0x7f/0xc0 [] warn_slowpath_null+0x1a/0x20 [] kvm_check_and_clear_guest_paused+0x52/0x60 [] watchdog_timer_fn+0x97/0x1b0 [] __run_hrtimer+0x83/0x1d0 [] ? touch_softlockup_watchdog_sync+0x40/0x40 [] hrtimer_interrupt+0x106/0x240 [] migrate_hrtimers+0x138/0x1a0 [] hrtimer_cpu_notify+0xe4/0xeb [] notifier_call_chain+0x55/0x80 [] __raw_notifier_call_chain+0xe/0x10 [] __cpu_notify+0x20/0x40 [] cpu_notify_nofail+0x15/0x30 [] _cpu_down+0x12d/0x270 [] cpu_down+0x36/0x50 [] store_online+0x74/0xd0 [] dev_attr_store+0x20/0x30 [] sysfs_write_file+0xef/0x170 [] vfs_write+0xc8/0x190 [] sys_write+0x51/0x90 [] system_call_fastpath+0x16/0x1b ---[ end trace 574db9e069f4f35e ]--- This is a regression introduced by the following commit: commit 3b5d56b9317fa7b5407dff1aa7b115bf6cdbd494 Author: Eric B Munson Date: Sat Mar 10 14:37:26 2012 -0500 kvmclock: Add functions to check if the host has stopped the vm When a host stops or suspends a VM it will set a flag to show this. The watchdog will use these functions to determine if a softlockup is real, or the result of a suspended VM. Signed-off-by: Eric B Munson asm-generic changes Acked-by: Arnd Bergmann Signed-off-by: Marcelo Tosatti Signed-off-by: Avi Kivity -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at arch/x86/kernel/kvmclock.c:127
On Tue, 5 Jun 2012 20:20:59 +0200, Frank Arnold wrote: Hi all, I observed the following warning during testing of 3.5-rc1, involving some intensive and random CPU online state toggling through sysfs. It only shows up about 1 out of 500 times. Also, there was no KVM guest running on the system. This warning kicks off if the watchdog_timer_fn function is called outside of an atomic context, the warning itself is not re [ cut here ] WARNING: at arch/x86/kernel/kvmclock.c:127 kvm_check_and_clear_guest_paused+0x52/0x60() Hardware name: Dinar Modules linked in: nfs lockd fscache auth_rpcgss nfs_acl sunrpc ipv6 ext4 jbd2 dm_mod powernow_k8 freq_table mperf kvm_amd kvm crc32c_intel ghash_clmulni_intel aesni_intel cryptd aes_x86_64 aes_generic microcode serio_raw pcspkr k10temp amd64_edac_mod edac_core edac_mce_amd i2c_piix4 sg bnx2 ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif ata_generic pata_acpi pata_atiixp ahci libahci usb_storage radeon ttm drm_kms_helper drm i2c_algo_bit i2c_core [last unloaded: scsi_wait_scan] Pid: 3593, comm: py_topology Not tainted 3.5.0-rc1+ #1 Call Trace: [] warn_slowpath_common+0x7f/0xc0 [] warn_slowpath_null+0x1a/0x20 [] kvm_check_and_clear_guest_paused+0x52/0x60 [] watchdog_timer_fn+0x97/0x1b0 [] __run_hrtimer+0x83/0x1d0 [] ? touch_softlockup_watchdog_sync+0x40/0x40 [] hrtimer_interrupt+0x106/0x240 [] migrate_hrtimers+0x138/0x1a0 [] hrtimer_cpu_notify+0xe4/0xeb [] notifier_call_chain+0x55/0x80 [] __raw_notifier_call_chain+0xe/0x10 [] __cpu_notify+0x20/0x40 [] cpu_notify_nofail+0x15/0x30 [] _cpu_down+0x12d/0x270 [] cpu_down+0x36/0x50 [] store_online+0x74/0xd0 [] dev_attr_store+0x20/0x30 [] sysfs_write_file+0xef/0x170 [] vfs_write+0xc8/0x190 [] sys_write+0x51/0x90 [] system_call_fastpath+0x16/0x1b ---[ end trace 574db9e069f4f35e ]--- This is a regression introduced by the following commit: commit 3b5d56b9317fa7b5407dff1aa7b115bf6cdbd494 Author: Eric B Munson Date: Sat Mar 10 14:37:26 2012 -0500 kvmclock: Add functions to check if the host has stopped the vm When a host stops or suspends a VM it will set a flag to show this. The watchdog will use these functions to determine if a softlockup is real, or the result of a suspended VM. Signed-off-by: Eric B Munson asm-generic changes Acked-by: Arnd Bergmann Signed-off-by: Marcelo Tosatti Signed-off-by: Avi Kivity -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Remove unneeded EXPORT macro
check_and_clear_guest_paused does not need to be exported as it isn't used
by any modules, remove the export.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: jan.kis...@siemens.com
Cc: a...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/kernel/kvmclock.c |1 -
1 files changed, 0 insertions(+), 1 deletions(-)
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 4ba090c..086eb58 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -133,7 +133,6 @@ bool kvm_check_and_clear_guest_paused(void)
return ret;
}
-EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
static struct clocksource kvm_clock = {
.name = "kvm-clock",
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/4 V16] Add functions to check if the host has stopped the vm
On Wed, 14 Mar 2012 11:12:35 +0100, Jan Kiszka wrote:
On 2012-03-10 20:37, Eric B Munson wrote:
When a host stops or suspends a VM it will set a flag to show this.
The
watchdog will use these functions to determine if a softlockup is
real, or the
result of a suspended VM.
...
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index f8492da..4ba090c 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+ * per_cpu() is safe here because this function is only called
from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
Which module is going to use this? I failed to find it, but I may
have
missed something.
The actual reason for this reply: this patches causes a compiler
warning
here, likely due to lacking include for EXPORT_SYMBOL_GPL:
CC arch/x86/kernel/kvmclock.o
/data/linux-kvm/arch/x86/kernel/kvmclock.c:136:1: warning: data
definition has no type or storage class
/data/linux-kvm/arch/x86/kernel/kvmclock.c:136:1: warning: type
defaults
to ‘int’ in declaration of ‘EXPORT_SYMBOL_GPL’
/data/linux-kvm/arch/x86/kernel/kvmclock.c:136:1: warning: parameter
names (without types) in function declaration
Jan
There isn't a module that will use it, I was under the (mistaken)
understanding that
the EXPORT was necessary for use in kernel/watchdog.c. Give that it
isn't, I will
post a patch to remove it shortly.
Eric
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/4 V16] Add functions to check if the host has stopped the vm
On Wed, 14 Mar 2012 11:12:35 +0100, Jan Kiszka wrote:
On 2012-03-10 20:37, Eric B Munson wrote:
When a host stops or suspends a VM it will set a flag to show this.
The
watchdog will use these functions to determine if a softlockup is
real, or the
result of a suspended VM.
...
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index f8492da..4ba090c 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+ * per_cpu() is safe here because this function is only called
from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
Which module is going to use this? I failed to find it, but I may
have
missed something.
The actual reason for this reply: this patches causes a compiler
warning
here, likely due to lacking include for EXPORT_SYMBOL_GPL:
CC arch/x86/kernel/kvmclock.o
/data/linux-kvm/arch/x86/kernel/kvmclock.c:136:1: warning: data
definition has no type or storage class
/data/linux-kvm/arch/x86/kernel/kvmclock.c:136:1: warning: type
defaults
to ‘int’ in declaration of ‘EXPORT_SYMBOL_GPL’
/data/linux-kvm/arch/x86/kernel/kvmclock.c:136:1: warning: parameter
names (without types) in function declaration
Jan
I will have a look at these this evening.
Eric
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4 V13] Avoid soft lockup message when KVM is stopped by host
On Mon, 05 Mar 2012, Marcelo Tosatti wrote: > On Tue, Feb 14, 2012 at 04:17:20PM -0500, Eric B Munson wrote: > > On Tue, 14 Feb 2012, Marcelo Tosatti wrote: > > > > > On Tue, Feb 14, 2012 at 10:50:13AM -0500, Eric B Munson wrote: > > > > On Tue, 14 Feb 2012, Marcelo Tosatti wrote: > > > > > > > > > On Tue, Feb 14, 2012 at 10:29:31AM -0500, Eric B Munson wrote: > > > > > > On Wed, 08 Feb 2012, Eric B Munson wrote: > > > > > > > > > > > > > > > > > > > > When a guest kernel is stopped by the host hypervisor it can look > > > > > > > like a soft > > > > > > > lockup to the guest kernel. This false warning can mask later > > > > > > > soft lockup > > > > > > > warnings which may be real. This patch series adds a method for > > > > > > > a host > > > > > > > hypervisor to communicate to a guest kernel that it is being > > > > > > > stopped. The > > > > > > > final patch in the series has the watchdog check this flag when > > > > > > > it goes to > > > > > > > issue a soft lockup warning and skip the warning if the guest > > > > > > > knows it was > > > > > > > stopped. > > > > > > > > > > > > > > It was attempted to solve this in Qemu, but the side effects of > > > > > > > saving and > > > > > > > restoring the clock and tsc for each vcpu put the wall clock of > > > > > > > the guest behind > > > > > > > by the amount of time of the pause. This forces a guest to have > > > > > > > ntp running > > > > > > > in order to keep the wall clock accurate. > > > > > > > > > > > > Avi, > > > > > > > > > > > > Is this set fit for merging or is there something else you want > > > > > > changed? > > > > > > > > > > Eric, > > > > > > > > > > On Message-ID: <20120210160536.ga23...@amt.cnet>, i asked: > > > > > > > > > > How is the stub getting included for other architectures again? > > > > > > > > > > > > > Marcelo, > > > > > > > > Sorry, I put out V13 to answer that. There is a stub in asm-generic > > > > that was > > > > lost in the V11-V12 rebase. This stub has be included in the V13 set. > > > > > > > > Eric > > > > > > Eric, > > > > > > I know the stub has been included in the series. But i am asking how > > > it is #include'ed for other architectures? (can't see that). > > > > Marcelo, > > > > kernel/watchdog.c now includes linux/kvm_para.h which includes > > asm/kvm_para.h. > > The check_and_clear function is defined in arch include/asm/kvm_para.h or in > > asm-generic/kvm_para.h for any arch lacking the specific header in their asm > > include dir. If I have misunderstood how these headers work, please let me > > know and I will fix it. > > There is no automatic inclusion of asm-generic/ headers. You must create > kvm_para.h in each architecture's include/asm/ directory, #including > asm-generic/kvm_para.h. > Hopefully V16 handles this properly, please let me know if there is anything that needs to be addressed. I'd like to see this set in for the 3.4 window. Eric signature.asc Description: Digital signature
[PATCH 4/4 V16] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index d117262..f6ffdc2 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V16] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The watchdog will use these functions to determine if a softlockup is real, or the result of a suspended VM. Signed-off-by: Eric B Munson asm-generic changes Acked-by: Arnd Bergmann Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org --- Changes from V15: Add stubs to include asm-generic/kvm_para.h where necessary Changes from V11: Re-add the missing asm-generic stub for check_and_clear_guest_stopped() Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic stubs into this patch check_and_clear_guest_stopped() takes no args and uses __get_cpu_var() Include individual definitions in ia64, s390, and powerpc arch/alpha/include/asm/kvm_para.h |1 + arch/arm/include/asm/kvm_para.h|1 + arch/avr32/include/asm/kvm_para.h |1 + arch/blackfin/include/asm/kvm_para.h |1 + arch/c6x/include/asm/kvm_para.h|1 + arch/frv/include/asm/kvm_para.h|1 + arch/h8300/include/asm/kvm_para.h |1 + arch/hexagon/include/asm/kvm_para.h|1 + arch/ia64/include/asm/kvm_para.h |5 + arch/m68k/include/asm/kvm_para.h |1 + arch/microblaze/include/asm/kvm_para.h |1 + arch/mips/include/asm/kvm_para.h |1 + arch/mn10300/include/asm/kvm_para.h|1 + arch/openrisc/include/asm/kvm_para.h |1 + arch/parisc/include/asm/kvm_para.h |1 + arch/powerpc/include/asm/kvm_para.h|5 + arch/s390/include/asm/kvm_para.h |5 + arch/score/include/asm/kvm_para.h |1 + arch/sh/include/asm/kvm_para.h |1 + arch/sparc/include/asm/kvm_para.h |1 + arch/tile/include/asm/kvm_para.h |1 + arch/um/include/asm/kvm_para.h |1 + arch/unicore32/include/asm/kvm_para.h |1 + arch/x86/include/asm/kvm_para.h|8 arch/x86/kernel/kvmclock.c | 21 + arch/xtensa/include/asm/kvm_para.h |1 + include/asm-generic/kvm_para.h | 14 ++ 27 files changed, 79 insertions(+), 0 deletions(-) create mode 100644 arch/alpha/include/asm/kvm_para.h create mode 100644 arch/arm/include/asm/kvm_para.h create mode 100644 arch/avr32/include/asm/kvm_para.h create mode 100644 arch/blackfin/include/asm/kvm_para.h create mode 100644 arch/c6x/include/asm/kvm_para.h create mode 100644 arch/frv/include/asm/kvm_para.h create mode 100644 arch/h8300/include/asm/kvm_para.h create mode 100644 arch/hexagon/include/asm/kvm_para.h create mode 100644 arch/m68k/include/asm/kvm_para.h create mode 100644 arch/microblaze/include/asm/kvm_para.h create mode 100644 arch/mips/include/asm/kvm_para.h create mode 100644 arch/mn10300/include/asm/kvm_para.h create mode 100644 arch/openrisc/include/asm/kvm_para.h create mode 100644 arch/parisc/include/asm/kvm_para.h create mode 100644 arch/score/include/asm/kvm_para.h create mode 100644 arch/sh/include/asm/kvm_para.h create mode 100644 arch/sparc/include/asm/kvm_para.h create mode 100644 arch/tile/include/asm/kvm_para.h create mode 100644 arch/um/include/asm/kvm_para.h create mode 100644 arch/unicore32/include/asm/kvm_para.h create mode 100644 arch/xtensa/include/asm/kvm_para.h create mode 100644 include/asm-generic/kvm_para.h diff --git a/arch/alpha/include/asm/kvm_para.h b/arch/alpha/include/asm/kvm_para.h new file mode 100644 index 000..14fab8f --- /dev/null +++ b/arch/alpha/include/asm/kvm_para.h @@ -0,0 +1 @@ +#include diff --git a/arch/arm/include/asm/kvm_para.h b/arch/arm/include/asm/kvm_para.h new file mode 100644 index 000..14fab8f --- /dev/null +++ b/arch/arm/include/asm/kvm_para.h @@ -0,0 +1 @@ +#include diff --git a/arch/avr32/include/asm/kvm_para.h b/arch/avr32/include/asm/kvm_para.h new file mode 100644 index 000..14fab8f --- /dev/null +++ b/arch/avr32/include/asm/kvm_para.h @@ -0,0 +1 @@ +#include diff --git a/arch/blackfin/include/asm/kvm_para.h b/arch/blackfin/include/asm/kvm_para.h new file mode 100644 index 000..14fab8f --- /dev/null +++ b/arch/blackfin/include/asm/kvm_para.h @@ -0,0 +1 @@ +#include diff --git a/arch/c6x/include/asm/kvm_para.h b/arch/c6x/include/asm/kvm_para.h new file mode 100644 index 000..14fab8f --- /dev/null +++ b/arch/c6x/include/asm/kvm_para.h @@ -0,0 +1 @@ +#include diff --git a/arch/frv/include/asm/kvm_para.h b/arch/frv/include/asm/kvm_para.h new file mode 100644 index 000..14fab8f --- /dev/null +++ b/arch/frv/include/asm/kvm_para.h @@ -0,0 +1 @@ +#include diff --git a/arch/h8300/include/asm/kvm_para.h b/arch/h8300/include/asm/kvm_para.h new file mode 100644 index 000..14fab8f --- /dev/null +++ b/arch/h8300/include/asm/kvm_para.h @@ -0,0 +1 @@ +#include diff --git a/arch
[PATCH 3/4 V16] Add ioctl for KVM_KVMCLOCK_CTRL
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V14:
Add description of flags bit used for host->guest communication
Changes from V13:
Expand the ioctl documentation
Changes from V12:
Rename KVM_CAP and KVM ioctl
Changes from V11:
Correct typo in api.txt
Add kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) call to kvm_set_guest_paused()
Changes from V10:
Return the ioctl to per vcpu instead of per vm
Changes from V9:
Use kvm_for_each_vcpu to iterate online vcpu's
Changes from V8:
Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 20
Documentation/virtual/kvm/msr.txt |4
arch/x86/kvm/x86.c| 22 ++
include/linux/kvm.h |3 +++
4 files changed, 49 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 6386f8c..81ff39f 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1669,6 +1669,26 @@ at the memory location pointed to by "addr".
The list of registers accessible using this interface is identical to the
list in 4.64.
+4.70 KVM_KVMCLOCK_CTRL
+
+Capability: KVM_CAP_KVMCLOCK_CTRL
+Architectures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog. The flag is part of the pvclock structure that
+is shared between guest and host, specifically the second bit of the flags
+field of the pvclock_vcpu_time_info structure. It will be set exclusively by
+the host and read/cleared exclusively by the guest. The guest operation of
+checking and clearing the flag must an atomic operation so
+load-link/store-conditional, or equivalent must be used. There are two cases
+where the guest will clear the flag: when the soft lockup watchdog timer resets
+itself or when a soft lockup is detected. This ioctl can be called any time
+after pausing the vcpu, but before it is resumed.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/Documentation/virtual/kvm/msr.txt
b/Documentation/virtual/kvm/msr.txt
index 5031780..96b41bd 100644
--- a/Documentation/virtual/kvm/msr.txt
+++ b/Documentation/virtual/kvm/msr.txt
@@ -109,6 +109,10 @@ MSR_KVM_SYSTEM_TIME_NEW: 0x4b564d01
0 | 24 | multiple cpus are guaranteed to
| | be monotonic
-
+ | | guest vcpu has been paused by
+1 | N/A | the host
+ | | See 4.70 in api.txt
+ -
Availability of this MSR must be checked via bit 3 in 0x401 cpuid
leaf prior to usage.
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 7ce5878..4e1bf3b 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2145,6 +2145,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_ASYNC_PF:
case KVM_CAP_GET_TSC_KHZ:
case KVM_CAP_PCI_2_3:
+ case KVM_CAP_KVMCLOCK_CTRL:
r = 1;
break;
case KVM_CAP_COALESCED_MMIO:
@@ -2595,6 +2596,23 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu
*vcpu,
return r;
}
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ mark_page_dirty(vcpu->kvm, vcpu->arch.time >> PAGE_SHIFT);
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
+ return 0;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp,
[PATCH 1/4 V16] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4 V16] Avoid soft lockup message when KVM is stopped by host
Changes from V15: Add arch/*/include/asm/kvm_para.h that includes the asm-generic version Changes from V14: Update documentation for the pvclock api Changes from V13: Expand on KVM_KVMCLOCK_CTRL ioctl documentation Changes from V12: Re-add missing kvm.c code after rebase Rename CAP to KVM_CAP_KVMCLOCK_CTRL Rename ioctl to KVM_KVMCLOCK_CTRL Changes from V11: Re-add asm-generic stub Correct api.txt typo add kvm_make_request() call after setting PVCLOCK_GUEST_STOPPED Changes from V10: Return ioctl to per vcpu instead of per vm Changes from V9: Use kvm_for_each_vcpu to iterate online vcpu's Changes from V8: Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu Changes from V7: Define KVM_CAP_GUEST_PAUSED and support check Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVM_KVMCLOCK_CTRL Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 20 Documentation/virtual/kvm/msr.txt |4 arch/alpha/include/asm/kvm_para.h |1 + arch/arm/include/asm/kvm_para.h|1 + arch/avr32/include/asm/kvm_para.h |1 + arch/blackfin/include/asm/kvm_para.h |1 + arch/c6x/include/asm/kvm_para.h|1 + arch/frv/include/asm/kvm_para.h|1 + arch/h8300/include/asm/kvm_para.h |1 + arch/hexagon/include/asm/kvm_para.h|1 + arch/ia64/include/asm/kvm_para.h |5 + arch/m68k/include/asm/kvm_para.h |1 + arch/microblaze/include/asm/kvm_para.h |1 + arch/mips/include/asm/kvm_para.h |1 + arch/mn10300/include/asm/kvm_para.h|1 + arch/openrisc/include/asm/kvm_para.h |1 + arch/parisc/include/asm/kvm_para.h |1 + arch/powerpc/include/asm/kvm_para.h|5 + arch/s390/include/asm/kvm_para.h |5 + arch/score/include/asm/kvm_para.h |1 + arch/sh/include/asm/kvm_para.h |1 + arch/sparc/include/asm/kvm_para.h |1 + arch/tile/include/asm/kvm_para.h |1 + arch/um/include/asm/kvm_para.h |1 + arch/unicore32/include/asm/kvm_para.h |1 + arch/x86/include/asm/kvm_para.h|8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 22 ++ arch/xtensa/include/asm/kvm_para.h |1 + include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h|3 +++ kernel/watchdog.c | 12 33 files changed, 141 insertions(+), 0 deletions(-) create mode 100644 arch/alpha/include/asm/kvm_para.h create mode 100644 arch/arm/include/asm/kvm_para.h create mode 100644 arch/avr32/include/asm/kvm_para.h create mode 100644 arch/blackfin/include/asm/kvm_para.h create mode 100644 arch/c6x/include/asm/kvm_para.h create mode 100644 arch/frv/include/asm/kvm_para.h c
Re: [PATCH 0/4 V13] Avoid soft lockup message when KVM is stopped by host
On Mon, 5 Mar 2012 13:39:43 -0300, Marcelo Tosatti wrote: On Tue, Feb 14, 2012 at 04:17:20PM -0500, Eric B Munson wrote: On Tue, 14 Feb 2012, Marcelo Tosatti wrote: > On Tue, Feb 14, 2012 at 10:50:13AM -0500, Eric B Munson wrote: > > On Tue, 14 Feb 2012, Marcelo Tosatti wrote: > > > > > On Tue, Feb 14, 2012 at 10:29:31AM -0500, Eric B Munson wrote: > > > > On Wed, 08 Feb 2012, Eric B Munson wrote: > > > > > > > > > > > > > > When a guest kernel is stopped by the host hypervisor it can look like a soft > > > > > lockup to the guest kernel. This false warning can mask later soft lockup > > > > > warnings which may be real. This patch series adds a method for a host > > > > > hypervisor to communicate to a guest kernel that it is being stopped. The > > > > > final patch in the series has the watchdog check this flag when it goes to > > > > > issue a soft lockup warning and skip the warning if the guest knows it was > > > > > stopped. > > > > > > > > > > It was attempted to solve this in Qemu, but the side effects of saving and > > > > > restoring the clock and tsc for each vcpu put the wall clock of the guest behind > > > > > by the amount of time of the pause. This forces a guest to have ntp running > > > > > in order to keep the wall clock accurate. > > > > > > > > Avi, > > > > > > > > Is this set fit for merging or is there something else you want changed? > > > > > > Eric, > > > > > > On Message-ID: <20120210160536.ga23...@amt.cnet>, i asked: > > > > > > How is the stub getting included for other architectures again? > > > > > > > Marcelo, > > > > Sorry, I put out V13 to answer that. There is a stub in asm-generic that was > > lost in the V11-V12 rebase. This stub has be included in the V13 set. > > > > Eric > > Eric, > > I know the stub has been included in the series. But i am asking how > it is #include'ed for other architectures? (can't see that). Marcelo, kernel/watchdog.c now includes linux/kvm_para.h which includes asm/kvm_para.h. The check_and_clear function is defined in arch include/asm/kvm_para.h or in asm-generic/kvm_para.h for any arch lacking the specific header in their asm include dir. If I have misunderstood how these headers work, please let me know and I will fix it. There is no automatic inclusion of asm-generic/ headers. You must create kvm_para.h in each architecture's include/asm/ directory, #including asm-generic/kvm_para.h. Okay, that will go into V16 then... Eric -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4 V15] Avoid soft lockup message when KVM is stopped by host
On Sun, 19 Feb 2012 21:22:53 -0500, Eric B Munson wrote: Changes from V14: Update documentation for the pvclock api ... snip When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Avi, Is this version acceptable for merging? I'd really like this to go in for the 3.4 merge window. Thanks, Eric Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVM_KVMCLOCK_CTRL Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 20 Documentation/virtual/kvm/msr.txt |4 arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 22 ++ include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |3 +++ kernel/watchdog.c | 12 12 files changed, 120 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4 V15] Add ioctl for KVM_KVMCLOCK_CTRL
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V14:
Add description of flags bit used for host->guest communication
Changes from V13:
Expand the ioctl documentation
Changes from V12:
Rename KVM_CAP and KVM ioctl
Changes from V11:
Correct typo in api.txt
Add kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) call to kvm_set_guest_paused()
Changes from V10:
Return the ioctl to per vcpu instead of per vm
Changes from V9:
Use kvm_for_each_vcpu to iterate online vcpu's
Changes from V8:
Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 20
Documentation/virtual/kvm/msr.txt |4
arch/x86/kvm/x86.c| 22 ++
include/linux/kvm.h |3 +++
4 files changed, 49 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 59a3826..c9617b3 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1628,6 +1628,26 @@ at the memory location pointed to by "addr".
The list of registers accessible using this interface is identical to the
list in 4.64.
+4.70 KVM_KVMCLOCK_CTRL
+
+Capability: KVM_CAP_KVMCLOCK_CTRL
+Architectures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog. The flag is part of the pvclock structure that
+is shared between guest and host, specifically the second bit of the flags
+field of the pvclock_vcpu_time_info structure. It will be set exclusively by
+the host and read/cleared exclusively by the guest. The guest operation of
+checking and clearing the flag must an atomic operation so
+load-link/store-conditional, or equivalent must be used. There are two cases
+where the guest will clear the flag: when the soft lockup watchdog timer resets
+itself or when a soft lockup is detected. This ioctl can be called any time
+after pausing the vcpu, but before it is resumed.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/Documentation/virtual/kvm/msr.txt
b/Documentation/virtual/kvm/msr.txt
index 5031780..96b41bd 100644
--- a/Documentation/virtual/kvm/msr.txt
+++ b/Documentation/virtual/kvm/msr.txt
@@ -109,6 +109,10 @@ MSR_KVM_SYSTEM_TIME_NEW: 0x4b564d01
0 | 24 | multiple cpus are guaranteed to
| | be monotonic
-
+ | | guest vcpu has been paused by
+1 | N/A | the host
+ | | See 4.70 in api.txt
+ -
Availability of this MSR must be checked via bit 3 in 0x401 cpuid
leaf prior to usage.
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index c9d99e5..7ba8213 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2143,6 +2143,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_XSAVE:
case KVM_CAP_ASYNC_PF:
case KVM_CAP_GET_TSC_KHZ:
+ case KVM_CAP_KVMCLOCK_CTRL:
r = 1;
break;
case KVM_CAP_COALESCED_MMIO:
@@ -2593,6 +2594,23 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu
*vcpu,
return r;
}
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ mark_page_dirty(vcpu->kvm, vcpu->arch.time >> PAGE_SHIFT);
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
+ return 0;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp,
[PATCH 4/4 V15] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index d117262..f6ffdc2 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V15] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4 V15] Avoid soft lockup message when KVM is stopped by host
Changes from V14: Update documentation for the pvclock api Changes from V13: Expand on KVM_KVMCLOCK_CTRL ioctl documentation Changes from V12: Re-add missing kvm.c code after rebase Rename CAP to KVM_CAP_KVMCLOCK_CTRL Rename ioctl to KVM_KVMCLOCK_CTRL Changes from V11: Re-add asm-generic stub Correct api.txt typo add kvm_make_request() call after setting PVCLOCK_GUEST_STOPPED Changes from V10: Return ioctl to per vcpu instead of per vm Changes from V9: Use kvm_for_each_vcpu to iterate online vcpu's Changes from V8: Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu Changes from V7: Define KVM_CAP_GUEST_PAUSED and support check Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVM_KVMCLOCK_CTRL Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 20 Documentation/virtual/kvm/msr.txt |4 arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 22 ++ include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |3 +++ kernel/watchdog.c | 12 12 files changed, 120 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V15] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
asm-generic changes Acked-by: Arnd Bergmann
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V11:
Re-add the missing asm-generic stub for check_and_clear_guest_stopped()
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 7b754e7..c18916b 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -206,6 +206,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index ca4e735..8562f77 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/4 V13] Add ioctl for KVM_KVMCLOCK_CTRL
On Thu, 16 Feb 2012, Eric B Munson wrote: > Now that we have a flag that will tell the guest it was suspended, create an > interface for that communication using a KVM ioctl. > > Signed-off-by: Eric B Munson Sorry, this is actually V14. signature.asc Description: Digital signature
[PATCH 2/4 V14] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
asm-generic changes Acked-by: Arnd Bergmann
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V11:
Re-add the missing asm-generic stub for check_and_clear_guest_stopped()
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 7b754e7..c18916b 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -206,6 +206,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index ca4e735..8562f77 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4 V14] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index d117262..f6ffdc2 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4 V13] Add ioctl for KVM_KVMCLOCK_CTRL
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V13:
Expand the ioctl documentation
Changes from V12:
Rename KVM_CAP and KVM ioctl
Changes from V11:
Correct typo in api.txt
Add kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) call to kvm_set_guest_paused()
Changes from V10:
Return the ioctl to per vcpu instead of per vm
Changes from V9:
Use kvm_for_each_vcpu to iterate online vcpu's
Changes from V8:
Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 21 +
arch/x86/kvm/x86.c| 22 ++
include/linux/kvm.h |3 +++
3 files changed, 46 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 59a3826..d868e80 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1628,6 +1628,27 @@ at the memory location pointed to by "addr".
The list of registers accessible using this interface is identical to the
list in 4.64.
+4.70 KVM_KVMCLOCK_CTRL
+
+Capability: KVM_CAP_KVMCLOCK_CTRL
+Architectures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog. The flag is part of the pvclock structure that
+is shared between guest and host, specifically the second bit of the flags
+field of the pvclock_vcpu_time_info structure. It will be set exclusively by
+the host and read/cleared exclusively by the guest. The guest operation of
+checking and clearing the flag must an atomic operation so use of the function
+check_and_clear_guest_paused() is encouraged, but it could also be done with
+load-link/store-conditional. There are two cases where the guest will clear
+the flag: when the soft lockup watchdog timer resets itself or when a soft
+lockup is detected. This ioctl can be called any time after pausing
+the vcpu, but before it is resumed.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index c9d99e5..7ba8213 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2143,6 +2143,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_XSAVE:
case KVM_CAP_ASYNC_PF:
case KVM_CAP_GET_TSC_KHZ:
+ case KVM_CAP_KVMCLOCK_CTRL:
r = 1;
break;
case KVM_CAP_COALESCED_MMIO:
@@ -2593,6 +2594,23 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu
*vcpu,
return r;
}
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ mark_page_dirty(vcpu->kvm, vcpu->arch.time >> PAGE_SHIFT);
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
+ return 0;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -2869,6 +2887,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
r = vcpu->arch.virtual_tsc_khz;
goto out;
}
+ case KVM_KVMCLOCK_CTRL: {
+ r = kvm_set_guest_paused(vcpu);
+ goto out;
+ }
default:
r = -EINVAL;
}
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index acbe429..a90db11 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -588,6 +588,7 @@ struct kvm_ppc_pvinfo {
#define KVM_CAP_TSC_DEADLINE_TIMER 72
#define KVM_CAP_S390_UCONTROL 73
#define KVM_CAP_SYNC_REGS 74
+#define KVM_CAP_KVMCLOCK_CTRL 75
#ifdef KVM_CAP_IRQ_ROUTING
@@ -855,6 +856,8 @@ struct kvm_s390_ucas_mapping {
/* Available with KVM_CAP_ONE_REG */
#define KVM_GET_ONE_REG _IOW(KVMIO, 0xab, struct kvm_one_reg)
#define
[PATCH 0/4 V14] Avoid soft lockup message when KVM is stopped by host
Changes from V13: Expand on KVM_KVMCLOCK_CTRL ioctl documentation Changes from V12: Re-add missing kvm.c code after rebase Rename CAP to KVM_CAP_KVMCLOCK_CTRL Rename ioctl to KVM_KVMCLOCK_CTRL Changes from V11: Re-add asm-generic stub Correct api.txt typo add kvm_make_request() call after setting PVCLOCK_GUEST_STOPPED Changes from V10: Return ioctl to per vcpu instead of per vm Changes from V9: Use kvm_for_each_vcpu to iterate online vcpu's Changes from V8: Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu Changes from V7: Define KVM_CAP_GUEST_PAUSED and support check Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVM_KVMCLOCK_CTRL Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 21 + arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 22 ++ include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |3 +++ kernel/watchdog.c | 12 11 files changed, 117 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V14] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4 V13] Avoid soft lockup message when KVM is stopped by host
On Tue, 14 Feb 2012, Marcelo Tosatti wrote: > On Tue, Feb 14, 2012 at 10:50:13AM -0500, Eric B Munson wrote: > > On Tue, 14 Feb 2012, Marcelo Tosatti wrote: > > > > > On Tue, Feb 14, 2012 at 10:29:31AM -0500, Eric B Munson wrote: > > > > On Wed, 08 Feb 2012, Eric B Munson wrote: > > > > > > > > > > > > > > When a guest kernel is stopped by the host hypervisor it can look > > > > > like a soft > > > > > lockup to the guest kernel. This false warning can mask later soft > > > > > lockup > > > > > warnings which may be real. This patch series adds a method for a > > > > > host > > > > > hypervisor to communicate to a guest kernel that it is being stopped. > > > > > The > > > > > final patch in the series has the watchdog check this flag when it > > > > > goes to > > > > > issue a soft lockup warning and skip the warning if the guest knows > > > > > it was > > > > > stopped. > > > > > > > > > > It was attempted to solve this in Qemu, but the side effects of > > > > > saving and > > > > > restoring the clock and tsc for each vcpu put the wall clock of the > > > > > guest behind > > > > > by the amount of time of the pause. This forces a guest to have ntp > > > > > running > > > > > in order to keep the wall clock accurate. > > > > > > > > Avi, > > > > > > > > Is this set fit for merging or is there something else you want changed? > > > > > > Eric, > > > > > > On Message-ID: <20120210160536.ga23...@amt.cnet>, i asked: > > > > > > How is the stub getting included for other architectures again? > > > > > > > Marcelo, > > > > Sorry, I put out V13 to answer that. There is a stub in asm-generic that > > was > > lost in the V11-V12 rebase. This stub has be included in the V13 set. > > > > Eric > > Eric, > > I know the stub has been included in the series. But i am asking how > it is #include'ed for other architectures? (can't see that). Marcelo, kernel/watchdog.c now includes linux/kvm_para.h which includes asm/kvm_para.h. The check_and_clear function is defined in arch include/asm/kvm_para.h or in asm-generic/kvm_para.h for any arch lacking the specific header in their asm include dir. If I have misunderstood how these headers work, please let me know and I will fix it. Eric signature.asc Description: Digital signature
Re: [PATCH 0/4 V13] Avoid soft lockup message when KVM is stopped by host
On Tue, 14 Feb 2012, Marcelo Tosatti wrote: > On Tue, Feb 14, 2012 at 10:29:31AM -0500, Eric B Munson wrote: > > On Wed, 08 Feb 2012, Eric B Munson wrote: > > > > > > > > When a guest kernel is stopped by the host hypervisor it can look like a > > > soft > > > lockup to the guest kernel. This false warning can mask later soft lockup > > > warnings which may be real. This patch series adds a method for a host > > > hypervisor to communicate to a guest kernel that it is being stopped. The > > > final patch in the series has the watchdog check this flag when it goes to > > > issue a soft lockup warning and skip the warning if the guest knows it was > > > stopped. > > > > > > It was attempted to solve this in Qemu, but the side effects of saving and > > > restoring the clock and tsc for each vcpu put the wall clock of the guest > > > behind > > > by the amount of time of the pause. This forces a guest to have ntp > > > running > > > in order to keep the wall clock accurate. > > > > Avi, > > > > Is this set fit for merging or is there something else you want changed? > > Eric, > > On Message-ID: <20120210160536.ga23...@amt.cnet>, i asked: > > How is the stub getting included for other architectures again? > Marcelo, Sorry, I put out V13 to answer that. There is a stub in asm-generic that was lost in the V11-V12 rebase. This stub has be included in the V13 set. Eric signature.asc Description: Digital signature
Re: [PATCH 0/4 V13] Avoid soft lockup message when KVM is stopped by host
On Wed, 08 Feb 2012, Eric B Munson wrote: > > When a guest kernel is stopped by the host hypervisor it can look like a soft > lockup to the guest kernel. This false warning can mask later soft lockup > warnings which may be real. This patch series adds a method for a host > hypervisor to communicate to a guest kernel that it is being stopped. The > final patch in the series has the watchdog check this flag when it goes to > issue a soft lockup warning and skip the warning if the guest knows it was > stopped. > > It was attempted to solve this in Qemu, but the side effects of saving and > restoring the clock and tsc for each vcpu put the wall clock of the guest > behind > by the amount of time of the pause. This forces a guest to have ntp running > in order to keep the wall clock accurate. Avi, Is this set fit for merging or is there something else you want changed? Eric > > Cc: mi...@redhat.com > Cc: h...@zytor.com > Cc: ry...@linux.vnet.ibm.com > Cc: aligu...@us.ibm.com > Cc: mtosa...@redhat.com > Cc: kvm@vger.kernel.org > Cc: linux-a...@vger.kernel.org > Cc: x...@kernel.org > Cc: linux-ker...@vger.kernel.org > > Eric B Munson (4): > Add flag to indicate that a vm was stopped by the host > Add functions to check if the host has stopped the vm > Add ioctl for KVM_KVMCLOCK_CTRL > Add check for suspended vm in softlockup detector > > Documentation/virtual/kvm/api.txt | 13 + > arch/ia64/include/asm/kvm_para.h|5 + > arch/powerpc/include/asm/kvm_para.h |5 + > arch/s390/include/asm/kvm_para.h|5 + > arch/x86/include/asm/kvm_para.h |8 > arch/x86/include/asm/pvclock-abi.h |1 + > arch/x86/kernel/kvmclock.c | 21 + > arch/x86/kvm/x86.c | 22 ++ > include/asm-generic/kvm_para.h | 14 ++ > include/linux/kvm.h |3 +++ > kernel/watchdog.c | 12 > 11 files changed, 109 insertions(+), 0 deletions(-) > create mode 100644 include/asm-generic/kvm_para.h > > -- > 1.7.5.4 > signature.asc Description: Digital signature
[PATCH 3/4 V13] Add ioctl for KVM_KVMCLOCK_CTRL
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V12:
Rename KVM_CAP and KVM ioctl
Changes from V11:
Correct typo in api.txt
Add kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) call to kvm_set_guest_paused()
Changes from V10:
Return the ioctl to per vcpu instead of per vm
Changes from V9:
Use kvm_for_each_vcpu to iterate online vcpu's
Changes from V8:
Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 13 +
arch/x86/kvm/x86.c| 22 ++
include/linux/kvm.h |3 +++
3 files changed, 38 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 5b03eee..c6f3ac7 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1482,6 +1482,19 @@ following algorithm:
Some guests configure the LINT1 NMI input to cause a panic, aiding in
debugging.
+4.65 KVM_KVMCLOCK_CTRL
+
+Capability: KVM_CAP_KVMCLOCK_CTRL
+Architectures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog. This ioctl can be called during pause or
+unpause.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 23c93fe..1895e4a 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2050,6 +2050,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_XSAVE:
case KVM_CAP_ASYNC_PF:
case KVM_CAP_GET_TSC_KHZ:
+ case KVM_CAP_KVMCLOCK_CTRL:
r = 1;
break;
case KVM_CAP_COALESCED_MMIO:
@@ -2493,6 +2494,23 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu
*vcpu,
return r;
}
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ mark_page_dirty(vcpu->kvm, vcpu->arch.time >> PAGE_SHIFT);
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
+ return 0;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -2782,6 +2800,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
goto out;
}
+ case KVM_KVMCLOCK_CTRL: {
+ r = kvm_set_guest_paused(vcpu);
+ goto out;
+ }
default:
r = -EINVAL;
}
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index c3892fc..27d5c46 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -557,6 +557,7 @@ struct kvm_ppc_pvinfo {
#define KVM_CAP_MAX_VCPUS 66 /* returns max vcpus per vm */
#define KVM_CAP_PPC_PAPR 68
#define KVM_CAP_S390_GMAP 71
+#define KVM_CAP_KVMCLOCK_CTRL 72
#ifdef KVM_CAP_IRQ_ROUTING
@@ -762,6 +763,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVM_KVMCLOCK_CTRL_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4 V13] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..91485e5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V13] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
asm-generic changes Acked-by: Arnd Bergmann
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V11:
Re-add the missing asm-generic stub for check_and_clear_guest_stopped()
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 50533f9..1f80293 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -169,6 +169,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..bdf6423 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4 V13] Avoid soft lockup message when KVM is stopped by host
Changes from V12: Re-add missing kvm.c code after rebase Rename CAP to KVM_CAP_KVMCLOCK_CTRL Rename ioctl to KVM_KVMCLOCK_CTRL Changes from V11: Re-add asm-generic stub Correct api.txt typo add kvm_make_request() call after setting PVCLOCK_GUEST_STOPPED Changes from V10: Return ioctl to per vcpu instead of per vm Changes from V9: Use kvm_for_each_vcpu to iterate online vcpu's Changes from V8: Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu Changes from V7: Define KVM_CAP_GUEST_PAUSED and support check Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVM_KVMCLOCK_CTRL Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 13 + arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 22 ++ include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |3 +++ kernel/watchdog.c | 12 11 files changed, 109 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V13] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4 V12] Avoid soft lockup message when KVM is stopped by host
I managed to miss the discussion from the V11 thread, I will work those comments in tomorrow and send V13 as soon as that is done. Eric -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V12] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
asm-generic changes Acked-by: Arnd Bergmann
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V11:
Re-add the missing asm-generic stub for check_and_clear_guest_stopped()
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 50533f9..1f80293 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -169,6 +169,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..bdf6423 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..91485e5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4 V12] Avoid soft lockup message when KVM is stopped by host
Changes from V11: Re-add asm-generic stub Correct api.txt typo add kvm_make_request() call after setting PVCLOCK_GUEST_STOPPED Changes from V10: Return ioctl to per vcpu instead of per vm Changes from V9: Use kvm_for_each_vcpu to iterate online vcpu's Changes from V8: Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu Changes from V7: Define KVM_CAP_GUEST_PAUSED and support check Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVMCLOCK_GUEST_STOPPED Add check for suspended vm in softlockup detector arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |3 +++ kernel/watchdog.c | 12 9 files changed, 74 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4 V12] Add ioctl for KVMCLOCK_GUEST_STOPPED
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V11:
Correct typo in api.txt
Add kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) call to kvm_set_guest_paused()
Changes from V10:
Return the ioctl to per vcpu instead of per vm
Changes from V9:
Use kvm_for_each_vcpu to iterate online vcpu's
Changes from V8:
Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
include/linux/kvm.h |3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index c3892fc..dbfc4f8 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -557,6 +557,7 @@ struct kvm_ppc_pvinfo {
#define KVM_CAP_MAX_VCPUS 66 /* returns max vcpus per vm */
#define KVM_CAP_PPC_PAPR 68
#define KVM_CAP_S390_GMAP 71
+#define KVM_CAP_GUEST_PAUSED 72
#ifdef KVM_CAP_IRQ_ROUTING
@@ -762,6 +763,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVMCLOCK_GUEST_PAUSED_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V12] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4 V11] Avoid soft lockup message when KVM is stopped by host
Changes from V10: Return ioctl to per vcpu instead of per vm Changes from V9: Use kvm_for_each_vcpu to iterate online vcpu's Changes from V8: Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu Changes from V7: Define KVM_CAP_GUEST_PAUSED and support check Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVMCLOCK_GUEST_STOPPED Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 13 + arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 21 + include/linux/kvm.h |3 +++ kernel/watchdog.c | 12 10 files changed, 94 insertions(+), 0 deletions(-) -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V11] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
asm-generic changes Acked-by: Arnd Bergmann
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
5 files changed, 44 insertions(+), 0 deletions(-)
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 50533f9..1f80293 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -169,6 +169,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..bdf6423 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4 V11] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..91485e5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V11] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4 V11] Add ioctl for KVMCLOCK_GUEST_STOPPED
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V10:
Return the ioctl to per vcpu instead of per vm
Changes from V9:
Use kvm_for_each_vcpu to iterate online vcpu's
Changes from V8:
Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 13 +
arch/x86/kvm/x86.c| 21 +
include/linux/kvm.h |3 +++
3 files changed, 37 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index e1d94bf..1931e5c 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1491,6 +1491,19 @@ following algorithm:
Some guests configure the LINT1 NMI input to cause a panic, aiding in
debugging.
+4.65 KVMCLOCK_GUEST_PAUSED
+
+Capability: KVM_CAP_GUEST_PAUSED
+Architechtures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog. This ioctl can be called during pause or
+unpause.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 14d6cad..c9cabba 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2056,6 +2056,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_X86_ROBUST_SINGLESTEP:
case KVM_CAP_XSAVE:
case KVM_CAP_ASYNC_PF:
+ case KVM_CAP_GUEST_PAUSED:
case KVM_CAP_GET_TSC_KHZ:
r = 1;
break;
@@ -2503,6 +2504,22 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu
*vcpu,
return r;
}
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ mark_page_dirty(vcpu->kvm, vcpu->arch.time >> PAGE_SHIFT);
+ return 0;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -2784,6 +2801,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
goto out;
}
+ case KVMCLOCK_GUEST_PAUSED: {
+ r = kvm_set_guest_paused(vcpu);
+ break;
+ }
default:
r = -EINVAL;
}
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index 68e67e5..4ffe0df 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -558,6 +558,7 @@ struct kvm_ppc_pvinfo {
#define KVM_CAP_PPC_PAPR 68
#define KVM_CAP_S390_GMAP 71
#define KVM_CAP_TSC_DEADLINE_TIMER 72
+#define KVM_CAP_GUEST_PAUSED 73
#ifdef KVM_CAP_IRQ_ROUTING
@@ -763,6 +764,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVMCLOCK_GUEST_PAUSED_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/4 V10] Add ioctl for KVMCLOCK_GUEST_STOPPED
On Mon, 30 Jan 2012, Avi Kivity wrote: > On 01/30/2012 05:32 PM, Eric B Munson wrote: > > > > > > Can you point me to the discussion that moved this to be a vm ioctl? In > > > general vm ioctls that do things for all vcpus are racy, like here. > > > You're accessing variables that are protected by the vcpu mutex, and not > > > taking the mutex (nor can you, since it is held while the guest is > > > running, unlike most kernel mutexes). > > > > > > > Jan Kiszka suggested that becuase there isn't a use case for notifying > > individual vcpus (can vcpu's be paused individually? > > They can, though the guest will grind to a halt very soon. > > > ) that it makes more sense > > to have a vm ioctl. > > > > http://thread.gmane.org/gmane.comp.emulators.qemu/131624 > > > > If the per vcpu ioctl is the right choice I can resend those patches. > > The races are solvable but I think it's easier in userspace. It's also > more flexible, though I don't really see a use for this flexibility. > Okay, I will rebase the per vcpu patches and resend those. Eric signature.asc Description: Digital signature
Re: [PATCH 3/4 V10] Add ioctl for KVMCLOCK_GUEST_STOPPED
On Mon, 30 Jan 2012, Avi Kivity wrote:
> On 01/30/2012 05:07 PM, Avi Kivity wrote:
> > > +Parameters: None
> > > +Returns: 0 on success, -1 on error
> > > +
> > > +This signals to the host kernel that the specified guest is being paused
> > > by
> > > +userspace. The host will set a flag in the pvclock structure that is
> > > checked
> > > +from the soft lockup watchdog. This ioctl can be called during pause or
> > > +unpause.
> > > +
> > > 5. The kvm_run structure
> > >
> > >
> > > +/*
> > > + * kvm_set_guest_paused() indicates to the guest kernel that it has been
> > > + * stopped by the hypervisor. This function will be called from the
> > > host only.
> > > + */
> > > +static int kvm_set_guest_paused(struct kvm *kvm)
> > > +{
> > > + struct kvm_vcpu *vcpu;
> > > + struct pvclock_vcpu_time_info *src;
> > > + int i;
> > > +
> > > + kvm_for_each_vcpu(i, vcpu, kvm) {
> > > + if (!vcpu->arch.time_page)
> > > + continue;
> > > + src = &vcpu->arch.hv_clock;
> > > + src->flags |= PVCLOCK_GUEST_STOPPED;
> >
> > This looks racy. The vcpu can remove its kvmclock concurrently with
> > this access, and src will be NULL.
> >
> > Can you point me to the discussion that moved this to be a vm ioctl? In
> > general vm ioctls that do things for all vcpus are racy, like here.
> > You're accessing variables that are protected by the vcpu mutex, and not
> > taking the mutex (nor can you, since it is held while the guest is
> > running, unlike most kernel mutexes).
> >
>
> Note, the standard way to fix this race is to
> kvm_make_request(KVM_REQ_KVMCLOCK_STOP) and kvm_vcpu_kick(vcpu), and do
> the update in vcpu_enter_guest(). But I think this is needless
> complexity and want to understand what motivated the move to a vm ioctl.
>
I will hold off on fixing this race until we settle the vm or vcpu ioctl
question.
Eric
signature.asc
Description: Digital signature
Re: [PATCH 3/4 V10] Add ioctl for KVMCLOCK_GUEST_STOPPED
On Mon, 30 Jan 2012, Avi Kivity wrote:
> On 01/17/2012 08:40 PM, Eric B Munson wrote:
> > Now that we have a flag that will tell the guest it was suspended, create an
> > interface for that communication using a KVM ioctl.
> >
> >
> > diff --git a/Documentation/virtual/kvm/api.txt
> > b/Documentation/virtual/kvm/api.txt
> > index e1d94bf..1931e5c 100644
> > --- a/Documentation/virtual/kvm/api.txt
> > +++ b/Documentation/virtual/kvm/api.txt
> > @@ -1491,6 +1491,19 @@ following algorithm:
> > Some guests configure the LINT1 NMI input to cause a panic, aiding in
> > debugging.
> >
> > +4.65 KVMCLOCK_GUEST_PAUSED
> > +
> > +Capability: KVM_CAP_GUEST_PAUSED
> > +Architechtures: Any that implement pvclocks (currently x86 only)
> > +Type: vcpu ioctl
>
> vm ioctl.
>
> > +Parameters: None
> > +Returns: 0 on success, -1 on error
> > +
> > +This signals to the host kernel that the specified guest is being paused by
> > +userspace. The host will set a flag in the pvclock structure that is
> > checked
> > +from the soft lockup watchdog. This ioctl can be called during pause or
> > +unpause.
> > +
> > 5. The kvm_run structure
> >
> >
> > +/*
> > + * kvm_set_guest_paused() indicates to the guest kernel that it has been
> > + * stopped by the hypervisor. This function will be called from the host
> > only.
> > + */
> > +static int kvm_set_guest_paused(struct kvm *kvm)
> > +{
> > + struct kvm_vcpu *vcpu;
> > + struct pvclock_vcpu_time_info *src;
> > + int i;
> > +
> > + kvm_for_each_vcpu(i, vcpu, kvm) {
> > + if (!vcpu->arch.time_page)
> > + continue;
> > + src = &vcpu->arch.hv_clock;
> > + src->flags |= PVCLOCK_GUEST_STOPPED;
>
> This looks racy. The vcpu can remove its kvmclock concurrently with
> this access, and src will be NULL.
>
> Can you point me to the discussion that moved this to be a vm ioctl? In
> general vm ioctls that do things for all vcpus are racy, like here.
> You're accessing variables that are protected by the vcpu mutex, and not
> taking the mutex (nor can you, since it is held while the guest is
> running, unlike most kernel mutexes).
>
Jan Kiszka suggested that becuase there isn't a use case for notifying
individual vcpus (can vcpu's be paused individually?) that it makes more sense
to have a vm ioctl.
http://thread.gmane.org/gmane.comp.emulators.qemu/131624
If the per vcpu ioctl is the right choice I can resend those patches.
Eric
signature.asc
Description: Digital signature
Re: [PATCH V7] Guest stop notification
On Tue, 17 Jan 2012, Eric B Munson wrote: > Often when a guest is stopped from the qemu console, it will report spurious > soft lockup warnings on resume. There are kernel patches being discussed that > will give the host the ability to tell the guest that it is being stopped and > should ignore the soft lockup warning that generates. This patch uses the > qemu > Notifier system to tell the guest it is about to be stopped. > > Signed-off-by: Eric B Munson Just poking to make sure this doesn't fall through the cracks. Eric signature.asc Description: Digital signature
Re: [PATCH 0/4 V10] Avoid soft lockup message when KVM is stopped by host
On Tue, 17 Jan 2012, Eric B Munson wrote: > Changes from V9: > Use kvm_for_each_vcpu to iterate online vcpu's > > Changes from V8: > Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu > > Changes from V7: > Define KVM_CAP_GUEST_PAUSED and support check > Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED > > Changes from V6: > Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag > > Changes from V5: > Collapse generic check_and_clear_guest_stopped into patch 2 > Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc > Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking > the > cpuid arg. > Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check > > Changes from V4: > Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED > Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt > > Changes from V3: > Include CC's on patch 3 > Drop clear flag ioctl and have the watchdog clear the flag when it is reset > > Changes from V2: > A new kvm functions defined in kvm_para.h, the only change to pvclock is the > initial flag definition > > Changes from V1: > (Thanks Marcelo) > Host code has all been moved to arch/x86/kvm/x86.c > KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED > > Eric B Munson (4): > Add flag to indicate that a vm was stopped by the host > Add functions to check if the host has stopped the vm > Add ioctl for KVMCLOCK_GUEST_STOPPED > Add check for suspended vm in softlockup detector > > Documentation/virtual/kvm/api.txt | 13 + > arch/ia64/include/asm/kvm_para.h|5 + > arch/powerpc/include/asm/kvm_para.h |5 + > arch/s390/include/asm/kvm_para.h|5 + > arch/x86/include/asm/kvm_para.h |8 > arch/x86/include/asm/pvclock-abi.h |1 + > arch/x86/kernel/kvmclock.c | 21 + > arch/x86/kvm/x86.c | 25 + > include/asm-generic/kvm_para.h | 14 ++ > include/linux/kvm.h |3 +++ > kernel/watchdog.c | 12 > 11 files changed, 112 insertions(+), 0 deletions(-) > create mode 100644 include/asm-generic/kvm_para.h > Just poking to make sure this set hasn't fallen through the cracks. Eric signature.asc Description: Digital signature
[PATCH 4/4 V10] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..91485e5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V10] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V10] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
asm-generic changes Acked-by: Arnd Bergmann
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 50533f9..1f80293 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -169,6 +169,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..bdf6423 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4 V10] Add ioctl for KVMCLOCK_GUEST_STOPPED
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V9:
Use kvm_for_each_vcpu to iterate online vcpu's
Changes from V8:
Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 13 +
arch/x86/kvm/x86.c| 25 +
include/linux/kvm.h |3 +++
3 files changed, 41 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index e1d94bf..1931e5c 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1491,6 +1491,19 @@ following algorithm:
Some guests configure the LINT1 NMI input to cause a panic, aiding in
debugging.
+4.65 KVMCLOCK_GUEST_PAUSED
+
+Capability: KVM_CAP_GUEST_PAUSED
+Architechtures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog. This ioctl can be called during pause or
+unpause.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 14d6cad..1341e3b 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2056,6 +2056,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_X86_ROBUST_SINGLESTEP:
case KVM_CAP_XSAVE:
case KVM_CAP_ASYNC_PF:
+ case KVM_CAP_GUEST_PAUSED:
case KVM_CAP_GET_TSC_KHZ:
r = 1;
break;
@@ -3061,6 +3062,26 @@ out:
return r;
}
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ */
+static int kvm_set_guest_paused(struct kvm *kvm)
+{
+ struct kvm_vcpu *vcpu;
+ struct pvclock_vcpu_time_info *src;
+ int i;
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (!vcpu->arch.time_page)
+ continue;
+ src = &vcpu->arch.hv_clock;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ mark_page_dirty(vcpu->kvm, vcpu->arch.time >> PAGE_SHIFT);
+ }
+ return 0;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -3351,6 +3372,10 @@ long kvm_arch_vm_ioctl(struct file *filp,
r = 0;
break;
}
+ case KVMCLOCK_GUEST_PAUSED: {
+ r = kvm_set_guest_paused(kvm);
+ break;
+ }
default:
;
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index 68e67e5..4ffe0df 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -558,6 +558,7 @@ struct kvm_ppc_pvinfo {
#define KVM_CAP_PPC_PAPR 68
#define KVM_CAP_S390_GMAP 71
#define KVM_CAP_TSC_DEADLINE_TIMER 72
+#define KVM_CAP_GUEST_PAUSED 73
#ifdef KVM_CAP_IRQ_ROUTING
@@ -763,6 +764,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVMCLOCK_GUEST_PAUSED_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V7] Guest stop notification
Often when a guest is stopped from the qemu console, it will report spurious
soft lockup warnings on resume. There are kernel patches being discussed that
will give the host the ability to tell the guest that it is being stopped and
should ignore the soft lockup warning that generates. This patch uses the qemu
Notifier system to tell the guest it is about to be stopped.
Signed-off-by: Eric B Munson
Cc: Avi Kivity
Cc: Marcelo Tosatti
Cc: Jan Kiszka
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: kvm@vger.kernel.org
---
Changes from V6:
Remove unnecessary include
Changes from V5:
KVM_GUEST_PAUSED is now a per vm ioctl instead of per vcpu
Changes from V4:
Test if the guest paused capability is available before use
Changes from V3:
Collapse new state change notification function into existsing function.
Correct whitespace issues
Change ioctl name to KVMCLOCK_GUEST_PAUSED
Use for loop to iterate vpcu's
Changes from V2:
Move ioctl into hw/kvmclock.c so as other arches can use it as it is
implemented
Changes from V1:
Remove unnecessary encapsulating function
hw/kvmclock.c | 11 +++
1 files changed, 11 insertions(+), 0 deletions(-)
diff --git a/hw/kvmclock.c b/hw/kvmclock.c
index 3b9fb20..ad79f52 100644
--- a/hw/kvmclock.c
+++ b/hw/kvmclock.c
@@ -64,10 +64,21 @@ static int kvmclock_post_load(void *opaque, int version_id)
static void kvmclock_vm_state_change(void *opaque, int running,
RunState state)
{
+int ret;
KVMClockState *s = opaque;
+int cap_guest_paused = kvm_check_extension(kvm_state,
KVM_CAP_GUEST_PAUSED);
if (running) {
s->clock_valid = false;
+
+if (!cap_guest_paused) {
+return;
+}
+
+ret = kvm_vm_ioctl(kvm_state, KVMCLOCK_GUEST_PAUSED, 0);
+if (ret) {
+fprintf(stderr, "kvmclock_vm_state_change: %s\n", strerror(-ret));
+}
}
}
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V6] Guest stop notification
Often when a guest is stopped from the qemu console, it will report spurious
soft lockup warnings on resume. There are kernel patches being discussed that
will give the host the ability to tell the guest that it is being stopped and
should ignore the soft lockup warning that generates. This patch uses the qemu
Notifier system to tell the guest it is about to be stopped.
Signed-off-by: Eric B Munson
Cc: Avi Kivity
Cc: Marcelo Tosatti
Cc: Jan Kiszka
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: kvm@vger.kernel.org
---
Changes from V5:
KVM_GUEST_PAUSED is now a per vm ioctl instead of per vcpu
Changes from V4:
Test if the guest paused capability is available before use
Changes from V3:
Collapse new state change notification function into existsing function.
Correct whitespace issues
Change ioctl name to KVMCLOCK_GUEST_PAUSED
Use for loop to iterate vpcu's
Changes from V2:
Move ioctl into hw/kvmclock.c so as other arches can use it as it is
implemented
Changes from V1:
Remove unnecessary encapsulating function
hw/kvmclock.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/hw/kvmclock.c b/hw/kvmclock.c
index 3b9fb20..a30504c 100644
--- a/hw/kvmclock.c
+++ b/hw/kvmclock.c
@@ -18,6 +18,7 @@
#include "sysbus.h"
#include "kvm.h"
#include "kvmclock.h"
+#include "cpu-all.h"
#include
#include
@@ -64,10 +65,21 @@ static int kvmclock_post_load(void *opaque, int version_id)
static void kvmclock_vm_state_change(void *opaque, int running,
RunState state)
{
+int ret;
KVMClockState *s = opaque;
+int cap_guest_paused = kvm_check_extension(kvm_state,
KVM_CAP_GUEST_PAUSED);
if (running) {
s->clock_valid = false;
+
+if (!cap_guest_paused) {
+return;
+}
+
+ret = kvm_vm_ioctl(kvm_state, KVMCLOCK_GUEST_PAUSED, 0);
+if (ret) {
+fprintf(stderr, "kvmclock_vm_state_change: %s\n", strerror(-ret));
+}
}
}
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4 V9] Add ioctl for KVMCLOCK_GUEST_STOPPED
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V8:
Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 13 +
arch/x86/kvm/x86.c| 26 ++
include/linux/kvm.h |3 +++
3 files changed, 42 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index e1d94bf..1931e5c 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1491,6 +1491,19 @@ following algorithm:
Some guests configure the LINT1 NMI input to cause a panic, aiding in
debugging.
+4.65 KVMCLOCK_GUEST_PAUSED
+
+Capability: KVM_CAP_GUEST_PAUSED
+Architechtures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog. This ioctl can be called during pause or
+unpause.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 14d6cad..4661abe 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2056,6 +2056,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_X86_ROBUST_SINGLESTEP:
case KVM_CAP_XSAVE:
case KVM_CAP_ASYNC_PF:
+ case KVM_CAP_GUEST_PAUSED:
case KVM_CAP_GET_TSC_KHZ:
r = 1;
break;
@@ -3061,6 +3062,27 @@ out:
return r;
}
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ */
+static int kvm_set_guest_paused(struct kvm *kvm)
+{
+ struct kvm_vcpu *vcpu;
+ struct pvclock_vcpu_time_info *src;
+ int i;
+
+ for (i = 0; i < atomic_read(&kvm->online_vcpus); i++) {
+ vcpu = kvm->vcpus[i];
+ if (!vcpu->arch.time_page)
+ continue;
+ src = &vcpu->arch.hv_clock;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ mark_page_dirty(vcpu->kvm, vcpu->arch.time >> PAGE_SHIFT);
+ }
+ return 0;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -3351,6 +3373,10 @@ long kvm_arch_vm_ioctl(struct file *filp,
r = 0;
break;
}
+ case KVMCLOCK_GUEST_PAUSED: {
+ r = kvm_set_guest_paused(kvm);
+ break;
+ }
default:
;
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index 68e67e5..4ffe0df 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -558,6 +558,7 @@ struct kvm_ppc_pvinfo {
#define KVM_CAP_PPC_PAPR 68
#define KVM_CAP_S390_GMAP 71
#define KVM_CAP_TSC_DEADLINE_TIMER 72
+#define KVM_CAP_GUEST_PAUSED 73
#ifdef KVM_CAP_IRQ_ROUTING
@@ -763,6 +764,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVMCLOCK_GUEST_PAUSED_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V9] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
asm-generic changes Acked-by: Arnd Bergmann
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 50533f9..1f80293 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -169,6 +169,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..bdf6423 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4 V9] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..91485e5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4 V9] Avoid soft lockup message when KVM is stopped by host
Changes from V8: Make KVM_GUEST_PAUSED a per vm ioctl instead of per vcpu Changes from V7: Define KVM_CAP_GUEST_PAUSED and support check Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: jeremy.fitzhardi...@citrix.com Cc: levinsasha...@gmail.com Cc: Jan Kiszka Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVMCLOCK_GUEST_STOPPED Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 13 + arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 26 ++ include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |3 +++ kernel/watchdog.c | 12 11 files changed, 113 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V9] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4 V8] Add ioctl for KVMCLOCK_GUEST_STOPPED
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V7:
Define KVM_CAP_GUEST_PAUSED and support check
Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 13 +
arch/x86/kvm/x86.c| 21 +
include/linux/kvm.h |3 +++
3 files changed, 37 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index e1d94bf..1931e5c 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1491,6 +1491,19 @@ following algorithm:
Some guests configure the LINT1 NMI input to cause a panic, aiding in
debugging.
+4.65 KVMCLOCK_GUEST_PAUSED
+
+Capability: KVM_CAP_GUEST_PAUSED
+Architechtures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog. This ioctl can be called during pause or
+unpause.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1171def..b0b51cb 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2056,6 +2056,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_X86_ROBUST_SINGLESTEP:
case KVM_CAP_XSAVE:
case KVM_CAP_ASYNC_PF:
+ case KVM_CAP_GUEST_PAUSED:
case KVM_CAP_GET_TSC_KHZ:
r = 1;
break;
@@ -2503,6 +2504,22 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu
*vcpu,
return r;
}
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ mark_page_dirty(vcpu->kvm, vcpu->arch.time >> PAGE_SHIFT);
+ return 0;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -2784,6 +2801,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
goto out;
}
+ case KVMCLOCK_GUEST_PAUSED: {
+ r = kvm_set_guest_paused(vcpu);
+ break;
+ }
default:
r = -EINVAL;
}
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index 68e67e5..4ffe0df 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -558,6 +558,7 @@ struct kvm_ppc_pvinfo {
#define KVM_CAP_PPC_PAPR 68
#define KVM_CAP_S390_GMAP 71
#define KVM_CAP_TSC_DEADLINE_TIMER 72
+#define KVM_CAP_GUEST_PAUSED 73
#ifdef KVM_CAP_IRQ_ROUTING
@@ -763,6 +764,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVMCLOCK_GUEST_PAUSED_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4 V8] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..91485e5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V5] Guest stop notification
Often when a guest is stopped from the qemu console, it will report spurious
soft lockup warnings on resume. There are kernel patches being discussed that
will give the host the ability to tell the guest that it is being stopped and
should ignore the soft lockup warning that generates. This patch uses the qemu
Notifier system to tell the guest it is about to be stopped.
Signed-off-by: Eric B Munson
Cc: Avi Kivity
Cc: Marcelo Tosatti
Cc: Jan Kiszka
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: kvm@vger.kernel.org
---
Changes from V4:
Test if the guest paused capability is available before use
Changes from V3:
Collapse new state change notification function into existsing function.
Correct whitespace issues
Change ioctl name to KVMCLOCK_GUEST_PAUSED
Use for loop to iterate vpcu's
Changes from V2:
Move ioctl into hw/kvmclock.c so as other arches can use it as it is
implemented
Changes from V1:
Remove unnecessary encapsulating function
hw/kvmclock.c | 20
1 files changed, 20 insertions(+), 0 deletions(-)
diff --git a/hw/kvmclock.c b/hw/kvmclock.c
index 5388bc4..d071d61 100644
--- a/hw/kvmclock.c
+++ b/hw/kvmclock.c
@@ -16,6 +16,7 @@
#include "sysbus.h"
#include "kvm.h"
#include "kvmclock.h"
+#include "cpu-all.h"
#include
#include
@@ -62,10 +63,29 @@ static int kvmclock_post_load(void *opaque, int version_id)
static void kvmclock_vm_state_change(void *opaque, int running,
RunState state)
{
+int ret;
+CPUState *penv = first_cpu;
KVMClockState *s = opaque;
+int cap_guest_paused = kvm_check_extension(kvm_state,
KVM_CAP_GUEST_PAUSED);
if (running) {
s->clock_valid = false;
+
+if (!cap_guest_paused) {
+return;
+}
+
+for (penv = first_cpu; penv != NULL; penv = penv->next_cpu) {
+ret = kvm_vcpu_ioctl(penv, KVMCLOCK_GUEST_PAUSED, 0);
+if (ret) {
+if (ret != -EINVAL) {
+fprintf(stderr,
+"kvmclock_vm_state_change: %s\n",
+strerror(-ret));
+}
+return;
+}
+}
}
}
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V8] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4 V8] Avoid soft lockup message when KVM is stopped by host
Changes from V7: Define KVM_CAP_GUEST_PAUSED and support check Call mark_page_dirty () after setting PVCLOCK_GUEST_STOPPED Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: jeremy.fitzhardi...@citrix.com Cc: levinsasha...@gmail.com Cc: Jan Kiszka Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVMCLOCK_GUEST_STOPPED Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 13 + arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 21 + include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |3 +++ kernel/watchdog.c | 12 11 files changed, 108 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V8] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
asm-generic changes Acked-by: Arnd Bergmann
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 50533f9..1f80293 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -169,6 +169,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..bdf6423 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4 V7] Add ioctl for KVMCLOCK_GUEST_STOPPED
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 12
arch/x86/include/asm/kvm_host.h |2 ++
arch/x86/kvm/x86.c| 20
include/linux/kvm.h |2 ++
4 files changed, 36 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 7945b0b..0f7dd99 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1450,6 +1450,18 @@ is supported; 2 if the processor requires all virtual
machines to have
an RMA, or 1 if the processor can use an RMA but doesn't require it,
because it supports the Virtual RMA (VRMA) facility.
+4.64 KVMCLOCK_GUEST_PAUSED
+
+Capability: basic
+Architechtures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index b4973f4..beb94c6 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -672,6 +672,8 @@ int kvm_pv_mmu_op(struct kvm_vcpu *vcpu, unsigned long
bytes,
gpa_t addr, unsigned long *ret);
u8 kvm_get_guest_memory_type(struct kvm_vcpu *vcpu, gfn_t gfn);
+int kvm_set_guest_paused(struct kvm_vcpu *vcpu);
+
extern bool tdp_enabled;
u64 vcpu_tsc_khz(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index c38efd7..1dab5fd 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3295,6 +3295,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
goto out;
}
+ case KVMCLOCK_GUEST_PAUSED: {
+ r = kvm_set_guest_paused(vcpu);
+ break;
+ }
default:
r = -EINVAL;
}
@@ -6117,6 +6121,22 @@ int kvm_task_switch(struct kvm_vcpu *vcpu, u16
tss_selector, int reason,
}
EXPORT_SYMBOL_GPL(kvm_task_switch);
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(kvm_set_guest_paused);
+
int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
struct kvm_sregs *sregs)
{
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index c3892fc..1d1ddef 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -762,6 +762,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVMCLOCK_GUEST_PAUSED_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4 V7] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..91485e5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4 V7] Avoid soft lockup message when KVM is stopped by host
Changes from V6: Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: a...@arndb.de Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: jeremy.fitzhardi...@citrix.com Cc: levinsasha...@gmail.com Cc: Jan Kiszka Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVMCLOCK_GUEST_STOPPED Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 12 arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_host.h |2 ++ arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 20 include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |2 ++ kernel/watchdog.c | 12 12 files changed, 107 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V7] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4 V7] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V6:
Use __this_cpu_and when clearing the PVCLOCK_GUEST_STOPPED flag
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 50533f9..1f80293 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -169,6 +169,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..bdf6423 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ __this_cpu_and(hv_clock.flags, ~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Thu, 15 Dec 2011, Avi Kivity wrote: > On 12/14/2011 07:58 PM, Eric B Munson wrote: > > > I don't think you should see a guest softlockup if the host is > > > overloaded. Nor should you see it due to a long live migration pause, > > > or STOP/CONT. You should see a guest softlockup if it is spinning due > > > to a guest bug, and not for any other reason. > > > > > > I think we need a mix of the ioctl (for STOP/CONT and voluntary pauses, > > > like live migration or qemu stop/cont) and preempt notifiers (together > > > with a check for TASK_RUNNING/TASK_UNINTERRUPTIBLE, for scheduling or > > > swap delays) > > > > > > > Do you want the preemption notifier work in before you will consider merging > > this set, or can that be a follow on? > > Yes please. > > -- > error compiling committee.c: too many arguments to function > I am working on V7 to incorporate the __this_cpu_and suggestion, would you consider that for inclusion and we can continue discussing the need for the preemption notification work? I think that having a guest complain when the host is so loaded that the guest watchdog threads can't make any progress is desirable behavior, it may be the only notification that an admin gets that a particular host is over loaded. Eric signature.asc Description: Digital signature
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Wed, 14 Dec 2011, Avi Kivity wrote: > On 12/08/2011 05:19 PM, Eric B Munson wrote: > > > error compiling committee.c: too many arguments to function > > > > > My concern for preempt notifiers is masking real soft lockup warnings. If > > the > > flag is set every time the vm is preempted, it becomes more likely that we > > will > > mask real warnings. The ioctl was choosen because it sets the flag only > > when > > the guest is being paused deliberately. > > > > AFAIK, SIGSTOP is not a supported way to stop a qemu vm so a soft lockup > > This is a kvm patch, not a qemu patch. In general we try to address > general requirements, not just those that are specific to one userspace, > even if it is near to our hearts. > > > warning would be working as designed there. If that isn't the case, or if > > it > > ever changes, we could always install a signal handler for SIGCONT that set > > the > > flag before resuming the vm. > > > > Scheduling delays are also beyond the scope of this problem and I see the > > soft > > lockup warning as appropriate in that case. > > I don't think you should see a guest softlockup if the host is > overloaded. Nor should you see it due to a long live migration pause, > or STOP/CONT. You should see a guest softlockup if it is spinning due > to a guest bug, and not for any other reason. > > I think we need a mix of the ioctl (for STOP/CONT and voluntary pauses, > like live migration or qemu stop/cont) and preempt notifiers (together > with a check for TASK_RUNNING/TASK_UNINTERRUPTIBLE, for scheduling or > swap delays) > > -- > error compiling committee.c: too many arguments to function > Do you want the preemption notifier work in before you will consider merging this set, or can that be a follow on? Eric signature.asc Description: Digital signature
Re: [PATCH 2/5 V5] Add functions to check if the host has stopped the vm
On Wed, 14 Dec 2011, Avi Kivity wrote: > On 12/14/2011 07:11 PM, Eric B Munson wrote: > > > > > > > > > > If I understand (please correct me if this is wrong) because this is > > > > > only > > > > > called from the watchdog, which disables preemption, we should be > > > > > protected > > > > > from something else writing to these flags. > > > > > > > > The host can write, but in that case race is harmless. > > > > > > Why is it harmless? You don't know what's in those other flags. > > > > > > -- > > > error compiling committee.c: too many arguments to function > > > > > > > Currently there is only one other flag in this byte (PVCLOCK_TSC_STABLE_BIT) > > and it isset once in kvmclock_init(). It is highly unlikely that the vm > > will > > be stopped during this init and have the flag clobbered. After the tsc > > stable > > bit is written in the init, the field is read only outside of the guest > > paused > > code. > > Let's not lay minefields for us later to step into. Making this a > cpu-local atomic (irq safe but not smp safe) is simple and easy, > somehting like __this_cpu_and(). > > -- > error compiling committee.c: too many arguments to function > Fair enough, I will add this to V7. signature.asc Description: Digital signature
Re: [PATCH 2/5 V5] Add functions to check if the host has stopped the vm
On Wed, 14 Dec 2011, Avi Kivity wrote:
> On 12/14/2011 02:11 PM, Marcelo Tosatti wrote:
> > On Thu, Dec 08, 2011 at 10:23:10AM -0500, Eric B Munson wrote:
> > > On Wed, 07 Dec 2011, Avi Kivity wrote:
> > >
> > > > On 12/05/2011 10:19 PM, Eric B Munson wrote:
> > > > > When a host stops or suspends a VM it will set a flag to show this.
> > > > > The
> > > > > watchdog will use these functions to determine if a softlockup is
> > > > > real, or the
> > > > > result of a suspended VM.
> > > > >
> > > > > +bool kvm_check_and_clear_guest_paused(int cpu)
> > > > > +{
> > > > > + bool ret = false;
> > > > > + struct pvclock_vcpu_time_info *src;
> > > > > +
> > > > > + /*
> > > > > + * per_cpu() is safe here because this function is only called
> > > > > from
> > > > > + * timer functions where preemption is already disabled.
> > > > > + */
> > > > > + WARN_ON(!in_atomic());
> > > > > + src = &per_cpu(hv_clock, cpu);
> > > >
> > > > __get_cpu_var(); drop the cpu argument
> > > >
> > >
> > > Will change for V6.
> > >
> > > > > + if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
> > > > > + src->flags = src->flags & (~PVCLOCK_GUEST_STOPPED);
> > > >
> > > > Isn't this racy? Between reading and writing src->flags, we can exit to
> > > > the hypervisor and add/remove new flags. The write then overrides those
> > > > new flags.
> > > >
> > >
> > > If I understand (please correct me if this is wrong) because this is only
> > > called from the watchdog, which disables preemption, we should be
> > > protected
> > > from something else writing to these flags.
> >
> > The host can write, but in that case race is harmless.
>
> Why is it harmless? You don't know what's in those other flags.
>
> --
> error compiling committee.c: too many arguments to function
>
Currently there is only one other flag in this byte (PVCLOCK_TSC_STABLE_BIT)
and it isset once in kvmclock_init(). It is highly unlikely that the vm will
be stopped during this init and have the flag clobbered. After the tsc stable
bit is written in the init, the field is read only outside of the guest paused
code.
Eric
signature.asc
Description: Digital signature
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Wed, 14 Dec 2011, Avi Kivity wrote: > On 12/14/2011 02:16 PM, Marcelo Tosatti wrote: > > > Having this controlled from userspace means it doesn't work for SIGSTOP > > > or for long scheduling delays. What about doing this automatically > > > based on preempt notifiers? > > > > Long scheduling delays should be considered hangups from the guest > > perspective. > > Why? To the guest it looks like slow hardware, but it will interpret it > as a softlockup. > > > About SIGSTOP, that is a corner case. Unsure if its even properly supported > > by QEMU. > > It works from my experience. > > -- > error compiling committee.c: too many arguments to function > And if the soft lockup warning is a concern here, we can have a SIGCONT handler set the flag the same way qemu stop/cont does. signature.asc Description: Digital signature
[PATCH 0/4 V6] Avoid soft lockup message when KVM is stopped by host
Changes from V5: Collapse generic check_and_clear_guest_stopped into patch 2 Include check_and_clear_guest_stopped defintion to ia64, s390, and powerpc Change check_and_clear_guest_stopped to use __get_cpu_var instead of taking the cpuid arg. Protect check_and_clear_guest_stopped declaration with CONFIG_KVM_CLOCK check Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: a...@arndb.de Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: jeremy.fitzhardi...@citrix.com Cc: levinsasha...@gmail.com Cc: Jan Kiszka Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVMCLOCK_GUEST_STOPPED Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 12 arch/ia64/include/asm/kvm_para.h|5 + arch/powerpc/include/asm/kvm_para.h |5 + arch/s390/include/asm/kvm_para.h|5 + arch/x86/include/asm/kvm_host.h |2 ++ arch/x86/include/asm/kvm_para.h |8 arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 20 include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h |2 ++ kernel/watchdog.c | 12 12 files changed, 107 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V5:
Collapse generic stubs into this patch
check_and_clear_guest_stopped() takes no args and uses __get_cpu_var()
Include individual definitions in ia64, s390, and powerpc
arch/ia64/include/asm/kvm_para.h|5 +
arch/powerpc/include/asm/kvm_para.h |5 +
arch/s390/include/asm/kvm_para.h|5 +
arch/x86/include/asm/kvm_para.h |8
arch/x86/kernel/kvmclock.c | 21 +
include/asm-generic/kvm_para.h | 14 ++
6 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/arch/ia64/include/asm/kvm_para.h b/arch/ia64/include/asm/kvm_para.h
index 1588aee..2019cb9 100644
--- a/arch/ia64/include/asm/kvm_para.h
+++ b/arch/ia64/include/asm/kvm_para.h
@@ -26,6 +26,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif
diff --git a/arch/powerpc/include/asm/kvm_para.h
b/arch/powerpc/include/asm/kvm_para.h
index 50533f9..1f80293 100644
--- a/arch/powerpc/include/asm/kvm_para.h
+++ b/arch/powerpc/include/asm/kvm_para.h
@@ -169,6 +169,11 @@ static inline unsigned int kvm_arch_para_features(void)
return r;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif /* __KERNEL__ */
#endif /* __POWERPC_KVM_PARA_H__ */
diff --git a/arch/s390/include/asm/kvm_para.h b/arch/s390/include/asm/kvm_para.h
index 6964db2..a988329 100644
--- a/arch/s390/include/asm/kvm_para.h
+++ b/arch/s390/include/asm/kvm_para.h
@@ -149,6 +149,11 @@ static inline unsigned int kvm_arch_para_features(void)
return 0;
}
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
#endif
#endif /* __S390_KVM_PARA_H */
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..99c4bbe 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,14 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+#ifdef CONFIG_KVM_CLOCK
+bool kvm_check_and_clear_guest_paused(void);
+#else
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+#endif /* CONFIG_KVMCLOCK */
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..33d2e22 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(void)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &__get_cpu_var(hv_clock);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ src->flags = src->flags & (~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..05ef7e7
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(void)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4] Add ioctl for KVMCLOCK_GUEST_STOPPED
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 12
arch/x86/include/asm/kvm_host.h |2 ++
arch/x86/kvm/x86.c| 20
include/linux/kvm.h |2 ++
4 files changed, 36 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 7945b0b..0f7dd99 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1450,6 +1450,18 @@ is supported; 2 if the processor requires all virtual
machines to have
an RMA, or 1 if the processor can use an RMA but doesn't require it,
because it supports the Virtual RMA (VRMA) facility.
+4.64 KVMCLOCK_GUEST_PAUSED
+
+Capability: basic
+Architechtures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index b4973f4..beb94c6 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -672,6 +672,8 @@ int kvm_pv_mmu_op(struct kvm_vcpu *vcpu, unsigned long
bytes,
gpa_t addr, unsigned long *ret);
u8 kvm_get_guest_memory_type(struct kvm_vcpu *vcpu, gfn_t gfn);
+int kvm_set_guest_paused(struct kvm_vcpu *vcpu);
+
extern bool tdp_enabled;
u64 vcpu_tsc_khz(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index c38efd7..1dab5fd 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3295,6 +3295,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
goto out;
}
+ case KVMCLOCK_GUEST_PAUSED: {
+ r = kvm_set_guest_paused(vcpu);
+ break;
+ }
default:
r = -EINVAL;
}
@@ -6117,6 +6121,22 @@ int kvm_task_switch(struct kvm_vcpu *vcpu, u16
tss_selector, int reason,
}
EXPORT_SYMBOL_GPL(kvm_task_switch);
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(kvm_set_guest_paused);
+
int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
struct kvm_sregs *sregs)
{
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index c3892fc..1d1ddef 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -762,6 +762,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVMCLOCK_GUEST_PAUSED_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..91485e5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused();
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused())
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4 V6] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 4/5 V5] Add generic stubs for kvm stop check functions
On Wed, 07 Dec 2011, Avi Kivity wrote:
> On 12/05/2011 10:19 PM, Eric B Munson wrote:
> >
> > diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
> > new file mode 100644
> > index 000..177e1eb
> > --- /dev/null
> > +++ b/include/asm-generic/kvm_para.h
> > @@ -0,0 +1,14 @@
> > +#ifndef _ASM_GENERIC_KVM_PARA_H
> > +#define _ASM_GENERIC_KVM_PARA_H
> > +
> > +
> > +/*
> > + * This function is used by architectures that support kvm to avoid issuing
> > + * false soft lockup messages.
> > + */
> > +static inline bool kvm_check_and_clear_guest_paused(int cpu)
> > +{
> > + return false;
> > +}
> > +
> > +#endif
>
> Please fold into patch 2.
Will do for V6.
>
> What about arch/{ia64,s390,powerpc}/include/asm/kvm_para.h?
>
I will add the above to those as well.
> --
> error compiling committee.c: too many arguments to function
>
signature.asc
Description: Digital signature
Re: [PATCH 3/5 V5] Add ioctl for KVMCLOCK_GUEST_STOPPED
On Wed, 07 Dec 2011, Avi Kivity wrote:
> On 12/05/2011 10:19 PM, Eric B Munson wrote:
> > Now that we have a flag that will tell the guest it was suspended, create an
> > interface for that communication using a KVM ioctl.
> >
> > @@ -3295,6 +3295,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
> >
> > goto out;
> > }
> > + case KVMCLOCK_GUEST_PAUSED: {
> > + r = kvm_set_guest_paused(vcpu);
> > + break;
> > + }
> > default:
> > r = -EINVAL;
> > }
> >
>
> You could also do this purely from userspace by reading the kvmclock msr
> and updating it. However, it's better to do this in the kernel to avoid
> distributing responsibility for kvmclock across too many cooks.
>
> --
> error compiling committee.c: too many arguments to function
>
As stated in the cover letter, per Marcelo's suggestion I already tried this
method and it resulted in the guest wall clock skewing by the amount of time
the guest was suspended. This places a new burden on the guest to be running
ntpd or equivslent to keep the wall clock correct.
Eric
signature.asc
Description: Digital signature
Re: [PATCH 2/5 V5] Add functions to check if the host has stopped the vm
On Wed, 07 Dec 2011, Avi Kivity wrote:
> On 12/05/2011 10:19 PM, Eric B Munson wrote:
> > When a host stops or suspends a VM it will set a flag to show this. The
> > watchdog will use these functions to determine if a softlockup is real, or
> > the
> > result of a suspended VM.
> >
> > +bool kvm_check_and_clear_guest_paused(int cpu)
> > +{
> > + bool ret = false;
> > + struct pvclock_vcpu_time_info *src;
> > +
> > + /*
> > +* per_cpu() is safe here because this function is only called from
> > +* timer functions where preemption is already disabled.
> > +*/
> > + WARN_ON(!in_atomic());
> > + src = &per_cpu(hv_clock, cpu);
>
> __get_cpu_var(); drop the cpu argument
>
Will change for V6.
> > + if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
> > + src->flags = src->flags & (~PVCLOCK_GUEST_STOPPED);
>
> Isn't this racy? Between reading and writing src->flags, we can exit to
> the hypervisor and add/remove new flags. The write then overrides those
> new flags.
>
If I understand (please correct me if this is wrong) because this is only
called from the watchdog, which disables preemption, we should be protected
from something else writing to these flags.
> > + ret = true;
> > + }
> > +
> > + return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
> > +
> > static struct clocksource kvm_clock = {
> > .name = "kvm-clock",
> > .read = kvm_clock_get_cycles,
>
>
> --
> error compiling committee.c: too many arguments to function
>
signature.asc
Description: Digital signature
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Wed, 07 Dec 2011, Avi Kivity wrote: > On 12/05/2011 10:18 PM, Eric B Munson wrote: > > Changes from V4: > > Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED > > Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt > > > > Changes from V3: > > Include CC's on patch 3 > > Drop clear flag ioctl and have the watchdog clear the flag when it is reset > > > > Changes from V2: > > A new kvm functions defined in kvm_para.h, the only change to pvclock is the > > initial flag definition > > > > Changes from V1: > > (Thanks Marcelo) > > Host code has all been moved to arch/x86/kvm/x86.c > > KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED > > > > When a guest kernel is stopped by the host hypervisor it can look like a > > soft > > lockup to the guest kernel. This false warning can mask later soft lockup > > warnings which may be real. This patch series adds a method for a host > > hypervisor to communicate to a guest kernel that it is being stopped. The > > final patch in the series has the watchdog check this flag when it goes to > > issue a soft lockup warning and skip the warning if the guest knows it was > > stopped. > > > > It was attempted to solve this in Qemu, but the side effects of saving and > > restoring the clock and tsc for each vcpu put the wall clock of the guest > > behind > > by the amount of time of the pause. This forces a guest to have ntp running > > in order to keep the wall clock accurate. > > Having this controlled from userspace means it doesn't work for SIGSTOP > or for long scheduling delays. What about doing this automatically > based on preempt notifiers? > > > -- > error compiling committee.c: too many arguments to function > My concern for preempt notifiers is masking real soft lockup warnings. If the flag is set every time the vm is preempted, it becomes more likely that we will mask real warnings. The ioctl was choosen because it sets the flag only when the guest is being paused deliberately. AFAIK, SIGSTOP is not a supported way to stop a qemu vm so a soft lockup warning would be working as designed there. If that isn't the case, or if it ever changes, we could always install a signal handler for SIGCONT that set the flag before resuming the vm. Scheduling delays are also beyond the scope of this problem and I see the soft lockup warning as appropriate in that case. Eric signature.asc Description: Digital signature
[PATCH 3/5 V5] Add ioctl for KVMCLOCK_GUEST_STOPPED
Now that we have a flag that will tell the guest it was suspended, create an
interface for that communication using a KVM ioctl.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: levinsasha...@gmail.com
Cc: Jan Kiszka
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V4:
Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED
Add new ioctl description to api.txt
Documentation/virtual/kvm/api.txt | 12
arch/x86/include/asm/kvm_host.h |2 ++
arch/x86/kvm/x86.c| 20
include/linux/kvm.h |2 ++
4 files changed, 36 insertions(+), 0 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 7945b0b..0f7dd99 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1450,6 +1450,18 @@ is supported; 2 if the processor requires all virtual
machines to have
an RMA, or 1 if the processor can use an RMA but doesn't require it,
because it supports the Virtual RMA (VRMA) facility.
+4.64 KVMCLOCK_GUEST_PAUSED
+
+Capability: basic
+Architechtures: Any that implement pvclocks (currently x86 only)
+Type: vcpu ioctl
+Parameters: None
+Returns: 0 on success, -1 on error
+
+This signals to the host kernel that the specified guest is being paused by
+userspace. The host will set a flag in the pvclock structure that is checked
+from the soft lockup watchdog.
+
5. The kvm_run structure
Application code obtains a pointer to the kvm_run structure by
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index b4973f4..beb94c6 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -672,6 +672,8 @@ int kvm_pv_mmu_op(struct kvm_vcpu *vcpu, unsigned long
bytes,
gpa_t addr, unsigned long *ret);
u8 kvm_get_guest_memory_type(struct kvm_vcpu *vcpu, gfn_t gfn);
+int kvm_set_guest_paused(struct kvm_vcpu *vcpu);
+
extern bool tdp_enabled;
u64 vcpu_tsc_khz(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index c38efd7..1dab5fd 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3295,6 +3295,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
goto out;
}
+ case KVMCLOCK_GUEST_PAUSED: {
+ r = kvm_set_guest_paused(vcpu);
+ break;
+ }
default:
r = -EINVAL;
}
@@ -6117,6 +6121,22 @@ int kvm_task_switch(struct kvm_vcpu *vcpu, u16
tss_selector, int reason,
}
EXPORT_SYMBOL_GPL(kvm_task_switch);
+/*
+ * kvm_set_guest_paused() indicates to the guest kernel that it has been
+ * stopped by the hypervisor. This function will be called from the host only.
+ * EINVAL is returned when the host attempts to set the flag for a guest that
+ * does not support pv clocks.
+ */
+int kvm_set_guest_paused(struct kvm_vcpu *vcpu)
+{
+ struct pvclock_vcpu_time_info *src = &vcpu->arch.hv_clock;
+ if (!vcpu->arch.time_page)
+ return -EINVAL;
+ src->flags |= PVCLOCK_GUEST_STOPPED;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(kvm_set_guest_paused);
+
int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
struct kvm_sregs *sregs)
{
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index c3892fc..1d1ddef 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -762,6 +762,8 @@ struct kvm_clock_data {
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct
kvm_create_spapr_tce)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
+/* VM is being stopped by host */
+#define KVMCLOCK_GUEST_PAUSED_IO(KVMIO, 0xaa)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/5 V5] Add generic stubs for kvm stop check functions
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: levinsasha...@gmail.com
Cc: Jan Kiszka
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
include/asm-generic/kvm_para.h | 14 ++
1 files changed, 14 insertions(+), 0 deletions(-)
create mode 100644 include/asm-generic/kvm_para.h
diff --git a/include/asm-generic/kvm_para.h b/include/asm-generic/kvm_para.h
new file mode 100644
index 000..177e1eb
--- /dev/null
+++ b/include/asm-generic/kvm_para.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_GENERIC_KVM_PARA_H
+#define _ASM_GENERIC_KVM_PARA_H
+
+
+/*
+ * This function is used by architectures that support kvm to avoid issuing
+ * false soft lockup messages.
+ */
+static inline bool kvm_check_and_clear_guest_paused(int cpu)
+{
+ return false;
+}
+
+#endif
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 5/5 V5] Add check for suspended vm in softlockup detector
A suspended VM can cause spurious soft lockup warnings. To avoid these, the
watchdog now checks if the kernel knows it was stopped by the host and skips
the warning if so. When the watchdog is reset successfully, clear the guest
paused flag.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: levinsasha...@gmail.com
Cc: Jan Kiszka
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
Changes from V3:
Clear the PAUSED flag when the watchdog is reset
kernel/watchdog.c | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 1d7bca7..7c62919 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -25,6 +25,7 @@
#include
#include
+#include
#include
int watchdog_enabled = 1;
@@ -280,6 +281,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
__this_cpu_write(softlockup_touch_sync, false);
sched_clock_tick();
}
+
+ /* Clear the guest paused flag on watchdog reset */
+ kvm_check_and_clear_guest_paused(smp_processor_id());
__touch_watchdog();
return HRTIMER_RESTART;
}
@@ -292,6 +296,14 @@ static enum hrtimer_restart watchdog_timer_fn(struct
hrtimer *hrtimer)
*/
duration = is_softlockup(touch_ts);
if (unlikely(duration)) {
+ /*
+* If a virtual machine is stopped by the host it can look to
+* the watchdog like a soft lockup, check to see if the host
+* stopped the vm before we issue the warning
+*/
+ if (kvm_check_and_clear_guest_paused(smp_processor_id()))
+ return HRTIMER_RESTART;
+
/* only warn once */
if (__this_cpu_read(soft_watchdog_warn) == true)
return HRTIMER_RESTART;
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: a...@arndb.de Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: jeremy.fitzhardi...@citrix.com Cc: levinsasha...@gmail.com Cc: Jan Kiszka Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (5): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVMCLOCK_GUEST_STOPPED Add generic stubs for kvm stop check functions Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 12 arch/x86/include/asm/kvm_host.h|2 ++ arch/x86/include/asm/kvm_para.h|1 + arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 20 include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h|2 ++ kernel/watchdog.c | 12 9 files changed, 85 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/5 V5] Add functions to check if the host has stopped the vm
When a host stops or suspends a VM it will set a flag to show this. The
watchdog will use these functions to determine if a softlockup is real, or the
result of a suspended VM.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: levinsasha...@gmail.com
Cc: Jan Kiszka
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/kvm_para.h |1 +
arch/x86/kernel/kvmclock.c | 21 +
2 files changed, 22 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 734c376..e9d63a6 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -95,6 +95,7 @@ struct kvm_vcpu_pv_apf_data {
extern void kvmclock_init(void);
extern int kvm_register_clock(char *txt);
+bool kvm_check_and_clear_guest_paused(int cpu);
/* This instruction is vmcall. On non-VT architectures, it will generate a
* trap that we will then rewrite to the appropriate instruction.
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 44842d7..f0c0599 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include
#include
@@ -114,6 +115,26 @@ static void kvm_get_preset_lpj(void)
preset_lpj = lpj;
}
+bool kvm_check_and_clear_guest_paused(int cpu)
+{
+ bool ret = false;
+ struct pvclock_vcpu_time_info *src;
+
+ /*
+* per_cpu() is safe here because this function is only called from
+* timer functions where preemption is already disabled.
+*/
+ WARN_ON(!in_atomic());
+ src = &per_cpu(hv_clock, cpu);
+ if ((src->flags & PVCLOCK_GUEST_STOPPED) != 0) {
+ src->flags = src->flags & (~PVCLOCK_GUEST_STOPPED);
+ ret = true;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_check_and_clear_guest_paused);
+
static struct clocksource kvm_clock = {
.name = "kvm-clock",
.read = kvm_clock_get_cycles,
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/5 V5] Add flag to indicate that a vm was stopped by the host
This flag will be used to check if the vm was stopped by the host when a soft
lockup was detected. The host will set the flag when it stops the guest. On
resume, the guest will check this flag if a soft lockup is detected and skip
issuing the warning.
Signed-off-by: Eric B Munson
Cc: mi...@redhat.com
Cc: h...@zytor.com
Cc: a...@arndb.de
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: mtosa...@redhat.com
Cc: jeremy.fitzhardi...@citrix.com
Cc: levinsasha...@gmail.com
Cc: Jan Kiszka
Cc: kvm@vger.kernel.org
Cc: linux-a...@vger.kernel.org
Cc: x...@kernel.org
Cc: linux-ker...@vger.kernel.org
---
arch/x86/include/asm/pvclock-abi.h |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/pvclock-abi.h
b/arch/x86/include/asm/pvclock-abi.h
index 35f2d19..6167fd7 100644
--- a/arch/x86/include/asm/pvclock-abi.h
+++ b/arch/x86/include/asm/pvclock-abi.h
@@ -40,5 +40,6 @@ struct pvclock_wall_clock {
} __attribute__((__packed__));
#define PVCLOCK_TSC_STABLE_BIT (1 << 0)
+#define PVCLOCK_GUEST_STOPPED (1 << 1)
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PVCLOCK_ABI_H */
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V4] Guest stop notification
Often when a guest is stopped from the qemu console, it will report spurious
soft lockup warnings on resume. There are kernel patches being discussed that
will give the host the ability to tell the guest that it is being stopped and
should ignore the soft lockup warning that generates. This patch uses the qemu
Notifier system to tell the guest it is about to be stopped.
Signed-off-by: Eric B Munson
Cc: Avi Kivity
Cc: Marcelo Tosatti
Cc: Jan Kiszka
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: kvm@vger.kernel.org
---
Changes from V3:
Collapse new state change notification function into existsing function.
Correct whitespace issues
Change ioctl name to KVMCLOCK_GUEST_PAUSED
Use for loop to iterate vpcu's
Changes from V2:
Move ioctl into hw/kvmclock.c so as other arches can use it as it is
implemented
Changes from V1:
Remove unnecessary encapsulating function
hw/kvmclock.c | 15 +++
1 files changed, 15 insertions(+), 0 deletions(-)
diff --git a/hw/kvmclock.c b/hw/kvmclock.c
index 5388bc4..fa11dd7 100644
--- a/hw/kvmclock.c
+++ b/hw/kvmclock.c
@@ -16,6 +16,7 @@
#include "sysbus.h"
#include "kvm.h"
#include "kvmclock.h"
+#include "cpu-all.h"
#include
#include
@@ -62,10 +63,24 @@ static int kvmclock_post_load(void *opaque, int version_id)
static void kvmclock_vm_state_change(void *opaque, int running,
RunState state)
{
+int ret;
+CPUState *penv = first_cpu;
KVMClockState *s = opaque;
if (running) {
s->clock_valid = false;
+
+for (penv = first_cpu; penv != NULL; penv = penv->next_cpu) {
+ret = kvm_vcpu_ioctl(penv, KVMCLOCK_GUEST_PAUSED, 0);
+if (ret) {
+if (ret != -EINVAL) {
+fprintf(stderr,
+"kvmclock_vm_state_change: %s\n",
+strerror(-ret));
+}
+return;
+}
+}
}
}
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/5 V4] Add ioctl for KVM_GUEST_STOPPED
On Sat, 03 Dec 2011, Sasha Levin wrote: > On Tue, 2011-11-29 at 16:35 -0500, Eric B Munson wrote: > > > > Now that we have a flag that will tell the guest it was suspended, > > create an interface for that communication using a KVM ioctl. > > > > Signed-off-by: Eric B Munson > > Can it be documented in api.txt as well? > > -- > > Sasha. > Thanks for the review, will do for V5. Eric signature.asc Description: Digital signature
Re: [PATCH V3] Guest stop notification
On Sat, 03 Dec 2011, Jan Kiszka wrote:
> On 2011-12-02 22:27, Eric B Munson wrote:
> > On Fri, 02 Dec 2011, Jan Kiszka wrote:
> >
> >> On 2011-12-02 20:19, Eric B Munson wrote:
> >>> Often when a guest is stopped from the qemu console, it will report
> >>> spurious
> >>> soft lockup warnings on resume. There are kernel patches being discussed
> >>> that
> >>> will give the host the ability to tell the guest that it is being stopped
> >>> and
> >>> should ignore the soft lockup warning that generates.
> >>>
> >>> Signed-off-by: Eric B Munson
> >>> Cc: Avi Kivity
> >>> Cc: Marcelo Tosatti
> >>> Cc: Jan Kiszka
> >>> Cc: ry...@linux.vnet.ibm.com
> >>> Cc: aligu...@us.ibm.com
> >>> Cc: kvm@vger.kernel.org
> >>>
> >>> ---
> >>> Changes from V2:
> >>> Move ioctl into hw/kvmclock.c so as other arches can use it as it is
> >>> implemented
> >>>
> >>> Changes from V1:
> >>> Remove unnecessary encapsulating function
> >>>
> >>> hw/kvmclock.c | 24
> >>> 1 files changed, 24 insertions(+), 0 deletions(-)
> >>>
> >>> diff --git a/hw/kvmclock.c b/hw/kvmclock.c
> >>> index 5388bc4..756839f 100644
> >>> --- a/hw/kvmclock.c
> >>> +++ b/hw/kvmclock.c
> >>> @@ -16,6 +16,7 @@
> >>> #include "sysbus.h"
> >>> #include "kvm.h"
> >>> #include "kvmclock.h"
> >>> +#include "cpu-all.h"
> >>>
> >>> #include
> >>> #include
> >>> @@ -69,11 +70,34 @@ static void kvmclock_vm_state_change(void *opaque,
> >>> int running,
> >>> }
> >>> }
> >>>
> >>> +static void kvmclock_vm_state_change_vcpu(void *opaque, int running,
> >>> + RunState state)
> >>> +{
> >>> +int ret;
> >>> +CPUState *penv = first_cpu;
> >>> +
> >>> +if (running) {
> >>> + while (penv) {
> >>
> >> or: for (cpu = first_cpu; cpu != NULL; cpu = cpu->next_cpu) {
> >>
> >
> > Functionally equivalent and I see both in the code, is there a standard?
>
> Not really. I once tried to introduce an iterator macro, but it was
> refused. The above is just more compact.
>
> But this is only a minor nit.
>
Fair enough, since there will be a V4 I will switch to the for loop.
> >
> >>> +ret = kvm_vcpu_ioctl(penv, KVM_GUEST_PAUSED, 0);
> >>> +if (ret) {
> >>> +if (ret != ENOSYS) {
> >>> +fprintf(stderr,
> >>> +"kvmclock_vm_state_change_vcpu: %s\n",
> >>> +strerror(-ret));
> >>> +}
> >>> +return;
> >>> +}
> >>> +penv = (CPUState *)penv->next_cpu;
> >>
> >> Unneeded cast.
> >>
> >
> > Also following an example seen elsewhere.
>
> Generally, we try to avoid those pointless casts.
>
Will remove for V4.
> >
> >>> +}
> >>> +}
> >>> +}
> >>> +
> >>
> >> Again: please use checkpatch.pl.
> >>
> >
> > Sorry, tough to get used to hitting space bar that many times...
> >
> >>> static int kvmclock_init(SysBusDevice *dev)
> >>> {
> >>> KVMClockState *s = FROM_SYSBUS(KVMClockState, dev);
> >>>
> >>> qemu_add_vm_change_state_handler(kvmclock_vm_state_change, s);
> >>> +qemu_add_vm_change_state_handler(kvmclock_vm_state_change_vcpu,
> >>> NULL);
> >>> return 0;
> >>> }
> >>>
> >>
> >> Why not extend the existing handler?
> >
> > Because the new handler doesn't touch the KVMClockState object. If this is
> > preferred, I have no objection.
>
> The separate registration looks strange to me. And the fact that you
> don't need to object doesn't justify a callback of its own.
>
I think you misunderstood me, I meant I have no object to doign it your way if
you have a strong opinion (as it seems you do).
> >
> >>
> >> I still wonder if the IOCTL interface is actually kvmclock specific. But
> >> Marcello asked for this, and we could still change it when some arch
> >> comes around that provides it independent of kvmclock.
> >
> > The flag itself is stored in the pvclock_vcpu_time_info structure, and
> > anything
> > else that touches that structure uses ioctls.
>
> That's the host-guest interface. But I'm talking about the kvm-qemu
> interface here which has no relation to how the "was paused" information
> is transferred to the guest.
>
> Jan
>
signature.asc
Description: Digital signature
Re: [PATCH V3] Guest stop notification
On Fri, 02 Dec 2011, Jan Kiszka wrote:
> On 2011-12-02 20:19, Eric B Munson wrote:
> > Often when a guest is stopped from the qemu console, it will report spurious
> > soft lockup warnings on resume. There are kernel patches being discussed
> > that
> > will give the host the ability to tell the guest that it is being stopped
> > and
> > should ignore the soft lockup warning that generates.
> >
> > Signed-off-by: Eric B Munson
> > Cc: Avi Kivity
> > Cc: Marcelo Tosatti
> > Cc: Jan Kiszka
> > Cc: ry...@linux.vnet.ibm.com
> > Cc: aligu...@us.ibm.com
> > Cc: kvm@vger.kernel.org
> >
> > ---
> > Changes from V2:
> > Move ioctl into hw/kvmclock.c so as other arches can use it as it is
> > implemented
> >
> > Changes from V1:
> > Remove unnecessary encapsulating function
> >
> > hw/kvmclock.c | 24
> > 1 files changed, 24 insertions(+), 0 deletions(-)
> >
> > diff --git a/hw/kvmclock.c b/hw/kvmclock.c
> > index 5388bc4..756839f 100644
> > --- a/hw/kvmclock.c
> > +++ b/hw/kvmclock.c
> > @@ -16,6 +16,7 @@
> > #include "sysbus.h"
> > #include "kvm.h"
> > #include "kvmclock.h"
> > +#include "cpu-all.h"
> >
> > #include
> > #include
> > @@ -69,11 +70,34 @@ static void kvmclock_vm_state_change(void *opaque, int
> > running,
> > }
> > }
> >
> > +static void kvmclock_vm_state_change_vcpu(void *opaque, int running,
> > + RunState state)
> > +{
> > +int ret;
> > +CPUState *penv = first_cpu;
> > +
> > +if (running) {
> > + while (penv) {
>
> or: for (cpu = first_cpu; cpu != NULL; cpu = cpu->next_cpu) {
>
Functionally equivalent and I see both in the code, is there a standard?
> > +ret = kvm_vcpu_ioctl(penv, KVM_GUEST_PAUSED, 0);
> > +if (ret) {
> > +if (ret != ENOSYS) {
> > +fprintf(stderr,
> > +"kvmclock_vm_state_change_vcpu: %s\n",
> > +strerror(-ret));
> > +}
> > +return;
> > +}
> > +penv = (CPUState *)penv->next_cpu;
>
> Unneeded cast.
>
Also following an example seen elsewhere.
> > +}
> > +}
> > +}
> > +
>
> Again: please use checkpatch.pl.
>
Sorry, tough to get used to hitting space bar that many times...
> > static int kvmclock_init(SysBusDevice *dev)
> > {
> > KVMClockState *s = FROM_SYSBUS(KVMClockState, dev);
> >
> > qemu_add_vm_change_state_handler(kvmclock_vm_state_change, s);
> > +qemu_add_vm_change_state_handler(kvmclock_vm_state_change_vcpu, NULL);
> > return 0;
> > }
> >
>
> Why not extend the existing handler?
Because the new handler doesn't touch the KVMClockState object. If this is
preferred, I have no objection.
>
> I still wonder if the IOCTL interface is actually kvmclock specific. But
> Marcello asked for this, and we could still change it when some arch
> comes around that provides it independent of kvmclock.
The flag itself is stored in the pvclock_vcpu_time_info structure, and anything
else that touches that structure uses ioctls.
>
> Jan
>
signature.asc
Description: Digital signature
[PATCH V3] Guest stop notification
Often when a guest is stopped from the qemu console, it will report spurious
soft lockup warnings on resume. There are kernel patches being discussed that
will give the host the ability to tell the guest that it is being stopped and
should ignore the soft lockup warning that generates.
Signed-off-by: Eric B Munson
Cc: Avi Kivity
Cc: Marcelo Tosatti
Cc: Jan Kiszka
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: kvm@vger.kernel.org
---
Changes from V2:
Move ioctl into hw/kvmclock.c so as other arches can use it as it is
implemented
Changes from V1:
Remove unnecessary encapsulating function
hw/kvmclock.c | 24
1 files changed, 24 insertions(+), 0 deletions(-)
diff --git a/hw/kvmclock.c b/hw/kvmclock.c
index 5388bc4..756839f 100644
--- a/hw/kvmclock.c
+++ b/hw/kvmclock.c
@@ -16,6 +16,7 @@
#include "sysbus.h"
#include "kvm.h"
#include "kvmclock.h"
+#include "cpu-all.h"
#include
#include
@@ -69,11 +70,34 @@ static void kvmclock_vm_state_change(void *opaque, int
running,
}
}
+static void kvmclock_vm_state_change_vcpu(void *opaque, int running,
+ RunState state)
+{
+int ret;
+CPUState *penv = first_cpu;
+
+if (running) {
+ while (penv) {
+ret = kvm_vcpu_ioctl(penv, KVM_GUEST_PAUSED, 0);
+if (ret) {
+if (ret != ENOSYS) {
+fprintf(stderr,
+"kvmclock_vm_state_change_vcpu: %s\n",
+strerror(-ret));
+}
+return;
+}
+penv = (CPUState *)penv->next_cpu;
+}
+}
+}
+
static int kvmclock_init(SysBusDevice *dev)
{
KVMClockState *s = FROM_SYSBUS(KVMClockState, dev);
qemu_add_vm_change_state_handler(kvmclock_vm_state_change, s);
+qemu_add_vm_change_state_handler(kvmclock_vm_state_change_vcpu, NULL);
return 0;
}
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Guest stop notification
On Thu, 01 Dec 2011, Marcelo Tosatti wrote:
> On Thu, Dec 01, 2011 at 06:36:17PM +0100, Jan Kiszka wrote:
> > On 2011-12-01 18:22, Eric B Munson wrote:
> > > On Thu, 01 Dec 2011, Jan Kiszka wrote:
> > >
> > >> On 2011-11-29 22:36, Eric B Munson wrote:
> > >>> Often when a guest is stopped from the qemu console, it will report
> > >>> spurious
> > >>> soft lockup warnings on resume. There are kernel patches being
> > >>> discussed that
> > >>> will give the host the ability to tell the guest that it is being
> > >>> stopped and
> > >>> should ignore the soft lockup warning that generates.
> > >>>
> > >>> Signed-off-by: Eric B Munson
> > >>> Cc: ry...@linux.vnet.ibm.com
> > >>> Cc: aligu...@us.ibm.com
> > >>> Cc: mtosa...@redhat.com
> > >>> Cc: a...@redhat.com
> > >>> Cc: kvm@vger.kernel.org
> > >>> Cc: linux-ker...@vger.kernel.org
> > >>> ---
> > >>> target-i386/kvm.c |6 ++
> > >>> 1 files changed, 6 insertions(+), 0 deletions(-)
> > >>>
> > >>> diff --git a/target-i386/kvm.c b/target-i386/kvm.c
> > >>> index 5bfc21f..defd364 100644
> > >>> --- a/target-i386/kvm.c
> > >>> +++ b/target-i386/kvm.c
> > >>> @@ -336,12 +336,18 @@ static int kvm_inject_mce_oldstyle(CPUState *env)
> > >>> return 0;
> > >>> }
> > >>>
> > >>> +static void kvm_put_guest_paused(CPUState *penv)
> > >>> +{
> > >>> +kvm_vcpu_ioctl(penv, KVM_GUEST_PAUSED, 0);
> > >>> +}
> > >>
> > >> I see no need in encapsulating this in a separate function.
> > >>
> > >>> +
> > >>> static void cpu_update_state(void *opaque, int running, RunState state)
> > >>> {
> > >>> CPUState *env = opaque;
> > >>>
> > >>> if (running) {
> > >>> env->tsc_valid = false;
> > >>> + kvm_put_guest_paused(env);
> > >>
> > >> checkpatch.pl would have asked you to remove this tab.
> > >>
> > >> More general:
> > >>
> > >> Why is this x86-only? If the kernel interface is x86-only, what prevents
> > >> making it generic right from the beginning?
> > >
> > > Sorry, missed this question on the first pass, this is x86 only because
> > > the
> > > flag used lives in the pvclock structure. AFAICT, there aren't any other
> > > architectures out there that implement paravirtualized clocks yet.
> >
> > That's an implementation "detail" of the kernel. The interface (IOCTL or
> > kvm_run field) is generic, no?
> >
> > I would just fire this notification from generic code, evaluate the
> > error (that was lacking so far), and only report it if it's something
> > else than "not supported".
>
> Yes, it should live in hw/kvmclock.c preferably.
>
Okay, I get a V3 with this moved around out tomorrow.
Thanks for the feedback,
Eric
signature.asc
Description: Digital signature
[PATCH V2] Guest stop notification
Often when a guest is stopped from the qemu console, it will report spurious
soft lockup warnings on resume. There are kernel patches being discussed that
will give the host the ability to tell the guest that it is being stopped and
should ignore the soft lockup warning that generates. This patch uses the qemu
Notifier system to tell the guest it is about to be stopped.
Signed-off-by: Eric B Munson
Cc: Avi Kivity
Cc: Marcelo Tosatti
Cc: Jan Kiszka
Cc: ry...@linux.vnet.ibm.com
Cc: aligu...@us.ibm.com
Cc: linux-ker...@vger.kernel.org
Cc: kvm@vger.kernel.org
---
target-i386/kvm.c |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 5bfc21f..12ac91a 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -342,6 +342,7 @@ static void cpu_update_state(void *opaque, int running,
RunState state)
if (running) {
env->tsc_valid = false;
+kvm_vcpu_ioctl(env, KVM_GUEST_PAUSED, 0);
}
}
--
1.7.5.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
