RE: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors
-Original Message- From: Vitaly Kuznetsov [mailto:vkuzn...@redhat.com] Sent: Monday, January 26, 2015 2:39 AM To: Greg Kroah-Hartman Cc: KY Srinivasan; de...@linuxdriverproject.org; Haiyang Zhang; linux- ker...@vger.kernel.org; Dexuan Cui Subject: Re: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors Greg Kroah-Hartman gre...@linuxfoundation.org writes: On Mon, Jan 12, 2015 at 05:50:11PM +0100, Vitaly Kuznetsov wrote: When an SMP Hyper-V guest is running on top of 2012R2 Server and secondary cpus are sent offline (with echo 0 /sys/devices/system/cpu/cpu$cpu/online) the system freeze is observed. This happens due to the fact that on newer hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c) and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining when vmbus is loaded until the issue is fixed host-side. This patch also disables hibernation but it is OK as it is also broken (MCE error is hit on resume). Suspend still works. Tested with WS2008R2 and WS2012R2. Signed-off-by: Vitaly Kuznetsov vkuzn...@redhat.com Signed-off-by: K. Y. Srinivasan k...@microsoft.com --- Changes since v2: - repair the build when vmbus is builded as a module [Greg KH] by saving current cpu_disable pointer to previous_cpu_disable and restoring it on unload; - return -ENOSYS (same as native_cpu_disable when !CONFIG_HOTPLUG_CPU) instead of -1 in hyperv_cpu_disable(). Changes since v1: - introduce hv_cpu_hotplug_quirk() function to not spread #ifdefs [Greg KH]; - add pr_notice() message hv_vmbus: CPU offlining is not supported by hypervisor. --- drivers/hv/vmbus_drv.c | 36 1 file changed, 36 insertions(+) Doesn't apply to my char-misc-test branch at all :( Another mid-air collision with K.Y's Drivers: hv: vmbus: Implement a clockevent device, please use the attached version. No functional changes are required, I just fixed the merge conflict (includes). Othere than that (and sorry for meddling), would it it be better if you switch to 'pull requests' workflow with K.Y? There is a lot of ongoing work in hyperv nowdays and such collisions seem otherwise inevitable ... I will co-ordinate and forward Hyper-V patches to Greg. K. Y -- Vitaly ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors
Greg Kroah-Hartman gre...@linuxfoundation.org writes:
On Mon, Jan 12, 2015 at 05:50:11PM +0100, Vitaly Kuznetsov wrote:
When an SMP Hyper-V guest is running on top of 2012R2 Server and secondary
cpus are sent offline (with echo 0 /sys/devices/system/cpu/cpu$cpu/online)
the system freeze is observed. This happens due to the fact that on newer
hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed
across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c)
and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining
when vmbus is loaded until the issue is fixed host-side.
This patch also disables hibernation but it is OK as it is also broken (MCE
error is hit on resume). Suspend still works.
Tested with WS2008R2 and WS2012R2.
Signed-off-by: Vitaly Kuznetsov vkuzn...@redhat.com
Signed-off-by: K. Y. Srinivasan k...@microsoft.com
---
Changes since v2:
- repair the build when vmbus is builded as a module [Greg KH] by saving
current cpu_disable pointer to previous_cpu_disable and restoring it on
unload;
- return -ENOSYS (same as native_cpu_disable when !CONFIG_HOTPLUG_CPU)
instead
of -1 in hyperv_cpu_disable().
Changes since v1:
- introduce hv_cpu_hotplug_quirk() function to not spread #ifdefs [Greg KH];
- add pr_notice() message hv_vmbus: CPU offlining is not supported by
hypervisor.
---
drivers/hv/vmbus_drv.c | 36
1 file changed, 36 insertions(+)
Doesn't apply to my char-misc-test branch at all :(
Another mid-air collision with K.Y's Drivers: hv: vmbus: Implement a
clockevent device, please use the attached version. No functional
changes are required, I just fixed the merge conflict (includes).
Othere than that (and sorry for meddling), would it it be better if
you switch to 'pull requests' workflow with K.Y? There is a lot of
ongoing work in hyperv nowdays and such collisions seem otherwise
inevitable ...
--
Vitaly
From 95f9b0ff3f73c3bbe8aa9c525414097c2c26a3ef Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov vkuzn...@redhat.com
Date: Mon, 12 Jan 2015 17:50:11 +0100
Subject: [PATCH] Drivers: hv: vmbus: prevent cpu offlining on newer
hypervisors
When an SMP Hyper-V guest is running on top of 2012R2 Server and secondary
cpus are sent offline (with echo 0 /sys/devices/system/cpu/cpu$cpu/online)
the system freeze is observed. This happens due to the fact that on newer
hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed
across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c)
and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining
when vmbus is loaded until the issue is fixed host-side.
This patch also disables hibernation but it is OK as it is also broken (MCE
error is hit on resume). Suspend still works.
Tested with WS2008R2 and WS2012R2.
Signed-off-by: Vitaly Kuznetsov vkuzn...@redhat.com
Signed-off-by: K. Y. Srinivasan k...@microsoft.com
---
drivers/hv/vmbus_drv.c | 36
1 file changed, 36 insertions(+)
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 35e3f42..90c3400 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -33,6 +33,7 @@
#include linux/hyperv.h
#include linux/kernel_stat.h
#include linux/clockchips.h
+#include linux/cpu.h
#include asm/hyperv.h
#include asm/hypervisor.h
#include asm/mshyperv.h
@@ -704,6 +705,39 @@ static void vmbus_isr(void)
}
}
+#ifdef CONFIG_HOTPLUG_CPU
+static int hyperv_cpu_disable(void)
+{
+ return -ENOSYS;
+}
+
+static void hv_cpu_hotplug_quirk(bool vmbus_loaded)
+{
+ static void *previous_cpu_disable;
+
+ /*
+ * Offlining a CPU when running on newer hypervisors (WS2012R2, Win8,
+ * ...) is not supported at this moment as channel interrupts are
+ * distributed across all of them.
+ */
+
+ if ((vmbus_proto_version == VERSION_WS2008) ||
+ (vmbus_proto_version == VERSION_WIN7))
+ return;
+
+ if (vmbus_loaded) {
+ previous_cpu_disable = smp_ops.cpu_disable;
+ smp_ops.cpu_disable = hyperv_cpu_disable;
+ pr_notice(CPU offlining is not supported by hypervisor\n);
+ } else if (previous_cpu_disable)
+ smp_ops.cpu_disable = previous_cpu_disable;
+}
+#else
+static void hv_cpu_hotplug_quirk(bool vmbus_loaded)
+{
+}
+#endif
+
/*
* vmbus_bus_init -Main vmbus driver initialization routine.
*
@@ -744,6 +778,7 @@ static int vmbus_bus_init(int irq)
if (ret)
goto err_alloc;
+ hv_cpu_hotplug_quirk(true);
vmbus_request_offers();
return 0;
@@ -997,6 +1032,7 @@ static void __exit vmbus_exit(void)
bus_unregister(hv_bus);
hv_cleanup();
acpi_bus_unregister_driver(vmbus_acpi_driver);
+ hv_cpu_hotplug_quirk(false);
}
--
1.9.3
___
devel mailing list
de...@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors
On Mon, Jan 26, 2015 at 11:38:54AM +0100, Vitaly Kuznetsov wrote: Greg Kroah-Hartman gre...@linuxfoundation.org writes: On Mon, Jan 12, 2015 at 05:50:11PM +0100, Vitaly Kuznetsov wrote: When an SMP Hyper-V guest is running on top of 2012R2 Server and secondary cpus are sent offline (with echo 0 /sys/devices/system/cpu/cpu$cpu/online) the system freeze is observed. This happens due to the fact that on newer hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c) and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining when vmbus is loaded until the issue is fixed host-side. This patch also disables hibernation but it is OK as it is also broken (MCE error is hit on resume). Suspend still works. Tested with WS2008R2 and WS2012R2. Signed-off-by: Vitaly Kuznetsov vkuzn...@redhat.com Signed-off-by: K. Y. Srinivasan k...@microsoft.com --- Changes since v2: - repair the build when vmbus is builded as a module [Greg KH] by saving current cpu_disable pointer to previous_cpu_disable and restoring it on unload; - return -ENOSYS (same as native_cpu_disable when !CONFIG_HOTPLUG_CPU) instead of -1 in hyperv_cpu_disable(). Changes since v1: - introduce hv_cpu_hotplug_quirk() function to not spread #ifdefs [Greg KH]; - add pr_notice() message hv_vmbus: CPU offlining is not supported by hypervisor. --- drivers/hv/vmbus_drv.c | 36 1 file changed, 36 insertions(+) Doesn't apply to my char-misc-test branch at all :( Another mid-air collision with K.Y's Drivers: hv: vmbus: Implement a clockevent device, please use the attached version. No functional changes are required, I just fixed the merge conflict (includes). Othere than that (and sorry for meddling), would it it be better if you switch to 'pull requests' workflow with K.Y? There is a lot of ongoing work in hyperv nowdays and such collisions seem otherwise inevitable ... This is a trivial number of patches for this subsystem, so no, a git workflow isn't needed. But if KY would collect the patches up and send them to me for all hyperv patches from now on, that would make these types of issues go away, KY, can you do that from now on? thanks, greg k-h ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
RE: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors
-Original Message- From: Greg Kroah-Hartman [mailto:gre...@linuxfoundation.org] Sent: Monday, January 26, 2015 2:54 PM To: Vitaly Kuznetsov Cc: KY Srinivasan; de...@linuxdriverproject.org; Haiyang Zhang; linux- ker...@vger.kernel.org; Dexuan Cui Subject: Re: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors On Mon, Jan 26, 2015 at 11:38:54AM +0100, Vitaly Kuznetsov wrote: Greg Kroah-Hartman gre...@linuxfoundation.org writes: On Mon, Jan 12, 2015 at 05:50:11PM +0100, Vitaly Kuznetsov wrote: When an SMP Hyper-V guest is running on top of 2012R2 Server and secondary cpus are sent offline (with echo 0 /sys/devices/system/cpu/cpu$cpu/online) the system freeze is observed. This happens due to the fact that on newer hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c) and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining when vmbus is loaded until the issue is fixed host-side. This patch also disables hibernation but it is OK as it is also broken (MCE error is hit on resume). Suspend still works. Tested with WS2008R2 and WS2012R2. Signed-off-by: Vitaly Kuznetsov vkuzn...@redhat.com Signed-off-by: K. Y. Srinivasan k...@microsoft.com --- Changes since v2: - repair the build when vmbus is builded as a module [Greg KH] by saving current cpu_disable pointer to previous_cpu_disable and restoring it on unload; - return -ENOSYS (same as native_cpu_disable when !CONFIG_HOTPLUG_CPU) instead of -1 in hyperv_cpu_disable(). Changes since v1: - introduce hv_cpu_hotplug_quirk() function to not spread #ifdefs [Greg KH]; - add pr_notice() message hv_vmbus: CPU offlining is not supported by hypervisor. --- drivers/hv/vmbus_drv.c | 36 1 file changed, 36 insertions(+) Doesn't apply to my char-misc-test branch at all :( Another mid-air collision with K.Y's Drivers: hv: vmbus: Implement a clockevent device, please use the attached version. No functional changes are required, I just fixed the merge conflict (includes). Othere than that (and sorry for meddling), would it it be better if you switch to 'pull requests' workflow with K.Y? There is a lot of ongoing work in hyperv nowdays and such collisions seem otherwise inevitable ... This is a trivial number of patches for this subsystem, so no, a git workflow isn't needed. But if KY would collect the patches up and send them to me for all hyperv patches from now on, that would make these types of issues go away, KY, can you do that from now on? Will do. Thanks, K. Y thanks, greg k-h ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors
On Mon, Jan 12, 2015 at 05:50:11PM +0100, Vitaly Kuznetsov wrote: When an SMP Hyper-V guest is running on top of 2012R2 Server and secondary cpus are sent offline (with echo 0 /sys/devices/system/cpu/cpu$cpu/online) the system freeze is observed. This happens due to the fact that on newer hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c) and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining when vmbus is loaded until the issue is fixed host-side. This patch also disables hibernation but it is OK as it is also broken (MCE error is hit on resume). Suspend still works. Tested with WS2008R2 and WS2012R2. Signed-off-by: Vitaly Kuznetsov vkuzn...@redhat.com Signed-off-by: K. Y. Srinivasan k...@microsoft.com --- Changes since v2: - repair the build when vmbus is builded as a module [Greg KH] by saving current cpu_disable pointer to previous_cpu_disable and restoring it on unload; - return -ENOSYS (same as native_cpu_disable when !CONFIG_HOTPLUG_CPU) instead of -1 in hyperv_cpu_disable(). Changes since v1: - introduce hv_cpu_hotplug_quirk() function to not spread #ifdefs [Greg KH]; - add pr_notice() message hv_vmbus: CPU offlining is not supported by hypervisor. --- drivers/hv/vmbus_drv.c | 36 1 file changed, 36 insertions(+) Doesn't apply to my char-misc-test branch at all :( ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
RE: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors
-Original Message-
From: Vitaly Kuznetsov [mailto:vkuzn...@redhat.com]
Sent: Monday, January 12, 2015 8:50 AM
To: KY Srinivasan; de...@linuxdriverproject.org
Cc: Haiyang Zhang; Greg Kroah-Hartman; linux-ker...@vger.kernel.org;
Dexuan Cui
Subject: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer
hypervisors
When an SMP Hyper-V guest is running on top of 2012R2 Server and
secondary
cpus are sent offline (with echo 0
/sys/devices/system/cpu/cpu$cpu/online)
the system freeze is observed. This happens due to the fact that on newer
hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed
across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c)
and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining
when vmbus is loaded until the issue is fixed host-side.
This patch also disables hibernation but it is OK as it is also broken (MCE
error is hit on resume). Suspend still works.
Tested with WS2008R2 and WS2012R2.
Signed-off-by: Vitaly Kuznetsov vkuzn...@redhat.com
Thank you.
Signed-off-by: K. Y. Srinivasan k...@microsoft.com
---
Changes since v2:
- repair the build when vmbus is builded as a module [Greg KH] by saving
current cpu_disable pointer to previous_cpu_disable and restoring it on
unload;
- return -ENOSYS (same as native_cpu_disable when
!CONFIG_HOTPLUG_CPU) instead
of -1 in hyperv_cpu_disable().
Changes since v1:
- introduce hv_cpu_hotplug_quirk() function to not spread #ifdefs [Greg
KH];
- add pr_notice() message hv_vmbus: CPU offlining is not supported by
hypervisor.
---
drivers/hv/vmbus_drv.c | 36
1 file changed, 36 insertions(+)
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 4d6b269..233da0b 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -32,6 +32,7 @@
#include linux/completion.h
#include linux/hyperv.h
#include linux/kernel_stat.h
+#include linux/cpu.h
#include asm/hyperv.h
#include asm/hypervisor.h
#include asm/mshyperv.h
@@ -671,6 +672,39 @@ static void vmbus_isr(void)
tasklet_schedule(msg_dpc);
}
+#ifdef CONFIG_HOTPLUG_CPU
+static int hyperv_cpu_disable(void)
+{
+ return -ENOSYS;
+}
+
+static void hv_cpu_hotplug_quirk(bool vmbus_loaded)
+{
+ static void *previous_cpu_disable;
+
+ /*
+ * Offlining a CPU when running on newer hypervisors (WS2012R2,
Win8,
+ * ...) is not supported at this moment as channel interrupts are
+ * distributed across all of them.
+ */
+
+ if ((vmbus_proto_version == VERSION_WS2008) ||
+ (vmbus_proto_version == VERSION_WIN7))
+ return;
+
+ if (vmbus_loaded) {
+ previous_cpu_disable = smp_ops.cpu_disable;
+ smp_ops.cpu_disable = hyperv_cpu_disable;
+ pr_notice(CPU offlining is not supported by hypervisor\n);
+ } else if (previous_cpu_disable)
+ smp_ops.cpu_disable = previous_cpu_disable;
+}
+#else
+static void hv_cpu_hotplug_quirk(bool vmbus_loaded)
+{
+}
+#endif
+
/*
* vmbus_bus_init -Main vmbus driver initialization routine.
*
@@ -711,6 +745,7 @@ static int vmbus_bus_init(int irq)
if (ret)
goto err_alloc;
+ hv_cpu_hotplug_quirk(true);
vmbus_request_offers();
return 0;
@@ -964,6 +999,7 @@ static void __exit vmbus_exit(void)
bus_unregister(hv_bus);
hv_cleanup();
acpi_bus_unregister_driver(vmbus_acpi_driver);
+ hv_cpu_hotplug_quirk(false);
}
--
1.9.3
___
devel mailing list
de...@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
