[v3 PATCH 2/4] powernv:smp: Add busy-wait loop as fall back for CPU-Hotplug

2017-03-22 Thread Gautham R. Shenoy 
From: "Gautham R. Shenoy" 

Currently, the powernv cpu-offline function assumes that platform idle
states such as stop on POWER9, winkle/sleep/nap on POWER8 are always
available. On POWER8, it picks nap as the default state if other deep
idle states like sleep/winkle are not available and enabled in the
platform.

On POWER9, nap is not available and all idle states are managed by
STOP instruction.  The parameters to the idle state are passed through
processor stop status control register (PSSCR).  Hence as such
executing STOP would take parameters from current PSSCR. We do not
want to make any assumptions in kernel on what STOP states and PSSCR
features are configured by the platform.

Ideally platform will configure a good set of stop states that can be
used in the kernel.  We would like to start with a clean slate, if the
platform choose to not configure any state or there is an error in
platform firmware that lead to no stop states being configured or
allowed to be requested.

This patch adds a fallback method for CPU-Hotplug that is similar to
snooze loop at idle where the threads are left to spin at low priority
and hence reduce the cycles consumed.

This is a safe fallback mechanism in the case when no stop state would
be requested if the platform firmware did not configure them most
likely due to an error condition.

Requesting a stop state when the platform has not configured them or
enabled them would lead to further error conditions which could be
difficult to debug.

[Changelog written with inputs from sva...@linux.vnet.ibm.com]
Reviewed-by: Nicholas Piggin 
Signed-off-by: Gautham R. Shenoy 
---
 arch/powerpc/platforms/powernv/idle.c | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 419edff..f335e0f 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -283,8 +283,16 @@ unsigned long pnv_cpu_offline(unsigned int cpu)
} else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
   (idle_states & OPAL_PM_SLEEP_ENABLED_ER1)) {
srr1 = power7_sleep();
-   } else {
+   } else if (idle_states & OPAL_PM_NAP_ENABLED) {
srr1 = power7_nap(1);
+   } else {
+   /* This is the fallback method. We emulate snooze */
+   while (!generic_check_cpu_restart(cpu)) {
+   HMT_low();
+   HMT_very_low();
+   }
+   srr1 = 0;
+   HMT_medium();
}
 
return srr1;
-- 
1.9.4

Re: [v3 PATCH 2/4] powernv:smp: Add busy-wait loop as fall back for CPU-Hotplug

2017-03-27 Thread Michael Ellerman 
"Gautham R. Shenoy"  writes:

> diff --git a/arch/powerpc/platforms/powernv/idle.c 
> b/arch/powerpc/platforms/powernv/idle.c
> index 419edff..f335e0f 100644
> --- a/arch/powerpc/platforms/powernv/idle.c
> +++ b/arch/powerpc/platforms/powernv/idle.c
> @@ -283,8 +283,16 @@ unsigned long pnv_cpu_offline(unsigned int cpu)
>   } else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
>  (idle_states & OPAL_PM_SLEEP_ENABLED_ER1)) {
>   srr1 = power7_sleep();
> - } else {
> + } else if (idle_states & OPAL_PM_NAP_ENABLED) {
>   srr1 = power7_nap(1);
> + } else {
> + /* This is the fallback method. We emulate snooze */
> + while (!generic_check_cpu_restart(cpu)) {

Breaks the SMP=n build :/

arch/powerpc/platforms/powernv/idle.c:299:11: error: implicit declaration of 
function 'generic_check_cpu_restart' [-Werror=implicit-function-declaration]

cheers

Re: [v3 PATCH 2/4] powernv:smp: Add busy-wait loop as fall back for CPU-Hotplug

2017-03-30 Thread Gautham R Shenoy 
On Mon, Mar 27, 2017 at 10:43:44PM +1100, Michael Ellerman wrote:
> "Gautham R. Shenoy"  writes:
> 
> > diff --git a/arch/powerpc/platforms/powernv/idle.c 
> > b/arch/powerpc/platforms/powernv/idle.c
> > index 419edff..f335e0f 100644
> > --- a/arch/powerpc/platforms/powernv/idle.c
> > +++ b/arch/powerpc/platforms/powernv/idle.c
> > @@ -283,8 +283,16 @@ unsigned long pnv_cpu_offline(unsigned int cpu)
> > } else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
> >(idle_states & OPAL_PM_SLEEP_ENABLED_ER1)) {
> > srr1 = power7_sleep();
> > -   } else {
> > +   } else if (idle_states & OPAL_PM_NAP_ENABLED) {
> > srr1 = power7_nap(1);
> > +   } else {
> > +   /* This is the fallback method. We emulate snooze */
> > +   while (!generic_check_cpu_restart(cpu)) {
> 
> Breaks the SMP=n build :/

Hmmm.. Looks like I have missed adding a CONFIG_HOTPLUG_CPU somewhere.
> 
> arch/powerpc/platforms/powernv/idle.c:299:11: error: implicit declaration of 
> function 'generic_check_cpu_restart' [-Werror=implicit-function-declaration]
>

Could you add the following patch as the PATCH 1 in the series instead
of what was present in this series ? It adds a CONFIG_HOTPLUG_CPU
around pnv_cpu_offline(). I have compiled it with both all
combinations of CONFIG_HOTPLUG_CPU and CONFIG_SMP and it builds fine.


x-x-x---
powernv: Move CPU-Offline idle state invocation from smp.c to idle.c

Move the piece of code in powernv/smp.c::pnv_smp_cpu_kill_self() which
transitions the CPU to the deepest available platform idle state to a
new function named pnv_cpu_offline() in powernv/idle.c. The rationale
behind this code movement is that the data required to determine the
deepest available platform state resides in powernv/idle.c.

Reviewed-by: Nicholas Piggin 
Signed-off-by: Gautham R. Shenoy 
---
Restricts pnv_cpu_offline() to CONFIG_HOTPLUG_CPU

 arch/powerpc/include/asm/cpuidle.h   |  1 +
 arch/powerpc/platforms/powernv/idle.c| 27 +++
 arch/powerpc/platforms/powernv/powernv.h |  2 --
 arch/powerpc/platforms/powernv/smp.c | 18 ++
 4 files changed, 30 insertions(+), 18 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h 
b/arch/powerpc/include/asm/cpuidle.h
index 1557315..4649ca0 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -46,6 +46,7 @@
 
 extern u64 pnv_first_deep_stop_state;
 
+unsigned long pnv_cpu_offline(unsigned int cpu);
 int validate_psscr_val_mask(u64 *psscr_val, u64 *psscr_mask, u32 flags);
 static inline void report_invalid_psscr_val(u64 psscr_val, int err)
 {
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 4ee837e..fafafa3 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -265,6 +265,33 @@ static void power9_idle(void)
 u64 pnv_deepest_stop_psscr_val;
 u64 pnv_deepest_stop_psscr_mask;
 
+#ifdef CONFIG_HOTPLUG_CPU
+/*
+ * pnv_cpu_offline: A function that puts the CPU into the deepest
+ * available platform idle state on a CPU-Offline.
+ */
+unsigned long pnv_cpu_offline(unsigned int cpu)
+{
+   unsigned long srr1;
+
+   u32 idle_states = pnv_get_supported_cpuidle_states();
+
+   if (cpu_has_feature(CPU_FTR_ARCH_300)) {
+   srr1 = power9_idle_stop(pnv_deepest_stop_psscr_val,
+   pnv_deepest_stop_psscr_mask);
+   } else if (idle_states & OPAL_PM_WINKLE_ENABLED) {
+   srr1 = power7_winkle();
+   } else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
+  (idle_states & OPAL_PM_SLEEP_ENABLED_ER1)) {
+   srr1 = power7_sleep();
+   } else {
+   srr1 = power7_nap(1);
+   }
+
+   return srr1;
+}
+#endif
+
 /*
  * Power ISA 3.0 idle initialization.
  *
diff --git a/arch/powerpc/platforms/powernv/powernv.h 
b/arch/powerpc/platforms/powernv/powernv.h
index 6130522..6dbc0a1 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -18,8 +18,6 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
-extern u64 pnv_deepest_stop_psscr_val;
-extern u64 pnv_deepest_stop_psscr_mask;
 
 extern void pnv_lpc_init(void);
 
diff --git a/arch/powerpc/platforms/powernv/smp.c 
b/arch/powerpc/platforms/powernv/smp.c
index 8b67e1e..914b456 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -35,6 +35,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "powernv.h"
 
@@ -140,7 +141,6 @@ static void pnv_smp_cpu_kill_self(void)
 {
unsigned int cpu;
unsigned long srr1, wmask;
-   u32 idle_states;
 
/* Standard hot unplug procedure */
local_irq_disable();
@@ -155,8 +155,6 @@ static void pnv_smp_cpu_kill_self(void)
if (cpu_has