From: Thomas Gleixner <[email protected]> Indirect Branch Speculation (IBS) is controlled per physical core. If one thread disables it then it's disabled for the core. If a thread enters idle it makes sense to reenable IBS so the sibling thread can run with full speculation enabled in user space.
This makes only sense in mwait_idle_with_hints() because mwait_idle() can serve an interrupt immediately before speculation can be stopped again. SKL which requires IBRS should use mwait_idle_with_hints() so this is a non issue and in the worst case a missed optimization. Originally-by: Tim Chen <[email protected]> Signed-off-by: Thomas Gleixner <[email protected]> --- arch/x86/include/asm/mwait.h | 14 ++++++++++++++ arch/x86/kernel/process.c | 14 ++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/arch/x86/include/asm/mwait.h b/arch/x86/include/asm/mwait.h index 39a2fb2..f173072 100644 --- a/arch/x86/include/asm/mwait.h +++ b/arch/x86/include/asm/mwait.h @@ -6,6 +6,7 @@ #include <linux/sched/idle.h> #include <asm/cpufeature.h> +#include <asm/nospec-branch.h> #define MWAIT_SUBSTATE_MASK 0xf #define MWAIT_CSTATE_MASK 0xf @@ -106,7 +107,20 @@ static inline void mwait_idle_with_hints(unsigned long eax, unsigned long ecx) mb(); } + /* + * Indirect Branch Speculation (IBS) is controlled per + * physical core. If one thread disables it, then it's + * disabled on all threads of the core. The kernel disables + * it on entry from user space. Reenable it on the thread + * which goes idle so the other thread has a chance to run + * with full speculation enabled in userspace. + */ + unrestrict_branch_speculation(); __monitor((void *)¤t_thread_info()->flags, 0, 0); + /* + * Restrict IBS again to protect kernel execution. + */ + restrict_branch_speculation(); if (!need_resched()) __mwait(eax, ecx); } diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 3cb2486..f941c5d 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -461,6 +461,20 @@ static __cpuidle void mwait_idle(void) mb(); /* quirk */ } + /* + * Indirect Branch Speculation (IBS) is controlled per + * physical core. If one thread disables it, then it's + * disabled on all threads of the core. The kernel disables + * it on entry from user space. For __sti_mwait() it's + * wrong to reenable it because an interrupt can be served + * before speculation can be stopped again. + * + * To plug that hole the interrupt entry code would need to + * save current state and restore. Not worth the trouble as + * SKL should not use mwait_idle(). It should use + * mwait_idle_with_hints() which can do speculation control + * safely. + */ __monitor((void *)¤t_thread_info()->flags, 0, 0); if (!need_resched()) __sti_mwait(0, 0); -- 2.7.4
