[RFC] mm/hugetlb: use mem policy when allocating surplus huge pages
Application allocating overcommitted hugepages behave differently when its mempolicy is set to bind with NUMA nodes containing CPUs and not containing CPUs. When memory is allocated on node with CPUs everything work as expected, when memory is allocated on CPU-less node: 1. Some memory is allocated from node with CPUs. 2. Application is terminated with SIGBUS (due to touching not allocated page). Reproduction (Node0: 90GB, 272 logical CPUs; Node1: 16GB, No CPUs): int main() { char *p = (char*)mmap(0, 4*1024*1024, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB, 0, 0); *p = 0; p += 2*1024*1024; *p=0; return 0; } echo 2 > /proc/sys/vm/nr_overcommit_hugepages numactl -m 0 ./test #works numactl -m 1 ./test #sigbus The reason for this behavior is hugetlb_reserve_pages(...) omits struct vm_area when calling hugetlb_acct_pages(..) and later allocation is unable to determine memory policy. To fix this issue memory policy is forwarded from hugetlb_reserved_pages to allocation routine. When policy is interleave, NUMA Node is computed by: page address >> huge_page_shift() % interleaved nodes count. This algorithm assumes that address is known, but in this case address is not known so to keep interleave working without it, dummy address is computed as vm_start + (1 << huge_page_shift())*n, where n is allocated page number. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- mm/hugetlb.c | 49 +++-- 1 file changed, 35 insertions(+), 14 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 418bf01..3913066 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -67,7 +67,8 @@ static int num_fault_mutexes; struct mutex *hugetlb_fault_mutex_table cacheline_aligned_in_smp; /* Forward declaration */ -static int hugetlb_acct_memory(struct hstate *h, long delta); +static int hugetlb_acct_memory(struct hstate *h, long delta, + struct vm_area_struct *vma); static inline void unlock_or_release_subpool(struct hugepage_subpool *spool) { @@ -81,7 +82,7 @@ static inline void unlock_or_release_subpool(struct hugepage_subpool *spool) if (free) { if (spool->min_hpages != -1) hugetlb_acct_memory(spool->hstate, - -spool->min_hpages); + -spool->min_hpages, NULL); kfree(spool); } } @@ -101,7 +102,7 @@ struct hugepage_subpool *hugepage_new_subpool(struct hstate *h, long max_hpages, spool->hstate = h; spool->min_hpages = min_hpages; - if (min_hpages != -1 && hugetlb_acct_memory(h, min_hpages)) { + if (min_hpages != -1 && hugetlb_acct_memory(h, min_hpages, NULL)) { kfree(spool); return NULL; } @@ -576,7 +577,7 @@ void hugetlb_fix_reserve_counts(struct inode *inode) if (rsv_adjust) { struct hstate *h = hstate_inode(inode); - hugetlb_acct_memory(h, 1); + hugetlb_acct_memory(h, 1, NULL); } } @@ -1690,10 +1691,12 @@ struct page *alloc_huge_page_node(struct hstate *h, int nid) * Increase the hugetlb pool such that it can accommodate a reservation * of size 'delta'. */ -static int gather_surplus_pages(struct hstate *h, int delta) +static int gather_surplus_pages(struct hstate *h, int delta, + struct vm_area_struct *vma) { struct list_head surplus_list; struct page *page, *tmp; + unsigned long address_offset = 0; int ret, i; int needed, allocated; bool alloc_ok = true; @@ -1711,7 +1714,20 @@ static int gather_surplus_pages(struct hstate *h, int delta) retry: spin_unlock(_lock); for (i = 0; i < needed; i++) { - page = __alloc_buddy_huge_page_no_mpol(h, NUMA_NO_NODE); + if (vma) { + unsigned long dummy_addr = vma->vm_start + + (address_offset << huge_page_shift(h)); + + if (dummy_addr >= vma->vm_end) { + address_offset = 0; + dummy_addr = vma->vm_start; + } + page = __alloc_buddy_huge_page_with_mpol(h, vma, +dummy_addr); + address_offset++; + } else { + page = __alloc_buddy_huge_page_no_mpol(h, NUMA_NO_NODE); + } if (!page) { alloc_ok = false; break; @@ -2057,7 +2073,7 @@ struct page *alloc_huge_page(struct vm_area_struct *vma, long rsv_adjust; rsv_adjust = hugepage_sub
[RFC] mm/hugetlb: use mem policy when allocating surplus huge pages
Application allocating overcommitted hugepages behave differently when its mempolicy is set to bind with NUMA nodes containing CPUs and not containing CPUs. When memory is allocated on node with CPUs everything work as expected, when memory is allocated on CPU-less node: 1. Some memory is allocated from node with CPUs. 2. Application is terminated with SIGBUS (due to touching not allocated page). Reproduction (Node0: 90GB, 272 logical CPUs; Node1: 16GB, No CPUs): int main() { char *p = (char*)mmap(0, 4*1024*1024, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB, 0, 0); *p = 0; p += 2*1024*1024; *p=0; return 0; } echo 2 > /proc/sys/vm/nr_overcommit_hugepages numactl -m 0 ./test #works numactl -m 1 ./test #sigbus The reason for this behavior is hugetlb_reserve_pages(...) omits struct vm_area when calling hugetlb_acct_pages(..) and later allocation is unable to determine memory policy. To fix this issue memory policy is forwarded from hugetlb_reserved_pages to allocation routine. When policy is interleave, NUMA Node is computed by: page address >> huge_page_shift() % interleaved nodes count. This algorithm assumes that address is known, but in this case address is not known so to keep interleave working without it, dummy address is computed as vm_start + (1 << huge_page_shift())*n, where n is allocated page number. Signed-off-by: Grzegorz Andrejczuk --- mm/hugetlb.c | 49 +++-- 1 file changed, 35 insertions(+), 14 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 418bf01..3913066 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -67,7 +67,8 @@ static int num_fault_mutexes; struct mutex *hugetlb_fault_mutex_table cacheline_aligned_in_smp; /* Forward declaration */ -static int hugetlb_acct_memory(struct hstate *h, long delta); +static int hugetlb_acct_memory(struct hstate *h, long delta, + struct vm_area_struct *vma); static inline void unlock_or_release_subpool(struct hugepage_subpool *spool) { @@ -81,7 +82,7 @@ static inline void unlock_or_release_subpool(struct hugepage_subpool *spool) if (free) { if (spool->min_hpages != -1) hugetlb_acct_memory(spool->hstate, - -spool->min_hpages); + -spool->min_hpages, NULL); kfree(spool); } } @@ -101,7 +102,7 @@ struct hugepage_subpool *hugepage_new_subpool(struct hstate *h, long max_hpages, spool->hstate = h; spool->min_hpages = min_hpages; - if (min_hpages != -1 && hugetlb_acct_memory(h, min_hpages)) { + if (min_hpages != -1 && hugetlb_acct_memory(h, min_hpages, NULL)) { kfree(spool); return NULL; } @@ -576,7 +577,7 @@ void hugetlb_fix_reserve_counts(struct inode *inode) if (rsv_adjust) { struct hstate *h = hstate_inode(inode); - hugetlb_acct_memory(h, 1); + hugetlb_acct_memory(h, 1, NULL); } } @@ -1690,10 +1691,12 @@ struct page *alloc_huge_page_node(struct hstate *h, int nid) * Increase the hugetlb pool such that it can accommodate a reservation * of size 'delta'. */ -static int gather_surplus_pages(struct hstate *h, int delta) +static int gather_surplus_pages(struct hstate *h, int delta, + struct vm_area_struct *vma) { struct list_head surplus_list; struct page *page, *tmp; + unsigned long address_offset = 0; int ret, i; int needed, allocated; bool alloc_ok = true; @@ -1711,7 +1714,20 @@ static int gather_surplus_pages(struct hstate *h, int delta) retry: spin_unlock(_lock); for (i = 0; i < needed; i++) { - page = __alloc_buddy_huge_page_no_mpol(h, NUMA_NO_NODE); + if (vma) { + unsigned long dummy_addr = vma->vm_start + + (address_offset << huge_page_shift(h)); + + if (dummy_addr >= vma->vm_end) { + address_offset = 0; + dummy_addr = vma->vm_start; + } + page = __alloc_buddy_huge_page_with_mpol(h, vma, +dummy_addr); + address_offset++; + } else { + page = __alloc_buddy_huge_page_no_mpol(h, NUMA_NO_NODE); + } if (!page) { alloc_ok = false; break; @@ -2057,7 +2073,7 @@ struct page *alloc_huge_page(struct vm_area_struct *vma, long rsv_adjust; rsv_adjust = hugepage_subpool_put_pages(spool, 1); - huge
[tip:x86/cpufeature] x86/cpufeature: Enable RING3MWAIT for Knights Landing
Commit-ID: e16fd002afe2b16d828bbf738b8a81a185fe9272 Gitweb: http://git.kernel.org/tip/e16fd002afe2b16d828bbf738b8a81a185fe9272 Author: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> AuthorDate: Fri, 20 Jan 2017 14:22:36 +0100 Committer: Thomas Gleixner <t...@linutronix.de> CommitDate: Sat, 4 Feb 2017 08:51:09 +0100 x86/cpufeature: Enable RING3MWAIT for Knights Landing Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> Cc: piotr@intel.com Cc: dave.han...@linux.intel.com Link: http://lkml.kernel.org/r/1484918557-15481-5-git-send-email-grzegorz.andrejc...@intel.com Signed-off-by: Thomas Gleixner <t...@linutronix.de> --- Documentation/admin-guide/kernel-parameters.txt | 4 +++ arch/x86/kernel/cpu/intel.c | 37 + 2 files changed, 41 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index be7c0d9..cfbb3fc 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3563,6 +3563,10 @@ rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait=disable + [KNL] Disable ring 3 MONITOR/MWAIT feature on supported + CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 203f860..da2401a 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -15,6 +15,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -62,6 +64,39 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static bool ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = true; + return 0; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + + if (c == _cpu_data) + ELF_HWCAP2 |= HWCAP2_RING3MWAIT; +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -560,6 +595,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32
[tip:x86/cpufeature] x86/cpufeature: Enable RING3MWAIT for Knights Landing
Commit-ID: e16fd002afe2b16d828bbf738b8a81a185fe9272 Gitweb: http://git.kernel.org/tip/e16fd002afe2b16d828bbf738b8a81a185fe9272 Author: Grzegorz Andrejczuk AuthorDate: Fri, 20 Jan 2017 14:22:36 +0100 Committer: Thomas Gleixner CommitDate: Sat, 4 Feb 2017 08:51:09 +0100 x86/cpufeature: Enable RING3MWAIT for Knights Landing Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk Cc: piotr@intel.com Cc: dave.han...@linux.intel.com Link: http://lkml.kernel.org/r/1484918557-15481-5-git-send-email-grzegorz.andrejc...@intel.com Signed-off-by: Thomas Gleixner --- Documentation/admin-guide/kernel-parameters.txt | 4 +++ arch/x86/kernel/cpu/intel.c | 37 + 2 files changed, 41 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index be7c0d9..cfbb3fc 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3563,6 +3563,10 @@ rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait=disable + [KNL] Disable ring 3 MONITOR/MWAIT feature on supported + CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 203f860..da2401a 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -15,6 +15,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -62,6 +64,39 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static bool ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = true; + return 0; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + + if (c == _cpu_data) + ELF_HWCAP2 |= HWCAP2_RING3MWAIT; +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -560,6 +595,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32
[tip:x86/cpufeature] x86/cpufeature: Add RING3MWAIT to CPU features
Commit-ID: 1d12d0ef0194ccc4dcebed3d96bb2301b26fc3ee Gitweb: http://git.kernel.org/tip/1d12d0ef0194ccc4dcebed3d96bb2301b26fc3ee Author: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> AuthorDate: Fri, 20 Jan 2017 14:22:35 +0100 Committer: Thomas Gleixner <t...@linutronix.de> CommitDate: Sat, 4 Feb 2017 08:51:09 +0100 x86/cpufeature: Add RING3MWAIT to CPU features Add software-defined CPUID bit for the non-architectural ring 3 MONITOR/MWAIT feature. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> Cc: piotr@intel.com Cc: dave.han...@linux.intel.com Link: http://lkml.kernel.org/r/1484918557-15481-4-git-send-email-grzegorz.andrejc...@intel.com Signed-off-by: Thomas Gleixner <t...@linutronix.de> --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index d9d7136..56e5184 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */
[tip:x86/cpufeature] x86/cpufeature: Add RING3MWAIT to CPU features
Commit-ID: 1d12d0ef0194ccc4dcebed3d96bb2301b26fc3ee Gitweb: http://git.kernel.org/tip/1d12d0ef0194ccc4dcebed3d96bb2301b26fc3ee Author: Grzegorz Andrejczuk AuthorDate: Fri, 20 Jan 2017 14:22:35 +0100 Committer: Thomas Gleixner CommitDate: Sat, 4 Feb 2017 08:51:09 +0100 x86/cpufeature: Add RING3MWAIT to CPU features Add software-defined CPUID bit for the non-architectural ring 3 MONITOR/MWAIT feature. Signed-off-by: Grzegorz Andrejczuk Cc: piotr@intel.com Cc: dave.han...@linux.intel.com Link: http://lkml.kernel.org/r/1484918557-15481-4-git-send-email-grzegorz.andrejc...@intel.com Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index d9d7136..56e5184 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */
[tip:x86/cpufeature] x86/elf: Add HWCAP2 to expose ring 3 MONITOR/MWAIT
Commit-ID: 0274f9551eff55dbd63b5f5f3efe30fe5d4c801c Gitweb: http://git.kernel.org/tip/0274f9551eff55dbd63b5f5f3efe30fe5d4c801c Author: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> AuthorDate: Fri, 20 Jan 2017 14:22:34 +0100 Committer: Thomas Gleixner <t...@linutronix.de> CommitDate: Sat, 4 Feb 2017 08:51:09 +0100 x86/elf: Add HWCAP2 to expose ring 3 MONITOR/MWAIT Introduce ELF_HWCAP2 variable for x86 and reserve its bit 0 to expose the ring 3 MONITOR/MWAIT. HWCAP variables contain bitmasks which can be used by userspace applications to detect which instruction sets are supported by CPU. On x86 architecture information about CPU capabilities can be checked via CPUID instructions, unfortunately presence of ring 3 MONITOR/MWAIT feature cannot be checked this way. ELF_HWCAP cannot be used as well, because on x86 it is set to CPUID[1].EDX which means that all bits are reserved there. HWCAP2 approach was chosen because it reuses existing solution present in other architectures, so only minor modifications are required to the kernel and userspace applications. When ELF_HWCAP2 is defined kernel maps it to AT_HWCAP2 during the start of the application. This way the ring 3 MONITOR/MWAIT feature can be detected using getauxval() API in a simple and fast manner. ELF_HWCAP2 type is u32 to be consistent with x86 ELF_HWCAP type. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> Cc: piotr@intel.com Cc: dave.han...@linux.intel.com Link: http://lkml.kernel.org/r/1484918557-15481-3-git-send-email-grzegorz.andrejc...@intel.com Signed-off-by: Thomas Gleixner <t...@linutronix.de> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..9d49c18 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern u32 elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 (elf_hwcap2) + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..0bd2be5 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* MONITOR/MWAIT enabled in Ring 3 */ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 9bab7a8..f879429 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +u32 elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask;
[tip:x86/cpufeature] x86/elf: Add HWCAP2 to expose ring 3 MONITOR/MWAIT
Commit-ID: 0274f9551eff55dbd63b5f5f3efe30fe5d4c801c Gitweb: http://git.kernel.org/tip/0274f9551eff55dbd63b5f5f3efe30fe5d4c801c Author: Grzegorz Andrejczuk AuthorDate: Fri, 20 Jan 2017 14:22:34 +0100 Committer: Thomas Gleixner CommitDate: Sat, 4 Feb 2017 08:51:09 +0100 x86/elf: Add HWCAP2 to expose ring 3 MONITOR/MWAIT Introduce ELF_HWCAP2 variable for x86 and reserve its bit 0 to expose the ring 3 MONITOR/MWAIT. HWCAP variables contain bitmasks which can be used by userspace applications to detect which instruction sets are supported by CPU. On x86 architecture information about CPU capabilities can be checked via CPUID instructions, unfortunately presence of ring 3 MONITOR/MWAIT feature cannot be checked this way. ELF_HWCAP cannot be used as well, because on x86 it is set to CPUID[1].EDX which means that all bits are reserved there. HWCAP2 approach was chosen because it reuses existing solution present in other architectures, so only minor modifications are required to the kernel and userspace applications. When ELF_HWCAP2 is defined kernel maps it to AT_HWCAP2 during the start of the application. This way the ring 3 MONITOR/MWAIT feature can be detected using getauxval() API in a simple and fast manner. ELF_HWCAP2 type is u32 to be consistent with x86 ELF_HWCAP type. Signed-off-by: Grzegorz Andrejczuk Cc: piotr@intel.com Cc: dave.han...@linux.intel.com Link: http://lkml.kernel.org/r/1484918557-15481-3-git-send-email-grzegorz.andrejc...@intel.com Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..9d49c18 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern u32 elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 (elf_hwcap2) + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..0bd2be5 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* MONITOR/MWAIT enabled in Ring 3 */ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 9bab7a8..f879429 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +u32 elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask;
[tip:x86/cpufeature] x86/msr: Add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
Commit-ID: ae47eda905e61ef6ba0b6f79b967c9de15ca4f8e Gitweb: http://git.kernel.org/tip/ae47eda905e61ef6ba0b6f79b967c9de15ca4f8e Author: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> AuthorDate: Fri, 20 Jan 2017 14:22:33 +0100 Committer: Thomas Gleixner <t...@linutronix.de> CommitDate: Sat, 4 Feb 2017 08:51:09 +0100 x86/msr: Add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit Define new MSR MISC_FEATURE_ENABLES (0x140). On supported CPUs if bit 1 of this MSR is set, then calling MONITOR and MWAIT instructions outside of ring 0 will not cause invalid-opcode exception. The MSR MISC_FEATURE_ENABLES is not yet documented in the SDM. Here is the relevant documentation: Hex Dec Name Scope 140H 320 MISC_FEATURE_ENABLES Thread 0Reserved 1If set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> Cc: piotr@intel.com Cc: dave.han...@linux.intel.com Link: http://lkml.kernel.org/r/1484918557-15481-2-git-send-email-grzegorz.andrejc...@intel.com Signed-off-by: Thomas Gleixner <t...@linutronix.de> --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 710273c..00293a9 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -543,6 +543,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* MISC_FEATURE_ENABLES non-architectural features */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 + +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */
[tip:x86/cpufeature] x86/msr: Add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
Commit-ID: ae47eda905e61ef6ba0b6f79b967c9de15ca4f8e Gitweb: http://git.kernel.org/tip/ae47eda905e61ef6ba0b6f79b967c9de15ca4f8e Author: Grzegorz Andrejczuk AuthorDate: Fri, 20 Jan 2017 14:22:33 +0100 Committer: Thomas Gleixner CommitDate: Sat, 4 Feb 2017 08:51:09 +0100 x86/msr: Add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit Define new MSR MISC_FEATURE_ENABLES (0x140). On supported CPUs if bit 1 of this MSR is set, then calling MONITOR and MWAIT instructions outside of ring 0 will not cause invalid-opcode exception. The MSR MISC_FEATURE_ENABLES is not yet documented in the SDM. Here is the relevant documentation: Hex Dec Name Scope 140H 320 MISC_FEATURE_ENABLES Thread 0Reserved 1If set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk Cc: piotr@intel.com Cc: dave.han...@linux.intel.com Link: http://lkml.kernel.org/r/1484918557-15481-2-git-send-email-grzegorz.andrejc...@intel.com Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 710273c..00293a9 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -543,6 +543,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* MISC_FEATURE_ENABLES non-architectural features */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 + +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */
[PATCH v12 2/5] x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT
Introduce ELF_HWCAP2 variable for x86 and reserve its bit 0 to expose the ring 3 MONITOR/MWAIT. HWCAP variables contain bitmask which can be used by userspace applications to detect what instruction sets are supported by CPU. On x86 architecture information about CPU capabilities can be checked via CPUID instructions, unfortunately presence of ring 3 MONITOR/MWAIT feature cannot be checked this way. ELF_HWCAP cannot be used as well, because on x86 it is set to CPUID[1].EDX which means that all bits are reserved there. HWCAP2 approach was chosen because it reuses existing solution present in other architectures, so only minor modifications are required to the kernel and userspace applications. When ELF_HWCAP2 is defined kernel maps it to AT_HWCAP2 during the start of the application. This way the ring 3 MONITOR/MWAIT feature can be detected using getauxval() API in a simple and fast manner. ELF_HWCAP2 type is u32 to be consistent with x86 ELF_HWCAP type. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..9d49c18 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern u32 elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 (elf_hwcap2) + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..0bd2be5 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* MONITOR/MWAIT enabled in Ring 3 */ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 9bab7a8..f879429 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +u32 elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v12 2/5] x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT
Introduce ELF_HWCAP2 variable for x86 and reserve its bit 0 to expose the ring 3 MONITOR/MWAIT. HWCAP variables contain bitmask which can be used by userspace applications to detect what instruction sets are supported by CPU. On x86 architecture information about CPU capabilities can be checked via CPUID instructions, unfortunately presence of ring 3 MONITOR/MWAIT feature cannot be checked this way. ELF_HWCAP cannot be used as well, because on x86 it is set to CPUID[1].EDX which means that all bits are reserved there. HWCAP2 approach was chosen because it reuses existing solution present in other architectures, so only minor modifications are required to the kernel and userspace applications. When ELF_HWCAP2 is defined kernel maps it to AT_HWCAP2 during the start of the application. This way the ring 3 MONITOR/MWAIT feature can be detected using getauxval() API in a simple and fast manner. ELF_HWCAP2 type is u32 to be consistent with x86 ELF_HWCAP type. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..9d49c18 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern u32 elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 (elf_hwcap2) + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..0bd2be5 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* MONITOR/MWAIT enabled in Ring 3 */ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 9bab7a8..f879429 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +u32 elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v12 3/5] x86/cpufeature: add RING3MWAIT to CPU features
Add software-defined CPUID bit for the non-architectural ring 3 MONITOR/MWAIT feature. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index eafee31..167e140 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v12 3/5] x86/cpufeature: add RING3MWAIT to CPU features
Add software-defined CPUID bit for the non-architectural ring 3 MONITOR/MWAIT feature. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index eafee31..167e140 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v12 5/5] x86/cpufeature: enable RING3MWAIT for Knights Mill
From: Piotr LucEnable ring 3 MONITOR/MWAIT for Intel Xeon Phi codenamed Knights Mill. We can't guarantee that this (KNM) will be the last CPU model that needs this hack. But, we do recognize that this is far from optimal, and there is an effort to ensure we don't keep doing extending this hack forever. Signed-off-by: Piotr Luc --- arch/x86/kernel/cpu/intel.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 213cbf0..b1b1af5 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -79,7 +79,9 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) * Ring 3 MONITOR/MWAIT feature cannot be detected without * cpu model and family comparison. */ - if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + if (c->x86 != 6 || + (c->x86_model != INTEL_FAM6_XEON_PHI_KNL && + c->x86_model != INTEL_FAM6_XEON_PHI_KNM)) return; if (ring3mwait_disabled) { -- 2.5.1
[PATCH v12 5/5] x86/cpufeature: enable RING3MWAIT for Knights Mill
From: Piotr Luc Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi codenamed Knights Mill. We can't guarantee that this (KNM) will be the last CPU model that needs this hack. But, we do recognize that this is far from optimal, and there is an effort to ensure we don't keep doing extending this hack forever. Signed-off-by: Piotr Luc --- arch/x86/kernel/cpu/intel.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 213cbf0..b1b1af5 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -79,7 +79,9 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) * Ring 3 MONITOR/MWAIT feature cannot be detected without * cpu model and family comparison. */ - if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + if (c->x86 != 6 || + (c->x86_model != INTEL_FAM6_XEON_PHI_KNL && + c->x86_model != INTEL_FAM6_XEON_PHI_KNM)) return; if (ring3mwait_disabled) { -- 2.5.1
[PATCH v12 4/5] x86/cpufeature: enable RING3MWAIT for Knights Landing
Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/admin-guide/kernel-parameters.txt | 4 +++ arch/x86/kernel/cpu/intel.c | 37 + 2 files changed, 41 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index be7c0d9..cfbb3fc 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3563,6 +3563,10 @@ rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait=disable + [KNL] Disable ring 3 MONITOR/MWAIT feature on supported + CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 203f860..213cbf0 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -15,6 +15,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -62,6 +64,39 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static int ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = 1; + return 0; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + + if (c == _cpu_data) + ELF_HWCAP2 |= HWCAP2_RING3MWAIT; +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -560,6 +595,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v12 4/5] x86/cpufeature: enable RING3MWAIT for Knights Landing
Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk --- Documentation/admin-guide/kernel-parameters.txt | 4 +++ arch/x86/kernel/cpu/intel.c | 37 + 2 files changed, 41 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index be7c0d9..cfbb3fc 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3563,6 +3563,10 @@ rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait=disable + [KNL] Disable ring 3 MONITOR/MWAIT feature on supported + CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 203f860..213cbf0 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -15,6 +15,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -62,6 +64,39 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static int ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = 1; + return 0; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + + if (c == _cpu_data) + ELF_HWCAP2 |= HWCAP2_RING3MWAIT; +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -560,6 +595,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v12 1/5] x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
Define new MSR MISC_FEATURE_ENABLES (0x140). On supported CPUs if bit 1 of this new register is set, then calling MONITOR and MWAIT instructions outside of ring 0 will not cause invalid-opcode exception. The MSR MISC_FEATURE_ENABLES is not yet documented in the SDM. Here is the relevant documentation: Hex Dec Name Scope 140H 320 MISC_FEATURE_ENABLES Thread 0Reserved 1If set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 710273c..00293a9 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -543,6 +543,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* MISC_FEATURE_ENABLES non-architectural features */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 + +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v12 1/5] x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
Define new MSR MISC_FEATURE_ENABLES (0x140). On supported CPUs if bit 1 of this new register is set, then calling MONITOR and MWAIT instructions outside of ring 0 will not cause invalid-opcode exception. The MSR MISC_FEATURE_ENABLES is not yet documented in the SDM. Here is the relevant documentation: Hex Dec Name Scope 140H 320 MISC_FEATURE_ENABLES Thread 0Reserved 1If set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 710273c..00293a9 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -543,6 +543,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* MISC_FEATURE_ENABLES non-architectural features */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 + +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v12 0/5] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
Following patches enable the use of the feature that allows the Intel Xeon Phi x200 devices to use MONITOR/MWAIT instructions outside ring 0. This feature allows userspace application to use more efficient synchronization operations, which improves performance and energy efficiency. v12: Removed unused define from msr-info.h Added braces in define ELF_HWCAP Set HWCAP2_RING3MWAIT bit only for boot cpu Replaced set_bit by bit OR operator Updated ring3mwait_disable return value Rebased to kernel 4.10rc4 Updated commit messages v11: Removed warning from 32-bit build Removed "This patch" from commit messages v10: Included Piotr's patch for Knights Mill Included Dave's comments from internal review Rewritten commit messages Remove x86_64 requirement Fixed kernel boot parameter description Used set_bit to update HWCAP2 bit Rebased to kernel 4.9 v9: Removed PHI from defines v8: Fixed commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT x86/cpufeature: add RING3MWAIT to CPU features x86/cpufeature: enable RING3MWAIT for Knights Landing Piotr Luc (1): x86/cpufeature: enable RING3MWAIT for Knights Mill Documentation/admin-guide/kernel-parameters.txt | 4 +++ arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/elf.h | 9 ++ arch/x86/include/asm/msr-index.h| 5 arch/x86/include/uapi/asm/hwcap2.h | 7 + arch/x86/kernel/cpu/common.c| 3 ++ arch/x86/kernel/cpu/intel.c | 39 + 7 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v12 0/5] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
Following patches enable the use of the feature that allows the Intel Xeon Phi x200 devices to use MONITOR/MWAIT instructions outside ring 0. This feature allows userspace application to use more efficient synchronization operations, which improves performance and energy efficiency. v12: Removed unused define from msr-info.h Added braces in define ELF_HWCAP Set HWCAP2_RING3MWAIT bit only for boot cpu Replaced set_bit by bit OR operator Updated ring3mwait_disable return value Rebased to kernel 4.10rc4 Updated commit messages v11: Removed warning from 32-bit build Removed "This patch" from commit messages v10: Included Piotr's patch for Knights Mill Included Dave's comments from internal review Rewritten commit messages Remove x86_64 requirement Fixed kernel boot parameter description Used set_bit to update HWCAP2 bit Rebased to kernel 4.9 v9: Removed PHI from defines v8: Fixed commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT x86/cpufeature: add RING3MWAIT to CPU features x86/cpufeature: enable RING3MWAIT for Knights Landing Piotr Luc (1): x86/cpufeature: enable RING3MWAIT for Knights Mill Documentation/admin-guide/kernel-parameters.txt | 4 +++ arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/elf.h | 9 ++ arch/x86/include/asm/msr-index.h| 5 arch/x86/include/uapi/asm/hwcap2.h | 7 + arch/x86/kernel/cpu/common.c| 3 ++ arch/x86/kernel/cpu/intel.c | 39 + 7 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[Patch v11 4/5] x86/cpufeature: enable RING3MWAIT for Knights Landing
Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. The patch: - Sets CPU feature X86_FEATURE_RING3MWAIT. - Sets HWCAP2_RING3MWAIT bit in ELF_HWCAP2. - Adds the ring3mwait=disable command line parameter. - Sets bit 1 of the MSR MISC_FEATURE_ENABLES or clears it when the ring3mwait=disable command line parameter is used. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/kernel-parameters.txt | 4 arch/x86/kernel/cpu/intel.c | 34 ++ 2 files changed, 38 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 37babf9..b8b4ac8 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3734,6 +3734,10 @@ bytes respectively. Such letter suffixes can also be entirely omitted. rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait=disable + [KNL] Disable ring 3 MONITOR/MWAIT feature on supported + CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..9d07bee 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,36 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static int ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = 1; + return 1; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + set_bit(HWCAP2_RING3MWAIT, (unsigned long *)_HWCAP2); +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +597,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[Patch v11 4/5] x86/cpufeature: enable RING3MWAIT for Knights Landing
Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. The patch: - Sets CPU feature X86_FEATURE_RING3MWAIT. - Sets HWCAP2_RING3MWAIT bit in ELF_HWCAP2. - Adds the ring3mwait=disable command line parameter. - Sets bit 1 of the MSR MISC_FEATURE_ENABLES or clears it when the ring3mwait=disable command line parameter is used. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk --- Documentation/kernel-parameters.txt | 4 arch/x86/kernel/cpu/intel.c | 34 ++ 2 files changed, 38 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 37babf9..b8b4ac8 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3734,6 +3734,10 @@ bytes respectively. Such letter suffixes can also be entirely omitted. rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait=disable + [KNL] Disable ring 3 MONITOR/MWAIT feature on supported + CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..9d07bee 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,36 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static int ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = 1; + return 1; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + set_bit(HWCAP2_RING3MWAIT, (unsigned long *)_HWCAP2); +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +597,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v11 1/5] x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
Define new MSR MISC_FEATURE_ENABLES (0x140). On supported CPUs if bit 1 of this new register is set, then calling MONITOR and MWAIT instructions outside of ring 0 will not cause invalid-opcode exception. The MSR MISC_FEATURE_ENABLES is not yet documented in the SDM. Here is the relevant documentation: Hex Dec Name Scope 140H 320 MISC_FEATURE_ENABLES Thread 0Reserved 1If set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/msr-index.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 78f3760..55ffae0 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -539,6 +539,12 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* MISC_FEATURE_ENABLES non-architectural features */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 + +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v11 1/5] x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
Define new MSR MISC_FEATURE_ENABLES (0x140). On supported CPUs if bit 1 of this new register is set, then calling MONITOR and MWAIT instructions outside of ring 0 will not cause invalid-opcode exception. The MSR MISC_FEATURE_ENABLES is not yet documented in the SDM. Here is the relevant documentation: Hex Dec Name Scope 140H 320 MISC_FEATURE_ENABLES Thread 0Reserved 1If set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/msr-index.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 78f3760..55ffae0 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -539,6 +539,12 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* MISC_FEATURE_ENABLES non-architectural features */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 + +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v11 3/5] x86/cpufeature: add RING3MWAIT to CPU features
Add software-defined CPUID bit for the non-architectural ring 3 MONITOR/MWAIT feature. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index a396292..dc0255e 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v11 3/5] x86/cpufeature: add RING3MWAIT to CPU features
Add software-defined CPUID bit for the non-architectural ring 3 MONITOR/MWAIT feature. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index a396292..dc0255e 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v11 5/5] x86/cpufeature: enable RING3MWAIT for Knights Mill
From: Piotr LucEnable ring 3 MONITOR/MWAIT for Intel Xeon Phi codenamed Knights Mill. We can't guarantee that this (KNM) will be the last CPU model that needs this hack. But, we do recognize that this is far from optimal, and there is an effort to ensure we don't keep doing extending this hack forever Signed-off-by: Piotr Luc --- arch/x86/kernel/cpu/intel.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 9d07bee..a1c28ea 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -78,7 +78,9 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) * Ring 3 MONITOR/MWAIT feature cannot be detected without * cpu model and family comparison. */ - if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + if (c->x86 != 6 || + (c->x86_model != INTEL_FAM6_XEON_PHI_KNL && + c->x86_model != INTEL_FAM6_XEON_PHI_KNM)) return; if (ring3mwait_disabled) { -- 2.5.1
[PATCH v11 5/5] x86/cpufeature: enable RING3MWAIT for Knights Mill
From: Piotr Luc Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi codenamed Knights Mill. We can't guarantee that this (KNM) will be the last CPU model that needs this hack. But, we do recognize that this is far from optimal, and there is an effort to ensure we don't keep doing extending this hack forever Signed-off-by: Piotr Luc --- arch/x86/kernel/cpu/intel.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 9d07bee..a1c28ea 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -78,7 +78,9 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) * Ring 3 MONITOR/MWAIT feature cannot be detected without * cpu model and family comparison. */ - if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + if (c->x86 != 6 || + (c->x86_model != INTEL_FAM6_XEON_PHI_KNL && + c->x86_model != INTEL_FAM6_XEON_PHI_KNM)) return; if (ring3mwait_disabled) { -- 2.5.1
[PATCH v11 2/5] x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT
Introduce ELF_HWCAP2 variable for x86 and reserve its bit 0 to expose the ring 3 MONITOR/MWAIT. HWCAP variables contain bitmask which can be used by userspace applications to detect what instruction sets are supported by CPU. On x86 architecture information about CPU capabilities can be checked via CPUID instructions, unfortunately presence of ring 3 MONITOR/MWAIT feature cannot be checked this way. ELF_HWCAP cannot be used as well, because on x86 it is set to CPUID[1].EDX which means that all bits are reserved there. HWCAP2 approach was chosen because it reuses existing solution present in other architectures, so only minor modifications are required to the kernel and userspace applications. When ELF_HWCAP2 is defined kernel maps it to AT_HWCAP2 during the start of the application. This way the ring 3 MONITOR/MWAIT feature can be detected using getauxval() API in a simple and fast manner. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..0bd2be5 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* MONITOR/MWAIT enabled in Ring 3 */ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index cc9e980..217697b 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v11 4/5] x86/cpufeature: enable RING3MWAIT for Knights Landing
Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. The patch: - Sets CPU feature X86_FEATURE_RING3MWAIT. - Sets HWCAP2_RING3MWAIT bit in ELF_HWCAP2. - Adds the ring3mwait=disable command line parameter. - Sets bit 1 of the MSR MISC_FEATURE_ENABLES or clears it when the ring3mwait=disable command line parameter is used. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/kernel-parameters.txt | 3 +++ arch/x86/kernel/cpu/intel.c | 34 ++ 2 files changed, 37 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 37babf9..c8bca65 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3734,6 +3734,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted. rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait= [KNL] + disable Disable ring 3 MONITOR/MWAIT feature on supported CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..9d07bee 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,36 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static int ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = 1; + return 1; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + set_bit(HWCAP2_RING3MWAIT, (unsigned long *)_HWCAP2); +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +597,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v11 2/5] x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT
Introduce ELF_HWCAP2 variable for x86 and reserve its bit 0 to expose the ring 3 MONITOR/MWAIT. HWCAP variables contain bitmask which can be used by userspace applications to detect what instruction sets are supported by CPU. On x86 architecture information about CPU capabilities can be checked via CPUID instructions, unfortunately presence of ring 3 MONITOR/MWAIT feature cannot be checked this way. ELF_HWCAP cannot be used as well, because on x86 it is set to CPUID[1].EDX which means that all bits are reserved there. HWCAP2 approach was chosen because it reuses existing solution present in other architectures, so only minor modifications are required to the kernel and userspace applications. When ELF_HWCAP2 is defined kernel maps it to AT_HWCAP2 during the start of the application. This way the ring 3 MONITOR/MWAIT feature can be detected using getauxval() API in a simple and fast manner. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..0bd2be5 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* MONITOR/MWAIT enabled in Ring 3 */ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index cc9e980..217697b 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v11 4/5] x86/cpufeature: enable RING3MWAIT for Knights Landing
Enable ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. The patch: - Sets CPU feature X86_FEATURE_RING3MWAIT. - Sets HWCAP2_RING3MWAIT bit in ELF_HWCAP2. - Adds the ring3mwait=disable command line parameter. - Sets bit 1 of the MSR MISC_FEATURE_ENABLES or clears it when the ring3mwait=disable command line parameter is used. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk --- Documentation/kernel-parameters.txt | 3 +++ arch/x86/kernel/cpu/intel.c | 34 ++ 2 files changed, 37 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 37babf9..c8bca65 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3734,6 +3734,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted. rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait= [KNL] + disable Disable ring 3 MONITOR/MWAIT feature on supported CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..9d07bee 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,36 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static int ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = 1; + return 1; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + set_bit(HWCAP2_RING3MWAIT, (unsigned long *)_HWCAP2); +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +597,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v11 0/5] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
Following patches enable the use of the feature that allows the Intel Xeon Phi x200 devices to use MONITOR/MWAIT instructions outside ring 0. It allows userspace applications to use more efficient synchronization operations, which improves performance and energy efficiency. v11: Removed warning from 32-bit build Removed "This patch" from commit messages v10: Included Piotr's patch for Knights Mill Included Dave's comments from internal review Rewritten commit messages Remove x86_64 requirement Fixed kernel boot parameter description Used set_bit to update HWCAP2 bit Rebased to kernel 4.9 v9: Removed PHI from defines v8: Fixed commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT x86/cpufeature: add RING3MWAIT to CPU features x86/cpufeature: enable RING3MWAIT for Knights Landing Piotr Luc (1): x86/cpufeature: enable RING3MWAIT for Knights Mill Documentation/kernel-parameters.txt | 3 +++ arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/elf.h | 9 + arch/x86/include/asm/msr-index.h| 6 ++ arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 36 7 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v11 0/5] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
Following patches enable the use of the feature that allows the Intel Xeon Phi x200 devices to use MONITOR/MWAIT instructions outside ring 0. It allows userspace applications to use more efficient synchronization operations, which improves performance and energy efficiency. v11: Removed warning from 32-bit build Removed "This patch" from commit messages v10: Included Piotr's patch for Knights Mill Included Dave's comments from internal review Rewritten commit messages Remove x86_64 requirement Fixed kernel boot parameter description Used set_bit to update HWCAP2 bit Rebased to kernel 4.9 v9: Removed PHI from defines v8: Fixed commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT x86/cpufeature: add RING3MWAIT to CPU features x86/cpufeature: enable RING3MWAIT for Knights Landing Piotr Luc (1): x86/cpufeature: enable RING3MWAIT for Knights Mill Documentation/kernel-parameters.txt | 3 +++ arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/elf.h | 9 + arch/x86/include/asm/msr-index.h| 6 ++ arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 36 7 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v10 4/5] x86/cpufeature: enable RING3MWAIT for Knights Landing
This patch enables ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. The patch: - Sets CPU feature X86_FEATURE_RING3MWAIT. - Sets HWCAP2_RING3MWAIT bit in ELF_HWCAP2. - Adds the ring3mwait=disable command line parameter. - Sets bit 1 of the MSR MISC_FEATURE_ENABLES or clears it when the ring3mwait=disable command line parameter is used. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/kernel-parameters.txt | 3 +++ arch/x86/kernel/cpu/intel.c | 34 ++ 2 files changed, 37 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 37babf9..c8bca65 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3734,6 +3734,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted. rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait= [KNL] + disable Disable ring 3 MONITOR/MWAIT feature on supported CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..70d4985 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,36 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static int ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = 1; + return 1; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + set_bit(HWCAP2_RING3MWAIT, _HWCAP2); +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +597,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v10 0/5] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
Following patches enable the use of the feature that allows the some Intel Xeon Phi processors to use MONITOR/MWAIT instructions outside ring 0. This feature allows userspace application to use more efficient synchronization operations, which improves performance and energy efficiency. v10: Included Piotr's patch for Knights Mill Included Dave's comments from internal review Rewritten commit messages Removed x86_64 config requirement Fixed kernel boot parameter description Used set_bit to update HWCAP2 bit Rebased to kernel 4.9 v9: Removed PHI prefix from defines v8: Updated commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Changed order of the patches Changed the name of MSR to MSR_MISC_FEATURE_ENABLES Used bit 25 from word 3 to expose feature v6: v5: Added phir3mwait=disable cmdline switch Fixed typos v4: Wrapped the enabling code by CONFIG_X86_64 Moved probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Dave's and Thomas' comments v2: Added check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT x86/cpufeature: add RING3MWAIT to CPU features x86/cpufeature: enable RING3MWAIT for Knights Landing Piotr Luc (1): x86/cpufeature: enable RING3MWAIT for Knights Mill Documentation/kernel-parameters.txt | 3 +++ arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/elf.h | 9 + arch/x86/include/asm/msr-index.h| 6 ++ arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 36 7 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v10 3/5] x86/cpufeature: add RING3MWAIT to CPU features
This commit adds software-defined CPUID bit for the non-architectural ring 3 MONITOR/MWAIT feature. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index a396292..dc0255e 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v10 2/5] x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT
This patch introduces ELF_HWCAP2 variable for x86 and reserves one bit in it to expose the ring 3 MONITOR/MWAIT. HWCAP variables contain bitmask which can be used by userspace applications to detect what instruction sets are supported by CPU. On x86 architecture information about CPU capabilities can be checked via CPUID instructions, unfortunately presence of ring 3 MONITOR/MWAIT feature cannot be checked this way. ELF_HWCAP cannot be used as well, because on x86 it is set to CPUID[1].EDX which means that all bits are reserved there. HWCAP2 approach was chosen because it reuses existing solution present in other architectures, so only minor modifications are required to the kernel and userspace applications. When ELF_HWCAP2 is defined kernel maps it to AT_HWCAP2 during the start of the application. This way the ring 3 MONITOR/MWAIT feature can be detected using getauxval() API in a simple and fast manner. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..0bd2be5 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* MONITOR/MWAIT enabled in Ring 3 */ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index cc9e980..217697b 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v10 3/5] x86/cpufeature: add RING3MWAIT to CPU features
This commit adds software-defined CPUID bit for the non-architectural ring 3 MONITOR/MWAIT feature. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index a396292..dc0255e 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v10 2/5] x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT
This patch introduces ELF_HWCAP2 variable for x86 and reserves one bit in it to expose the ring 3 MONITOR/MWAIT. HWCAP variables contain bitmask which can be used by userspace applications to detect what instruction sets are supported by CPU. On x86 architecture information about CPU capabilities can be checked via CPUID instructions, unfortunately presence of ring 3 MONITOR/MWAIT feature cannot be checked this way. ELF_HWCAP cannot be used as well, because on x86 it is set to CPUID[1].EDX which means that all bits are reserved there. HWCAP2 approach was chosen because it reuses existing solution present in other architectures, so only minor modifications are required to the kernel and userspace applications. When ELF_HWCAP2 is defined kernel maps it to AT_HWCAP2 during the start of the application. This way the ring 3 MONITOR/MWAIT feature can be detected using getauxval() API in a simple and fast manner. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..0bd2be5 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* MONITOR/MWAIT enabled in Ring 3 */ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index cc9e980..217697b 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v10 4/5] x86/cpufeature: enable RING3MWAIT for Knights Landing
This patch enables ring 3 MONITOR/MWAIT for Intel Xeon Phi x200 codenamed Knights Landing. The patch: - Sets CPU feature X86_FEATURE_RING3MWAIT. - Sets HWCAP2_RING3MWAIT bit in ELF_HWCAP2. - Adds the ring3mwait=disable command line parameter. - Sets bit 1 of the MSR MISC_FEATURE_ENABLES or clears it when the ring3mwait=disable command line parameter is used. Presence of this feature cannot be detected automatically (by reading any other MSR) therefore it is required to explicitly check for the family and model of the CPU before attempting to enable it. Signed-off-by: Grzegorz Andrejczuk --- Documentation/kernel-parameters.txt | 3 +++ arch/x86/kernel/cpu/intel.c | 34 ++ 2 files changed, 37 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 37babf9..c8bca65 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3734,6 +3734,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted. rhash_entries= [KNL,NET] Set number of hash buckets for route cache + ring3mwait= [KNL] + disable Disable ring 3 MONITOR/MWAIT feature on supported CPUs. + ro [KNL] Mount root device read-only on boot rodata= [KNL] diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..70d4985 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,36 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +static int ring3mwait_disabled __read_mostly; + +static int __init ring3mwait_disable(char *__unused) +{ + ring3mwait_disabled = 1; + return 1; +} +__setup("ring3mwait=disable", ring3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (ring3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + set_bit(HWCAP2_RING3MWAIT, _HWCAP2); +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +597,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v10 0/5] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
Following patches enable the use of the feature that allows the some Intel Xeon Phi processors to use MONITOR/MWAIT instructions outside ring 0. This feature allows userspace application to use more efficient synchronization operations, which improves performance and energy efficiency. v10: Included Piotr's patch for Knights Mill Included Dave's comments from internal review Rewritten commit messages Removed x86_64 config requirement Fixed kernel boot parameter description Used set_bit to update HWCAP2 bit Rebased to kernel 4.9 v9: Removed PHI prefix from defines v8: Updated commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Changed order of the patches Changed the name of MSR to MSR_MISC_FEATURE_ENABLES Used bit 25 from word 3 to expose feature v6: v5: Added phir3mwait=disable cmdline switch Fixed typos v4: Wrapped the enabling code by CONFIG_X86_64 Moved probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Dave's and Thomas' comments v2: Added check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit x86/elf: add HWCAP2 to expose ring 3 MONITOR/MWAIT x86/cpufeature: add RING3MWAIT to CPU features x86/cpufeature: enable RING3MWAIT for Knights Landing Piotr Luc (1): x86/cpufeature: enable RING3MWAIT for Knights Mill Documentation/kernel-parameters.txt | 3 +++ arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/elf.h | 9 + arch/x86/include/asm/msr-index.h| 6 ++ arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 36 7 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v10 1/5] x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
This patch defines new MSR MISC_FEATURE_ENABLES (0x140). On supported CPUs if bit 1 of this new register is set, then calling MONITOR and MWAIT instructions outside of ring 0 will not cause invalid-opcode exception. The MSR MISC_FEATURE_ENABLES is not yet documented in the SDM. Here is the relevant documentation: Hex Dec Name Scope 140H 320 MISC_FEATURE_ENABLES Thread 0Reserved 1If set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/msr-index.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 78f3760..55ffae0 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -539,6 +539,12 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* MISC_FEATURE_ENABLES non-architectural features */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 + +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v10 1/5] x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
This patch defines new MSR MISC_FEATURE_ENABLES (0x140). On supported CPUs if bit 1 of this new register is set, then calling MONITOR and MWAIT instructions outside of ring 0 will not cause invalid-opcode exception. The MSR MISC_FEATURE_ENABLES is not yet documented in the SDM. Here is the relevant documentation: Hex Dec Name Scope 140H 320 MISC_FEATURE_ENABLES Thread 0Reserved 1If set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/msr-index.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 78f3760..55ffae0 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -539,6 +539,12 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* MISC_FEATURE_ENABLES non-architectural features */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 + +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v10 5/5] x86/cpufeature: enable RING3MWAIT for Knights Mill
From: Piotr LucThis patch enables ring 3 MONITOR/MWAIT for Intel Xeon Phi codenamed Knights Mill. We can't guarantee that this (KNM) will be the last CPU model that needs this hack. But, we do recognize that this is far from optimal, and there is an effort to ensure we don't keep doing extending this hack forever. Signed-off-by: Piotr Luc --- arch/x86/kernel/cpu/intel.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 70d4985..1507c7c 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -78,7 +78,9 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) * Ring 3 MONITOR/MWAIT feature cannot be detected without * cpu model and family comparison. */ - if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + if (c->x86 != 6 || + (c->x86_model != INTEL_FAM6_XEON_PHI_KNL && + c->x86_model != INTEL_FAM6_XEON_PHI_KNM)) return; if (ring3mwait_disabled) { -- 2.5.1
[PATCH v10 5/5] x86/cpufeature: enable RING3MWAIT for Knights Mill
From: Piotr Luc This patch enables ring 3 MONITOR/MWAIT for Intel Xeon Phi codenamed Knights Mill. We can't guarantee that this (KNM) will be the last CPU model that needs this hack. But, we do recognize that this is far from optimal, and there is an effort to ensure we don't keep doing extending this hack forever. Signed-off-by: Piotr Luc --- arch/x86/kernel/cpu/intel.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 70d4985..1507c7c 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -78,7 +78,9 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) * Ring 3 MONITOR/MWAIT feature cannot be detected without * cpu model and family comparison. */ - if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + if (c->x86 != 6 || + (c->x86_model != INTEL_FAM6_XEON_PHI_KNL && + c->x86_model != INTEL_FAM6_XEON_PHI_KNM)) return; if (ring3mwait_disabled) { -- 2.5.1
[PATCH v9 4/4] x86/cpufeatures: handle RING3MWAIT on Xeon Phi models
Unfortunately presence of this feature cannot be detected automatically (by reading some other MSR) therefore it is required to do explicit check for the family and model of the cpu. If processor is Intel Xeon Phi x200 RING 3 MONITOR/MWAIT feature is enabled by setting cpu cap X86_FEATURE_RING3MWAIT and elf HWCAP2_RING3MWAIT. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/kernel-parameters.txt | 6 + Documentation/x86/x86_64/boot-options.txt | 5 arch/x86/kernel/cpu/intel.c | 39 +++ 3 files changed, 50 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..4683cc1 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86-64] Disable ring 3 MONITOR/MWAIT on Intel Xeon Phi + codenamed Knight Landing cpus. + It has no effect on cpus other than Intel Xeon Phi. + Format: { disable } + See Documentation/x86/x86_64/boot-options.txt + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/Documentation/x86/x86_64/boot-options.txt b/Documentation/x86/x86_64/boot-options.txt index 0965a71..1a515e8 100644 --- a/Documentation/x86/x86_64/boot-options.txt +++ b/Documentation/x86/x86_64/boot-options.txt @@ -281,6 +281,11 @@ Debugging kstack=N Print N words from the kernel stack in oops dumps. + phir3mwait=disable + Disables unconditional setting bit 1 of the MSR_MISC_FEATURE_ENABLES + for Intel Xeon Phi, this way administrator can switch off ring 3 mwait + feature. + Miscellaneous nogbpages diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..3bfc8e5 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,41 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (phi_r3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + ELF_HWCAP2 |= HWCAP2_RING3MWAIT; +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +602,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v9 4/4] x86/cpufeatures: handle RING3MWAIT on Xeon Phi models
Unfortunately presence of this feature cannot be detected automatically (by reading some other MSR) therefore it is required to do explicit check for the family and model of the cpu. If processor is Intel Xeon Phi x200 RING 3 MONITOR/MWAIT feature is enabled by setting cpu cap X86_FEATURE_RING3MWAIT and elf HWCAP2_RING3MWAIT. Signed-off-by: Grzegorz Andrejczuk --- Documentation/kernel-parameters.txt | 6 + Documentation/x86/x86_64/boot-options.txt | 5 arch/x86/kernel/cpu/intel.c | 39 +++ 3 files changed, 50 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..4683cc1 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86-64] Disable ring 3 MONITOR/MWAIT on Intel Xeon Phi + codenamed Knight Landing cpus. + It has no effect on cpus other than Intel Xeon Phi. + Format: { disable } + See Documentation/x86/x86_64/boot-options.txt + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/Documentation/x86/x86_64/boot-options.txt b/Documentation/x86/x86_64/boot-options.txt index 0965a71..1a515e8 100644 --- a/Documentation/x86/x86_64/boot-options.txt +++ b/Documentation/x86/x86_64/boot-options.txt @@ -281,6 +281,11 @@ Debugging kstack=N Print N words from the kernel stack in oops dumps. + phir3mwait=disable + Disables unconditional setting bit 1 of the MSR_MISC_FEATURE_ENABLES + for Intel Xeon Phi, this way administrator can switch off ring 3 mwait + feature. + Miscellaneous nogbpages diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..3bfc8e5 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,41 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Ring 3 MONITOR/MWAIT feature cannot be detected without +* cpu model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (phi_r3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_RING3MWAIT); + ELF_HWCAP2 |= HWCAP2_RING3MWAIT; +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +602,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v9 1/4] x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
Intel Xeon Phi x200 (codenamed Knights Landing) allows to enable MONITOR and MWAIT instructions outside of ring 0. The feature is controlled by MSR MISC_FEATURE_ENABLES (0x140). Setting bit 1 of this register enables it, so MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when invoked outside of ring 0. The feature MSR is not yet documented in the SDM. Here is the relevant documentation: Hex Dec NameScope 140H 320 MISC_FEATURE_ENABLESThread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..c95da90 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v9 1/4] x86/msr: add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit
Intel Xeon Phi x200 (codenamed Knights Landing) allows to enable MONITOR and MWAIT instructions outside of ring 0. The feature is controlled by MSR MISC_FEATURE_ENABLES (0x140). Setting bit 1 of this register enables it, so MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when invoked outside of ring 0. The feature MSR is not yet documented in the SDM. Here is the relevant documentation: Hex Dec NameScope 140H 320 MISC_FEATURE_ENABLESThread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..c95da90 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_RING3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_RING3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v9 3/4] x86/cpufeature: add RING3MWAIT to CPU features
Add Intel Xeon Phi x200 (KnightsLanding) CPU feature - ring 3 MONITOR/MWAIT. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..5b7f701 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v9 3/4] x86/cpufeature: add RING3MWAIT to CPU features
Add Intel Xeon Phi x200 (KnightsLanding) CPU feature - ring 3 MONITOR/MWAIT. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..5b7f701 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_RING3MWAIT ( 3*32+25) /* ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v9 2/4] x86/elf: use HWCAP2 to expose ring 3 MWAIT
Add HWCAP2 for x86 and reserve its bit 0 to expose ring 3 mwait. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..116cab3 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* Kernel enabled Ring 3 MONITOR/MWAIT*/ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v9 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference: https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v9: Removed PHI from defines v8: Fixed commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: Add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit x86/elf: Use HWCAP2 to expose ring 3 MWAIT x86/cpufeature: Add RING3MWAIT to CPU features x86/cpufeatures: Handle RING3MWAIT on Xeon Phi models Documentation/kernel-parameters.txt | 5 Documentation/x86/x86_64/boot-options.txt | 5 arch/x86/include/asm/cpufeatures.h| 2 +- arch/x86/include/asm/elf.h| 9 +++ arch/x86/include/asm/msr-index.h | 5 arch/x86/include/uapi/asm/hwcap2.h| 7 ++ arch/x86/kernel/cpu/common.c | 3 +++ arch/x86/kernel/cpu/intel.c | 39 +++ 8 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v9 2/4] x86/elf: use HWCAP2 to expose ring 3 MWAIT
Add HWCAP2 for x86 and reserve its bit 0 to expose ring 3 mwait. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..116cab3 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_X86_HWCAP2_H +#define _ASM_X86_HWCAP2_H + +/* Kernel enabled Ring 3 MONITOR/MWAIT*/ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v9 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference: https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v9: Removed PHI from defines v8: Fixed commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: Add MSR_MISC_FEATURE_ENABLES and RING3MWAIT bit x86/elf: Use HWCAP2 to expose ring 3 MWAIT x86/cpufeature: Add RING3MWAIT to CPU features x86/cpufeatures: Handle RING3MWAIT on Xeon Phi models Documentation/kernel-parameters.txt | 5 Documentation/x86/x86_64/boot-options.txt | 5 arch/x86/include/asm/cpufeatures.h| 2 +- arch/x86/include/asm/elf.h| 9 +++ arch/x86/include/asm/msr-index.h | 5 arch/x86/include/uapi/asm/hwcap2.h| 7 ++ arch/x86/kernel/cpu/common.c | 3 +++ arch/x86/kernel/cpu/intel.c | 39 +++ 8 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v8 3/4] x86/cpufeature: Add PHIR3MWAIT to CPU features
Add Intel Xeon Phi x200 (KnightsLanding) CPU feature - ring 3 MONITOR/MWAIT. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..98414c5 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_PHIR3MWAIT ( 3*32+25) /* Xeon Phi x200 ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v8 3/4] x86/cpufeature: Add PHIR3MWAIT to CPU features
Add Intel Xeon Phi x200 (KnightsLanding) CPU feature - ring 3 MONITOR/MWAIT. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..98414c5 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_PHIR3MWAIT ( 3*32+25) /* Xeon Phi x200 ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v8: 4/4] x86/cpufeatures: Handle RING3MWAIT on Xeon Phi models
Unfortunately presence of this feature cannot be detected automatically (by reading some other MSR) therefore it is required to do explicit check for the family and model of the cpu. If processor is Intel Xeon Phi x200 RING3MWAIT feature is enabled by setting cpu cap X86_FEATURE_PHIR3MWAIT and elf HWCAP2_RING3MWAIT. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/kernel-parameters.txt | 5 Documentation/x86/x86_64/boot-options.txt | 5 arch/x86/kernel/cpu/intel.c | 39 +++ 3 files changed, 49 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..7754310 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86-64] Do not enable Intel Xeon Phi x200 ring 3 MONITOR/MWAIT + feature for all cpus. + Format: { disable } + See Documentation/x86/x86_64/boot-options.txt + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/Documentation/x86/x86_64/boot-options.txt b/Documentation/x86/x86_64/boot-options.txt index 0965a71..1a515e8 100644 --- a/Documentation/x86/x86_64/boot-options.txt +++ b/Documentation/x86/x86_64/boot-options.txt @@ -281,6 +281,11 @@ Debugging kstack=N Print N words from the kernel stack in oops dumps. + phir3mwait=disable + Disables unconditional setting bit 1 of the MSR_MISC_FEATURE_ENABLES + for Intel Xeon Phi, this way administrator can switch off ring 3 mwait + feature. + Miscellaneous nogbpages diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..670dd98 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,41 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Rign 3 MWAIT feature cannot be detected without +* ugly model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (phi_r3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_PHIR3MWAIT); + ELF_HWCAP2 |= HWCAP2_RING3MWAIT; +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +602,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v8: 4/4] x86/cpufeatures: Handle RING3MWAIT on Xeon Phi models
Unfortunately presence of this feature cannot be detected automatically (by reading some other MSR) therefore it is required to do explicit check for the family and model of the cpu. If processor is Intel Xeon Phi x200 RING3MWAIT feature is enabled by setting cpu cap X86_FEATURE_PHIR3MWAIT and elf HWCAP2_RING3MWAIT. Signed-off-by: Grzegorz Andrejczuk --- Documentation/kernel-parameters.txt | 5 Documentation/x86/x86_64/boot-options.txt | 5 arch/x86/kernel/cpu/intel.c | 39 +++ 3 files changed, 49 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..7754310 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86-64] Do not enable Intel Xeon Phi x200 ring 3 MONITOR/MWAIT + feature for all cpus. + Format: { disable } + See Documentation/x86/x86_64/boot-options.txt + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/Documentation/x86/x86_64/boot-options.txt b/Documentation/x86/x86_64/boot-options.txt index 0965a71..1a515e8 100644 --- a/Documentation/x86/x86_64/boot-options.txt +++ b/Documentation/x86/x86_64/boot-options.txt @@ -281,6 +281,11 @@ Debugging kstack=N Print N words from the kernel stack in oops dumps. + phir3mwait=disable + Disables unconditional setting bit 1 of the MSR_MISC_FEATURE_ENABLES + for Intel Xeon Phi, this way administrator can switch off ring 3 mwait + feature. + Miscellaneous nogbpages diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..670dd98 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,41 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + /* +* Rign 3 MWAIT feature cannot be detected without +* ugly model and family comparison. +*/ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + if (phi_r3mwait_disabled) { + msr_clear_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT); + return; + } + + msr_set_bit(MSR_MISC_FEATURE_ENABLES, + MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT); + set_cpu_cap(c, X86_FEATURE_PHIR3MWAIT); + ELF_HWCAP2 |= HWCAP2_RING3MWAIT; +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +602,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v8: 2/4] x86/elf: Use HWCAP2 to expose ring 3 MWAIT
Add HWCAP2 for x86 and reserve its bit 0 to expose ring 3 mwait. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..9e7c117 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_HWCAP2_H +#define _ASM_HWCAP2_H + +/* Kernel enabled Ring 3 MWAIT for Xeon Phi*/ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v8: 2/4] x86/elf: Use HWCAP2 to expose ring 3 MWAIT
Add HWCAP2 for x86 and reserve its bit 0 to expose ring 3 mwait. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..9e7c117 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_HWCAP2_H +#define _ASM_HWCAP2_H + +/* Kernel enabled Ring 3 MWAIT for Xeon Phi*/ +#define HWCAP2_RING3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v8: 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference: https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v8: Fixed commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: Add MSR_MISC_FEATURE_ENABLES and PHIR3MWAIT bit x86/elf: Use HWCAP2 to expose ring 3 MWAIT x86/cpufeature: Add PHIR3MWAIT to CPU features x86/cpufeatures: Handle RING3MWAIT on Xeon Phi models Documentation/kernel-parameters.txt | 5 Documentation/x86/x86_64/boot-options.txt | 5 arch/x86/include/asm/cpufeatures.h| 2 +- arch/x86/include/asm/elf.h| 9 +++ arch/x86/include/asm/msr-index.h | 5 arch/x86/include/uapi/asm/hwcap2.h| 7 ++ arch/x86/kernel/cpu/common.c | 3 +++ arch/x86/kernel/cpu/intel.c | 39 +++ 8 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v8: 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference: https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v8: Fixed commit messages Removed logging Used msr_set/clear_bit functions instesd of wrmsrl Fixed documentation Renamed HWCAP2_PHIR3MWAIT to HWCAP2_RING3MWAIT v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: Add MSR_MISC_FEATURE_ENABLES and PHIR3MWAIT bit x86/elf: Use HWCAP2 to expose ring 3 MWAIT x86/cpufeature: Add PHIR3MWAIT to CPU features x86/cpufeatures: Handle RING3MWAIT on Xeon Phi models Documentation/kernel-parameters.txt | 5 Documentation/x86/x86_64/boot-options.txt | 5 arch/x86/include/asm/cpufeatures.h| 2 +- arch/x86/include/asm/elf.h| 9 +++ arch/x86/include/asm/msr-index.h | 5 arch/x86/include/uapi/asm/hwcap2.h| 7 ++ arch/x86/kernel/cpu/common.c | 3 +++ arch/x86/kernel/cpu/intel.c | 39 +++ 8 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v8: 1/4] x86/msr: Add MSR_MISC_FEATURE_ENABLES and PHIR3MWAIT bit
Intel Xeon Phi x200 (codenamed Knights Landing) allows to enable MONITOR and MWAIT instructions outside of ring 0. The feature is controlled by MSR MISC_FEATURE_ENABLES (0x140). Setting bit 1 of this register enables it, so MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when invoked outside of ring 0. The feature MSR is not yet documented in the SDM. Here is the relevant documentation: Hex Dec NameScope 140H 320 MISC_FEATURE_ENABLESThread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..0fc220d 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 +#define MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v8: 3/4] x86/cpufeature: Add PHIR3MWAIT to CPU features
Add Intel Xeon Phi x200 (KnightsLanding) cpu feature - ring 3 monitor/mwait Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..98414c5 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_PHIR3MWAIT ( 3*32+25) /* Xeon Phi x200 ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v8: 1/4] x86/msr: Add MSR_MISC_FEATURE_ENABLES and PHIR3MWAIT bit
Intel Xeon Phi x200 (codenamed Knights Landing) allows to enable MONITOR and MWAIT instructions outside of ring 0. The feature is controlled by MSR MISC_FEATURE_ENABLES (0x140). Setting bit 1 of this register enables it, so MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when invoked outside of ring 0. The feature MSR is not yet documented in the SDM. Here is the relevant documentation: Hex Dec NameScope 140H 320 MISC_FEATURE_ENABLESThread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..0fc220d 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 +#define MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v8: 3/4] x86/cpufeature: Add PHIR3MWAIT to CPU features
Add Intel Xeon Phi x200 (KnightsLanding) cpu feature - ring 3 monitor/mwait Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..98414c5 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -100,7 +100,7 @@ #define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* cpu topology enum extensions */ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ -/* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_PHIR3MWAIT ( 3*32+25) /* Xeon Phi x200 ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v7 1/4] x86/msr: Add MSR(140H) and PHIR3MWAIT bit to msr-info.h
Intel Xeon Phi x200 (codenamed Knights Landing) has MSR MISC_FEATURE_ENABLES 0x140. Setting 2nd bit of this register makes MONITOR and MWAIT instructions do not cause invalid-opcode exception when called from ring different than 0. Hex Dec NameScope 140H 320 MISC_FEATURE_ENABLESThread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..0fc220d 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 +#define MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v7 1/4] x86/msr: Add MSR(140H) and PHIR3MWAIT bit to msr-info.h
Intel Xeon Phi x200 (codenamed Knights Landing) has MSR MISC_FEATURE_ENABLES 0x140. Setting 2nd bit of this register makes MONITOR and MWAIT instructions do not cause invalid-opcode exception when called from ring different than 0. Hex Dec NameScope 140H 320 MISC_FEATURE_ENABLESThread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..0fc220d 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_MISC_FEATURE_ENABLES 0x0140 +#define MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT1 +#define MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT(1ULL << MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v7 3/4] x86/cpufeature: Add PHIR3MWAIT to CPU features
Add Intel Xeon Phi x200 (KnightsLanding) cpu feature - ring 3 monitor/mwait Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..eb88eeb 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -101,6 +101,7 @@ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ /* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_PHIR3MWAIT ( 3*32+25) /* Xeon Phi x200 ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v7 2/4] x86: Use HWCAP2 to expose Xeon Phi ring 3 MWAIT
Add HWCAP2 for x86 and reserve its 1st bit to expose Xeon Phi ring 3 monitor/mwait to userspace apps. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..90ef445 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_HWCAP2_H +#define _ASM_HWCAP2_H + +/* Kernel enabled Ring 3 MWAIT for Xeon Phi*/ +#define HWCAP2_PHIR3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v7 2/4] x86: Use HWCAP2 to expose Xeon Phi ring 3 MWAIT
Add HWCAP2 for x86 and reserve its 1st bit to expose Xeon Phi ring 3 monitor/mwait to userspace apps. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..90ef445 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_HWCAP2_H +#define _ASM_HWCAP2_H + +/* Kernel enabled Ring 3 MWAIT for Xeon Phi*/ +#define HWCAP2_PHIR3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v7 3/4] x86/cpufeature: Add PHIR3MWAIT to CPU features
Add Intel Xeon Phi x200 (KnightsLanding) cpu feature - ring 3 monitor/mwait Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..eb88eeb 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -101,6 +101,7 @@ #define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */ #define X86_FEATURE_NONSTOP_TSC( 3*32+24) /* TSC does not stop in C states */ /* free, was #define X86_FEATURE_CLFLUSH_MONITOR ( 3*32+25) * "" clflush reqd with monitor */ +#define X86_FEATURE_PHIR3MWAIT ( 3*32+25) /* Xeon Phi x200 ring 3 MONITOR/MWAIT */ #define X86_FEATURE_EXTD_APICID( 3*32+26) /* has extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */ -- 2.5.1
[PATCH v7 4/4] x86: Add enabling of the R3MWAIT during boot
If processor is Intel Xeon Phi x200 we enable user-level mwait feature. Enabling this feature suppresses invalid-opcode error, when MONITOR/MWAIT is called from ring 3. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/kernel-parameters.txt | 5 + arch/x86/kernel/cpu/intel.c | 43 + 2 files changed, 48 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..d58915b 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86] Disable Intel Xeon Phi x200 ring 3 MONITOR/MWAIT + feature for all cpus. + Format: { disable } + See arch/x86/kernel/cpu/intel.c + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..127dfdd 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,45 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + pr_warn("x86/phir3mwait: Disabled ring 3 MWAIT for Xeon Phi"); + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + u64 msr; + + /* + * Setting ring 3 MONITOR/MWAIT for each logical CPU + * return when CPU is not Xeon Phi Family x200 (KnightsLanding). + */ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + rdmsrl(MSR_MISC_FEATURE_ENABLES, msr); + + if (phi_r3mwait_disabled) { + msr &= ~MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT; + wrmsrl(MSR_MISC_FEATURE_ENABLES, msr); + } else { + msr |= MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT; + wrmsrl(MSR_MISC_FEATURE_ENABLES, msr); + set_cpu_cap(c, X86_FEATURE_PHIR3MWAIT); + ELF_HWCAP2 |= HWCAP2_PHIR3MWAIT; + } +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +606,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v7 4/4] x86: Add enabling of the R3MWAIT during boot
If processor is Intel Xeon Phi x200 we enable user-level mwait feature. Enabling this feature suppresses invalid-opcode error, when MONITOR/MWAIT is called from ring 3. Signed-off-by: Grzegorz Andrejczuk --- Documentation/kernel-parameters.txt | 5 + arch/x86/kernel/cpu/intel.c | 43 + 2 files changed, 48 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..d58915b 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86] Disable Intel Xeon Phi x200 ring 3 MONITOR/MWAIT + feature for all cpus. + Format: { disable } + See arch/x86/kernel/cpu/intel.c + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..127dfdd 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -61,6 +63,45 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + pr_warn("x86/phir3mwait: Disabled ring 3 MWAIT for Xeon Phi"); + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + u64 msr; + + /* + * Setting ring 3 MONITOR/MWAIT for each logical CPU + * return when CPU is not Xeon Phi Family x200 (KnightsLanding). + */ + if (c->x86 != 6 || c->x86_model != INTEL_FAM6_XEON_PHI_KNL) + return; + + rdmsrl(MSR_MISC_FEATURE_ENABLES, msr); + + if (phi_r3mwait_disabled) { + msr &= ~MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT; + wrmsrl(MSR_MISC_FEATURE_ENABLES, msr); + } else { + msr |= MSR_MISC_FEATURE_ENABLES_PHIR3MWAIT; + wrmsrl(MSR_MISC_FEATURE_ENABLES, msr); + set_cpu_cap(c, X86_FEATURE_PHIR3MWAIT); + ELF_HWCAP2 |= HWCAP2_PHIR3MWAIT; + } +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +606,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v7 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference: https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: Add MSR(140H) and PHIR3MWAIT bit to msr-info.h x86: Use HWCAP2 to expose Xeon Phi ring 3 MWAIT x86/cpufeature: Add PHIR3MWAIT to CPU features x86: Add enabling of the R3MWAIT during boot Documentation/kernel-parameters.txt | 5 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/elf.h | 9 arch/x86/include/asm/msr-index.h| 5 + arch/x86/include/uapi/asm/hwcap2.h | 7 ++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 43 + 7 files changed, 73 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v7 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference: https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v7: Change order of the patches, with this code looks cleaner. Changed the name of MSR to MSR_MISC_FEATURE_ENABLES. Used Word 3 25th bit to expose feature. v6: v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: Add MSR(140H) and PHIR3MWAIT bit to msr-info.h x86: Use HWCAP2 to expose Xeon Phi ring 3 MWAIT x86/cpufeature: Add PHIR3MWAIT to CPU features x86: Add enabling of the R3MWAIT during boot Documentation/kernel-parameters.txt | 5 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/elf.h | 9 arch/x86/include/asm/msr-index.h| 5 + arch/x86/include/uapi/asm/hwcap2.h | 7 ++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 43 + 7 files changed, 73 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v6 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference: https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: Add R3MWAIT register and bit to msr-info.h x86: Add enabling of the R3MWAIT during boot x86: Use HWCAP2 to expose Xeon Phi ring 3 MWAIT x86/cpufeature: Add R3MWAIT to CPU features Documentation/kernel-parameters.txt | 5 + arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/include/asm/elf.h | 9 arch/x86/include/asm/msr-index.h| 5 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 42 + 7 files changed, 73 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v6 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference: https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v5: When phir3mwait=disable is cmdline switch off r3 mwait feature Fix typos v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages v3: Included Daves and Thomas comments v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages Grzegorz Andrejczuk (4): x86/msr: Add R3MWAIT register and bit to msr-info.h x86: Add enabling of the R3MWAIT during boot x86: Use HWCAP2 to expose Xeon Phi ring 3 MWAIT x86/cpufeature: Add R3MWAIT to CPU features Documentation/kernel-parameters.txt | 5 + arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/include/asm/elf.h | 9 arch/x86/include/asm/msr-index.h| 5 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 42 + 7 files changed, 73 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v6: 3/4] x86: Use HWCAP2 to expose Xeon Phi ring 3 MWAIT
Add HWCAP2 for x86 and reserve its 1st bit to expose Xeon Phi ring 3 monitor/mwait to userspace apps. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..90ef445 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_HWCAP2_H +#define _ASM_HWCAP2_H + +/* Kernel enabled Ring 3 MWAIT for Xeon Phi*/ +#define HWCAP2_PHIR3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v6: 3/4] x86: Use HWCAP2 to expose Xeon Phi ring 3 MWAIT
Add HWCAP2 for x86 and reserve its 1st bit to expose Xeon Phi ring 3 monitor/mwait to userspace apps. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..90ef445 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_HWCAP2_H +#define _ASM_HWCAP2_H + +/* Kernel enabled Ring 3 MWAIT for Xeon Phi*/ +#define HWCAP2_PHIR3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v6: 1/4] x86/msr: Add R3MWAIT register and bit to msr-info.h
Intel Xeon Phi x200 (codenamed Knights Landing) has MSR MISC_THD_FEATURE_ENABLE 0x140. Setting 2nd bit of this register makes MONITOR and MWAIT instructions do not cause invalid-opcode exception when called from ring different than 0. Hex Dec NameScope 140H 320 MISC_THD_FEATURE_ENABLE Thread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..df9d8d3 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_PHI_MISC_THD_FEATURE 0x0140 +#define MSR_PHI_MISC_THD_FEATURE_R3MWAIT_BIT 1 +#define MSR_PHI_MISC_THD_FEATURE_R3MWAIT (1ULL << MSR_PHI_MISC_THD_FEATURE_R3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v6: 1/4] x86/msr: Add R3MWAIT register and bit to msr-info.h
Intel Xeon Phi x200 (codenamed Knights Landing) has MSR MISC_THD_FEATURE_ENABLE 0x140. Setting 2nd bit of this register makes MONITOR and MWAIT instructions do not cause invalid-opcode exception when called from ring different than 0. Hex Dec NameScope 140H 320 MISC_THD_FEATURE_ENABLE Thread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..df9d8d3 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_PHI_MISC_THD_FEATURE 0x0140 +#define MSR_PHI_MISC_THD_FEATURE_R3MWAIT_BIT 1 +#define MSR_PHI_MISC_THD_FEATURE_R3MWAIT (1ULL << MSR_PHI_MISC_THD_FEATURE_R3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v6: 4/4] x86/cpufeature: Add R3MWAIT to CPU features
Add cpu feature for ring 3 monitor/mwait. Set HWCAP2 1st bit during init. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/kernel/cpu/intel.c| 4 2 files changed, 6 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..d430200 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -71,6 +71,8 @@ #define X86_FEATURE_RECOVERY ( 2*32+ 0) /* CPU in recovery mode */ #define X86_FEATURE_LONGRUN( 2*32+ 1) /* Longrun power control */ #define X86_FEATURE_LRTI ( 2*32+ 3) /* LongRun table interface */ +/* Xeon Phi x200 ring 3 MONITOR/MWAIT enabled */ +#define X86_FEATURE_PHIR3MWAIT ( 2*32+ 4) /* Other features, Linux-defined mapping, word 3 */ /* This range is used for feature bits which conflict or are synthesized */ diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 2140ed3..3f02c7e 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -84,6 +86,8 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) } else { msr |= MSR_PHI_MISC_THD_FEATURE_R3MWAIT; wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + set_cpu_cap(c, X86_FEATURE_PHIR3MWAIT); + ELF_HWCAP2 |= HWCAP2_PHIR3MWAIT; } } -- 2.5.1
[PATCH v6: 4/4] x86/cpufeature: Add R3MWAIT to CPU features
Add cpu feature for ring 3 monitor/mwait. Set HWCAP2 1st bit during init. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/kernel/cpu/intel.c| 4 2 files changed, 6 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..d430200 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -71,6 +71,8 @@ #define X86_FEATURE_RECOVERY ( 2*32+ 0) /* CPU in recovery mode */ #define X86_FEATURE_LONGRUN( 2*32+ 1) /* Longrun power control */ #define X86_FEATURE_LRTI ( 2*32+ 3) /* LongRun table interface */ +/* Xeon Phi x200 ring 3 MONITOR/MWAIT enabled */ +#define X86_FEATURE_PHIR3MWAIT ( 2*32+ 4) /* Other features, Linux-defined mapping, word 3 */ /* This range is used for feature bits which conflict or are synthesized */ diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 2140ed3..3f02c7e 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -84,6 +86,8 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) } else { msr |= MSR_PHI_MISC_THD_FEATURE_R3MWAIT; wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + set_cpu_cap(c, X86_FEATURE_PHIR3MWAIT); + ELF_HWCAP2 |= HWCAP2_PHIR3MWAIT; } } -- 2.5.1
[PATCH v6: 2/4] x86: Add enabling of the R3MWAIT during boot
If processor is Intel Xeon Phi x200 we enable user-level mwait feature. Enabling this feature suppresses invalid-opcode error, when MONITOR/MWAIT is called from ring 3. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/kernel-parameters.txt | 5 + arch/x86/kernel/cpu/intel.c | 37 + 2 files changed, 42 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..d58915b 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86] Disable Intel Xeon Phi x200 ring 3 MONITOR/MWAIT + feature for all cpus. + Format: { disable } + See arch/x86/kernel/cpu/intel.c + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..2140ed3 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -61,6 +61,36 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + pr_warn("x86/phir3mwait: Disabled ring 3 MWAIT for Xeon Phi"); + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + u64 msr; + + rdmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + + if (phi_r3mwait_disabled) { + msr &= ~MSR_PHI_MISC_THD_FEATURE_R3MWAIT; + wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + } else { + msr |= MSR_PHI_MISC_THD_FEATURE_R3MWAIT; + wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + } +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +595,13 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + /* + * Setting ring 3 MONITOR/MWAIT for thread + * when CPU is Xeon Phi Family x200 (KnightsLanding). + */ + if (c->x86 == 6 && c->x86_model == INTEL_FAM6_XEON_PHI_KNL) + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v6: 2/4] x86: Add enabling of the R3MWAIT during boot
If processor is Intel Xeon Phi x200 we enable user-level mwait feature. Enabling this feature suppresses invalid-opcode error, when MONITOR/MWAIT is called from ring 3. Signed-off-by: Grzegorz Andrejczuk --- Documentation/kernel-parameters.txt | 5 + arch/x86/kernel/cpu/intel.c | 37 + 2 files changed, 42 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..d58915b 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86] Disable Intel Xeon Phi x200 ring 3 MONITOR/MWAIT + feature for all cpus. + Format: { disable } + See arch/x86/kernel/cpu/intel.c + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..2140ed3 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -61,6 +61,36 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + pr_warn("x86/phir3mwait: Disabled ring 3 MWAIT for Xeon Phi"); + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + u64 msr; + + rdmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + + if (phi_r3mwait_disabled) { + msr &= ~MSR_PHI_MISC_THD_FEATURE_R3MWAIT; + wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + } else { + msr |= MSR_PHI_MISC_THD_FEATURE_R3MWAIT; + wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + } +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +595,13 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + /* + * Setting ring 3 MONITOR/MWAIT for thread + * when CPU is Xeon Phi Family x200 (KnightsLanding). + */ + if (c->x86 == 6 && c->x86_model == INTEL_FAM6_XEON_PHI_KNL) + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v4 4/4] x86/phi: Add R3MWAIT to CPU features
Add cpu feature for ring 3 monitor/mwait. Set HWCAP2 1st bit during init. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/kernel/cpu/intel.c| 4 2 files changed, 6 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..d430200 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -71,6 +71,8 @@ #define X86_FEATURE_RECOVERY ( 2*32+ 0) /* CPU in recovery mode */ #define X86_FEATURE_LONGRUN( 2*32+ 1) /* Longrun power control */ #define X86_FEATURE_LRTI ( 2*32+ 3) /* LongRun table interface */ +/* Xeon Phi x200 ring 3 MONITOR/MWAIT enabled */ +#define X86_FEATURE_PHIR3MWAIT ( 2*32+ 4) /* Other features, Linux-defined mapping, word 3 */ /* This range is used for feature bits which conflict or are synthesized */ diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 1134dca..a2ea905 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -87,6 +89,8 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) rdmsrl(MSR_PHI_MISC_THD_FEATURE, msr); msr |= MSR_PHI_MISC_THD_FEATURE_R3MWAIT; wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + set_cpu_cap(c, X86_FEATURE_PHIR3MWAIT); + ELF_HWCAP2 |= HWCAP2_PHIR3MWAIT; } } -- 2.5.1
[PATCH v4 4/4] x86/phi: Add R3MWAIT to CPU features
Add cpu feature for ring 3 monitor/mwait. Set HWCAP2 1st bit during init. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/kernel/cpu/intel.c| 4 2 files changed, 6 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 92a8308..d430200 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -71,6 +71,8 @@ #define X86_FEATURE_RECOVERY ( 2*32+ 0) /* CPU in recovery mode */ #define X86_FEATURE_LONGRUN( 2*32+ 1) /* Longrun power control */ #define X86_FEATURE_LRTI ( 2*32+ 3) /* LongRun table interface */ +/* Xeon Phi x200 ring 3 MONITOR/MWAIT enabled */ +#define X86_FEATURE_PHIR3MWAIT ( 2*32+ 4) /* Other features, Linux-defined mapping, word 3 */ /* This range is used for feature bits which conflict or are synthesized */ diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 1134dca..a2ea905 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #ifdef CONFIG_X86_64 #include @@ -87,6 +89,8 @@ static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) rdmsrl(MSR_PHI_MISC_THD_FEATURE, msr); msr |= MSR_PHI_MISC_THD_FEATURE_R3MWAIT; wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + set_cpu_cap(c, X86_FEATURE_PHIR3MWAIT); + ELF_HWCAP2 |= HWCAP2_PHIR3MWAIT; } } -- 2.5.1
[PATCH v4 3/4] x86: Use HWCAP2 to expoose Xeon Phi ring 3 MWAIT
Add HWCAP2 for x86 and reserve its 1st bit to expose Xeon Phi ring 3 monitor/mwait to userspace apps. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..90ef445 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_HWCAP2_H +#define _ASM_HWCAP2_H + +/* Kernel enabled Ring 3 MWAIT for Xeon Phi*/ +#define HWCAP2_PHIR3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v4 2/4] x86/phi: Add enabling of the R3MWAIT during boot
If processor is Intel Xeon Phi we enable user-level mwait feature. Enabling this feature suppreses invalid-opcode error, when MONITOR/MWAIT is called from ring 3. Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- Documentation/kernel-parameters.txt | 5 + arch/x86/kernel/cpu/intel.c | 35 +++ 2 files changed, 40 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..d58915b 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86] Disable Intel Xeon Phi x200 ring 3 MONITOR/MWAIT + feature for all cpus. + Format: { disable } + See arch/x86/kernel/cpu/intel.c + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..1134dca 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -61,6 +61,39 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + pr_warn("x86/phir3mwait: Disabled ring 3 MWAIT for Xeon Phi"); + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + if (phi_r3mwait_disabled) + return; + + /* + * Setting ring 3 MONITOR/MWAIT for thread + * when CPU is Xeon Phi Family x200. + */ + if (c->x86 == 6 && c->x86_model == INTEL_FAM6_XEON_PHI_KNL) { + u64 msr; + + rdmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + msr |= MSR_PHI_MISC_THD_FEATURE_R3MWAIT; + wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + } +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +598,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v4 3/4] x86: Use HWCAP2 to expoose Xeon Phi ring 3 MWAIT
Add HWCAP2 for x86 and reserve its 1st bit to expose Xeon Phi ring 3 monitor/mwait to userspace apps. Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/elf.h | 9 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c | 3 +++ 3 files changed, 19 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h index e7f155c..59703aa 100644 --- a/arch/x86/include/asm/elf.h +++ b/arch/x86/include/asm/elf.h @@ -258,6 +258,15 @@ extern int force_personality32; #define ELF_HWCAP (boot_cpu_data.x86_capability[CPUID_1_EDX]) +extern unsigned int elf_hwcap2; + +/* + * HWCAP2 supplies mask with kernel enabled CPU features, so that + * the application can discover that it can safely use them. + * The bits are defined in uapi/asm/hwcap2.h. + */ +#define ELF_HWCAP2 elf_hwcap2 + /* This yields a string that ld.so will use to load implementation specific libraries for optimization. This is more specific in intent than poking at uname or /proc/cpuinfo. diff --git a/arch/x86/include/uapi/asm/hwcap2.h b/arch/x86/include/uapi/asm/hwcap2.h new file mode 100644 index 000..90ef445 --- /dev/null +++ b/arch/x86/include/uapi/asm/hwcap2.h @@ -0,0 +1,7 @@ +#ifndef _ASM_HWCAP2_H +#define _ASM_HWCAP2_H + +/* Kernel enabled Ring 3 MWAIT for Xeon Phi*/ +#define HWCAP2_PHIR3MWAIT (1 << 0) + +#endif diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bcc9ccc..fdbf708 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ #include "cpu.h" +unsigned int elf_hwcap2 __read_mostly; + /* all of these masks are initialized in setup_cpu_local_masks() */ cpumask_var_t cpu_initialized_mask; cpumask_var_t cpu_callout_mask; -- 2.5.1
[PATCH v4 2/4] x86/phi: Add enabling of the R3MWAIT during boot
If processor is Intel Xeon Phi we enable user-level mwait feature. Enabling this feature suppreses invalid-opcode error, when MONITOR/MWAIT is called from ring 3. Signed-off-by: Grzegorz Andrejczuk --- Documentation/kernel-parameters.txt | 5 + arch/x86/kernel/cpu/intel.c | 35 +++ 2 files changed, 40 insertions(+) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index a4f4d69..d58915b 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3120,6 +3120,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted. pg. [PARIDE] See Documentation/blockdev/paride.txt. + phir3mwait= [X86] Disable Intel Xeon Phi x200 ring 3 MONITOR/MWAIT + feature for all cpus. + Format: { disable } + See arch/x86/kernel/cpu/intel.c + pirq= [SMP,APIC] Manual mp-table setup See Documentation/x86/i386/IO-APIC.txt. diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index fcd484d..1134dca 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -61,6 +61,39 @@ void check_mpx_erratum(struct cpuinfo_x86 *c) } } +#ifdef CONFIG_X86_64 +static int phi_r3mwait_disabled __read_mostly; + +static int __init phir3mwait_disable(char *__unused) +{ + phi_r3mwait_disabled = 1; + pr_warn("x86/phir3mwait: Disabled ring 3 MWAIT for Xeon Phi"); + return 1; +} +__setup("phir3mwait=disable", phir3mwait_disable); + +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *c) +{ + if (phi_r3mwait_disabled) + return; + + /* + * Setting ring 3 MONITOR/MWAIT for thread + * when CPU is Xeon Phi Family x200. + */ + if (c->x86 == 6 && c->x86_model == INTEL_FAM6_XEON_PHI_KNL) { + u64 msr; + + rdmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + msr |= MSR_PHI_MISC_THD_FEATURE_R3MWAIT; + wrmsrl(MSR_PHI_MISC_THD_FEATURE, msr); + } +} + +#else +static void probe_xeon_phi_r3mwait(struct cpuinfo_x86 *__unused) {} +#endif + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; @@ -565,6 +598,8 @@ static void init_intel(struct cpuinfo_x86 *c) detect_vmx_virtcap(c); init_intel_energy_perf(c); + + probe_xeon_phi_r3mwait(c); } #ifdef CONFIG_X86_32 -- 2.5.1
[PATCH v4 1/4] x86/phi: Add R3MWAIT register and bit to msr-info.h
Intel Xeon Phi x200 (codenamed Knights Landing) has MSR MISC_THD_FEATURE_ENABLE 0x140. Setting 2nd bit of this register makes MONITOR and MWAIT instructions do not cause invalid-opcode exception when called from ring different than 0. Hex Dec NameScope 140H 320 MISC_THD_FEATURE_ENABLE Thread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk <grzegorz.andrejc...@intel.com> --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..df9d8d3 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_PHI_MISC_THD_FEATURE 0x0140 +#define MSR_PHI_MISC_THD_FEATURE_R3MWAIT_BIT 1 +#define MSR_PHI_MISC_THD_FEATURE_R3MWAIT (1ULL << MSR_PHI_MISC_THD_FEATURE_R3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v4 1/4] x86/phi: Add R3MWAIT register and bit to msr-info.h
Intel Xeon Phi x200 (codenamed Knights Landing) has MSR MISC_THD_FEATURE_ENABLE 0x140. Setting 2nd bit of this register makes MONITOR and MWAIT instructions do not cause invalid-opcode exception when called from ring different than 0. Hex Dec NameScope 140H 320 MISC_THD_FEATURE_ENABLE Thread 0Reserved 1if set to 1, the MONITOR and MWAIT instructions do not cause invalid-opcode exceptions when executed with CPL > 0 or in virtual-8086 mode. If MWAIT is executed when CPL > 0 or in virtual-8086 mode, and if EAX indicates a C-state other than C0 or C1, the instruction operates as if EAX indicated the C-state C1. 63:2 Reserved Signed-off-by: Grzegorz Andrejczuk --- arch/x86/include/asm/msr-index.h | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 56f4c66..df9d8d3 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -540,6 +540,11 @@ #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT 39 #define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE (1ULL << MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE_BIT) +/* Intel Xeon Phi x200 ring 3 MONITOR/MWAIT */ +#define MSR_PHI_MISC_THD_FEATURE 0x0140 +#define MSR_PHI_MISC_THD_FEATURE_R3MWAIT_BIT 1 +#define MSR_PHI_MISC_THD_FEATURE_R3MWAIT (1ULL << MSR_PHI_MISC_THD_FEATURE_R3MWAIT_BIT) + #define MSR_IA32_TSC_DEADLINE 0x06E0 /* P4/Xeon+ specific */ -- 2.5.1
[PATCH v4 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference (the solution is temporary MSR definition will be in next SDM document): https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages v3: Included Daves and Thomas comments v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages Grzegorz Andrejczuk (4): x86/phi: Add R3MWAIT register and bit to msr-info.h x86/phi: Add enabling of the R3MWAIT during boot x86: Use HWCAP2 to expoose Xeon Phi ring 3 MWAIT x86/phi: Add R3MWAIT to CPU features Documentation/kernel-parameters.txt | 5 + arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/include/asm/elf.h | 9 + arch/x86/include/asm/msr-index.h| 5 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 40 + 7 files changed, 71 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1
[PATCH v4 0/4] Enabling Ring 3 MONITOR/MWAIT feature for Knights Landing
These patches enable Intel Xeon Phi x200 feature to use MONITOR/MWAIT instruction in ring 3 (userspace) Patches set MSR 0x140 for all logical CPUs. Then expose it as CPU feature and introduces elf HWCAP capability for x86. Reference (the solution is temporary MSR definition will be in next SDM document): https://software.intel.com/en-us/blogs/2016/10/06/intel-xeon-phi-product-family-x200-knl-user-mode-ring-3-monitor-and-mwait v2: Check MSR before wrmsrl Shortened names Used Word 3 for feature init_scattered_cpuid_features() Fixed commit messages v3: Included Daves and Thomas comments v4: Wrapped the enabling code by CONFIG_X86_64 Add documentation for phir3mwait=disable cmdline switch Move probe_ function call from early_intel_init to intel_init Fixed commit messages Grzegorz Andrejczuk (4): x86/phi: Add R3MWAIT register and bit to msr-info.h x86/phi: Add enabling of the R3MWAIT during boot x86: Use HWCAP2 to expoose Xeon Phi ring 3 MWAIT x86/phi: Add R3MWAIT to CPU features Documentation/kernel-parameters.txt | 5 + arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/include/asm/elf.h | 9 + arch/x86/include/asm/msr-index.h| 5 + arch/x86/include/uapi/asm/hwcap2.h | 7 +++ arch/x86/kernel/cpu/common.c| 3 +++ arch/x86/kernel/cpu/intel.c | 40 + 7 files changed, 71 insertions(+) create mode 100644 arch/x86/include/uapi/asm/hwcap2.h -- 2.5.1