[RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
We have eager and lazy fpu modes, introduced in: 304bceda6a18 x86, fpu: use non-lazy fpu restore for processors supporting xsave The result is rather messy. There are two code paths in almost all of the FPU code, and only one of them (the eager case) is tested frequently, since most kernel developers have new enough hardware that we use eagerfpu. It seems that, on any remotely recent hardware, eagerfpu is a win: glibc uses SSE2, so laziness is probably overoptimistic, and, in any case, manipulating TS is far slower that saving and restoring the full state. To try to shake out any latent issues on old hardware, this changes the default to eager on all CPUs. If no performance or functionality problems show up, a subsequent patch could remove lazy mode entirely. Signed-off-by: Andy Lutomirski --- arch/x86/kernel/xsave.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/xsave.c b/arch/x86/kernel/xsave.c index 0de1fae2bdf0..1928c8f34ce5 100644 --- a/arch/x86/kernel/xsave.c +++ b/arch/x86/kernel/xsave.c @@ -637,8 +637,8 @@ static void __init xstate_enable_boot_cpu(void) prepare_fx_sw_frame(); setup_init_fpu_buf(); - /* Auto enable eagerfpu for xsaveopt */ - if (cpu_has_xsaveopt && eagerfpu != DISABLE) + /* Auto enable eagerfpu for everyone */ + if (eagerfpu != DISABLE) eagerfpu = ENABLE; if (pcntxt_mask & XSTATE_EAGER) { -- 2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Tue, Feb 24, 2015 at 11:15 AM, Denys Vlasenko wrote: > On Fri, Feb 20, 2015 at 7:58 PM, Andy Lutomirski wrote: >> We have eager and lazy fpu modes, introduced in: >> >> 304bceda6a18 x86, fpu: use non-lazy fpu restore for processors supporting >> xsave >> >> The result is rather messy. There are two code paths in almost all of the >> FPU code, and only one of them (the eager case) is tested frequently, since >> most kernel developers have new enough hardware that we use eagerfpu. >> >> It seems that, on any remotely recent hardware, eagerfpu is a win: >> glibc uses SSE2, so laziness is probably overoptimistic, and, in any >> case, manipulating TS is far slower that saving and restoring the full >> state. >> >> To try to shake out any latent issues on old hardware, this changes >> the default to eager on all CPUs. If no performance or functionality >> problems show up, a subsequent patch could remove lazy mode entirely. > > I'm a big fan of simplifying things, but. > > SIMD registers were growing in x86, and they are going to grow again, > this time four-fold in Intel MIC: > from sixteen 256-bit registers to thirty two 512-bit registers. > > That's 2 kbytes of data. Just moving this data out to/from memory > will take some time. > > And some people talk about 1024-bit registers already... > > Let's not completely remove lazy FPU saving code just yet. > Maybe we'll be forced to reinstate it. I'd prefer a different partial solution: encourage everyone to clear the xstate before making syscalls (using e.g. vzeroall). In fact, maybe user code should aggressively clear newly-unused xstate. --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Tue, Feb 24, 2015 at 04:07:07PM -0800, Andy Lutomirski wrote: > I'd prefer a different partial solution: encourage everyone to clear > the xstate before making syscalls (using e.g. vzeroall). In fact, > maybe user code should aggressively clear newly-unused xstate. We don't trust userspace. -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Andy Lutomirski wrote: > > I'm a big fan of simplifying things, but. > > > > SIMD registers were growing in x86, and they are going > > to grow again, this time four-fold in Intel MIC: from > > sixteen 256-bit registers to thirty two 512-bit > > registers. > > > > That's 2 kbytes of data. Just moving this data out > > to/from memory will take some time. > > > > And some people talk about 1024-bit registers > > already... > > > > Let's not completely remove lazy FPU saving code just > > yet. Maybe we'll be forced to reinstate it. > > I'd prefer a different partial solution: encourage > everyone to clear the xstate before making syscalls > (using e.g. vzeroall). In fact, maybe user code should > aggressively clear newly-unused xstate. Also, xstate has various compaction features and could grow new ones in the future as well, should the xsave area become overly sparse: see xstate_comp_*[] et al in arch/x86/kernel/xsave.c. This is the better, hardware driven, synchronous alternative to lazy, async register state save/restore, as it gets us similar benefits of not saving/restoring unused space, but avoids any async trap overhead. Also ... with more and wider vector CPU registers context switches between different sets of registers are going to be inevitably more expensive, no matter what. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Borislav Petkov wrote: > On Tue, Feb 24, 2015 at 04:07:07PM -0800, Andy Lutomirski wrote: > > > I'd prefer a different partial solution: encourage > > everyone to clear the xstate before making syscalls > > (using e.g. vzeroall). In fact, maybe user code should > > aggressively clear newly-unused xstate. > > We don't trust userspace. We certainly don't, but in this case it's a performance optimization detail: if user-space clears unused xstate in a way that the CPU recognizes it (for example VZEROALL) then it might get faster context switches. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
+ Linus. I'm sure he'll save something to say about it :-) On Fri, Feb 20, 2015 at 10:58:15AM -0800, Andy Lutomirski wrote: > We have eager and lazy fpu modes, introduced in: > > 304bceda6a18 x86, fpu: use non-lazy fpu restore for processors supporting > xsave > > The result is rather messy. There are two code paths in almost all of the > FPU code, and only one of them (the eager case) is tested frequently, since > most kernel developers have new enough hardware that we use eagerfpu. > > It seems that, on any remotely recent hardware, eagerfpu is a win: > glibc uses SSE2, so laziness is probably overoptimistic, and, in any > case, manipulating TS is far slower that saving and restoring the full > state. > > To try to shake out any latent issues on old hardware, this changes > the default to eager on all CPUs. If no performance or functionality > problems show up, a subsequent patch could remove lazy mode entirely. > > Signed-off-by: Andy Lutomirski > --- > arch/x86/kernel/xsave.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/arch/x86/kernel/xsave.c b/arch/x86/kernel/xsave.c > index 0de1fae2bdf0..1928c8f34ce5 100644 > --- a/arch/x86/kernel/xsave.c > +++ b/arch/x86/kernel/xsave.c > @@ -637,8 +637,8 @@ static void __init xstate_enable_boot_cpu(void) > prepare_fx_sw_frame(); > setup_init_fpu_buf(); > > - /* Auto enable eagerfpu for xsaveopt */ > - if (cpu_has_xsaveopt && eagerfpu != DISABLE) > + /* Auto enable eagerfpu for everyone */ > + if (eagerfpu != DISABLE) > eagerfpu = ENABLE; > > if (pcntxt_mask & XSTATE_EAGER) { > -- > 2.1.0 > > -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Andy Lutomirski wrote: > We have eager and lazy fpu modes, introduced in: > > 304bceda6a18 x86, fpu: use non-lazy fpu restore for processors supporting > xsave > > The result is rather messy. There are two code paths in > almost all of the FPU code, and only one of them (the > eager case) is tested frequently, since most kernel > developers have new enough hardware that we use eagerfpu. > > It seems that, on any remotely recent hardware, eagerfpu > is a win: glibc uses SSE2, so laziness is probably > overoptimistic, and, in any case, manipulating TS is far > slower that saving and restoring the full state. > > To try to shake out any latent issues on old hardware, > this changes the default to eager on all CPUs. If no > performance or functionality problems show up, a > subsequent patch could remove lazy mode entirely. So it would be nice to test this on at least one reasonably old (but not uncomfortably old - say 5 years old) system, to get a feel for what kind of performance impact it has there. But yes, this would enable a nice simplification in the end so I'm all for it as long as it doesn't cause unacceptable problems - and the FPU code needs simplification badly, because the current latency of bug discovery is too high IMO. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sat, Feb 21, 2015 at 10:31:50AM +0100, Ingo Molnar wrote: > So it would be nice to test this on at least one reasonably old (but > not uncomfortably old - say 5 years old) system, to get a feel for > what kind of performance impact it has there. Yeah, this is exactly what Andy and I were talking about yesterday on IRC. So let's measure our favourite workload - the kernel build! :-) My assumption is that libc uses SSE for memcpy and thus the FPU will be used. (I'll trace FPU-specific PMCs later to confirm). Machine is an AMD F10h which should be 5-10 years old depending on what you're looking at (uarch, revision, ...). Numbers look great to me in the sense that we have a very small improvement and the rest stays the same. Which would mean, killing lazy FPU does not bring slowdown, if no improvement, but will bring a huuge improvement in code quality and the handling of the FPU state by getting rid of the lazyness... IPC is the same, branch misses are *down* a bit, cache misses go up a bit probably because we're shuffling FPU state more often to mem, page faults go down and runtime goes down by half a second: plain 3.19: == perf stat -a -e task-clock,cycles,instructions,branch-misses,cache-misses,faults,context-switches,migrations --repeat 10 --sync --pre ~/bin/pre-build-kernel.sh make -s -j12 Performance counter stats for 'system wide' (10 runs): 1408897.576594 task-clock (msec) #6.003 CPUs utilized ( +- 0.15% ) [100.00%] 3,137,565,760,188 cycles#2.227 GHz ( +- 0.02% ) [100.00%] 2,849,228,161,721 instructions #0.91 insns per cycle ( +- 0.00% ) [100.00%] 32,391,188,891 branch-misses # 22.990 M/sec ( +- 0.02% ) [100.00%] 27,879,813,595 cache-misses # 19.788 M/sec ( +- 0.01% ) 27,195,402 faults#0.019 M/sec ( +- 0.01% ) [100.00%] 1,293,241 context-switches #0.918 K/sec ( +- 0.09% ) [100.00%] 69,548 migrations#0.049 K/sec ( +- 0.22% ) 234.681331200 seconds time elapsed ( +- 0.15% ) eagerfpu=ENABLE === Performance counter stats for 'system wide' (10 runs): 1405208.771580 task-clock (msec) #6.003 CPUs utilized ( +- 0.19% ) [100.00%] 3,137,381,829,748 cycles#2.233 GHz ( +- 0.03% ) [100.00%] 2,849,059,336,718 instructions #0.91 insns per cycle ( +- 0.00% ) [100.00%] 32,380,999,636 branch-misses # 23.044 M/sec ( +- 0.02% ) [100.00%] 27,884,281,327 cache-misses # 19.844 M/sec ( +- 0.01% ) 27,193,985 faults#0.019 M/sec ( +- 0.01% ) [100.00%] 1,293,300 context-switches #0.920 K/sec ( +- 0.08% ) [100.00%] 69,791 migrations#0.050 K/sec ( +- 0.18% ) 234.066525648 seconds time elapsed ( +- 0.19% ) -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sat, Feb 21, 2015 at 05:38:40PM +0100, Borislav Petkov wrote: > My assumption is that libc uses SSE for memcpy and thus the FPU will > be used. (I'll trace FPU-specific PMCs later to confirm). Ok, so I slapped a trace_printk() at the beginning of fpu_save_init() and did a kernel build once with default 3.19 and once with Andy's patch. So we end up with this count: default: 712000 eager: 78 This would explain the very small difference in the performance counters data from the previous email. Provided I've not made a mistake, this leads me to think that this simple workload and pretty much everything else uses the FPU through glibc which does the SSE memcpy and so on. Which basically kills the whole idea behind lazy FPU as practically you don't really encounter workloads nowadays which don't use the FPU thanks to glibc and the lazy strategy doesn't really bring anything. Which would then mean, we don't really need the lazy handling as userspace is making it eager, so to speak, for us. Thoughts? -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Borislav Petkov wrote: > plain 3.19: > > 234.681331200 seconds time elapsed >( +- 0.15% ) > > eagerfpu=ENABLE > > 234.066525648 seconds time elapsed >( +- 0.19% ) hm, a win of more than 600 milliseconds - more than I'd have expected. I really want to believe this, but I worry about the systematic boot-to-boot noise though, which cannot be eliminated via --repeat 10. Would it be possible to add a simple runtime debug switch to switch between the two FPU modes dynamically via a sysctl? It should not crash tasks I think if it's flipped around on a reasonably idle system, so should allow much better apples-to-apples comparisons with the same kind of page cache layout, etc. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Borislav Petkov wrote: > On Sat, Feb 21, 2015 at 05:38:40PM +0100, Borislav Petkov wrote: > > My assumption is that libc uses SSE for memcpy and thus the FPU will > > be used. (I'll trace FPU-specific PMCs later to confirm). > > Ok, so I slapped a trace_printk() at the beginning of fpu_save_init() > and did a kernel build once with default 3.19 and once with Andy's > patch. So we end up with this count: > > default: 712000 > eager: 78 > > This would explain the very small difference in the > performance counters data from the previous email. So the workload improved by ~600,000 usecs, and there's 68,000 less calls, so it saved 8.8 usecs per call. Isn't that a bit too high? I'd sleep a lot better if we had some runtime debug flag to be able to do run-to-run comparisons on the same booted up kernel, or so. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sat, Feb 21, 2015 at 07:39:52PM +0100, Ingo Molnar wrote: > So the workload improved by ~600,000 usecs, and there's > 68,000 less calls, so it saved 8.8 usecs per call. Isn't I think you mean more calls. The eager measurement has more calls. Let me do some primitive math: def =(234.681331200 / 712000)*10^6 = 329.6086112359550500 microsecs/call eager=(234.066525648 / 78)*10^6 = 300.0852892923076900 microsecs/call diff is 29.5233219436473600 microsecs speedup per call which could explain the cost of CR0.TS serialization semantics in the lazy mode. > that a bit too high? Now, is 29 microseconds too high? I'm not sure this is even correct and not some noise interfering. > I'd sleep a lot better if we had some runtime debug flag to > be able to do run-to-run comparisons on the same booted up > kernel, or so. Let me take a look whether we could so some knob... The nice thing is, code uses use_eager_fpu() to check stuff so we should be able to clear the cpufeature flag. -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Borislav Petkov wrote: > > I'd sleep a lot better if we had some runtime debug > > flag to be able to do run-to-run comparisons on the > > same booted up kernel, or so. > > Let me take a look whether we could so some knob... The > nice thing is, code uses use_eager_fpu() to check stuff > so we should be able to clear the cpufeature flag. Either that, or you could quickly hack those checks to use panic_timeout, add an 'extern int panic_timeout;' and after bootup do: echo 0 > /proc/sys/kernel/panic echo 1 > /proc/sys/kernel/panic to switch between the modes? Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sat, Feb 21, 2015 at 08:23:52PM +0100, Ingo Molnar wrote: > to switch between the modes? I went all out and did a debugfs file, see patch at the end, which counts FPU saves. Then I ran this script: --- #!/bin/bash D="/sys/kernel/debug/fpu/eager" echo "Lazy FPU: " echo 0 > $D echo -n " FPU saves before: "; cat $D perf stat -a -e task-clock,cycles,instructions,branch-misses,cache-misses,faults,context-switches,migrations --sync --pre ~/bin/pre-build-kernel.sh make -s -j12 echo -n " FPU saves after: "; cat $D echo "" echo "Eager FPU: " echo 1 > $D echo -n " FPU saves before: "; cat $D perf stat -a -e task-clock,cycles,instructions,branch-misses,cache-misses,faults,context-switches,migrations --sync --pre ~/bin/pre-build-kernel.sh make -s -j12 echo -n " FPU saves after: "; cat $D --- which spit this: Lazy FPU: FPU saves before: 3 Setup is 16252 bytes (padded to 16384 bytes). System is 4222 kB CRC c79a13ab Kernel: arch/x86/boot/bzImage is ready (#41) Performance counter stats for 'system wide': 1315527.989020 task-clock (msec) #6.003 CPUs utilized [100.00%] 3,042,312,057,208 cycles#2.313 GHz [100.00%] 2,790,807,863,402 instructions #0.92 insns per cycle [100.00%] 31,658,299,111 branch-misses # 24.065 M/sec [100.00%] 27,504,255,277 cache-misses # 20.907 M/sec 26,802,015 faults#0.020 M/sec [100.00%] 1,248,899 context-switches #0.949 K/sec [100.00%] 69,553 migrations#0.053 K/sec 219.127929718 seconds time elapsed FPU saves after: 704186 Eager FPU: FPU saves before: 4 Setup is 16252 bytes (padded to 16384 bytes). System is 4222 kB CRC 6767bb2e Kernel: arch/x86/boot/bzImage is ready (#42) Performance counter stats for 'system wide': 1321651.543922 task-clock (msec) #6.003 CPUs utilized [100.00%] 3,044,403,437,364 cycles#2.303 GHz [100.00%] 2,790,835,886,565 instructions #0.92 insns per cycle [100.00%] 31,638,090,259 branch-misses # 23.938 M/sec [100.00%] 27,491,643,095 cache-misses # 20.801 M/sec 26,869,732 faults#0.020 M/sec [100.00%] 1,252,034 context-switches #0.947 K/sec [100.00%] 69,247 migrations#0.052 K/sec 220.148034331 seconds time elapsed FPU saves after: 901638 --- so we have a second slowdown and 200K FPU saves more in eager mode. Provided I've not done a mistake, looks like the increase in cycles gets mirrored in 1 second time longer. I've not done the --repeat 10 thing again, maybe I should do it too, just to be fair as this is a single run. --- arch/x86/include/asm/fpu-internal.h | 4 arch/x86/kernel/xsave.c | 47 - 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/fpu-internal.h b/arch/x86/include/asm/fpu-internal.h index e97622f57722..7141f353e960 100644 --- a/arch/x86/include/asm/fpu-internal.h +++ b/arch/x86/include/asm/fpu-internal.h @@ -38,6 +38,8 @@ int ia32_setup_frame(int sig, struct ksignal *ksig, # define ia32_setup_rt_frame __setup_rt_frame #endif + +extern unsigned long fpu_saved; extern unsigned int mxcsr_feature_mask; extern void fpu_init(void); extern void eager_fpu_init(void); @@ -242,6 +244,8 @@ static inline void fpu_fxsave(struct fpu *fpu) */ static inline int fpu_save_init(struct fpu *fpu) { + fpu_saved++; + if (use_xsave()) { fpu_xsave(fpu); diff --git a/arch/x86/kernel/xsave.c b/arch/x86/kernel/xsave.c index 0de1fae2bdf0..029de8b629d0 100644 --- a/arch/x86/kernel/xsave.c +++ b/arch/x86/kernel/xsave.c @@ -14,6 +14,8 @@ #include #include +#include + /* * Supported feature mask by the CPU and the kernel. */ @@ -638,7 +640,7 @@ static void __init xstate_enable_boot_cpu(void) setup_init_fpu_buf(); /* Auto enable eagerfpu for xsaveopt */ - if (cpu_has_xsaveopt && eagerfpu != DISABLE) + if (eagerfpu != DISABLE) eagerfpu = ENABLE; if (pcntxt_mask & XSTATE_EAGER) { @@ -739,3 +741,46 @@ void *get_xsave_addr(struct xsave_struct *xsave, int xstate) return (void *)xsave + xstate_comp_offsets[feature]; } EXPORT_SYMBOL_GPL(get_xsave_addr); + +unsigned long fpu_saved; + +static int eager_get(void *data, u64 *val) +{ + *val = fpu_saved; + + return 0; +} + +static int eager_set(void *data, u64 val) +{ + if (val) + setup_force_cpu_cap(X86_FEATURE_EAGER_FPU); + else
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sat, 21 Feb 2015, Borislav Petkov wrote: > Provided I've not made a mistake, this leads me to think that this > simple workload and pretty much everything else uses the FPU through > glibc which does the SSE memcpy and so on. Which basically kills the > whole idea behind lazy FPU as practically you don't really encounter > workloads nowadays which don't use the FPU thanks to glibc and the lazy > strategy doesn't really bring anything. > > Which would then mean, we don't really need the lazy handling as > userspace is making it eager, so to speak, for us. Please correct me if I'm wrong, but it looks to me like you're confusing lazy FPU context allocation and lazy FPU context switching. These build on the same hardware principles, but they are different concepts. Your "userspace is making it eager" statement in the context of glibc using SSE for `memcpy' is certainly true for lazy FPU context allocation, however I wouldn't be so sure about lazy FPU context switching, and a kernel compilation (or in fact any compilation) does not appear to be a representative benchmark to me. I am sure lots of software won't be calling `memcpy' all the time, there should be context switches between which the FPU is not referred to at all. Also, does `__builtin_memcpy' also expand to SSE? I'd expect it rather than external `memcpy' to be used by GCC for copying fixed amounts of data, especially smaller ones such as when passing structures by value in function calls or for string operations like `strdup' or suchlike. These I'd expect to be ubiquitous, whereas external `memcpy' I'd expect to be called from time to time only. Additionally I believe long-executing FPU instructions (i.e. transcendentals) can take advantage of continuing to execute in parallel where the context has already been switched rather than stalling an eager FPU context switch until the FPU instruction has completed. And last but not least, why does the handling of CR0.TS traps have to be complicated? It does not look like rocket science to me, it should be a mere handful of instructions, the time required to move the two FP contexts out from and in to the FPU respectively should dominate processing time. Where quoted the optimisation manual states 250 cycles for FXSAVE and FXRSTOR combined. And of course you can install the right handler (i.e. FSAVE vs FXSAVE) at bootstrap depending on processor features, you don't have to do all the run-time check on every trap. You can even optimise the FSAVE handler away at the build time if you know it won't ever be used based on the minimal supported processor family selected. Do you happen to know or can determine how much time (in clock cycles) a CR0.TS trap itself takes, including any time required to preserve the execution state in the handler such as pushing/popping GPRs to/from the stack (as opposed to processing time spent on moving the FP contexts back and forth)? Is there no room for improvement left there? How many task scheduling slots say per million must be there poking at the FPU for eager FPU context switching to take advantage over lazy one? Maciej -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sat, Feb 21, 2015 at 4:34 PM, Maciej W. Rozycki wrote: > On Sat, 21 Feb 2015, Borislav Petkov wrote: > >> Provided I've not made a mistake, this leads me to think that this >> simple workload and pretty much everything else uses the FPU through >> glibc which does the SSE memcpy and so on. Which basically kills the >> whole idea behind lazy FPU as practically you don't really encounter >> workloads nowadays which don't use the FPU thanks to glibc and the lazy >> strategy doesn't really bring anything. >> >> Which would then mean, we don't really need the lazy handling as >> userspace is making it eager, so to speak, for us. > > Please correct me if I'm wrong, but it looks to me like you're confusing > lazy FPU context allocation and lazy FPU context switching. These build > on the same hardware principles, but they are different concepts. > > Your "userspace is making it eager" statement in the context of glibc > using SSE for `memcpy' is certainly true for lazy FPU context allocation, > however I wouldn't be so sure about lazy FPU context switching, and a > kernel compilation (or in fact any compilation) does not appear to be a > representative benchmark to me. I am sure lots of software won't be > calling `memcpy' all the time, there should be context switches between > which the FPU is not referred to at all. That's true. The question is whether there are enough of them, and whether twiddling TS is fast enough, that it's worth it. > > Additionally I believe long-executing FPU instructions (i.e. > transcendentals) can take advantage of continuing to execute in parallel > where the context has already been switched rather than stalling an eager > FPU context switch until the FPU instruction has completed. It seems highly unlikely to me that a slow FPU instruction can retire *after* a subsequent fxsave, which would need to happen for this to work. > > And last but not least, why does the handling of CR0.TS traps have to be > complicated? It does not look like rocket science to me, it should be a > mere handful of instructions, the time required to move the two FP > contexts out from and in to the FPU respectively should dominate > processing time. Where quoted the optimisation manual states 250 cycles > for FXSAVE and FXRSTOR combined. The TS traps aren't complicated -- they're just really slow. I think that each of setting *and* clearing TS serializes and takes well over a hundred cycles. A #NM interrupt (the thing that happens if you try to use the FPU with TS set) serializes and does all kinds of slow things, so it takes many hundreds of cycles. The handler needs to clear TS (another hundred cycles or more), load the FPU state (actually rather fast on modern CPUs), and then IRET back to userspace (hundreds of cycles). This adds up to a lot of cycles. A round trip through an exception handler seems to be thousands of cycles. > > And of course you can install the right handler (i.e. FSAVE vs FXSAVE) at > bootstrap depending on processor features, you don't have to do all the > run-time check on every trap. You can even optimise the FSAVE handler > away at the build time if you know it won't ever be used based on the > minimal supported processor family selected. None of this matters. A couple of branches in an #NM handler are completely lost in the noise. > > Do you happen to know or can determine how much time (in clock cycles) a > CR0.TS trap itself takes, including any time required to preserve the > execution state in the handler such as pushing/popping GPRs to/from the > stack (as opposed to processing time spent on moving the FP contexts back > and forth)? Is there no room for improvement left there? How many task > scheduling slots say per million must be there poking at the FPU for eager > FPU context switching to take advantage over lazy one? Thousands of cycles. Considerably worse in an SVM guest. x86 exception handling sucks. --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Borislav Petkov wrote: > which spit this: > > Lazy FPU: > 219.127929718 seconds time elapsed > Eager FPU: > 220.148034331 seconds time elapsed > so we have a second slowdown and 200K FPU saves more in eager mode. So am I interpreting the older and your latest numbers correctly in stating that the cost observation has flipped around 180 degrees: the first measurement showed eager FPU to be a win, but now that we can do more precise measurements, eager FPU has actually slowed down the kernel build by ~0.5%? That's not good, and kernel builds are just a random load that isn't even that FPU or context switch heavy - there will certainly be other loads that would be hurt even more. So just before we base wide reaching decisions based on any of these measurements, would you mind help us increase our confidence in the numbers some more: - It might make sense to do a 'perf stat --null --repeat' measurement as well [without any -e arguments], to make sure the rich PMU stats you are gathering are not interfering? With 'perf stat --null --repeat' perf acts essenially as a /usr/bin/time replacement, but can measure down to microseconds and will calculate noise/sttdev properly. - Perhaps also double check the debug switch: is it really properly switching FPU handling mode? - Do you have enough RAM that there's essentially no IO in the system worth speaking of? Do you have enough RAM to copy a whole kernel tree to /tmp/linux/ and do the measurement there, on ramfs? Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Ingo Molnar wrote: > - Do you have enough RAM that there's essentially no IO > in the system worth speaking of? Do you have enough RAM > to copy a whole kernel tree to /tmp/linux/ and do the > measurement there, on ramfs? Doing that will also pin down the page cache: kernel build times are very sensitive to the page cache layout, and once a page cache layout is established on systems with lots of RAM it does not tend to be flushed out. But the next bootup will generate another random page cache layout - which makes inter-kernel kernel build times comparisons much noiser than the run-to-run numbers suggest. So to get more precise measurements a 'pinned' page cache layout and the dynamic debug switch you implemented is very helpful and --repeat stddev will be pretty representative of the true noise of the measurement. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sun, Feb 22, 2015 at 09:18:40AM +0100, Ingo Molnar wrote: > So am I interpreting the older and your latest numbers > correctly in stating that the cost observation has flipped > around 180 degrees: the first measurement showed eager FPU > to be a win, but now that we can do more precise > measurements, eager FPU has actually slowed down the kernel > build by ~0.5%? Well, I wouldn't take the latest numbers too seriously - that was a single run without --repeat. > That's not good, and kernel builds are just a random load > that isn't even that FPU or context switch heavy - there > will certainly be other loads that would be hurt even more. That is my fear. > So just before we base wide reaching decisions based on any > of these measurements, would you mind help us increase our > confidence in the numbers some more: > > - It might make sense to do a 'perf stat --null --repeat' > measurement as well [without any -e arguments], to make > sure the rich PMU stats you are gathering are not > interfering? > > With 'perf stat --null --repeat' perf acts essenially > as a /usr/bin/time replacement, but can measure down to > microseconds and will calculate noise/sttdev properly. Cool, let me do that. > - Perhaps also double check the debug switch: is it > really properly switching FPU handling mode? I've changed the use_eager_fpu() test to do: static __always_inline __pure bool use_eager_fpu(void) { return boot_cpu_has(X86_FEATURE_EAGER_FPU); } and I'm clearing/setting eager FPU with setup_force_cpu_cap/setup_clear_cpu_cap, see full diff below. > - Do you have enough RAM that there's essentially no IO > in the system worth speaking of? Do you have enough RAM > to copy a whole kernel tree to /tmp/linux/ and do the > measurement there, on ramfs? /proc/meminfo says "MemTotal: 4011860 kB" which is probably not enough. But I could find one somewhere :-) --- arch/x86/include/asm/fpu-internal.h | 6 +++- arch/x86/kernel/xsave.c | 57 - 2 files changed, 61 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/fpu-internal.h b/arch/x86/include/asm/fpu-internal.h index e97622f57722..c8a161d02056 100644 --- a/arch/x86/include/asm/fpu-internal.h +++ b/arch/x86/include/asm/fpu-internal.h @@ -38,6 +38,8 @@ int ia32_setup_frame(int sig, struct ksignal *ksig, # define ia32_setup_rt_frame __setup_rt_frame #endif + +extern unsigned long fpu_saved; extern unsigned int mxcsr_feature_mask; extern void fpu_init(void); extern void eager_fpu_init(void); @@ -87,7 +89,7 @@ static inline int is_x32_frame(void) static __always_inline __pure bool use_eager_fpu(void) { - return static_cpu_has_safe(X86_FEATURE_EAGER_FPU); + return boot_cpu_has(X86_FEATURE_EAGER_FPU); } static __always_inline __pure bool use_xsaveopt(void) @@ -242,6 +244,8 @@ static inline void fpu_fxsave(struct fpu *fpu) */ static inline int fpu_save_init(struct fpu *fpu) { + fpu_saved++; + if (use_xsave()) { fpu_xsave(fpu); diff --git a/arch/x86/kernel/xsave.c b/arch/x86/kernel/xsave.c index 0de1fae2bdf0..943af0adacff 100644 --- a/arch/x86/kernel/xsave.c +++ b/arch/x86/kernel/xsave.c @@ -14,6 +14,8 @@ #include #include +#include + /* * Supported feature mask by the CPU and the kernel. */ @@ -638,7 +640,7 @@ static void __init xstate_enable_boot_cpu(void) setup_init_fpu_buf(); /* Auto enable eagerfpu for xsaveopt */ - if (cpu_has_xsaveopt && eagerfpu != DISABLE) + if (eagerfpu != DISABLE) eagerfpu = ENABLE; if (pcntxt_mask & XSTATE_EAGER) { @@ -739,3 +741,56 @@ void *get_xsave_addr(struct xsave_struct *xsave, int xstate) return (void *)xsave + xstate_comp_offsets[feature]; } EXPORT_SYMBOL_GPL(get_xsave_addr); + +unsigned long fpu_saved; + +static void my_clts(void *arg) +{ + asm volatile("clts"); +} + +static int eager_get(void *data, u64 *val) +{ + *val = fpu_saved; + + return 0; +} + +static int eager_set(void *data, u64 val) +{ + preempt_disable(); + if (val) { + on_each_cpu(my_clts, NULL, 1); + setup_force_cpu_cap(X86_FEATURE_EAGER_FPU); + } else { + setup_clear_cpu_cap(X86_FEATURE_EAGER_FPU); + stts(); + } + preempt_enable(); + + fpu_saved = 0; + + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(eager_fops, eager_get, eager_set, "%llu\n"); + +static int __init setup_eagerfpu_knob(void) +{ + static struct dentry *d_eager, *f_eager; + + d_eager = debugfs_create_dir("fpu", NULL); + if (!d_eager) { + pr_err("Error creating fpu debugfs dir\n"); + return -ENOMEM; + } + + f_eager = debugfs_create_file("eager", 0644, d_eager, NULL, &eager_fops); + if (!f_eager) { + pr_err("Error creating fpu debugfs node\n"); +
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sat, Feb 21, 2015 at 06:18:01PM -0800, Andy Lutomirski wrote: > That's true. The question is whether there are enough of them, and > whether twiddling TS is fast enough, that it's worth it. Yes, and let me make it clear what I'm trying to do here: I want to make sure that eager FPU handling (both allocation and switching - and no, I'm not confusing the concepts) *doesn't* *hurt* any relevant workload. If it does, then we'll stop wasting time right there. But(!), if the CR0.TS lazy dance turns out to be really slow and the eager handling doesn't bring a noticeable slowdown, in comparison, we should consider doing the eager thing by default. After running a lot more benchmarks, of course. Which brings me to the main reason why we're doing this: code simplification. If we switch to eager, we'll kill a lot of non-trivial code and the FPU handling in the kernel becomes dumb and nice again. > > And last but not least, why does the handling of CR0.TS traps have to be > > complicated? It does not look like rocket science to me, it should be a If you think so, just go and look at the latest tip/x86/fpu commits and the amount of WTF in the commit messages. -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sun, Feb 22, 2015 at 09:18:40AM +0100, Ingo Molnar wrote: > - It might make sense to do a 'perf stat --null --repeat' > measurement as well [without any -e arguments], to make > sure the rich PMU stats you are gathering are not > interfering? Well, the --repeat thing definitely is good to do and the previous single run is simply not meaningful due to variance I'd guess: perf stat --null --repeat 10 --sync --pre ~/bin/pre-build-kernel.sh make -s -j12 ... Lazy FPU: FPU saves before: 5 Setup is 16252 bytes (padded to 16384 bytes). System is 4222 kB CRC 5417fa4f Kernel: arch/x86/boot/bzImage is ready (#43) ... Kernel: arch/x86/boot/bzImage is ready (#52) Performance counter stats for 'make -s -j12' (10 runs): 219.406449195 seconds time elapsed ( +- 0.17% ) FPU saves after: 6974975 Eager FPU: FPU saves before: 4 Setup is 16252 bytes (padded to 16384 bytes). System is 4222 kB CRC 823c1801 Kernel: arch/x86/boot/bzImage is ready (#53) ... Kernel: arch/x86/boot/bzImage is ready (#62) Performance counter stats for 'make -s -j12' (10 runs): 218.791122148 seconds time elapsed ( +- 0.13% ) FPU saves after: 8939084 The counter numbers are consistent with before: ~20 FPU saves more in eager mode per kernel build. Timing improvement of 0.6 secs on average looks nice to me. Next I'll do the same thing in tmpfs to see how the numbers look without the page cache. -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
* Borislav Petkov wrote: > Lazy FPU: > 219.406449195 seconds time elapsed >( +- 0.17% ) > Eager FPU: > 218.791122148 seconds time elapsed >( +- 0.13% ) > Timing improvement of 0.6 secs on average looks nice to > me. Next I'll do the same thing in tmpfs to see how the > numbers look without the page cache. This is also very similar to the ~0.6 secs improvement your first set of numbers gave. So now that it appears we have consistent numbers, it would be nice to check it on older hardware (and other workloads) as well, to see whether it's a consistent win. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sun, Feb 22, 2015 at 01:57:36PM +0100, Ingo Molnar wrote: > This is also very similar to the ~0.6 secs improvement your > first set of numbers gave. Yeah, running without --repeat was simply misleading. > So now that it appears we have consistent numbers, it would > be nice to check it on older hardware (and other workloads) > as well, to see whether it's a consistent win. I'll try to dig out some old boxes we have here, maybe the build robot could do some measurements too. Fengguang et al, guys, is it possible for you to run this patch: https://lkml.kernel.org/r/b0ba174ea882ed36cf7011e872baf427c23b7e09.1424458621.git.l...@amacapital.net on your fleet - has to be baremetal - to check how it behaves, performance-wise? Older machines would be preferred... That would be lovely :-) Thanks! -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/22/2015 06:06 AM, Borislav Petkov wrote: > On Sat, Feb 21, 2015 at 06:18:01PM -0800, Andy Lutomirski wrote: >> That's true. The question is whether there are enough of them, >> and whether twiddling TS is fast enough, that it's worth it. > > Yes, and let me make it clear what I'm trying to do here: I want to > make sure that eager FPU handling (both allocation and switching - > and no, I'm not confusing the concepts) *doesn't* *hurt* any > relevant workload. If it does, then we'll stop wasting time right > there. > > But(!), if the CR0.TS lazy dance turns out to be really slow and > the eager handling doesn't bring a noticeable slowdown, in > comparison, we should consider doing the eager thing by default. > After running a lot more benchmarks, of course. > > Which brings me to the main reason why we're doing this: code > simplification. If we switch to eager, we'll kill a lot of > non-trivial code and the FPU handling in the kernel becomes dumb > and nice again. Currently the FPU handling does something really dumb for KVM VCPU threads. Specifically, every time we enter a KVM guest, we save the userspace FPU state of the VCPU thread, and every time we leave the KVM guest, we load the userspace FPU state of the VCPU thread. This is done for a thread that hardly ever exits to userspace, and generally just switches between kernel and guest mode. The reason for this acrobatics is that at every context switch time, the userspace FPU state is saved & loaded. I am working on a patch series to avoid that needless FPU save & restore, by moving the point at which the user FPU state is loaded out to the point where we return to userspace, in do_notify_resume. One implication of this is that in kernel mode, we can no longer just assume that the user space FPU state is always loaded, and we need to check for that (like the lazy FPU code does today). I would really like to keep that code around, for obvious reasons :) - -- All rights reversed -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJU6oZBAAoJEM553pKExN6Dk1oH/iJhvE96xM8Yo38eplaI73nC Bx8OAJk5ombiTroWTqU99Y/2iZmdt3k9KHYEQhYnvCu3RV/4N9GwVLobbh/EPN8Q v/gXJHOPT1uT7arpIu+XqcbqYCUUMnFdAOfjuLupGRjX64YgzBltd4TUC4yPdW/2 TXnj7OLW3jGIJVOKjnx7zQaKqolAAxbprXkfe8MsGwy0ARS4kXIvcBG7e8s92uQb oIIQrs5UxmhQo/8Sa+Q8jCF8bHrTJr/mkbPybT6o1et78kwT7FV+2d7oGQySn+v1 FMBRiQsUOdY6AZOtjkWxB+k73QDSwkdLivVWwXZMGICUcQz4756nINWQyPNL49U= =DlRc -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sun, Feb 22, 2015 at 5:45 PM, Rik van Riel wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 02/22/2015 06:06 AM, Borislav Petkov wrote: >> On Sat, Feb 21, 2015 at 06:18:01PM -0800, Andy Lutomirski wrote: >>> That's true. The question is whether there are enough of them, >>> and whether twiddling TS is fast enough, that it's worth it. >> >> Yes, and let me make it clear what I'm trying to do here: I want to >> make sure that eager FPU handling (both allocation and switching - >> and no, I'm not confusing the concepts) *doesn't* *hurt* any >> relevant workload. If it does, then we'll stop wasting time right >> there. >> >> But(!), if the CR0.TS lazy dance turns out to be really slow and >> the eager handling doesn't bring a noticeable slowdown, in >> comparison, we should consider doing the eager thing by default. >> After running a lot more benchmarks, of course. >> >> Which brings me to the main reason why we're doing this: code >> simplification. If we switch to eager, we'll kill a lot of >> non-trivial code and the FPU handling in the kernel becomes dumb >> and nice again. > > Currently the FPU handling does something really dumb for > KVM VCPU threads. Specifically, every time we enter a > KVM guest, we save the userspace FPU state of the VCPU > thread, and every time we leave the KVM guest, we load > the userspace FPU state of the VCPU thread. > > This is done for a thread that hardly ever exits to > userspace, and generally just switches between kernel > and guest mode. > > The reason for this acrobatics is that at every > context switch time, the userspace FPU state is > saved & loaded. > > I am working on a patch series to avoid that needless > FPU save & restore, by moving the point at which the > user FPU state is loaded out to the point where we > return to userspace, in do_notify_resume. > > One implication of this is that in kernel mode, we > can no longer just assume that the user space FPU > state is always loaded, and we need to check for that > (like the lazy FPU code does today). I would really > like to keep that code around, for obvious reasons :) I like that stuff, except for the fact that it still has code that depends on whether we're in eager or lazy mode, even though eager is a little less eager with your patches. Ideally I'd like to see your patches applied *and* lazy mode removed. --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/23/2015 12:22 AM, Andy Lutomirski wrote: > On Sun, Feb 22, 2015 at 5:45 PM, Rik van Riel > wrote: >> One implication of this is that in kernel mode, we can no longer >> just assume that the user space FPU state is always loaded, and >> we need to check for that (like the lazy FPU code does today). I >> would really like to keep that code around, for obvious reasons >> :) > > I like that stuff, except for the fact that it still has code that > depends on whether we're in eager or lazy mode, even though eager > is a little less eager with your patches. Ideally I'd like to see > your patches applied *and* lazy mode removed. The latest version of my code (which I should forward-port again to the latest tip bits) simplifies things a lot. It moves all new task handling to switch_fpu_finished(), which is called from do_notify_resume(). At that point we either load the FPU context, or we set CR0.TS. - -- All rights reversed -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJU6yI3AAoJEM553pKExN6DyQEH/13E0T6wsII5L+0W1D0WU4v5 F2SmU74zvCv+RhRl/0npvKRQC3IkZ/peM9h/1PU/r61Y39VhexUxiUZ86afzz4KJ ogPNEs6HZ8UYSji6RPlVsuJJ4vCsZ9GA+27JwKGzLUG1NfniL0qlc93rl+f3ZfFm 5H4bjtcO361Am15yKkFPRl/0PKqRhMDa4tzJGkvvCaJ+k195Ik0a7WXjOlGdIeJA 5Q1D0qaAccXtwfNjqhxIfTBi1LV2a1/jCKF8ktxGJ2+Ywvau9blJnPfD8vUf9JL2 9/le1kyNU3h8B2lU0jIi4a2Y/CcU47heEUy5htmoo6wChQWAy/pR8JaE6++qWqI= =ee/J -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On 02/20, Andy Lutomirski wrote: > > We have eager and lazy fpu modes, introduced in: > > 304bceda6a18 x86, fpu: use non-lazy fpu restore for processors supporting > xsave > > The result is rather messy. There are two code paths in almost all of the > FPU code, and only one of them (the eager case) is tested frequently, since > most kernel developers have new enough hardware that we use eagerfpu. > > It seems that, on any remotely recent hardware, eagerfpu is a win: > glibc uses SSE2, so laziness is probably overoptimistic, and, in any > case, manipulating TS is far slower that saving and restoring the full > state. > > To try to shake out any latent issues on old hardware, this changes > the default to eager on all CPUs. If no performance or functionality > problems show up, a subsequent patch could remove lazy mode entirely. > > Signed-off-by: Andy Lutomirski > --- > arch/x86/kernel/xsave.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/arch/x86/kernel/xsave.c b/arch/x86/kernel/xsave.c > index 0de1fae2bdf0..1928c8f34ce5 100644 > --- a/arch/x86/kernel/xsave.c > +++ b/arch/x86/kernel/xsave.c > @@ -637,8 +637,8 @@ static void __init xstate_enable_boot_cpu(void) > prepare_fx_sw_frame(); > setup_init_fpu_buf(); > > - /* Auto enable eagerfpu for xsaveopt */ > - if (cpu_has_xsaveopt && eagerfpu != DISABLE) > + /* Auto enable eagerfpu for everyone */ > + if (eagerfpu != DISABLE) > eagerfpu = ENABLE; Well, but if we want this change then perhaps we should simply change the default value? This way "AUTO" still can work. Oleg. --- x/arch/x86/kernel/xsave.c +++ x/arch/x86/kernel/xsave.c @@ -563,7 +563,7 @@ static void __init setup_init_fpu_buf(vo xsave_state_booting(init_xstate_buf, -1); } -static enum { AUTO, ENABLE, DISABLE } eagerfpu = AUTO; +static enum { AUTO, ENABLE, DISABLE } eagerfpu = ENABLE; static int __init eager_fpu_setup(char *s) { if (!strcmp(s, "on")) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, Feb 23, 2015 at 07:51:04AM -0500, Rik van Riel wrote: > At that point we either load the FPU context, or we > set CR0.TS. Right, but provided eager doesn't bring any slowdown, we can drop the TS fiddling altogether and only load FPU context. -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, Feb 23, 2015 at 03:59:29PM +0100, Oleg Nesterov wrote: > Well, but if we want this change then perhaps we should simply change > the default value? This way "AUTO" still can work. Yeah, sure, let's do some measurements first, to see whether this is even worth it. Btw, Mel pointed me at some cleanups he did a while ago in that area too: https://lkml.org/lkml/2014/8/6/201 If you guys could take a look, that would be awesome :) -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/23/2015 10:03 AM, Borislav Petkov wrote: > On Mon, Feb 23, 2015 at 07:51:04AM -0500, Rik van Riel wrote: >> At that point we either load the FPU context, or we set CR0.TS. > > Right, but provided eager doesn't bring any slowdown, we can drop > the TS fiddling altogether and only load FPU context. Agreed. However, we would still need the rest of the kernel code to check whether userspace FPU context is loaded in the registers or not. We cannot simplify to the point of assuming that the FPU state will always be loaded when in eager mode. - -- All rights reversed -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJU60x+AAoJEM553pKExN6DIMkIAJpZHQuojQMTUyy11EB7dx2N ZMm5PiQGW47f/BIbIwR3hwZbC385ZLdx1Nkrys8cXd3WNJInrMxf2PDaU9YtDFw1 NdybRbbsxAAaB3oJldLFUH5ouo9ePKV4RyXFgsgNH75uv6u4pesaWwUZNuLmxzrk h8/wns7Lpddr4C0ahPimEM1pXdfiFyo/0jaAiOj8o/IklXiJW25LBYUzE4ogD/2T 097wV5C8ojvS4doVD+GthxFuJFslZWxyNaRBRNqpE2LyhmjEdz2WwNTSrDtLsBK0 gKHjgD0eox/HpNkBzwaF6MOo1of2/WFNSJyRKwlwaWnRNpZ3jDfyU0VPg3yhF30= =aIKn -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/23/2015 10:11 AM, Borislav Petkov wrote: > On Mon, Feb 23, 2015 at 03:59:29PM +0100, Oleg Nesterov wrote: >> Well, but if we want this change then perhaps we should simply >> change the default value? This way "AUTO" still can work. > > Yeah, sure, let's do some measurements first, to see whether this > is even worth it. > > Btw, Mel pointed me at some cleanups he did a while ago in that > area too: https://lkml.org/lkml/2014/8/6/201 > > If you guys could take a look, that would be awesome :) I don't think any of those changes would survive the "defer FPU loading to do_notify_resume" patch series. - -- All rights reversed -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJU60zqAAoJEM553pKExN6DbIMH/A72/Lid9hho65Wh4notTa7a QMRsM+XgLvvohmYKWZQ+uS4by9jbLaH827PIQtcW3bSEgtWGjam+pUGC/gWXwZk7 51oYDhyAqvVr6vb3mpl13+N+xGN5TlLoZa7SC73DuSSD/6HGqDryrr7mcfX53Yc+ UOg7MpAx//5tLZ89amMirqqqhqEfQh9qlyjPNLVywpHeSDYfxlinsfJgWOUW0/4A 7wR51ToANH9cMcSXWQ72lUNbJpJLgkcW8AVKXicS/UtFnvgItcvfgy1TjOpuaZom MRVv0E7NUM10U9mZ13NRDE7JSCk5Euy8XgM+AOk5PXJlb0Lg8IYu1nVTn7Q+6A0= =JLIM -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, Feb 23, 2015 at 10:51:26AM -0500, Rik van Riel wrote: > However, we would still need the rest of the kernel code to ... Yeah, let's wait out first and see what the benchmarks say. Mel started a bunch of them on a couple of boxes here, we'll have results in the coming days. -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On 02/23, Rik van Riel wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 02/23/2015 10:11 AM, Borislav Petkov wrote: > > On Mon, Feb 23, 2015 at 03:59:29PM +0100, Oleg Nesterov wrote: > >> Well, but if we want this change then perhaps we should simply > >> change the default value? This way "AUTO" still can work. > > > > Yeah, sure, let's do some measurements first, to see whether this > > is even worth it. > > > > Btw, Mel pointed me at some cleanups he did a while ago in that > > area too: https://lkml.org/lkml/2014/8/6/201 > > > > If you guys could take a look, that would be awesome :) > > I don't think any of those changes would survive the > "defer FPU loading to do_notify_resume" patch series. Agreed, at least +static inline void switch_eagerfpu_prepare(struct task_struct *old, struct task_struct *new, int cpu, + fpu_switch_t *fpu) +{ + fpu->preload = tsk_used_math(new); doesn't look right if we change the rules. OTOH, perhaps it makes sense to redo these cleanups later. Oleg. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Sat, 21 Feb 2015, Andy Lutomirski wrote: > > Additionally I believe long-executing FPU instructions (i.e. > > transcendentals) can take advantage of continuing to execute in parallel > > where the context has already been switched rather than stalling an eager > > FPU context switch until the FPU instruction has completed. > > It seems highly unlikely to me that a slow FPU instruction can retire > *after* a subsequent fxsave, which would need to happen for this to > work. I meant something else -- a slow FPU instruction can retire after a task has been switched where the FP context has been left intact, i.e. in the lazy FP context switching case, where only the MMU context and GPRs have been replaced. Whereas in the eager FP context switching case you can get through to FXSAVE while a slow FPU instruction hasn't completed yet (e.g. started just as preemption was about to happen). Obviously that FXSAVE will have to stall until the FPU instruction has completed (IIRC the i486 aborted transcendental instructions on any exceptions/interrupts instead, leading to the risk of process starvation in heavily interrupt loaded systems, but I also believe it has been fixed as from the Pentium). Though if, as you say, the lone taking of a trap/interrupt gate can take hundreds of cycles, perhaps indeed no FPU instruction will execute *that* long on modern silicon. > > And last but not least, why does the handling of CR0.TS traps have to be > > complicated? It does not look like rocket science to me, it should be a > > mere handful of instructions, the time required to move the two FP > > contexts out from and in to the FPU respectively should dominate > > processing time. Where quoted the optimisation manual states 250 cycles > > for FXSAVE and FXRSTOR combined. > > The TS traps aren't complicated -- they're just really slow. I think > that each of setting *and* clearing TS serializes and takes well over > a hundred cycles. A #NM interrupt (the thing that happens if you try > to use the FPU with TS set) serializes and does all kinds of slow > things, so it takes many hundreds of cycles. The handler needs to > clear TS (another hundred cycles or more), load the FPU state > (actually rather fast on modern CPUs), and then IRET back to userspace > (hundreds of cycles). This adds up to a lot of cycles. A round trip > through an exception handler seems to be thousands of cycles. That sucks wet goat farts! :( I have to admit I got moved a bit away from the x86 world and didn't realise things have become so bad. Some 10 years ago or so taking a trap or interrupt gate would need some 30 cycles (of course task gates are unusable for anything that does not absolutely require them such as a #DF; their usability for anything real ended with the 80286 or suchlike). Similarly an IRET to reverse the actions taken. That was already rather bad, but understandable, after all the CPU had to read the gate descriptor, access the TSS, switch both CS and SS descriptors, etc. What I don't understand is why CLTS, a dedicated instruction that avoids the need to access whole CR0 (that again can understandably be costly, because of the grouping of all the important bits there), has to be so slow. It flips a single bit down there and does not to serialise anything, as any instruction down the pipeline it could affect would trigger a #NM anyway! And there's an IRET somewhere on the way too, before the instruction that originally triggered the fault will be reexecuted. And why the heck over all these years a mechanism similar to SYSENTER and its bunch of complementing MSRs hasn't been invented for the common exceptions, to avoid all this gate descriptor dance! > > And of course you can install the right handler (i.e. FSAVE vs FXSAVE) at > > bootstrap depending on processor features, you don't have to do all the > > run-time check on every trap. You can even optimise the FSAVE handler > > away at the build time if you know it won't ever be used based on the > > minimal supported processor family selected. > > None of this matters. A couple of branches in an #NM handler are > completely lost in the noise. Agreed, given what you state, completely understood. > > Do you happen to know or can determine how much time (in clock cycles) a > > CR0.TS trap itself takes, including any time required to preserve the > > execution state in the handler such as pushing/popping GPRs to/from the > > stack (as opposed to processing time spent on moving the FP contexts back > > and forth)? Is there no room for improvement left there? How many task > > scheduling slots say per million must be there poking at the FPU for eager > > FPU context switching to take advantage over lazy one? > > Thousands of cycles. Considerably worse in an SVM guest. x86 > exception handling sucks. I must have been spoilt with the MIPS exception handling. Taking an exception on a MIPS processor is virtually instantane
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/23/2015 04:17 PM, Maciej W. Rozycki wrote: > On Sat, 21 Feb 2015, Andy Lutomirski wrote: > >>> Additionally I believe long-executing FPU instructions (i.e. >>> transcendentals) can take advantage of continuing to execute in >>> parallel where the context has already been switched rather >>> than stalling an eager FPU context switch until the FPU >>> instruction has completed. >> >> It seems highly unlikely to me that a slow FPU instruction can >> retire *after* a subsequent fxsave, which would need to happen >> for this to work. > > I meant something else -- a slow FPU instruction can retire after a > task has been switched where the FP context has been left intact, > i.e. in the lazy FP context switching case, where only the MMU > context and GPRs have been replaced. I don't think that's true, because changing the MMU context and GPRs also includes changing the instruction pointer, and changing over the execution to the new task. After a context switch, the instructions from the old task are no longer in the pipeline. - -- All rights reversed -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJU65noAAoJEM553pKExN6DyVQH+gJDWuHyoFtgKFyELFi+tTGb vB/9uR267N4Hxu0D3kedhX9HNJwBtcs1rXm++p+r4RXpz2+gl/HxihoL8v1qk4zy 1lAa5se8T5hUL7MCsD5nOIs2ca3HAGwOFtVXrjvJQ6dSCCVD/SuqJC49ENc5Nd3M 1FcW4jg+MlQ2TNJoi4QRBllmxUFffACzv8M3VWVigIibMGqrOXXSraHp0k1hFFOE LeL122JKVp7YhVj5Dx7qx1DPUkT24RT9ip1f9r5DkxFmHwdCcY31WzT4asArqHb4 uwK8LVcpTax30VHghqy8KUBVKO5h7pHvdhTYSaoLWdUJfNFJ+8Hw5sDMfPdjN2k= =6lA5 -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, Feb 23, 2015 at 1:21 PM, Rik van Riel wrote: > > On 02/23/2015 04:17 PM, Maciej W. Rozycki wrote: >>> >>> It seems highly unlikely to me that a slow FPU instruction can >>> retire *after* a subsequent fxsave, which would need to happen >>> for this to work. >> >> I meant something else -- a slow FPU instruction can retire after a >> task has been switched where the FP context has been left intact, >> i.e. in the lazy FP context switching case, where only the MMU >> context and GPRs have been replaced. > > I don't think that's true, because changing the MMU context and GPRs > also includes changing the instruction pointer, and changing over the > execution to the new task. We have one traditional special case, which actually did something like Maciej's nightmare scenario: the completely broken "FPU errors over irq13" IBM PC/AT FPU linkage. But since we don't actually support old i386 machines any more, we don't really need to care. The only way you can get into that situation is with an external i387. I don't think we need to worry about it. But with the old horrid irq13 error handling, you literally could get into a situation that you got an error "exception" (irq) from the previous state, *after* you had already switched to the new one. We had some code to mitigate the problem, but as mentioned, I don't think it's an issue any more. Linus -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, 23 Feb 2015, Rik van Riel wrote: > > I meant something else -- a slow FPU instruction can retire after a > > task has been switched where the FP context has been left intact, > > i.e. in the lazy FP context switching case, where only the MMU > > context and GPRs have been replaced. > > I don't think that's true, because changing the MMU context and GPRs > also includes changing the instruction pointer, and changing over the > execution to the new task. That does not matter. The instructions in question only operate on x87 internal registers: the data stack registers, specifically ST(0) and possibly also ST(1), and consequently the Tag Word register, and the Status Word register. No CPU resource such as the MMU or GPRs need to be referred for an x87 instruction to complete. Any unmasked IEEE 754 FPU exception recorded on the way is only signalled at the next x87 instruction. > After a context switch, the instructions from the old task are no > longer in the pipeline. I'd say it's implementation-specific. As I mentioned the i486 aborted any transcendental x87 instruction in progress upon taking an exception or interrupt. That was a model like you refer to, but as I also mentioned it had its shortcomings. Any newer implementation I'd expect to, and Pentium class processors certainly did, continue executing these instructions in parallel in the FPU pipeline regardless of what the CPU does until completed. If WAIT or a waiting x87 instruction was encountered while a previous x87 instruction was still in progress, the CPU pipeline would stall until the earlier x87 instruction has completed. The FPU has no way to determine the CPU context has been switched and neither it recognises execution privilege levels. I can't speak of SIMD instructions, I don't know offhand. OTOH AFAIK they don't suffer from latencies so long as some x87 instructions that may be in the range of 400 clock cycles. Maciej -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, Feb 23, 2015 at 2:27 PM, Maciej W. Rozycki wrote: > On Mon, 23 Feb 2015, Rik van Riel wrote: > >> > I meant something else -- a slow FPU instruction can retire after a >> > task has been switched where the FP context has been left intact, >> > i.e. in the lazy FP context switching case, where only the MMU >> > context and GPRs have been replaced. >> >> I don't think that's true, because changing the MMU context and GPRs >> also includes changing the instruction pointer, and changing over the >> execution to the new task. > > That does not matter. The instructions in question only operate on x87 > internal registers: the data stack registers, specifically ST(0) and > possibly also ST(1), and consequently the Tag Word register, and the > Status Word register. No CPU resource such as the MMU or GPRs need to be > referred for an x87 instruction to complete. Any unmasked IEEE 754 FPU > exception recorded on the way is only signalled at the next x87 > instruction. > >> After a context switch, the instructions from the old task are no >> longer in the pipeline. > > I'd say it's implementation-specific. As I mentioned the i486 aborted > any transcendental x87 instruction in progress upon taking an exception or > interrupt. That was a model like you refer to, but as I also mentioned it > had its shortcomings. IRET is serializing, according to the the docs (I think) and according to the Intel engineers I asked (I'm absolutely certain about this part). So FPU ops are entirely done at the end of a normal context switch. We also always save the FPU context on every context switch away from a task that used the FPU, even in lazy mode. This is because we might switch the task back in on a different CPU, and we don't want to use an IPI to move the FPU context. Given that we're only talking about old CPUs here, I sincerely doubt that there's any relevant case in which an fxsave can usefully wait for a long-running transcendental op to finish while we continue doing useful work. *Especially* since there will almost certainly be several more mfences or atomic ops before the end of the context switch, even if we're lucky enough to complete the context switching using sysret. --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, Feb 23, 2015 at 4:56 PM, Maciej W. Rozycki wrote: > On Mon, 23 Feb 2015, Linus Torvalds wrote: > >> We have one traditional special case, which actually did something >> like Maciej's nightmare scenario: the completely broken "FPU errors >> over irq13" IBM PC/AT FPU linkage. >> >> But since we don't actually support old i386 machines any more, we >> don't really need to care. The only way you can get into that >> situation is with an external i387. I don't think we need to worry >> about it. >> >> But with the old horrid irq13 error handling, you literally could get >> into a situation that you got an error "exception" (irq) from the >> previous state, *after* you had already switched to the new one. We >> had some code to mitigate the problem, but as mentioned, I don't think >> it's an issue any more. > > Correct, the horrid hack is gone, it was so horrible (though I understand > why IBM had to do it with their PC/AT) that back in mid 1990s, some 10 > years after the inception of the problem, Intel felt so compelled to make > people get the handling of it right as to release a dedicated application > note: "AP-578 Software and Hardware Considerations for FPU Exception > Handlers for Intel Architecture Processors", Order Number 243291-002. > > Anyway, my point through this consideration has been about the > performance benefit from continuing the execution of an x87 instruction in > parallel, perhaps until after a context has been fully switched. Which > benefit is lost if a FSAVE/FXSAVE executed eagerly at the context switch > stalls waiting for the instruction to complete instead. And the save is indeed executed eagerly. Conceivably we could get rid of that on UP, but that seems to be a lot of complexity for extremely little gain. --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, 23 Feb 2015, Linus Torvalds wrote: > We have one traditional special case, which actually did something > like Maciej's nightmare scenario: the completely broken "FPU errors > over irq13" IBM PC/AT FPU linkage. > > But since we don't actually support old i386 machines any more, we > don't really need to care. The only way you can get into that > situation is with an external i387. I don't think we need to worry > about it. > > But with the old horrid irq13 error handling, you literally could get > into a situation that you got an error "exception" (irq) from the > previous state, *after* you had already switched to the new one. We > had some code to mitigate the problem, but as mentioned, I don't think > it's an issue any more. Correct, the horrid hack is gone, it was so horrible (though I understand why IBM had to do it with their PC/AT) that back in mid 1990s, some 10 years after the inception of the problem, Intel felt so compelled to make people get the handling of it right as to release a dedicated application note: "AP-578 Software and Hardware Considerations for FPU Exception Handlers for Intel Architecture Processors", Order Number 243291-002. Anyway, my point through this consideration has been about the performance benefit from continuing the execution of an x87 instruction in parallel, perhaps until after a context has been fully switched. Which benefit is lost if a FSAVE/FXSAVE executed eagerly at the context switch stalls waiting for the instruction to complete instead. Maciej -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, 23 Feb 2015, Andy Lutomirski wrote: > >> After a context switch, the instructions from the old task are no > >> longer in the pipeline. > > > > I'd say it's implementation-specific. As I mentioned the i486 aborted > > any transcendental x87 instruction in progress upon taking an exception or > > interrupt. That was a model like you refer to, but as I also mentioned it > > had its shortcomings. > > IRET is serializing, according to the the docs (I think) and according > to the Intel engineers I asked (I'm absolutely certain about this > part). So FPU ops are entirely done at the end of a normal context > switch. No question about the serialising property of IRET, it has been like this since the original Pentium implementation. Do you have an architecture specification reference to back up your claim though as far as the FPU is concerned? I'm asking because I am genuinely curious. The x87 case is so special, there isn't anything there really that is externally observable or should be affected by IRET or any other synchronisation barriers apart from WAIT (or a waiting x87 instruction) that has been there for this purpose since forever. And it would defeat some documented benefits of running the FP pipeline in the parallel. And certainly such synchronisation didn't happen in the old days. > We also always save the FPU context on every context switch away from > a task that used the FPU, even in lazy mode. This is because we might > switch the task back in on a different CPU, and we don't want to use > an IPI to move the FPU context. That's an interesting case too, although not necessarily related. If you say that we always save the FP context eagerly for the purpose of process migration, then sure, that invalidates any benefit we'd have from letting the x87 proceed. However I can see different ways to address this case avoiding the need of eager FP context saving or an IPI: 1. We could bind any currently suspended process with an unsaved FP context to the CPU it last executed on. 2. We could mark such a process for migration next time and let it execute on the CPU that holds its FP context once more, and then save the FP context eagerly on the way out. In some cases a lazily retained FP context would be preempted by another process before the process in question would resume anyway. In this case any temporary binding to a CPU could be given up. > Given that we're only talking about old CPUs here, I sincerely doubt > that there's any relevant case in which an fxsave can usefully wait > for a long-running transcendental op to finish while we continue doing > useful work. *Especially* since there will almost certainly be > several more mfences or atomic ops before the end of the context > switch, even if we're lucky enough to complete the context switching > using sysret. I am not sure what you mean by FXSAVE usefully waiting for an op, please elaborate. At the point you've reached FXSAVE and an earlier x87 instruction hasn't completed, you've already lost. The pipeline will be stalled until the x87 instruction has completed and it can be hundreds of cycles. My point therefore has been about avoiding to execute FXSAVE for the old task until absolutely necessary, that with the lazy FP context switching would be at the next x87 (or SSE) instruction reached by the new task. Likewise I don't see why MFENCE or an atomic operation should affect the excecution of say FSINCOS. Whether the results of FSINCOS arrive before or after MFENCE, etc. are not externally observable. And I'm not sure if this all affects old CPUs only -- I don't know how much x87 software is out there, but after all these years I'd expect quite some. Sure, lots of this can be recompiled to use SSE instead, but not all, and even where it is feasible, that's an extra burden for people, beyond say a routine hardware or Linux distribution or for that matter lone kernel upgrade. Therefore I think we need to be careful not to pessimise things for a subset of people too much and ideally at all. And to be clear, I am not against removing lazy FP context switching per se. I am just emphasizing to be careful with that and be absolutely sure that it does not cause excessive harm. I still wonder why Intel hasn't addressed some issues around this stuff -- is that there are not enough people using proper IEEE 754 arithmetic on x86 hardware to attract interest of hardware architecture maintainers? After all the same issue applies to enabled IEEE 754 exceptions, a #MF/#XM exception isn't going to take any less than a #NM fault. Or maybe I'm just missing something? Maciej -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Mon, Feb 23, 2015 at 6:14 PM, Maciej W. Rozycki wrote: > On Mon, 23 Feb 2015, Andy Lutomirski wrote: > >> >> After a context switch, the instructions from the old task are no >> >> longer in the pipeline. >> > >> > I'd say it's implementation-specific. As I mentioned the i486 aborted >> > any transcendental x87 instruction in progress upon taking an exception or >> > interrupt. That was a model like you refer to, but as I also mentioned it >> > had its shortcomings. >> >> IRET is serializing, according to the the docs (I think) and according >> to the Intel engineers I asked (I'm absolutely certain about this >> part). So FPU ops are entirely done at the end of a normal context >> switch. > > No question about the serialising property of IRET, it has been like this > since the original Pentium implementation. Do you have an architecture > specification reference to back up your claim though as far as the FPU is > concerned? I'm asking because I am genuinely curious. > > The x87 case is so special, there isn't anything there really that is > externally observable or should be affected by IRET or any other > synchronisation barriers apart from WAIT (or a waiting x87 instruction) > that has been there for this purpose since forever. And it would defeat > some documented benefits of running the FP pipeline in the parallel. It's plausible that this is special, but I doubt it. Especially since this optimization would be nuts post-SSE2. > > And certainly such synchronisation didn't happen in the old days. > >> We also always save the FPU context on every context switch away from >> a task that used the FPU, even in lazy mode. This is because we might >> switch the task back in on a different CPU, and we don't want to use >> an IPI to move the FPU context. > > That's an interesting case too, although not necessarily related. If you > say that we always save the FP context eagerly for the purpose of process > migration, then sure, that invalidates any benefit we'd have from letting > the x87 proceed. > > However I can see different ways to address this case avoiding the need > of eager FP context saving or an IPI: > > 1. We could bind any currently suspended process with an unsaved FP >context to the CPU it last executed on. This would be insane. > > 2. We could mark such a process for migration next time and let it execute >on the CPU that holds its FP context once more, and then save the FP >context eagerly on the way out. This would be worse than insane. Now, in order to wake such a process on a different CPU, we'd have to force a *context switch* on the source CPU. Now we're replacing a few hundred cycles at worse for a transcendental function with at least 10k cycles (at a guess) and possibly many orders of magnitude more if locks are held, plus priority issues, plus total craziness. > > In some cases a lazily retained FP context would be preempted by another > process before the process in question would resume anyway. In this case > any temporary binding to a CPU could be given up. > >> Given that we're only talking about old CPUs here, I sincerely doubt >> that there's any relevant case in which an fxsave can usefully wait >> for a long-running transcendental op to finish while we continue doing >> useful work. *Especially* since there will almost certainly be >> several more mfences or atomic ops before the end of the context >> switch, even if we're lucky enough to complete the context switching >> using sysret. > > I am not sure what you mean by FXSAVE usefully waiting for an op, please > elaborate. At the point you've reached FXSAVE and an earlier x87 > instruction hasn't completed, you've already lost. The pipeline will be > stalled until the x87 instruction has completed and it can be hundreds of > cycles. My point therefore has been about avoiding to execute FXSAVE for > the old task until absolutely necessary, that with the lazy FP context > switching would be at the next x87 (or SSE) instruction reached by the new > task. > > Likewise I don't see why MFENCE or an atomic operation should affect the > excecution of say FSINCOS. Whether the results of FSINCOS arrive before > or after MFENCE, etc. are not externally observable. FSINCOS; FXSAVE; MFENCE had better serialize all the way, no matter what weird architectural crud is going on. > > And I'm not sure if this all affects old CPUs only -- I don't know how > much x87 software is out there, but after all these years I'd expect quite > some. Sure, lots of this can be recompiled to use SSE instead, but not > all, and even where it is feasible, that's an extra burden for people, > beyond say a routine hardware or Linux distribution or for that matter > lone kernel upgrade. Therefore I think we need to be careful not to > pessimise things for a subset of people too much and ideally at all. > > And to be clear, I am not against removing lazy FP context switching per > se. I am just emphasizing to be c
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/23/2015 09:31 PM, Andy Lutomirski wrote: > On Mon, Feb 23, 2015 at 6:14 PM, Maciej W. Rozycki > wrote: >> That's an interesting case too, although not necessarily related. >> If you say that we always save the FP context eagerly for the >> purpose of process migration, then sure, that invalidates any >> benefit we'd have from letting the x87 proceed. >> >> However I can see different ways to address this case avoiding >> the need of eager FP context saving or an IPI: >> >> 1. We could bind any currently suspended process with an unsaved >> FP context to the CPU it last executed on. > > This would be insane. The task would only be bound to the CPU as long as nothing else ran that saved the FPU state to memory. This means the task would be bound to the CPU while the CPU is idle, or running a kernel thread. When another user space thread is about to run, the FPU state would be saved. This sounds slightly less insane already. Of course, once you figure in CPU power saving and CPU hot plug, the level of insanity is far beyond what you seem to imply above... In other words, almost certainly not worth doing :) - -- All rights reversed -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJU7I38AAoJEM553pKExN6DhKEIAKauUGBx/1sTshdYYZ1aBlLx xY7afNUjs8PIxjCbdcwrujbtNa9CFgDlRR6TegvmzQA3prXu/0XZ3vas3O/lD2lC ks8p3RBzIw4dECxZoCvTQ+VrULk07+LCI+AUNKSm/pNlBCSWeeo2nKqoTREh3oHU EWzJxn5aEfIA4vZQAnFP5TwkCwR2ob5COGx/I9l54brHEwhEqiRFrPwrIP2WJerx Lc1Wkmv2PtTN/oQkXOCVKN0hVab//eVnkUiTsY1TnfCQsZSbEMWgq6aqXlb/lhUs hhpNToBVlWF3LsCnGm6LfCrgX+VSBY9LQpYfaY1ltEmxE+nOplbI+JHQG4Yqgag= =ah5u -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs
On Fri, Feb 20, 2015 at 7:58 PM, Andy Lutomirski wrote: > We have eager and lazy fpu modes, introduced in: > > 304bceda6a18 x86, fpu: use non-lazy fpu restore for processors supporting > xsave > > The result is rather messy. There are two code paths in almost all of the > FPU code, and only one of them (the eager case) is tested frequently, since > most kernel developers have new enough hardware that we use eagerfpu. > > It seems that, on any remotely recent hardware, eagerfpu is a win: > glibc uses SSE2, so laziness is probably overoptimistic, and, in any > case, manipulating TS is far slower that saving and restoring the full > state. > > To try to shake out any latent issues on old hardware, this changes > the default to eager on all CPUs. If no performance or functionality > problems show up, a subsequent patch could remove lazy mode entirely. I'm a big fan of simplifying things, but. SIMD registers were growing in x86, and they are going to grow again, this time four-fold in Intel MIC: from sixteen 256-bit registers to thirty two 512-bit registers. That's 2 kbytes of data. Just moving this data out to/from memory will take some time. And some people talk about 1024-bit registers already... Let's not completely remove lazy FPU saving code just yet. Maybe we'll be forced to reinstate it. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Some results (was: Re: [RFC PATCH] x86, fpu: Use eagerfpu by default on all CPUs)
On Fri, Feb 20, 2015 at 10:58:15AM -0800, Andy Lutomirski wrote: > - /* Auto enable eagerfpu for xsaveopt */ > - if (cpu_has_xsaveopt && eagerfpu != DISABLE) > + /* Auto enable eagerfpu for everyone */ > + if (eagerfpu != DISABLE) > eagerfpu = ENABLE; So Mel did run some measurements with it on an old Intel and AMD box. Attached are netperf results from both. The Intel box is a quad core very similar to this one: http://ark.intel.com/products/28020/Intel-Xeon-Processor-5063-4M-Cache-3_20-GHz-1066-MHz-FSB netperf-udp-rr shows some impact of the eagerfpu patch which is outside of the noise level. netperf-tcp-rr not so much but eager is still a bit behind. The AMD box is an old K8 and results there look like eager is better :-) Not with all though - pipetest is worse. netperf-udp-rr is better at almost every data point and tcp-rr looks a bit useless with those high noise levels. As a summary, the patch has some, albeit small, impact. We would need more benchmarks. We have one speccpu run which is currently taking forever to finish... -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- amd.tar.bz2 Description: Binary data intel.tar.bz2 Description: Binary data