Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 08:25:37PM +, Elliott, Robert (Persistent Memory) wrote: > This isn't exactly what you're looking for, but here is > an example of fio doing reads from pmem devices (reading > from NVDIMMs, writing to DIMMs) with various transfer > sizes. ... and "fio" is? > At 256

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 08:25:37PM +, Elliott, Robert (Persistent Memory) wrote: > This isn't exactly what you're looking for, but here is > an example of fio doing reads from pmem devices (reading > from NVDIMMs, writing to DIMMs) with various transfer > sizes. ... and "fio" is? > At 256

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

L > ; linux-kernel@vger.kernel.org; Linux MM m...@kvack.org>; Andy Lutomirski ; Andrew Morton > ; Ingo Molnar > Subject: Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to > recover from machine checks > > On Tue, Dec 15, 2015 at 07:19:58PM +, Elliott, Robert (Persiste

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 07:19:58PM +, Elliott, Robert (Persistent Memory) wrote: ... > Due to the historic long latency of storage devices, > applications don't re-read from storage again; they > save the results. > So, the streaming-load instructions are beneficial: That's the theory...

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

ger.kernel.org; Linux MM ; Andy Lutomirski > ; Andrew Morton ; Ingo Molnar > > Subject: Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to > recover from machine checks > > On Tue, Dec 15, 2015 at 10:35:49AM -0800, Dan Williams wrote: > > Correction we have

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

>> ... and the non-temporal version is the optimal one even though we're >> defaulting to copy_user_enhanced_fast_string for memcpy on modern Intel >> CPUs...? My current generation cpu has a bit of an issue with recovering from a machine check in a "rep mov" ... so I'm working with a version of

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 5:11 AM, Borislav Petkov wrote: > On Thu, Dec 10, 2015 at 04:21:50PM -0800, Tony Luck wrote: >> Using __copy_user_nocache() as inspiration create a memory copy >> routine for use by kernel code with annotations to allow for >> recovery from machine checks. >> >> Notes: >>

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 10:35:49AM -0800, Dan Williams wrote: > Correction we have MOVNTDQA, but that requires saving the fpu state > and marking the memory as WC, i.e. probably not worth it. Not really. Last time I tried an SSE3 memcpy in the kernel like glibc does, it wasn't worth it. The

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 9:53 AM, Luck, Tony wrote: >>> ... and the non-temporal version is the optimal one even though we're >>> defaulting to copy_user_enhanced_fast_string for memcpy on modern Intel >>> CPUs...? > > My current generation cpu has a bit of an issue with recovering from a >

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 10:27 AM, Dan Williams wrote: > On Tue, Dec 15, 2015 at 9:53 AM, Luck, Tony wrote: ... and the non-temporal version is the optimal one even though we're defaulting to copy_user_enhanced_fast_string for memcpy on modern Intel CPUs...? >> >> My current

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 05:53:31PM +, Luck, Tony wrote: > My current generation cpu has a bit of an issue with recovering from a > machine check in a "rep mov" ... so I'm working with a version of memcpy > that unrolls into individual mov instructions for now. Ah. > I can drop the "nti" from

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Thu, Dec 10, 2015 at 04:21:50PM -0800, Tony Luck wrote: > Using __copy_user_nocache() as inspiration create a memory copy > routine for use by kernel code with annotations to allow for > recovery from machine checks. > > Notes: > 1) Unlike the original we make no attempt to copy all the bytes

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 9:53 AM, Luck, Tony wrote: >>> ... and the non-temporal version is the optimal one even though we're >>> defaulting to copy_user_enhanced_fast_string for memcpy on modern Intel >>> CPUs...? > > My current generation cpu has a bit of an issue with

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 10:35:49AM -0800, Dan Williams wrote: > Correction we have MOVNTDQA, but that requires saving the fpu state > and marking the memory as WC, i.e. probably not worth it. Not really. Last time I tried an SSE3 memcpy in the kernel like glibc does, it wasn't worth it. The

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

m nvd...@ml01.01.org>; X86 ML <x...@kernel.org>; linux- > ker...@vger.kernel.org; Linux MM <linux...@kvack.org>; Andy Lutomirski > <l...@kernel.org>; Andrew Morton <a...@linux-foundation.org>; Ingo Molnar > <mi...@kernel.org> > Subject: Re: [PATCHV2 3

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 05:53:31PM +, Luck, Tony wrote: > My current generation cpu has a bit of an issue with recovering from a > machine check in a "rep mov" ... so I'm working with a version of memcpy > that unrolls into individual mov instructions for now. Ah. > I can drop the "nti" from

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 10:27 AM, Dan Williams wrote: > On Tue, Dec 15, 2015 at 9:53 AM, Luck, Tony wrote: ... and the non-temporal version is the optimal one even though we're defaulting to copy_user_enhanced_fast_string for memcpy on

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 5:11 AM, Borislav Petkov wrote: > On Thu, Dec 10, 2015 at 04:21:50PM -0800, Tony Luck wrote: >> Using __copy_user_nocache() as inspiration create a memory copy >> routine for use by kernel code with annotations to allow for >> recovery from machine checks.

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Tue, Dec 15, 2015 at 07:19:58PM +, Elliott, Robert (Persistent Memory) wrote: ... > Due to the historic long latency of storage devices, > applications don't re-read from storage again; they > save the results. > So, the streaming-load instructions are beneficial: That's the theory...

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

>> ... and the non-temporal version is the optimal one even though we're >> defaulting to copy_user_enhanced_fast_string for memcpy on modern Intel >> CPUs...? My current generation cpu has a bit of an issue with recovering from a machine check in a "rep mov" ... so I'm working with a version of

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

Molnar <mi...@kernel.org> > Subject: Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to > recover from machine checks > > On Tue, Dec 15, 2015 at 07:19:58PM +, Elliott, Robert (Persistent > Memory) wrote: > > ... > > > Due to the historic long latency

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Thu, Dec 10, 2015 at 04:21:50PM -0800, Tony Luck wrote: > Using __copy_user_nocache() as inspiration create a memory copy > routine for use by kernel code with annotations to allow for > recovery from machine checks. > > Notes: > 1) Unlike the original we make no attempt to copy all the bytes

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Mon, Dec 14, 2015 at 11:46 AM, Luck, Tony wrote: > On Mon, Dec 14, 2015 at 09:36:25AM +0100, Ingo Molnar wrote: >> > /* deal with it */ >> > >> > That way the magic is isolated to the function that needs the magic. >> >> Seconded - this is the usual pattern we use in all assembly

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Mon, Dec 14, 2015 at 09:36:25AM +0100, Ingo Molnar wrote: > > /* deal with it */ > > > > That way the magic is isolated to the function that needs the magic. > > Seconded - this is the usual pattern we use in all assembly functions. Ok - you want me to write some x86 assembly code (you

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

* Andy Lutomirski wrote: > I still think it would be better if you get rid of BIT(63) and use a > pair of landing pads, though. They could be as simple as: > > .Lpage_fault_goes_here: > xorq %rax, %rax > jmp .Lbad > > .Lmce_goes_here: > /* set high bit of rax or whatever */ >

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Mon, Dec 14, 2015 at 09:36:25AM +0100, Ingo Molnar wrote: > > /* deal with it */ > > > > That way the magic is isolated to the function that needs the magic. > > Seconded - this is the usual pattern we use in all assembly functions. Ok - you want me to write some x86 assembly code (you

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Mon, Dec 14, 2015 at 11:46 AM, Luck, Tony wrote: > On Mon, Dec 14, 2015 at 09:36:25AM +0100, Ingo Molnar wrote: >> > /* deal with it */ >> > >> > That way the magic is isolated to the function that needs the magic. >> >> Seconded - this is the usual pattern we use in

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

* Andy Lutomirski wrote: > I still think it would be better if you get rid of BIT(63) and use a > pair of landing pads, though. They could be as simple as: > > .Lpage_fault_goes_here: > xorq %rax, %rax > jmp .Lbad > > .Lmce_goes_here: > /* set high bit of rax

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 2:45 PM, Luck, Tony wrote: >>> But a machine check safe copy_from_user() would be useful >>> current generation cpus that broadcast all the time. >> >> Fair enough. > > Thanks for spending the time to look at this. Coaxing me to re-write the > tail of do_machine_check()

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

>> But a machine check safe copy_from_user() would be useful >> current generation cpus that broadcast all the time. > > Fair enough. Thanks for spending the time to look at this. Coaxing me to re-write the tail of do_machine_check() has made that code much better. Too many years of one patch on

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 2:35 PM, Luck, Tony wrote: >> Also, are there really PCOMMIT-capable CPUs that still forcibly >> broadcast MCE? If, so, that's unfortunate. > > PCOMMIT and LMCE arrive together ... though BIOS is in the decision > path to enable LMCE, so it is possible that some systems

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

> Also, are there really PCOMMIT-capable CPUs that still forcibly > broadcast MCE? If, so, that's unfortunate. PCOMMIT and LMCE arrive together ... though BIOS is in the decision path to enable LMCE, so it is possible that some systems could still broadcast if the BIOS writer decides to not

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 2:20 PM, Dan Williams wrote: > On Fri, Dec 11, 2015 at 2:17 PM, Luck, Tony wrote: >>> Also, would it be more straightforward if the mcexception landing pad >>> looked up the va -> pa mapping by itself? Or is that somehow not >>> reliable? >> >> If we did get all the

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 2:17 PM, Luck, Tony wrote: >> Also, would it be more straightforward if the mcexception landing pad >> looked up the va -> pa mapping by itself? Or is that somehow not >> reliable? > > If we did get all the above right, then we could have > target use virt_to_phys() to

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

> I'm missing something, though. The normal fixup_exception path > doesn't touch rax at all. The memory_failure path does. But couldn't > you distinguish them by just pointing the exception handlers at > different landing pads? Perhaps I'm just trying to take a short cut to avoid writing some

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 1:19 PM, Luck, Tony wrote: >> I still don't get the BIT(63) thing. Can you explain it? > > It will be more obvious when I get around to writing copy_from_user(). > > Then we will have a function that can take page faults if there are pages > that are not present. If the

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

> I still don't get the BIT(63) thing. Can you explain it? It will be more obvious when I get around to writing copy_from_user(). Then we will have a function that can take page faults if there are pages that are not present. If the page faults can't be fixed we have a -EFAULT condition. We

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Thu, Dec 10, 2015 at 4:21 PM, Tony Luck wrote: > Using __copy_user_nocache() as inspiration create a memory copy > routine for use by kernel code with annotations to allow for > recovery from machine checks. > > Notes: > 1) Unlike the original we make no attempt to copy all the bytes >up

[PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

Using __copy_user_nocache() as inspiration create a memory copy routine for use by kernel code with annotations to allow for recovery from machine checks. Notes: 1) Unlike the original we make no attempt to copy all the bytes up to the faulting address. The original achieves that by

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 1:19 PM, Luck, Tony wrote: >> I still don't get the BIT(63) thing. Can you explain it? > > It will be more obvious when I get around to writing copy_from_user(). > > Then we will have a function that can take page faults if there are pages > that are

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 2:17 PM, Luck, Tony wrote: >> Also, would it be more straightforward if the mcexception landing pad >> looked up the va -> pa mapping by itself? Or is that somehow not >> reliable? > > If we did get all the above right, then we could have > target use

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

> Also, are there really PCOMMIT-capable CPUs that still forcibly > broadcast MCE? If, so, that's unfortunate. PCOMMIT and LMCE arrive together ... though BIOS is in the decision path to enable LMCE, so it is possible that some systems could still broadcast if the BIOS writer decides to not

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

> I still don't get the BIT(63) thing. Can you explain it? It will be more obvious when I get around to writing copy_from_user(). Then we will have a function that can take page faults if there are pages that are not present. If the page faults can't be fixed we have a -EFAULT condition. We

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 2:45 PM, Luck, Tony wrote: >>> But a machine check safe copy_from_user() would be useful >>> current generation cpus that broadcast all the time. >> >> Fair enough. > > Thanks for spending the time to look at this. Coaxing me to re-write the > tail of

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

> I'm missing something, though. The normal fixup_exception path > doesn't touch rax at all. The memory_failure path does. But couldn't > you distinguish them by just pointing the exception handlers at > different landing pads? Perhaps I'm just trying to take a short cut to avoid writing some

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 2:35 PM, Luck, Tony wrote: >> Also, are there really PCOMMIT-capable CPUs that still forcibly >> broadcast MCE? If, so, that's unfortunate. > > PCOMMIT and LMCE arrive together ... though BIOS is in the decision > path to enable LMCE, so it is

RE: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

>> But a machine check safe copy_from_user() would be useful >> current generation cpus that broadcast all the time. > > Fair enough. Thanks for spending the time to look at this. Coaxing me to re-write the tail of do_machine_check() has made that code much better. Too many years of one patch on

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Fri, Dec 11, 2015 at 2:20 PM, Dan Williams wrote: > On Fri, Dec 11, 2015 at 2:17 PM, Luck, Tony wrote: >>> Also, would it be more straightforward if the mcexception landing pad >>> looked up the va -> pa mapping by itself? Or is that somehow not

[PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

Using __copy_user_nocache() as inspiration create a memory copy routine for use by kernel code with annotations to allow for recovery from machine checks. Notes: 1) Unlike the original we make no attempt to copy all the bytes up to the faulting address. The original achieves that by

Re: [PATCHV2 3/3] x86, ras: Add mcsafe_memcpy() function to recover from machine checks

On Thu, Dec 10, 2015 at 4:21 PM, Tony Luck wrote: > Using __copy_user_nocache() as inspiration create a memory copy > routine for use by kernel code with annotations to allow for > recovery from machine checks. > > Notes: > 1) Unlike the original we make no attempt to copy