Re: gcc spectre stuff for netbsd-8
Additionally, why skipping -current entirely: - GCC 7.3 already has an official backport, and we might update to this. - I'm not good at compilers p.s. for people who desperately want -current patches, we can still use GCC 5.5 on current with HAVE_GCC=5.
Re: gcc spectre stuff for netbsd-8
sorry for being terse, I'm not very knowledgeable about GCC/x86 but applied it to our copy of GCC. what it should be adding Add -mindirect-branch-register to force indirect branch via register. This is implemented by disabling patterns of indirect branch via memory, similar to TARGET_X32. https://gcc.gnu.org/ml/gcc-patches/2018-01/msg00427.html Add -mindirect-branch= option to convert indirect call and jump to call and return thunks. The default is 'keep', which keeps indirect call and jump unmodified. 'thunk' converts indirect call and jump to call and return thunk. 'thunk-inline' converts indirect call and jump to inlined call and return thunk. 'thunk-extern' converts indirect call and jump to external call and return thunk provided in a separate object file. You can control this behavior for a specific function by using the function attribute indirect_branch. https://gcc.gnu.org/ml/gcc-patches/2018-01/msg00424.html Add -mfunction-return= option to convert function return to call and return thunks. The default is 'keep', which keeps function return unmodified. 'thunk' converts function return to call and return thunk. 'thunk-inline' converts function return to inlined call and return thunk. 'thunk-extern' converts function return to external call and return thunk provided in a separate object file. You can control this behavior for a specific function by using the function attribute function_return. https://gcc.gnu.org/ml/gcc-patches/2018-01/msg00426.html
gcc spectre stuff for netbsd-8
Hi, I've done the following, hopefully it will enable others. Completed: - Builds amd64 tools - Builds working amd64 kernel (*without new mitigations*!) Need: - Test full release, stress test - Test i386 too - Decide which spectre mitigation to use - Test that kernel with that mitigation works - Also, modules! - Run the new testcases from GCC testsuite which were added with this feature Maybe: - Figure out what I just did to the documentation? Patch: http://netbsd.org/~maya/netbsd8-gcc-spectre-20180312 (Originally from opensuse's gcc patch, and comparing a lot to gcc-HEAD where it was insufficient)
Re: Spectre on non-amd64
Hello, On Fri, 19 Jan 2018 21:55:12 + <paul.kon...@dell.com> wrote: > > On Jan 19, 2018, at 4:47 PM, <m...@netbsd.org> <m...@netbsd.org> wrote: > > > > Hi folks. > > > > I think that the spectre variant 2 situation is a lot worse for: > > - Speculative CPU > > - Weak memory protection > > > > Then I don't need a JIT for gadgets. > > > > Architectures that fall into this: > > - default i386 netbsd, because it is missing NX bit (PAE is optional) > > - MIPS for us, because we don't use kseg2 and then it doesn't go through > > MMU. > > which MIPS do speculative execution? R1xk, Loongson ( at least since L2 ), probably quite a few modern ones. have fun Michael
Re: Spectre on non-amd64
> On Jan 19, 2018, at 4:47 PM, <m...@netbsd.org> <m...@netbsd.org> wrote: > > Hi folks. > > I think that the spectre variant 2 situation is a lot worse for: > - Speculative CPU > - Weak memory protection > > Then I don't need a JIT for gadgets. > > Architectures that fall into this: > - default i386 netbsd, because it is missing NX bit (PAE is optional) > - MIPS for us, because we don't use kseg2 and then it doesn't go through > MMU. which MIPS do speculative execution? paul
Re: Spectre on non-amd64
Spectre variant 2 also relies on me being able to poison the branch target buffer. loongson had a similar issue where errant BTB entries would cause hangs, and they then claimed this at kernel entry clears out their BTB:* jal 1f nop 1: jal 1f nop 1: jal 1f nop 1: jal 1f nop 1: A question is whether that works for other branch predictors, and how many we would need. * https://github.com/torvalds/linux/blob/2d6349944d967129c1da3c47287376f10121dbe1/arch/mips/include/asm/stackframe.h#L152-L164
Spectre on non-amd64
Hi folks. I think that the spectre variant 2 situation is a lot worse for: - Speculative CPU - Weak memory protection Then I don't need a JIT for gadgets. Architectures that fall into this: - default i386 netbsd, because it is missing NX bit (PAE is optional) - MIPS for us, because we don't use kseg2 and then it doesn't go through MMU. No NX bit: - Make a file, the contents of it is a spectre gadget - Put it in buffer cache - Poison branch predictor, which will speculatively execute the contents of this file No SMEP: - Locally create a spectre gadget and make it executable - Poison branch predictor to jump to my user-memory gadget - Enter kernel (Maybe helped by Meltdown fixes, if they are early enough) Now I am not sure how MMUs work, but I think that even if - Kernel has its own ASID - But... we haven't switched to it yet before performing a branch Then at the early branches I could speculate-execute some user code.
Re: Spectre
m...@netbsd.org wrote: > Considering JITs are a much bigger risk, and how cheap this suggestion > is, should we use lfence / similar for other architectures within sljit > (and possibly lua)? While everyone seems to be concerned on negative performance impact of Spectre, I recently worked on preventing speculative loads to avoid trashing caches and keeping tails of performance in good shape. I found that lfence was ineffective and I had to insert data dependency and some extra work to distract processor. It worked much better and it improved latencies in tails. If you look at Intel's documentation of lfence, they make it very clear that lfence doesn't prevent speculative loads. -- Alex
Re: Spectre
Considering JITs are a much bigger risk, and how cheap this suggestion is, should we use lfence / similar for other architectures within sljit (and possibly lua)? The slides also talk about adding rules to detect likely gadgets to coverity. I wonder whether they can be added also to static analyzers we can actually use, like clang-static-analyzer.
Re: Spectre
> On Jan 18, 2018, at 10:31 AM, Mouse <mo...@rodents-montreal.org> wrote: > >> ... > >> The Spectre fixes all amount to a speculative barrier, which will do >> the job just as well (though it requires code change). > > Yes...but it requires a code change in the wrong place. > > That "if (access is ok)" check that needs a spec ex barrier could well > be inside a library that doesn't want to cripple performance for > non-sandboxed applications. See also the spectre paper's description > of use of code that doesn't think it's making an access check but > happens to contain an instruction sequence that can be used that way. > > I'd prefer to have a spec ex disable bit which the sandbox could set > for the duration of the sandboxed code. That's an option. But for regular (not indirect) branches like the example access check, most cases are not Spectre risks. There is only an issue if the speculatively loaded code is subjected to data dependent actions that are visible through side channels. For a lot of code, that doesn't apply. So yes, a spec ex barrier after such checks can affect performance somewhat. But a "spec ex disable" bit that utterly turns off speculative execution for sandboxed code will have an impact that's massively larger. Easier to apply, sure -- well, once you have those bits which of course right now you do not. But a great deal more costly than spec ex barriers applied with skill. paul
Re: Spectre
> It seems to me that blocking all observable side effects of > speculative execution can probably only be done by disabling > speculative execution outright. Hmm. Probably. > That clearly isn't a good thing. I agree it would not be a good thing to do globally. Too many things care too much about performance. But I do think it is a good thing to have available. Sandboxes, for example, might want to do so when executing sandboxed code. (Might. Not, of course, always will; see also below.) > The Spectre fixes all amount to a speculative barrier, which will do > the job just as well (though it requires code change). Yes...but it requires a code change in the wrong place. That "if (access is ok)" check that needs a spec ex barrier could well be inside a library that doesn't want to cripple performance for non-sandboxed applications. See also the spectre paper's description of use of code that doesn't think it's making an access check but happens to contain an instruction sequence that can be used that way. I'd prefer to have a spec ex disable bit which the sandbox could set for the duration of the sandboxed code. (Actually, I'd like to have "two and a half" such bits. I'd like a privileged three-way choice and an unprivileged bit: the privileged choice would be spec ex always on, spec ex always off, or spec ex controlled by the unprivileged bit. (I'm not sure there's much use case for the privileged "always on" choice, but it would be easy to do and could be useful for something like using normally-sandboxed environments for running known-non-malicious code.) /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Spectre
On Thu, Jan 18, 2018 at 7:58 AM, <paul.kon...@dell.com> wrote: > > > > On Jan 18, 2018, at 9:48 AM, Mouse <mo...@rodents-montreal.org> wrote: > > > >> Since this involves a speculative load that is legal from the > >> hardware definition point of view (the load is done by kernel code), > >> this isn't a hardware bug the way Meltdown is. > > > > Well, I'd say it's the same fundamental hardware bug as meltdown, but > > not compounded by an additional hardware property (which I'm not sure I > > would call a bug) which is made much worse by the actual bug. > > > > To my mind, the bug here is that annulling spec ex doesn't annul _all_ > > its effects. That, fundamentally, is what's behind both spectre and > > meltdown. In meltdown it's exacerbated by spec ex's failure to check > > permissions fully - but if the side effects were annulled correctly, > > even that failure wouldn't cause trouble. > > That's true. But the problem is that cache fill is only the most > obvious and easiest to exploit side channel. There are others, such > as timing due to execution units being busy, that are harder to exploit > but also harder to cure. It seems to me that blocking all observable > side effects of speculative execution can probably only be done by > disabling speculative execution outright. That clearly isn't a good > thing. The Spectre fixes all amount to a speculative barrier, which > will do the job just as well (though it requires code change). The > Meltdown fix is more obvious: don't omit mode dependent access checks > before launching a speculative load, as most CPU designers already did. > One difficulty with caches: You'd have to re-cache what you eject, otherwise there's an observable effect. That's the whole point of this family of attacks: the micro architecture does something that you can observe that you'd normally not be able to observe. It's really really hard to not leak any side-channel data at all. Side channel has become the new buffer overflow. Warner
Re: Spectre
> On Jan 18, 2018, at 9:48 AM, Mouse <mo...@rodents-montreal.org> wrote: > >> Since this involves a speculative load that is legal from the >> hardware definition point of view (the load is done by kernel code), >> this isn't a hardware bug the way Meltdown is. > > Well, I'd say it's the same fundamental hardware bug as meltdown, but > not compounded by an additional hardware property (which I'm not sure I > would call a bug) which is made much worse by the actual bug. > > To my mind, the bug here is that annulling spec ex doesn't annul _all_ > its effects. That, fundamentally, is what's behind both spectre and > meltdown. In meltdown it's exacerbated by spec ex's failure to check > permissions fully - but if the side effects were annulled correctly, > even that failure wouldn't cause trouble. That's true. But the problem is that cache fill is only the most obvious and easiest to exploit side channel. There are others, such as timing due to execution units being busy, that are harder to exploit but also harder to cure. It seems to me that blocking all observable side effects of speculative execution can probably only be done by disabling speculative execution outright. That clearly isn't a good thing. The Spectre fixes all amount to a speculative barrier, which will do the job just as well (though it requires code change). The Meltdown fix is more obvious: don't omit mode dependent access checks before launching a speculative load, as most CPU designers already did. paul
Re: Spectre
> Since this involves a speculative load that is legal from the > hardware definition point of view (the load is done by kernel code), > this isn't a hardware bug the way Meltdown is. Well, I'd say it's the same fundamental hardware bug as meltdown, but not compounded by an additional hardware property (which I'm not sure I would call a bug) which is made much worse by the actual bug. To my mind, the bug here is that annulling spec ex doesn't annul _all_ its effects. That, fundamentally, is what's behind both spectre and meltdown. In meltdown it's exacerbated by spec ex's failure to check permissions fully - but if the side effects were annulled correctly, even that failure wouldn't cause trouble. > But it's an issue that requires a fix -- which is a speculative > execution barrier between the software access check, and the > subsequent code that is legal only if the check is successful. The _right_ fix is for the side effects (cache, among others) to be annulled along with the overt effects when spec ex is annulled. Spec ex barriers are just a workaround to hide the worst of the effects of the bug in buggy hardware - there's so much buggy hardware out in the field that _some_ kind of mitigation measure is appropriate - but workarounds should not be confused with fixes. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Spectre
> Consider something like BPF [...] Yes, that's the "on the one hand" part: using this property to read outside a sandbox. > -- code executed in the kernel with an enforced security model to > prevent "undesirable" acceses. Such CPUs are inappropriate for such uses. Whether that's a bug in the CPU or a bug in its application depends on how you prefer to think of it. Since, as far as I know, such CPUs have historically not been documented with warnings about such things, I'm more inclined to see it as a bug in the CPU at present - but I can certainly understand the other point of view. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Spectre
> On Jan 18, 2018, at 8:49 AM, Joerg Sonnenberger <jo...@bec.de> wrote: > > On Wed, Jan 17, 2018 at 09:38:27PM -0500, Mouse wrote: >> But, on the other hand, I can easily imagine a CPU designer looking at >> it and saying "What's the big deal if this code can read that location? >> It can get it anytime it wants with a simple load instruction anyway.", >> something I have trouble disagreeing with. > > Consider something like BPF -- code executed in the kernel with an > enforced security model to prevent "undesirable" acceses. It will create > logic like: > >void *p = ...; >if (!is_accesible(p)) > raise_error(); >load(p); > > Now imagine that the expression for p is intentionally pointing into > userland and depends on the speculative execution of something else. > Loading the pointer speculatively results in a visible side effect that > defeats in part the access check. In short, it can effectively invert > access control checks for verified code. Yes, you've just described Spectre. Since this involves a speculative load that is legal from the hardware definition point of view (the load is done by kernel code), this isn't a hardware bug the way Meltdown is. But it's an issue that requires a fix -- which is a speculative execution barrier between the software access check, and the subsequent code that is legal only if the check is successful. paul
Re: Spectre
On Wed, Jan 17, 2018 at 09:38:27PM -0500, Mouse wrote: > But, on the other hand, I can easily imagine a CPU designer looking at > it and saying "What's the big deal if this code can read that location? > It can get it anytime it wants with a simple load instruction anyway.", > something I have trouble disagreeing with. Consider something like BPF -- code executed in the kernel with an enforced security model to prevent "undesirable" acceses. It will create logic like: void *p = ...; if (!is_accesible(p)) raise_error(); load(p); Now imagine that the expression for p is intentionally pointing into userland and depends on the speculative execution of something else. Loading the pointer speculatively results in a visible side effect that defeats in part the access check. In short, it can effectively invert access control checks for verified code. Joerg
Re: Spectre
It's a lot less obvious from a CPU designer perspective. One will make the speculative bits, declare 'all the actions I do are rolled back, so this is perfectly safe!' and someone else making the cache doesn't realize that the reads were speculative and their effects should have been rolled back. People were talking about timing attacks for a while, but somehow it hadn't clicked that: array[*malicious_address & 1] Actually leaks (via timing) the content of the first bit of malicious_address. Or maybe it's just hadn't for me.
Re: Spectre
> Spectre is unrelated and does not depend on a mistake of this kind, > since there you're dealing with speculative loads that ARE permitted > as far as access control goes; they just aren't wanted because they > are preceded by range checks or the like. Yes. I'm of two minds whether it's even fair to call spectre variants like that a vulnerability. (Spectre variants that exfiltrate values from other processes, or from the kernel, are quite another story.) On the one hand, of course, it is, in that it can be used to do things like read outside sandboxes. But, on the other hand, I can easily imagine a CPU designer looking at it and saying "What's the big deal if this code can read that location? It can get it anytime it wants with a simple load instruction anyway.", something I have trouble disagreeing with. So, I'm not sure whether I consider those spectre variants a CPU bug or just a misfeature that makes sandboxing more difficult (in that it provides unobvious ways to read memory). /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Spectre
On Thu, Jan 18, 2018 at 12:37:11AM +, co...@sdf.org wrote: > - Variant 1 seems possible to avoid with low cost. It will likely result > in an error somewhere along the line, which is detectable. Flushing > the entire cache on userret will make it hard to exploit. Do all > bound checks failing result in an easily noticed error? On further thought, this won't be sufficient protection. It only protects assuming we: - Cache memory - Time access to memory - Flush cache - Time access to now cached memory Flushing on userret/kernel entry is not sufficient because I am assuming it will need a second trip to userland. it does not, assuming we can construct our own perfect gadget.
Re: Spectre
On Wed, Jan 17, 2018 at 08:08:06PM -0500, Mouse wrote: > Unfortunately, they appear to be exported only on the Web, and even > then only over HTTPS. I can send copies privately to anyone for whom > those are obstacles (probably not very many, but they were for me). > https://spectreattack.com/ and https://meltdownattack.com/ are the URLs > I've found, though (as implied above) I haven't actually verified them > myself. Hi Mouse, 'wget -r' is often effective at fetching the contents of websites. It handles this website. If the problem is the website it might help.
Re: Spectre
> On Jan 17, 2018, at 8:08 PM, Mouse <mo...@rodents-montreal.org> wrote: > > ... >> - Even speculative execution obeys access restrictions, > > In some respects. Meltdown is possible because Intel spec ex does not > obey access restrictions in one particular respect; I don't know what > aspects may not be obeyed by what CPUs except for that. Indeed. I was surprised, but apparently that "obeys..." is wrong in the case of Intel, though it is correct, as you might expect, for AMD and ARM and probably most other architectures. More precisely, speculative execution obeys access restrictions in the sense that no architecturally visible (i.e., register/memory) changes occur that are prohibited by the access controls. But Intel does launch a speculative load without checking access; apparently the access check is done in parallel and will complete a while later, by which time the speculatively loaded data is in the cache and some other operations may be done based on it. Obviously, if speculative loads check permissions prior to launching the load, the issue goes away. If so, Meltdown is completely prevented. Spectre is unrelated and does not depend on a mistake of this kind, since there you're dealing with speculative loads that ARE permitted as far as access control goes; they just aren't wanted because they are preceded by range checks or the like. paul
Re: Spectre
> I'd suggest reading the papers describing spectre and meltdown. They are fairly readable - I would expect anyone working on the NetBSD kernel to be competent to understand them - and they describe the vulnerabilities, and how the authors exploited them, in reasonable detail. Unfortunately, they appear to be exported only on the Web, and even then only over HTTPS. I can send copies privately to anyone for whom those are obstacles (probably not very many, but they were for me). https://spectreattack.com/ and https://meltdownattack.com/ are the URLs I've found, though (as implied above) I haven't actually verified them myself. > Spectre is also a vulnerability. > - Even speculative execution obeys access restrictions, In some respects. Meltdown is possible because Intel spec ex does not obey access restrictions in one particular respect; I don't know what aspects may not be obeyed by what CPUs except for that. > - Variant 1 seems possible to avoid with low cost. It will likely > result in an error somewhere along the line, which is detectable. Sometimes. Doing the operation inside a transaction apparently will suppress the memory fault in at least some cases. Executing the whole thing under spec ex of a mispredicted branch definitely will annul the trap, but, from reading the papers, it appears they haven't tested it, so it's speculation (hah), albeit reasonable speculation, that it would be exploitable that way. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Spectre
On Thu, Jan 18, 2018 at 12:37:11AM +, co...@sdf.org wrote: > - Intel and AMD have pushed microcode updates that introduce > instructions, for Intel they are 'IBPB' and 'IBRS'. + STIBP for Intel. > - I'm not sure how easy it is to find a good enough gadget without > something like eBPF Lua is probably a good target (though we don't normally have it loaded).
Spectre
Hi folks. Spectre is also a vulnerability. It's been discussed in the context of web browsers, but it does have repercussions for kernel. The gist of it seems to be, if you have code that looks like this: Variant 1: if (malicious offset is safe) value = array[malicious_offset]; value2 = (value & 1) value3 = array2[value2] You can read arbitrary memory, as cached memory will be faster. repeat this bit by bit. Variant 2: relies on poisoning the branch predictor if_transmit = mydriver_transmit; .. regular code if_transmit(); /* indirect call */ .. malicious_ptr: value = array[malicious_offset]; value2 = (value & 1) value3 = array2[value2] Now, we poison the branch target buffer, to predict that if_transmit = malicious_ptr. It does not even appear like a possible code path, but it is. This means that code being loaded is enough to pose a risk, even if there is no easy path to it. Some things that help this: - Rather than look for a gadget like this, they went for eBPF, which is like dtrace. It also has a JIT. You can load bytecode and it will verify and execute it. So you can make your own speculation-gadget. - Some archs do not allow fine-grained cache control like x86 clflush, but this isn't necessary for abuse. just fill the cache before going. they have done similar from JavaScript. Some that harm it: - Even speculative execution obeys access restrictions, so SMEP, SMAP, marking memory NX will prevent execution. - Instructions like 'lfence' stop speculative execution. - Intel and AMD have pushed microcode updates that introduce instructions, for Intel they are 'IBPB' and 'IBRS'. - ARM has 'BPIALL' to invalidate the branch target to stop branch predictor poisoning - Retpoline, stuff_RSB etc. to modify jumps to be less likely to jump into user-controlled code. People seemed to occasionally mentioned that 'as many as 100 instructions will be executed speculatively', so that is how far we are talking. Own thoughts: - Variant 1 seems possible to avoid with low cost. It will likely result in an error somewhere along the line, which is detectable. Flushing the entire cache on userret will make it hard to exploit. Do all bound checks failing result in an easily noticed error? - Variant 2 however is harder without custom instructions. Those may rely on a CPU being new enough to receive microcode updates, and them being applied. - I'm not sure how easy it is to find a good enough gadget without something like eBPF, but the mere presence of the code in memory was a risk. - Linux could've marked eBPF non-executable when it is unused, so its presence in memory does not pose risk. Discuss.
