Re: resistance against single-even upsets
p...@delphinusdns.org(p...@delphinusdns.org) on 2023.03.14 17:32:09 +0100: > >Fix: > I propose all these variables to be monitored occasionally with a CRC > check and if there is a bit flip happening to unset it to the right value. You just invented data scrubbing. Don't forget to occasionally check your CRC check routine integrity as well. After all the bit might flip in the code as well as in the data. > OpenBSD would never be faring to space right? You are probably wrong.
Re: resistance against single-even upsets
On Tue, Mar 14, 2023 at 01:17:00PM -0500, Scott Cheloha wrote:
> > On Mar 14, 2023, at 11:32 AM, p...@delphinusdns.org wrote:
> >
> >> Synopsis: can we resist agains bit flipping?
> >> Category: system
> >> Environment:
> > System : OpenBSD 7.2
> > Details : OpenBSD 7.2 (GENERIC.MP) #2: Thu Nov 24 23:53:03 MST 2022
> > r...@syspatch-72-arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
> >
> > Architecture: OpenBSD.arm64
> > Machine : arm64
> >> Description:
> > https://en.wikipedia.org/wiki/Single-event_upset
> >
> > A single event upset gave someone in belgium who was in a poll, 4096
> > extra votes. When I think about this bit flip and look at the kernel
> > code for an ultra secure operating system there is not much stopping
> > someone to try an attack during a cosmic storm or increased solar
> > activity. Perhaps a bit flips somewhere in the CPU or RAM?
> >
> > pjp@polarstern$ grep sourceroute ip_input.c
> > int ip_dosourceroute = 0;
> >if (!ip_dosourceroute) {
> >if (!ip_dosourceroute)
> >_dosourceroute);
> >
> > Like here. As you know someone found something last week if this were
> > enabled. But the way this check is. It doesn't check for the low bit set
> > to
> > one but it checks for the inverted value, so if the 12th bit was flipped in
> > a
> > solar storm ip_dosourceroute would now be 4096. And the system would be
> > wide
> > open.
> >
> >> How-To-Repeat:
> > Hackers probably check the weather report like
> > https://spaceweather.com/ for increased solar activity and then fill
> > the CPU caches with attempts to get a bit flip happening. The odds
> > aren't in their favour but who knows they may get lucky.
> >> Fix:
> > I propose all these variables to be monitored occasionally with a CRC
> > check and if there is a bit flip happening to unset it to the right value.
> > This is a lot of work but may be worth it. OpenBSD would never be faring to
> > space right? I have no code but trying to think around how to do this.
>
> Why wouldn't you just buy ECC memory?
>
> https://en.wikipedia.org/wiki/ECC_memory
>
Good idea, but nothing is perfect of course :)
An interesting read and research:
https://www.vusec.net/projects/eccploit/
--
Kind regards,
Hiltjo
Re: resistance against single-even upsets
> On Mar 14, 2023, at 11:32 AM, p...@delphinusdns.org wrote:
>
>> Synopsis: can we resist agains bit flipping?
>> Category: system
>> Environment:
> System : OpenBSD 7.2
> Details : OpenBSD 7.2 (GENERIC.MP) #2: Thu Nov 24 23:53:03 MST 2022
> r...@syspatch-72-arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
>
> Architecture: OpenBSD.arm64
> Machine : arm64
>> Description:
> https://en.wikipedia.org/wiki/Single-event_upset
>
> A single event upset gave someone in belgium who was in a poll, 4096
> extra votes. When I think about this bit flip and look at the kernel
> code for an ultra secure operating system there is not much stopping
> someone to try an attack during a cosmic storm or increased solar
> activity. Perhaps a bit flips somewhere in the CPU or RAM?
>
> pjp@polarstern$ grep sourceroute ip_input.c
> int ip_dosourceroute = 0;
>if (!ip_dosourceroute) {
>if (!ip_dosourceroute)
>_dosourceroute);
>
> Like here. As you know someone found something last week if this were
> enabled. But the way this check is. It doesn't check for the low bit set to
> one but it checks for the inverted value, so if the 12th bit was flipped in a
> solar storm ip_dosourceroute would now be 4096. And the system would be wide
> open.
>
>> How-To-Repeat:
> Hackers probably check the weather report like
> https://spaceweather.com/ for increased solar activity and then fill
> the CPU caches with attempts to get a bit flip happening. The odds
> aren't in their favour but who knows they may get lucky.
>> Fix:
> I propose all these variables to be monitored occasionally with a CRC
> check and if there is a bit flip happening to unset it to the right value.
> This is a lot of work but may be worth it. OpenBSD would never be faring to
> space right? I have no code but trying to think around how to do this.
Why wouldn't you just buy ECC memory?
https://en.wikipedia.org/wiki/ECC_memory
Re: resistance against single-even upsets
Peter J. Philipp wrote: > On Tue, Mar 14, 2023 at 10:34:48AM -0600, Theo de Raadt wrote: > > Good god, imagine this bit flip happened *anywhere else*, like in the > > page tables, or in the code or data or stack of chrome, or basically > > *anywhere* > > > > Shall we change them all? > > The example I gave was the last resort other than pf enabled to a kernel > compromise afaik. There is a few of them perhaps, they've not been fixed > for decades lying dormant with a sysctl knob to turn them off. There are a few of them???!?! You must be pretty uneducated about how computation actually happens. There are billions.
Re: resistance against single-even upsets
On Tue, Mar 14, 2023 at 10:34:48AM -0600, Theo de Raadt wrote:
> Good god, imagine this bit flip happened *anywhere else*, like in the
> page tables, or in the code or data or stack of chrome, or basically
> *anywhere*
>
> Shall we change them all?
The example I gave was the last resort other than pf enabled to a kernel
compromise afaik. There is a few of them perhaps, they've not been fixed
for decades lying dormant with a sysctl knob to turn them off.
> Shall we change the compiler to not allow checks like this?
No not at all.
> Shall we wait for a compiler diff from you?
No it's above my head and it would take me decades. :-)
Happy Pi day Theo, it's not quite April 1st but I think this is a bit more
serious. Just think about it, and perhaps in 10-20 years you can consider
it?
Best Regards,
-peter
> p...@delphinusdns.org wrote:
>
> > >Synopsis: can we resist agains bit flipping?
> > >Category: system
> > >Environment:
> > System : OpenBSD 7.2
> > Details : OpenBSD 7.2 (GENERIC.MP) #2: Thu Nov 24 23:53:03 MST 2022
> >
> > r...@syspatch-72-arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
> >
> > Architecture: OpenBSD.arm64
> > Machine : arm64
> > >Description:
> > https://en.wikipedia.org/wiki/Single-event_upset
> >
> > A single event upset gave someone in belgium who was in a poll, 4096
> > extra votes. When I think about this bit flip and look at the kernel
> > code for an ultra secure operating system there is not much stopping
> > someone to try an attack during a cosmic storm or increased solar
> > activity. Perhaps a bit flips somewhere in the CPU or RAM?
> >
> > pjp@polarstern$ grep sourceroute ip_input.c
> > int ip_dosourceroute = 0;
> > if (!ip_dosourceroute) {
> > if (!ip_dosourceroute)
> > _dosourceroute);
> >
> > Like here. As you know someone found something last week if this were
> > enabled. But the way this check is. It doesn't check for the low bit set
> > to
> > one but it checks for the inverted value, so if the 12th bit was flipped in
> > a
> > solar storm ip_dosourceroute would now be 4096. And the system would be
> > wide
> > open.
> >
> > >How-To-Repeat:
> > Hackers probably check the weather report like
> > https://spaceweather.com/ for increased solar activity and then fill
> > the CPU caches with attempts to get a bit flip happening. The odds
> > aren't in their favour but who knows they may get lucky.
> > >Fix:
> > I propose all these variables to be monitored occasionally with a CRC
> > check and if there is a bit flip happening to unset it to the right value.
> > This is a lot of work but may be worth it. OpenBSD would never be faring to
> > space right? I have no code but trying to think around how to do this.
> >
> >
> > dmesg:
> > cut
> >
>
Re: resistance against single-even upsets
This is a hardware issue.
Hardware failure is best guarded against using redundant systems.
Since random errors tend to lead to different outcomes, machines which
agree on what they were told would be the ones which did not
experience hardware failure.
Personally, I'd recommend 5x redundancy for machines where public
safety is a critical issue.
--
Raul
On Tue, Mar 14, 2023 at 12:33 PM wrote:
>
> >Synopsis: can we resist agains bit flipping?
> >Category: system
> >Environment:
> System : OpenBSD 7.2
> Details : OpenBSD 7.2 (GENERIC.MP) #2: Thu Nov 24 23:53:03 MST
> 2022
>
> r...@syspatch-72-arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
>
> Architecture: OpenBSD.arm64
> Machine : arm64
> >Description:
> https://en.wikipedia.org/wiki/Single-event_upset
>
> A single event upset gave someone in belgium who was in a poll, 4096
> extra votes. When I think about this bit flip and look at the kernel
> code for an ultra secure operating system there is not much stopping
> someone to try an attack during a cosmic storm or increased solar
> activity. Perhaps a bit flips somewhere in the CPU or RAM?
>
> pjp@polarstern$ grep sourceroute ip_input.c
> int ip_dosourceroute = 0;
> if (!ip_dosourceroute) {
> if (!ip_dosourceroute)
> _dosourceroute);
>
> Like here. As you know someone found something last week if this were
> enabled. But the way this check is. It doesn't check for the low bit set to
> one but it checks for the inverted value, so if the 12th bit was flipped in a
> solar storm ip_dosourceroute would now be 4096. And the system would be wide
> open.
>
> >How-To-Repeat:
> Hackers probably check the weather report like
> https://spaceweather.com/ for increased solar activity and then fill
> the CPU caches with attempts to get a bit flip happening. The odds
> aren't in their favour but who knows they may get lucky.
> >Fix:
> I propose all these variables to be monitored occasionally with a CRC
> check and if there is a bit flip happening to unset it to the right value.
> This is a lot of work but may be worth it. OpenBSD would never be faring to
> space right? I have no code but trying to think around how to do this.
>
>
> dmesg:
> cut
>
Re: resistance against single-even upsets
Good god, imagine this bit flip happened *anywhere else*, like in the
page tables, or in the code or data or stack of chrome, or basically
*anywhere*
Shall we change them all?
Shall we change the compiler to not allow checks like this?
Shall we wait for a compiler diff from you?
p...@delphinusdns.org wrote:
> >Synopsis:can we resist agains bit flipping?
> >Category:system
> >Environment:
> System : OpenBSD 7.2
> Details : OpenBSD 7.2 (GENERIC.MP) #2: Thu Nov 24 23:53:03 MST 2022
>
> r...@syspatch-72-arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
>
> Architecture: OpenBSD.arm64
> Machine : arm64
> >Description:
> https://en.wikipedia.org/wiki/Single-event_upset
>
> A single event upset gave someone in belgium who was in a poll, 4096
> extra votes. When I think about this bit flip and look at the kernel
> code for an ultra secure operating system there is not much stopping
> someone to try an attack during a cosmic storm or increased solar
> activity. Perhaps a bit flips somewhere in the CPU or RAM?
>
> pjp@polarstern$ grep sourceroute ip_input.c
> int ip_dosourceroute = 0;
> if (!ip_dosourceroute) {
> if (!ip_dosourceroute)
> _dosourceroute);
>
> Like here. As you know someone found something last week if this were
> enabled. But the way this check is. It doesn't check for the low bit set to
> one but it checks for the inverted value, so if the 12th bit was flipped in a
> solar storm ip_dosourceroute would now be 4096. And the system would be wide
> open.
>
> >How-To-Repeat:
> Hackers probably check the weather report like
> https://spaceweather.com/ for increased solar activity and then fill
> the CPU caches with attempts to get a bit flip happening. The odds
> aren't in their favour but who knows they may get lucky.
> >Fix:
> I propose all these variables to be monitored occasionally with a CRC
> check and if there is a bit flip happening to unset it to the right value.
> This is a lot of work but may be worth it. OpenBSD would never be faring to
> space right? I have no code but trying to think around how to do this.
>
>
> dmesg:
> cut
>
resistance against single-even upsets
>Synopsis: can we resist agains bit flipping?
>Category: system
>Environment:
System : OpenBSD 7.2
Details : OpenBSD 7.2 (GENERIC.MP) #2: Thu Nov 24 23:53:03 MST 2022
r...@syspatch-72-arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
Architecture: OpenBSD.arm64
Machine : arm64
>Description:
https://en.wikipedia.org/wiki/Single-event_upset
A single event upset gave someone in belgium who was in a poll, 4096
extra votes. When I think about this bit flip and look at the kernel
code for an ultra secure operating system there is not much stopping
someone to try an attack during a cosmic storm or increased solar
activity. Perhaps a bit flips somewhere in the CPU or RAM?
pjp@polarstern$ grep sourceroute ip_input.c
int ip_dosourceroute = 0;
if (!ip_dosourceroute) {
if (!ip_dosourceroute)
_dosourceroute);
Like here. As you know someone found something last week if this were
enabled. But the way this check is. It doesn't check for the low bit set to
one but it checks for the inverted value, so if the 12th bit was flipped in a
solar storm ip_dosourceroute would now be 4096. And the system would be wide
open.
>How-To-Repeat:
Hackers probably check the weather report like
https://spaceweather.com/ for increased solar activity and then fill
the CPU caches with attempts to get a bit flip happening. The odds
aren't in their favour but who knows they may get lucky.
>Fix:
I propose all these variables to be monitored occasionally with a CRC
check and if there is a bit flip happening to unset it to the right value.
This is a lot of work but may be worth it. OpenBSD would never be faring to
space right? I have no code but trying to think around how to do this.
dmesg:
cut
