Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Ard Biesheuvel]

On 30 November 2015 at 10:39, Michael Zimmermann
<sigmaepsilo...@gmail.com> wrote:
>> Unrelated note: *please* get rid of the ARM BDS and use the Intel BDS
>> instead. The ARM BDS violates the UEFI spec, (i.e., you cannot boot
>> installer images from it without a lot of hassle) and does not allow
>> you to enable UEFI secure boot.
>>
>> Look at the development history of ArmVExpress-FVP-AArch64.dsc for
>> hints how to do this.
>
> To make this easier for other, here is my commit for switching to Intel's
> BDS(my code is based on ArmPlatformPkg/ArmPlatformPkg-2ndstage):
> https://github.com/efidroid/uefi_edk2packages_LittleKernelPkg/commit/dbbf212823046e00db3879c6846d96532f703e30
>
> You may also need dynamic PCD support:
> https://github.com/efidroid/uefi_edk2packages_LittleKernelPkg/commit/673cd3bc2d71bf077e5744d9de1597a356b6d2d5
>

Thanks for sharing that. I hope it will be useful to other.

Since the Intel BDS support is fairly recent, please don't hesitate to
share observations or questions. There may be some issues lurking that
we haven't spotted yet ourselves.

Thanks,
Ard.


> On Tue, Nov 24, 2015 at 8:28 AM, Ard Biesheuvel <ard.biesheu...@linaro.org>
> wrote:
>>
>> On 24 November 2015 at 00:05, Vladimir Olovyannikov
>> <volov...@broadcom.com> wrote:
>> >> -Original Message-
>> >> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>> >> Sent: Wednesday, November 18, 2015 11:32 PM
>> >> To: Vladimir Olovyannikov
>> >> Cc: Mark Rutland; edk2-devel@lists.01.org
>> >> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the
>> >> UEFI
>> >>
>> >> On 19 November 2015 at 05:48, Vladimir Olovyannikov
>> >> <volov...@broadcom.com> wrote:
>> >> >
>> >> >
>> > [...]
>> >> > A side note: I got the u-boot source for that board and there are
>> >> > several
>> >> hacks made to avoid SError (writing 0x20 to the HCR register (reroute
>> >> SError
>> >> to EL2), and
>> >> > just ERET from SError exception handler, and then write 0x0 to HCR
>> >> > before
>> >> Linux boots), so it could be an HW issue I am not aware of as of yet.
>> >>
>> >> OK, that would explain it. Note that the kernel will replace the EL2
>> >> vector table if booted at EL2, so this is definitely not a workaround
>> >> that you would want to reuse in UEFI.
>> > [...]
>> >
>> > [...]
>> >> >>
>> >> >> I'd still like to double check the value of VBAR_EL2 as it is
>> >> >> written,
>> >> >> it should be a multiple of 2 KB
>> >> > It is always at 0x85008800 which is a multiple of 2K
>> >> > Any other things to verify?
>> >> >
>> >>
>> >> No, that looks fine. You need to get in touch with the authors of the
>> >> U-Boot code to figure out what it is they are working around. Simply
>> >> ignoring SErrors is obviously not a long term solution.
>> >>
>> > Thanks a lot for your help Ard, Mark,
>> > It is the ATF which has an issue. I will post additional information if
>> > there is still a problem after ATF implementation is fixed
>> >
>>
>> OK.
>> Thanks for reporting back.
>>
>> --
>> Ard.
>> ___
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel
>
>
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Vladimir Olovyannikov]

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Wednesday, November 18, 2015 11:32 PM
> To: Vladimir Olovyannikov
> Cc: Mark Rutland; edk2-devel@lists.01.org
> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
> 
> On 19 November 2015 at 05:48, Vladimir Olovyannikov
> <volov...@broadcom.com> wrote:
> >
> >
[...]
> > A side note: I got the u-boot source for that board and there are several
> hacks made to avoid SError (writing 0x20 to the HCR register (reroute SError
> to EL2), and
> > just ERET from SError exception handler, and then write 0x0 to HCR before
> Linux boots), so it could be an HW issue I am not aware of as of yet.
> 
> OK, that would explain it. Note that the kernel will replace the EL2
> vector table if booted at EL2, so this is definitely not a workaround
> that you would want to reuse in UEFI.
[...]

[...]
> >>
> >> I'd still like to double check the value of VBAR_EL2 as it is written,
> >> it should be a multiple of 2 KB
> > It is always at 0x85008800 which is a multiple of 2K
> > Any other things to verify?
> >
> 
> No, that looks fine. You need to get in touch with the authors of the
> U-Boot code to figure out what it is they are working around. Simply
> ignoring SErrors is obviously not a long term solution.
> 
Thanks a lot for your help Ard, Mark,
It is the ATF which has an issue. I will post additional information if there 
is still a problem after ATF implementation is fixed

Thank you,
Vladimir
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Ard Biesheuvel]

On 19 November 2015 at 05:48, Vladimir Olovyannikov
<volov...@broadcom.com> wrote:
>
>
>> -Original Message-
>> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>> Sent: Tuesday, November 17, 2015 11:03 PM
>> To: Mark Rutland
>> Cc: Vladimir Olovyannikov; edk2-devel@lists.01.org
>> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
>>
>> On 17 November 2015 at 12:22, Mark Rutland <mark.rutl...@arm.com>
>> wrote:
>> [...]
>> >> Did that. Regardless of ArmInstructionSycnhronizationBarrier() and
>> subsequent enabling of the async abort
>> >> SError happens right in the ArmWriteVBar.
>> >
>> > Ok.
>> >
>> > Two theories:
>> >
>> > * When we take an exception, SError is masked. So perhaps we take
>> >   another exception immediately after writing to VBAR_EL2, and that
>> >   exception handler triggers the SError. I imagine that must be an
>> >   asynchronous exception (i.e. one we can mask with DAIF).
>> >
>> >   We should be able to see if that's the case if we change the
>> >   ArmWriteVbar logic for EL2 to something like:
>> >
>> >   mrs   x1, daif
>> >   msr   daifset, #(0xb << 6) // D_IF masked
> After this instruction DAIF is 0x2C0 (bit A is not masked)
>> >   mrs   vbar_el2, x0
>  After msr vbar_el2, x0, isb+nop, and msr DAIFClr,#4 no SError - bit A was 
> not masked.
>> >   isb
>> >   nop
>> >   mrs   daif, x1
>> >   nop
>> >
>> >   If that is the case, I'd expect to take the SError immediately after
>> >   the write back to daif.
> Mark, I tried this and made some other experiments with DAIF masked.
> It looks like the x0 address is not accepted by msr vbar_el2,x0.
> I tried to write vector in the very beginning, like this:
> mov x0, #0800
> movk x0, #8500 lsl 16
> msr vbar_el2, x0
> isb
> nop
> and hit SError on the first msr DAIFClr,#4
> I run UEFI from within the u-boot. Probably better would be just flash the 
> UEFI and run UEFI only, and boot that way.
> Anyway when I setup vbar_el2 with u-boot exception vector address, at least  
> no SError happens.
> If I setup u-boot vector with the UEFI vector address, it hits SError right 
> away.
> (Boot into u-boot, load UEFI, then load u-boot and start execution with 
> 0x85008800 vector address)
>
> A side note: I got the u-boot source for that board and there are several 
> hacks made to avoid SError (writing 0x20 to the HCR register (reroute SError 
> to EL2), and
> just ERET from SError exception handler, and then write 0x0 to HCR before 
> Linux boots), so it could be an HW issue I am not aware of as of yet.

OK, that would explain it. Note that the kernel will replace the EL2
vector table if booted at EL2, so this is definitely not a workaround
that you would want to reuse in UEFI.

>> >
>> >   Ard, do we ever expect to take (non-fatal) synchronous exceptions?
>> >
>>
>> Never this early, since this version of the vector table is installed
>> very early, and deadloops on any exception that it takes.
>>
>> Only after CpuDxe is initialized, we can handle dispatch exceptions
>> and interrupts normally,  but this is typically only used for the
>> timer interrupt. I could not find any invocations of
>> RegisterInterruptHandler() that installs a handler for synchronous
>> exceptions.
>>
>> > * As Ard originally suggested, the VBAR_EL2 address is close to some
>> >   memory region we shouldn't speculatively fetch from, but for some
>> >   reason do.
>> >
>> >   Can you take a look at the new VBAR_EL2 value and your memory map,
>> and
>> >   see if it's close to anything that shouldn't be fetched from?
>> >
>>
>> I'd still like to double check the value of VBAR_EL2 as it is written,
>> it should be a multiple of 2 KB
> It is always at 0x85008800 which is a multiple of 2K
> Any other things to verify?
>

No, that looks fine. You need to get in touch with the authors of the
U-Boot code to figure out what it is they are working around. Simply
ignoring SErrors is obviously not a long term solution.

-- 
Ard.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Mark Rutland]

> > Is the SError taken directly to EL2? I understood from your previous
> > reply that the EDK2 exception handler was invoked at this point, so is
> > there anything at EL3 which is trying to catch exceptions and then
> > re-inject them down to EL2?
> None that I would be aware of.

Ok.

> > The fact that we're clearly able to take the SError via the table
> > confuses me; it makes it sound like the table is fine.
> > 
> > Is that exception taken via the old vector table, or the new one we're
> > trying to install with this write to VBAR_EL2?
> The exception is taken from the NEW vector table.

Interesting!

> > > What would you advise?
> > 
> > My hunch is that we have a pending SError that gets "flushed" by an ISB.
> > 
> > You should be able to add ArmInstructionSynchronizationBarrier() calls
> > in the general vicinity of InitializeDebugAgent() (e.g. before the call
> > to ArmWriteVBar(), in the caller of InitializeDebugAgent(), before it
> > and the previous few functions are called).
> > 
> > That may cause the SError to be taken closer to its trigger, and if so
> > it would be possible to bisect down to the trigger.
> Did that. Regardless of ArmInstructionSycnhronizationBarrier() and subsequent 
> enabling of the async abort
> SError happens right in the ArmWriteVBar.

Ok.

Two theories:

* When we take an exception, SError is masked. So perhaps we take
  another exception immediately after writing to VBAR_EL2, and that
  exception handler triggers the SError. I imagine that must be an
  asynchronous exception (i.e. one we can mask with DAIF).

  We should be able to see if that's the case if we change the
  ArmWriteVbar logic for EL2 to something like:

  mrs   x1, daif
  msr   daifset, #(0xb << 6) // D_IF masked
  mrs   vbar_el2, x0
  isb
  nop
  mrs   daif, x1
  nop

  If that is the case, I'd expect to take the SError immediately after
  the write back to daif.

  Ard, do we ever expect to take (non-fatal) synchronous exceptions?

* As Ard originally suggested, the VBAR_EL2 address is close to some
  memory region we shouldn't speculatively fetch from, but for some
  reason do.

  Can you take a look at the new VBAR_EL2 value and your memory map, and
  see if it's close to anything that shouldn't be fetched from?

Thanks,
Mark.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Vladimir Olovyannikov]

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Monday, November 16, 2015 10:28 AM
> To: Vladimir Olovyannikov
> Cc: Mark Rutland; edk2-devel@lists.01.org
> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
> 
[...]
> >
> > Async abort occurs in ArmWriteVBar() called by InitializeDebugAgent(),
> DebugAgentSymbolsBaseLib.c.
> > Prior to this call I can easily enable async aborts with no "bad"
> consequences.
> >
> > Here is the exact instruction causing the SError in the ArmWriteVBar():
> > 2: msr   vbar_el2, x0// Set the Address of the EL2 Vector Table 
> > in the
> VBAR register
> 
> Are you using a release build? If so, you should check whether x0 is
> correctly aligned to 2 KB. The ASSERT() tries to establish that, but
> it is only active in DEBUG builds.
> 
Ard, it is a DEBUG build.
> 
> > Could it mean that I do not have enough privileges in the UEFI for this
> operation?
> > What would you advise?
> >>
> >> > The boot sequence is BL2->BL3.1->UEFI (plus grub.efi app)->Linux
> >>
> >> I take it per the naming that you are running ARM Trusted Firmware.
> >>
> >> Are you able to unmask SError during BL2 or BL3.1, and if so, does it fire
> >> prior to entering EDK2?
> >>
> >> [...]
> > It fires in the ArmWriteVBar as I mentioned above.
> >>
[...]

Thank you,
Vladimir
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Ard Biesheuvel]

On 16 November 2015 at 19:41, Vladimir Olovyannikov
<volov...@broadcom.com> wrote:
>> -Original Message-
>> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>> Sent: Monday, November 16, 2015 10:28 AM
>> To: Vladimir Olovyannikov
>> Cc: Mark Rutland; edk2-devel@lists.01.org
>> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
>>
> [...]
>> >
>> > Async abort occurs in ArmWriteVBar() called by InitializeDebugAgent(),
>> DebugAgentSymbolsBaseLib.c.
>> > Prior to this call I can easily enable async aborts with no "bad"
>> consequences.
>> >
>> > Here is the exact instruction causing the SError in the ArmWriteVBar():
>> > 2: msr   vbar_el2, x0// Set the Address of the EL2 Vector 
>> > Table in the
>> VBAR register
>>
>> Are you using a release build? If so, you should check whether x0 is
>> correctly aligned to 2 KB. The ASSERT() tries to establish that, but
>> it is only active in DEBUG builds.
>>
> Ard, it is a DEBUG build.

OK. So that should confirm that x0 is aligned to 2 KB.

Perhaps the write to VBAR_EL2 enables the delivery in some way. Could
you check the A bit in ESR_EL1 before and after?


>>
>> > Could it mean that I do not have enough privileges in the UEFI for this
>> operation?
>> > What would you advise?
>> >>
>> >> > The boot sequence is BL2->BL3.1->UEFI (plus grub.efi app)->Linux
>> >>
>> >> I take it per the naming that you are running ARM Trusted Firmware.
>> >>
>> >> Are you able to unmask SError during BL2 or BL3.1, and if so, does it fire
>> >> prior to entering EDK2?
>> >>
>> >> [...]
>> > It fires in the ArmWriteVBar as I mentioned above.
>> >>
> [...]
>
> Thank you,
> Vladimir
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Ard Biesheuvel]

On 16 November 2015 at 20:13, Mark Rutland <mark.rutl...@arm.com> wrote:
> On Mon, Nov 16, 2015 at 08:08:18PM +0100, Ard Biesheuvel wrote:
>> On 16 November 2015 at 19:41, Vladimir Olovyannikov
>> <volov...@broadcom.com> wrote:
>> >> -Original Message-
>> >> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>> >> Sent: Monday, November 16, 2015 10:28 AM
>> >> To: Vladimir Olovyannikov
>> >> Cc: Mark Rutland; edk2-devel@lists.01.org
>> >> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
>> >>
>> > [...]
>> >> >
>> >> > Async abort occurs in ArmWriteVBar() called by InitializeDebugAgent(),
>> >> DebugAgentSymbolsBaseLib.c.
>> >> > Prior to this call I can easily enable async aborts with no "bad"
>> >> consequences.
>> >> >
>> >> > Here is the exact instruction causing the SError in the ArmWriteVBar():
>> >> > 2: msr   vbar_el2, x0// Set the Address of the EL2 Vector 
>> >> > Table in the
>> >> VBAR register
>> >>
>> >> Are you using a release build? If so, you should check whether x0 is
>> >> correctly aligned to 2 KB. The ASSERT() tries to establish that, but
>> >> it is only active in DEBUG builds.
>> >>
>> > Ard, it is a DEBUG build.
>>
>> OK. So that should confirm that x0 is aligned to 2 KB.
>>
>> Perhaps the write to VBAR_EL2 enables the delivery in some way. Could
>> you check the A bit in ESR_EL1 before and after?
>
> Do you mean in DAIF / PSTATE? I'm not sure what ESR_ELx would have to do
> with this.
>

If I am reading the ARM ARM correctly, ISR_EL1.A will be 1 if an
SError interrupt is pending. Is that gated by DAIF / PSTATE in some
way?
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Ard Biesheuvel]

On 16 November 2015 at 19:23, Vladimir Olovyannikov
<volov...@broadcom.com> wrote:
>> -Original Message-
>> From: Mark Rutland [mailto:mark.rutl...@arm.com]
>> Sent: Friday, November 13, 2015 5:18 PM
>> To: Vladimir Olovyannikov
>> Cc: Ard Biesheuvel; edk2-devel@lists.01.org
>> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
>>
>> On Fri, Nov 13, 2015 at 10:39:34PM +, Vladimir Olovyannikov wrote:
>> >
>> >
>> > > -Original Message-
>> > > From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
>> > > Sent: Friday, November 13, 2015 3:00 AM
>> > > To: Vladimir Olovyannikov
>> > > Cc: edk2-devel@lists.01.org
>> > > Subject: Re: Armv8 64bit: System error booting linux from the UEFI
>> > >
>> > (removed extra text)
>> >
>> > > >> This is a firmware problem, not a kernel problem. The kernel is
>> > > >> entered with an pending asynchronous external abort, which may
>> have
>> > > >> several causes. You need to instrument the firmware (i.e., unmask the
>> > > >> aborts earlier and run in the debugger) to get a feeling for what is
>> > > >> going on there. It could be something like speculative accesses to
>> > > >> unassigned physical ranges that are mapped cacheable inadvertently,
>> > > >> but it could be lots of other things as well.
>> > > >
>> > > >
>> > >
>> > > FYI http://article.gmane.org/gmane.comp.bios.edk2.devel/4246
>> > >
>> > > This fixes a bug that could potentially result in speculative reads to
>> > > device regions. We have also made some other changes regarding
>> > > cacheability and shareability recently (and the patch above will
>> > > likely go in today) so it is probably worthwhile to make sure you are
>> > > based on the latest upstream.
>> >
>> > Thank you Ard, I got the latest upstream. Same issue... It is not related 
>> > to
>> the cache.
>> >
>> > I investigated and found that as soon as I do
>> > msr DAIFClr, #4 (or call ArmEnableAsynchronousAbort()) anywhere in the
>> UEFI,
>> > I immediately get this exception.
>>
>> This implies that either code very early in EDK2, or code prior to EDK2 is 
>> the
>> problem, given that when masked, SError will pend until it is unmasked
>> (whereupon it should be taken).
>>
>> What is the earliest point in EDK2 that you have unmasked SError?
>>
>> Are you doing this in PrePeiCoreEntryPoint.S, or later?
> Thanks for the pointers Mark.
>
> Async abort occurs in ArmWriteVBar() called by InitializeDebugAgent(), 
> DebugAgentSymbolsBaseLib.c.
> Prior to this call I can easily enable async aborts with no "bad" 
> consequences.
>
> Here is the exact instruction causing the SError in the ArmWriteVBar():
> 2: msr   vbar_el2, x0// Set the Address of the EL2 Vector Table 
> in the VBAR register

Are you using a release build? If so, you should check whether x0 is
correctly aligned to 2 KB. The ASSERT() tries to establish that, but
it is only active in DEBUG builds.


> Could it mean that I do not have enough privileges in the UEFI for this 
> operation?
> What would you advise?
>>
>> > The boot sequence is BL2->BL3.1->UEFI (plus grub.efi app)->Linux
>>
>> I take it per the naming that you are running ARM Trusted Firmware.
>>
>> Are you able to unmask SError during BL2 or BL3.1, and if so, does it fire
>> prior to entering EDK2?
>>
>> [...]
> It fires in the ArmWriteVBar as I mentioned above.
>>
>> > UEFI is running in EL2.
>> > Maybe I am missing some initialization in the UEFI and that is the reason
>> any
>> > msr DAIFClr,#4 causes this SError exception?
>>
>> It is possible that some initialisation is missing at an early stage in the
>> boot sequence, and that this contributes ot the SError.
>>
>> As an asynchronous exception, SError pends until it is unmasked, and should
>> trigger as soon as it is unmasked. The DAIFClr itself is vanishingly unlikely
>> to be the root cause of the SError, but rather makes it apparent.
>>
>> The earlier you are able to unmask SError, the closer you will be able to get
>> to the root cause. Are you able to capture it were it earlier in the boot
>> sequence (e.g. can you register a handler earlier, or capture the exception
>> with a hardware debugger)?
>>
>> Thanks,
>> Mark.
>
> Thank you,
> Vladimir
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Mark Rutland]

On Mon, Nov 16, 2015 at 06:23:20PM +, Vladimir Olovyannikov wrote:
> > -Original Message-
> > From: Mark Rutland [mailto:mark.rutl...@arm.com]

[...]

> > What is the earliest point in EDK2 that you have unmasked SError?
> > 
> > Are you doing this in PrePeiCoreEntryPoint.S, or later?
> Thanks for the pointers Mark.
> 
> Async abort occurs in ArmWriteVBar() called by InitializeDebugAgent(), 
> DebugAgentSymbolsBaseLib.c.

Interesting.

Is the SError taken directly to EL2? I understood from your previous
reply that the EDK2 exception handler was invoked at this point, so is
there anything at EL3 which is trying to catch exceptions and then
re-inject them down to EL2?

The fact that we're clearly able to take the SError via the table
confuses me; it makes it sound like the table is fine.

Is that exception taken via the old vector table, or the new one we're
trying to install with this write to VBAR_EL2?

> Prior to this call I can easily enable async aborts with no "bad" 
> consequences.
> 
> Here is the exact instruction causing the SError in the ArmWriteVBar():
> 2: msr   vbar_el2, x0// Set the Address of the EL2 Vector Table 
> in the VBAR register

As SError is asynchronous, it's not necessarily the case that the write
to VBAR_EL2 is the trigger.

I note that in ArmPkg/Library/ArmLib/AArch64/AArch64Support.S, there is
an ISB at the end of ArmWriteVBar(). That might happen to "flush" a
pending SError on the CPU and cause it to be taken.

> Could it mean that I do not have enough privileges in the UEFI for this 
> operation?

There are no traps or enables affecting VBAR_EL2. So long as EDK2 is
running at EL2 or higher it should be sufficiently privileged to read or
write VBAR_EL2.

> What would you advise?

My hunch is that we have a pending SError that gets "flushed" by an ISB.

You should be able to add ArmInstructionSynchronizationBarrier() calls
in the general vicinity of InitializeDebugAgent() (e.g. before the call
to ArmWriteVBar(), in the caller of InitializeDebugAgent(), before it
and the previous few functions are called).

That may cause the SError to be taken closer to its trigger, and if so
it would be possible to bisect down to the trigger.

Thanks,
Mark.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Mark Rutland]

On Mon, Nov 16, 2015 at 08:08:18PM +0100, Ard Biesheuvel wrote:
> On 16 November 2015 at 19:41, Vladimir Olovyannikov
> <volov...@broadcom.com> wrote:
> >> -Original Message-
> >> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> >> Sent: Monday, November 16, 2015 10:28 AM
> >> To: Vladimir Olovyannikov
> >> Cc: Mark Rutland; edk2-devel@lists.01.org
> >> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
> >>
> > [...]
> >> >
> >> > Async abort occurs in ArmWriteVBar() called by InitializeDebugAgent(),
> >> DebugAgentSymbolsBaseLib.c.
> >> > Prior to this call I can easily enable async aborts with no "bad"
> >> consequences.
> >> >
> >> > Here is the exact instruction causing the SError in the ArmWriteVBar():
> >> > 2: msr   vbar_el2, x0// Set the Address of the EL2 Vector 
> >> > Table in the
> >> VBAR register
> >>
> >> Are you using a release build? If so, you should check whether x0 is
> >> correctly aligned to 2 KB. The ASSERT() tries to establish that, but
> >> it is only active in DEBUG builds.
> >>
> > Ard, it is a DEBUG build.
> 
> OK. So that should confirm that x0 is aligned to 2 KB.
> 
> Perhaps the write to VBAR_EL2 enables the delivery in some way. Could
> you check the A bit in ESR_EL1 before and after?

Do you mean in DAIF / PSTATE? I'm not sure what ESR_ELx would have to do
with this.

Thanks,
Mark.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Vladimir Olovyannikov]

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Monday, November 16, 2015 11:08 AM
> To: Vladimir Olovyannikov
> Cc: Mark Rutland; edk2-devel@lists.01.org
> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
> 
> On 16 November 2015 at 19:41, Vladimir Olovyannikov
> <volov...@broadcom.com> wrote:
> >> -Original Message-
> >> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> >> Sent: Monday, November 16, 2015 10:28 AM
> >> To: Vladimir Olovyannikov
> >> Cc: Mark Rutland; edk2-devel@lists.01.org
> >> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
> >>
> > [...]
> >> >
> >> > Async abort occurs in ArmWriteVBar() called by InitializeDebugAgent(),
> >> DebugAgentSymbolsBaseLib.c.
> >> > Prior to this call I can easily enable async aborts with no "bad"
> >> consequences.
> >> >
> >> > Here is the exact instruction causing the SError in the ArmWriteVBar():
> >> > 2: msr   vbar_el2, x0// Set the Address of the EL2 Vector 
> >> > Table in
> the
> >> VBAR register
> >>
> >> Are you using a release build? If so, you should check whether x0 is
> >> correctly aligned to 2 KB. The ASSERT() tries to establish that, but
> >> it is only active in DEBUG builds.
> >>
> > Ard, it is a DEBUG build.
> 
> OK. So that should confirm that x0 is aligned to 2 KB.
> 
> Perhaps the write to VBAR_EL2 enables the delivery in some way. Could
> you check the A bit in ESR_EL1 before and after?
Ard, ESR_EL1 is all 0x0 before and after.
> 

Thank you,
Vladimir
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Vladimir Olovyannikov]

> -Original Message-
> From: Mark Rutland [mailto:mark.rutl...@arm.com]
> Sent: Monday, November 16, 2015 11:11 AM
> To: Vladimir Olovyannikov
> Cc: Ard Biesheuvel; edk2-devel@lists.01.org
> Subject: Re: [edk2] Armv8 64bit: System error booting linux from the UEFI
> 
> On Mon, Nov 16, 2015 at 06:23:20PM +, Vladimir Olovyannikov wrote:
> > > -Original Message-
> > > From: Mark Rutland [mailto:mark.rutl...@arm.com]
> 
> [...]
> 
> > > What is the earliest point in EDK2 that you have unmasked SError?
> > >
> > > Are you doing this in PrePeiCoreEntryPoint.S, or later?
> > Thanks for the pointers Mark.
> >
> > Async abort occurs in ArmWriteVBar() called by InitializeDebugAgent(),
> DebugAgentSymbolsBaseLib.c.
> 
> Interesting.
> 
> Is the SError taken directly to EL2? I understood from your previous
> reply that the EDK2 exception handler was invoked at this point, so is
> there anything at EL3 which is trying to catch exceptions and then
> re-inject them down to EL2?
None that I would be aware of.
> 
> The fact that we're clearly able to take the SError via the table
> confuses me; it makes it sound like the table is fine.
> 
> Is that exception taken via the old vector table, or the new one we're
> trying to install with this write to VBAR_EL2?
The exception is taken from the NEW vector table.
> 
> > Prior to this call I can easily enable async aborts with no "bad"
> consequences.
> >
> > Here is the exact instruction causing the SError in the ArmWriteVBar():
> > 2: msr   vbar_el2, x0// Set the Address of the EL2 Vector Table 
> > in the
> VBAR register
> 
> As SError is asynchronous, it's not necessarily the case that the write
> to VBAR_EL2 is the trigger.
> 
> I note that in ArmPkg/Library/ArmLib/AArch64/AArch64Support.S, there is
> an ISB at the end of ArmWriteVBar(). That might happen to "flush" a
> pending SError on the CPU and cause it to be taken.
> 
> > Could it mean that I do not have enough privileges in the UEFI for this
> operation?
> 
> There are no traps or enables affecting VBAR_EL2. So long as EDK2 is
> running at EL2 or higher it should be sufficiently privileged to read or
> write VBAR_EL2.

I see. UEFI is running at EL2.
> 
> > What would you advise?
> 
> My hunch is that we have a pending SError that gets "flushed" by an ISB.
> 
> You should be able to add ArmInstructionSynchronizationBarrier() calls
> in the general vicinity of InitializeDebugAgent() (e.g. before the call
> to ArmWriteVBar(), in the caller of InitializeDebugAgent(), before it
> and the previous few functions are called).
> 
> That may cause the SError to be taken closer to its trigger, and if so
> it would be possible to bisect down to the trigger.
Did that. Regardless of ArmInstructionSycnhronizationBarrier() and subsequent 
enabling of the async abort
SError happens right in the ArmWriteVBar.

Thank you,
Vladimir 
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Vladimir Olovyannikov]

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Friday, November 13, 2015 3:00 AM
> To: Vladimir Olovyannikov
> Cc: edk2-devel@lists.01.org
> Subject: Re: Armv8 64bit: System error booting linux from the UEFI
> 
(removed extra text)

> >> This is a firmware problem, not a kernel problem. The kernel is
> >> entered with an pending asynchronous external abort, which may have
> >> several causes. You need to instrument the firmware (i.e., unmask the
> >> aborts earlier and run in the debugger) to get a feeling for what is
> >> going on there. It could be something like speculative accesses to
> >> unassigned physical ranges that are mapped cacheable inadvertently,
> >> but it could be lots of other things as well.
> >
> >
> 
> FYI http://article.gmane.org/gmane.comp.bios.edk2.devel/4246
> 
> This fixes a bug that could potentially result in speculative reads to
> device regions. We have also made some other changes regarding
> cacheability and shareability recently (and the patch above will
> likely go in today) so it is probably worthwhile to make sure you are
> based on the latest upstream.

Thank you Ard, I got the latest upstream. Same issue... It is not related to 
the cache.

I investigated and found that as soon as I do
msr DAIFClr, #4 (or call ArmEnableAsynchronousAbort()) anywhere in the UEFI, 
I immediately get this exception.

Here is slightly modified efi_entry.S (immediately on ENTRY):

ENTRY(efi_stub_entry)
0x995a440   msr DAIFClr, #4 <-- crashes here right away
/*
 * Create a stack frame to save FP/LR with extra space
 * for image_addr variable passed to efi_entry().
 */
0x995a444 stp   x29, x30, [sp, #-32]!
...

So for me just performing this instruction causes SError immediately.
Here is the excerpt from the UEFI exception handler on this command:

SError Exception at 0xA995A444

  X0 0xBEDB8798   X1 0xBEFE0018   X2 0xA995A440   X3 
0x00B0
  X4 0xBEDB8598   X5 0xBF4BDE8C   X6 0x005C   X7 
0x
  X8 0xB986DF9C   X9 0xB9F8  X10 0x  X11 
0xD800
 X12 0xDC00  X13 0x200C  X14 0x0003  X15 
0x001D
 X16 0xB5B0  X17 0x0A880C885C60448A  X18 0x0A880C88  X19 
0xBF4D0780
 X20 0x00B8  X21 0xB986DF9B  X22 0xBE31A040  X23 
0xB98E727A
 X24 0xB98E7274  X25 0xB998D7A4  X26 0x  X27 
0x
 X28 0x   FP 0xB630   LR 0xBF4A36C0  
...


  
 SP 0xB5B0  ELR 0xA995A444  SPSR 0x6209  FPSR 
0x  
   
 ESR 0xBF00  FAR 0x 

  


  
 ESR : EC 0x2F  IL 0x1  ISS 0x0100  

  
ASSERT  
/uefi/ArmPkg/Library/DefaultExceptionHandlerLib/AArch64/DefaultExceptionHandler.c(186):
 ((BOOLEAN)(0==1))

The boot sequence is BL2->BL3.1->UEFI (plus grub.efi app)->Linux
UEFI is running in EL2.
Maybe I am missing some initialization in the UEFI and that is the reason any 
msr DAIFClr,#4 causes this SError exception?

Thank you,
Vladimir
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Mark Rutland]

On Fri, Nov 13, 2015 at 10:39:34PM +, Vladimir Olovyannikov wrote:
> 
> 
> > -Original Message-
> > From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> > Sent: Friday, November 13, 2015 3:00 AM
> > To: Vladimir Olovyannikov
> > Cc: edk2-devel@lists.01.org
> > Subject: Re: Armv8 64bit: System error booting linux from the UEFI
> > 
> (removed extra text)
> 
> > >> This is a firmware problem, not a kernel problem. The kernel is
> > >> entered with an pending asynchronous external abort, which may have
> > >> several causes. You need to instrument the firmware (i.e., unmask the
> > >> aborts earlier and run in the debugger) to get a feeling for what is
> > >> going on there. It could be something like speculative accesses to
> > >> unassigned physical ranges that are mapped cacheable inadvertently,
> > >> but it could be lots of other things as well.
> > >
> > >
> > 
> > FYI http://article.gmane.org/gmane.comp.bios.edk2.devel/4246
> > 
> > This fixes a bug that could potentially result in speculative reads to
> > device regions. We have also made some other changes regarding
> > cacheability and shareability recently (and the patch above will
> > likely go in today) so it is probably worthwhile to make sure you are
> > based on the latest upstream.
> 
> Thank you Ard, I got the latest upstream. Same issue... It is not related to 
> the cache.
> 
> I investigated and found that as soon as I do
> msr DAIFClr, #4 (or call ArmEnableAsynchronousAbort()) anywhere in the UEFI, 
> I immediately get this exception.

This implies that either code very early in EDK2, or code prior to EDK2 is the
problem, given that when masked, SError will pend until it is unmasked
(whereupon it should be taken).

What is the earliest point in EDK2 that you have unmasked SError?

Are you doing this in PrePeiCoreEntryPoint.S, or later?

> The boot sequence is BL2->BL3.1->UEFI (plus grub.efi app)->Linux

I take it per the naming that you are running ARM Trusted Firmware.

Are you able to unmask SError during BL2 or BL3.1, and if so, does it fire
prior to entering EDK2?

[...]

> UEFI is running in EL2.
> Maybe I am missing some initialization in the UEFI and that is the reason any
> msr DAIFClr,#4 causes this SError exception?

It is possible that some initialisation is missing at an early stage in the
boot sequence, and that this contributes ot the SError.

As an asynchronous exception, SError pends until it is unmasked, and should
trigger as soon as it is unmasked. The DAIFClr itself is vanishingly unlikely
to be the root cause of the SError, but rather makes it apparent.

The earlier you are able to unmask SError, the closer you will be able to get
to the root cause. Are you able to capture it were it earlier in the boot
sequence (e.g. can you register a handler earlier, or capture the exception
with a hardware debugger)?

Thanks,
Mark.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Ard Biesheuvel]

On 11 November 2015 at 01:20, Vladimir Olovyannikov
 wrote:
> Hello,
>
> I am not sure this is the right forum to ask this question, so I am sorry in
> advance if this is an offtopic here (please point me to the proper one).
>
> I brought up UEFI on the device and am trying to load linux from the SD card
> (loading from the network using GRUB is a topic for another message).
>
> Linux kernel starts and shortly after that (as soon as local_async is
> enabled which is simple “MSR DAIfclr,4”) there is a kernel crash with
>
> “Bad mode in Error handler detected, code 0xbf00 -- SError”
> Here is the log:
>
>
>
> The default boot selection will start in  10 seconds
>

Unrelated note: *please* get rid of the ARM BDS and use the Intel BDS
instead. The ARM BDS violates the UEFI spec, (i.e., you cannot boot
installer images from it without a lot of hassle) and does not allow
you to enable UEFI secure boot.

Look at the development history of ArmVExpress-FVP-AArch64.dsc for
hints how to do this.

> [1] Linux Kernel from SD Card
>
> - SD(0x0)/HD(1,MBR,0x,0x89,0x3A9F77)/Image
>
> - Arguments: dtb=ns2-svk.dtb initrd=initrd console=ttyS0,115200n8
> earlycon=uart8250,mmio32,0x6613,maxcpus=1
>
> [2] GRUB
>
> - MAC(001019D0B2A4,0x1)/IPv4(10.136.12.136)/grub.efi
>
> - Arguments:
>
> [3] Shell
>
> [4] Boot Manager
>
> Start: 1
>
> InstallProtocolInterface: CE345171-BA0B-11D2-8E4F-00A0C969723B BEDB9DA0
>
> BlockSize : 512
>
> LastBlock : 3A9FFF
>
> InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B BEDB9798
>
> InstallProtocolInterface: 964E5B21-6459-11D2-8E39-00A0C969723B BEDB9670
>
> InstallProtocolInterface: CE345171-BA0B-11D2-8E4F-00A0C969723B BEDB95A0
>
> InstallProtocolInterface: 964E5B22-6459-11D2-8E39-00A0C969723B BEDAF030
>
> InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B BE2E6040
>
> Loading driver at 0x000B8C0C000 EntryPoint=0x000B9297440
>
> Loading driver at 0x000B8C0C000 EntryPoint=0x000B9297440
>
> InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF BEDB0718
>
> EFI stub: Booting Linux Kernel...
>
> ConvertPages: Incompatible memory types
>
> EFI stub: Using DTB from command line
>
> EFI stub: Exiting boot services and installing virtual address map...
>
> MmcHost: ExitBootServicesEvent+
>
> Marked card brcm-SDIO1 as removed
>
> Resetting Card brcm-SDIO1
>
> brcm-SDIO1 Card reset done.
>
> MmcHost: ExitBootServicesEvent-
>
> MmcHost: ExitBootServicesEvent+
>
> Marked card brcm-SDIO0 as removed
>
> Resetting Card brcm-SDIO0
>
> brcm-SDIO0 Card reset done.
>
> MmcHost: ExitBootServicesEvent-
>
> [0.00] Booting Linux on physical CPU 0x0
>
> [0.00] Initializing cgroup subsys cpu
>
> [0.00] Linux version 4.2.0+ (volovyan@LBRMN-LNXUB114) (gcc version
> 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro
> GCC 4.9-2014.09) ) #1 SMP PREEMPT Tue Oct 13 11:33:09 PDT 2015
>
> [0.00] CPU: AArch64 Processor [411fd073] revision 3
>
> [0.00] Detected PIPT I-cache on CPU0
>
> [0.00] earlycon: Early serial console at MMIO32 0x6613 (options
> 'maxcpus=1')
>
> [0.00] bootconsole [uart0] enabled
>
> [0.00] Bad mode in Error handler detected, code 0xbf00 -- SError
>
> [0.00] CPU: 0 PID: 0 Comm: swapper Not tainted 4.2.0+ #1
>
> [0.00] Hardware name: SVK proto (DT)
>
> [0.00] task: ffc0007584b0 ti: ffc00074c000 task.ti:
> ffc00074c000
>
> [0.00] PC is at setup_arch+0x260/0x5b4
>
> [0.00] LR is at setup_arch+0x25c/0x5b4
>
> [0.00] pc : [] lr : [] pstate:
> 02c5
>
> [0.00] sp : ffc00074ff20
>
> [0.00] x29: ffc00074ff20 x28: 
>
> [0.00] x27: ffc81198 x26: 807cd000
>
> [0.00] x25: 807ca000 x24: 8000
>
> [0.00] x23:  x22: ffc000752000
>
> [0.00] x21: ffc000752610 x20: ffbffa80
>
> [0.00] x19: ffc8 x18: 
>
> [0.00] x17: 05e3 x16: 1000
>
> [0.00] x15: 1c00 x14: 0ffe
>
> [0.00] x13: 0020 x12: 0028
>
> [0.00] x11: 0007 x10: 0101010101010101
>
> [0.00] x9 : fffb x8 : 0008
>
> [0.00] x7 : 0003 x6 : 0080
>
> [0.00] x5 : 005f x4 : 
>
> [0.00] x3 :  x2 : 0065
>
> [0.00] x1 :  x0 : 0001
>
> [0.00]
>
> [0.00] Internal error: Oops - bad mode: 0 [#1] PREEMPT SMP
>
> [0.00] Modules linked in:
>
> [0.00] CPU: 0 PID: 0 Comm: swapper Not tainted 4.2.0+ #1
>
> [0.00] Hardware name: SVK proto (DT)
>
> [0.00] task: ffc0007584b0 ti: 

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Vladimir Olovyannikov]

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Wednesday, November 11, 2015 1:08 AM
> To: Vladimir Olovyannikov
> Cc: edk2-devel@lists.01.org
> Subject: Re: Armv8 64bit: System error booting linux from the UEFI
> 
> On 11 November 2015 at 01:20, Vladimir Olovyannikov
>  wrote:
> > Hello,
> >
> > I am not sure this is the right forum to ask this question, so I am sorry in
> > advance if this is an offtopic here (please point me to the proper one).
> >
> > I brought up UEFI on the device and am trying to load linux from the SD
> card
> > (loading from the network using GRUB is a topic for another message).
> >
> > Linux kernel starts and shortly after that (as soon as local_async is
> > enabled which is simple “MSR DAIfclr,4”) there is a kernel crash with
> >
> > “Bad mode in Error handler detected, code 0xbf00 -- SError”
> > Here is the log:
> >
> >
> >
> > The default boot selection will start in  10 seconds
> >
> 
> Unrelated note: *please* get rid of the ARM BDS and use the Intel BDS
> instead. The ARM BDS violates the UEFI spec, (i.e., you cannot boot
> installer images from it without a lot of hassle) and does not allow
> you to enable UEFI secure boot.
> 
> Look at the development history of ArmVExpress-FVP-AArch64.dsc for
> hints how to do this.
Thank you for the hint. I will follow your advice to get rid of the ARM BDS.
> 
> > [1] Linux Kernel from SD Card
> >
> > - SD(0x0)/HD(1,MBR,0x,0x89,0x3A9F77)/Image
> >
> > - Arguments: dtb=ns2-svk.dtb initrd=initrd console=ttyS0,115200n8
> > earlycon=uart8250,mmio32,0x6613,maxcpus=1
> >
> > [2] GRUB
> >
> > - MAC(001019D0B2A4,0x1)/IPv4(10.136.12.136)/grub.efi
> >
> > - Arguments:
> >
> > [3] Shell
> >
> > [4] Boot Manager
> >
> > Start: 1
> >
> > InstallProtocolInterface: CE345171-BA0B-11D2-8E4F-00A0C969723B
> BEDB9DA0
> >
> > BlockSize : 512
> >
> > LastBlock : 3A9FFF
> >
> > InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B
> BEDB9798
> >
> > InstallProtocolInterface: 964E5B21-6459-11D2-8E39-00A0C969723B
> BEDB9670
> >
> > InstallProtocolInterface: CE345171-BA0B-11D2-8E4F-00A0C969723B
> BEDB95A0
> >
> > InstallProtocolInterface: 964E5B22-6459-11D2-8E39-00A0C969723B
> BEDAF030
> >
> > InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B
> BE2E6040
> >
> > Loading driver at 0x000B8C0C000 EntryPoint=0x000B9297440
> >
> > Loading driver at 0x000B8C0C000 EntryPoint=0x000B9297440
> >
> > InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF
> BEDB0718
> >
> > EFI stub: Booting Linux Kernel...
> >
> > ConvertPages: Incompatible memory types
> >
> > EFI stub: Using DTB from command line
> >
> > EFI stub: Exiting boot services and installing virtual address map...
> >
> > MmcHost: ExitBootServicesEvent+
> >
> > Marked card brcm-SDIO1 as removed
> >
> > Resetting Card brcm-SDIO1
> >
> > brcm-SDIO1 Card reset done.
> >
> > MmcHost: ExitBootServicesEvent-
> >
> > MmcHost: ExitBootServicesEvent+
> >
> > Marked card brcm-SDIO0 as removed
> >
> > Resetting Card brcm-SDIO0
> >
> > brcm-SDIO0 Card reset done.
> >
> > MmcHost: ExitBootServicesEvent-
> >
> > [0.00] Booting Linux on physical CPU 0x0
> >
> > [0.00] Initializing cgroup subsys cpu
> >
> > [0.00] Linux version 4.2.0+ (volovyan@LBRMN-LNXUB114) (gcc
> version
> > 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro
> > GCC 4.9-2014.09) ) #1 SMP PREEMPT Tue Oct 13 11:33:09 PDT 2015
> >
> > [0.00] CPU: AArch64 Processor [411fd073] revision 3
> >
> > [0.00] Detected PIPT I-cache on CPU0
> >
> > [0.00] earlycon: Early serial console at MMIO32 0x6613 (options
> > 'maxcpus=1')
> >
> > [0.00] bootconsole [uart0] enabled
> >
> > [0.00] Bad mode in Error handler detected, code 0xbf00 -- SError
> >
> > [0.00] CPU: 0 PID: 0 Comm: swapper Not tainted 4.2.0+ #1
> >
> > [0.00] Hardware name: SVK proto (DT)
> >
> > [0.00] task: ffc0007584b0 ti: ffc00074c000 task.ti:
> > ffc00074c000
> >
> > [0.00] PC is at setup_arch+0x260/0x5b4
> >
> > [0.00] LR is at setup_arch+0x25c/0x5b4
> >
> > [0.00] pc : [] lr : [] pstate:
> > 02c5
> >
> > [0.00] sp : ffc00074ff20
> >
> > [0.00] x29: ffc00074ff20 x28: 
> >
> > [0.00] x27: ffc81198 x26: 807cd000
> >
> > [0.00] x25: 807ca000 x24: 8000
> >
> > [0.00] x23:  x22: ffc000752000
> >
> > [0.00] x21: ffc000752610 x20: ffbffa80
> >
> > [0.00] x19: ffc8 x18: 
> >
> > [0.00] x17: 05e3 x16: 1000
> >
> > [0.00] x15: 1c00 x14: 0ffe
> >
> > [0.00] x13: 0020 x12: 0028
> >
> > [

[edk2] Armv8 64bit: System error booting linux from the UEFI

[Vladimir Olovyannikov]

Hello,

I am not sure this is the right forum to ask this question, so I am sorry in 
advance if this is an offtopic here (please point me to the proper one).

I brought up UEFI on the device and am trying to load linux from the SD card 
(loading from the network using GRUB is a topic for another message).

Linux kernel starts and shortly after that (as soon as local_async is enabled 
which is simple "MSR DAIfclr,4") there is a kernel crash with
"Bad mode in Error handler detected, code 0xbf00 -- SError"

Here is the log:

The default boot selection will start in  10 seconds
[1] Linux Kernel from SD Card
- SD(0x0)/HD(1,MBR,0x,0x89,0x3A9F77)/Image
- Arguments: dtb=ns2-svk.dtb initrd=initrd console=ttyS0,115200n8 
earlycon=uart8250,mmio32,0x6613,maxcpus=1
[2] GRUB
- MAC(001019D0B2A4,0x1)/IPv4(10.136.12.136)/grub.efi
- Arguments:
[3] Shell
[4] Boot Manager
Start: 1
InstallProtocolInterface: CE345171-BA0B-11D2-8E4F-00A0C969723B BEDB9DA0
BlockSize : 512
LastBlock : 3A9FFF
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B BEDB9798
InstallProtocolInterface: 964E5B21-6459-11D2-8E39-00A0C969723B BEDB9670
InstallProtocolInterface: CE345171-BA0B-11D2-8E4F-00A0C969723B BEDB95A0
InstallProtocolInterface: 964E5B22-6459-11D2-8E39-00A0C969723B BEDAF030
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B BE2E6040
Loading driver at 0x000B8C0C000 EntryPoint=0x000B9297440
Loading driver at 0x000B8C0C000 EntryPoint=0x000B9297440
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF BEDB0718
EFI stub: Booting Linux Kernel...
ConvertPages: Incompatible memory types
EFI stub: Using DTB from command line
EFI stub: Exiting boot services and installing virtual address map...
MmcHost: ExitBootServicesEvent+
Marked card brcm-SDIO1 as removed
Resetting Card brcm-SDIO1
brcm-SDIO1 Card reset done.
MmcHost: ExitBootServicesEvent-
MmcHost: ExitBootServicesEvent+
Marked card brcm-SDIO0 as removed
Resetting Card brcm-SDIO0
brcm-SDIO0 Card reset done.
MmcHost: ExitBootServicesEvent-
[0.00] Booting Linux on physical CPU 0x0
[0.00] Initializing cgroup subsys cpu
[0.00] Linux version 4.2.0+ (volovyan@LBRMN-LNXUB114) (gcc version 
4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro 
GCC 4.9-2014.09) ) #1 SMP PREEMPT Tue Oct 13 11:33:09 PDT 2015
[0.00] CPU: AArch64 Processor [411fd073] revision 3
[0.00] Detected PIPT I-cache on CPU0
[0.00] earlycon: Early serial console at MMIO32 0x6613 (options 
'maxcpus=1')
[0.00] bootconsole [uart0] enabled
[0.00] Bad mode in Error handler detected, code 0xbf00 -- SError
[0.00] CPU: 0 PID: 0 Comm: swapper Not tainted 4.2.0+ #1
[0.00] Hardware name: SVK proto (DT)
[0.00] task: ffc0007584b0 ti: ffc00074c000 task.ti: 
ffc00074c000
[0.00] PC is at setup_arch+0x260/0x5b4
[0.00] LR is at setup_arch+0x25c/0x5b4
[0.00] pc : [] lr : [] pstate: 
02c5
[0.00] sp : ffc00074ff20
[0.00] x29: ffc00074ff20 x28: 
[0.00] x27: ffc81198 x26: 807cd000
[0.00] x25: 807ca000 x24: 8000
[0.00] x23:  x22: ffc000752000
[0.00] x21: ffc000752610 x20: ffbffa80
[0.00] x19: ffc8 x18: 
[0.00] x17: 05e3 x16: 1000
[0.00] x15: 1c00 x14: 0ffe
[0.00] x13: 0020 x12: 0028
[0.00] x11: 0007 x10: 0101010101010101
[0.00] x9 : fffb x8 : 0008
[0.00] x7 : 0003 x6 : 0080
[0.00] x5 : 005f x4 : 
[0.00] x3 :  x2 : 0065
[0.00] x1 :  x0 : 0001
[0.00]
[0.00] Internal error: Oops - bad mode: 0 [#1] PREEMPT SMP
[0.00] Modules linked in:
[0.00] CPU: 0 PID: 0 Comm: swapper Not tainted 4.2.0+ #1
[0.00] Hardware name: SVK proto (DT)
[0.00] task: ffc0007584b0 ti: ffc00074c000 task.ti: 
ffc00074c000
[0.00] PC is at setup_arch+0x260/0x5b4
[0.00] LR is at setup_arch+0x25c/0x5b4
[0.00] pc : [] lr : [] pstate: 
02c5
[0.00] sp : ffc00074ff20
[0.00] x29: ffc00074ff20 x28: 
[0.00] x27: ffc81198 x26: 807cd000
[0.00] x25: 807ca000 x24: 8000
[0.00] x23:  x22: ffc000752000
[0.00] x21: ffc000752610 x20: ffbffa80
[0.00] x19: ffc8 x18: 
[0.00] x17: 05e3 x16: 1000
[0.00] x15: 1c00 x14: 0ffe
[0.00] x13: 0020 

Re: [edk2] Armv8 64bit: System error booting linux from the UEFI

[Haojian Zhuang]

On 11 November 2015 at 08:20, Vladimir Olovyannikov
 wrote:
> Hello,
>
> I am not sure this is the right forum to ask this question, so I am sorry in 
> advance if this is an offtopic here (please point me to the proper one).
>
> I brought up UEFI on the device and am trying to load linux from the SD card 
> (loading from the network using GRUB is a topic for another message).
>
> Linux kernel starts and shortly after that (as soon as local_async is enabled 
> which is simple "MSR DAIfclr,4") there is a kernel crash with
> "Bad mode in Error handler detected, code 0xbf00 -- SError"
>
> Here is the log:
>
> The default boot selection will start in  10 seconds
> [1] Linux Kernel from SD Card
> - SD(0x0)/HD(1,MBR,0x,0x89,0x3A9F77)/Image
> - Arguments: dtb=ns2-svk.dtb initrd=initrd console=ttyS0,115200n8 
> earlycon=uart8250,mmio32,0x6613,maxcpus=1
> [2] GRUB
> - MAC(001019D0B2A4,0x1)/IPv4(10.136.12.136)/grub.efi
> - Arguments:
> [3] Shell
> [4] Boot Manager
> Start: 1
> InstallProtocolInterface: CE345171-BA0B-11D2-8E4F-00A0C969723B BEDB9DA0
> BlockSize : 512
> LastBlock : 3A9FFF
> InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B BEDB9798
> InstallProtocolInterface: 964E5B21-6459-11D2-8E39-00A0C969723B BEDB9670
> InstallProtocolInterface: CE345171-BA0B-11D2-8E4F-00A0C969723B BEDB95A0
> InstallProtocolInterface: 964E5B22-6459-11D2-8E39-00A0C969723B BEDAF030
> InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B BE2E6040
> Loading driver at 0x000B8C0C000 EntryPoint=0x000B9297440
> Loading driver at 0x000B8C0C000 EntryPoint=0x000B9297440
> InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF BEDB0718
> EFI stub: Booting Linux Kernel...
> ConvertPages: Incompatible memory types
> EFI stub: Using DTB from command line
> EFI stub: Exiting boot services and installing virtual address map...
> MmcHost: ExitBootServicesEvent+
> Marked card brcm-SDIO1 as removed
> Resetting Card brcm-SDIO1
> brcm-SDIO1 Card reset done.
> MmcHost: ExitBootServicesEvent-
> MmcHost: ExitBootServicesEvent+
> Marked card brcm-SDIO0 as removed
> Resetting Card brcm-SDIO0
> brcm-SDIO0 Card reset done.
> MmcHost: ExitBootServicesEvent-
> [0.00] Booting Linux on physical CPU 0x0
> [0.00] Initializing cgroup subsys cpu
> [0.00] Linux version 4.2.0+ (volovyan@LBRMN-LNXUB114) (gcc version 
> 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro 
> GCC 4.9-2014.09) ) #1 SMP PREEMPT Tue Oct 13 11:33:09 PDT 2015
> [0.00] CPU: AArch64 Processor [411fd073] revision 3
> [0.00] Detected PIPT I-cache on CPU0
> [0.00] earlycon: Early serial console at MMIO32 0x6613 (options 
> 'maxcpus=1')
> [0.00] bootconsole [uart0] enabled
> [0.00] Bad mode in Error handler detected, code 0xbf00 -- SError
> [0.00] CPU: 0 PID: 0 Comm: swapper Not tainted 4.2.0+ #1
> [0.00] Hardware name: SVK proto (DT)
> [0.00] task: ffc0007584b0 ti: ffc00074c000 task.ti: 
> ffc00074c000
> [0.00] PC is at setup_arch+0x260/0x5b4
> [0.00] LR is at setup_arch+0x25c/0x5b4
> [0.00] pc : [] lr : [] pstate: 
> 02c5
> [0.00] sp : ffc00074ff20
> [0.00] x29: ffc00074ff20 x28: 
> [0.00] x27: ffc81198 x26: 807cd000
> [0.00] x25: 807ca000 x24: 8000
> [0.00] x23:  x22: ffc000752000
> [0.00] x21: ffc000752610 x20: ffbffa80
> [0.00] x19: ffc8 x18: 
> [0.00] x17: 05e3 x16: 1000
> [0.00] x15: 1c00 x14: 0ffe
> [0.00] x13: 0020 x12: 0028
> [0.00] x11: 0007 x10: 0101010101010101
> [0.00] x9 : fffb x8 : 0008
> [0.00] x7 : 0003 x6 : 0080
> [0.00] x5 : 005f x4 : 
> [0.00] x3 :  x2 : 0065
> [0.00] x1 :  x0 : 0001
> [0.00]
> [0.00] Internal error: Oops - bad mode: 0 [#1] PREEMPT SMP
> [0.00] Modules linked in:
> [0.00] CPU: 0 PID: 0 Comm: swapper Not tainted 4.2.0+ #1
> [0.00] Hardware name: SVK proto (DT)
> [0.00] task: ffc0007584b0 ti: ffc00074c000 task.ti: 
> ffc00074c000
> [0.00] PC is at setup_arch+0x260/0x5b4
> [0.00] LR is at setup_arch+0x25c/0x5b4
> [0.00] pc : [] lr : [] pstate: 
> 02c5
> [0.00] sp : ffc00074ff20
> [0.00] x29: ffc00074ff20 x28: 
> [0.00] x27: ffc81198 x26: 807cd000
> [0.00] x25: 807ca000 x24: 8000
> [0.00] x23:  x22: