Re: [REGRESSION] Re: [PATCH v4 5/5] efi: Use generic EFI loader for x86_64 and i386
Hi Ard, everyone, On 03. 07. 24 19:35, Jan Čermák wrote: Sure, below is output from the board I have available. I'll ask the community members who reported the issues to get me output from their systems. in the attachments I'm sending all complete dmidecode outputs I was able to gather so far from systems affected by the bug, namely Intel Atom D2500, D2700, N2800 and D525 (the last one was already a part of my previous message). If there's anything more needed, please let me know. Also, if you want to test that the patch resolves the issues reported before submitting it for review, I'm available for testing, just let me know here or privately. Thanks, Jan# dmidecode 3.5 Getting SMBIOS data from sysfs. SMBIOS 2.5 present. 27 structures occupying 1318 bytes. Table at 0x7EE97000. Handle 0x, DMI type 4, 35 bytes Processor Information Socket Designation: CPU 1 Type: Central Processor Family: Atom Manufacturer: Intel(R) Corporation ID: CA 06 01 00 FF FB EB BF Signature: Type 0, Family 6, Model 28, Stepping 10 Flags: FPU (Floating-point unit on-chip) VME (Virtual mode extension) DE (Debugging extension) PSE (Page size extension) TSC (Time stamp counter) MSR (Model specific registers) PAE (Physical address extension) MCE (Machine check exception) CX8 (CMPXCHG8 instruction supported) APIC (On-chip APIC hardware supported) SEP (Fast system call) MTRR (Memory type range registers) PGE (Page global enable) MCA (Machine check architecture) CMOV (Conditional move instruction supported) PAT (Page attribute table) PSE-36 (36-bit page size extension) CLFSH (CLFLUSH instruction supported) DS (Debug store) ACPI (ACPI supported) MMX (MMX technology supported) FXSR (FXSAVE and FXSTOR instructions supported) SSE (Streaming SIMD extensions) SSE2 (Streaming SIMD extensions 2) SS (Self-snoop) HTT (Multi-threading) TM (Thermal monitor supported) PBE (Pending break enabled) Version: Intel(R) Atom(TM) CPU D525 @ 1.80GHz Voltage: 1.1 V External Clock: 200 MHz Max Speed: 4000 MHz Current Speed: 1801 MHz Status: Populated, Enabled Upgrade: Socket LGA775 L1 Cache Handle: 0x0003 L2 Cache Handle: 0x0001 L3 Cache Handle: Not Provided Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x0001, DMI type 7, 19 bytes Cache Information Socket Designation: Unknown Configuration: Enabled, Not Socketed, Level 2 Operational Mode: Write Back Location: Internal Installed Size: 512 kB Maximum Size: 512 kB Supported SRAM Types: Asynchronous Installed SRAM Type: Asynchronous Speed: Unknown Error Correction Type: Single-bit ECC System Type: Data Associativity: 8-way Set-associative Handle 0x0002, DMI type 7, 19 bytes Cache Information Socket Designation: Unknown Configuration: Enabled, Not Socketed, Level 1 Operational Mode: Write Back Location: Internal Installed Size: 32 kB Maximum Size: 32 kB Supported SRAM Types: Asynchronous Installed SRAM Type: Asynchronous Speed: Unknown Error Correction Type: Single-bit ECC System Type: Instruction Associativity: 8-way Set-associative Handle 0x0003, DMI type 7, 19 bytes Cache Information Socket Designation: Unknown Configuration: Enabled, Not Socketed, Level 1 Operational Mode: Write Back Location: Internal Installed Size: 24 kB Maximum Size: 24 kB Supported SRAM Types: Asynchronous Installed SRAM Type: Asynchronous Speed: Unknown Error Correction Type: Single-bit ECC System Type: Data Associativity: 32-way Set-associative Handle 0x0004, DMI type 0, 24 bytes BIOS Information Vendor: Intel Corp. Version: MWPNT10N.86A.0132.2013.0726.1534 Release Date: 07/26/2013 Address: 0xF Runtime Size: 64 kB ROM Size: 1 MB Characteristics: PCI is supported BIOS is upgradeable BIOS shadowing is allowed Boot from CD is supported Selectable boot is supported EDD is supported 8042 keyboard services are supported (int 9h) Serial services are supported (int 14h)
Re: [REGRESSION] Re: [PATCH v4 5/5] efi: Use generic EFI loader for x86_64 and i386
On 03. 07. 24 16:47, Ard Biesheuvel wrote: If GRUB no longer works correctly on these systems due to the move to the generic EFI loader, and we can straight-forwardly identify them using SMBIOS metadata, I don't anticipate any objections from the GRUB maintainers. Cool, that sounds great! If you can get me the output of 'dmidecode' from preferably more than one variant of such a system, I can hack up a GRUB patch that implements the SMBIOS matching and the opt-out. Sure, below is output from the board I have available. I'll ask the community members who reported the issues to get me output from their systems. Regards, Jan -- # dmidecode 3.5 Getting SMBIOS data from sysfs. SMBIOS 2.5 present. 27 structures occupying 1318 bytes. Table at 0x7EE97000. Handle 0x, DMI type 4, 35 bytes Processor Information Socket Designation: CPU 1 Type: Central Processor Family: Atom Manufacturer: Intel(R) Corporation ID: CA 06 01 00 FF FB EB BF Signature: Type 0, Family 6, Model 28, Stepping 10 Flags: FPU (Floating-point unit on-chip) VME (Virtual mode extension) DE (Debugging extension) PSE (Page size extension) TSC (Time stamp counter) MSR (Model specific registers) PAE (Physical address extension) MCE (Machine check exception) CX8 (CMPXCHG8 instruction supported) APIC (On-chip APIC hardware supported) SEP (Fast system call) MTRR (Memory type range registers) PGE (Page global enable) MCA (Machine check architecture) CMOV (Conditional move instruction supported) PAT (Page attribute table) PSE-36 (36-bit page size extension) CLFSH (CLFLUSH instruction supported) DS (Debug store) ACPI (ACPI supported) MMX (MMX technology supported) FXSR (FXSAVE and FXSTOR instructions supported) SSE (Streaming SIMD extensions) SSE2 (Streaming SIMD extensions 2) SS (Self-snoop) HTT (Multi-threading) TM (Thermal monitor supported) PBE (Pending break enabled) Version: Intel(R) Atom(TM) CPU D525 @ 1.80GHz Voltage: 1.1 V External Clock: 200 MHz Max Speed: 4000 MHz Current Speed: 1802 MHz Status: Populated, Enabled Upgrade: Socket LGA775 L1 Cache Handle: 0x0003 L2 Cache Handle: 0x0001 L3 Cache Handle: Not Provided Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x0001, DMI type 7, 19 bytes Cache Information Socket Designation: Unknown Configuration: Enabled, Not Socketed, Level 2 Operational Mode: Write Back Location: Internal Installed Size: 512 kB Maximum Size: 512 kB Supported SRAM Types: Asynchronous Installed SRAM Type: Asynchronous Speed: Unknown Error Correction Type: Single-bit ECC System Type: Data Associativity: 8-way Set-associative Handle 0x0002, DMI type 7, 19 bytes Cache Information Socket Designation: Unknown Configuration: Enabled, Not Socketed, Level 1 Operational Mode: Write Back Location: Internal Installed Size: 32 kB Maximum Size: 32 kB Supported SRAM Types: Asynchronous Installed SRAM Type: Asynchronous Speed: Unknown Error Correction Type: Single-bit ECC System Type: Instruction Associativity: 8-way Set-associative Handle 0x0003, DMI type 7, 19 bytes Cache Information Socket Designation: Unknown Configuration: Enabled, Not Socketed, Level 1 Operational Mode: Write Back Location: Internal Installed Size: 24 kB Maximum Size: 24 kB Supported SRAM Types: Asynchronous Installed SRAM Type: Asynchronous Speed: Unknown Error Correction Type: Single-bit ECC System Type: Data Associativity: 32-way Set-associative Handle 0x0004, DMI type 0, 24 bytes BIOS Information Vendor: Intel Corp. Version: MWPNT10N.86A.0132.2013.0726.1534 Release Date: 07/26/2013 Address: 0xF Runtime Size: 64 kB ROM Size: 1 MB Characteristics: PCI is supported BIOS is upgradeable BIOS shadowing is allowed Boot from CD is supported Selectable boot is supported EDD is supported 8042 keyboard services are supported (int 9h) Serial services are supported (int 14h) Printer
Re: [REGRESSION] Re: [PATCH v4 5/5] efi: Use generic EFI loader for x86_64 and i386
On Wed, 3 Jul 2024 at 16:25, Jan Čermák wrote: > > Hi Ard > > On 28. 06. 24 19:28, Ard Biesheuvel wrote: > > Given that you carry your own GRUB build, I would recommend reverting > > to non-EFI stub boot for affected Atom systems, identified by DMI > > data, which should be easy to access on such systems. > > > > Look for 'goto fallback' in the existing loader logic in > > grub-core/loader/efi/linux.c - you'd need to jump to the same point to > > use the GRUB 2.06 logic for systems that fail that cannot load the EFI > > stub kernel image. > > Thanks for the suggestion, that sounds like a more acceptable solution > (than the revert). However, would a quirk like that be acceptable in > upstream code as well? In long term, we'd like to only support hardware > that is supported in upstream, instead of taking the maintenance burden > of carrying custom patches indefinitely, so if it is GRUB's maintainers' > decision to abandon support for old buggy hardware, we shall declare it > abandoned as well. > If GRUB no longer works correctly on these systems due to the move to the generic EFI loader, and we can straight-forwardly identify them using SMBIOS metadata, I don't anticipate any objections from the GRUB maintainers. @Daniel? > > Out of curiosity - these are all x86_64 Atoms right? How old are they? > > Yes, they are, e.g. Atom D525 [1], but from the reports we've received > it seems the problem is with the NM10 chipset [2] in general (as I > stated in the first message in the thread). They are ~14 years old so > pretty archaic I'd say O:-) > If you can get me the output of 'dmidecode' from preferably more than one variant of such a system, I can hack up a GRUB patch that implements the SMBIOS matching and the opt-out. ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [REGRESSION] Re: [PATCH v4 5/5] efi: Use generic EFI loader for x86_64 and i386
Hi Ard On 28. 06. 24 19:28, Ard Biesheuvel wrote: Given that you carry your own GRUB build, I would recommend reverting to non-EFI stub boot for affected Atom systems, identified by DMI data, which should be easy to access on such systems. Look for 'goto fallback' in the existing loader logic in grub-core/loader/efi/linux.c - you'd need to jump to the same point to use the GRUB 2.06 logic for systems that fail that cannot load the EFI stub kernel image. Thanks for the suggestion, that sounds like a more acceptable solution (than the revert). However, would a quirk like that be acceptable in upstream code as well? In long term, we'd like to only support hardware that is supported in upstream, instead of taking the maintenance burden of carrying custom patches indefinitely, so if it is GRUB's maintainers' decision to abandon support for old buggy hardware, we shall declare it abandoned as well. Out of curiosity - these are all x86_64 Atoms right? How old are they? Yes, they are, e.g. Atom D525 [1], but from the reports we've received it seems the problem is with the NM10 chipset [2] in general (as I stated in the first message in the thread). They are ~14 years old so pretty archaic I'd say O:-) Regards, Jan [1] https://ark.intel.com/content/www/us/en/ark/products/49490/intel-atom-processor-d525-1m-cache-1-80-ghz.html [2] https://www.intel.com/content/www/us/en/products/sku/47610/intel-nm10-express-chipset/specifications.html ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [REGRESSION] Re: [PATCH v4 5/5] efi: Use generic EFI loader for x86_64 and i386
On Thu, 27 Jun 2024 at 12:27, Jan Čermák wrote: > > Hi Ard, > > sorry, I feel a little ashamed for replying after such a long time but I > wanted to do some due diligence first and didn't have time (or the Atom > board around) until now. > > > Does your Kconfig have EFI_DISABLE_PCI_DMA enabled by any chance? That > > could definitely produce the issues you are observing. > > No, this option is not set in our kernel. > > > In any case, given that you never relied on the EFI stub in the past > > on x86_64 (as GRUB 2.06 does not rely on it), you could just disable > > that in your Kconfig. > > > > That of course does not solve the Fujitsu issue, which apparently > > requires boot via the EFI stub, but I don't have a solution for that - > > I suppose that simply never worked with the old 'working' version of > > the OS? > > The goal is to have a single configuration that works for both (or > ideally - all amd64) targets so disabling EFI stub is not an option. > With no changes in the kernel in the meantime, just GRUB 2.06 loaded > kernel without issues on both devices. > > Anyway, what held me up a bit is that I wanted to try any of the major > distributions that already adopter GRUB 2.12. I tested Ubuntu 24.04 and > Arch's build of GRUB. While Arch's GRUB behaves exactly the same way > (error loading image) I was a bit puzzled that it's not the case of > Ubuntu. It took me a while to figure out why because there are quite > many patches applied but in the end it boiled up to the following two > from the grub package repo [1]: > > - loader-framework.patch > - efi-use-peimage-shim.patch > > Vanilla GRUB with these two patches AND with the introduced `peimage` > module added to the GRUB binary boots our kernel correctly. > > At this point I don't have the knowledge to figure out if it's just side > effect of that changes or if they indeed make any difference. And I'm > not sure what effect it has on the Fujitsu board but I can't find any > mentions of it in Ubuntu's issue tracker or anywhere on the internet so > I *guess* it's fine. > > I don't know what else to try at this point. If you'd like to look into > the issue yourself, I can give you the board I have, in the end it has > no other use for me than making sure that our OS boots correctly there, > and if the issue gets resolved, it will be just gathering dust. Let me > know if you're interested - if you give me an EU address to ship to, I > can arrange that. Outside the EU I'm afraid the shipping costs and > duties would be higher than the cost of the hardware itself :) > Hello Jan, I appreciate the gesture but please don't send me stuff. Given that you carry your own GRUB build, I would recommend reverting to non-EFI stub boot for affected Atom systems, identified by DMI data, which should be easy to access on such systems. Look for 'goto fallback' in the existing loader logic in grub-core/loader/efi/linux.c - you'd need to jump to the same point to use the GRUB 2.06 logic for systems that fail that cannot load the EFI stub kernel image. Out of curiosity - these are all x86_64 Atoms right? How old are they? ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [REGRESSION] Re: [PATCH v4 5/5] efi: Use generic EFI loader for x86_64 and i386
This is likely an issue stemming from a bad interaction between the firmware's PE loader and the kernel's efi stub. The reason peimage can appear to fix this as it bypasses the firmware's PE loader for secure boot reasons. Hiding bugs in said PE loader is a coincidental side benefit and not an intentional fix. On Thu, Jun 27, 2024 at 11:27 AM Jan Čermák wrote: > > Hi Ard, > > sorry, I feel a little ashamed for replying after such a long time but I > wanted to do some due diligence first and didn't have time (or the Atom > board around) until now. > > > Does your Kconfig have EFI_DISABLE_PCI_DMA enabled by any chance? That > > could definitely produce the issues you are observing. > > No, this option is not set in our kernel. > > > In any case, given that you never relied on the EFI stub in the past > > on x86_64 (as GRUB 2.06 does not rely on it), you could just disable > > that in your Kconfig. > > > > That of course does not solve the Fujitsu issue, which apparently > > requires boot via the EFI stub, but I don't have a solution for that - > > I suppose that simply never worked with the old 'working' version of > > the OS? > > The goal is to have a single configuration that works for both (or > ideally - all amd64) targets so disabling EFI stub is not an option. > With no changes in the kernel in the meantime, just GRUB 2.06 loaded > kernel without issues on both devices. > > Anyway, what held me up a bit is that I wanted to try any of the major > distributions that already adopter GRUB 2.12. I tested Ubuntu 24.04 and > Arch's build of GRUB. While Arch's GRUB behaves exactly the same way > (error loading image) I was a bit puzzled that it's not the case of > Ubuntu. It took me a while to figure out why because there are quite > many patches applied but in the end it boiled up to the following two > from the grub package repo [1]: > > - loader-framework.patch > - efi-use-peimage-shim.patch > > Vanilla GRUB with these two patches AND with the introduced `peimage` > module added to the GRUB binary boots our kernel correctly. > > At this point I don't have the knowledge to figure out if it's just side > effect of that changes or if they indeed make any difference. And I'm > not sure what effect it has on the Fujitsu board but I can't find any > mentions of it in Ubuntu's issue tracker or anywhere on the internet so > I *guess* it's fine. > > I don't know what else to try at this point. If you'd like to look into > the issue yourself, I can give you the board I have, in the end it has > no other use for me than making sure that our OS boots correctly there, > and if the issue gets resolved, it will be just gathering dust. Let me > know if you're interested - if you give me an EU address to ship to, I > can arrange that. Outside the EU I'm afraid the shipping costs and > duties would be higher than the cost of the hardware itself :) > > Thanks, > Jan > > [1] > https://git.launchpad.net/~ubuntu-core-dev/grub/+git/ubuntu/tree/debian/patches/secure-boot > > ___ > Grub-devel mailing list > Grub-devel@gnu.org > https://lists.gnu.org/mailman/listinfo/grub-devel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [REGRESSION] Re: [PATCH v4 5/5] efi: Use generic EFI loader for x86_64 and i386
Hi Ard, sorry, I feel a little ashamed for replying after such a long time but I wanted to do some due diligence first and didn't have time (or the Atom board around) until now. Does your Kconfig have EFI_DISABLE_PCI_DMA enabled by any chance? That could definitely produce the issues you are observing. No, this option is not set in our kernel. In any case, given that you never relied on the EFI stub in the past on x86_64 (as GRUB 2.06 does not rely on it), you could just disable that in your Kconfig. That of course does not solve the Fujitsu issue, which apparently requires boot via the EFI stub, but I don't have a solution for that - I suppose that simply never worked with the old 'working' version of the OS? The goal is to have a single configuration that works for both (or ideally - all amd64) targets so disabling EFI stub is not an option. With no changes in the kernel in the meantime, just GRUB 2.06 loaded kernel without issues on both devices. Anyway, what held me up a bit is that I wanted to try any of the major distributions that already adopter GRUB 2.12. I tested Ubuntu 24.04 and Arch's build of GRUB. While Arch's GRUB behaves exactly the same way (error loading image) I was a bit puzzled that it's not the case of Ubuntu. It took me a while to figure out why because there are quite many patches applied but in the end it boiled up to the following two from the grub package repo [1]: - loader-framework.patch - efi-use-peimage-shim.patch Vanilla GRUB with these two patches AND with the introduced `peimage` module added to the GRUB binary boots our kernel correctly. At this point I don't have the knowledge to figure out if it's just side effect of that changes or if they indeed make any difference. And I'm not sure what effect it has on the Fujitsu board but I can't find any mentions of it in Ubuntu's issue tracker or anywhere on the internet so I *guess* it's fine. I don't know what else to try at this point. If you'd like to look into the issue yourself, I can give you the board I have, in the end it has no other use for me than making sure that our OS boots correctly there, and if the issue gets resolved, it will be just gathering dust. Let me know if you're interested - if you give me an EU address to ship to, I can arrange that. Outside the EU I'm afraid the shipping costs and duties would be higher than the cost of the hardware itself :) Thanks, Jan [1] https://git.launchpad.net/~ubuntu-core-dev/grub/+git/ubuntu/tree/debian/patches/secure-boot ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [REGRESSION] Re: [PATCH v4 5/5] efi: Use generic EFI loader for x86_64 and i386
On Thu, 16 May 2024 at 14:24, Jan Čermák wrote: > > Hi Ard, everyone, > > On 23. 05. 23 17:31, Ard Biesheuvel wrote: > > Switch the x86 based EFI platform builds to the generic EFI loader, > > ... > > We use GRUB as the loader for the Home Assistant Operating System (based > on Buildroot, using mostly unpatched GRUB 2 build [1]) and after > updating to the latest 2.12 release, this patch (commit > cfbfae1aef0694b416aa199291cfef7596cdfc20) has been identified to break > boot on Intel Atom NM10, at least with upstream kernel 6.6 with > CONFIG_EFI_STUB enabled. I reproduced it on Intel D525MW board, and > there are some more reports from HAOS users on Github [2]. > > Initially, we decided to revert the patch [3] for the time being, > however, while it fixed issue for users running on those rather old > boards, it broke boot [4] on the comparatively newer Fujitsu Esprimo > Q920. From the user reports, there is no BIOS update available that will > make any difference but reverting to a "vanilla" 2.12 fixes that. > > Do you have a clue what could have gone wrong, either with the original > patch, or why the revert breaks the other platform? I'll be happy to get > any details and perform tests on the NM10 board I have here. > Alternatively, I can also ask users with the Q920 for more details or do > some tests. > Upstream GRUB 2.06 does not boot Linux via the EFI stub, it uses the legacy EFI boot method where the Linux kernel is only entered after the EFI firmware has been shut down. GRUB 2.12 changed this, and will know always use the pure EFI boot sequence, relying on the EFI stub in the Linux kernel to shut down the firmware services. Does your Kconfig have EFI_DISABLE_PCI_DMA enabled by any chance? That could definitely produce the issues you are observing. In any case, given that you never relied on the EFI stub in the past on x86_64 (as GRUB 2.06 does not rely on it), you could just disable that in your Kconfig. That of course does not solve the Fujitsu issue, which apparently requires boot via the EFI stub, but I don't have a solution for that - I suppose that simply never worked with the old 'working' version of the OS? ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
