Re: Let's talk about boot loaders
On Fri, May 06, 2016 at 02:03:42PM -0400, Jon Masters wrote: > On 05/06/2016 01:10 PM, Mark Brown wrote: > > On Fri, May 06, 2016 at 12:20:40PM -0400, Jon Masters wrote: > > > But is it really worth trying after so long of the right thing not > > > happening? If anyone really cared about making general purpose distros > > > boot > > > on embedded boards, efforts to compel standards would have happened years > > > ago. To do it right, we would need to have a couple of vendors involved > > > who > > > could compel vendors to comply. > Note: by standards above, I specifically mean "separate platform flash" in > addition to all of the other associated things. Actually, to do it right you Oh, right. In that case this is all irrelevant anyway, there's no need to worry about magic areas of the disk and distros can just do whatever. > > Distros care and currently do ship on such systems - Debian stable lists > > a bunch of boards (something like 20 IIRC) as actively tested for > > example. The board and SoC vendors are to an extent irrelevant here, > I get your point, but for separate flash and getting vendors to ship > firmware, they very much need to be involved. Today, we can't agree as an > industry who is on the hook for this. With an enterprise hat on, I get to Personally I'm not convinced it's particularly worth worrying about - there's enough sensible ways to build systems where the separate storage for bootloaders just isn't solving problems people have that such things are going to be around for a while and inevitably people will end up wanting to run distros on them so at least the community distros need to work with them. > compel vendors to do the only sane thing (in my opinion) which is "thou > shalt ship EFI, on flash that we don't touch". And those who screwed up and > put EFI parameters on hidden disk partitions, or thought EFI variables were > a place to store MAC and platform parameters are slowly found and forced to > comply with the way the industry works. But on embedded, spending a few > cents to do the "right" thing is something that isn't going to happen unless > everyone mandates and pushes for it. It's not just cents, it's also things like board area, manufacturing process, usage models and for some use cases customers who want full control over the software stack. For some kinds of system like the enterprise market what you're describing is absolutely the right way to go but there's other segments where it either isn't solving problems people have or is currently actively worse so there's no compelling reason to adopt. But this is a bit off topic... signature.asc Description: PGP signature ___ linaro-dev mailing list linaro-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Let's talk about boot loaders
On Fri, May 06, 2016 at 12:20:40PM -0400, Jon Masters wrote: > But is it really worth trying after so long of the right thing not > happening? If anyone really cared about making general purpose distros boot > on embedded boards, efforts to compel standards would have happened years > ago. To do it right, we would need to have a couple of vendors involved who > could compel vendors to comply. Distros care and currently do ship on such systems - Debian stable lists a bunch of boards (something like 20 IIRC) as actively tested for example. The board and SoC vendors are to an extent irrelevant here, when people do this they often don't use any of the software the board vendors provide and just use things like upstream u-boot. Coming up with something that covers all the cases with minimal code for the distros will make life easier for them, if board vendors want to pick it up as well in what they're shipping (and ideally in how they spec their media usage) that's great but we're already winning even if all the board vendors totally ignore it. signature.asc Description: PGP signature ___ linaro-dev mailing list linaro-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Let's talk about boot loaders
On Fri, May 6, 2016 at 12:15 PM, Alexander Graf wrote: > > > On 06.05.16 13:03, Grant Likely wrote: >> On Fri, May 6, 2016 at 2:10 AM, Tom Rini wrote: >>> On Thu, May 05, 2016 at 10:21:25PM +0200, Alexander Graf wrote: So what's the goal here? Are we trying to force GPT on systems whose vendors never intended them to run with GPT? It really shouldn't matter at the end of the day whether we use GPT or MBR. All uEFI firmware implementations I'm aware of support both. So if you have a device whose bootloader collides with the GPT, just use MBR. As for the "protection" mechanism, I don't think it's a problem either. IIRC parted starts to create partitions with a sensible alignment (1 or 2MB). Most boot loaders fit perfectly fine within that gap. So this really boils down to - use GPT for systems that were designed for it or - use MBR with alignment gap to first partition end of story. There shouldn't be any middle ground. At least I haven't seen any so far :). >> >> That is a good point. MBR is a pain to deal with, but I don't think >> there is anything that it is absolutely required for in UEFI land. >> >>> This, for many use cases is also true. A reason that various SoCs pick >>> a magic location that is MBR compatible and not strictly GPT compatible >>> is that they don't see a use case for GPT-not-MBR being used. >>> By-and-large saying your SD card shall have an MBR and shall leave a gap >>> that is also well aligned for the card anyhow is enough for (enough) >>> firmware to reside in the magic locations. >> >> I think the key issue is how do the tools know what they are allowed >> to do. If (for example) the system is booted into a recovery/install >> image and needs to repartition and install onto the eMMC, can we get > > We really only have 2 cases: eMMC and SD. In both cases, I would expect > that there is some workable partition label on the target installation > medium. > >> away from the tools requiring board specific knowledge? A generic >> partitioning/install tool needs to know: >> - Is an MBR required (ie. SoC reads MBR to find firmware) >> - Is FW location at a fixed offset? >> - Is GPT supported? > > So none of these matter. If we can guarantee that > > * labels don't get recreated, only partitions change and > * partitions start earliest at 2MB > > then we have it all covered, because ... > >> >> If the tools can get that information, then it can make good decisions >> when reimaging a device, and it will make Marcin happier I'm sure. :-) >> >> I /don't/ think the general tools should need to know how to install >> the firmware itself. That is still a nasty board-specific problem >> (although even here we could make things better if we had a spec for >> managing the SoC's FW partition. I would like to see separate steps >> for FW provisioning, and OS install. ie. A board-specific tool to prep >> an SD card with the bare minimum for FW, and then generic tools to >> complete partitioning and install. > > ... as you mention here, the installation path on such a system would > involve a specialized image as your "installer image" which includes FW > at whereever it has to be plus the installer binaries. That installer > would usually load itself into RAM, use a server on the internet as > installation source and simply repartition the SD card / eMMC. > > So while it's repartitioning, we need to make sure that we don't remove > the RPi FAT partition for example. But that's easy - mark it as MBR EFI > partition (Label type 0xef) and fix any issues in your installer if it > thinks it wants to remove it. I think today the RPi firmware doesn't > boot from 0xef labels, but I'm sure we can talk to them to fix that. > > For all hard coded offset firmware, just make sure that your > repartitioning doesn't touch the first 2MB and your firmware will stay > alive from the days when your SD card contained the installer. Yes, that makes sense. One tweak I'd make, is I'd still create a partition table entry to cover the fixed-address region. Then the information is explicitly there. > >> For on-board devices (eMMC), FW provisioning should only be needed once >> For removable devices (SD), FW provisioning is only needed when FW >> must be on SD. (ie. no-onboard eMMC, or for FW recovery/upgrade) > > So that only leaves the "install from SD to eMMC" case. And that is > simply an installation issue, there's nothing we can do generically to > fix that. If you want to support that case, put the respective logic to > dump firmware to the eMMC into your installer. Or into some pre-stage on > the SD card. Or create a separate SD image that "upgrades firmware" on > the system. Agreed. Okay, that simplifies things quite a bit. It still needs to be documented, and I'd like to get some of this discussion into the UEFI spec. I'll draft something. BTW, I've looked into the limitations on MBR vs. GPT. From the UEFI sp
Re: Let's talk about boot loaders
On 06.05.16 13:03, Grant Likely wrote: > On Fri, May 6, 2016 at 2:10 AM, Tom Rini wrote: >> On Thu, May 05, 2016 at 10:21:25PM +0200, Alexander Graf wrote: >>> So what's the goal here? Are we trying to force GPT on systems whose >>> vendors never intended them to run with GPT? >>> >>> It really shouldn't matter at the end of the day whether we use GPT or >>> MBR. All uEFI firmware implementations I'm aware of support both. So if >>> you have a device whose bootloader collides with the GPT, just use MBR. >>> >>> As for the "protection" mechanism, I don't think it's a problem either. >>> IIRC parted starts to create partitions with a sensible alignment (1 or >>> 2MB). Most boot loaders fit perfectly fine within that gap. >>> >>> So this really boils down to >>> >>> - use GPT for systems that were designed for it or >>> - use MBR with alignment gap to first partition >>> >>> end of story. There shouldn't be any middle ground. At least I haven't >>> seen any so far :). > > That is a good point. MBR is a pain to deal with, but I don't think > there is anything that it is absolutely required for in UEFI land. > >> This, for many use cases is also true. A reason that various SoCs pick >> a magic location that is MBR compatible and not strictly GPT compatible >> is that they don't see a use case for GPT-not-MBR being used. >> By-and-large saying your SD card shall have an MBR and shall leave a gap >> that is also well aligned for the card anyhow is enough for (enough) >> firmware to reside in the magic locations. > > I think the key issue is how do the tools know what they are allowed > to do. If (for example) the system is booted into a recovery/install > image and needs to repartition and install onto the eMMC, can we get We really only have 2 cases: eMMC and SD. In both cases, I would expect that there is some workable partition label on the target installation medium. > away from the tools requiring board specific knowledge? A generic > partitioning/install tool needs to know: > - Is an MBR required (ie. SoC reads MBR to find firmware) > - Is FW location at a fixed offset? > - Is GPT supported? So none of these matter. If we can guarantee that * labels don't get recreated, only partitions change and * partitions start earliest at 2MB then we have it all covered, because ... > > If the tools can get that information, then it can make good decisions > when reimaging a device, and it will make Marcin happier I'm sure. :-) > > I /don't/ think the general tools should need to know how to install > the firmware itself. That is still a nasty board-specific problem > (although even here we could make things better if we had a spec for > managing the SoC's FW partition. I would like to see separate steps > for FW provisioning, and OS install. ie. A board-specific tool to prep > an SD card with the bare minimum for FW, and then generic tools to > complete partitioning and install. ... as you mention here, the installation path on such a system would involve a specialized image as your "installer image" which includes FW at whereever it has to be plus the installer binaries. That installer would usually load itself into RAM, use a server on the internet as installation source and simply repartition the SD card / eMMC. So while it's repartitioning, we need to make sure that we don't remove the RPi FAT partition for example. But that's easy - mark it as MBR EFI partition (Label type 0xef) and fix any issues in your installer if it thinks it wants to remove it. I think today the RPi firmware doesn't boot from 0xef labels, but I'm sure we can talk to them to fix that. For all hard coded offset firmware, just make sure that your repartitioning doesn't touch the first 2MB and your firmware will stay alive from the days when your SD card contained the installer. > For on-board devices (eMMC), FW provisioning should only be needed once > For removable devices (SD), FW provisioning is only needed when FW > must be on SD. (ie. no-onboard eMMC, or for FW recovery/upgrade) So that only leaves the "install from SD to eMMC" case. And that is simply an installation issue, there's nothing we can do generically to fix that. If you want to support that case, put the respective logic to dump firmware to the eMMC into your installer. Or into some pre-stage on the SD card. Or create a separate SD image that "upgrades firmware" on the system. Alex ___ linaro-dev mailing list linaro-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Let's talk about boot loaders
On Fri, May 6, 2016 at 2:10 AM, Tom Rini wrote: > On Thu, May 05, 2016 at 10:21:25PM +0200, Alexander Graf wrote: >> On 05.05.16 17:21, Grant Likely wrote: >> > On Thu, May 5, 2016 at 12:45 PM, Marcin Juszkiewicz >> > wrote: >> >> Recently my angry post on Google+ [1] got so many comments that it was >> >> clear >> >> that it would be better to move to some mailing list with discussion. >> >> >> >> As it is about boot loaders and Linaro has engineers from most of SoC >> >> vendor >> >> companies I thought that this will be best one. >> >> >> >> 1. https://plus.google.com/u/0/+MarcinJuszkiewicz/posts/J79qhndV6FY >> >> >> >> >> >> All started when I got Pine64 board (based on Allwinner A64 SoC) and had >> >> same issue as on several boards in past - boot loader written in some >> >> random >> >> place on SD card. >> >> >> >> Days where people used Texas Instruments SoC chips were great - in-cpu >> >> boot >> >> loader knew how to read MBR partition table and was able to load 1st stage >> >> boot loader (called MLO) from it as long it was FAT filesystem. >> >> >> >> GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and >> >> reads >> >> firmware files from there as long it is FAT. >> >> >> >> Chromebooks have some SPI flash to keep boot loaders and use GPT >> >> partitioning to find where from load kernel (or another boot loader). >> >> >> >> And then we have all those boards where vendors decided that SPI flash for >> >> boot loader is too expensive so it will be read from SD card instead. From >> >> any random place of course... >> >> >> >> >> >> Then we have distributions. And instead of generating bunch of images per >> >> board they want to make one clean image which will be able to handle as >> >> much >> >> as possible. >> >> >> >> If there are UEFI machines on a list of supported ones then GPT >> >> partitioning >> >> will be used, boot loader will be stored in "EFI system area" and it >> >> boots. >> >> This is how AArch64 SBSA/SBBR machines work. >> >> >> >> But there are also all those U-Boot (or fastboot/redboot/whateverboot) >> >> ones. >> >> They are usually handled by taking image from previous stage and adding >> >> boot >> >> loader(s) by separate script. And this is where "fun" starts... >> >> >> >> GPT takes first 17KB of storage media as it allow to store information >> >> about >> >> 128 partitions. Sure, no one is using so many on such devices but still >> >> space is reserved. >> >> >> >> But most of chips expects boot loader(s) to be stored: >> >> >> >> - right after MBR >> >> - from 1KB >> >> - from 8KB >> >> - any other random place >> >> >> >> So scripts start to be sets of magic written to handle all those SoCs... >> >> >> >> Solution for existing SoCs is usually adding 1MB of SPI flash during >> >> design >> >> phase of device and store boot loader(s) there. But it is so expensive >> >> someone would say when it is in 10-30 cents range... >> >> >> > >> > To try and summarize, what you're asking for is to define the usage >> > model for eMMC/SD when both the firmware* and OS are stored on the >> > same media. Some argue that these things should always be on separate >> > devices, but while the debate is interesting, it doesn't match the >> > reality of how hardware is being built. In which case, the derived >> > requirements are: >> > >> > 1) Co-exist with MBR partitioning >> > 2) Co-exist with GPT partitioning >> > 3) Be detectable --- partitioning tools must respect it >> > 4) Be speced. Write it down so that tool and SoC developers can see it >> > as a requirement >> > 5) Be usable regardless of firmware type (UEFI, U-Boot, Little Kernel, etc) >> > 6) Support some form of firmware non-volatile storage (variable storage) >> > >> > It would be really nice if we could also have: >> > 7) Support SoCs that hardcode boot code to specific locations >> > (after-MBR, 1K, 8K, random) >> > - May not be able to support all variants, but it is a worthy design >> > goal. >> > >> > Agreed? >> > >> > * I'm ignoring eMMC's separate boot area because that solution has >> > firmware and OS logically separated. Strong recommendation is for SoCs >> > to boot from boot area. Then normal GPT/MBR partitioning works just >> > fine. The rest of this discussion only applies If the SoC cannot do >> > that >> > >> > (For the following discussion, I refer to the UEFI spec because that >> > is where GPT is defined, but the expectation is that anything >> > described here can equally be used by non-UEFI platforms) >> > >> > I've just read through the UEFI GPT spec, and here are the constraints: >> > - MBR must be at the start of LBA0 (0 - 0.5k) >> > - Primary GPT must be at the start of LBA1 (0.5k to 4k, but may >> > collide with fw), >> > - It /seems/ like the GPT Header and GPT table can be separated by >> > some blocks. The GPT header has a PartitionEntryLBA field which >> > describes where the actual table of partitions starts. >> > - GPTHeader is only 92 bytes. >> > -
Re: Let's talk about boot loaders
On Fri, May 6, 2016 at 2:10 AM, Tom Rini wrote: > On Thu, May 05, 2016 at 05:30:55PM +0100, Grant Likely wrote: >> On Thu, May 5, 2016 at 4:59 PM, Mark Brown wrote: >> > On Thu, May 05, 2016 at 09:01:05PM +0530, Amit Kucheria wrote: >> >> On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz >> > >> >> > Solution for existing SoCs is usually adding 1MB of SPI flash during >> >> > design >> >> > phase of device and store boot loader(s) there. But it is so expensive >> >> > someone would say when it is in 10-30 cents range... >> > >> >> > Even 96boards CE specification totally ignored that fact while it could >> >> > be a >> >> > way of showing how to make popular board. Instead it became >> >> > yet-another-board-to-laugh (EE spec did not improve much). >> > >> >> > Is there a way to get it improved? At least for new designs? >> > >> >> Yes! I've added this suggestion to a list of suggestions for evolution >> >> of the 96boards spec. >> > >> > We already went round the houses repeatedly on that one :( >> >> It's not so dismal as all that! With the board size and price point >> the CE spec aims at, it is very unlikely that vendors will build >> boards with a separate storage device for FW.* However, by and large >> these platforms will be built with eMMC and eMMC has a separate boot >> area that is logically separate from the main pool. We can add a >> requirement for future versions to boot from the boot area when >> booting from eMMC. > > Saying that supporting the eMMC boot partitions shall be bootable AND > documenting how the various relevant EXT_CSD flags need to be set be > public is important. It won't do everyone good if you can't program the > board to boot from the boot partition. Yes, of course. This is definitely an aspirational item. It doesn't do any good for current SoCs that already have the boot scheme fixed, but it is a place where we can put pressure on vendors for what to do in future SoCs. g. ___ linaro-dev mailing list linaro-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-dev
