Re: Linux DRTM on UEFI platforms

[Brendan Trotter]

Hi,

On Fri, Aug 12, 2022 at 3:55 AM Matthew Garrett  wrote:
> On Thu, Aug 11, 2022 at 07:25:58PM +0930, Brendan Trotter wrote:
> > On Thu, Aug 11, 2022 at 3:16 AM Matthew Garrett  wrote:
> > > The kernel has no way to know this - *any* code you've run before
> > > performing a measurement could tamper with the kernel such that it
> > > believes it's fine. This is just as true in DRTM as it is in SRTM. But
> > > you know what the expected measurements should be, so you're able to
> > > either seal secrets to those PCR values or rely on remote attestation.
> >
> > In this scenario the kernel has no idea what the measurement should
> > be, it only knows the measurement that a potentially malicious boot
> > loader felt like giving the kernel previously (e.g. when the kernel
> > was installed).
>
> Even if the kernel has an idea of what the measurement should be, it has
> no way to verify that what it believes to be true is true - any
> malicious code could simply have modified the kernel to believe that
> anything it asks the TPM returns the "correct" answer.

Right. To achieve the best possible security; you'd need Secure Boot
to ensure that the kernel itself wasn't modified, and then the kernel
establishes a dynamic root of trust itself.

Anything involving a boot loader (e.g. Secure Boot ensure's boot
loader wasn't modified, then boot loader ensure's kernel wasn't
modified, then ) increases the attack surface for no benefit.

> > > Measurements are not opaque objects. If you're not able to reconstruct
> > > the expected measurement then you're doing it wrong.
> >
> > OK; so to detect if boot loader has always given kernel a bad/forged
> > measurement; the kernel repeats all of the steps involved in creating
> > the measurement itself exactly the same as the boot loader should have
> > (but might not have) so that kernel can compare a "known
> > good/trustworthy" measurement with the useless measurement that the
> > boot loader created for no sane reason whatsoever?
>
> No, some external agent does. Code running on the local machine can
> never determine whether the machine is trustworthy.

This part of the conversation evolved from "there's no way for kernel
to detect a MiTM boot loader (that provides a bad/forged
measurement)".

I'm not convinced an external agent can detect "bad/forged
measurement" either. E.g. if a MiTM boot loader always provided a
bad/forged measurement the external agent won't detect it (as the
measurement always matches the expected measurement), and then if the
MiTM boot loader is replaced by a good/honest boot loader the external
agent will decide that the good/honest boot loader is malicious
because its measurement doesn't match the expected bad/forged
measurement.

- Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: Linux DRTM on UEFI platforms

[Brendan Trotter]

Hi,

On Thu, Aug 11, 2022 at 3:16 AM Matthew Garrett  wrote:
> On Wed, Aug 10, 2022 at 06:37:18PM +0930, Brendan Trotter wrote:
>
> > [1] doesn't provide any useful information. How does a kernel know
> > that the callback provided by boot loader actually measures what it's
> > supposed to measure, or even does anything at all?
>
> The kernel has no way to know this - *any* code you've run before
> performing a measurement could tamper with the kernel such that it
> believes it's fine. This is just as true in DRTM as it is in SRTM. But
> you know what the expected measurements should be, so you're able to
> either seal secrets to those PCR values or rely on remote attestation.

In this scenario the kernel has no idea what the measurement should
be, it only knows the measurement that a potentially malicious boot
loader felt like giving the kernel previously (e.g. when the kernel
was installed).

> > [1] doesn't provide any useful information. Senter and skinit don't
> > provide a method for kernel to detect that (e.g.) a MiTM boot loader
> > has always measured a forgery and has changed unmeasured code in a
> > different way every time you boot.
>
> Measurements are not opaque objects. If you're not able to reconstruct
> the expected measurement then you're doing it wrong.

OK; so to detect if boot loader has always given kernel a bad/forged
measurement; the kernel repeats all of the steps involved in creating
the measurement itself exactly the same as the boot loader should have
(but might not have) so that kernel can compare a "known
good/trustworthy" measurement with the useless measurement that the
boot loader created for no sane reason whatsoever?

- Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: Linux DRTM on UEFI platforms

[Brendan Trotter]

Hi,

On Tue, Aug 9, 2022 at 8:25 PM Daniel P. Smith
 wrote:
> On 7/23/22 01:15, Brendan Trotter wrote:
> > On Sat, Jul 23, 2022 at 2:53 AM Daniel P. Smith
> >  wrote:
> >> On 7/7/22 23:36, Brendan Trotter wrote:
> >>> On Thu, Jul 7, 2022 at 7:18 PM Daniel P. Smith
> >>>  wrote:
> >>>> On 7/5/22 20:03, Brendan Trotter wrote:
> >>>>> On Wed, Jul 6, 2022 at 4:52 AM Daniel P. Smith
> >>>>>  wrote:
> >>>>>> On 6/10/22 12:40, Ard Biesheuvel wrote:> On Thu, 19 May 2022 at 22:59,
> >>>>>> To help provide clarity, consider the following flows for comparison,
> >>>>>>
> >>>>>> Normal/existing efi-stub:
> >>>>>> EFI -> efi-stub -> head_64.S
> >>>>>>
> >>>>>> Proposed secure launch:
> >>>>>> EFI -> efi-stub -> dl-handler -> [cpu] -> sl_stub ->head_64.S
> >>>>>
> >>>>> For more clarity; the entire point is to ensure that the kernel only
> >>>>> has to trust itself and the CPU/TPM hardware (and does not have to
> >>>>> trust a potentially malicious boot loader)..Any attempt to avoid a
> >>>>> one-off solution for Linux is an attempt to weaken security.
> >>>>
> >>>> Please elaborate so I might understand how this entrypoint allows for
> >>>> the kernel to only trust itself and the CPU/TPM.
> >>>
> >>> Is this a serious request?
> >>
> >> Yes, it was serious because I found the statements to be terse and open
> >> to interpretation. Specifically, when I read it, it almost seemed that
> >> the position is that dynamic launch is not needed, and that the kernel
> >> could just establish its own RoT.
> >>
> >>> Kernel is started (via. firmware using the kernel's efi-stub, or via.
> >>> "kexec()", or..); and regardless of how the kernel was started the
> >>> kernel establishes its own dynamic root of trust.(e.g. AMD"s SKINIT or
> >>> Intel's TXT, followed by measuring the remainder of itself and
> >>> anything passed from firmware like APCI tables) without relying on a
> >>> call-back provided by "untrusted by kernel" third-parties that don't
> >>> exist in most cases. The dynamic root of trust that kernel creates
> >>> depends on the kernel, CPU, TPM, etc (and excludes untrusted and
> >>> unnecessary third parties)..
> >>
> >> This clarifies the previous statements and I would say, yes, this is one
> >> approach. Considering the challenges we have/are facing in getting a
> >> minimal post-launch handling (sl-stub) into the setup kernel, I would be
> >> hard-pressed to believe adding all the pre-launch handling would be
> >> found to be acceptable. If the intent is to have it completely
> >> self-contained, this would require,
> >>1) the introduction of a TPM driver into the setup kernel, a hole I
> >>   would rather not go down again
> >
> > ..which is no more work than the introduction of a TPM driver into one
> > boot loader (probably less so, as Linux already has TPM drivers that
> > are used later), and significantly less work than introducing a TPM
> > driver into many boot loaders.
>
> Already been there and no it is not as straight forward as one may
> think. A minimal driver was attempted but rejected by the TPM
> maintainers with the requirement to reuse the existing driver code. So
> we would gladly welcome someone to come rewrite the TPM drive.
> Specifically to de-tangle the core TPM interface code away from all of
> its kernel proper dependencies so that it can be used by both the setup
> kernel and the kernel proper.
>
> > Note that to do this well (with some protection against malicious
> > devices) you'd disable all PCI devices, then set up IOMMU as "allow
> > the least possible", then establish a dynamic root of trust, then
> > enable and allow individual devices only if/when their driver is
> > started by kernel and not sooner. Without doing it like this there's
> > less guarantee that the code/data being measured is the same as the
> > code/data being executed/used (as a malicious device can modify
> > anything before it's measured, restore the original extremely briefly
> > while measurement is done, then modify anything after it's measured
> > and before it's used). Also note that I mean "some protection" and not
> > 100% protection (

Re: Linux DRTM on UEFI platforms

[Brendan Trotter]

Hi,

On Sat, Jul 23, 2022 at 2:53 AM Daniel P. Smith
 wrote:
> On 7/7/22 23:36, Brendan Trotter wrote:
> > On Thu, Jul 7, 2022 at 7:18 PM Daniel P. Smith
> >  wrote:
> >> On 7/5/22 20:03, Brendan Trotter wrote:
> >>> On Wed, Jul 6, 2022 at 4:52 AM Daniel P. Smith
> >>>  wrote:
> >>>> On 6/10/22 12:40, Ard Biesheuvel wrote:> On Thu, 19 May 2022 at 22:59,
> >>>> To help provide clarity, consider the following flows for comparison,
> >>>>
> >>>> Normal/existing efi-stub:
> >>>>EFI -> efi-stub -> head_64.S
> >>>>
> >>>> Proposed secure launch:
> >>>>EFI -> efi-stub -> dl-handler -> [cpu] -> sl_stub ->head_64.S
> >>>
> >>> For more clarity; the entire point is to ensure that the kernel only
> >>> has to trust itself and the CPU/TPM hardware (and does not have to
> >>> trust a potentially malicious boot loader)..Any attempt to avoid a
> >>> one-off solution for Linux is an attempt to weaken security.
> >>
> >> Please elaborate so I might understand how this entrypoint allows for
> >> the kernel to only trust itself and the CPU/TPM.
> >
> > Is this a serious request?
>
> Yes, it was serious because I found the statements to be terse and open
> to interpretation. Specifically, when I read it, it almost seemed that
> the position is that dynamic launch is not needed, and that the kernel
> could just establish its own RoT.
>
> > Kernel is started (via. firmware using the kernel's efi-stub, or via.
> > "kexec()", or..); and regardless of how the kernel was started the
> > kernel establishes its own dynamic root of trust.(e.g. AMD"s SKINIT or
> > Intel's TXT, followed by measuring the remainder of itself and
> > anything passed from firmware like APCI tables) without relying on a
> > call-back provided by "untrusted by kernel" third-parties that don't
> > exist in most cases. The dynamic root of trust that kernel creates
> > depends on the kernel, CPU, TPM, etc (and excludes untrusted and
> > unnecessary third parties)..
>
> This clarifies the previous statements and I would say, yes, this is one
> approach. Considering the challenges we have/are facing in getting a
> minimal post-launch handling (sl-stub) into the setup kernel, I would be
> hard-pressed to believe adding all the pre-launch handling would be
> found to be acceptable. If the intent is to have it completely
> self-contained, this would require,
>   1) the introduction of a TPM driver into the setup kernel, a hole I
>  would rather not go down again

..which is no more work than the introduction of a TPM driver into one
boot loader (probably less so, as Linux already has TPM drivers that
are used later), and significantly less work than introducing a TPM
driver into many boot loaders.

Note that to do this well (with some protection against malicious
devices) you'd disable all PCI devices, then set up IOMMU as "allow
the least possible", then establish a dynamic root of trust, then
enable and allow individual devices only if/when their driver is
started by kernel and not sooner. Without doing it like this there's
less guarantee that the code/data being measured is the same as the
code/data being executed/used (as a malicious device can modify
anything before it's measured, restore the original extremely briefly
while measurement is done, then modify anything after it's measured
and before it's used). Also note that I mean "some protection" and not
100% protection (it's unsolvable unless firmware uses an "allow the
least necessary by default" strategy) but "some protection" is better
than none.

In other words; it isn't necessarily just TPM driver alone that would
need to be started "earlier" (ideally it's also IOMMU and PCI
enumeration too). However, "earlier" is "earlier than the
establishment of dynamic root of trust"; which can be achieved by
moving the establishment of dynamic root of trust to a later stage
(rather than bringing things like TPM driver to an earlier stage)..

>   2) potentially the ability to load files from disk if it is not
>  acceptable for the bootloader to load the DCE (ACM/SLB)

..which is both unnecessary and ill-advised.

>   3) miscellaneous system evaluations, memory table, machine check, etc.

..which is the same regardless of where it's done.

> The only thing that is gained from such an approach is to make dynamic
> launch for Linux be self-contained in the kernel.

Making it self-contained in the kernel means:

a) it actually works, for the most important use case (where UEFI
firmware boots Linux's efi-stub di

Re: Linux DRTM on UEFI platforms

[Brendan Trotter]

Hi,

On Thu, Jul 7, 2022 at 7:18 PM Daniel P. Smith
 wrote:
> On 7/5/22 20:03, Brendan Trotter wrote:
> Greetings!
>
> Not sure why I got dropped from distro, but no worries.
>
> > On Wed, Jul 6, 2022 at 4:52 AM Daniel P. Smith
> >  wrote:
> >> On 6/10/22 12:40, Ard Biesheuvel wrote:> On Thu, 19 May 2022 at 22:59,
> >> To help provide clarity, consider the following flows for comparison,
> >>
> >> Normal/existing efi-stub:
> >>EFI -> efi-stub -> head_64.S
> >>
> >> Proposed secure launch:
> >>EFI -> efi-stub -> dl-handler -> [cpu] -> sl_stub ->head_64.S
> >
> > For more clarity; the entire point is to ensure that the kernel only
> > has to trust itself and the CPU/TPM hardware (and does not have to
> > trust a potentially malicious boot loader)..Any attempt to avoid a
> > one-off solution for Linux is an attempt to weaken security.
>
> Please elaborate so I might understand how this entrypoint allows for
> the kernel to only trust itself and the CPU/TPM.

Is this a serious request?

Kernel is started (via. firmware using the kernel's efi-stub, or via.
"kexec()", or..); and regardless of how the kernel was started the
kernel establishes its own dynamic root of trust.(e.g. AMD"s SKINIT or
Intel's TXT, followed by measuring the remainder of itself and
anything passed from firmware like APCI tables) without relying on a
call-back provided by "untrusted by kernel" third-parties that don't
exist in most cases. The dynamic root of trust that kernel creates
depends on the kernel, CPU, TPM, etc (and excludes untrusted and
unnecessary third parties)..

The only potential benefit that the callback solution provides is that
it, in theory, it could reduce duplication of work for other operating
systems (FreeBSD, Solaris, Haiku, Fuchsia, .. could use the same
callback instead of doing it themselves); but previous discussions
(talk of formalising the contract between the boot stub and the Linux
kernel) suggest that you aren't interested in any other OS.

This leaves me wondering what your true motivation is. Are you trying
to benefit GRUB/Trenchboot (at the expense of security, end-user
convenience, distro installer hassle, etc); or trying to manufacture
scope for future man-in-the middle attacks (by promoting a solution
that requires something between firmware and kernel)?

- Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: Linux DRTM on UEFI platforms

[Brendan Trotter]

Hi,

On Wed, Jul 6, 2022 at 4:52 AM Daniel P. Smith
 wrote:
> On 6/10/22 12:40, Ard Biesheuvel wrote:> On Thu, 19 May 2022 at 22:59,
> To help provide clarity, consider the following flows for comparison,
>
> Normal/existing efi-stub:
>   EFI -> efi-stub -> head_64.S
>
> Proposed secure launch:
>   EFI -> efi-stub -> dl-handler -> [cpu] -> sl_stub ->head_64.S

For more clarity; the entire point is to ensure that the kernel only
has to trust itself and the CPU/TPM hardware (and does not have to
trust a potentially malicious boot loader)..Any attempt to avoid a
one-off solution for Linux is an attempt to weaken security.

The only correct approach is "efi-stub -> head_64.S -> kernel's own
secure init"; where (on UEFI systems) neither GRUB nor Trenchboot has
a valid reason to exist and should never be installed.


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: the rootcause of the grub2 that can support USB keyboard

[Brendan Trotter]

Hi,

On 8/21/18, Sherry Zhang(BJ-RD)  wrote:
> But I have found the USB keyboard module-usb_keyboard.mod that I don't load
> ,
> So why the usb keyboard can be used when I don't load usb modules?

A better question is, why does "module-usb_keyboard.mod" exist in the
first place? In theory, GRUB should be able to use whatever the
firmware provides to access key presses and shouldn't need a module at
all.

However, there were a few incredibly dodgy computers (mostly "late
1990s" time frame?) where the firmware expects a PS/2 keyboard and
firmware's keyboard support doesn't work with a USB keyboard. GRUB
probably provides a "module-usb_keyboard.mod" so that USB keyboards
will work properly on these incredibly dodgy computers.

Note: I'm not a GRUB developer or anything, I just know how much
computers can suck. ;-)

- Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Brendan Trotter]

Hi,

> Le 7 mars 2017 17:24, "Vladimir 'phcoder' Serbinenko" 
a écrit :
>
> I'd like to know more about the usecase. Generally you should avoid
downloading or loading too large files in bootloader. I.a. TFTP protocol
has problems with files over about 100MIB. Generally you should download
only kernel + initrd and rest of the system should be on iSCSI or NFS.

"Ancient TFTP" (with 512-byte blocks and 16-bit block numbers that aren't
allowed to roll over) has problems with files over 32 MiB. "Modern
TFTP" (with larger/negotiated block size and 16-bit block numbers that are
allowed to roll over) has no limit at all.


> On Tue, Mar 7, 2017, 09:09 Michel Hermier 
wrote:
>
> Because I don't trust automatic detection. Even if one say it is 200%
safe, there is allways that machine that nobody heard of that will fail. So
having user being able to force some values is usually a good idea.

What do you think is more reliable: well designed auto-detection code that
was written and tested by competent developer/s that know exactly what
they're doing and why; or a random user who failed read the documentation,
thought it did something else, screwed everything up, then blames you for
their mistake, then blames you for giving them the ability to make the
mistake?


Cheers,

Brendan
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: chained grub2 derivative bootauto system

[Brendan Trotter]

Hi,

On Fri, Aug 31, 2012 at 11:59 PM, Lennart Sorensen
lsore...@csclub.uwaterloo.ca wrote:
 On Fri, Aug 31, 2012 at 09:19:43AM +1000, Philip Rhoades wrote:
 Ivo,

 Interesting idea - I particularly like the idea of booting from
 arbitrary isos.

 Too bad that accessing an iso on a usb key is nothing like an actual cd
 or dvd and the sytem you boot must have support for the fact you put the
 iso on the usb key rather than as a real disc.  Nothing the boot loader
 can do to change that.

 If it was really that simple, someone would already have made such a
 boot loader for use on usb keys.  The reason it doesn't exist yet is
 that it can't exist.

For PC BIOS, having a USB with multiple ISO images would actually be
relatively easy.

The USB's boot code would need to hook Int 0x15 to steal some memory,
then install code that emulates a CD (and El Torito) into the stolen memory.
Obviously you'd also hook Int 0x13 so that the BIOS disk services are
redirected to your CD emulation code.

I'd expect that similar would be possible for UEFI (e.g. create a special
UEFI driver that emulates CD/s).

The main problem is after the OS on the ISO tries to take control of hardware
and fails to find its (emulated) CD. For some OSs this may not be a problem -
e.g. MS-DOS and FreeDOS (which continue to use the BIOS services
and don't take control of hardware), Linux (as long as the root
partition isn't on
the emulated CD), etc. For other OS's (Windows) it can't work.

However, someone that wants to create a USB like this might not care about
those unsupportable OSs anyway.

Finally; I'm not sure how a scheme like this would involve GRUB - you'd want
relatively specialised USB boot code (not GRUB). GRUB could (potentially)
be installed inside one or more of the ISOs, but this would be a normal
El Torito (or UEFI) boot as far as GRUB would know.


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: RFC: New multiboot2 memory map entry type

[Brendan Trotter]

Hi,

On Sat, Dec 24, 2011 at 3:01 PM, Seth Goldberg seth.goldb...@oracle.com wrote:
 On 09.11.2011 06:25, Seth Goldberg wrote:
 The proposal is to add an additional type (value = 6) that denotes runtime 
 memory that some firmware marks as required to be mapped to take advantage 
 of services after the end of boot (UEFI is the canonical example).  Without 
 this information, it's impossible for a multiboot2-compliant OS to set up 
 proper mappings for this memory.

   I'm actually withdrawing this request in exchange for a multiboot2 info tag 
 that includes the EFI memory map (as returned by GetMemoryMap()).  I think 
 this is better because it's more complete and provides an OS with a much more 
 complete set of map information (an array of EFI_MEMORY_DESCRIPTOR 
 structures, as per the UEFI 2.0 spec) for UEFI systems (and considering the 
 workarounds required to fully utilize UEFI, this is a necessity), so the 
 proposal is (based on the 
 http://bzr.savannah.gnu.org/r/grub/branches/multiboot2/ repo):

I think the multiboot memory map should be in a single, clear,
consistent and complete format; and shouldn't just be a dumb copy of
whatever each different type of firmware felt like providing; and a
multiboot OS should be able to use that single, clear, consistent and
complete memory map without caring about where it came from or what
type of firmware booted GRUB.

My recommendation would be base_address, size, type, attributes,
NUMA_domain, where the multiboot specification determines the
meanings for type and the meanings of bits in the attributes
bit-field without any regard for any other specification; and where
the boot loader (GRUB) also uses the ACPI SRAT (or any other
information, if possible) to try to determine the NUMA domain that
each area belongs to (and any hot-plug RAM areas).

Suggested types would be:
 - reserved (includes unknown, areas used by legacy devices, ROMs, APICs, etc)
 - usable RAM (combines AddressRangeMemory, EfiConventionalMemory,
EfiBootServicesCode, EfiBootServicesData)
 - ACPI reclaimable (OS can reuse after it has finished with all ACPI tables)
 - boot code reclaimable (OS can reuse after it has finished with all
multi-boot information)
 - boot module reclaimable (used to store the kernel and any modules
that were loaded with the kernel)
 - preserved (combines ACPI NVS, EfiRuntimeServiceCode,
EfiRuntimeServicesData and EfiPalCode)
 - faulty RAM
 - not-present RAM (area reserved for hot-add)
 - unused (the only type of area that is suitable for use for memory
mapped PCI devices)
 - unusable (not used, but can't be used for memory mapped PCI devices either)

Suggested attribute flags would be:
- uncached (same as UEFI spec)
- write combining (same as UEFI spec)
- write-through (same as UEFI spec)
- write back (same as UEFI spec)
- uncached exportable (same as UEFI spec)
- write protectable (same as UEFI spec)
- read protectable (same as UEFI spec)
- execute protectable (same as UEFI spec)
- runtime (same as UEFI spec)
- volatile (same as ACPI spec)
- slow (same as ACPI spec)
- errorLog (same as ACPI spec)
- is hot-plug (e.g. either usable RAM that may be unplugged or not
present RAM that may be plugged in)
- has NUMA domain (NUMA domain field is valid if this flag is set)

In addition, multiboot should guarantee that the memory map:
- is sorted in order from lowest base address to highest base address
- contains no overlapping areas
- contains no adjacent areas of the same type, attributes and NUMA domain
- contains no unmentioned holes (every byte in the 64-bit physical
address space is accounted for; for example, a 32-bit physical
addresses only system would have a massive 1757500617114 GiB area
marked as unusable at the top of the 64-bit physical address space).

Note: on 80x86, determining the size of the physical address space
that the system supports involves using CPUID.function 0x8008 if
present (and working around the errata for Pentium 4 (family = 0xF,
model = 3) which incorrectly reports support for 40-bit physical
addresses when it only supports 36-bit physical addresses). If this
CPUID function isn't supported then check feature flags to see if
PSE36 or PAE is supported and assume 36-bit if they are, and assume
32-bit if they aren't (or if CPUID itself isn't supported).

The boot loader (GRUB) would do the best it can with available
information. For example, for an ancient PC BIOS system that doesn't
even support int 0x15, eax=0xE820 it might use int 0x12 to get one
usable RAM area (from zero to the start of the EBDA), then assume the
area immediately above that up to 0x0010 is reserved, then use
other old BIOS functions to determine if there's more RAM above
0x0010, then have an assumed unusable area (in case the old BIOS
functions didn't report all extended memory), then have an assumed
unused area up to about 0xFE00, then have an assumed reserved
area up to 0x (in case of APICs, ROM, etc), then have an
assumed unusable area for 

Re: Legacy GRUB to GRUB2

[Brendan Trotter]

Hi,

On Thu, Jul 21, 2011 at 2:44 PM, Yash Jain yash2le...@gmail.com wrote:
 Could you please let me know how can i upgrade to GRUB2(because
 trusted GRUB is based on legacy GRUB) or it is still fine if i can use
 the legacy GRUB to load kernel 2.6.37.

I'm still using GRUB-legacy with linux-2.6.38-gentoo-r6, and it
works perfectly. I'm not using any patches provided by GRUB-IMA, but I
doubt Linux developers have changed anything that would break those
patches (or any other boot loader that supports Linux), and I doubt
there's any reason for you to switch to GRUB2 in the short term.

In the longer term, eventually you'll need to switch to a boot loader
that supports UEFI (the firmware/BIOS that GRUB-legacy relies on is
slowly being deprecated). For Linux your choices are limited to GRUB2
and ELILO, and neither of these boot loaders have any support for
ensuring boot files haven't been tampered with. If you have a strict
requirement for that (e.g. Dept. of Defence or something), then you
may have no choice but to switch to an OS that cares more about your
security than the freedom of malicious attackers.


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: Contribution: a file loading module

[Brendan Trotter]

Hi,

Note: I am me. I speak on behalf of myself only, and don't represent
the views of the GRUB team in any way.

On Fri, Jun 24, 2011 at 7:03 PM, Pierre-Nicolas Clauss
p...@tuxfamily.org wrote:
 Are you saying that :

 kernel /my_multiboot_kernel
 chainloader /my_chainloader
 boot

 will load the kernel as per the multiboot header and then call the 
 chainloader ?
 does grub guarantee that the kernel will be kept in memory ?

The terminology GRUB uses is very misleading. What they call kernel
really means the module that the boot loader should pass control to,
and it does not need to be a kernel as such. The kernel may just be
a layer that sets things up for the real kernel, where the real kernel
is loaded by GRUB as a module (or possibly not loaded by GRUB at all).

For example, if your chainloader is happy to relocate itself and then
copy the real kernel to 0x0010, then you might do:

kernel /boot/my_chainloader
module /boot/my_kernel
boot

Otherwise, if you want to keep your chainloader as is for other
reasons, then you might want an extra layer (thunk code) to set
things up in the state your chainloader expects (e.g. relocate things,
potentially switch CPU modes, etc):

kernel /boot/my_thunk_code
module /boot/my_chainloader
module /boot/my_kernel
boot

In GRUB, chainloading is chainloading in the old MBR partition way -
e.g. loading a 512-byte boot sector from the start of a partition and
passing control to it (in real mode). GRUB can't pass control to a
512-byte boot sector while also passing control to the module that
the boot loader should pass control to (what GRUB incorrectly calls
kernel) - it simply doesn't make sense.

If your chainloader actually is an old MBR partition style
chainloader, then you're not really using GRUB at all (you're using
the de facto boot method that's been standard for about 30 years,
which GRUB and just about every other boot manager for 80x86 PC BIOS
happens to support).If your chainloader isn't an old MBR partition
style chainloader, then you should call it something different to
avoid confusion (maybe your OS name stage 1.or something).



Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: Two Small Patches (x86 VolId Sun Label Checking)

[Brendan Trotter]

Hi,

2010/12/27 Vladimir 'φ-coder/phcoder' Serbinenko phco...@gmail.com:
 On 12/26/2010 10:15 PM, ehem+g...@m5p.com wrote:
 Quite simple, the disk slice scheme detection routines vary in the
 quality of their detection. In particular, the MSDOS-style detection is
 *extremely* brittle. The only even mildly distinguishing characteristic
 it finds is ensuring only the msb of the boot-flag byte is set. The other
 thing it looks for is the 0xAA55 signature, but that is merely a signal
 to PC-BIOSes that the disk is bootable; as such *any* bootable disk for a
 IBM-PC will have that signature, whether or not it is actually using the
 MSDOS-style header. A 1 in 65536 chance of a false positive is bad.

If a disk is bootable then it should have the 0xAA55 signature,
regardless of whether or not there's a partition table.

If a disk is *not* bootable then it should *not* have the 0xAA55
signature, regardless of whether or not there's a partition table.
Lots of OSs (including Linux/fdisk) have the same stupid bug where
they put the 0xAA55 signature on pure data disks where it doesn't
belong. The worst case here is that the computer crashes during boot
if the disk ever becomes the first disk (rather than the BIOS moving
on to the next type of bootable device, and/or rather than the user
getting a No OS found BIOS error message), so it's usually just an
annoying bug (and not a serious bug, unless someone is perverted
enough to attempt to use a bootable CD-ROM or something as a type of
fail-safe in case their first drive goes missing), but it's still
wrong.

To check if there is/isn't an MBR partition table, first check for
GPT. If there's no GPT then; for each (potential) MBR partition table
entry that isn't type = unused check for (boot-flag  0x7F == 0)
then check if the starting CHS address is sane (e.g. not an invalid
sector, and with work-arounds for unrepresentable) and if the ending
CHS address is sane (e.g. not invalid and not less than or equal to
the starting CHS, and with work-arounds for unrepresentable) and
then do the same for the starting and ending LBA addresses. Then you
could check for overlapping partitions.

If a partition is marked as active then you could check for the
0xAA55 signature in the first sector of that partition, but that is
the only case where checking the 0xAA55 signature makes any sense.

 Actually 1 in 2^(7*4+16) =2^44 in you take into account the both checks
 and consider every possible sector equiprobable.

Actually, if you take into account that the second check is worthless
and/or wrong; then for the current code; if there's 4 (assumed)
usable partitions there's 1 chance in 4*127 of thinking there's a
partition table when there isn't, and if there's only one (assumed)
usable partition then it drops to 1 chance in 127; and if the MBR
contains random data the former is much more likely than the latter (I
was too lazy to do the calculation properly).


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: RFC: Plan for new hwmatch command

[Brendan Trotter]

Hi,

On Thu, Nov 18, 2010 at 3:28 PM, Evan Broder e...@ebroder.net wrote:
   Based on some off-list discussion, I'd like to try a different
 angle for the Lua patches I submitted a week or two ago. For context,
 Ubuntu is interested in setting gfxpayload=keep as often as we can in
 the next release [1]. Since gfxpayload=keep doesn't work with all
 hardware/driver combinations, we need a way to selectively turn it on,
 based on a whitelist or blacklist.

I'm curious why setting gfxpayload=keep doesn't work with all
hardware/driver combinations; and by extension, also wondering if
Ubuntu is relying on extending an already over-complicated generic
boot loader (GRUB) to hide symptoms of problems that exist elsewhere
(rather than fixing the problem/s that exist elsewhere, and maybe not
setting gfxpayload=keep until those problems are fixed).

Mostly, I'm wondering where GRUB's responsibilities end and OS
responsibilities start, and how this can effect the GRUB developer's
ability to maintain GRUB in the upcoming months/years/decades; given
that it's easy to add features but almost impossible to remove them.


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Brendan Trotter]

Hi,

Just thought I'd throw my 2 cents in..

Any software (except the software that owns the MBR) that uses any
sectors that are in the first track with the MBR and outside of any
partition (e.g. before the first partition) is broken. Not only will
this broken software (potentially) conflict with the software that
owns the MBR (not just GRUB), but it will also (potentially)
conflict with any other pieces of broken software.

It does not matter if the broken software is proprietory of not, or if
broken software restrict users' freedom or not, or if broken software
is popular or well known or not, or if the broken software is
extremely useful and/or has an excuse. All of these considerations
are irrelevant. The software touches something it shouldn't, therefore
it is broken.

I doubt that Windows is *directly* at fault. However, allowing broken
software access to these sectors is a security flaw. If Windows allows
software to tamper with these sectors, then what else does Windows
allow software to tamper with? Does Windows allow software to install
a virus in the MBR? Does Windows allow software to install a virus in
one or more sectors that are loaded (and executed) by the MBR? At a
minimum Windows should have a UAC warning (a dialog box requiring
admin privileges that alerts the user to the attempted access, and
gives them the option to deny permission), but I wouldn't be surprised
if there's none. If there is no such warning, then the issue should be
reported to Microsoft as a security vulnerability, because that's
exactly what it is.

While the situation is unfortunate, Vladimir's suggested use of
error-correcting code is a very good idea. However, I think it should
go one step further. If the user has several OSs, and each OS happens
to have several pieces of broken software that trash different
sectors, then simply avoiding those trashed sectors won't stop the
system from becoming unbootable. GRUB has to restore the rightful
contents of trashed sectors during boot to minimise the total number
of sectors that are trashed at any point in time. In addition to
increasing GRUB's ability to tolerate trashed sectors, this would also
help to discourage broken software (and possibly, make it easier for
users to identify which piece/s of software is broken).

As a service to end-users; it would also be very nice if GRUB
displayed a high visibility warning that something is tampering with
the system's security and attempted to identify which pieces of
software may be tampering with the system's security (so the user can
more easily identify the cause). This warning could/should include a
URL for a web page that explains the issue in more detail, and maybe a
hex dump from each trashed sector so that signatures can be obtained
more easily.

Also, as a service to end-users; if/when broken software is correctly
identified the publisher of that software should be informed of their
mistake - something like a formal email that (politely) explains to
them why their software is broken and (even more politely) requests
them to fix their broken software.


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: Improperly nested partitions, help needed!

[Brendan Trotter]

Hi,

2010/9/23 Vladimir 'φ-coder/phcoder' Serbinenko phco...@gmail.com:
 On 09/22/2010 07:44 PM, Lennart Sorensen wrote:
 On Wed, Sep 22, 2010 at 07:08:49PM +0200, Grégoire Sutre wrote:
 On 09/22/2010 18:44, Lennart Sorensen wrote:

 After all msdos partitions tables may only exist in MBR and extended
 partitions

 According to which standard?

 Well, I think DOS and common practice.

 Can you think of any OS that supports any other cases?


 Minix.

Lennart Sorensen is entirely correct. If both Minix and GRUB are
broken, then both Minix and GRUB should be fixed...


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: 3. Re: LiveCD bootloader

[Brendan Trotter]

Hi,

2010/9/15 Dmitry Ilyin idv1...@mail.ru:
 I guess it is possible to chainload to real cdrom in your drive same way we 
 chainload to windows partition, you can also use script (grub or lua) to 
 determine that your drive is not empty and show chainload menu entry.

 Or if you cannot chainload perhaps you could boot this (maybe with memdisk 
 from syslinux)
 http://btmgr.sourceforge.net/about.html

The problem with chainloading a CD-ROM (more correct would be
emulating El Torito for a CD-ROM) is that often the BIOS only
provides a device number (and support via. int 0x13) for the CD-ROM
when the BIOS itself has booted the CD-ROM. This means that to
reliably access the CD-ROM you'd need to do device detection (e.g.
scan PCI buses looking for likely controllers), provide drivers for
different types of controllers (ATA, AHCI, 3 or 4 types of USB, lots
of different SCSI, etc, and maybe the legacy proprietory interfaces
you find on ancient sound cards) and drivers for different CD-ROMs
that can be attached to those controllers. In addition to that you'd
need to hook the BIOS's int 0x13, so that when the boot loader on
the CD-ROM tries to load sectors from CD using the BIOS it works like
it should (with all types of booting from CD - emulated floppy,
emulated hard disk and no emulation); and to do that properly
you'd need to steal some RAM from the BIOS (to store GRUB's drivers
in); which is typically done by intercepting the BIOS functions that
are used to get the amount of free memory (e.g. int 0x15,
eax=0xE820, plus int 0x12 and the segment of EBDA value in the
BIOS data area). Of course after the OS boots there's no way to tell
it that it can use the RAM that GRUB's drivers were using so that RAM
is lost; and because OS's tend to make assumptions (like there's at
least 512 KiB of usable RAM starting at 0x) you'd need to
make sure the drivers are small (e.g. no more than about 64 KiB for
everything). You also wouldn't be able to assume that the boot loader
on the CD doesn't constantly switch between real mode and protected
mode, or use unreal mode, which means you'd be limited to 16-bit
real mode code, and you couldn't switch to 32-bit code when you want
because there's no way of doing that without trashing CPU state that
the CD's boot loader could be relying on.

Basically it's not impossible, but it is a lot of work, especially if
you don't limit it to all kinds of IDE ATAPI CD-ROMs (and
forget/ignore SATA/AHCI, USB, SCSI, etc).


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: LiveCD bootloader

[Brendan Trotter]

Hi,

On Wed, Sep 8, 2010 at 12:12 PM, Teresa e Junior
teresaejun...@gmail.com wrote:
 But due to some features I need (not the eyecandy), I noticed the CD
 only boots without gfx support (which is the most important for me).

You can't really safely use graphics support in a Live-CD anyway.

GRUB and Linux use video mode numbers to request a video mode. These
video mode numbers can be different for different video cards. Early
versions of VBE did define some standard mode numbers, but these
became obsolete in VBE version 2.0 (back in 1994). Software written
after 1994 shouldn't rely on these obsolete video mode numbers. For
example, if you're lucky mode 0x118 might be 1024*768 with 24-BPP, but
nothing guarantees that anymore, and it could easily be any other
video mode. There's also no guarantee that the the video card supports
the video mode (e.g. a lot of video cards only support 32-BPP video
modes and don't support any 24-BPP video modes; and some only support
24-BPP video modes and not 32-BPP video modes; and the same is true
for 15-BPP vs. 16-BPP).

Finally, even if you're lucky and VBE does support the video mode (and
it's the right one, not something else); there's no guarantee that the
monitor also supports the video mode (neither VBE nor GRUB nor Linux
bother the check the monitors EDID information to make sure). This can
be worse than user can't see anything. For some (very old) VGA only
monitors exceeding the monitor's maximum timing frequencies can cause
the monitor to blow up (but these are quite rare now, for obvious
reasons). I've also heard rumours that for some LCDs built into
cheaper notebook type devices, exceeding the screen's maximum timing
frequencies can cause permanent damage.

Basically, the entire thing only works when the user tells GRUB/Linux
which video mode to use (and then it's the user's fault if they choose
something that doesn't work). It fails for a Live-CD which is meant to
work as is and isn't pre-configured by the user.

The only least likely to fail option would be to use standard VGA
video modes instead - e..g mode 0x12 (320*200*256 colours) or mode
0x13 (640*480*16 colours). These mode numbers are guaranteed (by VGA
compatibility) to be what you asked for, and are also guaranteed to
work with the monitor (by VGA compatibility for older monitors and by
VESA's Safe Mode Timing specification for newer monitors).
Unfortunately they also suck - very low resolution and/or poor colour
depth.

Of course, I'm not saying that you can't unsafely use graphics support
in a Live-CD - I'm sure plenty of people have done worse.



Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: LiveCD bootloader

[Brendan Trotter]

Hi,

On Wed, Sep 8, 2010 at 2:39 PM, Brendan Trotter btrot...@gmail.com wrote:
 GRUB and Linux use video mode numbers to request a video mode. These
 video mode numbers can be different for different video cards. Early
 versions of VBE did define some standard mode numbers, but these
 became obsolete in VBE version 2.0 (back in 1994). Software written
 after 1994 shouldn't rely on these obsolete video mode numbers. For
 example, if you're lucky mode 0x118 might be 1024*768 with 24-BPP, but
 nothing guarantees that anymore, and it could easily be any other
 video mode. There's also no guarantee that the the video card supports
 the video mode (e.g. a lot of video cards only support 32-BPP video
 modes and don't support any 24-BPP video modes; and some only support
 24-BPP video modes and not 32-BPP video modes; and the same is true
 for 15-BPP vs. 16-BPP).

My apologies.

It seems GRUB2 has improved a lot, and doesn't rely on the obsolete
standard video mode numbers anymore. Except for problems caused by
video modes that are supported by the video card but not supported by
the monitor; it's mostly only a problem with Linux itself now.


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: [RFC] Detect other software using embedding area

[Brendan Trotter]

Hi,

On Tue, Aug 31, 2010 at 8:28 PM, Colin Watson cjwat...@ubuntu.com wrote:
 When I blogged about this recently (and rather unexpectedly ended up on
 Slashdot), several people followed up to say that GRUB shouldn't be
 using the embedding area because it was never defined to be used for
 anything in particular and has no protocol for arbitrating among
 conflicts like this.

Unfortunately, where the PC BIOS is concerned a lot of things are de
facto standards, rather than actual standards; but I've always held a
far simpler view: The MBR and the remaining sectors in the first track
(before the start of the first partition) are reserved for the sole
use of the boot manager. If GRUB is the boot manager, then no OS
(and no application running under any OS) has the right to touch these
sectors; and the opposite is also true - e.g. if GRUB is not the MBR
(e.g. GRUB is chain-loaded by something else) then GRUB has no right
to touch any of these sectors.

If applications running on Windows are causing problems (overwriting
something they have no right to touch), then the best solution would
be a public web page (maybe in the GRUB wiki?) that explains the
problem and lists which applications are buggy/broken crap...


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: my thoughts about grub 2

[Brendan Trotter]

Hi,

On Wed, Aug 18, 2010 at 12:35 PM, BVK Chaitanya bvk.gro...@gmail.com wrote:
 If you could list out specific instances of what you felt is
 difficult, then it would be a lot helpful.  A general comment like,
 its not good or difficult does not give any chance for improvements.

If the English language was radically changed, almost everyone that
became accustomed to the current/old version of English would become
confused and frustrated when they're confronted with the new English.
This is relatively easy to understand - radical change means
confusion, relearning/retraining and lost productivity.

The syntax used for GRUB's configuration (and the structure and
location of GRUB's configuration files) isn't really any different -
it's like a special language people use to tell GRUB what they want.
Not only was it radically changed, it became a lot more complex too.

Here's some selected excerpts from the Configuration section of the
(draft?) GRUB2 documentation (from
http://grub.enbug.org/Manual#head-8bd9ce00a71835c5e4a44b1ae459a8de023e55cd
):

- Before you edit some file/entry in /boot/grub that mysteriously disappears
- Configuration for GRUB 2 is much different than the original GRUB.
- A multistep configuration creation process (i.e. a configuration
for creating the configuration) might seem convoluted.
- Certain items embed configuration information in a non-obvious way
- Users must depend on the configurations fed to the utilities that
generate these images to infer what they will try to do on reboot.

Of course this is an intentionally biased and misleading selection of
excerpts intended to illustrate a point of view; but can you imagine
what normal users are thinking when they try to understand GRUB2
configuration for the first time?

To make thing even worse, (if Ubuntu is a reasonable indication)
different OS distributions will add their own non-standard extras to
GRUB2 configuration (and probably add their own non-standard patches
to GRUB2's code too). This will mean that if someone learn to
configure GRUB2 on one OS distribution (with one set of extra
features) then they'll be confused if/when they shift to a different
OS distribution (with a different set of extra features); and if
anyone is silly enough to attempt to install 2 different OS
distributions on the same computer (e.g. dual boot) they'll need to be
extremely careful that the distributions don't fight over the GRUB
configuration (and screw each other up).

In my honest opinion, the design of GRUB2 should have began with the
question What do normal users expect?. As far as I can tell, most
users expect a boot manager to be self contained (e.g. doesn't depend
on any OS for installation or configuration) and includes built-in
configuration tools that are easy to use (e.g. menu driven).

To confirm my suspicions I spent an hour or 2 doing a survey. I
started with a Google search for boot manager, and selected each
entry in the search results for the first page and a half (before
deciding I'd spent enough time) and tried to find out about the
installation and configuration of each different boot manager.

For the following list of boot managers they all have 2 things in
common - they don't require any OS for installation, and all
configuration is done inside the boot manager itself (typically with a
simple menu driven interface):
- SyMon (http://symon.da.ru/)
- GAG (http://gag.sourceforge.net/index.html),
- Plop (http://www.plop.at/en/home.html)
- Master Booter (http://www.masterbooter.com)
- Smart Boot Manager (http://btmgr.sourceforge.net/about.htm)
- System Selector/BootManager
(http://www.bootmanager.com/uebersicht.html.en.html)
- BootIt (http://www.terabyteunlimited.com/bootit-next-generation.htm)
- Multiple Boot Manager (http://elm-chan.org/fsw/mbm/mbm_e.html)
- MATTsoft Boot Manager (http://martin.hinner.info/mbtmgr/)

I only found 3 exceptions. The first is Acronis OS Selector
(http://www.acronis.com/homecomputing/products/diskdirector/os-selector.html)
which is part of a much larger toolkit (backup/recovery, partition
management, etc). It requires a version of Windows for Installation
and configuration. OSL2000 (http://www.osloader.com/) and boot-us
(http://www.boot-us.com/) also require an OS (windows) for
installation and configuration.

The other thing I noticed is that when you it comes to
user-friendliness all of the boot managers (even the ones that
require an OS for installation/configuration) make GRUB2 look like an
extremely overcomplicated mess.


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: my thoughts about grub 2

[Brendan Trotter]

Hi,

On Thu, Aug 19, 2010 at 12:14 AM, Lennart Sorensen
lsore...@csclub.uwaterloo.ca wrote:
 On Wed, Aug 18, 2010 at 04:42:20PM +0930, Brendan Trotter wrote:
 On Wed, Aug 18, 2010 at 12:35 PM, BVK Chaitanya bvk.gro...@gmail.com wrote:
  If you could list out specific instances of what you felt is
  difficult, then it would be a lot helpful.  A general comment like,
  its not good or difficult does not give any chance for improvements.

 If the English language was radically changed, almost everyone that
 became accustomed to the current/old version of English would become
 confused and frustrated when they're confronted with the new English.
 This is relatively easy to understand - radical change means
 confusion, relearning/retraining and lost productivity.

 The syntax used for GRUB's configuration (and the structure and
 location of GRUB's configuration files) isn't really any different -
 it's like a special language people use to tell GRUB what they want.
 Not only was it radically changed, it became a lot more complex too.

 Here's some selected excerpts from the Configuration section of the
 (draft?) GRUB2 documentation (from
 http://grub.enbug.org/Manual#head-8bd9ce00a71835c5e4a44b1ae459a8de023e55cd
 ):

 - Before you edit some file/entry in /boot/grub that mysteriously 
 disappears
 - Configuration for GRUB 2 is much different than the original GRUB.
 - A multistep configuration creation process (i.e. a configuration
 for creating the configuration) might seem convoluted.
 - Certain items embed configuration information in a non-obvious way
 - Users must depend on the configurations fed to the utilities that
 generate these images to infer what they will try to do on reboot.

 Of course this is an intentionally biased and misleading selection of
 excerpts intended to illustrate a point of view; but can you imagine
 what normal users are thinking when they try to understand GRUB2
 configuration for the first time?

 To make thing even worse, (if Ubuntu is a reasonable indication)
 different OS distributions will add their own non-standard extras to
 GRUB2 configuration (and probably add their own non-standard patches
 to GRUB2's code too). This will mean that if someone learn to
 configure GRUB2 on one OS distribution (with one set of extra
 features) then they'll be confused if/when they shift to a different
 OS distribution (with a different set of extra features); and if
 anyone is silly enough to attempt to install 2 different OS
 distributions on the same computer (e.g. dual boot) they'll need to be
 extremely careful that the distributions don't fight over the GRUB
 configuration (and screw each other up).

 That was certainly the case with grub1 as well.  Debian had a patch to
 add 'save default' to grub1, which upstream did not have.  This
 certainly puzzled some people who wondered why it didn't work on other
 distributions.

 In my honest opinion, the design of GRUB2 should have began with the
 question What do normal users expect?. As far as I can tell, most
 users expect a boot manager to be self contained (e.g. doesn't depend
 on any OS for installation or configuration) and includes built-in
 configuration tools that are easy to use (e.g. menu driven).

 No, that's not a good design method.  Starting with 'what should it be
 able to do' is much better (and appears to be what happened).

 grub2 is multi architecture, modular, extendible, and much more robust
 than grub1.  The fact it no longer depends on any block maps to work is
 a great thing.  The fact it uses modules to build the required features
 in and loads any others needed on demand means it can support a lot more
 filesystems than grub1 since grub1 would have gotten too big adding all
 those features.

 Sure grub1's config was simple and the syntax had a lot less in it, but it
 was also limiting the ability to add new features.  Now for debian users,
 they already had an update-grub command that generated the grub config
 file for them, so going to grub2 really doesn't change anything from the
 users point of view, unless they happen to want to custize something.
 Now those customizations happen in /etc not /boot/grub/menu.lst.  That's
 actually a good thing, since all config SHOULD be in /etc, not /boot.
 So grub1 was actually the one that was doing the wrong thing before.

Um, what?

Imagine you've got 3 OSs: Hiaku, FreeDOS and ReactOS. Given that none
of these OSs normally have an /etc directory, which /etc should be
used to store GRUB's configuration?

Perhaps you're saying that GRUB should be useless for anything that
isn't a Unix clone. In that case, imagine you've got 3 Unix clones. Of
course all of them want to automatically update their boot loader's
configuration when their kernel is updated, and they can't all share
the same /etc. Does the user nominate one Unix clone as working
and let the other 2 OSs fail?

Using a separate partition for /boot that contains GRUB's
configuration for all OSs worked (at least

Re: my thoughts about grub 2

[Brendan Trotter]

Hi,

On Thu, Aug 19, 2010 at 3:27 AM, Lennart Sorensen
lsore...@csclub.uwaterloo.ca wrote:
 On Thu, Aug 19, 2010 at 03:18:53AM +0930, Brendan Trotter wrote:
 Um, what?

 Well at least update-grub reads from /etc to generate the final config
 (which is still /boot/grub/grub.cfg, so it does go in /boot, but since
 it is generated (at least on my debian system), I don't consider it
 config anymore).

 Imagine you've got 3 OSs: Hiaku, FreeDOS and ReactOS. Given that none
 of these OSs normally have an /etc directory, which /etc should be
 used to store GRUB's configuration?

 Well whichever one is responsible for generating the grub.cfg could
 store the files wherever is normal on that OS.

 Perhaps you're saying that GRUB should be useless for anything that
 isn't a Unix clone. In that case, imagine you've got 3 Unix clones. Of
 course all of them want to automatically update their boot loader's
 configuration when their kernel is updated, and they can't all share
 the same /etc. Does the user nominate one Unix clone as working
 and let the other 2 OSs fail?

 I honestly don't personally care at all about any OS that isn't a unix
 clone anymore.  Fortunately, I am only a grub user and not one of the
 developers.  They seem to care.

You missed my point - it doesn't work for Unix clones either. You
could even have 2 copies of the exact same OS (something like Ubuntu)
installed on the same computer (in different partitions), and it fails
because both copies of the OS can't share the same /etc. Any changes
to /etc done by one copy won't be seen by the other copy.

 Using a separate partition for /boot that contains GRUB's
 configuration for all OSs worked (at least in theory) because all OSs
 that are installed could mount that partition without conflicts (as
 long as you use a file system that all OSs understand).

 grub2 certainly has no issue with that.  The default is to use grub.cfg
 in the /boot/grub directory.

 I was talking about boot managers, not boot loaders.

 Why should there be a difference?

A few sayings come to mind:
- Write programs that do one thing and do it well (
http://en.wikipedia.org/wiki/Unix_philosophy )
- jack of all trades (
http://en.wikipedia.org/wiki/Jack_of_all_trades,_master_of_none )

A boot manager can be very simple to install and very simple to
configure. You should probably take a look at a few (I created a list
including URLs earlier). A few of them even claimed to have single
click configuration because they automatically detect installed OSs.

A pure boot loader (something without boot manager features, that is
designed specifically for one OS only) typically needs a single
command to install and doesn't need any configuration.

 Conceptually you have a boot manager (to select which OS to boot) that
 doesn't really need to care about any of the details for any
 particular OS; plus a boot loader for each OS which is designed
 specifically for that OS (and doesn't really need to care about other
 file systems, etc). The difference between them often gets blurred
 because feature creep is tempting (for example, a lot of the boot
 managers I looked at earlier had features for creating/removing
 partitions, even though this is normally done using separate utilities
 designed for the purpose, like fdisk, parted, etc; and a lot of boot
 loaders are probably able to chainload).

 GRUB is different in that it's intended to be a boot manager and a
 boot loader for many OSs (and isn't primarily intended for a single
 role); and I'd guess that is the reason it has to be too complex to be
 user friendly for any specific role.

 Almost every x86 boot loader for linux has also been a boot manager
 (through chainloading if nothing else).  Even the ntldr can do that.  It
 seems to me that a boot manager is a stripped down boot loader that
 doens't do very much.  Seems like a completely useless piece of software
 to me.  I don't get it.

SYSLINUX is a collection of pure boot loaders. ELILO is a pure boot
loader too. In the old days so was loadlin. If I remember correctly,
early versions of Linux also had a boot loader built directly into the
kernel (but I think that was only for booting from floppies). GRUB,
LILO, SILO and PALO are a mixture. I can't think of any other Linux
loaders.


Cheers,

Brendan

___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: Multiboot2 Suggestions

[Brendan Trotter]

Hi,

2010/4/10 Vladimir 'φ-coder/phcoder' Serbinenko phco...@gmail.com:
 Making sense out of arbitrary termcap strings isn't easy - it would
 add a large amount of mess to early OS initialisation code (which
 typically doesn't even have C library functions to rely on). A single
 integer saying which protocol is much easier to parse and use,
 especially as only a few standard protocols (e.g. VT100) would need to
 be supported.

 It doesn't need to be termcap string in its normal form. It can be sth like:
 uint16_t offset_to_gotoxy_string;
 uint16_t offset_to_cls_string;
 ...
 uint16_t offset_to_end_of_last_string;
 strings
 This way if OS wants to e.g. clear screen it should have no trouble
 retrieving and replaying string.

That could work (and would be much better than a termcap string in its
normal form).

 For 32-bit MMIO you could use 64-bit MMIO with the high bits of the
 address in I/O space set to zero.

 Do all platforms zero-expand? Or do some sign-extend? if it depends only
 on ISA there is no problem in just saying if it's zero or sign-extended
 depending on platform. Perhaps like
 0 = you can access even if you're not 64-bit aware
 1 = you need to be 64-bit aware

I've never seen an architecture where physical addresses are
interpreted as signed (instead of unsigned).

 Another field I forgot which will go instead of 16-bit padding is flags.
 First flag will be CONSOLE_ACTIVE which will indicate that console was
 used by bootloader. This way bootloader can inform OS of additional
 serial ports which it didn't use.

That's a good idea. :-)

 I currently use PC speaker as my critical error notification
 method - it's about 15 instructions that use I/O ports only and
 doesn't require memory allocations or anything else. I doubt setting
 keyboard LEDs (for a PS/2 keyboard) would be much larger or rely on
 anything more than I/O ports.

 In comparison, my video setup code is around 64 KiB of code that
 starts with trying to get EDID information from the monitor, filtering
 a list of video modes (from VGA and/or VBE), allocating several MiB of
 RAM for video buffers and font data, scaling font data, etc. If
 there's a problem setting up the memory management it's all useless
 and I fall back to the critical error notification method so the
 user knows the OS failed to initialise something (e.g. couldn't
 allocate several MiB of RAM).

 If you use frmaebuffer info in mbi you can have basic video much faster.

I'm not planning to have any support for basic video (only eye
candy video). I don't support text modes either (and have no desire
to add support for text modes).

I'm also wondering if a primary monitor's EDID tag could be added to
the multi-boot information. This tag would optional - if the boot
loader can't get the information, or if the boot loader doesn't even
try to get the information, then the boot loader can skip the EDID tag
(but the EDID information is easy to obtain from VBE and U/EFI if
you're writing firmware specific code).

Where possible, I currently use the EDID information to determine the
physical size of the monitor (e.g. 520 mm wide and 320 mm high), and
then scale font data, etc to suit; so that everything is the same
shape regardless of the monitor's aspect ratio, so that things aren't
too small on a small screen or too big on a large screen, and so that
everything looks the same in all video modes (resolution independent).
For an example, for a 1600*1200 video mode on a small 4:3 monitor I
might end up with 32*42 characters, for a 800*600 video mode on the
same small 4:3 monitor I might end up with 16*21 characters, and for a
800*600 video mode on a large 16:9 monitor I might end up with 8*19
characters; and in all of these cases I can draw a square box (that
doesn't look like a rectangle in any case).

 Alternatively, (for my OS) for headless systems; I use RTS/CTS and the
 VT100 identify command to detect if anything is listening on the
 serial port (and if it's a terminal or something else). When nothing
 is listening on the other end the OS can't talk so it uses the PC
 speaker as a fallback (but continues monitoring the serial port).
 Basically if something goes wrong at any stage, the OS beeps, and the
 user can plug a terminal in afterwards to find out what went wrong
 (rather than having no idea what caused the problem, then connecting a
 terminal and rebooting to see if it happens again while they are
 watching).

 I feel like here it's not anymore about present hardware or its state
 but about user configuration. Generally for this type of parameters
 command line is better suited.

It's about what does the OS do when it needs to tell the user there's
been a problem but can't talk to the user using the normal console/s
for any reason (regardless of what the normal console/s are and
regardless of what the reason/s may be).

 What's the difference between type 2 and type 5

 For a server environment, would your OS automatically send an email to
 an administrator 

Re: Multiboot2 Suggestions

[Brendan Trotter]

Hi,

2010/4/3 Vladimir 'φ-coder/phcoder' Serbinenko phco...@gmail.com:
 Brendan Trotter wrote:
 1) If GRUB was using a serial port as a console device (e.g. on a
 headless system) it'd be nice if the OS could continue using the same
 serial port with the same configuration instead of resetting the
 serial port, etc. A new tag (for 80x86, 8250/6550 compatible serial
 ports) would include base I/O port, the serial line configuration
 (e.g. 8 data bits, no stop bit, no parity), the baud rate (e.g. 9600),

 The kernel can read data bits, stop bits, parity and divisor from
 registers itself. I think it's more useful to supply a base frequency
 since there are a lot of almost compatible cards which differ only in
 base frequency.

You're right.

 the IRQ number (if known/used by the boot loader) and the protocol
 being used (ASCII, VT100, etc).
 I think it's more useful to supply directly usable strings termcap
 strings rather than an abstract ID

Here's an example termcap string:

ca|concept100|c100|concept|c104|concept100-4p|HDS Concept-100:\
 :al=3*\E^R:am:bl=^G:cd=16*\E^C:ce=16\E^U:cl=2*^L:cm=\Ea%+ %+ :\
 
:co#80:.cr=9^M:db:dc=16\E^A:dl=3*\E^B:do=^J:ei=\E\200:eo:im=\E^P:in:\
 :ip=16*:is=\EU\Ef\E7\E5\E8\El\ENH\EK\E\200\Eo\200\Eo\47\E:k1=\E5:\
 :k2=\E6:k3=\E7:kb=^h:kd=\E:ke=\Ex:kh=\E?:kl=\E:kr=\E=:ks=\EX:\
 :ku=\E;:le=^H:li#24:mb=\EC:me=\EN\200:mh=\EE:mi:mk=\EH:mp=\EI:\
 :mr=\ED:nd=\E=:pb#9600:rp=0.2*\Er%.%+ :se=\Ed\Ee:sf=^J:so=\EE\ED:\
 :.ta=8\t:te=\Ev\200\200\200\200\200\200\Ep\r\n:\
 :ti=\EU\Ev  8p\Ep\r:ue=\Eg:ul:up=\E;:us=\EG:\
 
:vb=\Ek\200\200\200\200\200\200\200\200\200\200\200\200\200\200\EK:\
 :ve=\Ew:vs=\EW:vt#8:xn:\
 :bs:cr=^M:dC#9:dT#8:nl=^J:ta=^I:pt:

Making sense out of arbitrary termcap strings isn't easy - it would
add a large amount of mess to early OS initialisation code (which
typically doesn't even have C library functions to rely on). A single
integer saying which protocol is much easier to parse and use,
especially as only a few standard protocols (e.g. VT100) would need to
be supported.

  If the boot loader doesn't know which
 IRQ the serial port uses (e.g. it uses polling) then it sets the IRQ
 number to 0x.

 Where the serial port has a different interface
 (e.g. if the serial port uses MMIO or if it's not 8250/6550
 compatible) a different tag with different fields are used to
 describe it.
 I think it's more useful to have an I/O selector since yeeloong serial 
 interface is basically the same, just attached differently

An I/O selector (for 8250/6550 compatible) makes sense, and
different tags for not 8250/6550 compatible serial ports.

 I think we need following fields:
 1) I/O space selector (e.g. 0 = 32-bit MMIO, 1 = 64-bit MMIO, 2 = i386 I/O)

For 32-bit MMIO you could use 64-bit MMIO with the high bits of the
address in I/O space set to zero.

 2) IRQ type selector (0 = None, 1 = standard platform interrupt)
 3) 16-bit padding
 4) address in I/O space (up to 64-bits)
 5) Base frequency in Hz (32-bit)
 6) IRQ (32-bit or even 64-bit since it's not excluded some platforms would 
 use 64-bit IRQ ids, though I'm not aware of such).

Looks good to me.

 telnet+ethernet
 It would need network tag first

Just an example of a different tag for a different type of console device.

 2) The OS image format information should be expanded, so that the
 OS can tell the boot loader if it supports 8250/6550 compatible
 serial ports (and which protocols), and any other console devices (for
 the same reason the OS image format already has flags, etc for
 video).

 Console flags tag is for this

Yes - the console flags tag needs to be expanded...

 3) There should be an (optional?) critical error notification method
 tag that tells the OS which method/s it can/should use to tell the
 user it encountered a problem before it was able to setup it's console
 output. For example, can/should the OS return to the boot loader, or
 use the PC speaker to beep, or make a PS/2 keyboard's LEDs flash, or
 something else.

 Processing such a selector may prove as difficult as setting up a
 console based on console tags. So I doubt its usefullness

I currently use PC speaker as my critical error notification
method - it's about 15 instructions that use I/O ports only and
doesn't require memory allocations or anything else. I doubt setting
keyboard LEDs (for a PS/2 keyboard) would be much larger or rely on
anything more than I/O ports.

In comparison, my video setup code is around 64 KiB of code that
starts with trying to get EDID information from the monitor, filtering
a list of video modes (from VGA and/or VBE), allocating several MiB of
RAM for video buffers and font data, scaling font data, etc. If
there's a problem setting up the memory management it's all useless
and I fall back to the critical error notification method so the
user knows the OS failed to initialise

Multiboot2 Suggestions

[Brendan Trotter]

Hi,

2010/3/28 Vladimir 'φ-coder/phcoder' Serbinenko phco...@gmail.com:
 Also I'm aware that at least some people want more tags. Feel free to
 propose new ones.
 In short all ammendment ideas are welcome.

Here's my list.. :-)

1) If GRUB was using a serial port as a console device (e.g. on a
headless system) it'd be nice if the OS could continue using the same
serial port with the same configuration instead of resetting the
serial port, etc. A new tag (for 80x86, 8250/6550 compatible serial
ports) would include base I/O port, the serial line configuration
(e.g. 8 data bits, no stop bit, no parity), the baud rate (e.g. 9600),
the IRQ number (if known/used by the boot loader) and the protocol
being used (ASCII, VT100, etc). If the boot loader doesn't know which
IRQ the serial port uses (e.g. it uses polling) then it sets the IRQ
number to 0x. Where the serial port has a different interface
(e.g. if the serial port uses MMIO or if it's not 8250/6550
compatible) a different tag with different fields are used to
describe it. The same idea should be used for any other console device
in future. For example, if a boot loader supports parallel ports or
telnet+ethernet or something in future then a new tag describing it
is created; and if a boot loader supports new protocols for existing
devices then they should added to the documentation (e.g. if a boot
loader supports a Braille terminal connected to serial port, then a
new protocol type is added to the existing serial port tags).

2) The OS image format information should be expanded, so that the
OS can tell the boot loader if it supports 8250/6550 compatible
serial ports (and which protocols), and any other console devices (for
the same reason the OS image format already has flags, etc for
video).

3) There should be an (optional?) critical error notification method
tag that tells the OS which method/s it can/should use to tell the
user it encountered a problem before it was able to setup it's console
output. For example, can/should the OS return to the boot loader, or
use the PC speaker to beep, or make a PS/2 keyboard's LEDs flash, or
something else.

4) The section on Machine State is missing lots of information, and
needs to indicate the state of *all* hardware on all architectures
(regardles of firmware type). For example; for 80x86/PC it should say
that PCI devices are left in an undefined state (so that the boot
loader is not responsible for configuring PCI devices if the firmware
didn't for any reason), except for any PCI device that is indirectly
mentioned in the multi-boot information (e.g. if there's a serial port
tag then the OS can assume that serial port is configured, if there's
a video information tag then the OS can assume the video cards is
configured, etc).

5) The boot loader should always provide a memory map. If the boot
loader is unable to get a memory map from the firmware then the boot
loader constructs a fake memory map from any/all information it can
find, including known areas that aren't RAM. For e.g. on 80x86 BIOS
if int 0x15, eax=0xE820 isn't supported, then other BIOS functions
are used to find usable RAM and the boot loader creates an entry that
marks the area from 0x000A to 0x000F as system or unknown.
The mem_upper/mem_lower tag should removed from the specification.

6) For any areas that aren't mentioned in the memory map, the OS can
assume these areas are safe to use when configuring memory mapped PCI
device's BARs. For 80x86 this means that if int 0x15, eax=0xE820
leaves holes in the area from 0x000A to 0x000F (which is
almost always the case) then the boot loader creates entries that
describe these areas as unknown, so that the area/s are mentioned in
the memory map and the OS doesn't assume the area can be used for PCI
devices.

7) The memory map needs more area types. Any RAM that is reported by
firmware as faulty should use area type 5 so that the OS can know
that some RAM is faulty, and (for e.g.) could tell a system
administrator about it. Also, area type 0x should be used
for unknown, as this allows the OS to determine the difference
between boot loader doesn't know what the area is and boot loader
does know what the area is but the area type is defined in a newer
version of the multi-boot specification.

8) Any RAM that is not immediately usable by the OS should not be
reported as usable RAM in the memory map. An example of this is the
ACPI reclaimable area (which is RAM that isn't usable until the OS
has finished using the ACPI tables). RAM used to store the multi-boot
information, RAM used to store the kernel and RAM used to store any
modules should be treated in the same way. I suggest using area type
0xFFFE = RAM used for multi-boot information and area type
0xFFFD = RAM used for kernel/modules. This makes it much easier
to write a kernel's early initialisation code, because it can use RAM
without worrying about trashing data that is needed by the kernel/OS
later.

9) 

Re: help ragarding Multiprocessor(MP) Configuration Table

[Brendan Trotter]

Hi,

On Thu, Dec 24, 2009 at 1:09 AM, mandar naik mandar...@yahoo.co.in wrote:
 So we wanted to know how  what MP information is made available to grub on
 Intel core2duo architecture if MP Floating Point structure  MP
 configuration table are not present.
 If Intel core2duo architecture is one of default configuration type then is
 it possible to modify smp-imps.c file to use default configuration settings.
 Also we would like to know if there is similar SMP support provided in GRUB
 1.97.
 Thanking in anticipation,

The default configurations mentioned in Intel's Multi-Processor
Specification haven't been used for over 15 years.

About a decade ago everything started shifting to ACPI, and in the
last few years I've seen a few systems without Intel's (much older) MP
Specification tables or with dummy/incomplete MP specification tables
(although they are still rare - almost all modern computers have both
MP and ACPI). The ACPI tables include a Multiple APIC Description
Table which tells you everything that the MP Tables did, and modern
software should check for ACPI tables and use them (and should only
use MP Specification tables if no ACPI tables are found).

Modifying GRUB to suit your specific hardware is a bad idea
(especially when the problem is your software and not the hardware,
firmware or GRUB); and learning how to make software that doesn't work
on other computers and stops working when you change some of the BIOS
settings is also a bad idea.

Use the ACPI tables.


Cheers,

Brendan


___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: Using network informations from PXE as grub2 enviroment variables

[Brendan Trotter]

Hi,

On Sat, Oct 17, 2009 at 4:42 AM, Pavel Pisa p...@cmp.felk.cvut.cz wrote:
 As for scalability, I agree, that use of single config file with checking
 for each IP is horrible hack, but there is no problem to source
 to another IP specific file (menu-xxx.xxx.xxx.xxx.lst) with separate
 menu configuration when own IP is known and if file is not found
 switch do default one.

 So I strongly vote for a way to be able to use server IP provided
 by DHCP/PXE.

I vote for using the ethernet card's MAC address (which never changes)
instead of whatever IP address the DHCP server felt like *dynamically*
assigning...

GRUB could (should?) download a configuration file from the TFTP
server that's called 123456789ABC.cfg or 123456789ABC/grub.cfg
(where 123456789ABC is the ethernet card's MAC address) ; and if
that's not found it should try something like default.cfg.

Putting the ethernet card's MAC address and the current IP address
into an environment variable (e.g. for use in command line parameters)
might be fun too.

Note 1: the easiest way to find the ethernet card's MAC address and
the computer's IP address is to use the GET_CACHED_INFO function
(opcode 7), with the Packet type field set to 2 (DHCP ACK from
server); because you need to use this function to find the IP address
of the TFTP server anyway.

Note 2: the name your_ip probably came from the PXE specification.


Cheers,

Brendan


___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: Handling multiple cores by GRUB

[Brendan Trotter]

Hi,

On Mon, Oct 12, 2009 at 10:51 PM, Swapnil Pimpale
swapnil.p...@gmail.com wrote:
 Actually we are doing an academic project in which we need to have two cores
 of Intel Core 2 Duo, running inside the GRUB simultaneously. For that we
 need to send an IPI from primary core to the secondary core.  Can you
 suggest any pointers to implement this? Has someone done anything similar to
 this before?

Is there any sane reason for this? None of GRUB's code is designed for
multiple CPUs, and I'd assume that no firmware is designed to handle
multiple CPUs either (I know that PC BIOS isn't), and I expect that
if you wasted your time and got multiple CPUs running inside GRUB then
one of the CPUs would do everything while any extra CPU/s would do
nothing (until an OS is started and you need to shut down the do
nothing CPU/s).

The method for starting extra CPUs (for 80x86 only) is:
- find the ACPI APIC table and get the address of the local APIC and
the APIC ID for the CPU/s you want to start
- if there's no ACPI, use the Multi-Processor Specification tables
to find the address of the local APIC and the APIC ID for the CPU/s
you want to start (you shouldn't need to do this on modern systems)
- get the local APIC version (from either ACPI, MP spec or directly
from the BSP's local APIC) and determine what type of APIC it is
- (for xAPIC only) send an INIT IPI
- (for xAPIC only) wait 10 ms
- (for xAPIC only) send a SIPI IPI
- (for xAPIC only) wait for up to 200 us for the other CPU to set a
I started flag
- (for xAPIC only) if the I started flag wasn't set in time, send
another SIPI IPI
- (for xAPIC only) if the I started flag wasn't set in time, wait
for a while (at least 200 s, but typically several seconds for the
other CPU to set a I started flag

The SIPI IPI includes the address that you want the other CPU to start
executing. This needs to be on a page boundary and must be below
0x0010. The other CPU will start in real mode, and one of the
first things it should do is set some sort of I started flag (to
tell the first CPU that it can stop waiting and that there hasn't been
some sort of CPU failed to start error).

For different types of local APIC there's variations, but you probably
don't need to care about the old 82489DX local APICs (only used on
80486 and early Pentium); and the new x2APIC is compatible with the
xAPIC (unless there's a very large number of CPUs present and the
APIC ID for the CPU/s you want to start is above 0x00FF).

For reference materials, see Intel's System Programming Guide, Intel's
Multi-Processor Specification (even if you're using ACPI), the ACPI
specification, and generic OS development resources (e.g.
http://www.osdev.org/, the #osdev IRC channel, etc).


Cheers,

Brendan


___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: About firmware facilities

[Brendan Trotter]

Hi,

On Sat, Sep 19, 2009 at 11:36 PM, Vladimir 'phcoder' Serbinenko
phco...@gmail.com wrote:
 Brendan Trotter wrote:
 No. Usuable means only that firmware isn't destroyed. Any device may
 be in a different state

 Any device (that the firmware assumes is in a certain state) may be
 left in a different state (that the firmware no longer knows about)?

 For a very simple example, imagine if the BIOS leaves the floppy motor
 on, and GRUB's own floppy driver uses the floppy and then turns the
 motor off. Then the OS uses the firmware to read from floppy, but the
 firmware thinks the floppy motor is still on and attempts to read from
 the floppy without turning the floppy motor on.

 If GRUB has it's own device drivers, and GRUB doesn't restore devices
 to the state that the firmware expects the devices to be in, then the
 firmware is unusable.

 Most OSes should use their own drivers to access devices.

Most device manufacturers should provide full documentation so that
programmers can write drivers to access the devices; and manufacturers
should provide hardware samples (and documentation) to these
programmers so that the device driver is ready before the device is
made available to the general public. Unfortunately the real world
just doesn't work the same as should.

The multi-boot specification says the firmware is left in a usable
state. If GRUB doesn't leave the firmware in a usable state, then
either GRUB is wrong or the multi-boot specification is wrong. You
can't have it both ways.

Of course I'm forgetting that GRUB also supports chainloading (e.g.
the chainloaded OS tries to use the firmware to load more of it's
data, and the firmware fails because GRUB left a device in an
unexpected state) - non-compliance with the multi-boot specification
isn't the only issue.

 If an OS can't use the firmware, then the OS must rely on GRUB for
 everything instead, including strange OS specific things that nobody
 has seen any other OS do before.

 If nobody uses a particular feature in firmware then you shouldn't use
 it either. Unused firmware features are often buggy. Moreover firmware
 on x86 is useful only for bootstrap and once bootstrap is completed you
 should forget it exists except some firmware-specific tasks as setting
 boot device.

So, can I rely on GRUB to (for e.g.) setup video in a way that is
suitable for my code, or do I need to use the firmware myself (and
hope that GRUB hasn't left a device in an unexpected state)?

 Due to limitations in the original multi-boot specification my code
 switches back to real mode and uses the BIOS to do memory detection,
 do video mode detection, switch video modes and gather other
 information.

 Have you actually read the multiboot specification? Booter passes info
 about memory and video mode in mbi (video for multiboot isn't
 implemented yet). If you need firmware for basic bootup you're clearly
 doing something wrong and are firmware-dependent. Of course it's your
 freedom to make suboptimal software.

 I've read the multi-boot specification. I've also read the code in
 GRUB-legacy that does memory detection, and I'm unwilling to allow my
 code to rely on it for quality control reasons. Without going into
 details, GRUB-legacy tends to do a minimal job and then expects the
 user to fix the problem if/when it goes wrong (but even then it only
 offers a uppermem command without providing a way for the user to
 specify a complete system memory map).


 What is minimal job and quality control? We use standard
 E820+(optionally)badram command. I've seen no OS do any more than
 this.


 My code tries int 0x15, eax=0xE820 expecting 24 bytes per area (ACPI
 3.0); then it tries int 0x15, eax=0xE820 expecting 20 bytes per
 area. If int 0x15, eax=0xE820 isn't supported by the BIOS then you
 can assume it's an old computer (and old computers are painful).

 It tries int 0x15, ax=0xE801, then int 0x15, ah=0xC7, then int
 0x15, ah=0x8A, then int 0x15, ah=0xDA88, then int 0x15, ah=0x88,
 then CMOS locations 0x70 and 0x71.
 Read code. GRUB fallback to old methods if newer aren't available.

I read the code (for both GRUB 1.96 and GRUB 0.97) and wrote down
exactly which BIOS functions GRUB does use in my last post. You didn't
read the code (and didn't read what I wrote either), and now you're
telling me to read the code?

  If all of this fails (which does
 happen on some computers) then it does manual probing.

 Some of the old BIOS functions have limited range - for example
 they'll return number of KiB blocks at 0x0010 as a 16-bit
 integer, and can't return more than 64 MiB. In this case my code won't
 know if the value returned by the BIOS has been limited to 0x, so
 it'll do manual probing to detect any more RAM above the reported
 area. Some computers have an ISA hole from 0x00F0 to 0x00FF.
 Because of this all older BIOS functions that report amount of RAM at
 0x0010 may return 14 MiB (from 0x0010 to 0x00EF) when
 there's actually

Re: About firmware facilities

[Brendan Trotter]

Hi,

On Tue, Sep 15, 2009 at 6:29 PM, Vladimir 'phcoder' Serbinenko
phco...@gmail.com wrote:
 On Tue, Sep 15, 2009 at 1:23 AM, Brendan Trotter btrot...@gmail.com wrote:
 On Tue, Sep 15, 2009 at 6:19 AM, Vladimir 'phcoder' Serbinenko
 phco...@gmail.com wrote:
 On Mon, Sep 14, 2009 at 10:12 PM, Brendan Trotter btrot...@gmail.com 
 wrote:
 On Tue, Sep 15, 2009 at 4:41 AM, Pavel Roskin pro...@gnu.org wrote:
 On Tue, 2009-09-15 at 04:27 +0930, Brendan Trotter wrote:

 GRUB can serve as BIOS together with Coreboot.

 I know. It'll break my code.

 The multi-boot specification says However, other machine state should
 be left by the boot loader in normal working order, i.e. as
 initialized by the bios (or DOS, if that's what the boot loader runs
 from).; although I seem to remember it saying words to the effect of
 the firmware should be left in a usable state.

 Firmware if present is left in usable state. However it may simply not
 be present.

 So if the firmware is present, GRUB won't alter the state that the
 firmware left the hardware in (or if it does it'll restore all
 hardware to the default firmware state before starting the OS);
 including all devices that GRUB uses via. it's own device drivers
 (except for specific things mentioned in the multi-boot specification,
 like the A20 gate)?

 No. Usuable means only that firmware isn't destroyed. Any device may
 be in a different state

Any device (that the firmware assumes is in a certain state) may be
left in a different state (that the firmware no longer knows about)?

For a very simple example, imagine if the BIOS leaves the floppy motor
on, and GRUB's own floppy driver uses the floppy and then turns the
motor off. Then the OS uses the firmware to read from floppy, but the
firmware thinks the floppy motor is still on and attempts to read from
the floppy without turning the floppy motor on.

If GRUB has it's own device drivers, and GRUB doesn't restore devices
to the state that the firmware expects the devices to be in, then the
firmware is unusable.

If an OS can't use the firmware, then the OS must rely on GRUB for
everything instead, including strange OS specific things that nobody
has seen any other OS do before.

 Due to limitations in the original multi-boot specification my code
 switches back to real mode and uses the BIOS to do memory detection,
 do video mode detection, switch video modes and gather other
 information.

 Have you actually read the multiboot specification? Booter passes info
 about memory and video mode in mbi (video for multiboot isn't
 implemented yet). If you need firmware for basic bootup you're clearly
 doing something wrong and are firmware-dependent. Of course it's your
 freedom to make suboptimal software.

 I've read the multi-boot specification. I've also read the code in
 GRUB-legacy that does memory detection, and I'm unwilling to allow my
 code to rely on it for quality control reasons. Without going into
 details, GRUB-legacy tends to do a minimal job and then expects the
 user to fix the problem if/when it goes wrong (but even then it only
 offers a uppermem command without providing a way for the user to
 specify a complete system memory map).

 What is minimal job and quality control? We use standard
 E820+(optionally)badram command. I've seen no OS do any more than
 this.

My code tries int 0x15, eax=0xE820 expecting 24 bytes per area (ACPI
3.0); then it tries int 0x15, eax=0xE820 expecting 20 bytes per
area. If int 0x15, eax=0xE820 isn't supported by the BIOS then you
can assume it's an old computer (and old computers are painful).

It tries int 0x15, ax=0xE801, then int 0x15, ah=0xC7, then int
0x15, ah=0x8A, then int 0x15, ah=0xDA88, then int 0x15, ah=0x88,
then CMOS locations 0x70 and 0x71. If all of this fails (which does
happen on some computers) then it does manual probing.

Some of the old BIOS functions have limited range - for example
they'll return number of KiB blocks at 0x0010 as a 16-bit
integer, and can't return more than 64 MiB. In this case my code won't
know if the value returned by the BIOS has been limited to 0x, so
it'll do manual probing to detect any more RAM above the reported
area. Some computers have an ISA hole from 0x00F0 to 0x00FF.
Because of this all older BIOS functions that report amount of RAM at
0x0010 may return 14 MiB (from 0x0010 to 0x00EF) when
there's actually another area of RAM at 0x0100. In this case my
code won't know if the value returned by the BIOS is right or not, and
it'll do manual probing to detect any more RAM at 0x0010. If my
code has to do manual probing, then it assumes there's an ISA hole
from 0x00F0 to 0x00FF (regardless of whether there is or not)
as this hole was used for memory mapped video cards (which would seem
like RAM).

For all BIOS functions used my code avoids all known BIOS bugs (and
there's plenty of them). This includes sanitizing the data returned
from int 0x15, eax=0xE820 - sorting the list

Re: About firmware facilities

[Brendan Trotter]

Hi,

On Tue, Sep 15, 2009 at 1:02 AM, Robert Millan r...@aybabtu.com wrote:
 Well, you have the freedom to disagree with anything we do and bring your
 customized GRUB to a different direction :-)

 Anyhow, my priority for GRUB is strong driver-based support.  We could recruit
 someone to develop the framework in next year's GSoC (unless somebody steps
 in, of course).

Why stop there?

If proprietory ethernet ROMs aren't good enough, then what about
proprietory SCSI ROMs, and proprietory firmware/BIOS?

Surely a boot manager wouldn't be a good manager if it didn't include
it's own replacements for all of these things; and perhaps it should
also include it's own replacement for proprietory OS's too...

Why are you worrying about such silly things when the multi-boot
specification (which actually is relevant) is still severely borked?

  The DRM opt-in fallacy: Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all.

Sigh. I think I understand now - lack of logical thinking leads to
lack of rational behavior.

Our data belongs to us. We will decide when (and how) other people
may access our data (potentially including allowing them to lease it
under terms and conditions that include DRM, if they agree to these
terms and conditions)


Cheers,

Brendan


___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: About firmware facilities

[Brendan Trotter]

Hi,

On Tue, Sep 15, 2009 at 4:41 AM, Pavel Roskin pro...@gnu.org wrote:
 On Tue, 2009-09-15 at 04:27 +0930, Brendan Trotter wrote:
 Hi,

 On Tue, Sep 15, 2009 at 1:02 AM, Robert Millan r...@aybabtu.com wrote:
  Well, you have the freedom to disagree with anything we do and bring your
  customized GRUB to a different direction :-)
 
  Anyhow, my priority for GRUB is strong driver-based support.  We could 
  recruit
  someone to develop the framework in next year's GSoC (unless somebody steps
  in, of course).

 Why stop there?

 If proprietory ethernet ROMs aren't good enough, then what about
 proprietory SCSI ROMs, and proprietory firmware/BIOS?

 We are already doing it.  There is functional ATA support, USB support
 is under development.

But, are you doing it for valid technical reasons?

 GRUB can serve as BIOS together with Coreboot.

I know. It'll break my code.

The multi-boot specification says However, other machine state should
be left by the boot loader in normal working order, i.e. as
initialized by the bios (or DOS, if that's what the boot loader runs
from).; although I seem to remember it saying words to the effect of
the firmware should be left in a usable state.

Due to limitations in the original multi-boot specification my code
switches back to real mode and uses the BIOS to do memory detection,
do video mode detection, switch video modes and gather other
information. If GRUB is running on coreboot or UEFI I can't do this
(and can't work around the limitations any other way). Unless the
multi-boot specification is changed significantly (such that a
multi-boot compliant code needs no work-arounds, or at least so that
multi-boot compliant code can determine what sort of firmware there
was and use the firmware) then GRUB as a whole becomes useless to me.

The current draft (http://grub.enbug.org/MultibootDraft) is a small
step in the right direction; but it has changed much in 3 years.

 Why are you worrying about such silly things when the multi-boot
 specification (which actually is relevant) is still severely borked?

 Patches are welcome.

I'm not sure it's possible to write patches for a multi-boot compliant
boot loader without a specification that defines multi-boot
compliant.


Cheers,

Brendan


___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: About firmware facilities

[Brendan Trotter]

Hi,

On Tue, Sep 15, 2009 at 6:19 AM, Vladimir 'phcoder' Serbinenko
phco...@gmail.com wrote:
 On Mon, Sep 14, 2009 at 10:12 PM, Brendan Trotter btrot...@gmail.com wrote:
 On Tue, Sep 15, 2009 at 4:41 AM, Pavel Roskin pro...@gnu.org wrote:
 On Tue, 2009-09-15 at 04:27 +0930, Brendan Trotter wrote:
 We are already doing it.  There is functional ATA support, USB support
 is under development.

 But, are you doing it for valid technical reasons?

 Yes. I have a borked firmware right on this laptop.

Sounds like you need better firmware - Coreboot might be a good
alternative. I assume GRUB is meant to be a boot loader though (rather
than a project to rewrite every piece of dodgy software ever written).

 GRUB can serve as BIOS together with Coreboot.

 I know. It'll break my code.

 The multi-boot specification says However, other machine state should
 be left by the boot loader in normal working order, i.e. as
 initialized by the bios (or DOS, if that's what the boot loader runs
 from).; although I seem to remember it saying words to the effect of
 the firmware should be left in a usable state.

 Firmware if present is left in usable state. However it may simply not
 be present.

So if the firmware is present, GRUB won't alter the state that the
firmware left the hardware in (or if it does it'll restore all
hardware to the default firmware state before starting the OS);
including all devices that GRUB uses via. it's own device drivers
(except for specific things mentioned in the multi-boot specification,
like the A20 gate)?

 Due to limitations in the original multi-boot specification my code
 switches back to real mode and uses the BIOS to do memory detection,
 do video mode detection, switch video modes and gather other
 information.

 Have you actually read the multiboot specification? Booter passes info
 about memory and video mode in mbi (video for multiboot isn't
 implemented yet). If you need firmware for basic bootup you're clearly
 doing something wrong and are firmware-dependent. Of course it's your
 freedom to make suboptimal software.

I've read the multi-boot specification. I've also read the code in
GRUB-legacy that does memory detection, and I'm unwilling to allow my
code to rely on it for quality control reasons. Without going into
details, GRUB-legacy tends to do a minimal job and then expects the
user to fix the problem if/when it goes wrong (but even then it only
offers a uppermem command without providing a way for the user to
specify a complete system memory map).

For video, I believe there's patches to make GRUB-legacy comply with
the original multi-boot specification (but for dual boot setups I
can't expect people to replace their main OS's existing GRUB with a
patched version). Even if all versions of GRUB did support the video
stuff in multi-boot, the multi-boot specification is still
ridiculously inadequate.

My code gives the user a choice of any video modes that are supported
by the video card and my code (and doesn't allow them to choose video
modes that aren't supported by my code), including double scanned
modes (if supported by the video card) and a variety of refresh rates
(if supported by the video card). It also uses the monitor's EDID
information to try to prevent video modes that are supported by the
video card but not supported by the monitor from being used. This is
mostly automated, so that (for e.g.) the user can replace the monitor
with another monitor that doesn't support the video mode that was
previously used, and the software will find the best video mode that
the new monitor does support without the user changing anything.

 If GRUB is running on coreboot or UEFI I can't do this
 (and can't work around the limitations any other way). Unless the
 multi-boot specification is changed significantly (such that a
 multi-boot compliant code needs no work-arounds, or at least so that
 multi-boot compliant code can determine what sort of firmware there
 was and use the firmware) then GRUB as a whole becomes useless to me.

 Just say in clean text what changes you need so we can discuss it.
 Don't expect us to understand the statement your standard is borked,
 change it.

Fair enough...

The video support is already mentioned above.

It'd be nice if other output methods were exposed to the OS too. For
example, if GRUB is configured to use a serial terminal it'd be good
if it told the OS details for the serial port, so that the OS can
continue using the same serial terminal without resetting/restarting
an established connection or requiring the user to configure it in 2
separate places (in GRUB and in the OS).

The same would apply to internationalization - GRUB should tell the OS
which language/locale it used, so that the OS can automatically use
the same language as a default (especially for OS installation CDs).

For memory detection, ACPI 3.0 allows the BIOS ( INT 15H, E820H) to
return extended attributes - mostly only a volatile/non-volatile flag.
This isn't in GRUB's

Re: About firmware facilities

[Brendan Trotter]

Hi,

On Tue, Sep 15, 2009 at 9:13 AM, Colin Watson cjwat...@ubuntu.com wrote:
 I don't have relevant experience regarding multiboot in general, but ...

 On Tue, Sep 15, 2009 at 08:53:12AM +0930, Brendan Trotter wrote:
 The same would apply to internationalization - GRUB should tell the OS
 which language/locale it used, so that the OS can automatically use
 the same language as a default (especially for OS installation CDs).

 Speaking as an OS installer developer, I'd actively prefer GRUB not to
 try to do this via specialised methods like multiboot. I don't want to
 have to maintain language lists and what-have-you in yet another place,
 cope with all the usual slight variations (e.g. things like pt_BR, or
 God-help-us something like the hopelessly inadequate language list in
 the USB keyboard specification), and cope with the fact that there are
 still more boot loaders out there than just GRUB and so some other
 mechanism needs to be maintained anyway.

An OS could choose to use the information provided by GRUB, or choose
to ignore the information provided by GRUB.

Assuming GRUB implements something that isn't hopelessly inadequate,
I'd prefer to attempt to use the information provided by GRUB (and
then have a fall-back). However, I may be a little unusual, in that
I'm willing to do a lot of extra work to avoid a little unnecessary
end-user configuration.

With this in mind, as an OS installer developer, what form of language
identifier would you consider the 'least hopelessly inadequate? What
if GRUB used the exact same language identifiers that (for e.g.)
Ubuntu already uses?



Cheers,

Brendan


___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel

Re: About firmware facilities

[Brendan Trotter]

Hi,

On Tue, Sep 15, 2009 at 9:58 AM, Colin Watson cjwat...@ubuntu.com wrote:
 On Tue, Sep 15, 2009 at 09:26:55AM +0930, Brendan Trotter wrote:
 An OS could choose to use the information provided by GRUB, or choose
 to ignore the information provided by GRUB.

 Sure, but the less code that needs to go in the boot loader the better.

 Assuming GRUB implements something that isn't hopelessly inadequate,
 I'd prefer to attempt to use the information provided by GRUB (and
 then have a fall-back). However, I may be a little unusual, in that
 I'm willing to do a lot of extra work to avoid a little unnecessary
 end-user configuration.

 Don't get me wrong, I wasn't suggesting end-user configuration - I'd
 rather have it in grub-mkconfig than in the core, that's all.

 With this in mind, as an OS installer developer, what form of language
 identifier would you consider the 'least hopelessly inadequate? What
 if GRUB used the exact same language identifiers that (for e.g.)
 Ubuntu already uses?

 You mean glibc locale identifiers with any territory and character set
 information stripped off, and the variant stripped off unless it's
 Portuguese or Chinese since those are the only currently significant
 cases where language variants have significant enough differences to be
 worth considering as essentially separate languages?

 Oh, and locale configuration tends to go together with keyboard
 configuration to some extent (at least for the purpose of setting
 defaults); since text entry is possible in GRUB it makes some sense to
 be able to configure the keymap, and you want to be able to pass that on
 to the operating system as well. Language-to-keymap mapping is a
 non-trivial problem often involving dispute resolution among local users
 to figure out the most appropriate default, and, for us, the keymaps
 themselves are maintained as part of the xkeyboard-config project and
 change quite frequently.

 Beyond the provision of some generic core facilities such as setting a
 keymap, this honestly doesn't seem suitable for the boot loader core,
 and nor does it seem likely to be particularly portable among anything
 other than quite closely related operating systems. Of course it does
 pretty much exactly what we need, but it's far too ad-hoc to live
 outside a scripting language, IMO.

 If GRUB made up its own set of identifiers, then somebody would have to
 maintain the list, and my expectation would be that churn would be
 substantial and frequent. If it used some kind of generic superset, I'd
 expect that we'd end up ignoring it and doing our own thing anyway.

If GRUB 2 supports internationalization for it's own menu, etc, then
they'll need to do something to track which language and which
keyboard mapping anyway. All I'm suggesting is that (in addition to
all the work needed anyway) the multi-boot information structure
(passed to the OS/kernel) include identifiers that indicate which
language (and keyboard mapping) that GRUB used. It's probably about 6
additional lines in the code that creates the multi-boot information
structure.

I honestly don't care where GRUB gets this information from (whether
it can be modified during boot using GRUB's menu, or if it's set when
GRUB is installed or compiled, or if it's specified by some sort of
script, or whatever). I only care about the header in a multi-boot
compliant OS image and the data passed by multi-boot compliant boot
loaders to the OS (where multi-boot compliant includes GRUB, but
doesn't exclude any other implementations of the multi-boot
specification).


Cheers,

Brendan


___
Grub-devel mailing list
Grub-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/grub-devel