Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

Raphael Hertzog wrote:
> Let the work continue!

Are there any result ISOs published yet ?

I'd like to include them in my xorriso regression tests.


Have a nice day :)

Thomas

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Raphael Hertzog]

Hi,

On Fri, 26 Aug 2016, adrian15 wrote:
> I don't know how to add Debian Installer to a Debian Live so I have not been
> able to test it.
> 
> So your feedback as user of the Debian Installer is welcome.

Your patch seems to work. We get a lot of new top-level entries (install,
expert, rescue, auto) all in text and gui variants.

I pushed your branch to the git repository. Let the work continue!

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 26/08/16 a las 09:52, Raphael Hertzog escribió:

(...)

Can you explain to me why such Installation entries are not available in
binary_syslinux?
Maybe they should be put there also?


Well, the menu entries are there by default:
$ cat share/bootloaders/isolinux/menu.cfg
menu hshift 0
menu width 82

menu title Boot menu
include stdmenu.cfg
include live.cfg
include install.cfg
menu begin advanced
menu title Advanced options
include stdmenu.cfg
label mainmenu
menu label ^Back..
menu exit
include advanced.cfg
menu end

menu clear
$ cat share/bootloaders/isolinux/install.cfg
label install
menu label ^Install
linux /install/vmlinuz
initrd /install/initrd.gz
append vga=788 @APPEND_INSTALL@ --- quiet

label installgui
menu label ^Graphical install
linux /install/gtk/vmlinuz
initrd /install/gtk/initrd.gz
append video=vesa:ywrap,mtrr vga=788 @APPEND_INSTALL@ --- quiet

In fact I have not found any code to drop those entries when you don't
want debian-installer in your live image... so I would rather see the opposite
problem.

So please add those entries in the grub menu.

(...)

I guess that some of those issues are not due to your changes, they
are probably longstanding issues in the generic grub code but still
it would be nice to have this fixed to have a more consistent experience
between grub and syslinux.


Well, there are some possible solutions to this problem:

(...)


3. Commit the patch as is and later on add more patches on the minimal set
needed for prettyfing this.


I'm ok for this but I would still like you to re-add the installer entries 
first.

Cheers,



So, here's my current uefi patch + the installation boot entries.

This recycled code from old binary_grub-pc does drop the installation 
entries if LB_DEBIAN_INSTALLER is false.


I don't know how to add Debian Installer to a Debian Live so I have not 
been able to test it.


So your feedback as user of the Debian Installer is welcome.

https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_10

adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/
>From 01a9df8ce325c5df9762f0db86128614b4d3476c Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:04:00 +
Subject: [PATCH 01/11] functions/default.sh : Define LB_PRIMARY_BOOTLOADER at
 the Set_defaults function which it's the right place where to do it

---
 functions/defaults.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/functions/defaults.sh b/functions/defaults.sh
index ddf4b19..334984f 100755
--- a/functions/defaults.sh
+++ b/functions/defaults.sh
@@ -537,6 +537,8 @@ Set_defaults ()
 		esac
 	fi
 
+	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
+
 	# Setting checksums
 	case "${LB_MODE}" in
 		progress-linux)
@@ -845,9 +847,6 @@ Check_defaults ()
 		fi
 	fi
 
-
-	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 	if [ "${LB_PRIMARY_BOOTLOADER}" = "syslinux" ]
 	then
 		# syslinux + fat or ntfs, or extlinux + ext[234] or btrfs
-- 
2.1.4

>From 0624064d44ed811aec5c43fabfd7b928688ed8e1 Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Wed, 20 Jan 2016 00:53:53 +0100
Subject: [PATCH 02/11] Remove repeated LB_PRIMARY_BOOTLOADER definition

---
 scripts/build/binary_hdd | 2 --
 scripts/build/binary_iso | 2 --
 2 files changed, 4 deletions(-)

diff --git a/scripts/build/binary_hdd b/scripts/build/binary_hdd
index 407901a..b45b2a9 100755
--- a/scripts/build/binary_hdd
+++ b/scripts/build/binary_hdd
@@ -67,8 +67,6 @@ do
 	esac
 done
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 case ${LB_PRIMARY_BOOTLOADER} in
 		syslinux)
 			case ${LB_BINARY_FILESYSTEM} in
diff --git a/scripts/build/binary_iso b/scripts/build/binary_iso
index d8b1553..67dfc85 100755
--- a/scripts/build/binary_iso
+++ b/scripts/build/binary_iso
@@ -107,8 +107,6 @@ then
 	XORRISO_OPTIONS="${XORRISO_OPTIONS} -V \"${LB_ISO_VOLUME}\""
 fi
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 # Handle xorriso architecture specific options
 case "${LB_PRIMARY_BOOTLOADER}" in
 	grub)
-- 
2.1.4

>From 9d1a983cc8fe12966d1a2c24a6ee0cfb419b3ce5 Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:07:48 +
Subject: [PATCH 03/11] Added new multi bootloader helper functions * Added:
 functions/bootloaders.sh . This file adds bootloader functions that are
 heavily used in efi scenarios where a bootloader can act as a first or an
 extra bootloader.

Since the introduction of the new switch:

--bootloaders

you can setup it like this:

--bootloaders=syslinux,grub-efi

.

This means that syslinux is the first 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

On 26 August 2016 at 13:34, Raphael Hertzog  wrote:
> On Fri, 26 Aug 2016, adrian15 wrote:
>> > Well, it sucks compared to the default visual appearance of
>> > isolinux/syslinux in live-build.
>> I know, but the purpose of my patch is to add UEFI support. Not to improve
>> visual appearance of grub2 so that it matches the isolinux/syslinux one.
>
> Well, my initial patch added EFI support on top of syslinux-efi and it
> thus had a nice visual appearance by default. So for me it's a
> regression...
>
> But I assume it's not a regression in terms of computers supported because
> I believe that syslinux-efi works for less cases than grub-efi and hence
> why I did not commit my own patch in the first place. Also I expect that
> secure boot will be easier to add on top of grub than on top of syslinux.

It works for Ubuntu so you can just pull the packages from there either through
apt pinning or by building a personal repo with them.

It seems some parts of the stuff are landing in experimental/sid but it would
be nice to have support for this in debian-live when it's complete.

I am probably not going to look into this any time soon, though.

>
> So I agree to apply your patch but I hope that you will stick around to
> help improve the visual appearance.

This bug is totally not about look of grub menus in debian-live.

If you are concerned about that file a separate bug about it.

>
>> These: /install/vmlinuz and /install/initrd.gz strings seem to be hardcoded.
>> That explains why I did not see them in binary_syslinux .
>> What does happen if you request both x86 and amd64 in your Debian Live? The
>> first kernel gets into the /install/vmlinuz and the second kernel gets
>> discarded?
>
> I don't know, I never tried to build images for multiple architectures.
> It's not a need for me.

It depends on the grub live scripts/templates, obviously.

Presumably fixing the installer support is needed to get full UEFI live support
but just filing it as an additional bug after this one is resolved may
be also fine.

It will be easier to fix once live media with *some* UEFI support are available.

Also I am fine with debootstrapping my system so that's not complete
installation blocker.

Unfortunately, most systems that need UEFI support also need secure boot
so solving that is probably most urgent UEFI problem.

The few I have access to right now do require it.

Thanks

Michal

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 26/08/16 a las 13:34, Raphael Hertzog escribió:

On Fri, 26 Aug 2016, adrian15 wrote:

Well, it sucks compared to the default visual appearance of
isolinux/syslinux in live-build.

I know, but the purpose of my patch is to add UEFI support. Not to improve
visual appearance of grub2 so that it matches the isolinux/syslinux one.


Well, my initial patch added EFI support on top of syslinux-efi and it
thus had a nice visual appearance by default. So for me it's a
regression...
I consider myself syslinux to be a regression in terms of functionality 
compared to grub2 but I stick to syslinux because I don't want delta 
between Rescatux and Debian Live to be too large.


I compare this new grub2 appearance to be the same one had the debian-cd 
when it started to support UEFI back in the day.



But I assume it's not a regression in terms of computers supported because
I believe that syslinux-efi works for less cases than grub-efi and hence
why I did not commit my own patch in the first place. Also I expect that
secure boot will be easier to add on top of grub than on top of syslinux.


That's true. There does not seem to be a secure boot support in the 
works in the syslinux side of things.


So I agree to apply your patch but I hope that you will stick around to
help improve the visual appearance.
I will try to check what jnqnfe did but I don't promise anything. I 
commited myself to work on this bug one year ago and I went silent on 
this bug for 3 or 4 months.



These: /install/vmlinuz and /install/initrd.gz strings seem to be hardcoded.
That explains why I did not see them in binary_syslinux .
What does happen if you request both x86 and amd64 in your Debian Live? The
first kernel gets into the /install/vmlinuz and the second kernel gets
discarded?


I don't know, I never tried to build images for multiple architectures.
It's not a need for me.

I see.



I will apply those patches as an addendum to my current changes. I don't
think it's worth the rebased needed for applying first the changes into the
grub-pc code.


Fine for me.

Ok.


Cheers,



adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

Raphael Hertzog wrote:
> I believe that syslinux-efi works for less cases than grub-efi

Discussions on syslinux mailing lists indicate that syslinux-efi does
not work with optical drives. (And i know of no bootable ISO 9660
image which would have syslinux-efi in its El Torito boot image
which serves as EFI System Partition.)


Have a nice day :)

Thomas

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Raphael Hertzog]

On Fri, 26 Aug 2016, adrian15 wrote:
> > Well, it sucks compared to the default visual appearance of
> > isolinux/syslinux in live-build.
> I know, but the purpose of my patch is to add UEFI support. Not to improve
> visual appearance of grub2 so that it matches the isolinux/syslinux one.

Well, my initial patch added EFI support on top of syslinux-efi and it
thus had a nice visual appearance by default. So for me it's a
regression...

But I assume it's not a regression in terms of computers supported because
I believe that syslinux-efi works for less cases than grub-efi and hence
why I did not commit my own patch in the first place. Also I expect that
secure boot will be easier to add on top of grub than on top of syslinux.

So I agree to apply your patch but I hope that you will stick around to
help improve the visual appearance.

> These: /install/vmlinuz and /install/initrd.gz strings seem to be hardcoded.
> That explains why I did not see them in binary_syslinux .
> What does happen if you request both x86 and amd64 in your Debian Live? The
> first kernel gets into the /install/vmlinuz and the second kernel gets
> discarded?

I don't know, I never tried to build images for multiple architectures.
It's not a need for me.

> I will apply those patches as an addendum to my current changes. I don't
> think it's worth the rebased needed for applying first the changes into the
> grub-pc code.

Fine for me.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 26/08/16 a las 09:52, Raphael Hertzog escribió:

On Thu, 25 Aug 2016, adrian15 wrote:

That's how the grub-pc menu (BIOS) shows currently in live-build.


Well, it sucks compared to the default visual appearance of
isolinux/syslinux in live-build.
I know, but the purpose of my patch is to add UEFI support. Not to 
improve visual appearance of grub2 so that it matches the 
isolinux/syslinux one.




- there are no menu entries to start debian-installer even though
I built my image with "--debian-installer live"


There were not such entries in the isolinux menu (BIOS). I already commented
such incongruence when I submitted my loopback patch but Daniel (or irl
maybe?. It was around the time dba quitted the project) agreed to merge my
code.

Compared to binary_grub2 script I have removed the installation
entries because I did not see any of them in binary_syslinux.

Can you explain to me why such Installation entries are not available in
binary_syslinux?
Maybe they should be put there also?


Well, the menu entries are there by default:
$ cat share/bootloaders/isolinux/menu.cfg
menu hshift 0
menu width 82

menu title Boot menu
include stdmenu.cfg
include live.cfg
include install.cfg
menu begin advanced
menu title Advanced options
include stdmenu.cfg
label mainmenu
menu label ^Back..
menu exit
include advanced.cfg
menu end

menu clear
$ cat share/bootloaders/isolinux/install.cfg
label install
menu label ^Install
linux /install/vmlinuz
initrd /install/initrd.gz
append vga=788 @APPEND_INSTALL@ --- quiet

label installgui
menu label ^Graphical install
linux /install/gtk/vmlinuz
initrd /install/gtk/initrd.gz
append video=vesa:ywrap,mtrr vga=788 @APPEND_INSTALL@ --- quiet

In fact I have not found any code to drop those entries when you don't
want debian-installer in your live image... so I would rather see the opposite
problem.

So please add those entries in the grub menu.

These: /install/vmlinuz and /install/initrd.gz strings seem to be 
hardcoded. That explains why I did not see them in binary_syslinux .
What does happen if you request both x86 and amd64 in your Debian Live? 
The first kernel gets into the /install/vmlinuz and the second kernel 
gets discarded?



- the menu entries hardcode "Debian GNU/Linux" as name of the project,
in the default syslinux configuration the entries are agnostic as in
"Live (@FLAVOUR@)".

That @FLAVOUR@ is from Debian's live-build?


Yes:

$ cat share/bootloaders/isolinux/live.cfg.in
label live-@FLAVOUR@
menu label ^Live (@FLAVOUR@)
menu default
linux @LINUX@
initrd @INITRD@
append @APPEND_LIVE@

label live-@FLAVOUR@-failsafe
menu label ^Live (@FLAVOUR@ failsafe)
linux @LINUX@
initrd @INITRD@
append @APPEND_LIVE_FAILSAFE@


I see.

I guess that some of those issues are not due to your changes, they
are probably longstanding issues in the generic grub code but still
it would be nice to have this fixed to have a more consistent experience
between grub and syslinux.


Well, there are some possible solutions to this problem:

1. Try to reuse some code from jnqnfe. He did quite some work on improving
bootloaders design which I don't think got into GIT:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775322 .


His patch set is massive... but yes we should probably review them
and merge what's appropriate.

Yeah, probably.



2. Try to reuse the debian-cd scripts which try to convert syslinux cfg
files into grub ones (including the design).

Here:

https://anonscm.debian.org/cgit/debian-cd/debian-cd.git/tree/tools/boot/jessie/parse_isolinux

Not sure if this is the correct version to use but be warned that Sledge
himself warns us that it's not pretty.


Yeah, I looked that code in the past and I would not want to rely on this even 
though
the principle would be nice...

:)



3. Commit the patch as is and later on add more patches on the minimal set
needed for prettyfing this.


I'm ok for this but I would still like you to re-add the installer entries 
first.


Although I do not like those hardcoded /install/vmlinuz strings I think 
I can do that.


I will apply those patches as an addendum to my current changes. I don't 
think it's worth the rebased needed for applying first the changes into 
the grub-pc code.




Cheers,


Thank you for your feedback.

adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Raphael Hertzog]

On Thu, 25 Aug 2016, adrian15 wrote:
> That's how the grub-pc menu (BIOS) shows currently in live-build.

Well, it sucks compared to the default visual appearance of
isolinux/syslinux in live-build.

> > - there are no menu entries to start debian-installer even though
> >I built my image with "--debian-installer live"
> 
> There were not such entries in the isolinux menu (BIOS). I already commented
> such incongruence when I submitted my loopback patch but Daniel (or irl
> maybe?. It was around the time dba quitted the project) agreed to merge my
> code.
>
> Compared to binary_grub2 script I have removed the installation
> entries because I did not see any of them in binary_syslinux.
> 
> Can you explain to me why such Installation entries are not available in
> binary_syslinux?
> Maybe they should be put there also?

Well, the menu entries are there by default:
$ cat share/bootloaders/isolinux/menu.cfg 
menu hshift 0
menu width 82

menu title Boot menu
include stdmenu.cfg
include live.cfg
include install.cfg
menu begin advanced
menu title Advanced options
include stdmenu.cfg
label mainmenu
menu label ^Back..
menu exit
include advanced.cfg
menu end

menu clear
$ cat share/bootloaders/isolinux/install.cfg 
label install
menu label ^Install
linux /install/vmlinuz
initrd /install/initrd.gz
append vga=788 @APPEND_INSTALL@ --- quiet

label installgui
menu label ^Graphical install
linux /install/gtk/vmlinuz
initrd /install/gtk/initrd.gz
append video=vesa:ywrap,mtrr vga=788 @APPEND_INSTALL@ --- quiet

In fact I have not found any code to drop those entries when you don't
want debian-installer in your live image... so I would rather see the opposite
problem.

So please add those entries in the grub menu.

> > - the menu entries hardcode "Debian GNU/Linux" as name of the project,
> >in the default syslinux configuration the entries are agnostic as in
> >"Live (@FLAVOUR@)".
> That @FLAVOUR@ is from Debian's live-build?

Yes: 

$ cat share/bootloaders/isolinux/live.cfg.in 
label live-@FLAVOUR@
menu label ^Live (@FLAVOUR@)
menu default
linux @LINUX@
initrd @INITRD@
append @APPEND_LIVE@

label live-@FLAVOUR@-failsafe
menu label ^Live (@FLAVOUR@ failsafe)
linux @LINUX@
initrd @INITRD@
append @APPEND_LIVE_FAILSAFE@

> > I guess that some of those issues are not due to your changes, they
> > are probably longstanding issues in the generic grub code but still
> > it would be nice to have this fixed to have a more consistent experience
> > between grub and syslinux.
> 
> Well, there are some possible solutions to this problem:
> 
> 1. Try to reuse some code from jnqnfe. He did quite some work on improving
> bootloaders design which I don't think got into GIT:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775322 .

His patch set is massive... but yes we should probably review them
and merge what's appropriate.

> 2. Try to reuse the debian-cd scripts which try to convert syslinux cfg
> files into grub ones (including the design).
> 
> Here:
> 
> https://anonscm.debian.org/cgit/debian-cd/debian-cd.git/tree/tools/boot/jessie/parse_isolinux
> 
> Not sure if this is the correct version to use but be warned that Sledge
> himself warns us that it's not pretty.

Yeah, I looked that code in the past and I would not want to rely on this even 
though
the principle would be nice...

> 3. Commit the patch as is and later on add more patches on the minimal set
> needed for prettyfing this.

I'm ok for this but I would still like you to re-add the installer entries 
first.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 25/08/16 a las 15:36, Raphael Hertzog escribió:

Hello Adrian,

On Tue, 16 Aug 2016, adrian15 wrote:

Kristian Klausen thinks is a good idea to wait for your tests.
So your feedback is welcome.


I just built a test Kali image with your patch applied. It works:
I can boot the live system in UEFI mode.


Great!

> But I also saw a few problems:

- the grub menu looks ugly (see attached picture), it's in plain
   text mode and the background is not even uniform (sometimes blue,
   sometimes black)

That's how the grub-pc menu (BIOS) shows currently in live-build.


- there are no menu entries to start debian-installer even though
   I built my image with "--debian-installer live"


There were not such entries in the isolinux menu (BIOS). I already 
commented such incongruence when I submitted my loopback patch but 
Daniel (or irl maybe?. It was around the time dba quitted the project) 
agreed to merge my code.


In my commit:
https://anonscm.debian.org/cgit/debian-live/live-build.git/commit/?id=b6771e736022d31b0cb53c552a0f7a42a1028a09

I already said:

Compared to binary_grub2 script I have removed the installation
entries because I did not see any of them in binary_syslinux.

Can you explain to me why such Installation entries are not available in 
binary_syslinux?

Maybe they should be put there also?


- the menu entries hardcode "Debian GNU/Linux" as name of the project,
   in the default syslinux configuration the entries are agnostic as in
   "Live (@FLAVOUR@)".

That @FLAVOUR@ is from Debian's live-build?

That was not the case for grub2 (BIOS) files.

You have:

https://anonscm.debian.org/cgit/debian-live/live-build.git/tree/share/bootloaders/grub-pc/grub.cfg?id=489a09ba92328cef846598a15329e9ff91519e3c

LINUX_LIVE
and
LINUX_INSTALL

strings which get replaced by the grub-pc script.

https://anonscm.debian.org/cgit/debian-live/live-build.git/tree/scripts/build/binary_grub-pc?id=489a09ba92328cef846598a15329e9ff91519e3c

In that case 'Debian Live' is hardcoded.



I guess that some of those issues are not due to your changes, they
are probably longstanding issues in the generic grub code but still
it would be nice to have this fixed to have a more consistent experience
between grub and syslinux.


Well, there are some possible solutions to this problem:

1. Try to reuse some code from jnqnfe. He did quite some work on 
improving bootloaders design which I don't think got into GIT: 
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775322 .


2. Try to reuse the debian-cd scripts which try to convert syslinux cfg 
files into grub ones (including the design).


Here:

https://anonscm.debian.org/cgit/debian-cd/debian-cd.git/tree/tools/boot/jessie/parse_isolinux

Not sure if this is the correct version to use but be warned that Sledge 
himself warns us that it's not pretty.


3. Commit the patch as is and later on add more patches on the minimal 
set needed for prettyfing this.


I advocate for commiting the patch as soon as possible so that I can ask 
new commits such as the Suchanek's '
"support multiple kernel versions with same flavour" to be reworked on 
over my uefi support.


adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 04/08/16 a las 14:51, Raphael Hertzog escribió:

Hi,

On Sun, 31 Jul 2016, adrian15 wrote:

Is there anyone else than can provide feedback on this patch / branch?

Either by:

* Installing live-build with this applied patch
* Building your iso and check if it boots in both BIOS and UEFI mode.
* Check non usual UEFI machines.

I suspect it's fine to wait for one week for feedback.
After that time I'll try to request a proper pull / insertion into Debian's
live-build repo and probably into live-build package binaries.


I can give a try to your patchset but I'm currently in vacation so it
would not be before the third week of August.

This is just FYI but you don't have to wait on my feedback to get your
work merged if people are happy with it.

Cheers,



Hi Raphael,

  Kristian Klausen thinks is a good idea to wait for your tests.

So your feedback is welcome.

  Thank you.


adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 04/08/16 a las 14:51, Raphael Hertzog escribió:

Hi,

On Sun, 31 Jul 2016, adrian15 wrote:

Is there anyone else than can provide feedback on this patch / branch?

Either by:

* Installing live-build with this applied patch
* Building your iso and check if it boots in both BIOS and UEFI mode.
* Check non usual UEFI machines.

I suspect it's fine to wait for one week for feedback.
After that time I'll try to request a proper pull / insertion into Debian's
live-build repo and probably into live-build package binaries.


I can give a try to your patchset but I'm currently in vacation so it
would not be before the third week of August.

This is just FYI but you don't have to wait on my feedback to get your
work merged if people are happy with it.

Cheers,


I have CCed Kristian Klausen whom it's willing to pull/push my patches.

By the way my patches are aimed at master branch. Not sure if the 
current protocol allows to add new changes to master branch. But, well, 
if I'm not mistaken this is the Jessie branch and we need these changes 
for Jessie.


If it makes sense from the Debian release team point of view of course. 
Maybe the Jessie's live-build package cannot have any more updates. Can 
anyone shed light on this concern of mine?


The branch to merge would be: 
https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_9 
.


I forward to Kristian Klausen the decision on whether to wait for 
Raphael Hertzog feedback or not.



adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Raphael Hertzog]

Hi,

On Sun, 31 Jul 2016, adrian15 wrote:
> Is there anyone else than can provide feedback on this patch / branch?
> 
> Either by:
> 
> * Installing live-build with this applied patch
> * Building your iso and check if it boots in both BIOS and UEFI mode.
> * Check non usual UEFI machines.
> 
> I suspect it's fine to wait for one week for feedback.
> After that time I'll try to request a proper pull / insertion into Debian's
> live-build repo and probably into live-build package binaries.

I can give a try to your patchset but I'm currently in vacation so it
would not be before the third week of August.

This is just FYI but you don't have to wait on my feedback to get your
work merged if people are happy with it.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

1) I'm glad Michal finally agrees with the overall implementation of 
multi bootloaders support.


2)

I have squashed some commits were it made sense.
I have fixed some commits descriptions.
I have removed a whitespace.

Here's the new rebased branch:

https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_9

git diff betweenefi_support_based_on_debian_cd_rebased_7 and 
efi_support_based_on_debian_cd_rebased_9 shows only a whitespace as a 
difference, so I have not even bothered to test this new branch.


3) wCPO from debian-live irc is willing to pull it.

Is there anyone else than can provide feedback on this patch / branch?

Either by:

* Installing live-build with this applied patch
* Building your iso and check if it boots in both BIOS and UEFI mode.
* Check non usual UEFI machines.

I suspect it's fine to wait for one week for feedback.
After that time I'll try to request a proper pull / insertion into 
Debian's live-build repo and probably into live-build package binaries.


adrian15

El 31/07/16 a las 10:12, Michal Suchanek escribió:

Hello,

On 31 July 2016 at 09:35, adrian15  wrote:

This new update tries to implement actual support for multiple bootloaders.
It only enforces grub-legacy not to be an extra bootloader because it's
current implementation in binary-iso is not compatible (without hacking)
with multiple bootloaders.


* Branch were I added these last improvement / fixes:
https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_michal
* Final rebased branched:
https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_7


I skimmed through this final rebased branch and it looks generally good.

I would suggest removing the unrelated whitespace changes and
squashing commits that
rewrite new code with commits introducing said code.

I have no idea who can pull this so it can be released in Debian.

Thanks

Michal



--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/
>From 01a9df8ce325c5df9762f0db86128614b4d3476c Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:04:00 +
Subject: [PATCH 01/10] functions/default.sh : Define LB_PRIMARY_BOOTLOADER at
 the Set_defaults function which it's the right place where to do it

---
 functions/defaults.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/functions/defaults.sh b/functions/defaults.sh
index ddf4b19..334984f 100755
--- a/functions/defaults.sh
+++ b/functions/defaults.sh
@@ -537,6 +537,8 @@ Set_defaults ()
 		esac
 	fi
 
+	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
+
 	# Setting checksums
 	case "${LB_MODE}" in
 		progress-linux)
@@ -845,9 +847,6 @@ Check_defaults ()
 		fi
 	fi
 
-
-	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 	if [ "${LB_PRIMARY_BOOTLOADER}" = "syslinux" ]
 	then
 		# syslinux + fat or ntfs, or extlinux + ext[234] or btrfs
-- 
2.1.4

>From 0624064d44ed811aec5c43fabfd7b928688ed8e1 Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Wed, 20 Jan 2016 00:53:53 +0100
Subject: [PATCH 02/10] Remove repeated LB_PRIMARY_BOOTLOADER definition

---
 scripts/build/binary_hdd | 2 --
 scripts/build/binary_iso | 2 --
 2 files changed, 4 deletions(-)

diff --git a/scripts/build/binary_hdd b/scripts/build/binary_hdd
index 407901a..b45b2a9 100755
--- a/scripts/build/binary_hdd
+++ b/scripts/build/binary_hdd
@@ -67,8 +67,6 @@ do
 	esac
 done
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 case ${LB_PRIMARY_BOOTLOADER} in
 		syslinux)
 			case ${LB_BINARY_FILESYSTEM} in
diff --git a/scripts/build/binary_iso b/scripts/build/binary_iso
index d8b1553..67dfc85 100755
--- a/scripts/build/binary_iso
+++ b/scripts/build/binary_iso
@@ -107,8 +107,6 @@ then
 	XORRISO_OPTIONS="${XORRISO_OPTIONS} -V \"${LB_ISO_VOLUME}\""
 fi
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 # Handle xorriso architecture specific options
 case "${LB_PRIMARY_BOOTLOADER}" in
 	grub)
-- 
2.1.4

>From 9d1a983cc8fe12966d1a2c24a6ee0cfb419b3ce5 Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:07:48 +
Subject: [PATCH 03/10] Added new multi bootloader helper functions * Added:
 functions/bootloaders.sh . This file adds bootloader functions that are
 heavily used in efi scenarios where a bootloader can act as a first or an
 extra bootloader.

Since the introduction of the new switch:

--bootloaders

you can setup it like this:

--bootloaders=syslinux,grub-efi

.

This means that syslinux is the first bootloader and grub-efi is the extra bootloader.

* Added new bootloader functions: Check_Non_First_Bootloader and Check_Non_Extra_Bootloader.

These functions let each one of the bootloaders abort 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

This new update tries to implement actual support for multiple 
bootloaders. It only enforces grub-legacy not to be an extra bootloader 
because it's current implementation in binary-iso is not compatible 
(without hacking) with multiple bootloaders.



* Branch were I added these last improvement / fixes: 
https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_michal
* Final rebased branched: 
https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_7



Rescatux 0.40 beta 7 has been built with 
efi_support_based_on_debian_cd_rebased_7 branch
and can be found here: 
http://www.supergrubdisk.org/2016/07/31/rescatux-0-40-beta-7-released/ .


As always feedback is welcome.

El 22/03/16 a las 07:18, Michal Suchanek escribió:

On 21 March 2016 at 23:06, adrian15  wrote:

El 21/03/16 a las 22:19, Michal Suchanek escribió:

So please consider either

1) fixing your current patch so there is no primary or first
bootloader and all installed bootloaders are equal

2) don't pretend you add support for multiple bootloaders when you are
not wiling to do so and just and some option like --bolt-on-grub-efi
which installs grub-efi if image type and filesystem is compatible
with grub-efi and fails the build otherwise

BTW it has been pointed out already that -eltorito-alt-boot is just
separator that starts new boot entry so there are no special
secondary/extra bootloader options. Any bootloader can be
first/second/third/whatever.

Thanks

Michal



--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/
>From 723661c38b48d0617c5c660ea140edfaef4699ce Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:04:00 +
Subject: [PATCH 01/13] functions/default.sh : Define LB_PRIMARY_BOOTLOADER at
 the Set_defaults function which it's the right place where to do it

---
 functions/defaults.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/functions/defaults.sh b/functions/defaults.sh
index ddf4b19..334984f 100755
--- a/functions/defaults.sh
+++ b/functions/defaults.sh
@@ -537,6 +537,8 @@ Set_defaults ()
 		esac
 	fi
 
+	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
+
 	# Setting checksums
 	case "${LB_MODE}" in
 		progress-linux)
@@ -845,9 +847,6 @@ Check_defaults ()
 		fi
 	fi
 
-
-	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 	if [ "${LB_PRIMARY_BOOTLOADER}" = "syslinux" ]
 	then
 		# syslinux + fat or ntfs, or extlinux + ext[234] or btrfs
-- 
2.1.4

>From a506c9c03d61f9c09d816a9e24285c599c9bacdf Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Wed, 20 Jan 2016 00:53:53 +0100
Subject: [PATCH 02/13] Remove repeated LB_PRIMARY_BOOTLOADER definition

---
 scripts/build/binary_hdd | 2 --
 scripts/build/binary_iso | 2 --
 2 files changed, 4 deletions(-)

diff --git a/scripts/build/binary_hdd b/scripts/build/binary_hdd
index 407901a..b45b2a9 100755
--- a/scripts/build/binary_hdd
+++ b/scripts/build/binary_hdd
@@ -67,8 +67,6 @@ do
 	esac
 done
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 case ${LB_PRIMARY_BOOTLOADER} in
 		syslinux)
 			case ${LB_BINARY_FILESYSTEM} in
diff --git a/scripts/build/binary_iso b/scripts/build/binary_iso
index d8b1553..67dfc85 100755
--- a/scripts/build/binary_iso
+++ b/scripts/build/binary_iso
@@ -107,8 +107,6 @@ then
 	XORRISO_OPTIONS="${XORRISO_OPTIONS} -V \"${LB_ISO_VOLUME}\""
 fi
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 # Handle xorriso architecture specific options
 case "${LB_PRIMARY_BOOTLOADER}" in
 	grub)
-- 
2.1.4

>From 06ad8acdc81e6bc2b460954bae57191ddb7d238a Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:07:48 +
Subject: [PATCH 03/13] * Added: functions/bootloaders.sh . It has new
 bootloader functions that are heavily used in efi scenarios where a
 bootloader can act as a primary or a secondary bootloader.

Since the introduction of the new switch:

--bootloaders

you can setup it like this:

--bootloaders=syslinux,grub-efi

.

This means that syslinux is the primary bootloader and grub-efi is the secondary bootloader.

* Added new bootloader functions: Check_Non_Primary_Bootloader and Check_Non_Secondary_Bootloader.

These functions let each one of the bootloaders abort the build because
they cannot perform a role either as a primary bootloader or as a secondary bootloader.

* Added bootloader functions: Check_Primary_Bootloader_Role, Check_Secondary_Bootloader_Role and Check_Any_Bootloader_Role

These functions let bootloaders to force their default role in a single line.
---
 functions/bootloaders.sh | 128 +++
 1 file changed, 128 insertions(+)
 create mode 100644 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

Hello,

On 31 July 2016 at 09:35, adrian15  wrote:
> This new update tries to implement actual support for multiple bootloaders.
> It only enforces grub-legacy not to be an extra bootloader because it's
> current implementation in binary-iso is not compatible (without hacking)
> with multiple bootloaders.
>
>
> * Branch were I added these last improvement / fixes:
> https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_michal
> * Final rebased branched:
> https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_7

I skimmed through this final rebased branch and it looks generally good.

I would suggest removing the unrelated whitespace changes and
squashing commits that
rewrite new code with commits introducing said code.

I have no idea who can pull this so it can be released in Debian.

Thanks

Michal

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

On 21 March 2016 at 23:06, adrian15  wrote:
> El 21/03/16 a las 22:19, Michal Suchanek escribió:
>

>> The bootloader support in live-build is limited. With your patches it
>> becomes wrong. eg. compatibility of bootloader with selected
>> filesystem and image type is only checked for first bootloader and EFI
>> support is added only when grub-efi extra bootloader but not when it
>> is the first bootloader.
>>
>> This is not fixed by renaming the variables.
>
>
> Ok. So I recognise that my patch:
>
> * Adds a limited support of UEFI only available when it's used as an extra
> bootloader
> * Does not check compatibility with selected filesystem in the extra
> bootloader because it blindly relies on selected filesystem selected in
> first bootloader being compatible with the extra bootloader too.
> * It does not work for hdd binaries (only iso binaries)
>
> and it does in addition to what live-build already did.

No, it does not.

live-build only installed bootloader which was compatible with
selected filesystem and image type which is no longer true with your
patch. In fact, if I choose grub-efi as first bootloader the efi
support is not added and compatibility of any extra bootloaders with
the filesystem chosen is not checked so the image may be completely
unbootable.

So please consider either

1) fixing your current patch so there is no primary or first
bootloader and all installed bootloaders are equal

2) don't pretend you add support for multiple bootloaders when you are
not wiling to do so and just and some option like --bolt-on-grub-efi
which installs grub-efi if image type and filesystem is compatible
with grub-efi and fails the build otherwise

BTW it has been pointed out already that -eltorito-alt-boot is just
separator that starts new boot entry so there are no special
secondary/extra bootloader options. Any bootloader can be
first/second/third/whatever.

Thanks

Michal

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 21/03/16 a las 22:19, Michal Suchanek escribió:

Hello,

On 21 March 2016 at 21:09, adrian15  wrote:



The branch which include specifically the commits I attach here as patches
is:

https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_5

.

About the variable names issue: I think the new terms: first and extra are
ok because they are not implying some sort of rank while explaining what's
the difference between them. Also, notice, that these are internal variables
which final user of live-build does not see. I think we should focus on
other aspects of my patch if there are more problems for it.


The problem is not with the name of the variable.

The problem is that you use it at all. In most places when you check
for primary or secondary bootloader you should just loop all
bootloaders and check each. In fact, in the previous batch of patches
I found no place where checking for primary or secondary bootloader
made any sense.



If you think that the way bootloaders is currently managed by live-build is
wrong please file up a new bug and send there your patch with your
improvements so that it's get discussed.


The bootloader support in live-build is limited. With your patches it
becomes wrong. eg. compatibility of bootloader with selected
filesystem and image type is only checked for first bootloader and EFI
support is added only when grub-efi extra bootloader but not when it
is the first bootloader.

This is not fixed by renaming the variables.

Thanks

Michal


Ok, what about now? Actually from my former email I only added one 
additional patch (0011) with some sort of modular support for bootloaders.


There probably needs some work to be done on binary_hdd but I first 
wanted to gather information on what to improve on this last patch.



https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_6


adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/
>From ff8206aea1985760dfea9ec94b93686a242f137e Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:04:00 +
Subject: [PATCH 01/11] functions/default.sh : Define LB_PRIMARY_BOOTLOADER at
 the Set_defaults function which it's the right place where to do it

---
 functions/defaults.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/functions/defaults.sh b/functions/defaults.sh
index ddf4b19..334984f 100755
--- a/functions/defaults.sh
+++ b/functions/defaults.sh
@@ -537,6 +537,8 @@ Set_defaults ()
 		esac
 	fi
 
+	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
+
 	# Setting checksums
 	case "${LB_MODE}" in
 		progress-linux)
@@ -845,9 +847,6 @@ Check_defaults ()
 		fi
 	fi
 
-
-	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 	if [ "${LB_PRIMARY_BOOTLOADER}" = "syslinux" ]
 	then
 		# syslinux + fat or ntfs, or extlinux + ext[234] or btrfs
-- 
2.1.4

>From f895f653dffb614639fa37a318e62c51104a4b2d Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Wed, 20 Jan 2016 00:53:53 +0100
Subject: [PATCH 02/11] Remove repeated LB_PRIMARY_BOOTLOADER definition

---
 scripts/build/binary_hdd | 2 --
 scripts/build/binary_iso | 2 --
 2 files changed, 4 deletions(-)

diff --git a/scripts/build/binary_hdd b/scripts/build/binary_hdd
index d0db382..0c9c5af 100755
--- a/scripts/build/binary_hdd
+++ b/scripts/build/binary_hdd
@@ -67,8 +67,6 @@ do
 	esac
 done
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 case ${LB_PRIMARY_BOOTLOADER} in
 		syslinux)
 			case ${LB_BINARY_FILESYSTEM} in
diff --git a/scripts/build/binary_iso b/scripts/build/binary_iso
index d8b1553..67dfc85 100755
--- a/scripts/build/binary_iso
+++ b/scripts/build/binary_iso
@@ -107,8 +107,6 @@ then
 	XORRISO_OPTIONS="${XORRISO_OPTIONS} -V \"${LB_ISO_VOLUME}\""
 fi
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 # Handle xorriso architecture specific options
 case "${LB_PRIMARY_BOOTLOADER}" in
 	grub)
-- 
2.1.4

>From 9175a624b7d88bfea80448a5a1e6ac1b11d651aa Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:07:48 +
Subject: [PATCH 03/11] * Added: functions/bootloaders.sh . It has new
 bootloader functions that are heavily used in efi scenarios where a
 bootloader can act as a primary or a secondary bootloader.

Since the introduction of the new switch:

--bootloaders

you can setup it like this:

--bootloaders=syslinux,grub-efi

.

This means that syslinux is the primary bootloader and grub-efi is the secondary bootloader.

* Added new bootloader functions: Check_Non_Primary_Bootloader and Check_Non_Secondary_Bootloader.

These functions let each one of the bootloaders abort the build because
they cannot perform a role either as a primary 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 21/03/16 a las 22:19, Michal Suchanek escribió:

Hello,

On 21 March 2016 at 21:09, adrian15  wrote:



The branch which include specifically the commits I attach here as patches
is:

https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_5

.

About the variable names issue: I think the new terms: first and extra are
ok because they are not implying some sort of rank while explaining what's
the difference between them. Also, notice, that these are internal variables
which final user of live-build does not see. I think we should focus on
other aspects of my patch if there are more problems for it.


The problem is not with the name of the variable.

The problem is that you use it at all. In most places when you check
for primary or secondary bootloader you should just loop all
bootloaders and check each. In fact, in the previous batch of patches
I found no place where checking for primary or secondary bootloader
made any sense.



If you think that the way bootloaders is currently managed by live-build is
wrong please file up a new bug and send there your patch with your
improvements so that it's get discussed.


The bootloader support in live-build is limited. With your patches it
becomes wrong. eg. compatibility of bootloader with selected
filesystem and image type is only checked for first bootloader and EFI
support is added only when grub-efi extra bootloader but not when it
is the first bootloader.

This is not fixed by renaming the variables.


Ok. So I recognise that my patch:

* Adds a limited support of UEFI only available when it's used as an 
extra bootloader
* Does not check compatibility with selected filesystem in the extra 
bootloader because it blindly relies on selected filesystem selected in 
first bootloader being compatible with the extra bootloader too.

* It does not work for hdd binaries (only iso binaries)

and it does in addition to what live-build already did.

I also think that my patch does not remove the current live-build 
functionality and if that happens I want to know about it.


Yes, my patch does not bring UEFI complete support, but a minimal one. 
So according to you the many changes I make on the bootloader functions 
do not compensate the minimal UEFI support I add to live-build?


Thank you.

adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

Hello,

On 21 March 2016 at 21:09, adrian15  wrote:

>
> The branch which include specifically the commits I attach here as patches
> is:
>
> https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_5
>
> .
>
> About the variable names issue: I think the new terms: first and extra are
> ok because they are not implying some sort of rank while explaining what's
> the difference between them. Also, notice, that these are internal variables
> which final user of live-build does not see. I think we should focus on
> other aspects of my patch if there are more problems for it.

The problem is not with the name of the variable.

The problem is that you use it at all. In most places when you check
for primary or secondary bootloader you should just loop all
bootloaders and check each. In fact, in the previous batch of patches
I found no place where checking for primary or secondary bootloader
made any sense.

>
> If you think that the way bootloaders is currently managed by live-build is
> wrong please file up a new bug and send there your patch with your
> improvements so that it's get discussed.

The bootloader support in live-build is limited. With your patches it
becomes wrong. eg. compatibility of bootloader with selected
filesystem and image type is only checked for first bootloader and EFI
support is added only when grub-efi extra bootloader but not when it
is the first bootloader.

This is not fixed by renaming the variables.

Thanks

Michal

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

This is my updated set of patches.

Changes since last set of patches:

* Renamed primary and secondary to first and extra terms.
* Forced insmod all_video command in grub.cfg to avoid problems in UEFI 
mode.

* Minor changes

Rescatux 0.40b6 which I will release soon will feature this branch which 
include these changes:


https://github.com/rescatux/live-build/tree/rescatux-0.40b6

.

The branch which include specifically the commits I attach here as 
patches is:


https://github.com/rescatux/live-build/tree/efi_support_based_on_debian_cd_rebased_5

.

About the variable names issue: I think the new terms: first and extra 
are ok because they are not implying some sort of rank while explaining 
what's the difference between them. Also, notice, that these are 
internal variables which final user of live-build does not see. I think 
we should focus on other aspects of my patch if there are more problems 
for it.


If you think that the way bootloaders is currently managed by live-build 
is wrong please file up a new bug and send there your patch with your 
improvements so that it's get discussed.


Then, if it gets approved I can improve my patch over yours.

My patch tries to make the minimum improvements so that other live-build 
functionality does not get affected by it.


Let's please focus on getting this set of patches accepted. Don't get me 
wrong I'm not asking for a carte blanche but, let's focus on other 
aspects from the patch. (E.g. please test it on actual hardware, in your 
distro builds, even if you don't use UEFI does the ISO boot as always in 
BIOS mode?) And give us feedback on it.


Thank you.

adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/
>From ff8206aea1985760dfea9ec94b93686a242f137e Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:04:00 +
Subject: [PATCH 01/10] functions/default.sh : Define LB_PRIMARY_BOOTLOADER at
 the Set_defaults function which it's the right place where to do it

---
 functions/defaults.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/functions/defaults.sh b/functions/defaults.sh
index ddf4b19..334984f 100755
--- a/functions/defaults.sh
+++ b/functions/defaults.sh
@@ -537,6 +537,8 @@ Set_defaults ()
 		esac
 	fi
 
+	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
+
 	# Setting checksums
 	case "${LB_MODE}" in
 		progress-linux)
@@ -845,9 +847,6 @@ Check_defaults ()
 		fi
 	fi
 
-
-	LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 	if [ "${LB_PRIMARY_BOOTLOADER}" = "syslinux" ]
 	then
 		# syslinux + fat or ntfs, or extlinux + ext[234] or btrfs
-- 
2.1.4

>From f895f653dffb614639fa37a318e62c51104a4b2d Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Wed, 20 Jan 2016 00:53:53 +0100
Subject: [PATCH 02/10] Remove repeated LB_PRIMARY_BOOTLOADER definition

---
 scripts/build/binary_hdd | 2 --
 scripts/build/binary_iso | 2 --
 2 files changed, 4 deletions(-)

diff --git a/scripts/build/binary_hdd b/scripts/build/binary_hdd
index d0db382..0c9c5af 100755
--- a/scripts/build/binary_hdd
+++ b/scripts/build/binary_hdd
@@ -67,8 +67,6 @@ do
 	esac
 done
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 case ${LB_PRIMARY_BOOTLOADER} in
 		syslinux)
 			case ${LB_BINARY_FILESYSTEM} in
diff --git a/scripts/build/binary_iso b/scripts/build/binary_iso
index d8b1553..67dfc85 100755
--- a/scripts/build/binary_iso
+++ b/scripts/build/binary_iso
@@ -107,8 +107,6 @@ then
 	XORRISO_OPTIONS="${XORRISO_OPTIONS} -V \"${LB_ISO_VOLUME}\""
 fi
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 # Handle xorriso architecture specific options
 case "${LB_PRIMARY_BOOTLOADER}" in
 	grub)
-- 
2.1.4

>From 9175a624b7d88bfea80448a5a1e6ac1b11d651aa Mon Sep 17 00:00:00 2001
From: Adrian Gibanel Lopez 
Date: Mon, 18 Jan 2016 03:07:48 +
Subject: [PATCH 03/10] * Added: functions/bootloaders.sh . It has new
 bootloader functions that are heavily used in efi scenarios where a
 bootloader can act as a primary or a secondary bootloader.

Since the introduction of the new switch:

--bootloaders

you can setup it like this:

--bootloaders=syslinux,grub-efi

.

This means that syslinux is the primary bootloader and grub-efi is the secondary bootloader.

* Added new bootloader functions: Check_Non_Primary_Bootloader and Check_Non_Secondary_Bootloader.

These functions let each one of the bootloaders abort the build because
they cannot perform a role either as a primary bootloader or as a secondary bootloader.

* Added bootloader functions: Check_Primary_Bootloader_Role, Check_Secondary_Bootloader_Role and Check_Any_Bootloader_Role

These functions let bootloaders to force their default role in a single line.
---
 functions/bootloaders.sh | 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

On 26 January 2016 at 23:20, adrian15  wrote:
> El 26/01/16 a las 10:18, Michal Suchanek escribió:

>>
>>>
>>> My use case is the following one. The final user requests:
>>>
>>> --bootloaders=grub-efi,syslinux
>>>
>>> so I show him:
>>>
>>> "Warning. You are using: syslinux as a non first bootloader. This might
>>> work
>>> but it is not advised."
>>>
>>> How do I know that I have to output this message?
>>
>>
>> for bootloader in $BOOTLOADERS ; do
>>
>>  # some bootloader foo
>>
>>  if echo $BIOS_BOOTLOADERS | grep "${bootloader}" >/dev/null; then
>> # a fixed list of bootloaders that load through legacy BIOS -
>> currently should be "syslinux grub-pc" although grub is not well
>> supported
>>  case $MEDIA_TYPE in # or whatever the variable
>>  iso*)
>>  case "${BOOTLOADERS}" in
>>  "${bootloader}"*);;
>>  *) echo "Warning: it is recommended to specify
>> $bootloader first in bootloader list so it gets installed as first
>> el-torito boot entry."
>>  ;;
>>  esac ;;
>>  esac
>>  fi
>> done
>
> Here you are creating a new variable: $BIOS_BOOTLOADERS which will have to
> be updated manually each time a new bios bootloader binary_ file is added.

So what? Any other way to recognize a BIOS bootloader as such?

>
> The grep part should be improved to avoid problems (e.g. syslinux is inside
> syslinux-efi).

Except $BIOS_BOOTLOADERS does not include syslinux-efi so it does not happen ;-)

>
>>> Because I compare the internal variable:
>>>
>>> LB_FIRST_BOOTLOADER="grub-efi"
>>>
>>> with the bootloader name "syslinux" and I see they are not the same one.
>>>
>>> So, as you see I need to use:
>>>
>>> "non first bootloader" term
>>
>>
>> Why that has to be a term? Just say it should come first in the list
>> of bootloaders if specified at all.
>
>
> "It should come first". "It should not come first". Ok. I can do that and
> ditch the "non first bootloader" or "first bootloader" from my messages.
>
>>
>>> and
>>> LB_FIRST_BOOTLOADER variable.
>>
>>
>> what for?
>
>
> For two reasons:
>
> 1) Being able to use current live-build code which used LB_BOOTLOADER in the
> past.
>
> Check what it's in current alioth git (Seach for LB_PRIMARY_BOOTLOADER on
> there):
>
> https://anonscm.debian.org/cgit/debian-live/live-build.git/tree/scripts/build/binary_hdd?id=1ccb41623046f2a8f823d62a5f417cdae724c22b
>
> https://anonscm.debian.org/cgit/debian-live/live-build.git/tree/scripts/build/binary_iso?id=1ccb41623046f2a8f823d62a5f417cdae724c22b
>
> https://anonscm.debian.org/cgit/debian-live/live-build.git/tree/functions/defaults.sh?id=1ccb41623046f2a8f823d62a5f417cdae724c22b

Skimming through that code in every place LB_PRIMARY_BOOTLOADER is
used the code is broken.

If a bootloader is requested then you should install its files/check
that the filesystem is compatible/... regardless of the el-torito
record number it is installed in.

>
> 2) Not having to use awk each time I need to compare a first / default
> bootloader... when I can just use a variable with previous (once only)
> calculated value.

Which if used correctly would be used once or twice if at all.

Thanks

Michal

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

On 25 January 2016 at 21:33, adrian15  wrote:
>
>
> El 25/01/16 a las 16:12, Michal Suchanek escribió:
>>
>> On 25 January 2016 at 03:05, adrian15  wrote:
>>>
>>> El 24/01/16 a las 16:51, Michal Suchanek escribió:
>>
>>
>>> What you are describing here is what it's actually implemented in my
>>> patch
>>> (Well, actually the first patch version because the current one enforces
>>> bootloader roles).
>>
>>
>> Actually, no.
>>
>> Nowhere in the description is any bootloader designated primary or
>> secondary or first or second. On purpose.
>
> Neither it is on my patch (initial implementation). Yes, the term
> PRIMARY_BOOTLOADER is used there for reusing old code. But using:
>
> --bootloaders=syslinux,grub-efi
>
> did not enforce syslinux to be in the first place or grub-efi to be in the
> second place.
>
> That's the specific part I meant.
>
>>
>>> So what about primary and secondary terms? Or first or
>>> second terms?
>>
>>
>> Both are broken and confusing.
>
> Ok...
>>>
>>>
>>> These terms are used in two places:
>>> * Internal variables and functions to handle bootloaders
>>> * Information shown to the final user
>>>
>>> I'm most convinced to use the first and non-first notation. So that the
>>> old
>>> code that referred to LB_BOOTLOADER can just refer to:
>>> LB_FIRST_BOOTLOADER.
>>
>>
>> For what piece of code we have does it make sense to reference
>> LB_FIRST_BOOTLOADER when not also referencing LB_SECOND_BOOTLOADER?
>> Will that be extended to LB_THIRD_BOOTLOADER once x86 grows support
>> for coreboot or l-b grows support for some other platform with many
>> firmware variants?
>>
>> If you set bootloaders like
>>
>> LB_BOOTLOADERS="syslinux grub-efi"
>>
>> then you can just do
>>
>> for bootloader in $LB_BOOTLOADERS ; do some $bootloader foo
>
> Mostly what current path does but with commas instead.

IIRC multivalue options use mostly spaces for separator in l-b so far.

>>
>>
>> after you check that you have at most two bootloaders and if you have
>> more than one then only the latter one ends with -efi.
>
>
> This is not a good approach. You are requesting the bootloaders to end in
> "-efi". The current approach is to name them based on the Debian package
> name. These packages do not need to end in "-efi".

It so happens that bootloaders that support efi are packaged as
packages with -efi suffix (well, except elilo). Maybe there is some
intent there?

However, grub-pc is now split in grub-pc scripts that are meant to
install the bootloader in the system which you probably don't want and
grub-pc-bin which just includes the binaries. The situation is even
more complicated with the 32bit and 64bit efi packages. Also expect
that at some point the maintainers figure out some new completely
awesome way to split the bootloader into packages and then the package
sets will be different in stable/testing/oldstable.

So naming the l-b option *exactly* like the package is not that good idea.

>
> My use case is the following one. The final user requests:
>
> --bootloaders=grub-efi,syslinux
>
> so I show him:
>
> "Warning. You are using: syslinux as a non first bootloader. This might work
> but it is not advised."
>
> How do I know that I have to output this message?

for bootloader in $BOOTLOADERS ; do

# some bootloader foo

if echo $BIOS_BOOTLOADERS | grep "${bootloader}" >/dev/null; then
# a fixed list of bootloaders that load through legacy BIOS -
currently should be "syslinux grub-pc" although grub is not well
supported
case $MEDIA_TYPE in # or whatever the variable
iso*)
case "${BOOTLOADERS}" in
"${bootloader}"*);;
*) echo "Warning: it is recommended to specify
$bootloader first in bootloader list so it gets installed as first
el-torito boot entry."
;;
esac ;;
esac
fi
done


>
> Because I compare the internal variable:
>
> LB_FIRST_BOOTLOADER="grub-efi"
>
> with the bootloader name "syslinux" and I see they are not the same one.
>
> So, as you see I need to use:
>
> "non first bootloader" term

Why that has to be a term? Just say it should come first in the list
of bootloaders if specified at all.

> and
> LB_FIRST_BOOTLOADER variable.

what for?

>
> So...
>
> 1) I don't mind renaming "non first bootloader" or LB_FIRST_BOOTLOADER to
> another terminology which makes more technical sense.
> 2) I prefer this approach over yours (Michal) because it's the own
> bootloader which decides if it is more suited for "first bootloader" or not.

How does it decide that? It can install equally well in 1st, 2nd or
10th el-torito record. The only reason we want BIOS record first is


1) some tools for butchering bootable CDs expect it to be first.

2) it's the traditional place for it (since it was the only record for
a long time) and some ancient BIOSes might potentially break if it's
not first because somebody missed there *can* be 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 26/01/16 a las 10:18, Michal Suchanek escribió:

If you set bootloaders like

LB_BOOTLOADERS="syslinux grub-efi"

then you can just do

for bootloader in $LB_BOOTLOADERS ; do some $bootloader foo


Mostly what current path does but with commas instead.


IIRC multivalue options use mostly spaces for separator in l-b so far.


dba requested it like this. I agree on that criteria for doing so. You 
are not obliged to use quotes when defining bootloaders on the command 
line or scripts.


What are these multivalue options so that I take a look at them? Having 
to deal with IFS it's not ideal but I think using commas it's the way to 
go... although I'm not 100% convinced.



after you check that you have at most two bootloaders and if you have
more than one then only the latter one ends with -efi.



This is not a good approach. You are requesting the bootloaders to end in
"-efi". The current approach is to name them based on the Debian package
name. These packages do not need to end in "-efi".


It so happens that bootloaders that support efi are packaged as
packages with -efi suffix (well, except elilo). Maybe there is some
intent there?

However, grub-pc is now split in grub-pc scripts that are meant to
install the bootloader in the system which you probably don't want and
grub-pc-bin which just includes the binaries. The situation is even
more complicated with the 32bit and 64bit efi packages. Also expect
that at some point the maintainers figure out some new completely
awesome way to split the bootloader into packages and then the package
sets will be different in stable/testing/oldstable.

So naming the l-b option *exactly* like the package is not that good idea.
dba commited a change ( 
https://anonscm.debian.org/cgit/debian-live/live-build.git/commit/?id=f93fa286d5e7348150aab4874794f7d96dac0894 
) for that behaviour. I think the reasoning of that was avoid file 
naming collisions in the future because package names are not repeated.






My use case is the following one. The final user requests:

--bootloaders=grub-efi,syslinux

so I show him:

"Warning. You are using: syslinux as a non first bootloader. This might work
but it is not advised."

How do I know that I have to output this message?


for bootloader in $BOOTLOADERS ; do

 # some bootloader foo

 if echo $BIOS_BOOTLOADERS | grep "${bootloader}" >/dev/null; then
# a fixed list of bootloaders that load through legacy BIOS -
currently should be "syslinux grub-pc" although grub is not well
supported
 case $MEDIA_TYPE in # or whatever the variable
 iso*)
 case "${BOOTLOADERS}" in
 "${bootloader}"*);;
 *) echo "Warning: it is recommended to specify
$bootloader first in bootloader list so it gets installed as first
el-torito boot entry."
 ;;
 esac ;;
 esac
 fi
done
Here you are creating a new variable: $BIOS_BOOTLOADERS which will have 
to be updated manually each time a new bios bootloader binary_ file is 
added.


The grep part should be improved to avoid problems (e.g. syslinux is 
inside syslinux-efi).



Because I compare the internal variable:

LB_FIRST_BOOTLOADER="grub-efi"

with the bootloader name "syslinux" and I see they are not the same one.

So, as you see I need to use:

"non first bootloader" term


Why that has to be a term? Just say it should come first in the list
of bootloaders if specified at all.


"It should come first". "It should not come first". Ok. I can do that 
and ditch the "non first bootloader" or "first bootloader" from my messages.





and
LB_FIRST_BOOTLOADER variable.


what for?


For two reasons:

1) Being able to use current live-build code which used LB_BOOTLOADER in 
the past.


Check what it's in current alioth git (Seach for LB_PRIMARY_BOOTLOADER 
on there):


https://anonscm.debian.org/cgit/debian-live/live-build.git/tree/scripts/build/binary_hdd?id=1ccb41623046f2a8f823d62a5f417cdae724c22b

https://anonscm.debian.org/cgit/debian-live/live-build.git/tree/scripts/build/binary_iso?id=1ccb41623046f2a8f823d62a5f417cdae724c22b

https://anonscm.debian.org/cgit/debian-live/live-build.git/tree/functions/defaults.sh?id=1ccb41623046f2a8f823d62a5f417cdae724c22b

2) Not having to use awk each time I need to compare a first / default 
bootloader... when I can just use a variable with previous (once only) 
calculated value.




So...

1) I don't mind renaming "non first bootloader" or LB_FIRST_BOOTLOADER to
another terminology which makes more technical sense.
2) I prefer this approach over yours (Michal) because it's the own
bootloader which decides if it is more suited for "first bootloader" or not.


How does it decide that? It can install equally well in 1st, 2nd or
10th el-torito record. The only reason we want BIOS record first is


Inside the binary_bootloader file by running a function that shows that 
warning. The author of the binary_bootloader it's who decides where it's 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

On 25 January 2016 at 03:05, adrian15  wrote:
> El 24/01/16 a las 16:51, Michal Suchanek escribió:

> What you are describing here is what it's actually implemented in my patch
> (Well, actually the first patch version because the current one enforces
> bootloader roles).

Actually, no.

Nowhere in the description is any bootloader designated primary or
secondary or first or second. On purpose.

> So what about primary and secondary terms? Or first or
> second terms?

Both are broken and confusing.

>
> These terms are used in two places:
> * Internal variables and functions to handle bootloaders
> * Information shown to the final user
>
> I'm most convinced to use the first and non-first notation. So that the old
> code that referred to LB_BOOTLOADER can just refer to: LB_FIRST_BOOTLOADER.

For what piece of code we have does it make sense to reference
LB_FIRST_BOOTLOADER when not also referencing LB_SECOND_BOOTLOADER?
Will that be extended to LB_THIRD_BOOTLOADER once x86 grows support
for coreboot or l-b grows support for some other platform with many
firmware variants?

If you set bootloaders like

LB_BOOTLOADERS="syslinux grub-efi"

then you can just do

for bootloader in $LB_BOOTLOADERS ; do some $bootloader foo

after you check that you have at most two bootloaders and if you have
more than one then only the latter one ends with -efi.

Thanks

Michal

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 25/01/16 a las 16:12, Michal Suchanek escribió:

On 25 January 2016 at 03:05, adrian15  wrote:

El 24/01/16 a las 16:51, Michal Suchanek escribió:



What you are describing here is what it's actually implemented in my patch
(Well, actually the first patch version because the current one enforces
bootloader roles).


Actually, no.

Nowhere in the description is any bootloader designated primary or
secondary or first or second. On purpose.
Neither it is on my patch (initial implementation). Yes, the term 
PRIMARY_BOOTLOADER is used there for reusing old code. But using:


--bootloaders=syslinux,grub-efi

did not enforce syslinux to be in the first place or grub-efi to be in 
the second place.


That's the specific part I meant.




So what about primary and secondary terms? Or first or
second terms?


Both are broken and confusing.

Ok...


These terms are used in two places:
* Internal variables and functions to handle bootloaders
* Information shown to the final user

I'm most convinced to use the first and non-first notation. So that the old
code that referred to LB_BOOTLOADER can just refer to: LB_FIRST_BOOTLOADER.


For what piece of code we have does it make sense to reference
LB_FIRST_BOOTLOADER when not also referencing LB_SECOND_BOOTLOADER?
Will that be extended to LB_THIRD_BOOTLOADER once x86 grows support
for coreboot or l-b grows support for some other platform with many
firmware variants?

If you set bootloaders like

LB_BOOTLOADERS="syslinux grub-efi"

then you can just do

for bootloader in $LB_BOOTLOADERS ; do some $bootloader foo

Mostly what current path does but with commas instead.


after you check that you have at most two bootloaders and if you have
more than one then only the latter one ends with -efi.


This is not a good approach. You are requesting the bootloaders to end 
in "-efi". The current approach is to name them based on the Debian 
package name. These packages do not need to end in "-efi".


My use case is the following one. The final user requests:

--bootloaders=grub-efi,syslinux

so I show him:

"Warning. You are using: syslinux as a non first bootloader. This might 
work but it is not advised."


How do I know that I have to output this message?

Because I compare the internal variable:

LB_FIRST_BOOTLOADER="grub-efi"

with the bootloader name "syslinux" and I see they are not the same one.

So, as you see I need to use:

"non first bootloader" term
and
LB_FIRST_BOOTLOADER variable.

So...

1) I don't mind renaming "non first bootloader" or LB_FIRST_BOOTLOADER 
to another terminology which makes more technical sense.
2) I prefer this approach over yours (Michal) because it's the own 
bootloader which decides if it is more suited for "first bootloader" or 
not. Let's not repeat the current binary_iso design which has many 
references to the different available binary_bootloaders available.




Thanks

Michal


adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 24/01/16 a las 16:51, Michal Suchanek escribió:

The model does not imply any ranking. "First", "Second", "Third"
could be justified, because there are lists in El Torito and
partition tables where the boot entries have to line up in sequence.

For my taste, "Main" or "Primary" too much implies a rank.



So... what about using:

FIRST_BOOTLOADER
and
SECOND_BOOTLOADER

instead of my current:

PRIMARY_BOOTLOADER
and
SECONDARY_BOOTLOADER




why not BOOTLOADERS?

The concept of being primary or secondary is just broken.

You can store 64 el-torito catalog entries on a CD. The different
entries are separated by -eltorito-alt-boot option. When you put the
CD with a BIOS and EFI entry in a machine that prefers UEFI the efi
entry is primary even when there is a CSM. When you put the CD into a
machine with legacy BIOS the BIOS entry is primary in the EFI one
useless.

There are some technical details like it is desirable to put the BIOS
entry before the EFI entry. This may be either enforced by reordering
the bootloader list automagically or just by setting the default to
syslinux,grub-efi and explaining in the docs that this is the usual
order and changing the order may potentially cause issues.

There are more technical details like the bootloader for i386 and
amd64 BIOS is the same one and on EFI it's two different binaries. Or
that when there is an EFI bootloader it's desirable to create another
e-torito entry for it as APM to boot on Macs. The technical details of
creating multiple partitions, boot entries, and whatnot are
implementation details. It's nice to document these details somewhere
for people who want to further butcher the created medium, add support
for more bootloaders, etc.

So basically what we have is support for two PC firmware types:

BIOS
EFI

and two bootloader alternatives for each firmware type

syslinux
grub

and for simlicity only one bootloader for one firmware type is
supported on single medium. Since booting on BIOS platform is well
supported with syslinux and booting on EFI platform is well-supported
with grub it is desirable to have the option to specify different
bootloader type for each supported platform and make it the default.
So rather than

BOOTLOADER=syslinux
BOOT_FIRMWARE=bios,efi

we have

BOOTLOADERS=syslinux,grub-efi

That is the bootloader name encodes the whole bootloader-firmware
pair. There is no need to specify more. It is well possible to specify
to not include 32bit efi binaries when building a 64bit CD build but
that's *-efi installation option and not a separate bootloader type.

This means the user can have 1-2 bootloaders on a single medium. And
if coreboot or whatnot support is added in the future there is
potential to have more.


Thanks

Michal



What you are describing here is what it's actually implemented in my 
patch (Well, actually the first patch version because the current one 
enforces bootloader roles). So what about primary and secondary terms? 
Or first or second terms?


These terms are used in two places:
* Internal variables and functions to handle bootloaders
* Information shown to the final user

I'm most convinced to use the first and non-first notation. So that the 
old code that referred to LB_BOOTLOADER can just refer to: 
LB_FIRST_BOOTLOADER.


Later on, if we want to improve this old code we can modify it so that 
it handles LB_BOOTLOADERS thanks to the bootloader functions directly.


I agree with you that a good approach would be to document the different 
supported or suggested bootloader combinations rather than trying to 
enforce it by code.


That would match my last thoughts which I quote here:


And, well, I have some ideas on how to add some special functions to 
binary_bootloader files so that we have some sort of Object-Oriented / Hook 
programming when defining what goes into the mkisofs options.

If you check current: binary_iso file it just relies on existing 
binary_bootloaders without having an agnostic bootloader approach.

Here it's what I'm talking about:

https://github.com/adrian15/live-build/blob/5eba3dff5a16a34c3c1eb5d54e3767339654e2d0/scripts/build/binary_iso#L111-L145

case "${LB_PRIMARY_BOOTLOADER}" in
grub)
(...)
grub-pc)
(...)

esac


We could just invent the:

grub-pc-xorriso-options()
or
syslinux-xorriso-options()

functions which would be defined in:

binary_grub-pc
or
binary_syslinux
files

and handle all of these with only 3 or 4 lines of code.

E.g. binary_iso would add '-eltorito-alt-boot' just before a second bootloader 
so that the bootloader only needs to take care of their own boot options.


Is there anyone opposed to such a big change on live-build handling of 
bootloaders?



adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

On 24 January 2016 at 00:41, adrian15  wrote:
> El 23/01/16 a las 09:21, Thomas Schmitt escribió:
>>
>> There is a fourth dimension to be expressed: Bootloader.
>> Then there is the dimension of ISO filesystem objects.
>> A user wish would contain at least
>>ISO-Object, Bootloader, Medium, Firmware, Architecture
>>
>> E.g.
>>
>>Appended Partition, with GRUB2 content, for CDROM and HDD, via EFI, on
>> i386
>>
>>ISO Data File, with SYSLINUX content, for HDD, via BIOS, on i386 and
>> amd64
>>
>> I am not sure whether this list of more or less combinable
>> dimensions is complete yet.
>>
>> Further one will want to express whether the gaps between partitions
>> should be filled, which kinds of partition tables shall emerge, ...
>> These properties are global to the ISO, not specific to a single
>> boot image. Some are combinable, some are mutally exclusive.
>>
>> (Maybe it is time to break out an UML editor.)
>
>
> I propose you to send these concerns to live-wrapper project which has just
> begun and it's advertised as highly modular by Iain. It would be quite nice
> if all these concerns were properly addressed by Iain's tool.
>
>   I am not saying that they are not welcomed to live-build but, right now, I
> think we should focus on making UEFI to boot and not re-thinking all the
> bootloader handling.
>
>> grub-mkrescue layout is rarely used for distro ISOs.
>> I blame this on the existing SYSLINUX based production software
>> for distro ISOs. In principle, a pure GRUB2 or a pure SYSLINUX
>> equipment appears to be the more reasonable choice.
>
>
> I once asked in a Debconf why the people were so stubborn to use syslinux
> instead of using grub2 (which seems technically superior to me). It would
> seem that the same people that maintain syslinux happen to maintain the
> kernel boot stack (or whatever it is called, I mean what happens just after
> the bootloader handling the execution to the kernel). So, that means, it's
> far easier to them to debug why the kernel is refusing to boot.
>

I was wondering that also. The GRUB is so cool and can do everything, right?

It also means it can crumble and fall apart in so many interesting ways.

I mean if you install grub on a LVM soft-RAID array and it then does
not boot you are wondering why it happened when it has all those RAID
modules. And you find that it has RAID modules only for the *previous*
revision of the LVM format.

So you do what you would plan for from the start with Syslinux - plug
in an USB stick and put a boot partition on it, yay.

Could have saved hours of debugging by going with the stupid limited
Syslinux from the start, right?

>>> Maybe we can just name them as:
>>> "Main bootloader"
>>> and
>>> "Alternate bootloader".
>>
>>
>> The model does not imply any ranking. "First", "Second", "Third"
>> could be justified, because there are lists in El Torito and
>> partition tables where the boot entries have to line up in sequence.
>>
>> For my taste, "Main" or "Primary" too much implies a rank.
>
>
> So... what about using:
>
> FIRST_BOOTLOADER
> and
> SECOND_BOOTLOADER
>
> instead of my current:
>
> PRIMARY_BOOTLOADER
> and
> SECONDARY_BOOTLOADER
>


why not BOOTLOADERS?

The concept of being primary or secondary is just broken.

You can store 64 el-torito catalog entries on a CD. The different
entries are separated by -eltorito-alt-boot option. When you put the
CD with a BIOS and EFI entry in a machine that prefers UEFI the efi
entry is primary even when there is a CSM. When you put the CD into a
machine with legacy BIOS the BIOS entry is primary in the EFI one
useless.

There are some technical details like it is desirable to put the BIOS
entry before the EFI entry. This may be either enforced by reordering
the bootloader list automagically or just by setting the default to
syslinux,grub-efi and explaining in the docs that this is the usual
order and changing the order may potentially cause issues.

There are more technical details like the bootloader for i386 and
amd64 BIOS is the same one and on EFI it's two different binaries. Or
that when there is an EFI bootloader it's desirable to create another
e-torito entry for it as APM to boot on Macs. The technical details of
creating multiple partitions, boot entries, and whatnot are
implementation details. It's nice to document these details somewhere
for people who want to further butcher the created medium, add support
for more bootloaders, etc.

So basically what we have is support for two PC firmware types:

BIOS
EFI

and two bootloader alternatives for each firmware type

syslinux
grub

and for simlicity only one bootloader for one firmware type is
supported on single medium. Since booting on BIOS platform is well
supported with syslinux and booting on EFI platform is well-supported
with grub it is desirable to have the option to specify different
bootloader type for each supported platform and make it the default.
So rather than

BOOTLOADER=syslinux

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

adrian15 wrote:
> I propose you to send these concerns to live-wrapper project which has just
> begun and it's advertised as highly modular by Iain.

New customers ? Welcome !


> I think we should focus on making UEFI to boot and not re-thinking all the
> bootloader handling.

I well understand that each project has its own goals and priorities.
My remarks shall serve as background info.


> I once asked in a Debconf why the people were so stubborn to use syslinux
> instead of using grub2 [...] It would
> seem that the same people that maintain syslinux happen to maintain the
> kernel boot stack

The kernels on an Installation-ISO or Live-ISO should not be
adventurous enough to make this important. The devices on which
the ISO gets presented are supposed to be oldfashioned.
(A 4K disk might cause failure. But who puts an ISO on such a disk ?)

As provider of xorriso i stay neutral towards the boot loaders.


Michal Suchanek wrote:
> I was wondering that also. The GRUB is so cool and can do everything, right?

It seems to be more apt with EFI and ISO 9660.
debian-cd needs a few hundred KB to hop out of FAT into ISO.
SYSLINUX needs another operating system to finally get out of FAT.


> I mean if you install grub on a LVM soft-RAID array

Another unlikely device type for presenting ISO 9660, too our luck.


> Could have saved hours of debugging by going with the stupid limited
> Syslinux from the start, right?

Never change a working system.
I assume that this is the true reason why nearly everybody else
still uses ISOLINUX for BIOS booting from ISO 9660.

A well maintained iLive CD system based entirely on GRUB2 would
help to find out whether this is supersticion or deep wisdom.


Have a nice day :)

Thomas

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

adrian15 wrote:
> It seems you are proposing to add like three tags: Medium?, Firmware?,
> Architecture? but I don't get how that would transform into options that the
> final user can use.

For now i only state that this is a model by which one
can describe the purpose of boot loaders in an ISO filesystem.


> And, well, how would you go into enabling or not a given bootloader given a
> Medium?, Firmware?, Architecture? combination.

I am pondering a better language for describing the boot
equipment to xorriso. It would have to be less tricky than the
zoo of inherited and self-invented mkisofs options.
Not easy.


> I'm not sure I understand why we need this.

Mainly take care not to create a terminology which contradicts
the correct model. (As far as it is identified yet.)


> Your proposal begins to make sense although I'm not 100% convinced ;).

I am not convinced either. The model matches the facts, so far.
But its straightforward implementation is not yet clear to me.

There is a fourth dimension to be expressed: Bootloader.
Then there is the dimension of ISO filesystem objects.
A user wish would contain at least
  ISO-Object, Bootloader, Medium, Firmware, Architecture

E.g.

  Appended Partition, with GRUB2 content, for CDROM and HDD, via EFI, on i386 

  ISO Data File, with SYSLINUX content, for HDD, via BIOS, on i386 and amd64

I am not sure whether this list of more or less combinable
dimensions is complete yet.

Further one will want to express whether the gaps between partitions
should be filled, which kinds of partition tables shall emerge, ...
These properties are global to the ISO, not specific to a single
boot image. Some are combinable, some are mutally exclusive.

(Maybe it is time to break out an UML editor.)


> > [SYSLINUX]
> > Did you try whether it boots via EFI if no BIOS boot equipment
> > is in the ISO ?
> Well, I tried: kvm -bios /usr/share/ovmf/OVMF.fd -boot d -cdrom file.iso .

More we cannot expect. :)
The fact that there is a minimal operating system in the EFI
System Partition explains why it works.


> > debian-cd has an MBR partition 0xef and a GPT
> > partition which point to that EFI boot image. (The GPT is to be
> > ignored by all sane EFI implementations.)

> Ok. Did you mean "NOT to be ignored"?

No. The GPT is surplus, according to UEFI 2.4 specs.
Either you have an MBR partition 0xef or you have GPT. Not both.

But Matthew Garrett had reason to produce it.
Re-reading his blog
  http://mjg59.dreamwidth.org/11285.html
i get the suspicion, that he first tried GPT alone, then added
MBR partition 0xef, and so invalidated his original GPT approach. 


> Maybe that's why the syslinux-efi does not boot. I could give it a try
> changing the partition type and see what happens.

Rather not. The presence of MBR partition 0xef should cause
EFI to simply not look for GPT partitions.


> > I recently tested on my Sid VM with various combinations of grub-pc,
> > grub-efi-ia32-bin, and grub-efi-amd64-bin. All together, pair-wise, alone.
> > They all booted with qemu via its default BIOS and via OVMF (as EFI).

> Did you try to boot them either as a cdrom or as a hard disk? I think that
> EFI + hard disk was the one failed for me.

Hopefully i tested all reasonable combinations. (But i begin to doubt
that i tried an i386 VM. Will have to check.)


> That's interesting. So we have Fedora, debian-cd, grub-mkimage ways of
> dealing with UEFI. Probably Ubuntu has also its own way of handling it too.

Fedora, Debian, Ubuntu use the Matthew Garrett layout.
Debian and Ubuntu omit the HFS+ aspect of that layout, though.

OpenSuSE is different. They have a mountable ISO partition after
the EFI System Partition. For that stunt they cut off the head
of a first ISO and prepend it to a second, reproducibly built
ISO, which lacks the EFI partition data file.
(In american length units, Frankenstein castle is not very far
 from Nuremberg.)

grub-mkrescue layout is rarely used for distro ISOs.
I blame this on the existing SYSLINUX based production software
for distro ISOs. In principle, a pure GRUB2 or a pure SYSLINUX
equipment appears to be the more reasonable choice.


> So you don't like the primary or secondary terms.
> How does xorriso name them?

They have no partitcular name in the docs.
Joerg Schilling obviously considered them "alternative" when he
invented option name -eltorito-alt-boot.
We should not put too much weight on that options naming tradition.
It grew slowly and in part cluelessly. (Joerg's and mine alike.)


> Maybe we can just name them as:
> "Main bootloader"
> and
> "Alternate bootloader".

The model does not imply any ranking. "First", "Second", "Third"
could be justified, because there are lists in El Torito and
partition tables where the boot entries have to line up in sequence.

For my taste, "Main" or "Primary" too much implies a rank.


> So that we can force a given bootloader to be used only as a "Main eltorito
entry" ?

Currently this would be done by 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

after checking El Torito specs, i have to correct my statements
about ranking of El Torito boot images.

The first entry in the boot catalog not only has the title
"Initial" but "Initial/Default".

"4.4 Boot Entry Selection
If the CD has several boot entries, a default entry which boots a 
election program may be provided as the Default/Initial catalog entry.
This image will usually be a floppy image which loads a program that
selects the proper boot image by examining the system configuration or
questioning the user."

Insofar, the first boot image in the catalog is something special.
It is to be preferred over the other boot images with the same
Platform Id. (Nevertheless, it is not obliged to really offer
booting of the others of the same Platform Id.)

I would plan for only one El Torito boot image per platform:
0 = 80x86 BIOS , 0xef = EFI.
But if there is a use case for offering BIOS boot images from GRUB2
and ISOLINUX in the same El Torito catalog, then the sequence matters
if one of them occupies the first entry in the catalog.
If you put the EFI boot image first, then both BIOS boot images get
equal rank.


In the light of this, it seems wise to stay with the sequence that is
tested since a few years:
First the BIOS boot image, then the EFI boot image, no further BIOS
boot images.

This makes it clear to the firmware:
BIOS will hop on the first entry, because it has the right Platform Id
and is Default. BIOS will most probably not enforce that the first
boot image is a floppy image, because there is no other BIOS boot image.
(You always have to expect that the BIOS programmers read the specs
with tired eyes.)
EFI will not hop on the Default entry, because the Platform Id
does not match.

The MBR x86 BIOS code, too, would have to decide on which BIOS boot
image binary to hop. If there is only one, the decision is easy.


Have a nice day :)

Thomas

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 23/01/16 a las 09:21, Thomas Schmitt escribió:

There is a fourth dimension to be expressed: Bootloader.
Then there is the dimension of ISO filesystem objects.
A user wish would contain at least
   ISO-Object, Bootloader, Medium, Firmware, Architecture

E.g.

   Appended Partition, with GRUB2 content, for CDROM and HDD, via EFI, on i386

   ISO Data File, with SYSLINUX content, for HDD, via BIOS, on i386 and amd64

I am not sure whether this list of more or less combinable
dimensions is complete yet.

Further one will want to express whether the gaps between partitions
should be filled, which kinds of partition tables shall emerge, ...
These properties are global to the ISO, not specific to a single
boot image. Some are combinable, some are mutally exclusive.

(Maybe it is time to break out an UML editor.)


I propose you to send these concerns to live-wrapper project which has 
just begun and it's advertised as highly modular by Iain. It would be 
quite nice if all these concerns were properly addressed by Iain's tool.


  I am not saying that they are not welcomed to live-build but, right 
now, I think we should focus on making UEFI to boot and not re-thinking 
all the bootloader handling.



grub-mkrescue layout is rarely used for distro ISOs.
I blame this on the existing SYSLINUX based production software
for distro ISOs. In principle, a pure GRUB2 or a pure SYSLINUX
equipment appears to be the more reasonable choice.


I once asked in a Debconf why the people were so stubborn to use 
syslinux instead of using grub2 (which seems technically superior to 
me). It would seem that the same people that maintain syslinux happen to 
maintain the kernel boot stack (or whatever it is called, I mean what 
happens just after the bootloader handling the execution to the kernel). 
So, that means, it's far easier to them to debug why the kernel is 
refusing to boot.


I am personally in a favour of GRUB2-only Live CDs so that it's much 
easier to have native Super Grub2 Disk in them :) .



Maybe we can just name them as:
"Main bootloader"
and
"Alternate bootloader".


The model does not imply any ranking. "First", "Second", "Third"
could be justified, because there are lists in El Torito and
partition tables where the boot entries have to line up in sequence.

For my taste, "Main" or "Primary" too much implies a rank.


So... what about using:

FIRST_BOOTLOADER
and
SECOND_BOOTLOADER

instead of my current:

PRIMARY_BOOTLOADER
and
SECONDARY_BOOTLOADER

?

I could even add a third bootloader if needed. And, well, I have some 
ideas on how to add some special functions to binary_bootloader files so 
that we have some sort of Object-Oriented / Hook programming when 
defining what goes into the mkisofs options.


If you check current: binary_iso file it just relies on existing 
binary_bootloaders without having an agnostic bootloader approach.


Here it's what I'm talking about:

https://github.com/adrian15/live-build/blob/5eba3dff5a16a34c3c1eb5d54e3767339654e2d0/scripts/build/binary_iso#L111-L145

case "${LB_PRIMARY_BOOTLOADER}" in
grub)
(...)
grub-pc)
(...)

esac


We could just invent the:

grub-pc-xorriso-options()
or
syslinux-xorriso-options()

functions which would be defined in:

binary_grub-pc
or
binary_syslinux
files

and handle all of these with only 3 or 4 lines of code.

E.g. binary_iso would add '-eltorito-alt-boot' just before a second 
bootloader so that the bootloader only needs to take care of their own 
boot options.



Is there anyone opposed to such a big change on live-build handling of 
bootloaders?



adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 21/01/16 a las 10:13, Thomas Schmitt escribió:

Hi,

For the nomenclature: "USB" alone is misleading because there are
also optical drives attachable to USB. Better distinguish the boot
media families CDROM (CD, DVD, BD) and HDD (hard disk, USB stick,
memory card, ...).

Sorry. I was refering to USB stick then.



adrian15 wrote:

Grub-pc would be the one installed to be boot but syslinux files would be
there for Multi-USB tools to know how to understand the iso and put it into
an USB.


You mean the capability to boot the ISO via BIOS from USB stick ?
(Known with SYSLINUX as "isohybrid".)

grub-pc is supposed to be bootable that way too. Have a look at the
result of grub-mkrescue.
When running xorriso -as mkisofs, the decisive trick of grub-mkrescue
for BIOS bootability from HDD is:

   --grub2-mbr /usr/lib/grub/i386-pc/boot_hybrid.img

(It is obvious that only one bootloader can put its MBR at the
start of the ISO. So additional -isohybrid-mbr for SYSLINUX
would conflict.)

CDROM bootability via BIOS is caused by

-b boot/grub/i386-pc/eltorito.img \
  -no-emul-boot -boot-load-size 4 -boot-info-table \
  --grub2-boot-info

(Options learned from grub-mkrescue tests on Debian Sid.)
Well, yes, I know that you can use grub-pc to be able to boot a single 
image that boots from either a hard disk / USB stick / CDROM media in a 
BIOS system.


I did use that setup for building earlier versions of Rescatux (which 
included native Super Grub2 Disk in them).


My first will on putting both syslinux and grub-pc in the same media had 
not the purpose of making it bootable with two methods but rather to:

* Default boot into Grub2 (so that Super Grub2 Disk could be used natively).
* Confuse multi usb stick tools such as yumi to think that the iso was 
actually isolinux based. That way they can easily extract iso contents 
and put it into a multi usb stick device.


As I said before this was the original idea that made me add the primary 
and secondary bootloaders concept to live-build.




--bootloaders="syslinux,grub-efi,syslinux-efi"


Here i wonder whether (or how) your syslinux-efi boots out of an
ISO 9660 filesystem. My current knowledge is that SYSLINUX cannot
do this.
Yeah, you are right. You can take a look at what I have left from 
Hertzog patch here:


https://github.com/adrian15/live-build/commit/1397b6fc3dbf7d7f1e6a077a37b9f2e6ee7d7f39

His workaround for syslinux-efi not being able to boot out of an ISO 
9660 filesystem is to put the bootloader files, kernel and initrd on the 
efi image.


That way the efi image should be able to load the initrd in the ram so 
that the rest of the system can be found.


(Back in the day in some old systems you had to do this trick, e.g., add 
these files to the old IDE attached disk because BIOS was unable to boot 
the SATA attached disk. But the initrd, yes, knew how to handle it.)




Now primary means: "First lure" and secondary means "Second lure" by your
definition.


There are normally two lures per firmware-hardware combination.
Depending on the medium, the lures are recognized in El Torito,
or in MBR, or in partition tables.

In general we have theses dimensions
   {Medium: CDROM, HDD} x
   {Firmware: BIOS, EFI, ... exotic others ...} x
   {Hardware: i386, amd64, ... exotic others ...}

Not all tuples chosen from these sets are valid and not all
valid tuples can be combined in one ISO filesystem.
But the 8 main combinations for PC hardware are valid and
combinable.

I would avoid ranking terms like "first" or "primary".
Job descriptions for bootloaders could rather look like
   (CDROM + HDD, BIOS, i386 + amd64)
   (HDD, EFI, i386)
   (HDD, EFI, amd64)

Some of them can hardly be separated from each other.
E.g. (HDD, BIOS, i386) and (HDD, BIOS, amd64) have identical
technical properties.
I am sorry. I agree with you but I'm not going to change my current 
naming without further input. I would need a patch from you or a 
concrete new naming system with examples that replace the current names.


It seems you are proposing to add like three tags: Medium?, Firmware?, 
Architecture? but I don't get how that would transform into options that 
the final user can use.


And, well, how would you go into enabling or not a given bootloader 
given a Medium?, Firmware?, Architecture? combination.


I mean, don't get me wrong, I want to code this right but I'm not sure I 
understand why we need this. I mean, I know that the medium already 
choosing by either binary_hdd or binary_iso . And, the architectures can 
also be choosen if needed (LB_ARCHITECTURES if I'm not mistaken). Not 
sure why you want to question of all that out of a sudden.


Anyway, as I said, maybe giving me concrete examples might make me 
change my mind. If we are going to implement UEFI support into 
live-build let's do it right.



Given the way that both grub-efi and syslinux-efi use:
/efi/boot/bootia32.efi
and
/efi/boot/bootx64.efi
the last one to be run would overwrite the first one efi 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 21/01/16 a las 12:57, Thomas Schmitt escribió:



adrian15 wrote:

Do you mean if you have:
xorriso bunch-of-options-1 -eltorito-alt-boot bunch-of-options-2
you could just re-arrange them as:
xorriso bunch-of-options-2 -eltorito-alt-boot bunch-of-options-1
and it would be fine?



From the view of xorriso and El Torito specs this is ok.

You have to keep the modifier options (e.g. -boot-info-table) in
the same -eltorito-alt-boot department as their main options (e.g. -b).

The program isohybrid of SYSLINUX expects BIOS in El Torito catalog
entry 1, EFI in entry 2, and HFS+ in entry 3.
So with genisoimage + isohybrid, you'd have to keep the sequence.

One never knows what firmware does with such changes. Normally
it should accept any sequence. But tradition is: BIOS first, EFI second.


I see. So when using syslinux it's better for it to be the primary 
bootloader. Ok.


And, if I'm not mistaken doing a:

--bootloaders="grub-efi" should be quite easy.

That way people that want EFI only images would be able to make them. In 
order to do that I would just need to implement the primary bootloader 
behaviour of grub-efi and, well, remove the current grub-efi as a 
secondary bootloader role enforcement.


I might finally do that.


I wonder what EFI firmware would hop on an MBR partition of type 0x01.
An EFI System Partition in MBR should have type 0xef to be recognized.

That needs to be asked to Hertzog. I just tried to use his original work.


Did you try whether it boots via EFI if no BIOS boot equipment
is in the ISO ?

Well, I tried: kvm -bios /usr/share/ovmf/OVMF.fd -boot d -cdrom file.iso .

I probably need to do more tests later.


If I'm not mistaken Hertzog removed that option because you suggested to
remove it on:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731709#66
Are you requesting to put this option back ?


Only if you augment the boot equipment by a third boot image that
contains a small HFS+ filesystem. (See Fedora Live CD.)
As long as there is no HFS+, the option -isohybrid-apm-hfsplus
is inappropriate but seems to do no harm. (See Debian amd64 netinst.)


I see. So the Fedora Live CD would seem even better. I wonder why the 
debian-cd team did not consider adding this third boot image with a 
small HFS+ filesystem. I'll probably ask them in the irc.



There are two ways known to me how to get a boot path via HFS+:

The debian-cd method is one of them? Or is it not?


debian-cd has no HFS+ boot image. This makes it different from
Fedora Live CD.

Understood.


The xorriso run could be easily adapted to include a HFS+ image.
But i do not know how Matthew Garrett produced this image.
I was doing some research on SElinux support and I found these set of 
tools for creating the Fedora Live CDs:


https://github.com/rhinstaller/livecd-tools

It's quite nice learning how other build their own live cd systems. 
Anyways, back to the main subject, I have no idea if Matthew Garrett 
used this tool or not.



One would have to investigate its content and find some Mac
filesystem tool to produce such content.

Probably.


The current debian-cd ( debian-8.2.0-amd64-i386-netinst.iso ) xorriso
options are: [well known from file /.disk/mkisofs in the ISO]
where you can see they use:
-isohybrid-apm-hfsplus
So... this is not useful for creating an HFS+ image file?


No. It just announces the EFI boot image in an Apple Partition Map.
This might be of use for GRUB2 if no other partition table points
to that image. But debian-cd has an MBR partition 0xef and a GPT
partition which point to that EFI boot image. (The GPT is to be
ignored by all sane EFI implementations.)

Ok. Did you mean "NOT to be ignored"?




What does this option do instead then?


It is surplus, currently.



Maybe I'm just repeating
what you are going to saying later in this email about how Grub2 handles
boot in Mac systems.


Regreattably i can only forward the rumor that Some (TM) Macs
do not boot without HFS+ and its blessed files.

They are in the time window between Apple Inc. giving up PowerPC
and Apple Inc. adopting EFI.
I have one of them in that window. I'm not sure about the blessing being 
needed though. Maybe I have disabled it. Not sure.




A) Are you complaining about syslinux-efi on:
-append_partition 2 0x01
that should be:
-append_partition 2 0xef
instead?


Yes. UEFI 2.4 5.2.2 says about booting from MBR partitions:
"* 0xEF (i.e., UEFI System Partition) defines a UEFI system partition."

The presence of GPT would have to be announced by a "protective MBR"
partition of type 0xee with start at LBA 1. Any MBR partition type other
than 0x00 and 0xee prevents the recognition of GPT.
Maybe that's why the syslinux-efi does not boot. I could give it a try 
changing the partition type and see what happens.



grub-mkrescue -o output.iso minimal_dir

Last time I checked it did not work ok in Debian because the commit
the grub package was based was too old.


I recently tested on my Sid VM with various combinations of 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 21/01/16 a las 10:13, Thomas Schmitt escribió:


Now primary means: "First lure" and secondary means "Second lure" by your
definition.


There are normally two lures per firmware-hardware combination.
Depending on the medium, the lures are recognized in El Torito,
or in MBR, or in partition tables.

In general we have theses dimensions
   {Medium: CDROM, HDD} x
   {Firmware: BIOS, EFI, ... exotic others ...} x
   {Hardware: i386, amd64, ... exotic others ...}

Not all tuples chosen from these sets are valid and not all
valid tuples can be combined in one ISO filesystem.
But the 8 main combinations for PC hardware are valid and
combinable.

I would avoid ranking terms like "first" or "primary".
Job descriptions for bootloaders could rather look like
   (CDROM + HDD, BIOS, i386 + amd64)
   (HDD, EFI, i386)
   (HDD, EFI, amd64)

Some of them can hardly be separated from each other.
E.g. (HDD, BIOS, i386) and (HDD, BIOS, amd64) have identical
technical properties.


I was thinking on this. So... what if there is a fourth dimension for 
choosing if the job description goes first or second in the mkisofs command?


E.g.:
xorriso bunch-of-options-1 -eltorito-alt-boot bunch-of-options-2
versus
xorriso bunch-of-options-2 -eltorito-alt-boot bunch-of-options-1

So you don't like the primary or secondary terms.

How does xorriso name them?

Given that man xorrisofs talks about:

-eltorito-alt-boot

maybe we can just name them as:

"Main bootloader"
and
"Alternate bootloader".

Or maybe even better:

"Main eltorito entry"
and
"Alternate eltorito entry"
?

So that we can force a given bootloader to be used only as a "Main 
eltorito entry" ?



What do you think about this idea?


adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

Despite of being discussing how to implement all of these properly. I 
feel it's right to show you my current work so that you can comment on it.


I attach my updated patches.

1) The main differences from my original patches are:

* I use more additional bootloader functions
* The code ensures that each bootloader can be used in its role, being 
it a primary bootloader o a secondary bootloader. That way we avoid the 
user being able to generate non bootable isos.


2) For the time being I am not maintaining the syslinux-efi branch till 
we manage to get a consensus on this one. Then I will rebase it, change 
the partition type and re-test it.


3) As I have mentioned in another message I plan to work on giving the 
user the ability of creating isos using:

--bootloaders="grub-efi" thus using grub-efi as a primary bootloader.

4) The patches can be found as a git branch (based on live-build master 
branch) here:


https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd_rebased_4


adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/
commit ff8206aea1985760dfea9ec94b93686a242f137e
Author: Adrian Gibanel Lopez 
Date:   Mon Jan 18 03:04:00 2016 +

functions/default.sh : Define LB_PRIMARY_BOOTLOADER at the Set_defaults 
function which it's the right place where to do it

diff --git a/functions/defaults.sh b/functions/defaults.sh
index ddf4b19..334984f 100755
--- a/functions/defaults.sh
+++ b/functions/defaults.sh
@@ -537,6 +537,8 @@ Set_defaults ()
esac
fi
 
+   LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 
}')
+
# Setting checksums
case "${LB_MODE}" in
progress-linux)
@@ -845,9 +847,6 @@ Check_defaults ()
fi
fi
 
-
-   LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 
}')
-
if [ "${LB_PRIMARY_BOOTLOADER}" = "syslinux" ]
then
# syslinux + fat or ntfs, or extlinux + ext[234] or btrfs
commit b3ec155600f79b6ac078ae8003e806735044a372
Author: Adrian Gibanel Lopez 
Date:   Wed Jan 20 00:53:53 2016 +0100

Remove repeated LB_PRIMARY_BOOTLOADER definition

diff --git a/scripts/build/binary_hdd b/scripts/build/binary_hdd
index d0db382..0c9c5af 100755
--- a/scripts/build/binary_hdd
+++ b/scripts/build/binary_hdd
@@ -67,8 +67,6 @@ do
esac
 done
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 case ${LB_PRIMARY_BOOTLOADER} in
syslinux)
case ${LB_BINARY_FILESYSTEM} in
diff --git a/scripts/build/binary_iso b/scripts/build/binary_iso
index d8b1553..67dfc85 100755
--- a/scripts/build/binary_iso
+++ b/scripts/build/binary_iso
@@ -107,8 +107,6 @@ then
XORRISO_OPTIONS="${XORRISO_OPTIONS} -V \"${LB_ISO_VOLUME}\""
 fi
 
-LB_PRIMARY_BOOTLOADER=$(echo "${LB_BOOTLOADERS}" | awk -F, '{ print $1 }')
-
 # Handle xorriso architecture specific options
 case "${LB_PRIMARY_BOOTLOADER}" in
grub)
commit e0abb8214b9ff71630220a24564a8770cea8658e
Author: Adrian Gibanel Lopez 
Date:   Mon Jan 18 03:07:48 2016 +

* Added: functions/bootloaders.sh . It has new bootloader functions that 
are heavily used in efi scenarios where a bootloader can act as a primary or a 
secondary bootloader.

Since the introduction of the new switch:

--bootloaders

you can setup it like this:

--bootloaders=syslinux,grub-efi

.

This means that syslinux is the primary bootloader and grub-efi is the 
secondary bootloader.

* Added new bootloader functions: Check_Non_Primary_Bootloader and 
Check_Non_Secondary_Bootloader.

These functions let each one of the bootloaders abort the build because
they cannot perform a role either as a primary bootloader or as a secondary 
bootloader.

* Added bootloader functions: Check_Primary_Bootloader_Role, 
Check_Secondary_Bootloader_Role and Check_Any_Bootloader_Role

These functions let bootloaders to force their default role in a single 
line.

diff --git a/functions/bootloaders.sh b/functions/bootloaders.sh
new file mode 100644
index 000..0829903
--- /dev/null
+++ b/functions/bootloaders.sh
@@ -0,0 +1,128 @@
+#!/bin/sh
+
+## live-build(7) - System Build Scripts
+## Copyright (C) 2016 Adrian Gibanel Lopez 
+##
+## This program comes with ABSOLUTELY NO WARRANTY; for details see COPYING.
+## This is free software, and you are welcome to redistribute it
+## under certain conditions; see COPYING for details.
+
+Is_Primary_Bootloader ()
+{
+   EVAL_PRIMARY_BOOTLOADER="${1}"
+
+   if [ "${LB_PRIMARY_BOOTLOADER}" = "${EVAL_PRIMARY_BOOTLOADER}" ]
+   then
+   return 0
+   else
+   return 1
+   fi
+
+}
+

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

i wrote:
> http://bazaar.launchpad.net/~libburnia-team/libisofs/scdbackup/view/head:/doc/boot_sectors.txt

adrian15 wrote:
> I want to document the different ways of how BIOS boot, UEFI boot and
> Secure Boot work and that might be helpful.

Secure Boot is not covered. (I am not sure whether it belongs there.
Actually i don't know enough about it.)


adrian15 wrote:
> Do you mean if you have:
> xorriso bunch-of-options-1 -eltorito-alt-boot bunch-of-options-2
> you could just re-arrange them as:
> xorriso bunch-of-options-2 -eltorito-alt-boot bunch-of-options-1
> and it would be fine?

>From the view of xorriso and El Torito specs this is ok.
You have to keep the modifier options (e.g. -boot-info-table) in
the same -eltorito-alt-boot department as their main options (e.g. -b).

The program isohybrid of SYSLINUX expects BIOS in El Torito catalog
entry 1, EFI in entry 2, and HFS+ in entry 3.
So with genisoimage + isohybrid, you'd have to keep the sequence.

One never knows what firmware does with such changes. Normally
it should accept any sequence. But tradition is: BIOS first, EFI second.


> > I wonder what EFI firmware would hop on an MBR partition of type 0x01.
> > An EFI System Partition in MBR should have type 0xef to be recognized.
> That needs to be asked to Hertzog. I just tried to use his original work.

Did you try whether it boots via EFI if no BIOS boot equipment
is in the ISO ?

Raphaël ? Can you shed light on this ?
Does the 0x01 partition stem from the "Firmware Partition" of mini.iso ?
(https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776317
reports about quite unintentional boot success via that partition.)


> > (Does "syslinux-efi" mean that you can boot from ISO via EFI
> > and SYSLINUX stuff to an operating system ? That would be new.)
> I don't know what you mean by 'SYLINUX stuff to an operating system.

Afaik, SYSLINUX begins to boot from an EFI System Partition inside
or after the ISO filesystem. It does not get to booting the operating
system files in the ISO filesystem, though.


> The EFI image needs to include the configuration file

Given the reports on sysli...@zytor.com i'd expect that enough of
operating system has to reside in the FAT filesystem so it is able
to boot to the capability to find and read the ISO filesystem.

But in practice, all EFI bootable ISOs known to me use GRUB for the job.
(One may boot a kernel directly from EFI, without intermediate boot
loader. Examples are rare but seem to work.)


> If I'm not mistaken Hertzog removed that option because you suggested to
> remove it on:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731709#66
> Are you requesting to put this option back ?

Only if you augment the boot equipment by a third boot image that
contains a small HFS+ filesystem. (See Fedora Live CD.)
As long as there is no HFS+, the option -isohybrid-apm-hfsplus
is inappropriate but seems to do no harm. (See Debian amd64 netinst.)


> > There are two ways known to me how to get a boot path via HFS+:
> The debian-cd method is one of them? Or is it not?

debian-cd has no HFS+ boot image. This makes it different from
Fedora Live CD.

The xorriso run could be easily adapted to include a HFS+ image.
But i do not know how Matthew Garrett produced this image.
One would have to investigate its content and find some Mac
filesystem tool to produce such content.


> The current debian-cd ( debian-8.2.0-amd64-i386-netinst.iso ) xorriso
> options are: [well known from file /.disk/mkisofs in the ISO]
> where you can see they use:
> -isohybrid-apm-hfsplus
> So... this is not useful for creating an HFS+ image file?

No. It just announces the EFI boot image in an Apple Partition Map.
This might be of use for GRUB2 if no other partition table points
to that image. But debian-cd has an MBR partition 0xef and a GPT
partition which point to that EFI boot image. (The GPT is to be
ignored by all sane EFI implementations.)


> What does this option do instead then?

It is surplus, currently.


> Maybe I'm just repeating
> what you are going to saying later in this email about how Grub2 handles
> boot in Mac systems.

Regreattably i can only forward the rumor that Some (TM) Macs
do not boot without HFS+ and its blessed files.

They are in the time window between Apple Inc. giving up PowerPC
and Apple Inc. adopting EFI.


> > (Options picked out of proposal of
> > xorriso -hfsplus on \
> > -indev Fedora-Live-Xfce-x86_64-rawhide-20150704.iso \
> > -report_el_torito as_mkisofs

> Interesting. I did not know that you could request what an ISO
> 'layout/configuration' was.

This is a xorriso feature which is present in the Sid package
xorriso-1.4.2 but not in xorriso-1.3.2 as of Debian 8.

You can request a listing of the boot equipment

  xorriso -hfsplus on -indev ...iso \
  -report_el_torito plain -report_system_area plain

and you can request option proposals by above
  -report_el_torito as_mkisofs


> A) Are you complaining 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

For the nomenclature: "USB" alone is misleading because there are
also optical drives attachable to USB. Better distinguish the boot
media families CDROM (CD, DVD, BD) and HDD (hard disk, USB stick,
memory card, ...). 


adrian15 wrote:
> Grub-pc would be the one installed to be boot but syslinux files would be
> there for Multi-USB tools to know how to understand the iso and put it into
> an USB.

You mean the capability to boot the ISO via BIOS from USB stick ?
(Known with SYSLINUX as "isohybrid".)

grub-pc is supposed to be bootable that way too. Have a look at the
result of grub-mkrescue.
When running xorriso -as mkisofs, the decisive trick of grub-mkrescue
for BIOS bootability from HDD is:

  --grub2-mbr /usr/lib/grub/i386-pc/boot_hybrid.img

(It is obvious that only one bootloader can put its MBR at the
start of the ISO. So additional -isohybrid-mbr for SYSLINUX
would conflict.)

CDROM bootability via BIOS is caused by

   -b boot/grub/i386-pc/eltorito.img \
 -no-emul-boot -boot-load-size 4 -boot-info-table \
 --grub2-boot-info 

(Options learned from grub-mkrescue tests on Debian Sid.)


> --bootloaders="syslinux,grub-efi,syslinux-efi"

Here i wonder whether (or how) your syslinux-efi boots out of an
ISO 9660 filesystem. My current knowledge is that SYSLINUX cannot
do this.


> Now primary means: "First lure" and secondary means "Second lure" by your
> definition.

There are normally two lures per firmware-hardware combination.
Depending on the medium, the lures are recognized in El Torito,
or in MBR, or in partition tables. 

In general we have theses dimensions
  {Medium: CDROM, HDD} x
  {Firmware: BIOS, EFI, ... exotic others ...} x
  {Hardware: i386, amd64, ... exotic others ...}

Not all tuples chosen from these sets are valid and not all
valid tuples can be combined in one ISO filesystem.
But the 8 main combinations for PC hardware are valid and
combinable.

I would avoid ranking terms like "first" or "primary".
Job descriptions for bootloaders could rather look like
  (CDROM + HDD, BIOS, i386 + amd64)
  (HDD, EFI, i386)
  (HDD, EFI, amd64)

Some of them can hardly be separated from each other.
E.g. (HDD, BIOS, i386) and (HDD, BIOS, amd64) have identical
technical properties.

  
> Given the way that both grub-efi and syslinux-efi use:
> /efi/boot/bootia32.efi
> and
> /efi/boot/bootx64.efi
> the last one to be run would overwrite the first one efi files.

You could combine bootia32.efi from the one boot loader and
bootx64.efi from the other boot loader, if you find a use case
for that stunt.
Without such a use case, i would decide for one boot loader per
firmware. Just to keep things simple.


> So... How would you go about for a:
> --bootloaders="syslinux,grub-efi,syslinux-efi"

I'd state that you don't need syslinux-efi if you have grub-efi
which is much better exercised with ISO 9660.

As said, nobody ever reported about success of SYSLINUX from
CDROM via EFI. I dimly remember that hpa, the author of SYSLINUX,
once stated on sysli...@zytor.com that the capability was missing
in SYSLINUX EFI to hop from the FAT filesystem into the ISO filesystem.


> they would overwrite their /efi/boot/boot*efi files ?

The EFI startup binaries of grub-efi and syslinux-efi would conflict
inside your FAT filesystem, indeed. There are supposed to be ways to
solve this conflict and offer the choice at boot time. For that you'd
need to have own EFI startup binaries which then hop on the programs
of one of the offered boot loaders.

But, well, what would be the use case for such a choice ?


Have a nice day :)

Thomas

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 18/01/16 a las 12:57, Michal Suchanek escribió:


Unless
the user requests two bootloaders that are incompatible the medium can
be created.

Ignoring bootloaders is not a good idea. If the user requested
something that is not possible the build should report an error and
stop.


Yeah, that it's hard to implement from the perspective of a single
bootloader script. So I decided not to implement it.

You could add an additional script or function named:
check_compatible_bootable_pairs (as you propose) but then you need to
maintain that fictional database each time a new bootloader gets in.

That's why I decide to avoid it using it.

What it's straight-forward to implement is one of the only-secondary
bootloaders finding themselves as a primary bootloader and outputting a
warning message. But, I'm not sure what to put there as a warning. Any
suggestion?


Unsupported bootloader combination or whatever.

It should not be a warning however. The script should avoid making
unbootable medium.

Thanks

Michal


I have implemented: Check_Primary_Bootloader_Role, 
Check_Secondary_Bootloader_Role functions for avoiding non supported 
bootloader combinations on:


https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd_rebased

So that part is solved.

(I'll send a proper rebased set of patches in the future).

adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 18/01/16 a las 13:38, Thomas Schmitt escribió:

Hi,

adrian15 wrote:

* What is it a secondary bootloader?
It's what happens when you request mkisofs that your bootloader to be
boot in second place or as a second partition. I don't know how it
actually works.


An ISO may contain several lures for boot firmware.

If it is booted from CD/DVD/BD, then the lures are in the El Torito
boot catalog. In case of debian-cd it contains two entries which
point to boot images. One entry is marked as suitable for BIOS and
one marked as suitable for EFI. The boot firmware then picks what
it deems right and starts it, or offers it to the user for starting.

Interesting.


If the ISO is presented on USB stick or alike, then BIOS looks
for the magic number of an MBR and if so, mindlessly executes
the x86 machine code that starts at byte 0 of the ISO.
This code then hops onto the same boot code that is advertised
by El Torito (because xorriso or isohybrid patched its block
address into the MBR code).

Aha.


EFI looks on USB stick for an MBR partition of type 0xef or
for a single partition of type 0xee. In the latter case it assumes
the presence of GPT and looks for a GPT partition with type GUID
28732ac11ff8d211ba4b00a0c93ec93b.
Inside the found partition it expects a FAT filesystem and
particular binaries for each supported processor archictecture.
E.g. \EFI\BOOT\BOOTX64.EFI

The El Torito EFI boot image has the same content specification
as the EFI System Partition. Therefore both boot paths can share
the same data.

My cheat sheet is at
   
http://bazaar.launchpad.net/~libburnia-team/libisofs/scdbackup/view/head:/doc/boot_sectors.txt


Great! I want to document the different ways of how BIOS boot, UEFI boot 
and Secure Boot work and that might be helpful.




-


So, I guess the -eltorito-alt-boot does the magic but I'm not sure.
Maybe someone else can explain it better than me.


-eltorito-alt-boot is simply a separator between the options of
El Torito boot images.
I assume your first boot image is announced by -b and gets
the usual options
-no-emul-boot -boot-load-size 4 -boot-info-table

Before the announcement of the EFI boot image begins, option
-eltorito-alt-boot is needed to protect your -b and its helpers
from being overwritten.


That it's quite interesting. Do you mean if you have:

xorriso bunch-of-options-1 -eltorito-alt-boot bunch-of-options-2

you could just re-arrange them as:

xorriso bunch-of-options-2 -eltorito-alt-boot bunch-of-options-1

and it would be fine?

So that it's not strictly necessary for syslinux or grub-pc to be a 
primary bootloader (or first lure in your definition) ?


That would also simplify my contributed code to live-build by not 
needing to handle if a bootloader is primary/secondary or not.





So in grub-efi we just add to the xorriso options:
-eltorito-alt-boot \
  -e boot/grub/efi.img \
  -no-emul-boot \
  -isohybrid-gpt-basdat \
  -isohybrid-apm-hfsplus


Here a second boot image gets announced by -e and the next three
options apply to it.

Ok.




And in syslinux-efi (currently) we add:
-eltorito-alt-boot \
--efi-boot boot/efi.img \


Option --efi-boot is a shortcut for
   -eltorito-alt-boot -e ... -no-emul-boot -eltorito-alt-boot
normally used by grub-mkrescue. The SYSLINUX community prefers -e.
Knowing that I will probably expand that so that the syslinux-efi and 
grub-efi options are more similar and we can compare them in a more easy 
manner.




-append_partition 2 0x01 \
  binary/boot/efi.img


I wonder what EFI firmware would hop on an MBR partition of type 0x01.
An EFI System Partition in MBR should have type 0xef to be recognized.

That needs to be asked to Hertzog. I just tried to use his original work.


It looks like this variant will not get GPT. That's fully compliant
to UEFI 2.4. Just the MBR partition type should be 0xef.

(Does "syslinux-efi" mean that you can boot from ISO via EFI
and SYSLINUX stuff to an operating system ? That would be new.)
I don't know what you mean by 'SYLINUX stuff to an operating system. I 
understand that Hertzog work let's you boot into the cd booting by the 
means of EFI. The EFI image needs to include the configuration file (as 
opossed to the grub-efi implementation which only stores its 
configuration file in the CD/DVD/BD media).


---

Above options do not produce HFS+ with blessing.
Option -isohybrid-apm-hfsplus causes an Apple Partition Map entry
which points to the data content of boot/grub/efi.img.


If I'm not mistaken Hertzog removed that option because you suggested to 
remove it on:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731709#66

Are you requesting to put this option back ?



There are two ways known to me how to get a boot path via HFS+:
Fedora Live CD as of Matthew Garrett
or grub-mkrescue as of Vladimir Serbinenko.


The debian-cd method is one of them? Or is it not?

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 18/01/16 a las 14:00, Thomas Schmitt escribió:

So any bootloader is made primary by leaving out -eltorito-alt-boot.


There is no "primary" or "secondary" on the level of boot images
and loaders. (Of course you may call them this way in your project.)

There are first, second, third ... El Torito boot images or MBR, GPT,
APM partition entries.
The sequence is just needed because not all can be first in the list.

The boot firmware inspects the medium for its known lures,
picks a tasty one, and follows it to the boot loader or kernel.


So, yes, I'm the one who invented the term primary for live-build. 
Actually, it was not meant for alt El Torito boot entries (or non first 
entries as you clarify).


The first idea was to add both syslinux and grub-pc in the same image. 
Grub-pc would be the one installed to be boot but syslinux files would 
be there for Multi-USB tools to know how to understand the iso and put 
it into an USB.


The Grub-pc bits were thought in first place for Super Grub2 Disk (which 
it is Grub2 cfg files mainly).


(Now that I think of the current 'check primary bootloader role' 
functions unable me to perform that Super Grub2 Disk Debian Live that 
included the Syslinux bits too. Anyways I'll find a workaround for that 
in the future. Not a priority right now.)


Now primary means: "First lure" and secondary means "Second lure" by 
your definition.


Currently there's no way to stop you from requesting:

--bootloaders="syslinux,grub-efi,syslinux-efi"

Given the way that both grub-efi and syslinux-efi use:

/efi/boot/bootia32.efi
and
/efi/boot/bootx64.efi

the last one to be run would overwrite the first one efi files.



You can probably use only one legacy bootloader but syslinux-efi and
grub-efi use different files so it should be possible to install both.
I am not sure how selecting one or the other would work, though.


You can offer several EL Torito boot images for BIOS, indeed.
It will depend on your BIOS what it does in this case.

Interesting.


There can be only one MBR, though. So you would have to hop
by MBR x86 code to a standalone program which chooses among
the BIOS suitable El Torito boot images.

With EFI it makes few sense to offer more than one El Torito
boot image or EFI System Partition. UEFI 2.4 rather prescribes to
handle all alternatives inside the FAT filesystem. The firmware
will start the \EFI\BOOT\BOOT*.EFI binary which it deems suitable
for its processor type.
This binary is quite free in its further proceedings.

So... How would you go about for a:

--bootloaders="syslinux,grub-efi,syslinux-efi"

to make sense if, as I have stated earlier (although I might have missed 
something there) they would overwrite their /efi/boot/boot*efi files ?




adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

Michal Suchanek wrote:
> -eltorito-alt-boot is not documented option of xorriso.

You need to look into man xorrisofs for the options of the -as mkisofs
emulation.

 -eltorito-alt-boot
  Finalize the current El Torito boot catalog entry  and  begin  a
  new  one.  A boot image file and all its necessary options shall
  be specified before option -eltorito-alt-boot.  All  further  El
  Torito  boot  options  apply  to the new catalog entry. Up to 32
  catalog entries are possible.


> So any bootloader is made primary by leaving out -eltorito-alt-boot.

There is no "primary" or "secondary" on the level of boot images
and loaders. (Of course you may call them this way in your project.)

There are first, second, third ... El Torito boot images or MBR, GPT,
APM partition entries.
The sequence is just needed because not all can be first in the list.

The boot firmware inspects the medium for its known lures,
picks a tasty one, and follows it to the boot loader or kernel.


> You can probably use only one legacy bootloader but syslinux-efi and
> grub-efi use different files so it should be possible to install both.
> I am not sure how selecting one or the other would work, though.

You can offer several EL Torito boot images for BIOS, indeed.
It will depend on your BIOS what it does in this case.

There can be only one MBR, though. So you would have to hop
by MBR x86 code to a standalone program which chooses among
the BIOS suitable El Torito boot images.

With EFI it makes few sense to offer more than one El Torito
boot image or EFI System Partition. UEFI 2.4 rather prescribes to
handle all alternatives inside the FAT filesystem. The firmware
will start the \EFI\BOOT\BOOT*.EFI binary which it deems suitable
for its processor type.
This binary is quite free in its further proceedings.


Have a nice day :)

Thomas

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

A quick update on my grub-efi work:

1) I have noticed I have missed to give execution permissions to many 
files when I rebased.


These are:
scripts/build/binary_grub-efi
scripts/build/binary_syslinux-efi
scripts/build/efi-image
scripts/build/grub-cpmodules

You can find additional commits to fix this situation on:
https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd_rebased
and
https://github.com/adrian15/live-build/commits/syslinux-efi-2016
.

(I will probably do another rebase in another branch in the future with 
other of your suggestions and these fixes.)



2) I have released Rescatux 0.40 beta 5 based on: 
https://github.com/adrian15/live-build/tree/rescatux-0.40b5 (The patches 
you have already seen + execution permissions fixed) . It is built by using:

--bootloaders="syslinux,grub-efi" .

You can find more information about it and download links on:

http://www.supergrubdisk.org/2016/01/18/rescatux-0-40-beta-5-released/

Remember that if you want to test it into a USB drive you need to do the 
'dd' if you want it to work ok and that means DESTROYING your data in 
the USB drive.



adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

El 18/01/16 a las 07:31, Michal Suchanek escribió:

Hello,

thanks for working on this.

On 18 January 2016 at 05:24, adrian15  wrote:

In my last message I forgot to CC many people who are involved in this bug
so I'm going to refer to my former message, CC some people and finally point
you to my repo/branches where you might find interesting commits.


1) My original message with attached patches (which you can download to your
email program and reply from there) can be found at:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731709#153


2) Repo / Branches:

* efi_support_based_on_debian_cd  (
https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd )
: Original dirty branch where I worked in both syslinux-efi and grub-efi
bootloaders at the same time

* efi_support_based_on_debian_cd_rebased (
https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd_rebased
) : The same branch rebased to have minimal useful commits. syslinux-efi was
removed from it because it did not boot.


I have checked what EFI platforms I have and it seems neither would be
bootable with this.

1) a Windows tablet that needs signed bootloader - signed grub is
available as non-free package (because it cannot be modified to not
break the signature) and presumably allows booting on such systems
without going through the unlocking stuff which may break your windows
installation.
Yeah, that's the Microsoft Secure Boot. debian-cd people are working on 
bringing it into official Debian CDs. There is not a fixed date for that 
but everything points that it will be finished within 2016. Once they 
manage to do so I'll see how easy it is to port into this work.


You can always try virtual systems. This is how I currently test this:

kdesudo "kvm -bios /usr/share/ovmf/OVMF.fd -cdrom rescatux-0.40b3.iso 
-boot d -m 512"


.


2) a mac mini which probably needs a GPT and the bootloader stored on
a HFS+ volume or something. Also the bootloader needs to be 'blessed'.
Command for that exists in grub and crashes. Never got this working on
an USB stick.
Well, I have not mentioned that this implementation (based on debian-cd 
team work) also would support intel based mac systems because it has an 
additional HFS+ volume.


I also have a mac book pro myself and the 'hybrid' version of Super 
Grub2 Disk 2.02s3 works for me (when using ALT key at boot). I don't 
remember having blessed it so... I'm unsure about the need (or not) to 
bless it.


So, the 'hybrid' Super Grub2 Disk 2.02s3 also has an additional HFS+ 
partition (as our current implementation here in these patches) put in 
place for mac book compatibility so I guess that this build will also 
boot in my system but I still need to test it.




As to the primary and secondary bootloader - how is the efi bootloader
secondary? It boots the same as the legacy bootloader. You probably
want a concept of compatible bootloaders - that is pairs of
bootloaders that can be installed alongside on the same medium.

* What is it a secondary bootloader?

It's what happens when you request mkisofs that your bootloader to be 
boot in second place or as a second partition. I don't know how it 
actually works.


So in grub-efi we just add to the xorriso options:

-eltorito-alt-boot \
 -e boot/grub/efi.img \
 -no-emul-boot \
 -isohybrid-gpt-basdat \
 -isohybrid-apm-hfsplus

And in syslinux-efi (currently) we add:

-eltorito-alt-boot \
 --efi-boot boot/efi.img \
 -append_partition 2 0x01 \
 binary/boot/efi.img

.

So, I guess the -eltorito-alt-boot does the magic but I'm not sure. 
Maybe someone else can explain it better than me.



* About current compatible bootloaders.
The technology is there. I mean. This is what it's currently available:

binary_grub-efi : Secondary bootloader
binary_grub-legacy : Primary bootloader
binary_grub-pc : Primary bootloader
binary_syslinux : Primary bootloader
binary_syslinux-efi : Secondary bootloader

so that means, in theory you can request one of these 9 combinations:

grub-legacy
grub-pc
syslinux

grub-legacy,grub-efi
grub-pc,grub-efi
syslinux,grub-efi

grub-legacy,syslinux-efi
grub-pc,syslinux-efi
syslinux,syslinux-efi


Why can't you request:

grub-efi
or
grub-efi,syslinux-efi

?

Because nobody has coded grub-efi installed as a primary bootloader yet. 
But the 'framework' is there.


You either need to improve binary_grub-efi or create another script file 
that does its job when ' Is_Primary_Bootloader "grub-efi" ' is true.




Unless
the user requests two bootloaders that are incompatible the medium can
be created.

Ignoring bootloaders is not a good idea. If the user requested
something that is not possible the build should report an error and
stop.
Yeah, that it's hard to implement from the perspective of a single 
bootloader script. So I decided not to implement it.


You could add an additional script or function named: 
check_compatible_bootable_pairs (as you propose) but then you need to 
maintain that 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

On 18 January 2016 at 10:43, adrian15  wrote:
> El 18/01/16 a las 07:31, Michal Suchanek escribió:
>>
>> Hello,
>>
>> thanks for working on this.
>>

>
>> As to the primary and secondary bootloader - how is the efi bootloader
>> secondary? It boots the same as the legacy bootloader. You probably
>> want a concept of compatible bootloaders - that is pairs of
>> bootloaders that can be installed alongside on the same medium.
>
> * What is it a secondary bootloader?
>
> It's what happens when you request mkisofs that your bootloader to be boot
> in second place or as a second partition. I don't know how it actually
> works.
>
> So in grub-efi we just add to the xorriso options:
>
> -eltorito-alt-boot \
>  -e boot/grub/efi.img \
>  -no-emul-boot \
>  -isohybrid-gpt-basdat \
>  -isohybrid-apm-hfsplus
>
> And in syslinux-efi (currently) we add:
>
> -eltorito-alt-boot \
>  --efi-boot boot/efi.img \
>  -append_partition 2 0x01 \
>  binary/boot/efi.img

-eltorito-alt-boot is not documented option of xorriso. For
genisoimage -eltorito-alt-boot denotes start of new bootloader
parameters. So any bootloader is made primary by leaving out
-eltorito-alt-boot.

>
> .
>
> So, I guess the -eltorito-alt-boot does the magic but I'm not sure. Maybe
> someone else can explain it better than me.
>
>
> * About current compatible bootloaders.
> The technology is there. I mean. This is what it's currently available:
>
> binary_grub-efi : Secondary bootloader
> binary_grub-legacy : Primary bootloader
> binary_grub-pc : Primary bootloader
> binary_syslinux : Primary bootloader
> binary_syslinux-efi : Secondary bootloader
>
> so that means, in theory you can request one of these 9 combinations:
>
> grub-legacy
> grub-pc
> syslinux
>
> grub-legacy,grub-efi
> grub-pc,grub-efi
> syslinux,grub-efi
>
> grub-legacy,syslinux-efi
> grub-pc,syslinux-efi
> syslinux,syslinux-efi

You can probably use only one legacy bootloader but syslinux-efi and
grub-efi use different files so it should be possible to install both.
I am not sure how selecting one or the other would work, though.

>
>> Unless
>> the user requests two bootloaders that are incompatible the medium can
>> be created.
>>
>> Ignoring bootloaders is not a good idea. If the user requested
>> something that is not possible the build should report an error and
>> stop.
>
> Yeah, that it's hard to implement from the perspective of a single
> bootloader script. So I decided not to implement it.
>
> You could add an additional script or function named:
> check_compatible_bootable_pairs (as you propose) but then you need to
> maintain that fictional database each time a new bootloader gets in.
>
> That's why I decide to avoid it using it.
>
> What it's straight-forward to implement is one of the only-secondary
> bootloaders finding themselves as a primary bootloader and outputting a
> warning message. But, I'm not sure what to put there as a warning. Any
> suggestion?

Unsupported bootloader combination or whatever.

It should not be a warning however. The script should avoid making
unbootable medium.

Thanks

Michal

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Thomas Schmitt]

Hi,

adrian15 wrote:
> * What is it a secondary bootloader?
> It's what happens when you request mkisofs that your bootloader to be 
> boot in second place or as a second partition. I don't know how it 
> actually works.

An ISO may contain several lures for boot firmware.

If it is booted from CD/DVD/BD, then the lures are in the El Torito
boot catalog. In case of debian-cd it contains two entries which
point to boot images. One entry is marked as suitable for BIOS and
one marked as suitable for EFI. The boot firmware then picks what
it deems right and starts it, or offers it to the user for starting.

If the ISO is presented on USB stick or alike, then BIOS looks
for the magic number of an MBR and if so, mindlessly executes
the x86 machine code that starts at byte 0 of the ISO.
This code then hops onto the same boot code that is advertised
by El Torito (because xorriso or isohybrid patched its block
address into the MBR code).

EFI looks on USB stick for an MBR partition of type 0xef or
for a single partition of type 0xee. In the latter case it assumes
the presence of GPT and looks for a GPT partition with type GUID
28732ac11ff8d211ba4b00a0c93ec93b.
Inside the found partition it expects a FAT filesystem and
particular binaries for each supported processor archictecture.
E.g. \EFI\BOOT\BOOTX64.EFI

The El Torito EFI boot image has the same content specification
as the EFI System Partition. Therefore both boot paths can share
the same data. 

My cheat sheet is at
  
http://bazaar.launchpad.net/~libburnia-team/libisofs/scdbackup/view/head:/doc/boot_sectors.txt

-

> So, I guess the -eltorito-alt-boot does the magic but I'm not sure. 
> Maybe someone else can explain it better than me.

-eltorito-alt-boot is simply a separator between the options of
El Torito boot images. 
I assume your first boot image is announced by -b and gets
the usual options
   -no-emul-boot -boot-load-size 4 -boot-info-table

Before the announcement of the EFI boot image begins, option
-eltorito-alt-boot is needed to protect your -b and its helpers
from being overwritten.

> So in grub-efi we just add to the xorriso options:
> -eltorito-alt-boot \
>  -e boot/grub/efi.img \
>  -no-emul-boot \
>  -isohybrid-gpt-basdat \
>  -isohybrid-apm-hfsplus

Here a second boot image gets announced by -e and the next three
options apply to it.


> And in syslinux-efi (currently) we add:
> -eltorito-alt-boot \
> --efi-boot boot/efi.img \

Option --efi-boot is a shortcut for
  -eltorito-alt-boot -e ... -no-emul-boot -eltorito-alt-boot
normally used by grub-mkrescue. The SYSLINUX community prefers -e.


> -append_partition 2 0x01 \
>  binary/boot/efi.img

I wonder what EFI firmware would hop on an MBR partition of type 0x01.
An EFI System Partition in MBR should have type 0xef to be recognized.

It looks like this variant will not get GPT. That's fully compliant
to UEFI 2.4. Just the MBR partition type should be 0xef.

(Does "syslinux-efi" mean that you can boot from ISO via EFI
and SYSLINUX stuff to an operating system ? That would be new.)

---

Above options do not produce HFS+ with blessing.
Option -isohybrid-apm-hfsplus causes an Apple Partition Map entry
which points to the data content of boot/grub/efi.img.

There are two ways known to me how to get a boot path via HFS+:
Fedora Live CD as of Matthew Garrett
or grub-mkrescue as of Vladimir Serbinenko.

---

Fedora Live, e.g. 
  Fedora-Live-Xfce-x86_64-rawhide-20150704.iso
is the ancestor of debian-cd's layout. It contains a small HFS+
filesystem image as ISO data file
  /isolinux/macboot.img

debian-cd does not have such an HFS+ image file.
Image content and production is out of my scope.

xorriso would only bond it to El Torito, MBR, GPT, and APM. I understand
that APM leads some Macs to that HFS+ filesystem. Whether there are
customers for GPT is unclear. The lures in El Torito and MBR are
probably unused.

The essential xorriso -as mkisofs options are a superset of those
of debian-cd for amd64:

  -isohybrid-mbr /path/to/isohdpfx.bin
  -apm-block-size 2048
  -c '/isolinux/boot.cat'
  -b '/isolinux/isolinux.bin'
-no-emul-boot
-boot-load-size 4
-boot-info-table
  -eltorito-alt-boot
  -e '/isolinux/efiboot.img'
-no-emul-boot
-isohybrid-gpt-basdat
-isohybrid-apm-hfsplus
  -eltorito-alt-boot
  -e '/isolinux/macboot.img'
-no-emul-boot
-isohybrid-gpt-hfsplus
-isohybrid-apm-hfsplus

(Options picked out of proposal of
   xorriso -hfsplus on \
   -indev Fedora-Live-Xfce-x86_64-rawhide-20150704.iso \
   -report_el_torito as_mkisofs
)

No UEFI compliant firmware should use the GPT in there, because
the MBR partition table contains a partition of type other than
0x00 and 0xee. EFI firmware should rather follow the MBR partition
of type 0xef if the ISO is presented on USB 

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[adrian15]

In my last message I forgot to CC many people who are involved in this 
bug so I'm going to refer to my former message, CC some people and 
finally point you to my repo/branches where you might find interesting 
commits.



1) My original message with attached patches (which you can download to 
your email program and reply from there) can be found at:


https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731709#153


2) Repo / Branches:

* efi_support_based_on_debian_cd  ( 
https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd 
) : Original dirty branch where I worked in both syslinux-efi and 
grub-efi bootloaders at the same time


* efi_support_based_on_debian_cd_rebased ( 
https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd_rebased 
) : The same branch rebased to have minimal useful commits. syslinux-efi 
was removed from it because it did not boot.


* syslinux-efi-2016 ( 
https://github.com/adrian15/live-build/tree/syslinux-efi-2016 ) : This 
branch adds and enables the latest version of syslinux-efi (rebased) but 
it does not boot.



adrian15
--
Support free software. Donate to Super Grub Disk. Apoya el software 
libre. Dona a Super Grub Disk. http://www.supergrubdisk.org/donate/

Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)

[Michal Suchanek]

Hello,

thanks for working on this.

On 18 January 2016 at 05:24, adrian15  wrote:
> In my last message I forgot to CC many people who are involved in this bug
> so I'm going to refer to my former message, CC some people and finally point
> you to my repo/branches where you might find interesting commits.
>
>
> 1) My original message with attached patches (which you can download to your
> email program and reply from there) can be found at:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731709#153
>
>
> 2) Repo / Branches:
>
> * efi_support_based_on_debian_cd  (
> https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd )
> : Original dirty branch where I worked in both syslinux-efi and grub-efi
> bootloaders at the same time
>
> * efi_support_based_on_debian_cd_rebased (
> https://github.com/adrian15/live-build/tree/efi_support_based_on_debian_cd_rebased
> ) : The same branch rebased to have minimal useful commits. syslinux-efi was
> removed from it because it did not boot.

I have checked what EFI platforms I have and it seems neither would be
bootable with this.

1) a Windows tablet that needs signed bootloader - signed grub is
available as non-free package (because it cannot be modified to not
break the signature) and presumably allows booting on such systems
without going through the unlocking stuff which may break your windows
installation.

2) a mac mini which probably needs a GPT and the bootloader stored on
a HFS+ volume or something. Also the bootloader needs to be 'blessed'.
Command for that exists in grub and crashes. Never got this working on
an USB stick.

As to the primary and secondary bootloader - how is the efi bootloader
secondary? It boots the same as the legacy bootloader. You probably
want a concept of compatible bootloaders - that is pairs of
bootloaders that can be installed alongside on the same medium. Unless
the user requests two bootloaders that are incompatible the medium can
be created.

Ignoring bootloaders is not a good idea. If the user requested
something that is not possible the build should report an error and
stop.

Finally it seems that some of the patches reference grub-efi before
the patch that adds support for it. Support for grub-efi should
probably be added before it's referenced in other code (eg set as
default).

Thanks

Michal