Re: Xenial/grub2: Changes for Xen

[Stefan Bader]

On 30.11.2015 18:22, Mathieu Trudel-Lapierre wrote:
> On Mon, Nov 30, 2015 at 10:01 AM, Stefan Bader  > wrote:
> [...]
> 
> Currently I do a /etc/grub.d/xen.cfg which, apart from adding a nicely 
> separated
> place for Xen specific grub options (which I think is worth keeping), 
> adds an
> override string to boot into Xen by default. A better way for that long 
> term
> seems to be to simply change the order of the generator script
> (/etc/grub.d/20_linux_xen). This only generates a real section if there 
> is a Xen
> hypervisor installed and doing that a user likely also wants that to 
> become the
> default. So the question is whether it sounds reasonable to rename 
> 20_linux_xen
> into something like 09_xen?
> 
> 
> I'm not opposed to that, but it's worth checking with the Debian GRUB
> maintainers too, since we usually try to keep grub in sync.

Fair point. I added Colin and Ian. Actually Ian may remember some of the details
about multiboot that I forgot. And maybe it makes sense to reach out to
pkg-xen-devel and if a similar list exists for grub2 to that as well.

>  
> 
> The the other thing probably needs more change than only grub: For a 
> while now
> xen-hypervisor ships a version that is normally used from grub (using 
> multiboot)
> and an EFI executable. The normal version cannot be used on UEFI systems 
> because
> multiboot protocol has some shortcomings and there is no way to transfer 
> control
> in a way to allow to get the memory layout (as one example).
> Currently 20_linux_xen generates two grub entries, one for xen-*.gz and 
> one for
> xen-*.efi. The latter plainly is wrong and has only gone unnoticed 
> because the
> former is selected by default. But I would propose the following change:
> 
> 
> We most likely don't want to use the .efi image at all, if we want to maintain
> the behavior of simply booting via grub for both methods. One use of the .efi
> image is probably because you can more easily enforce the signature on that 
> EFI
> binary, but it doesn't seem to me like something we'd go out of our way to 
> sign
> anyway.

I agree. Its also simpler to find the choice between Xen and normal boot there.
So I, too, would prefer any solution that keeps grub as the integration point.

> 
> [...]
> 
> 
> As for the question on how to handle UEFI boot, I don't know what can be 
> done
> about that. The *.efi executable likely needs to be rather loaded 
> directly from
> the shim layer, and then sooner than later also needs to become signed. Or
> 
> 
> As above, I think we'd probably just keep using the kernels loaded from grub. 
> On
> top of not requiring the separate signature of another EFI image (and either
> that signature coming from Microsoft or chainloading from shim and changing 
> the
> EFI boot entries to account for that), it would have the advantage of already
> working, being the same for both the EFI and legacy BIOS cases.
> 
> We also already sign at least the standard shipping kernel. Signing the Xen 
> bits
> may require a bit of work though, since it's in universe and we may want to 
> sign
> it with a different key. At least for now, you'll still benefit from the
> bootloader being signed, just like it is in the non-Xen case.

Right now it would be ok. But there could be the time when the whole boot chain
must be signed while secure boot is on. Which will also suck in the case of
quickly giving people test kernels (but that is kind of a different issue).
But I guess this is a good time to talk together with the Debian side and figure
out a plan for this (and maybe I then have an email archive to fill the holes in
my memory).

> 
> that would allow using the normal grub2->xen chain for the UEFI boot 
> case. But I
> am not sure there is an outtcome, yet. So I guess for now the primary 
> target
> would be to ignore the *.efi file when generating the grub.cfg.
> 
> 
> I don't know enough about Xen to know why the normal loading using grub2 would
> behave differently in UEFI (as you can no doubt notice above), but yes, the
> short answer IMO is that we should just ignore the .efi file.

I am not sure I remember it correctly. I think the problem was somewhere around
taking the multiboot route would exit boot services before transferring to the
Xen hypervisor code. And at least one problem there is that E820 has been
replaced by some other mapping which is no longer accessible then.
I have a vague feeling there was also something mentioned about needing to pass
on ... something. But I fail to remember what and why it was not possible with
the existing multiboot protocol.

-Stefan
> 
> Regards,
> 
> Mathieu Trudel-Lapierre >
> Freenode: cyphermox, Jabber: mathieu.tl @gmail.com
> 
> 4096R/DC95CA5A 36E2 CF22 B077 FEFE 725C  80D3 

Re: Xenial/grub2: Changes for Xen

[Stefan Bader]

On 01.12.2015 01:05, Seth Arnold wrote:
> On Mon, Nov 30, 2015 at 04:01:44PM +0100, Stefan Bader wrote:
>> 20_linux_xen:
>>  xen_list=`for i in /boot/xen*; do
>> +if echo "$i" | grep -q '\.efi$'; then continue; fi
>>  if grub_file_is_not_garbage "$i" && file_is_not_sym "$i" ; then 
>> echo -n
>> "$i " ; fi
>>done`
> 
> Could this be changesd to use the same xen-*.gz you used in the
> descriptive text?
> 
>xen_list=`for i in /boot/xen-*.gz; do
>if grub_file_is_not_garbage "$i" && file_is_not_sym "$i" ; then 
> echo -n
>   "$i " ; fi
>  done`

Maybe. Though one advantage of my variant (no, not currently planned on as far
as I know) would be that mine would work even if the .gz would one day be
replaced by a .bz2 or .xz or ...

-Stefan
> 
> Thanks
> 




signature.asc
Description: OpenPGP digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Xenial/grub2: Changes for Xen

[Seth Arnold]

On Mon, Nov 30, 2015 at 04:01:44PM +0100, Stefan Bader wrote:
> 20_linux_xen:
>  xen_list=`for i in /boot/xen*; do
> +if echo "$i" | grep -q '\.efi$'; then continue; fi
>  if grub_file_is_not_garbage "$i" && file_is_not_sym "$i" ; then echo 
> -n
> "$i " ; fi
>done`

Could this be changesd to use the same xen-*.gz you used in the
descriptive text?

   xen_list=`for i in /boot/xen-*.gz; do
   if grub_file_is_not_garbage "$i" && file_is_not_sym "$i" ; then echo 
-n
  "$i " ; fi
 done`

Thanks


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Xenial/grub2: Changes for Xen

[Colin Watson]

On Tue, Dec 01, 2015 at 09:49:55AM +0100, Stefan Bader wrote:
> On 30.11.2015 18:22, Mathieu Trudel-Lapierre wrote:
> > On Mon, Nov 30, 2015 at 10:01 AM, Stefan Bader  > > wrote:
> > Currently I do a /etc/grub.d/xen.cfg which, apart from adding a nicely 
> > separated
> > place for Xen specific grub options (which I think is worth keeping),

We already have GRUB_CMDLINE_XEN; no need for a separate file.

> > adds an override string to boot into Xen by default. A better
> > way for that long term seems to be to simply change the order of
> > the generator script (/etc/grub.d/20_linux_xen). This only
> > generates a real section if there is a Xen hypervisor installed
> > and doing that a user likely also wants that to become the
> > default. So the question is whether it sounds reasonable to
> > rename 20_linux_xen into something like 09_xen?
> > 
> > I'm not opposed to that, but it's worth checking with the Debian GRUB
> > maintainers too, since we usually try to keep grub in sync.
> 
> Fair point. I added Colin and Ian. Actually Ian may remember some of the 
> details
> about multiboot that I forgot. And maybe it makes sense to reach out to
> pkg-xen-devel and if a similar list exists for grub2 to that as well.

It's been suggested before, and the current situation is certainly
unfortunate, but this really needs to be sorted out upstream
(grub-de...@gnu.org).

Also, moving the conffile around is a cumbersome way to do this
ordering; if somebody wants to move it back then they won't get any
updates to the generator script, so it's anything but "simple" to make
this change in practice, unfortunately.  I've long thought that this is
an indication that the entire configuration system needs to be
rethought.

> > As above, I think we'd probably just keep using the kernels loaded from 
> > grub. On
> > top of not requiring the separate signature of another EFI image (and either
> > that signature coming from Microsoft or chainloading from shim and changing 
> > the
> > EFI boot entries to account for that), it would have the advantage of 
> > already
> > working, being the same for both the EFI and legacy BIOS cases.
> > 
> > We also already sign at least the standard shipping kernel. Signing the Xen 
> > bits
> > may require a bit of work though, since it's in universe and we may want to 
> > sign
> > it with a different key. At least for now, you'll still benefit from the
> > bootloader being signed, just like it is in the non-Xen case.
> 
> Right now it would be ok. But there could be the time when the whole boot 
> chain
> must be signed while secure boot is on. Which will also suck in the case of
> quickly giving people test kernels (but that is kind of a different issue).
> But I guess this is a good time to talk together with the Debian side and 
> figure
> out a plan for this (and maybe I then have an email archive to fill the holes 
> in
> my memory).

It would be reasonable to emit the Xen EFI image as another thing to be
signed, but how would further signature chaining work after that?

No need for a different key just because it's in universe.  The archive
key is for the archive, not just for main.  We force all signature
requests into unapproved for manual attention by archive admins so that
random packages can't get signatures.

-- 
Colin Watson   [cjwat...@ubuntu.com]

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Xenial/grub2: Changes for Xen

[Ian Campbell]

On Tue, 2015-12-01 at 09:49 +0100, Stefan Bader wrote:
> On 30.11.2015 18:22, Mathieu Trudel-Lapierre wrote:
> > On Mon, Nov 30, 2015 at 10:01 AM, Stefan Bader  > com
> > > wrote:
> > [...]
> > 
> > Currently I do a /etc/grub.d/xen.cfg which, apart from adding a
> > nicely separated
> > place for Xen specific grub options (which I think is worth
> > keeping), adds an
> > override string to boot into Xen by default. A better way for that
> > long term
> > seems to be to simply change the order of the generator script
> > (/etc/grub.d/20_linux_xen). This only generates a real section if
> > there is a Xen
> > hypervisor installed and doing that a user likely also wants that
> > to become the
> > default. So the question is whether it sounds reasonable to rename
> > 20_linux_xen
> > into something like 09_xen?
> > 
> > 
> > I'm not opposed to that, but it's worth checking with the Debian GRUB
> > maintainers too, since we usually try to keep grub in sync.
> 
> Fair point. I added Colin and Ian. Actually Ian may remember some of the 
> details
> about multiboot that I forgot. And maybe it makes sense to reach out to
> pkg-xen-devel and if a similar list exists for grub2 to that as well.

I've long thought that reordering 10_linux and 20_linux_xen would make
sense as an upstream fix even, I just never got around to doing anything
about it (IIRC).

> > The the other thing probably needs more change than only grub: For a 
> > while now
> > xen-hypervisor ships a version that is normally used from grub (using 
> > multiboot)
> > and an EFI executable. The normal version cannot be used on UEFI 
> > systems because
> > multiboot protocol has some shortcomings and there is no way to 
> > transfer control
> > in a way to allow to get the memory layout (as one example).
> > Currently 20_linux_xen generates two grub entries, one for xen-*.gz and 
> > one for
> > xen-*.efi. The latter plainly is wrong and has only gone unnoticed 
> > because the
> > former is selected by default. But I would propose the following change:
> > 
> > 
> > We most likely don't want to use the .efi image at all, if we want to 
> > maintain
> > the behavior of simply booting via grub for both methods. One use of the 
> > .efi
> > image is probably because you can more easily enforce the signature on that 
> > EFI
> > binary, but it doesn't seem to me like something we'd go out of our way to 
> > sign
> > anyway.
> 
> I agree. Its also simpler to find the choice between Xen and normal boot 
> there.
> So I, too, would prefer any solution that keeps grub as the integration point.

You currently can't boot xen.efi via grub in the normal way, you have to
chainload and provide (somehow) a xen.cfg per http://xenbits.xen.org/docs/u
nstable/misc/efi.html otherwise all sorts of things fail. I think most
people just register xen.efi with the firmware rather than laundering via
grub, although with chainloading I think both are supposed to work equally
well.

Daniel Kiper from Oracle is working with upstream grub2 and Xen to make
"normal" booting work properly, by defining a multiboot handover mechanism
for EFI apps (I've not been keeping up with the specifics).

Probably all of this is best discussed on either pkg-grub-devel and/or the
upstream xen/grub devel lists.

Ian.

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Xenial/grub2: Changes for Xen

[Stefan Bader]

As a summary, I agree I should prepare a post across various mailing lists.
Though a bit better structured and with feedback so far added in. Thanks for the
replies so far.

-Stefan




signature.asc
Description: OpenPGP digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Xenial/grub2: Changes for Xen

[Mathieu Trudel-Lapierre]

On Mon, Nov 30, 2015 at 10:01 AM, Stefan Bader 
wrote:
[...]

> Currently I do a /etc/grub.d/xen.cfg which, apart from adding a nicely
> separated
> place for Xen specific grub options (which I think is worth keeping), adds
> an
> override string to boot into Xen by default. A better way for that long
> term
> seems to be to simply change the order of the generator script
> (/etc/grub.d/20_linux_xen). This only generates a real section if there is
> a Xen
> hypervisor installed and doing that a user likely also wants that to
> become the
> default. So the question is whether it sounds reasonable to rename
> 20_linux_xen
> into something like 09_xen?
>

I'm not opposed to that, but it's worth checking with the Debian GRUB
maintainers too, since we usually try to keep grub in sync.


> The the other thing probably needs more change than only grub: For a while
> now
> xen-hypervisor ships a version that is normally used from grub (using
> multiboot)
> and an EFI executable. The normal version cannot be used on UEFI systems
> because
> multiboot protocol has some shortcomings and there is no way to transfer
> control
> in a way to allow to get the memory layout (as one example).
> Currently 20_linux_xen generates two grub entries, one for xen-*.gz and
> one for
> xen-*.efi. The latter plainly is wrong and has only gone unnoticed because
> the
> former is selected by default. But I would propose the following change:
>

We most likely don't want to use the .efi image at all, if we want to
maintain the behavior of simply booting via grub for both methods. One use
of the .efi image is probably because you can more easily enforce the
signature on that EFI binary, but it doesn't seem to me like something we'd
go out of our way to sign anyway.

[...]

>
> As for the question on how to handle UEFI boot, I don't know what can be
> done
> about that. The *.efi executable likely needs to be rather loaded directly
> from
> the shim layer, and then sooner than later also needs to become signed. Or
>

As above, I think we'd probably just keep using the kernels loaded from
grub. On top of not requiring the separate signature of another EFI image
(and either that signature coming from Microsoft or chainloading from shim
and changing the EFI boot entries to account for that), it would have the
advantage of already working, being the same for both the EFI and legacy
BIOS cases.

We also already sign at least the standard shipping kernel. Signing the Xen
bits may require a bit of work though, since it's in universe and we may
want to sign it with a different key. At least for now, you'll still
benefit from the bootloader being signed, just like it is in the non-Xen
case.

that would allow using the normal grub2->xen chain for the UEFI boot case.
> But I
> am not sure there is an outtcome, yet. So I guess for now the primary
> target
> would be to ignore the *.efi file when generating the grub.cfg.
>

I don't know enough about Xen to know why the normal loading using grub2
would behave differently in UEFI (as you can no doubt notice above), but
yes, the short answer IMO is that we should just ignore the .efi file.

Regards,

Mathieu Trudel-Lapierre 
Freenode: cyphermox, Jabber: mathieu...@gmail.com
4096R/DC95CA5A 36E2 CF22 B077 FEFE 725C  80D3 C7DA A946 DC95 CA5A
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel