Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
On Mon, Jul 20, 2015 at 04:29:05PM +0200, Daniel Kiper wrote:
> Build xen.gz with EFI code. We need this to support multiboot2
> protocol on EFI platforms.
>
> If we wish to load not ELF file using multiboot (v1) or multiboot2 then
> it must contain "linear" (or "flat") representation of code and data.
> Currently, PE file contains many sections which are not "linear" (one
> after another without any holes) or even do not have representation
> in a file (e.g. BSS). In theory there is a chance that we could build
> proper PE file using current build system. However, it means that
> xen.efi further diverge from xen ELF file (in terms of contents and
> build method). ELF have all needed properties. So, it means that this
> is good starting point for further development. Additionally, I think
> that this is also good starting point for further xen.efi code and
> build optimizations. It looks that there is a chance that finally we
> can generate xen.efi directly from xen ELF using just simple objcopy.
>
> Signed-off-by: Daniel Kiper
> ---
> v2 - suggestions/fixes:
>- build EFI code only if it is supported in a given build environment
> (suggested by Jan Beulich).
> ---
> xen/arch/x86/Makefile | 13 +
> xen/arch/x86/efi/Makefile | 16 +---
> xen/arch/x86/mm.c |3 ++-
> xen/common/efi/runtime.c |6 ++
> 4 files changed, 22 insertions(+), 16 deletions(-)
>
> diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
> index 5f24951..0335445 100644
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -80,7 +80,7 @@ ALL_OBJS := $(BASEDIR)/arch/x86/boot/built_in.o
> $(BASEDIR)/arch/x86/efi/built_in
>
> ifeq ($(lto),y)
> # Gather all LTO objects together
> -prelink_lto.o: $(ALL_OBJS)
> +prelink_lto.o: $(ALL_OBJS) efi/relocs-dummy.o
> $(LD_LTO) -r -o $@ $^
>
> prelink-efi_lto.o: $(ALL_OBJS) efi/runtime.o efi/compat.o
> @@ -90,14 +90,14 @@ prelink-efi_lto.o: $(ALL_OBJS) efi/runtime.o efi/compat.o
> prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS))
> prelink_lto.o
> $(LD) $(LDFLAGS) -r -o $@ $^
>
> -prelink-efi.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS))
> prelink-efi_lto.o efi/boot.init.o
> +prelink-efi.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS))
> prelink-efi_lto.o
> $(guard) $(LD) $(LDFLAGS) -r -o $@ $^
> else
> -prelink.o: $(ALL_OBJS)
> +prelink.o: $(ALL_OBJS) efi/relocs-dummy.o
> $(LD) $(LDFLAGS) -r -o $@ $^
>
> -prelink-efi.o: $(ALL_OBJS) efi/boot.init.o efi/runtime.o efi/compat.o
> - $(guard) $(LD) $(LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
> +prelink-efi.o: $(ALL_OBJS)
> + $(guard) $(LD) $(LDFLAGS) -r -o $@ $^
> endif
>
> $(BASEDIR)/common/symbols-dummy.o:
> @@ -146,9 +146,6 @@ $(TARGET).efi: prelink-efi.o efi.lds efi/relocs-dummy.o
> $(BASEDIR)/common/symbol
> if $(guard) false; then rm -f $@; echo 'EFI support disabled'; fi
> rm -f $(@D)/.$(@F).[0-9]*
>
> -efi/boot.init.o efi/runtime.o efi/compat.o:
> $(BASEDIR)/arch/x86/efi/built_in.o
> -efi/boot.init.o efi/runtime.o efi/compat.o: ;
> -
> asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c
> $(CC) $(filter-out -flto,$(CFLAGS)) -S -o $@ $<
>
> diff --git a/xen/arch/x86/efi/Makefile b/xen/arch/x86/efi/Makefile
> index 1daa7ac..b1e8883 100644
> --- a/xen/arch/x86/efi/Makefile
> +++ b/xen/arch/x86/efi/Makefile
> @@ -1,14 +1,16 @@
> CFLAGS += -fshort-wchar
>
> -obj-y += stub.o
> -
> -create = test -e $(1) || touch -t 19990101 $(1)
> -
> efi := $(filter y,$(x86_64)$(shell rm -f disabled))
> efi := $(if $(efi),$(shell $(CC) $(filter-out $(CFLAGS-y) .%.d,$(CFLAGS)) -c
> check.c 2>disabled && echo y))
> efi := $(if $(efi),$(shell $(LD) -mi386pep --subsystem=10 -o check.efi
> check.o 2>disabled && echo y))
> -efi := $(if $(efi),$(shell rm disabled)y,$(shell $(call create,boot.init.o);
> $(call create,runtime.o)))
> +efi := $(if $(efi),$(shell rm disabled)y)
>
> -extra-$(efi) += boot.init.o relocs-dummy.o runtime.o compat.o
> +extra-y += relocs-dummy.o
>
> -stub.o: $(extra-y)
> +ifeq ($(efi),y)
> +obj-y += boot.init.o
> +obj-y += compat.o
> +obj-y += runtime.o
> +else
> +obj-y += stub.o
> +endif
That makefile magic I skipped over, but the C code below looks good, so
Half-Reviewed-by: Konrad Rzeszutek Wilk
> diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
> index 342414f..cef2eb6 100644
> --- a/xen/arch/x86/mm.c
> +++ b/xen/arch/x86/mm.c
> @@ -344,7 +344,8 @@ void __init arch_init_memory(void)
>
> subarch_init_memory();
>
> -efi_init_memory();
> +if ( efi_enabled(EFI_PLATFORM) )
> +efi_init_memory();
>
> mem_sharing_init();
>
> diff --git a/xen/common/efi/runtime.c b/xen/common/efi/runtime.c
> index aa064e7..3eb21c1 100644
> --- a/xen/common/efi/runtime.c
> +++ b/xen/common/efi/runtime.c
> @@ -167,6 +167,9 @@ int efi_get_info(uint32_t idx, union xenpf_efi_info *info)
> {
> unsigned int i, n;
>
> +if (
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
>>> On 20.07.15 at 16:29, wrote: > Build xen.gz with EFI code. We need this to support multiboot2 > protocol on EFI platforms. > > If we wish to load not ELF file using multiboot (v1) or multiboot2 then DYM "a non-ELF file"? > it must contain "linear" (or "flat") representation of code and data. Why? Please don't just put out statements, but also reasons (i.e. at least which component is unable to deal with the current [valid afaict] PE image we have). > Currently, PE file contains many sections which are not "linear" (one > after another without any holes) or even do not have representation > in a file (e.g. BSS). In theory there is a chance that we could build > proper PE file using current build system. However, it means that What is "improper" about the currently built PE file? And if there is anything improper, did you inform the binutils maintainers of the problem? > --- a/xen/arch/x86/efi/Makefile > +++ b/xen/arch/x86/efi/Makefile > @@ -1,14 +1,16 @@ > CFLAGS += -fshort-wchar > > -obj-y += stub.o > - > -create = test -e $(1) || touch -t 19990101 $(1) > - > efi := $(filter y,$(x86_64)$(shell rm -f disabled)) > efi := $(if $(efi),$(shell $(CC) $(filter-out $(CFLAGS-y) .%.d,$(CFLAGS)) -c > check.c 2>disabled && echo y)) > efi := $(if $(efi),$(shell $(LD) -mi386pep --subsystem=10 -o check.efi > check.o 2>disabled && echo y)) > -efi := $(if $(efi),$(shell rm disabled)y,$(shell $(call create,boot.init.o); > $(call create,runtime.o))) > +efi := $(if $(efi),$(shell rm disabled)y) > > -extra-$(efi) += boot.init.o relocs-dummy.o runtime.o compat.o > +extra-y += relocs-dummy.o Why is this no longer extra-$(efi)? > -stub.o: $(extra-y) With this dependency removed (instead of perhaps replaced or extended) - what will trigger relocs-dummy.o to be (re)built? > +ifeq ($(efi),y) > +obj-y += boot.init.o > +obj-y += compat.o > +obj-y += runtime.o > +else > +obj-y += stub.o > +endif obj-y := stub.o obj-$(efi) := boot.init.o compat.o runtime.o > --- a/xen/arch/x86/mm.c > +++ b/xen/arch/x86/mm.c > @@ -344,7 +344,8 @@ void __init arch_init_memory(void) > > subarch_init_memory(); > > -efi_init_memory(); > +if ( efi_enabled(EFI_PLATFORM) ) > +efi_init_memory(); I think I'd prefer such checks to be constrained to EFI code where possible, i.e. in this case do the check inside efi_init_memory(). Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
On Thu, Aug 20, 2015 at 09:39:39AM -0600, Jan Beulich wrote: > >>> On 20.07.15 at 16:29, wrote: > > Build xen.gz with EFI code. We need this to support multiboot2 > > protocol on EFI platforms. > > > > If we wish to load not ELF file using multiboot (v1) or multiboot2 then > > DYM "a non-ELF file"? > > > it must contain "linear" (or "flat") representation of code and data. > > Why? Please don't just put out statements, but also reasons (i.e. > at least which component is unable to deal with the current [valid > afaict] PE image we have). This is a requirement of multiboot (v1) or multiboot2 protocol. They both know nothing about PE image format. > > Currently, PE file contains many sections which are not "linear" (one > > after another without any holes) or even do not have representation > > in a file (e.g. BSS). In theory there is a chance that we could build > > proper PE file using current build system. However, it means that > > What is "improper" about the currently built PE file? And if there is > anything improper, did you inform the binutils maintainers of the > problem? >From PE loader point of view everything is OK. However, current Xen PE image (at least build on my machines) is not usable by multiboot (v1) or multiboot2 protocol compatible loader because it is not linear (one section does not live immediately after another without any voids). Potentially we can change that (probably by playing with linker script or using objcopy or just simply building proper PE file ourselves; I did some experiments once but they are not exhaustive and I am not able to tell which solution will be the best in our case), however, this is next step and it will be investigated after applying this patch series to Xen source. > > --- a/xen/arch/x86/efi/Makefile > > +++ b/xen/arch/x86/efi/Makefile > > @@ -1,14 +1,16 @@ > > CFLAGS += -fshort-wchar > > > > -obj-y += stub.o > > - > > -create = test -e $(1) || touch -t 19990101 $(1) > > - > > efi := $(filter y,$(x86_64)$(shell rm -f disabled)) > > efi := $(if $(efi),$(shell $(CC) $(filter-out $(CFLAGS-y) .%.d,$(CFLAGS)) > > -c check.c 2>disabled && echo y)) > > efi := $(if $(efi),$(shell $(LD) -mi386pep --subsystem=10 -o check.efi > > check.o 2>disabled && echo y)) > > -efi := $(if $(efi),$(shell rm disabled)y,$(shell $(call > > create,boot.init.o); $(call create,runtime.o))) > > +efi := $(if $(efi),$(shell rm disabled)y) > > > > -extra-$(efi) += boot.init.o relocs-dummy.o runtime.o compat.o > > +extra-y += relocs-dummy.o > > Why is this no longer extra-$(efi)? Because we need proper EFI code in xen.gz to support boot via multiboot2 on EFI platforms. > > -stub.o: $(extra-y) > > With this dependency removed (instead of perhaps replaced or > extended) - what will trigger relocs-dummy.o to be (re)built? It is triggered by prelink.o build rule in xen/arch/x86/Makefile. Looks that it will suffice but I am not sure right now this is good solution. Hmmm... If it is OK then we can remove "extra-y += relocs-dummy.o". Daniel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
>>> On 22.08.15 at 15:59, wrote: > On Thu, Aug 20, 2015 at 09:39:39AM -0600, Jan Beulich wrote: >> >>> On 20.07.15 at 16:29, wrote: >> > Build xen.gz with EFI code. We need this to support multiboot2 >> > protocol on EFI platforms. >> > >> > If we wish to load not ELF file using multiboot (v1) or multiboot2 then >> >> DYM "a non-ELF file"? >> >> > it must contain "linear" (or "flat") representation of code and data. >> >> Why? Please don't just put out statements, but also reasons (i.e. >> at least which component is unable to deal with the current [valid >> afaict] PE image we have). > > This is a requirement of multiboot (v1) or multiboot2 protocol. They both > know nothing about PE image format. And hence how specifically we arrange data inside the image should be benign to them, as they won't be able to load the file _anyway_. >> > Currently, PE file contains many sections which are not "linear" (one >> > after another without any holes) or even do not have representation >> > in a file (e.g. BSS). In theory there is a chance that we could build >> > proper PE file using current build system. However, it means that >> >> What is "improper" about the currently built PE file? And if there is >> anything improper, did you inform the binutils maintainers of the >> problem? > > From PE loader point of view everything is OK. However, current Xen PE > image (at least build on my machines) is not usable by multiboot (v1) > or multiboot2 protocol compatible loader because it is not linear (one > section does not live immediately after another without any voids). Again - either I'm missing something (and then your explanation is not good enough) or this is (as said above) a pointless adjustment. >> > --- a/xen/arch/x86/efi/Makefile >> > +++ b/xen/arch/x86/efi/Makefile >> > @@ -1,14 +1,16 @@ >> > CFLAGS += -fshort-wchar >> > >> > -obj-y += stub.o >> > - >> > -create = test -e $(1) || touch -t 19990101 $(1) >> > - >> > efi := $(filter y,$(x86_64)$(shell rm -f disabled)) >> > efi := $(if $(efi),$(shell $(CC) $(filter-out $(CFLAGS-y) .%.d,$(CFLAGS)) >> > -c check.c 2>disabled && echo y)) >> > efi := $(if $(efi),$(shell $(LD) -mi386pep --subsystem=10 -o check.efi >> > check.o >disabled && echo y)) >> > -efi := $(if $(efi),$(shell rm disabled)y,$(shell $(call >> > create,boot.init.o); $(call create,runtime.o))) >> > +efi := $(if $(efi),$(shell rm disabled)y) >> > >> > -extra-$(efi) += boot.init.o relocs-dummy.o runtime.o compat.o >> > +extra-y += relocs-dummy.o >> >> Why is this no longer extra-$(efi)? > > Because we need proper EFI code in xen.gz to support boot > via multiboot2 on EFI platforms. What would we need that for when not building an EFI-capable binary anyway? >> > -stub.o: $(extra-y) >> >> With this dependency removed (instead of perhaps replaced or >> extended) - what will trigger relocs-dummy.o to be (re)built? > > It is triggered by prelink.o build rule in xen/arch/x86/Makefile. No. Or better - if it is, then this is wrong. With the way our build logic works, unless there are exceptional circumstances things in a certain directory should be built _only_ when recursion reaches that particular directory (i.e. not from any of its parents). Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
On Mon, Aug 24, 2015 at 05:35:21AM -0600, Jan Beulich wrote: > >>> On 22.08.15 at 15:59, wrote: > > On Thu, Aug 20, 2015 at 09:39:39AM -0600, Jan Beulich wrote: > >> >>> On 20.07.15 at 16:29, wrote: > >> > Build xen.gz with EFI code. We need this to support multiboot2 > >> > protocol on EFI platforms. > >> > > >> > If we wish to load not ELF file using multiboot (v1) or multiboot2 then > >> > >> DYM "a non-ELF file"? > >> > >> > it must contain "linear" (or "flat") representation of code and data. > >> > >> Why? Please don't just put out statements, but also reasons (i.e. > >> at least which component is unable to deal with the current [valid > >> afaict] PE image we have). > > > > This is a requirement of multiboot (v1) or multiboot2 protocol. They both > > know nothing about PE image format. > > And hence how specifically we arrange data inside the image should > be benign to them, as they won't be able to load the file _anyway_. > > >> > Currently, PE file contains many sections which are not "linear" (one > >> > after another without any holes) or even do not have representation > >> > in a file (e.g. BSS). In theory there is a chance that we could build > >> > proper PE file using current build system. However, it means that > >> > >> What is "improper" about the currently built PE file? And if there is > >> anything improper, did you inform the binutils maintainers of the > >> problem? > > > > From PE loader point of view everything is OK. However, current Xen PE > > image (at least build on my machines) is not usable by multiboot (v1) > > or multiboot2 protocol compatible loader because it is not linear (one > > section does not live immediately after another without any voids). > > Again - either I'm missing something (and then your explanation is > not good enough) or this is (as said above) a pointless adjustment. Let's focus on multiboot2 protocol (multiboot (v1) is similar to multiboot2 in discussed case). In general multiboot2 is able to load any file which has: 1. proper multiboot2 header in first 32 KiB of a given file, 2. "the text and data segments must be consecutive in the OS image" (The Multiboot Specification version 1.6). This implies that we can e.g. build valid ELF file which is also multiboot2 protocol compatible image. And we does. However, we can go further. Potentially we can build valid PE image which is also valid multiboot2 protocol image. Although current build method does not satisfy requirement number 2 because, e.g.: Sections: Idx Name Size VMA LMA File off Algn 0 .text 001513d0 82d08020 82d08020 1000 2**12 ^^ CONTENTS, ALLOC, LOAD, CODE 1 .rodata 0004de12 82d0803513e0 82d0803513e0 00153000 2**5 ^^ CONTENTS, ALLOC, LOAD, READONLY, DATA Hence, we must use special method to build PE image (I discussed that in my earlier email in that topic) to do it compatible with multiboot2 protocol. This way one file could be loaded by native PE loader, mulitboot (v1) protocol (it requires relevant header but it does not interfere with PE and multiboot2 protocol stuff) and mutliboot2 protocol compatible loaders. Additionally, if it is signed with Secure Boot signature then potentially signature could be verified by UEFI itself and e.g. GRUB2. However, as I said earlier this requires more work and this is next step which I am going to do after applying this series. Currently I am going to embed EFI support into ELF file because it is easy (less changes; currently used ELF file has required properties because multiboot (v1) which we use has similar requirements like multiboot2 protocol) to make it compatible with multiboot2 protocol. I hope that helps. > >> > --- a/xen/arch/x86/efi/Makefile > >> > +++ b/xen/arch/x86/efi/Makefile > >> > @@ -1,14 +1,16 @@ > >> > CFLAGS += -fshort-wchar > >> > > >> > -obj-y += stub.o > >> > - > >> > -create = test -e $(1) || touch -t 19990101 $(1) > >> > - > >> > efi := $(filter y,$(x86_64)$(shell rm -f disabled)) > >> > efi := $(if $(efi),$(shell $(CC) $(filter-out $(CFLAGS-y) > >> > .%.d,$(CFLAGS)) -c check.c 2>disabled && echo y)) > >> > efi := $(if $(efi),$(shell $(LD) -mi386pep --subsystem=10 -o check.efi > >> > check.o >disabled && echo y)) > >> > -efi := $(if $(efi),$(shell rm disabled)y,$(shell $(call > >> > create,boot.init.o); $(call create,runtime.o))) > >> > +efi := $(if $(efi),$(shell rm disabled)y) > >> > > >> > -extra-$(efi) += boot.init.o relocs-dummy.o runtime.o compat.o > >> > +extra-y += relocs-dummy.o > >> > >> Why is this no longer extra-$(efi)? > > > > Because we need proper EFI code in xen.gz to support boot > > via multiboot2 on EFI platforms. > > What would we need that for when not building an EFI-capable > binary anyway? xen/arch/x86/ef
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
On 24/08/15 21:54, Daniel Kiper wrote: > > Currently, PE file contains many sections which are not "linear" (one > after another without any holes) or even do not have representation > in a file (e.g. BSS). In theory there is a chance that we could build > proper PE file using current build system. However, it means that What is "improper" about the currently built PE file? And if there is anything improper, did you inform the binutils maintainers of the problem? >>> From PE loader point of view everything is OK. However, current Xen PE >>> image (at least build on my machines) is not usable by multiboot (v1) >>> or multiboot2 protocol compatible loader because it is not linear (one >>> section does not live immediately after another without any voids). >> Again - either I'm missing something (and then your explanation is >> not good enough) or this is (as said above) a pointless adjustment. > Let's focus on multiboot2 protocol (multiboot (v1) is similar to multiboot2 > in discussed case). In general multiboot2 is able to load any file which has: > 1. proper multiboot2 header in first 32 KiB of a given file, > 2. "the text and data segments must be consecutive in the OS image" > (The Multiboot Specification version 1.6). What is the reasoning for the restriction in 2. ? ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
>>> On 24.08.15 at 22:54, wrote: > On Mon, Aug 24, 2015 at 05:35:21AM -0600, Jan Beulich wrote: >> >>> On 22.08.15 at 15:59, wrote: >> > On Thu, Aug 20, 2015 at 09:39:39AM -0600, Jan Beulich wrote: >> >> >>> On 20.07.15 at 16:29, wrote: >> >> > Currently, PE file contains many sections which are not "linear" (one >> >> > after another without any holes) or even do not have representation >> >> > in a file (e.g. BSS). In theory there is a chance that we could build >> >> > proper PE file using current build system. However, it means that >> >> >> >> What is "improper" about the currently built PE file? And if there is >> >> anything improper, did you inform the binutils maintainers of the >> >> problem? >> > >> > From PE loader point of view everything is OK. However, current Xen PE >> > image (at least build on my machines) is not usable by multiboot (v1) >> > or multiboot2 protocol compatible loader because it is not linear (one >> > section does not live immediately after another without any voids). >> >> Again - either I'm missing something (and then your explanation is >> not good enough) or this is (as said above) a pointless adjustment. > > Let's focus on multiboot2 protocol (multiboot (v1) is similar to multiboot2 > in discussed case). In general multiboot2 is able to load any file which > has: > 1. proper multiboot2 header in first 32 KiB of a given file, > 2. "the text and data segments must be consecutive in the OS image" > (The Multiboot Specification version 1.6). > > This implies that we can e.g. build valid ELF file which is also multiboot2 > protocol compatible image. And we does. However, we can go further. > Potentially we can build valid PE image which is also valid multiboot2 > protocol image. Although current build method does not satisfy requirement > number 2 because, e.g.: > > Sections: > Idx Name Size VMA LMA File off > Algn > 0 .text 001513d0 82d08020 82d08020 1000 > 2**12 > ^^ > CONTENTS, ALLOC, LOAD, CODE > 1 .rodata 0004de12 82d0803513e0 82d0803513e0 00153000 > 2**5 > ^^ > CONTENTS, ALLOC, LOAD, READONLY, DATA > > Hence, we must use special method to build PE image (I discussed that in > my earlier email in that topic) to do it compatible with multiboot2 > protocol. And you realize that we use a "special method" for building the current "flat" ELF image too? > This way one file could be loaded by native PE loader, mulitboot (v1) > protocol > (it requires relevant header but it does not interfere with PE and > multiboot2 > protocol stuff) and mutliboot2 protocol compatible loaders. Additionally, > if it is signed with Secure Boot signature then potentially signature could > be verified by UEFI itself and e.g. GRUB2. However, as I said earlier this > requires more work and this is next step which I am going to do after > applying > this series. Currently I am going to embed EFI support into ELF file because > it is easy (less changes; currently used ELF file has required properties > because multiboot (v1) which we use has similar requirements like multiboot2 > protocol) to make it compatible with multiboot2 protocol. I think whether what you do now makes sense depends on the ultimate goal: If we want a single binary usable for all three cases, then while yes, having EFI code available in the ELF image makes sense, using an ELF Image won't work. And we can't have an image being both ELF and PE. Hence the goal ought to be to have a single PE image, and with that the direction you move seems wrong. >> >> > --- a/xen/arch/x86/efi/Makefile >> >> > +++ b/xen/arch/x86/efi/Makefile >> >> > @@ -1,14 +1,16 @@ >> >> > CFLAGS += -fshort-wchar >> >> > >> >> > -obj-y += stub.o >> >> > - >> >> > -create = test -e $(1) || touch -t 19990101 $(1) >> >> > - >> >> > efi := $(filter y,$(x86_64)$(shell rm -f disabled)) >> >> > efi := $(if $(efi),$(shell $(CC) $(filter-out $(CFLAGS-y) >> >> > .%.d,$(CFLAGS)) -c check.c 2>disabled && echo y)) >> >> > efi := $(if $(efi),$(shell $(LD) -mi386pep --subsystem=10 -o check.efi >> >> > heck.o >disabled && echo y)) >> >> > -efi := $(if $(efi),$(shell rm disabled)y,$(shell $(call >> >> > create,boot.init.o); $(call create,runtime.o))) >> >> > +efi := $(if $(efi),$(shell rm disabled)y) >> >> > >> >> > -extra-$(efi) += boot.init.o relocs-dummy.o runtime.o compat.o >> >> > +extra-y += relocs-dummy.o >> >> >> >> Why is this no longer extra-$(efi)? >> > >> > Because we need proper EFI code in xen.gz to support boot >> > via multiboot2 on EFI platforms. >> >> What would we need that for when not building an EFI-capable >> binary anyway? > > xen/arch/x86/efi/stub.c This is still too unspecific: I can't see any reference from that file to any of the symbols reloc
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
On Tue, Aug 25, 2015 at 11:50:05AM +0100, Andrew Cooper wrote: > On 24/08/15 21:54, Daniel Kiper wrote: > > > > Currently, PE file contains many sections which are not "linear" (one > > after another without any holes) or even do not have representation > > in a file (e.g. BSS). In theory there is a chance that we could build > > proper PE file using current build system. However, it means that > What is "improper" about the currently built PE file? And if there is > anything improper, did you inform the binutils maintainers of the > problem? > >>> From PE loader point of view everything is OK. However, current Xen PE > >>> image (at least build on my machines) is not usable by multiboot (v1) > >>> or multiboot2 protocol compatible loader because it is not linear (one > >>> section does not live immediately after another without any voids). > >> Again - either I'm missing something (and then your explanation is > >> not good enough) or this is (as said above) a pointless adjustment. > > Let's focus on multiboot2 protocol (multiboot (v1) is similar to multiboot2 > > in discussed case). In general multiboot2 is able to load any file which > > has: > > 1. proper multiboot2 header in first 32 KiB of a given file, > > 2. "the text and data segments must be consecutive in the OS image" > > (The Multiboot Specification version 1.6). > > What is the reasoning for the restriction in 2. ? You are able to describe following segments using only multiboot2 features: 1) text and data segment bundled into one thing; in general boot loader does not care a lot about its internal structure; it should be just string of bytes which is loaded as is into memory (without any conversion, mangling, etc.), 2) BSS segment which follows one mentioned above; it does not have any representation in image; boot loader just reserve and zero memory of a given size. So, as you can see Xen text and data segment must live in one place in file to satisfy multiboot2 protocol requirements. However, it does not mean that from Xen point of view image cannot have more then one section/segment with required alignments, etc. Daniel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
On Tue, Aug 25, 2015 at 06:09:09AM -0600, Jan Beulich wrote: > >>> On 24.08.15 at 22:54, wrote: > > On Mon, Aug 24, 2015 at 05:35:21AM -0600, Jan Beulich wrote: > >> >>> On 22.08.15 at 15:59, wrote: > >> > On Thu, Aug 20, 2015 at 09:39:39AM -0600, Jan Beulich wrote: > >> >> >>> On 20.07.15 at 16:29, wrote: > >> >> > Currently, PE file contains many sections which are not "linear" (one > >> >> > after another without any holes) or even do not have representation > >> >> > in a file (e.g. BSS). In theory there is a chance that we could build > >> >> > proper PE file using current build system. However, it means that > >> >> > >> >> What is "improper" about the currently built PE file? And if there is > >> >> anything improper, did you inform the binutils maintainers of the > >> >> problem? > >> > > >> > From PE loader point of view everything is OK. However, current Xen PE > >> > image (at least build on my machines) is not usable by multiboot (v1) > >> > or multiboot2 protocol compatible loader because it is not linear (one > >> > section does not live immediately after another without any voids). > >> > >> Again - either I'm missing something (and then your explanation is > >> not good enough) or this is (as said above) a pointless adjustment. > > > > Let's focus on multiboot2 protocol (multiboot (v1) is similar to multiboot2 > > in discussed case). In general multiboot2 is able to load any file which > > has: > > 1. proper multiboot2 header in first 32 KiB of a given file, > > 2. "the text and data segments must be consecutive in the OS image" > > (The Multiboot Specification version 1.6). > > > > This implies that we can e.g. build valid ELF file which is also multiboot2 > > protocol compatible image. And we does. However, we can go further. > > Potentially we can build valid PE image which is also valid multiboot2 > > protocol image. Although current build method does not satisfy requirement > > number 2 because, e.g.: > > > > Sections: > > Idx Name Size VMA LMA File off > > Algn > > 0 .text 001513d0 82d08020 82d08020 1000 > > 2**12 > > ^^ > > CONTENTS, ALLOC, LOAD, CODE > > 1 .rodata 0004de12 82d0803513e0 82d0803513e0 00153000 > > 2**5 > > ^^ > > CONTENTS, ALLOC, LOAD, READONLY, DATA > > > > Hence, we must use special method to build PE image (I discussed that in > > my earlier email in that topic) to do it compatible with multiboot2 > > protocol. > > And you realize that we use a "special method" for building the > current "flat" ELF image too? Yes, I know about that. > > This way one file could be loaded by native PE loader, mulitboot (v1) > > protocol > > (it requires relevant header but it does not interfere with PE and > > multiboot2 > > protocol stuff) and mutliboot2 protocol compatible loaders. Additionally, > > if it is signed with Secure Boot signature then potentially signature could > > be verified by UEFI itself and e.g. GRUB2. However, as I said earlier this > > requires more work and this is next step which I am going to do after > > applying > > this series. Currently I am going to embed EFI support into ELF file because > > it is easy (less changes; currently used ELF file has required properties > > because multiboot (v1) which we use has similar requirements like multiboot2 > > protocol) to make it compatible with multiboot2 protocol. > > I think whether what you do now makes sense depends on the > ultimate goal: If we want a single binary usable for all three cases, > then while yes, having EFI code available in the ELF image makes > sense, using an ELF Image won't work. And we can't have an > image being both ELF and PE. Hence the goal ought to be to have > a single PE image, and with that the direction you move seems > wrong. It depends how we want to generate proper PE file. There are two options. We can put manually proper PE header into xen/arch/x86/boot/head.S (maybe with some additional needed stuff). Then after build we will have ELF file which is loadable by multiboot protocols and has extra PE header. Of course it is unusable directly by EFI loader. However, using simple objcopy we can extract all interesting stuff from ELF file. This way we get proper PE file which is usable by three different boot protocols. Going that way we can also remove strict dependency on exact version of binutils which must have enabled i386pep support if we wish to build PE image. Potentially we can choose second way and build proper PE image using ld and objcopy/objdump tools with proper options. However, this require more work (maybe we will be forced to build something similar to mkelf32) and we bind Xen build machinery more tightly with exact version of binutils which is not nice. So, I decided to choose optio
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
>>> On 25.08.15 at 18:31, wrote: > On Tue, Aug 25, 2015 at 06:09:09AM -0600, Jan Beulich wrote: >> >>> On 24.08.15 at 22:54, wrote: >> > On Mon, Aug 24, 2015 at 05:35:21AM -0600, Jan Beulich wrote: >> >> >>> On 22.08.15 at 15:59, wrote: >> >> > On Thu, Aug 20, 2015 at 09:39:39AM -0600, Jan Beulich wrote: >> >> >> >>> On 20.07.15 at 16:29, wrote: >> >> >> > Currently, PE file contains many sections which are not "linear" (one >> >> >> > after another without any holes) or even do not have representation >> >> >> > in a file (e.g. BSS). In theory there is a chance that we could build >> >> >> > proper PE file using current build system. However, it means that >> >> >> >> >> >> What is "improper" about the currently built PE file? And if there is >> >> >> anything improper, did you inform the binutils maintainers of the >> >> >> problem? >> >> > >> >> > From PE loader point of view everything is OK. However, current Xen PE >> >> > image (at least build on my machines) is not usable by multiboot (v1) >> >> > or multiboot2 protocol compatible loader because it is not linear (one >> >> > section does not live immediately after another without any voids). >> >> >> >> Again - either I'm missing something (and then your explanation is >> >> not good enough) or this is (as said above) a pointless adjustment. >> > >> > Let's focus on multiboot2 protocol (multiboot (v1) is similar to multiboot2 >> > in discussed case). In general multiboot2 is able to load any file which >> > has: >> > 1. proper multiboot2 header in first 32 KiB of a given file, >> > 2. "the text and data segments must be consecutive in the OS image" >> > (The Multiboot Specification version 1.6). >> > >> > This implies that we can e.g. build valid ELF file which is also multiboot2 >> > protocol compatible image. And we does. However, we can go further. >> > Potentially we can build valid PE image which is also valid multiboot2 >> > protocol image. Although current build method does not satisfy requirement >> > number 2 because, e.g.: >> > >> > Sections: >> > Idx Name Size VMA LMA File off >> > Algn >> > 0 .text 001513d0 82d08020 82d08020 1000 >> > 2**12 >> > ^^ >> > CONTENTS, ALLOC, LOAD, CODE >> > 1 .rodata 0004de12 82d0803513e0 82d0803513e0 00153000 >> > 2**5 >> > ^^ >> > CONTENTS, ALLOC, LOAD, READONLY, DATA >> > >> > Hence, we must use special method to build PE image (I discussed that in >> > my earlier email in that topic) to do it compatible with multiboot2 >> > protocol. >> >> And you realize that we use a "special method" for building the >> current "flat" ELF image too? > > Yes, I know about that. And with that I wonder ... >> > This way one file could be loaded by native PE loader, mulitboot (v1) >> > protocol >> > (it requires relevant header but it does not interfere with PE and >> > multiboot2 >> > protocol stuff) and mutliboot2 protocol compatible loaders. Additionally, >> > if it is signed with Secure Boot signature then potentially signature could >> > be verified by UEFI itself and e.g. GRUB2. However, as I said earlier this >> > requires more work and this is next step which I am going to do after >> > applying >> > this series. Currently I am going to embed EFI support into ELF file >> > because >> > it is easy (less changes; currently used ELF file has required properties >> > because multiboot (v1) which we use has similar requirements like >> > multiboot2 >> > protocol) to make it compatible with multiboot2 protocol. >> >> I think whether what you do now makes sense depends on the >> ultimate goal: If we want a single binary usable for all three cases, >> then while yes, having EFI code available in the ELF image makes >> sense, using an ELF Image won't work. And we can't have an >> image being both ELF and PE. Hence the goal ought to be to have >> a single PE image, and with that the direction you move seems >> wrong. > > It depends how we want to generate proper PE file. There are two options. > > We can put manually proper PE header into xen/arch/x86/boot/head.S (maybe > with some additional needed stuff). Then after build we will have ELF file > which is loadable by multiboot protocols and has extra PE header. Of course > it is unusable directly by EFI loader. However, using simple objcopy we can > extract all interesting stuff from ELF file. This way we get proper PE file > which is usable by three different boot protocols. Going that way we can > also remove strict dependency on exact version of binutils which must have > enabled i386pep support if we wish to build PE image. > > Potentially we can choose second way and build proper PE image using ld and > objcopy/objdump tools with proper options. However, this require more work > (mayb
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
On Wed, Aug 26, 2015 at 12:46:22AM -0600, Jan Beulich wrote: > >>> On 25.08.15 at 18:31, wrote: > > On Tue, Aug 25, 2015 at 06:09:09AM -0600, Jan Beulich wrote: > >> >>> On 24.08.15 at 22:54, wrote: [...] > >> And you realize that we use a "special method" for building the > >> current "flat" ELF image too? > > > > Yes, I know about that. > > And with that I wonder ... > > >> > This way one file could be loaded by native PE loader, mulitboot (v1) > >> > protocol > >> > (it requires relevant header but it does not interfere with PE and > >> > multiboot2 > >> > protocol stuff) and mutliboot2 protocol compatible loaders. Additionally, > >> > if it is signed with Secure Boot signature then potentially signature > >> > could > >> > be verified by UEFI itself and e.g. GRUB2. However, as I said earlier > >> > this > >> > requires more work and this is next step which I am going to do after > >> > applying > >> > this series. Currently I am going to embed EFI support into ELF file > >> > because > >> > it is easy (less changes; currently used ELF file has required properties > >> > because multiboot (v1) which we use has similar requirements like > >> > multiboot2 > >> > protocol) to make it compatible with multiboot2 protocol. > >> > >> I think whether what you do now makes sense depends on the > >> ultimate goal: If we want a single binary usable for all three cases, > >> then while yes, having EFI code available in the ELF image makes > >> sense, using an ELF Image won't work. And we can't have an > >> image being both ELF and PE. Hence the goal ought to be to have > >> a single PE image, and with that the direction you move seems > >> wrong. > > > > It depends how we want to generate proper PE file. There are two options. > > > > We can put manually proper PE header into xen/arch/x86/boot/head.S (maybe > > with some additional needed stuff). Then after build we will have ELF file > > which is loadable by multiboot protocols and has extra PE header. Of course > > it is unusable directly by EFI loader. However, using simple objcopy we can > > extract all interesting stuff from ELF file. This way we get proper PE file > > which is usable by three different boot protocols. Going that way we can > > also remove strict dependency on exact version of binutils which must have > > enabled i386pep support if we wish to build PE image. > > > > Potentially we can choose second way and build proper PE image using ld and > > objcopy/objdump tools with proper options. However, this require more work > > (maybe we will be forced to build something similar to mkelf32) and we bind > > Xen build machinery more tightly with exact version of binutils which is > > not nice. > > > > So, I decided to choose option #1. > > ... why there's no option #3 here: Build a suitable PE image using a > tool similar to mkelf32 _without_ involving ld/objcopy (i.e. straight > from the full ELF binary that mkelf32 today uses as its input). This is variant of #1 and make sense too. However, this way we do not get extra PE header in ELF file which is also good. > > It looks simpler because we have a lot of > > needed stuff in place (e.g. Xen ELF image is currently in format usable by > > multiboot protocols). However, I think that in first step we should add EFI > > code to xen.gz because we want to load Xen using GRUB2 on EFI platforms > > ASAP. > > This patch allows us to do that. Later after getting this feature into > > upstream > > we can focus on building proper PE image with multiboot protocols support > > embedded in it. > > But for whatever we do now we should keep in mind what the end > goal is, and at least avoid making it more cumbersome to reach that > end goal. And in the end I'm not sure not going the full way at once #1 and #3 need EFI code in xen.gz. So, I do not think that we do anything wrong adding this functionality here because we need it for GRUB2 support on EFI platforms too. Hence, both things benefit from that change but one does not depend on another. > will actually turn out to be the easier route. Do you suggest that I should put this functionality (PE with multiboot headers) on top of this patch series? Well, it is possible but this series is big and I would like to avoid to make it bigger. I prefer to get current patches into Xen tree and then work on new features (it should not take so long because as I can see we almost agreed most of stuff in that case). Or if at least half patches of current series will be accepted (as I can see there is a chance to do that with v3) then I can work on this functionality and put it on top of second not applied part. Does it make sense? Daniel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
>>> On 26.08.15 at 14:33, wrote: > Do you suggest that I should put this functionality (PE with multiboot > headers) on top of this patch series? Well, it is possible but this > series is big and I would like to avoid to make it bigger. I prefer to > get current patches into Xen tree and then work on new features (it > should not take so long because as I can see we almost agreed most > of stuff in that case). Or if at least half patches of current series > will be accepted (as I can see there is a chance to do that with v3) > then I can work on this functionality and put it on top of second not > applied part. Does it make sense? Yes. I'm not objecting to deferring that part. All I want is you to make sure not to submit any changes potentially conflicting with the end goal of having a single binary (which as I understand it can only be a PE one). Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 10/23] efi: build xen.gz with EFI code
On Wed, Aug 26, 2015 at 06:40:22AM -0600, Jan Beulich wrote: > >>> On 26.08.15 at 14:33, wrote: > > Do you suggest that I should put this functionality (PE with multiboot > > headers) on top of this patch series? Well, it is possible but this > > series is big and I would like to avoid to make it bigger. I prefer to > > get current patches into Xen tree and then work on new features (it > > should not take so long because as I can see we almost agreed most > > of stuff in that case). Or if at least half patches of current series > > will be accepted (as I can see there is a chance to do that with v3) > > then I can work on this functionality and put it on top of second not > > applied part. Does it make sense? > > Yes. I'm not objecting to deferring that part. All I want is you to make Great! > sure not to submit any changes potentially conflicting with the end OK. > goal of having a single binary (which as I understand it can only be a > PE one). This is the end goal but I think that we should have transitional phase when both formats (ELF and PE) are avaibale for users. Just in case. Daniel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
