Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Greg KH]

On Tue, May 03, 2016 at 10:10:24AM -0700, Luis R. Rodriguez wrote:
> On Tue, May 3, 2016 at 10:07 AM, Luis R. Rodriguez  wrote:
> > Thanks! Can you confirm if any Android or Brillo builds are already using 
> > it?
> 
> Also more importantly, any chance you can provide any technical
> reasons why initramfs cannot be used, or it was decided to not use it
> on these systems? It should help others in the future as well.

Some systems just don't need it, nor want it.  We can't break working
systems, so this is not something we can remove, sorry.

thanks,

greg k-h

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Kees Cook]

On Tue, May 3, 2016 at 10:10 AM, Luis R. Rodriguez  wrote:
> On Tue, May 3, 2016 at 10:07 AM, Luis R. Rodriguez  wrote:
>> Thanks! Can you confirm if any Android or Brillo builds are already using it?
>
> Also more importantly, any chance you can provide any technical
> reasons why initramfs cannot be used, or it was decided to not use it
> on these systems? It should help others in the future as well.

In Chrome OS, the kernels are built specifically for the hardware
they're going to be on, so an initramfs was seen as a needless
additional boot step. Since Chrome OS was heavily optimized for boot
speed, it was designed to not need the initramfs at all. This is
actually enforced by the read-only boot firmware, so there's no
trivial way to _start_ using an initramfs on (existing) Chrome OS
devices either.

-Kees

-- 
Kees Cook
Chrome OS & Brillo Security

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Luis R. Rodriguez]

On Tue, May 3, 2016 at 10:10 AM, Luis R. Rodriguez  wrote:
> or it was decided

Sorry I meant to say: 'or _why_ it was decided to not use initramfs on
these systems?'

  Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Luis R. Rodriguez]

On Tue, May 3, 2016 at 10:07 AM, Luis R. Rodriguez  wrote:
> Thanks! Can you confirm if any Android or Brillo builds are already using it?

Also more importantly, any chance you can provide any technical
reasons why initramfs cannot be used, or it was decided to not use it
on these systems? It should help others in the future as well.

  Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Luis R. Rodriguez]

On Mon, May 2, 2016 at 11:41 AM, Greg KH  wrote:
> On Mon, May 02, 2016 at 11:34:33AM -0700, Kees Cook wrote:
>> On Mon, Feb 29, 2016 at 10:56 AM, Luis R. Rodriguez  wrote:
>> > On Mon, Feb 29, 2016 at 10:12:50AM +, David Woodhouse wrote:
>> >> On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
>> >> > This ports built-in firmware to use linker tables,
>> >> > this replaces the custom section solution with a
>> >> > generic solution.
>> >> >
>> >> > This also demos the use of the .rodata (SECTION_RO)
>> >> > linker tables.
>> >> >
>> >> > Tested with 0 built-in firmware, 1 and 2 built-in
>> >> > firmwares successfully.
>> >>
>> >> I think we'd do better to rip this support out entirely. It just isn't
>> >> needed; firmware can live in an initramfs and don't even need *any*
>> >> actual running userspace support to load it from there these days, do
>> >> we?
>> >
>> > I think this is reasonable if and only if we really don't know of anyone
>> > out there not able to use initramfs. I'm happy to rip it out.
>>
>> The changelog for this doesn't say anything about _why_ the change is
>> being made? (and what about other architectures.) Also, Chrome OS
>> doesn't use an initramfs (and plenty of other things don't too). Being
>> able to build monolithic kernels (e.g. Android and Brillo) with
>> builtin firmware is very handy. Please don't remove built-in firmware
>> support.
>
> I second this, we can't break existing systems at all.  I thought we
> were going to keep built-in firmware, right Luis?

Removing built-in firmware was simply a suggestion by David which we
were evaluating here -- patches were not even yet produced, although I
have them now if we wanted to rip it out. Since Kees noted it has
users, we'll keep it.

 Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Luis R. Rodriguez]

On Mon, May 2, 2016 at 11:34 AM, Kees Cook  wrote:
> On Mon, Feb 29, 2016 at 10:56 AM, Luis R. Rodriguez  wrote:
>> On Mon, Feb 29, 2016 at 10:12:50AM +, David Woodhouse wrote:
>>> On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
>>> > This ports built-in firmware to use linker tables,
>>> > this replaces the custom section solution with a
>>> > generic solution.
>>> >
>>> > This also demos the use of the .rodata (SECTION_RO)
>>> > linker tables.
>>> >
>>> > Tested with 0 built-in firmware, 1 and 2 built-in
>>> > firmwares successfully.
>>>
>>> I think we'd do better to rip this support out entirely. It just isn't
>>> needed; firmware can live in an initramfs and don't even need *any*
>>> actual running userspace support to load it from there these days, do
>>> we?
>>
>> I think this is reasonable if and only if we really don't know of anyone
>> out there not able to use initramfs. I'm happy to rip it out.
>
> The changelog for this doesn't say anything about _why_ the change is
> being made? (and what about other architectures.)

To be clear the RFC patch here is about linker table use and porting
custom table uses for a generic linker table solution. The topic of
conversation later changed to suggest that instead of porting built-in
firmware to linker tables we should just consider removing built-in
firmware all together. As for the reason _why_ we'd port built-in
firmware to linker tables, I'll be sure to add that in the next
iteration. The reason is that Linux has scattered strategies to both
extend and use custom linker sections, often requiring modifying the
custom linker script. The effort behind the linker script provides a
unified mechanism to do this, and also enables us to avoid having to
extend the custom linker script for this type of use.

> Also, Chrome OS
> doesn't use an initramfs (and plenty of other things don't too). Being
> able to build monolithic kernels (e.g. Android and Brillo) with
> builtin firmware is very handy. Please don't remove built-in firmware
> support.

Thanks! Can you confirm if any Android or Brillo builds are already using it?

  Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Greg KH]

On Mon, May 02, 2016 at 11:34:33AM -0700, Kees Cook wrote:
> On Mon, Feb 29, 2016 at 10:56 AM, Luis R. Rodriguez  wrote:
> > On Mon, Feb 29, 2016 at 10:12:50AM +, David Woodhouse wrote:
> >> On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
> >> > This ports built-in firmware to use linker tables,
> >> > this replaces the custom section solution with a
> >> > generic solution.
> >> >
> >> > This also demos the use of the .rodata (SECTION_RO)
> >> > linker tables.
> >> >
> >> > Tested with 0 built-in firmware, 1 and 2 built-in
> >> > firmwares successfully.
> >>
> >> I think we'd do better to rip this support out entirely. It just isn't
> >> needed; firmware can live in an initramfs and don't even need *any*
> >> actual running userspace support to load it from there these days, do
> >> we?
> >
> > I think this is reasonable if and only if we really don't know of anyone
> > out there not able to use initramfs. I'm happy to rip it out.
> 
> The changelog for this doesn't say anything about _why_ the change is
> being made? (and what about other architectures.) Also, Chrome OS
> doesn't use an initramfs (and plenty of other things don't too). Being
> able to build monolithic kernels (e.g. Android and Brillo) with
> builtin firmware is very handy. Please don't remove built-in firmware
> support.

I second this, we can't break existing systems at all.  I thought we
were going to keep built-in firmware, right Luis?

thanks,

greg k-h

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Kees Cook]

On Mon, Feb 29, 2016 at 10:56 AM, Luis R. Rodriguez  wrote:
> On Mon, Feb 29, 2016 at 10:12:50AM +, David Woodhouse wrote:
>> On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
>> > This ports built-in firmware to use linker tables,
>> > this replaces the custom section solution with a
>> > generic solution.
>> >
>> > This also demos the use of the .rodata (SECTION_RO)
>> > linker tables.
>> >
>> > Tested with 0 built-in firmware, 1 and 2 built-in
>> > firmwares successfully.
>>
>> I think we'd do better to rip this support out entirely. It just isn't
>> needed; firmware can live in an initramfs and don't even need *any*
>> actual running userspace support to load it from there these days, do
>> we?
>
> I think this is reasonable if and only if we really don't know of anyone
> out there not able to use initramfs. I'm happy to rip it out.

The changelog for this doesn't say anything about _why_ the change is
being made? (and what about other architectures.) Also, Chrome OS
doesn't use an initramfs (and plenty of other things don't too). Being
able to build monolithic kernels (e.g. Android and Brillo) with
builtin firmware is very handy. Please don't remove built-in firmware
support.

-Kees

-- 
Kees Cook
Chrome OS & Brillo Security

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Luis R. Rodriguez]

On Tue, Mar 1, 2016 at 9:54 AM, Luis R. Rodriguez  wrote:
> On Tue, Mar 01, 2016 at 08:10:24AM -0800, James Bottomley wrote:
>> On Mon, 2016-02-29 at 10:12 +, David Woodhouse wrote:
>> > On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
>> > > This ports built-in firmware to use linker tables,
>> > > this replaces the custom section solution with a
>> > > generic solution.
>> > >
>> > > This also demos the use of the .rodata (SECTION_RO)
>> > > linker tables.
>> > >
>> > > Tested with 0 built-in firmware, 1 and 2 built-in
>> > > firmwares successfully.
>> >
>> > I think we'd do better to rip this support out entirely. It just
>> > isn't needed; firmware can live in an initramfs and don't even need
>> > *any* actual running userspace support to load it from there these
>> > days, do we?
>>
>> We have lots of SCSI drivers with built in firmware.  The obvious
>> examples are 53c700, aic7xxx and aic79xx.  For them, we actually have
>> the firmware compilers in tree.  The firmware model they use just isn't
>> amenable to the firmware loader: they're not monolithic blobs, it's a
>> set of firmware scripts we use to handle particular operations before
>> giving control back to the host, so the firmware and the driver are
>> very much symbiotic.
>
> I'm in the process of doing some other cleanups with the firmware_class
> stuff so that odd requirements get supported but clean interfaces are
> also not hampered by these odd requirements, so I'll take a look at
> this later. If you have other oddball firmware requirements please
> let me know so I can also keep in mind.
>
>> On the other hand, I don't think any of them uses firmware sections, so
>> it's not an argument for not ripping out this type.
>
> Thanks for confirming. I'll rip this out.

Just a heads up, I've put the removal through 0-day without issues,
I'll be folding the removal into another series of changes for the
firmware API which should help compartmentalize the user mode helper
stuff as well.

 Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Luis R. Rodriguez]

On Tue, Mar 01, 2016 at 08:10:24AM -0800, James Bottomley wrote:
> On Mon, 2016-02-29 at 10:12 +, David Woodhouse wrote:
> > On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
> > > This ports built-in firmware to use linker tables,
> > > this replaces the custom section solution with a
> > > generic solution.
> > > 
> > > This also demos the use of the .rodata (SECTION_RO)
> > > linker tables.
> > > 
> > > Tested with 0 built-in firmware, 1 and 2 built-in
> > > firmwares successfully.
> > 
> > I think we'd do better to rip this support out entirely. It just 
> > isn't needed; firmware can live in an initramfs and don't even need 
> > *any* actual running userspace support to load it from there these 
> > days, do we?
> 
> We have lots of SCSI drivers with built in firmware.  The obvious
> examples are 53c700, aic7xxx and aic79xx.  For them, we actually have
> the firmware compilers in tree.  The firmware model they use just isn't
> amenable to the firmware loader: they're not monolithic blobs, it's a
> set of firmware scripts we use to handle particular operations before
> giving control back to the host, so the firmware and the driver are
> very much symbiotic.

I'm in the process of doing some other cleanups with the firmware_class
stuff so that odd requirements get supported but clean interfaces are
also not hampered by these odd requirements, so I'll take a look at
this later. If you have other oddball firmware requirements please
let me know so I can also keep in mind.

> On the other hand, I don't think any of them uses firmware sections, so
> it's not an argument for not ripping out this type.

Thanks for confirming. I'll rip this out.

  Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[James Bottomley]

On Mon, 2016-02-29 at 10:12 +, David Woodhouse wrote:
> On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
> > This ports built-in firmware to use linker tables,
> > this replaces the custom section solution with a
> > generic solution.
> > 
> > This also demos the use of the .rodata (SECTION_RO)
> > linker tables.
> > 
> > Tested with 0 built-in firmware, 1 and 2 built-in
> > firmwares successfully.
> 
> I think we'd do better to rip this support out entirely. It just 
> isn't needed; firmware can live in an initramfs and don't even need 
> *any* actual running userspace support to load it from there these 
> days, do we?

We have lots of SCSI drivers with built in firmware.  The obvious
examples are 53c700, aic7xxx and aic79xx.  For them, we actually have
the firmware compilers in tree.  The firmware model they use just isn't
amenable to the firmware loader: they're not monolithic blobs, it's a
set of firmware scripts we use to handle particular operations before
giving control back to the host, so the firmware and the driver are
very much symbiotic.

On the other hand, I don't think any of them uses firmware sections, so
it's not an argument for not ripping out this type.

James


smime.p7s
Description: S/MIME cryptographic signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Luis R. Rodriguez]

On Mon, Feb 29, 2016 at 10:12:50AM +, David Woodhouse wrote:
> On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
> > This ports built-in firmware to use linker tables,
> > this replaces the custom section solution with a
> > generic solution.
> > 
> > This also demos the use of the .rodata (SECTION_RO)
> > linker tables.
> > 
> > Tested with 0 built-in firmware, 1 and 2 built-in
> > firmwares successfully.
> 
> I think we'd do better to rip this support out entirely. It just isn't
> needed; firmware can live in an initramfs and don't even need *any*
> actual running userspace support to load it from there these days, do
> we?

I think this is reasonable if and only if we really don't know of anyone
out there not able to use initramfs. I'm happy to rip it out.

  Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[David Woodhouse]

On Fri, 2016-02-19 at 05:45 -0800, Luis R. Rodriguez wrote:
> This ports built-in firmware to use linker tables,
> this replaces the custom section solution with a
> generic solution.
> 
> This also demos the use of the .rodata (SECTION_RO)
> linker tables.
> 
> Tested with 0 built-in firmware, 1 and 2 built-in
> firmwares successfully.

I think we'd do better to rip this support out entirely. It just isn't
needed; firmware can live in an initramfs and don't even need *any*
actual running userspace support to load it from there these days, do
we?

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [RFC v2 3/7] firmware: port built-in section to linker table

[Luis R. Rodriguez]

This ports built-in firmware to use linker tables,
this replaces the custom section solution with a
generic solution.

This also demos the use of the .rodata (SECTION_RO)
linker tables.

Tested with 0 built-in firmware, 1 and 2 built-in
firmwares successfully.

Signed-off-by: Luis R. Rodriguez 
---
 arch/x86/kernel/cpu/microcode/core.c |  7 +++
 drivers/base/firmware_class.c| 11 +--
 firmware/Makefile|  2 +-
 include/asm-generic/vmlinux.lds.h|  7 ---
 4 files changed, 9 insertions(+), 18 deletions(-)

diff --git a/arch/x86/kernel/cpu/microcode/core.c 
b/arch/x86/kernel/cpu/microcode/core.c
index faec7120c508..7ee73cd64c95 100644
--- a/arch/x86/kernel/cpu/microcode/core.c
+++ b/arch/x86/kernel/cpu/microcode/core.c
@@ -99,15 +99,14 @@ static bool __init check_loader_disabled_bsp(void)
return *res;
 }
 
-extern struct builtin_fw __start_builtin_fw[];
-extern struct builtin_fw __end_builtin_fw[];
+DECLARE_LINKTABLE_RO(struct builtin_fw, builtin_fw);
 
 bool get_builtin_firmware(struct cpio_data *cd, const char *name)
 {
 #ifdef CONFIG_FW_LOADER
-   struct builtin_fw *b_fw;
+   const struct builtin_fw *b_fw;
 
-   for (b_fw = __start_builtin_fw; b_fw != __end_builtin_fw; b_fw++) {
+   LINKTABLE_FOR_EACH(b_fw, builtin_fw) {
if (!strcmp(name, b_fw->name)) {
cd->size = b_fw->size;
cd->data = b_fw->data;
diff --git a/drivers/base/firmware_class.c b/drivers/base/firmware_class.c
index b9250e564ebf..50b9cf3d0294 100644
--- a/drivers/base/firmware_class.c
+++ b/drivers/base/firmware_class.c
@@ -42,14 +42,13 @@ MODULE_LICENSE("GPL");
 
 #ifdef CONFIG_FW_LOADER
 
-extern struct builtin_fw __start_builtin_fw[];
-extern struct builtin_fw __end_builtin_fw[];
+DEFINE_LINKTABLE_RO(struct builtin_fw, builtin_fw);
 
 static bool fw_get_builtin_firmware(struct firmware *fw, const char *name)
 {
-   struct builtin_fw *b_fw;
+   const struct builtin_fw *b_fw;
 
-   for (b_fw = __start_builtin_fw; b_fw != __end_builtin_fw; b_fw++) {
+   LINKTABLE_FOR_EACH(b_fw, builtin_fw) {
if (strcmp(name, b_fw->name) == 0) {
fw->size = b_fw->size;
fw->data = b_fw->data;
@@ -62,9 +61,9 @@ static bool fw_get_builtin_firmware(struct firmware *fw, 
const char *name)
 
 static bool fw_is_builtin_firmware(const struct firmware *fw)
 {
-   struct builtin_fw *b_fw;
+   const struct builtin_fw *b_fw;
 
-   for (b_fw = __start_builtin_fw; b_fw != __end_builtin_fw; b_fw++)
+   LINKTABLE_FOR_EACH(b_fw, builtin_fw)
if (fw->data == b_fw->data)
return true;
 
diff --git a/firmware/Makefile b/firmware/Makefile
index e297e1b52636..e13549362577 100644
--- a/firmware/Makefile
+++ b/firmware/Makefile
@@ -164,7 +164,7 @@ quiet_cmd_fwbin = MK_FW   $@
  echo ".p2align $${ASM_ALIGN}" >>$@;\
  echo "_fw_$${FWSTR}_name:">>$@;\
  echo ".string \"$$FWNAME\""   >>$@;\
- echo ".section .builtin_fw,\"a\",$${PROGBITS}">>$@;\
+ echo ".section 
.rodata.tbl.builtin_fw.all,\"a\",$${PROGBITS}" >>$@;\
  echo ".p2align $${ASM_ALIGN}" >>$@;\
  echo "$${ASM_WORD} _fw_$${FWSTR}_name">>$@;\
  echo "$${ASM_WORD} _fw_$${FWSTR}_bin" >>$@;\
diff --git a/include/asm-generic/vmlinux.lds.h 
b/include/asm-generic/vmlinux.lds.h
index eb23738ef4bd..91815fb1f2fa 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -302,13 +302,6 @@
VMLINUX_SYMBOL(__end_pci_fixups_suspend_late) = .;  \
}   \
\
-   /* Built-in firmware blobs */   \
-   .builtin_fw: AT(ADDR(.builtin_fw) - LOAD_OFFSET) {  \
-   VMLINUX_SYMBOL(__start_builtin_fw) = .; \
-   *(.builtin_fw)  \
-   VMLINUX_SYMBOL(__end_builtin_fw) = .;   \
-   }   \
-   \
TRACEDATA   \
\
/* Kernel symbol table: Normal symbols */   \
-- 
2.7.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel