Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-21 Thread Henrique de Moraes Holschuh
On Wed, 20 Aug 2014, Tadeusz Struk wrote:
> What for? The point is that you don't have the FW binary file and you
> have to download it, which can happen automatically and this is what
> this patch does. The director where it will be stored doesn't really
> matter here.

"git pull" will at least tell you something is seriously wrong (if you know
the firmware tree should never require a forced update, that is).

I'd really appreciate if the firmware tree had signed tags or signed
commits, though.  Those at least provide decent end-to-end security (both at
rest, and while in transit) for data once it has been "git push"'d.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-21 Thread Henrique de Moraes Holschuh
On Wed, 20 Aug 2014, Tadeusz Struk wrote:
 What for? The point is that you don't have the FW binary file and you
 have to download it, which can happen automatically and this is what
 this patch does. The director where it will be stored doesn't really
 matter here.

git pull will at least tell you something is seriously wrong (if you know
the firmware tree should never require a forced update, that is).

I'd really appreciate if the firmware tree had signed tags or signed
commits, though.  Those at least provide decent end-to-end security (both at
rest, and while in transit) for data once it has been git push'd.

-- 
  One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie. -- The Silicon Valley Tarot
  Henrique Holschuh
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread Tadeusz Struk
On 08/20/2014 01:39 PM, David Woodhouse wrote:
> On Wed, 2014-08-20 at 12:21 -0700, Tadeusz Struk wrote:
>> Hi David,
>> On 08/20/2014 11:34 AM, David Woodhouse wrote:
>>> I'm not sure I understand. Precisely what fails?
>>
>> I clone a subsystem, configure it to use
>> CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:
>>
>> MK_FW   firmware/qat_895xxc.bin.gen.S
>> make[1]: *** No rule to make target `firmware/qat_895xxc.bin', needed by
>> `firmware/qat_895xxc.bin.gen.o'.  Stop.
> 
> Can't you already just use CONFIG_EXTRA_FIRMWARE_DIR ? 


What for? The point is that you don't have the FW binary file and you
have to download it, which can happen automatically and this is what
this patch does. The director where it will be stored doesn't really
matter here.

> 
>> Yes, if you use udev helper. When you want to compile in the blobs to
>> your kernel it is needed in build time, right?
> 
> Yes. But seriously: don't do that. Let firmware get loaded from
> userspace the normal way. Don't build kernel images that you can't distribute
> because they include non-GPL parts.
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread David Woodhouse
On Wed, 2014-08-20 at 12:21 -0700, Tadeusz Struk wrote:
> Hi David,
> On 08/20/2014 11:34 AM, David Woodhouse wrote:
> > I'm not sure I understand. Precisely what fails?
> 
> I clone a subsystem, configure it to use
> CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:
> 
> MK_FW   firmware/qat_895xxc.bin.gen.S
> make[1]: *** No rule to make target `firmware/qat_895xxc.bin', needed by
> `firmware/qat_895xxc.bin.gen.o'.  Stop.

Can't you already just use CONFIG_EXTRA_FIRMWARE_DIR ? 

> Yes, if you use udev helper. When you want to compile in the blobs to
> your kernel it is needed in build time, right?

Yes. But seriously: don't do that. Let firmware get loaded from
userspace the normal way. Don't build kernel images that you can't distribute
because they include non-GPL parts.

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread Valdis . Kletnieks
On Wed, 20 Aug 2014 12:21:23 -0700, Tadeusz Struk said:
> Hi David,
> On 08/20/2014 11:34 AM, David Woodhouse wrote:
> > I'm not sure I understand. Precisely what fails?
>
> I clone a subsystem, configure it to use
> CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:
>
> MK_FW   firmware/qat_895xxc.bin.gen.S
> make[1]: *** No rule to make target `firmware/qat_895xxc.bin', needed by
> `firmware/qat_895xxc.bin.gen.o'.  Stop.

Is this what you were looking for?

config EXTRA_FIRMWARE_DIR
string "Firmware blobs root directory"
depends on EXTRA_FIRMWARE != ""
default "firmware"
help
  This option controls the directory in which the kernel build system
  looks for the firmware files listed in the EXTRA_FIRMWARE option.
  The default is firmware/ in the kernel source tree, but by changing
  this option you can point it elsewhere, such as /lib/firmware/ or
  some other directory containing the firmware files.




pgp8mAFfTEg_V.pgp
Description: PGP signature


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread Tadeusz Struk
Hi David,
On 08/20/2014 11:34 AM, David Woodhouse wrote:
> I'm not sure I understand. Precisely what fails?

I clone a subsystem, configure it to use
CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:

MK_FW   firmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make target `firmware/qat_895xxc.bin', needed by
`firmware/qat_895xxc.bin.gen.o'.  Stop.

I thought it might be useful if it would pull whatever FW it needs and
not just give up. It also might be useful if one wants to "refresh" the
FW binaries. In this case one can do rm firmware/*.[bin|ihex] && make &&
make install

> 
> I don't like this patch very much. We should be removing the legacy
> firmware/ directory entirely, not patching it up.
> 
> Userspace is responsible for providing the firmware, and it should
> generally come from an entirely separate checkout of the linux-firmware
> repository.

Yes, if you use udev helper. When you want to compile in the blobs to
your kernel it is needed in build time, right?
In both cases you need the binary anyway so you can copy it manually
from linux-firmware or use this nice feature to do it for you.
If you want to remove firmware/ directory entirely then this makefile
will be gone as well so what's the problem?

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread David Woodhouse
On Wed, 2014-08-20 at 11:21 -0700, Tadeusz Struk wrote:
> Hi,
> When a binary FW file is not present in the firmware dir then the
> build will fail. 

I'm not sure I understand. Precisely what fails?

I don't like this patch very much. We should be removing the legacy
firmware/ directory entirely, not patching it up.

Userspace is responsible for providing the firmware, and it should
generally come from an entirely separate checkout of the linux-firmware
repository.

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature


[PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread Tadeusz Struk
Hi,
When a binary FW file is not present in the firmware dir then the build
will fail. This patch enables auto-download any missing FW file for both shipped
and external targets.
Generated against linux-next.

Signed-off-by: Tadeusz Struk 
---
 firmware/Makefile |   25 +++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/firmware/Makefile b/firmware/Makefile
index 5747417..3ad12f6 100644
--- a/firmware/Makefile
+++ b/firmware/Makefile
@@ -178,10 +178,31 @@ wordsize_deps := $(wildcard include/config/64bit.h 
include/config/32bit.h \
include/config/superh32.h include/config/superh64.h \
include/config/x86_32.h include/config/x86_64.h)
 
-$(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)): %: $(wordsize_deps)
+FW_URL := 
git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/plain
+
+.NOTPARALLEL: get_missing_external $(patsubst %,$(obj)/%.gen.S, 
$(fw-external-y))
+.NOTPARALLEL: get_missing_shipped $(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y))
+
+get_missing_shipped:
+   @for file in $(fw-shipped-y) $(fw-shipped-m); do \
+   if [ ! -f $(obj)/$$file ]; then \
+   wget $(FW_URL)/$$file -o /dev/null -O $(obj)/$$file; \
+   fi; \
+   done;
+
+get_missing_external:
+   @for file in $(fw-external-y); do \
+   if [ ! -f $(obj)/$$file ]; then \
+   wget $(FW_URL)/$$file -o /dev/null -O $(obj)/$$file; \
+   fi; \
+   done;
+
+$(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)): %: $(wordsize_deps) \
+   get_missing_shipped
$(call cmd,fwbin,$(patsubst %.gen.S,%,$@))
+
 $(patsubst %,$(obj)/%.gen.S, $(fw-external-y)): %: $(wordsize_deps) \
-   include/config/extra/firmware/dir.h
+   include/config/extra/firmware/dir.h get_missing_external
$(call cmd,fwbin,$(fwabs)/$(patsubst $(obj)/%.gen.S,%,$@))
 
 # The .o files depend on the binaries directly; the .S files don't.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread Tadeusz Struk
Hi,
When a binary FW file is not present in the firmware dir then the build
will fail. This patch enables auto-download any missing FW file for both shipped
and external targets.
Generated against linux-next.

Signed-off-by: Tadeusz Struk tadeusz.st...@intel.com
---
 firmware/Makefile |   25 +++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/firmware/Makefile b/firmware/Makefile
index 5747417..3ad12f6 100644
--- a/firmware/Makefile
+++ b/firmware/Makefile
@@ -178,10 +178,31 @@ wordsize_deps := $(wildcard include/config/64bit.h 
include/config/32bit.h \
include/config/superh32.h include/config/superh64.h \
include/config/x86_32.h include/config/x86_64.h)
 
-$(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)): %: $(wordsize_deps)
+FW_URL := 
git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/plain
+
+.NOTPARALLEL: get_missing_external $(patsubst %,$(obj)/%.gen.S, 
$(fw-external-y))
+.NOTPARALLEL: get_missing_shipped $(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y))
+
+get_missing_shipped:
+   @for file in $(fw-shipped-y) $(fw-shipped-m); do \
+   if [ ! -f $(obj)/$$file ]; then \
+   wget $(FW_URL)/$$file -o /dev/null -O $(obj)/$$file; \
+   fi; \
+   done;
+
+get_missing_external:
+   @for file in $(fw-external-y); do \
+   if [ ! -f $(obj)/$$file ]; then \
+   wget $(FW_URL)/$$file -o /dev/null -O $(obj)/$$file; \
+   fi; \
+   done;
+
+$(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)): %: $(wordsize_deps) \
+   get_missing_shipped
$(call cmd,fwbin,$(patsubst %.gen.S,%,$@))
+
 $(patsubst %,$(obj)/%.gen.S, $(fw-external-y)): %: $(wordsize_deps) \
-   include/config/extra/firmware/dir.h
+   include/config/extra/firmware/dir.h get_missing_external
$(call cmd,fwbin,$(fwabs)/$(patsubst $(obj)/%.gen.S,%,$@))
 
 # The .o files depend on the binaries directly; the .S files don't.

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread David Woodhouse
On Wed, 2014-08-20 at 11:21 -0700, Tadeusz Struk wrote:
 Hi,
 When a binary FW file is not present in the firmware dir then the
 build will fail. 

I'm not sure I understand. Precisely what fails?

I don't like this patch very much. We should be removing the legacy
firmware/ directory entirely, not patching it up.

Userspace is responsible for providing the firmware, and it should
generally come from an entirely separate checkout of the linux-firmware
repository.

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread Tadeusz Struk
Hi David,
On 08/20/2014 11:34 AM, David Woodhouse wrote:
 I'm not sure I understand. Precisely what fails?

I clone a subsystem, configure it to use
CONFIG_EXTRA_FIRMWARE=qat_895xcc.bin, type make  make install and get:

MK_FW   firmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make target `firmware/qat_895xxc.bin', needed by
`firmware/qat_895xxc.bin.gen.o'.  Stop.

I thought it might be useful if it would pull whatever FW it needs and
not just give up. It also might be useful if one wants to refresh the
FW binaries. In this case one can do rm firmware/*.[bin|ihex]  make 
make install

 
 I don't like this patch very much. We should be removing the legacy
 firmware/ directory entirely, not patching it up.
 
 Userspace is responsible for providing the firmware, and it should
 generally come from an entirely separate checkout of the linux-firmware
 repository.

Yes, if you use udev helper. When you want to compile in the blobs to
your kernel it is needed in build time, right?
In both cases you need the binary anyway so you can copy it manually
from linux-firmware or use this nice feature to do it for you.
If you want to remove firmware/ directory entirely then this makefile
will be gone as well so what's the problem?

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread Valdis . Kletnieks
On Wed, 20 Aug 2014 12:21:23 -0700, Tadeusz Struk said:
 Hi David,
 On 08/20/2014 11:34 AM, David Woodhouse wrote:
  I'm not sure I understand. Precisely what fails?

 I clone a subsystem, configure it to use
 CONFIG_EXTRA_FIRMWARE=qat_895xcc.bin, type make  make install and get:

 MK_FW   firmware/qat_895xxc.bin.gen.S
 make[1]: *** No rule to make target `firmware/qat_895xxc.bin', needed by
 `firmware/qat_895xxc.bin.gen.o'.  Stop.

Is this what you were looking for?

config EXTRA_FIRMWARE_DIR
string Firmware blobs root directory
depends on EXTRA_FIRMWARE != 
default firmware
help
  This option controls the directory in which the kernel build system
  looks for the firmware files listed in the EXTRA_FIRMWARE option.
  The default is firmware/ in the kernel source tree, but by changing
  this option you can point it elsewhere, such as /lib/firmware/ or
  some other directory containing the firmware files.




pgp8mAFfTEg_V.pgp
Description: PGP signature


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread David Woodhouse
On Wed, 2014-08-20 at 12:21 -0700, Tadeusz Struk wrote:
 Hi David,
 On 08/20/2014 11:34 AM, David Woodhouse wrote:
  I'm not sure I understand. Precisely what fails?
 
 I clone a subsystem, configure it to use
 CONFIG_EXTRA_FIRMWARE=qat_895xcc.bin, type make  make install and get:
 
 MK_FW   firmware/qat_895xxc.bin.gen.S
 make[1]: *** No rule to make target `firmware/qat_895xxc.bin', needed by
 `firmware/qat_895xxc.bin.gen.o'.  Stop.

Can't you already just use CONFIG_EXTRA_FIRMWARE_DIR ? 

 Yes, if you use udev helper. When you want to compile in the blobs to
 your kernel it is needed in build time, right?

Yes. But seriously: don't do that. Let firmware get loaded from
userspace the normal way. Don't build kernel images that you can't distribute
because they include non-GPL parts.

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature


Re: [PATCH] firmware: Automatically pull missing FW files

2014-08-20 Thread Tadeusz Struk
On 08/20/2014 01:39 PM, David Woodhouse wrote:
 On Wed, 2014-08-20 at 12:21 -0700, Tadeusz Struk wrote:
 Hi David,
 On 08/20/2014 11:34 AM, David Woodhouse wrote:
 I'm not sure I understand. Precisely what fails?

 I clone a subsystem, configure it to use
 CONFIG_EXTRA_FIRMWARE=qat_895xcc.bin, type make  make install and get:

 MK_FW   firmware/qat_895xxc.bin.gen.S
 make[1]: *** No rule to make target `firmware/qat_895xxc.bin', needed by
 `firmware/qat_895xxc.bin.gen.o'.  Stop.
 
 Can't you already just use CONFIG_EXTRA_FIRMWARE_DIR ? 


What for? The point is that you don't have the FW binary file and you
have to download it, which can happen automatically and this is what
this patch does. The director where it will be stored doesn't really
matter here.

 
 Yes, if you use udev helper. When you want to compile in the blobs to
 your kernel it is needed in build time, right?
 
 Yes. But seriously: don't do that. Let firmware get loaded from
 userspace the normal way. Don't build kernel images that you can't distribute
 because they include non-GPL parts.
 

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/