[LEDE-DEV] MT7621 support Jumbo frames

2017-04-23 Thread Alberto Bursi
(resent to the list because my client decided to use HTML for some reason)

That patch is a raw kernel patch, you need to integrate it in LEDE's 
buildsystem and hope that he is working with LEDE patches already applied.

Copy the patch's text in a text file and place this file in the 
/patches-* folder in your device's source folder

Your SoC is a ramips so it should be target/linux/ramips/patches-4.4 as 
he said it is for kernel 4.4.7

Give it a name that makes sure it will be added last, like 999-mypatch.patch

Then you can select your device with make menuconfig and then start a 
compilation with make and see if it goes well.

If that fails, you need to follow the tutorial as linked, and create a 
new kernel patch where you write these changes manually to each file he 
changed, then save the new patch, and then move the patch file you 
created from the quilt kernel patch folder (|platform/)| to the patch 
folder as said above.

There might be better ways, but that's what I did when I had to add new 
kernel patches.

-Alberto


On 04/22/2017 01:53 PM, Jaap Buurman wrote:
> I have never applied patches before, so I am probably making mistakes.
> The first problem I'm running into, is that it is trying to patch the
> following file:
>
> /drivers/net/ethernet/mediatek/gsw_mt7620.h
>
> However, such a file does not exist in that path in my build_dir. I
> can find gsw_mt7620.c and gsw_mt7620.o files just fine in the
> aforementioned path, but not the .h file it is trying to patch. To
> which branch does the patch apply cleanly? I am currently trying to
> patch the master branch. Should I try the 17.01 branch instead?
>
> On Wed, Apr 19, 2017 at 10:32 AM, Jaap Buurman  wrote:
>> Ah, that sounds even better :) I will try to compile and test this
>> patch tomorrow or the day after tomorrow. Will let you know if it
>> works. Thanks again for the effort you're putting into this!
>>
>> On Wed, Apr 19, 2017 at 10:23 AM, Gaetano Catalli
>>   wrote:
>>> I'm still working on this since I would like to raise the limit up to
>>> 9KB if possible. Please, let me know if this works for you.
>>>
>>> On Wed, Apr 19, 2017 at 10:18 AM, Jaap Buurman  
>>> wrote:
 Wow, this is perfect. Thank you very much. I will try to use this
 patch and compile my own image with up to 2kb frame support. Do you
 have any plans on submitting this as a PR to the LEDE git?

 On Wed, Apr 19, 2017 at 9:32 AM, Gaetano Catalli
   wrote:
> I've been working on this for a while. Apparently the embedded 5-port
> gigabit switch is able to handle packets with size up to 15KB. On the
> contrary, the GMAC, to which the switch is attached, has a limit of
> 2KB. The following is a patch that changes the max recv frame length
> to 2KB and allows to set the MTU up to that value. It is based on
> kernel 4.4.7.
>
> Signed-off-by: Gaetano Catalli
> ---
>   drivers/net/ethernet/mediatek/gsw_mt7620.h  |  6 +
>   drivers/net/ethernet/mediatek/gsw_mt7621.c  | 13 +++
>   drivers/net/ethernet/mediatek/mtk_eth_soc.c | 36 
> ++---
>   drivers/net/ethernet/mediatek/mtk_eth_soc.h |  1 +
>   drivers/net/ethernet/mediatek/soc_mt7621.c  |  2 +-
>   5 files changed, 39 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/net/ethernet/mediatek/gsw_mt7620.h
> b/drivers/net/ethernet/mediatek/gsw_mt7620.h
> index dcef9a8..ce3cd10 100644
> --- a/drivers/net/ethernet/mediatek/gsw_mt7620.h
> +++ b/drivers/net/ethernet/mediatek/gsw_mt7620.h
> @@ -45,6 +45,12 @@
>   #define GSW_REG_ISR 0x700c
>   #define GSW_REG_GPC1 0x7014
>
> +#define GSW_REG_MAC_P0_MCR 0x100
> +#define GSW_REG_MAC_P1_MCR 0x200
> +
> +// Global MAC control register
> +#define GSW_REG_GMACCR 0x30E0
> +
>   #define SYSC_REG_CHIP_REV_ID 0x0c
>   #define SYSC_REG_CFG1 0x14
>   #define RST_CTRL_MCM BIT(2)
> diff --git a/drivers/net/ethernet/mediatek/gsw_mt7621.c
> b/drivers/net/ethernet/mediatek/gsw_mt7621.c
> index 96280b4..db5d56d 100644
> --- a/drivers/net/ethernet/mediatek/gsw_mt7621.c
> +++ b/drivers/net/ethernet/mediatek/gsw_mt7621.c
> @@ -99,17 +99,20 @@ static void mt7621_hw_init(struct mt7620_gsw *gsw,
> struct device_node *np)
>usleep_range(10, 20);
>
>if ((rt_sysc_r32(SYSC_REG_CHIP_REV_ID) & 0x) == 0x0101) {
> - /* (GE1, Force 1000M/FD, FC ON, MAX_RX_LENGTH 1536) */
> - mtk_switch_w32(gsw, 0x2105e30b, 0x100);
> + /* (GE1, Force 1000M/FD, FC ON, MAX_RX_LENGTH 2k) */
> + mtk_switch_w32(gsw, 0x2305e30b, GSW_REG_MAC_P0_MCR);
>mt7530_mdio_w32(gsw, 0x3600, 0x5e30b);
>} else {
> - /* (GE1, Force 1000M/FD, FC ON, MAX_RX_LENGTH 1536) */
> - mtk_switch_w32(gsw, 0x2105e33b, 0x100);
> + /* (GE1, Force 1000M/FD, FC ON, MAX_RX_LENGTH 2k) */
> + mtk_switch_w32(gsw, 0x2305e33b, GSW_REG_MAC_P0_MCR);
>mt7530_mdio_w32(gsw, 0x

Re: [LEDE-DEV] MT7621 support Jumbo frames

2017-04-23 Thread Jaap Buurman
Thank you all for the suggestions. I've tried the following steps:

1) Add the patch in a file with a name so that it will be added last:
999-mtu.patch. This failed during the build.
2) Next, I wanted to write these changes manually to a patch with
quilt. However, the file /drivers/net/ethernet/mediatek/gsw_mt7620.h
was not there, so I could not apply those changes manually either.
That was the only file that gave me issues. The other files could be
found in the build_dir, and thus I could apply all the changes
manually. So that's where I'm stuck at the moment.

On Sun, Apr 23, 2017 at 11:34 AM, Alberto Bursi
 wrote:
> That patch is a raw kernel patch, you need to integrate it in LEDE's
> buildsystem and hope that he is working with LEDE patches already applied.
>
> Copy the patch's text in a text file and place this file in the /patches-*
> folder in your device's source folder
>
> Your SoC is a ramips so it should be target/linux/ramips/patches-4.4 as he
> said it is for kernel 4.4.7
>
> Give it a name that makes sure it will be added last, like 999-mypatch.patch
>
> Then you can select your device with make menuconfig and then start a
> compilation with make and see if it goes well.
>
> If that fails, you need to follow the tutorial as linked, and create a new
> kernel patch where you write these changes manually to each file he changed,
> then save the new patch, and then move the patch file you created from the
> quilt kernel patch folder (platform/) to the patch folder as said above.
>
> There might be better ways, but that's what I did when I had to add new
> kernel patches.
>
> -Alberto
>
>
> On 04/22/2017 01:53 PM, Jaap Buurman wrote:
>
> I have never applied patches before, so I am probably making mistakes.
> The first problem I'm running into, is that it is trying to patch the
> following file:
>
> /drivers/net/ethernet/mediatek/gsw_mt7620.h
>
> However, such a file does not exist in that path in my build_dir. I
> can find gsw_mt7620.c and gsw_mt7620.o files just fine in the
> aforementioned path, but not the .h file it is trying to patch. To
> which branch does the patch apply cleanly? I am currently trying to
> patch the master branch. Should I try the 17.01 branch instead?
>
> On Wed, Apr 19, 2017 at 10:32 AM, Jaap Buurman 
> wrote:
>
> Ah, that sounds even better :) I will try to compile and test this
> patch tomorrow or the day after tomorrow. Will let you know if it
> works. Thanks again for the effort you're putting into this!
>
> On Wed, Apr 19, 2017 at 10:23 AM, Gaetano Catalli
>  wrote:
>
> I'm still working on this since I would like to raise the limit up to
> 9KB if possible. Please, let me know if this works for you.
>
> On Wed, Apr 19, 2017 at 10:18 AM, Jaap Buurman 
> wrote:
>
> Wow, this is perfect. Thank you very much. I will try to use this
> patch and compile my own image with up to 2kb frame support. Do you
> have any plans on submitting this as a PR to the LEDE git?
>
> On Wed, Apr 19, 2017 at 9:32 AM, Gaetano Catalli
>  wrote:
>
> I've been working on this for a while. Apparently the embedded 5-port
> gigabit switch is able to handle packets with size up to 15KB. On the
> contrary, the GMAC, to which the switch is attached, has a limit of
> 2KB. The following is a patch that changes the max recv frame length
> to 2KB and allows to set the MTU up to that value. It is based on
> kernel 4.4.7.
>
> Signed-off-by: Gaetano Catalli 
> ---
>  drivers/net/ethernet/mediatek/gsw_mt7620.h  |  6 +
>  drivers/net/ethernet/mediatek/gsw_mt7621.c  | 13 +++
>  drivers/net/ethernet/mediatek/mtk_eth_soc.c | 36
> ++---
>  drivers/net/ethernet/mediatek/mtk_eth_soc.h |  1 +
>  drivers/net/ethernet/mediatek/soc_mt7621.c  |  2 +-
>  5 files changed, 39 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/net/ethernet/mediatek/gsw_mt7620.h
> b/drivers/net/ethernet/mediatek/gsw_mt7620.h
> index dcef9a8..ce3cd10 100644
> --- a/drivers/net/ethernet/mediatek/gsw_mt7620.h
> +++ b/drivers/net/ethernet/mediatek/gsw_mt7620.h
> @@ -45,6 +45,12 @@
>  #define GSW_REG_ISR 0x700c
>  #define GSW_REG_GPC1 0x7014
>
> +#define GSW_REG_MAC_P0_MCR 0x100
> +#define GSW_REG_MAC_P1_MCR 0x200
> +
> +// Global MAC control register
> +#define GSW_REG_GMACCR 0x30E0
> +
>  #define SYSC_REG_CHIP_REV_ID 0x0c
>  #define SYSC_REG_CFG1 0x14
>  #define RST_CTRL_MCM BIT(2)
> diff --git a/drivers/net/ethernet/mediatek/gsw_mt7621.c
> b/drivers/net/ethernet/mediatek/gsw_mt7621.c
> index 96280b4..db5d56d 100644
> --- a/drivers/net/ethernet/mediatek/gsw_mt7621.c
> +++ b/drivers/net/ethernet/mediatek/gsw_mt7621.c
> @@ -99,17 +99,20 @@ static void mt7621_hw_init(struct mt7620_gsw *gsw,
> struct device_node *np)
>   usleep_range(10, 20);
>
>   if ((rt_sysc_r32(SYSC_REG_CHIP_REV_ID) & 0x) == 0x0101) {
> - /* (GE1, Force 1000M/FD, FC ON, MAX_RX_LENGTH 1536) */
> - mtk_switch_w32(gsw, 0x2105e30b, 0x100);
> + /* (GE1, Force 1000M/FD, FC ON, MAX_RX_LENGTH 2k) */
> + mtk_switch_w32(gsw, 0x2305e30b, GSW_R

Re: [LEDE-DEV] MT7621 support Jumbo frames

2017-04-23 Thread Alberto Bursi
Weird, I'm not seeing the whole "mediatek" folder if I try to follow 
your steps from a fresh source download with LEDE master (trunk/HEAD, 
whatever).

I've checked the LEDE 17.01 branch and yes, there the "mediatek" folder 
is there and also that file. Kernel version is the same for both, 
4.4.61, and that "mediatek" folder isn't there in the source package 
downloaded so it must come from LEDE patches.

So go in main source folder and write "git checkout lede-17.01" then it 
should work.

-Alberto


On 04/23/2017 12:52 PM, Jaap Buurman wrote:
> Thank you all for the suggestions. I've tried the following steps:
>
> 1) Add the patch in a file with a name so that it will be added last:
> 999-mtu.patch. This failed during the build.
> 2) Next, I wanted to write these changes manually to a patch with
> quilt. However, the file /drivers/net/ethernet/mediatek/gsw_mt7620.h
> was not there, so I could not apply those changes manually either.
> That was the only file that gave me issues. The other files could be
> found in the build_dir, and thus I could apply all the changes
> manually. So that's where I'm stuck at the moment.
>
> On Sun, Apr 23, 2017 at 11:34 AM, Alberto Bursi
>  wrote:
>> That patch is a raw kernel patch, you need to integrate it in LEDE's
>> buildsystem and hope that he is working with LEDE patches already applied.
>>
>> Copy the patch's text in a text file and place this file in the /patches-*
>> folder in your device's source folder
>>
>> Your SoC is a ramips so it should be target/linux/ramips/patches-4.4 as he
>> said it is for kernel 4.4.7
>>
>> Give it a name that makes sure it will be added last, like 999-mypatch.patch
>>
>> Then you can select your device with make menuconfig and then start a
>> compilation with make and see if it goes well.
>>
>> If that fails, you need to follow the tutorial as linked, and create a new
>> kernel patch where you write these changes manually to each file he changed,
>> then save the new patch, and then move the patch file you created from the
>> quilt kernel patch folder (platform/) to the patch folder as said above.
>>
>> There might be better ways, but that's what I did when I had to add new
>> kernel patches.
>>
>> -Alberto
>>
>>
>> On 04/22/2017 01:53 PM, Jaap Buurman wrote:
>>
>> I have never applied patches before, so I am probably making mistakes.
>> The first problem I'm running into, is that it is trying to patch the
>> following file:
>>
>> /drivers/net/ethernet/mediatek/gsw_mt7620.h
>>
>> However, such a file does not exist in that path in my build_dir. I
>> can find gsw_mt7620.c and gsw_mt7620.o files just fine in the
>> aforementioned path, but not the .h file it is trying to patch. To
>> which branch does the patch apply cleanly? I am currently trying to
>> patch the master branch. Should I try the 17.01 branch instead?
>>
>> On Wed, Apr 19, 2017 at 10:32 AM, Jaap Buurman 
>> wrote:
>>
>> Ah, that sounds even better :) I will try to compile and test this
>> patch tomorrow or the day after tomorrow. Will let you know if it
>> works. Thanks again for the effort you're putting into this!
>>
>> On Wed, Apr 19, 2017 at 10:23 AM, Gaetano Catalli
>>  wrote:
>>
>> I'm still working on this since I would like to raise the limit up to
>> 9KB if possible. Please, let me know if this works for you.
>>
>> On Wed, Apr 19, 2017 at 10:18 AM, Jaap Buurman 
>> wrote:
>>
>> Wow, this is perfect. Thank you very much. I will try to use this
>> patch and compile my own image with up to 2kb frame support. Do you
>> have any plans on submitting this as a PR to the LEDE git?
>>
>> On Wed, Apr 19, 2017 at 9:32 AM, Gaetano Catalli
>>  wrote:
>>
>> I've been working on this for a while. Apparently the embedded 5-port
>> gigabit switch is able to handle packets with size up to 15KB. On the
>> contrary, the GMAC, to which the switch is attached, has a limit of
>> 2KB. The following is a patch that changes the max recv frame length
>> to 2KB and allows to set the MTU up to that value. It is based on
>> kernel 4.4.7.
>>
>> Signed-off-by: Gaetano Catalli 
>> ---
>>   drivers/net/ethernet/mediatek/gsw_mt7620.h  |  6 +
>>   drivers/net/ethernet/mediatek/gsw_mt7621.c  | 13 +++
>>   drivers/net/ethernet/mediatek/mtk_eth_soc.c | 36
>> ++---
>>   drivers/net/ethernet/mediatek/mtk_eth_soc.h |  1 +
>>   drivers/net/ethernet/mediatek/soc_mt7621.c  |  2 +-
>>   5 files changed, 39 insertions(+), 19 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/mediatek/gsw_mt7620.h
>> b/drivers/net/ethernet/mediatek/gsw_mt7620.h
>> index dcef9a8..ce3cd10 100644
>> --- a/drivers/net/ethernet/mediatek/gsw_mt7620.h
>> +++ b/drivers/net/ethernet/mediatek/gsw_mt7620.h
>> @@ -45,6 +45,12 @@
>>   #define GSW_REG_ISR 0x700c
>>   #define GSW_REG_GPC1 0x7014
>>
>> +#define GSW_REG_MAC_P0_MCR 0x100
>> +#define GSW_REG_MAC_P1_MCR 0x200
>> +
>> +// Global MAC control register
>> +#define GSW_REG_GMACCR 0x30E0
>> +
>>   #define SYSC_REG_CHIP_REV_ID 0x0c
>>   #define S

Re: [LEDE-DEV] MT7621 support Jumbo frames

2017-04-23 Thread Jaap Buurman
The mediatek folder is not there by default. It gets generated in the
during the "make" command when compiling the image by the patches. It
can be found at the following full path:

source/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7621/linux-4.4.61/drivers/net/ethernet/mediatek

I will try if I can patch the 17.01 branch. Does anyone have an idea
why gsw_mt7620.h is missing from the master branch?

On Sun, Apr 23, 2017 at 2:30 PM, Alberto Bursi  wrote:
> Weird, I'm not seeing the whole "mediatek" folder if I try to follow
> your steps from a fresh source download with LEDE master (trunk/HEAD,
> whatever).
>
> I've checked the LEDE 17.01 branch and yes, there the "mediatek" folder
> is there and also that file. Kernel version is the same for both,
> 4.4.61, and that "mediatek" folder isn't there in the source package
> downloaded so it must come from LEDE patches.
>
> So go in main source folder and write "git checkout lede-17.01" then it
> should work.
>
> -Alberto
>
>
> On 04/23/2017 12:52 PM, Jaap Buurman wrote:
>> Thank you all for the suggestions. I've tried the following steps:
>>
>> 1) Add the patch in a file with a name so that it will be added last:
>> 999-mtu.patch. This failed during the build.
>> 2) Next, I wanted to write these changes manually to a patch with
>> quilt. However, the file /drivers/net/ethernet/mediatek/gsw_mt7620.h
>> was not there, so I could not apply those changes manually either.
>> That was the only file that gave me issues. The other files could be
>> found in the build_dir, and thus I could apply all the changes
>> manually. So that's where I'm stuck at the moment.
>>
>> On Sun, Apr 23, 2017 at 11:34 AM, Alberto Bursi
>>  wrote:
>>> That patch is a raw kernel patch, you need to integrate it in LEDE's
>>> buildsystem and hope that he is working with LEDE patches already applied.
>>>
>>> Copy the patch's text in a text file and place this file in the /patches-*
>>> folder in your device's source folder
>>>
>>> Your SoC is a ramips so it should be target/linux/ramips/patches-4.4 as he
>>> said it is for kernel 4.4.7
>>>
>>> Give it a name that makes sure it will be added last, like 999-mypatch.patch
>>>
>>> Then you can select your device with make menuconfig and then start a
>>> compilation with make and see if it goes well.
>>>
>>> If that fails, you need to follow the tutorial as linked, and create a new
>>> kernel patch where you write these changes manually to each file he changed,
>>> then save the new patch, and then move the patch file you created from the
>>> quilt kernel patch folder (platform/) to the patch folder as said above.
>>>
>>> There might be better ways, but that's what I did when I had to add new
>>> kernel patches.
>>>
>>> -Alberto
>>>
>>>
>>> On 04/22/2017 01:53 PM, Jaap Buurman wrote:
>>>
>>> I have never applied patches before, so I am probably making mistakes.
>>> The first problem I'm running into, is that it is trying to patch the
>>> following file:
>>>
>>> /drivers/net/ethernet/mediatek/gsw_mt7620.h
>>>
>>> However, such a file does not exist in that path in my build_dir. I
>>> can find gsw_mt7620.c and gsw_mt7620.o files just fine in the
>>> aforementioned path, but not the .h file it is trying to patch. To
>>> which branch does the patch apply cleanly? I am currently trying to
>>> patch the master branch. Should I try the 17.01 branch instead?
>>>
>>> On Wed, Apr 19, 2017 at 10:32 AM, Jaap Buurman 
>>> wrote:
>>>
>>> Ah, that sounds even better :) I will try to compile and test this
>>> patch tomorrow or the day after tomorrow. Will let you know if it
>>> works. Thanks again for the effort you're putting into this!
>>>
>>> On Wed, Apr 19, 2017 at 10:23 AM, Gaetano Catalli
>>>  wrote:
>>>
>>> I'm still working on this since I would like to raise the limit up to
>>> 9KB if possible. Please, let me know if this works for you.
>>>
>>> On Wed, Apr 19, 2017 at 10:18 AM, Jaap Buurman 
>>> wrote:
>>>
>>> Wow, this is perfect. Thank you very much. I will try to use this
>>> patch and compile my own image with up to 2kb frame support. Do you
>>> have any plans on submitting this as a PR to the LEDE git?
>>>
>>> On Wed, Apr 19, 2017 at 9:32 AM, Gaetano Catalli
>>>  wrote:
>>>
>>> I've been working on this for a while. Apparently the embedded 5-port
>>> gigabit switch is able to handle packets with size up to 15KB. On the
>>> contrary, the GMAC, to which the switch is attached, has a limit of
>>> 2KB. The following is a patch that changes the max recv frame length
>>> to 2KB and allows to set the MTU up to that value. It is based on
>>> kernel 4.4.7.
>>>
>>> Signed-off-by: Gaetano Catalli 
>>> ---
>>>   drivers/net/ethernet/mediatek/gsw_mt7620.h  |  6 +
>>>   drivers/net/ethernet/mediatek/gsw_mt7621.c  | 13 +++
>>>   drivers/net/ethernet/mediatek/mtk_eth_soc.c | 36
>>> ++---
>>>   drivers/net/ethernet/mediatek/mtk_eth_soc.h |  1 +
>>>   drivers/net/ethernet/mediatek/soc_mt7621.c  |  2 +-
>>>   5 files changed, 3

[LEDE-DEV] Proper way to deal with "dual firmware" ar71xx devices

2017-04-23 Thread Bjørn Mork
Hello,

Many devices make use of "dual firmware" configurations, splitting the
available flash and allowing two complete and independent installations.
This works fine for devices like the Linksys WRT1900AC etc, where the
boot loader make sure the kernel command line "root=" parameter matches
the booted kernel.

It does not work so well with ar71xx devices like the Ubiquiti UniFi AC
Pro. The original firmware use this layout:

dev:size   erasesize  name
mtd0: 0006 0001 "u-boot"
mtd1: 0001 0001 "u-boot-env"
mtd2: 0079 0001 "kernel0"
mtd3: 0079 0001 "kernel1"
mtd4: 0002 0001 "bs"
mtd5: 0004 0001 "cfg"
mtd6: 0001 0001 "EEPROM"


The current LEDE images configure this as:
  MTDPARTS = 
spi0.0:384k(u-boot)ro,64k(u-boot-env)ro,7744k(firmware),7744k(ubnt-airos)ro,128k(bs)ro,256k(cfg)ro,64k(EEPROM)ro


Note that "kernel0" is statically mapped to "firmware", and that
"kernel1" (or "ubnt-airos") is made read-only.  This sort of works as
long as LEDE is installed on "kernel0". But LEDE/OpenWrt does its magic
partition splitting based of the "firmware" partifion name.  And it will
do this even if the currently booting LEDE kernel is located on
"ubnt-airos"/"kernel1".

Due to limited understanding of how the Ubiquiti U-Boot selects between
"kernel0" and "kernel1", there are instructions out there telling users
to try to install LEDE on both "kernel0" and "kernel1".  But what
happens if the boot loader is actually loading the "kernel1" image? We
will then have a system with the kernel loaded from "kernel1" but the
rootfs loaded from "kernel0".  This is bad.  When sysupgrading, the
image on "kernel0" (aka "firmare") is replaced, But the boot loader will
still continue to load the old LEDE kernel from "kernel1".  If you are
lucky, it will boot successfully using the new rootfs.  You can then use
the mtd-rw package to make "ubnt-airos" writeable and copy the new
kernel there.  Extremely confusing and unfriendly to users...

This should be fixed somehow.  But I don't know how.  The best would be
to make the kernel dynamically figure out which of the partitions it
booted from and then force the rootfs there.  But I don't know if this
can be done without the help of the boot loader?

Another option would be to make two different systems, where the command
line for the "kernel1" installation switched the order of the "firmware"
and "ubnt-airos" partitions.  But this would require the user to select
the correct image on installation. Not exactly user friendy...

Any better ideas or advice is appreciated.

Until this problem is resolved, I believe all installation instructions
for such devices should emphasize that LEDE/OpenWrt *must* be installed
on "kernel0" only!

Note that the boot loader appears to select "kernel0" or "kernel1" based
on the first bit of the "bs" partition.  This partition contains two
32bit numbers, where the first one is 0x8000 if "kernel1" is booted
and 0x if  "kernel0".  The second number appears to be a magic,
and is always 0xa34de82b (both numbers given as big endian here).  The
rest of the partition are zeroes.




Bjørn

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] early printk breaks kernel on the clearfog board

2017-04-23 Thread Josua Mayer
Hi everybody,

I noticed a serious problem with a Clearfog build that enables
CONFIG_KERNEL_EARLY_PRINTK=y.
In short: after Loading Kernel, the serial console stays completely
silent, and apparently the board does *not* boot.

This came as a serious surprise to me.
Turns out the reason for this is the usage of the old uart physical address:
CONFIG_DEBUG_UART_PHYS=0xd0012000
This is the default for if DEBUG_MVEBU_UART0
However the Clearfog Pro uses mainline u-boot, and requires:
CONFIG_DEBUG_UART_PHYS=0xf1012000

Any opinions what should be done about this?
I personally suggest to switch to what the kernel config calls "new
bootloaders".
After all this is quite the trap when somebody is so thoughtful to start
with a debug build, only to find out that is the reason his builds don't
boot.

Imre Kaloz: you are the one who committed this setting with config-4.4,
so I figured you may have some valuable insight here.

br
Josua Mayer


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Proper way to deal with "dual firmware" ar71xx devices

2017-04-23 Thread Alberto Bursi


On 04/23/2017 08:40 PM, Bjørn Mork wrote:
> Hello,
>
> Many devices make use of "dual firmware" configurations, splitting the
> available flash and allowing two complete and independent installations.
> This works fine for devices like the Linksys WRT1900AC etc, where the
> boot loader make sure the kernel command line "root=" parameter matches
> the booted kernel.
>
> It does not work so well with ar71xx devices like the Ubiquiti UniFi AC
> Pro. The original firmware use this layout:
>
> dev:size   erasesize  name
> mtd0: 0006 0001 "u-boot"
> mtd1: 0001 0001 "u-boot-env"
> mtd2: 0079 0001 "kernel0"
> mtd3: 0079 0001 "kernel1"
> mtd4: 0002 0001 "bs"
> mtd5: 0004 0001 "cfg"
> mtd6: 0001 0001 "EEPROM"
>
>
> The current LEDE images configure this as:
>MTDPARTS = 
> spi0.0:384k(u-boot)ro,64k(u-boot-env)ro,7744k(firmware),7744k(ubnt-airos)ro,128k(bs)ro,256k(cfg)ro,64k(EEPROM)ro
>
>
> Note that "kernel0" is statically mapped to "firmware", and that
> "kernel1" (or "ubnt-airos") is made read-only.  This sort of works as
> long as LEDE is installed on "kernel0". But LEDE/OpenWrt does its magic
> partition splitting based of the "firmware" partifion name.  And it will
> do this even if the currently booting LEDE kernel is located on
> "ubnt-airos"/"kernel1".
>
> Due to limited understanding of how the Ubiquiti U-Boot selects between
> "kernel0" and "kernel1", there are instructions out there telling users
> to try to install LEDE on both "kernel0" and "kernel1".  But what
> happens if the boot loader is actually loading the "kernel1" image? We
> will then have a system with the kernel loaded from "kernel1" but the
> rootfs loaded from "kernel0".  This is bad.  When sysupgrading, the
> image on "kernel0" (aka "firmare") is replaced, But the boot loader will
> still continue to load the old LEDE kernel from "kernel1".  If you are
> lucky, it will boot successfully using the new rootfs.  You can then use
> the mtd-rw package to make "ubnt-airos" writeable and copy the new
> kernel there.  Extremely confusing and unfriendly to users...
>
> This should be fixed somehow.  But I don't know how.  The best would be
> to make the kernel dynamically figure out which of the partitions it
> booted from and then force the rootfs there.  But I don't know if this
> can be done without the help of the boot loader?
>
> Another option would be to make two different systems, where the command
> line for the "kernel1" installation switched the order of the "firmware"
> and "ubnt-airos" partitions.  But this would require the user to select
> the correct image on installation. Not exactly user friendy...
>
> Any better ideas or advice is appreciated.
>
> Until this problem is resolved, I believe all installation instructions
> for such devices should emphasize that LEDE/OpenWrt *must* be installed
> on "kernel0" only!
>
> Note that the boot loader appears to select "kernel0" or "kernel1" based
> on the first bit of the "bs" partition.  This partition contains two
> 32bit numbers, where the first one is 0x8000 if "kernel1" is booted
> and 0x if  "kernel0".  The second number appears to be a magic,
> and is always 0xa34de82b (both numbers given as big endian here).  The
> rest of the partition are zeroes.
>
>
>
>
> Bjørn
>
>

Can this  "bs" partition be manipulated on installation or later? If it 
can be manipulated it's not that hard to just erase the flash with the 
first 32bit number so it is 0x00 or maybe add a check in sysupgrade 
procedure that reads that and automatically erases the cells if it finds 
them not at 0x00 already.

-Alberto
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] Proper way to deal with "dual firmware" ar71xx devices

2017-04-23 Thread Bjørn Mork
Alberto Bursi  writes:

> Can this  "bs" partition be manipulated on installation or later?

Yes.  The only thing preventing that is the default "(ro)" in the LEDE
images.  But by using mtd-rw it can be manipulated.  That's how I fixed
my installation.

I believe there are tools to manipulate it in the original firmware, but
I don't have the details.

> If it 
> can be manipulated it's not that hard to just erase the flash with the 
> first 32bit number so it is 0x00 or maybe add a check in sysupgrade 
> procedure that reads that and automatically erases the cells if it finds 
> them not at 0x00 already.

Yes.  But if we want sysupgrade to fix it, then we should make the
partition writable by default first.

We could maybe add this functionality to the "mtd" utility, similar to
the Linksys "bootcounter" feature?


Bjørn

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] dnsmasq-full should deselect dnsmasq

2017-04-23 Thread Andrew McConachie

Hi LEDE Devs,

If this is not the right place to report this please let me know.

When selecting package dnsmasq-full from make menuconfig I believe it 
should deselect package dnsmasq.


Otherwise this happens when selecting dnsmasq-full from an otherwise 
default config.



Configuring ppp-mod-pppoe.
Collected errors:
 * check_data_file_clashes: Package dnsmasq-full wants to install file 
/data/lede/build_dir/target-mips_24kc_musl/root-ar71xx/etc/hotplug.d/ntp/25-dnsmasqsec

But that file is already provided by package  * dnsmasq
 * check_data_file_clashes: Package dnsmasq-full wants to install file 
/data/lede/build_dir/target-mips_24kc_musl/root-ar71xx/etc/init.d/dnsmasq

But that file is already provided by package  * dnsmasq
 * check_data_file_clashes: Package dnsmasq-full wants to install file 
/data/lede/build_dir/target-mips_24kc_musl/root-ar71xx/usr/sbin/dnsmasq

But that file is already provided by package  * dnsmasq
 * check_data_file_clashes: Package dnsmasq-full wants to install file 
/data/lede/build_dir/target-mips_24kc_musl/root-ar71xx/usr/share/dnsmasq/rfc6761.conf

But that file is already provided by package  * dnsmasq
 * opkg_install_cmd: Cannot install package dnsmasq-full.


Thanks,
Andrew

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [RFC 04/13] procd: upgraded: add support for passing a "command" argument to stage2

2017-04-23 Thread Matthias Schiffer
This allows us to make use of upgraded in "snapshot convert" as well.

Signed-off-by: Matthias Schiffer 

Note: obviously, this patch should go into the procd repo when it is
actually applied; it is provided like this for now to allow easier testing
of the whole series.
---
 ...d-support-for-passing-a-command-argument-.patch | 104 +
 1 file changed, 104 insertions(+)
 create mode 100644 
package/system/procd/patches/0003-upgraded-add-support-for-passing-a-command-argument-.patch

diff --git 
a/package/system/procd/patches/0003-upgraded-add-support-for-passing-a-command-argument-.patch
 
b/package/system/procd/patches/0003-upgraded-add-support-for-passing-a-command-argument-.patch
new file mode 100644
index 00..5d6f39204d
--- /dev/null
+++ 
b/package/system/procd/patches/0003-upgraded-add-support-for-passing-a-command-argument-.patch
@@ -0,0 +1,104 @@
+From a90626d588abf71426f57513174427b0bf8feeab Mon Sep 17 00:00:00 2001
+Message-Id: 

+In-Reply-To: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492975837.git.mschif...@universe-factory.net>
+References: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492975837.git.mschif...@universe-factory.net>
+From: Matthias Schiffer 
+Date: Sun, 23 Apr 2017 19:04:25 +0200
+Subject: [PATCH 3/4] upgraded: add support for passing a "command" argument to
+ stage2
+
+This allows us to make use of upgraded in "snapshot convert" as well.
+
+Signed-off-by: Matthias Schiffer 
+---
+ system.c| 10 +++---
+ upgraded/upgraded.c | 13 +++--
+ 2 files changed, 14 insertions(+), 9 deletions(-)
+
+diff --git a/system.c b/system.c
+index 193c9b0..69ba57e 100644
+--- a/system.c
 b/system.c
+@@ -346,19 +346,21 @@ static int proc_signal(struct ubus_context *ctx, struct 
ubus_object *obj,
+ enum {
+   SYSUPGRADE_PATH,
+   SYSUPGRADE_PREFIX,
++  SYSUPGRADE_COMMAND,
+   __SYSUPGRADE_MAX
+ };
+ 
+ static const struct blobmsg_policy sysupgrade_policy[__SYSUPGRADE_MAX] = {
+   [SYSUPGRADE_PATH] = { .name = "path", .type = BLOBMSG_TYPE_STRING },
+   [SYSUPGRADE_PREFIX] = { .name = "prefix", .type = BLOBMSG_TYPE_STRING },
++  [SYSUPGRADE_COMMAND] = { .name = "command", .type = BLOBMSG_TYPE_STRING 
},
+ };
+ 
+ static void
+-procd_exec_upgraded(const char *prefix, char *path)
++procd_exec_upgraded(const char *prefix, char *path, char *command)
+ {
+   char *wdt_fd = watchdog_fd();
+-  char *argv[] = { "/sbin/upgraded", NULL, NULL};
++  char *argv[] = { "/sbin/upgraded", NULL, NULL, NULL};
+ 
+   if (chroot(prefix)) {
+   fprintf(stderr, "Failed to chroot for upgraded exec.\n");
+@@ -366,6 +368,7 @@ procd_exec_upgraded(const char *prefix, char *path)
+   }
+ 
+   argv[1] = path;
++  argv[2] = command;
+ 
+   DEBUG(2, "Exec to upgraded now\n");
+   if (wdt_fd) {
+@@ -395,7 +398,8 @@ static int sysupgrade(struct ubus_context *ctx, struct 
ubus_object *obj,
+   return UBUS_STATUS_INVALID_ARGUMENT;
+ 
+   procd_exec_upgraded(blobmsg_get_string(tb[SYSUPGRADE_PREFIX]),
+-  blobmsg_get_string(tb[SYSUPGRADE_PATH]));
++  blobmsg_get_string(tb[SYSUPGRADE_PATH]),
++  tb[SYSUPGRADE_COMMAND] ? 
blobmsg_get_string(tb[SYSUPGRADE_COMMAND]) : NULL);
+   return 0;
+ }
+ 
+diff --git a/upgraded/upgraded.c b/upgraded/upgraded.c
+index aa0b4ff..303edb7 100644
+--- a/upgraded/upgraded.c
 b/upgraded/upgraded.c
+@@ -39,11 +39,12 @@ static void upgrade_proc_cb(struct uloop_process *proc, 
int ret)
+   uloop_end();
+ }
+ 
+-static void sysupgrade(char *folder)
++static void sysupgrade(char *path, char *command)
+ {
+-  char *args[] = { "/sbin/sysupgrade", "nand", NULL, NULL };
++  char *args[] = { "/sbin/sysupgrade", "nand", NULL, NULL, NULL };
+ 
+-  args[2] = folder;
++  args[2] = path;
++  args[3] = command;
+   upgrade_proc.cb = upgrade_proc_cb;
+   upgrade_proc.pid = fork();
+   if (!upgrade_proc.pid) {
+@@ -80,14 +81,14 @@ int main(int argc, char **argv)
+   }
+   close(fd);
+ 
+-  if (argc != 2) {
+-  fprintf(stderr, "sysupgrade stage 2 failed, no folder 
specified\n");
++  if (argc != 2 && argc != 3) {
++  fprintf(stderr, "sysupgrade stage 2 failed, invalid command 
line\n");
+   return -1;
+   }
+ 
+   uloop_init();
+   watchdog_init(0);
+-  sysupgrade(argv[1]);
++  sysupgrade(argv[1], (argc == 3) ? argv[2] : NULL);
+   uloop_run();
+ 
+   reboot(RB_AUTOBOOT);
+-- 
+2.12.2
+
-- 
2.12.2


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [RFC 00/13] Universal staged sysupgrade

2017-04-23 Thread Matthias Schiffer
When talking about
https://bugs.lede-project.org/index.php?do=details&task_id=685 with jow
and lynxis, we came to the conclusion that the best way to fix sysupgrades
would be to generalize the staged sysupgrade used on NAND devices to work
on all targets. That is what this patchset does, with some nice extras
like staged sysupgrade support from failsafe mode (which did not work at
all on NAND devices before).

I have tested these patches on ar71xx (squashfs+jffs2), ramips (squashfs+
ubifs) and x86 (ext4), and they seems to work perfectly; they should be
tested on a lot more targets before they are applied though.

The same patchset can be found the "sysupgrade" branch of my staging tree
at https://git.lede-project.org/?p=lede/neoraider/staging.git;a=summary

Note: the diffstat looks a lot scarier than it should; most of the added
lines are the procd patches that I have included in the LEDE repo for now
to facilitate testing the series.

Matthias


Matthias Schiffer (13):
  procd: prepare NAND sysupgrade for making upgraded dynamically linked
  procd: system: always support staged sysupgrade
  procd: upgraded: link dynamically, chroot during exec
  procd: upgraded: add support for passing a "command" argument to
stage2
  procd: remove procd-nand package
  base-files: always use staged sysupgrade
  fstools: snapshot: handle jffs2 conversion using upgraded
  procd: remove code that has become unnecessary after sysupgrade
changes
  procd: init: add support for sysupgrades triggered from preinit
  base-files: sysupgrade cleanup
  base-files: add support for staged sysupgrades from failsafe mode
  ramips: sysupgrade: move nand_do_upgrade call to platform_do_upgrade
  x86: sysupgrade: explicitly rescan disk after writing partition table

 package/base-files/Makefile|  13 +-
 .../files/lib/preinit/40_run_failsafe_hook |   6 +-
 .../files/lib/preinit/99_10_failsafe_login |  11 +-
 package/base-files/files/lib/upgrade/common.sh | 167 +++--
 .../files => base-files/files/lib/upgrade}/nand.sh |  63 +
 package/base-files/files/lib/upgrade/stage2| 150 
 package/base-files/files/sbin/sysupgrade   |  73 +++---
 package/system/fstools/Makefile|   2 +-
 package/system/fstools/files/snapshot  |  16 +-
 package/system/procd/Makefile  |  41 +---
 package/system/procd/files/nand-preinit.sh |  21 --
 ...1-system-always-support-staged-sysupgrade.patch | 112 +
 ...raded-link-dynamically-chroot-during-exec.patch | 231 +
 ...d-support-for-passing-a-command-argument-.patch | 104 
 ...-that-has-become-unnecessary-after-sysupg.patch | 120 +
 ...pport-for-sysupgrades-triggered-from-prei.patch | 272 +
 .../ramips/base-files/lib/upgrade/platform.sh  |   9 +-
 .../linux/x86/base-files/lib/upgrade/platform.sh   |   6 +-
 18 files changed, 1112 insertions(+), 305 deletions(-)
 rename package/{system/procd/files => base-files/files/lib/upgrade}/nand.sh 
(89%)
 create mode 100755 package/base-files/files/lib/upgrade/stage2
 delete mode 100644 package/system/procd/files/nand-preinit.sh
 create mode 100644 
package/system/procd/patches/0001-system-always-support-staged-sysupgrade.patch
 create mode 100644 
package/system/procd/patches/0002-upgraded-link-dynamically-chroot-during-exec.patch
 create mode 100644 
package/system/procd/patches/0003-upgraded-add-support-for-passing-a-command-argument-.patch
 create mode 100644 
package/system/procd/patches/0004-Remove-code-that-has-become-unnecessary-after-sysupg.patch
 create mode 100644 
package/system/procd/patches/0005-init-add-support-for-sysupgrades-triggered-from-prei.patch

-- 
2.12.2


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [RFC 02/13] procd: system: always support staged sysupgrade

2017-04-23 Thread Matthias Schiffer
In preparation for switching all targets to the staged sysupgrade
mechanism, upgraded is always built, and the "nandupgrade" ubus method is
renamed to "sysupgrade".

To make the migration easier, support for the old name "nandupgrade" and
the "upgrade" method that will become unused with the staged sysupgrade is
retained for now.

Signed-off-by: Matthias Schiffer 

Note: obviously, this patch should go into the procd repo when it is
actually applied; it is provided like this for now to allow easier testing
of the whole series.
---
 ...1-system-always-support-staged-sysupgrade.patch | 112 +
 1 file changed, 112 insertions(+)
 create mode 100644 
package/system/procd/patches/0001-system-always-support-staged-sysupgrade.patch

diff --git 
a/package/system/procd/patches/0001-system-always-support-staged-sysupgrade.patch
 
b/package/system/procd/patches/0001-system-always-support-staged-sysupgrade.patch
new file mode 100644
index 00..a7e66e8a78
--- /dev/null
+++ 
b/package/system/procd/patches/0001-system-always-support-staged-sysupgrade.patch
@@ -0,0 +1,112 @@
+From 3462ccc0c91964ec92c1c61cde803a6504b2fb24 Mon Sep 17 00:00:00 2001
+Message-Id: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492975837.git.mschif...@universe-factory.net>
+From: Matthias Schiffer 
+Date: Fri, 21 Apr 2017 20:06:59 +0200
+Subject: [PATCH 1/4] system: always support staged sysupgrade
+
+In preparation for switching all targets to the staged sysupgrade
+mechanism, upgraded is always built, and the "nandupgrade" ubus method is
+renamed to "sysupgrade".
+
+To make the migration easier, support for the old name "nandupgrade" and
+the "upgrade" method that will become unused with the staged sysupgrade is
+retained for now.
+
+Signed-off-by: Matthias Schiffer 
+---
+ CMakeLists.txt |  4 +---
+ system.c   | 31 +--
+ 2 files changed, 14 insertions(+), 21 deletions(-)
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 444dd20..0729459 100644
+--- a/CMakeLists.txt
 b/CMakeLists.txt
+@@ -40,9 +40,7 @@ IF(ZRAM_TMPFS)
+   SET(SOURCES_ZRAM initd/zram.c)
+ ENDIF()
+ 
+-IF(BUILD_UPGRADED)
+-  add_subdirectory(upgraded)
+-ENDIF()
++add_subdirectory(upgraded)
+ 
+ ADD_EXECUTABLE(procd ${SOURCES})
+ TARGET_LINK_LIBRARIES(procd ${LIBS})
+diff --git a/system.c b/system.c
+index 1e31ce6..bb2abe5 100644
+--- a/system.c
 b/system.c
+@@ -344,12 +344,12 @@ static int proc_signal(struct ubus_context *ctx, struct 
ubus_object *obj,
+ }
+ 
+ enum {
+-  NAND_PATH,
+-  __NAND_MAX
++  SYSUPGRADE_PATH,
++  __SYSUPGRADE_MAX
+ };
+ 
+-static const struct blobmsg_policy nand_policy[__NAND_MAX] = {
+-  [NAND_PATH] = { .name = "path", .type = BLOBMSG_TYPE_STRING },
++static const struct blobmsg_policy sysupgrade_policy[__SYSUPGRADE_MAX] = {
++  [SYSUPGRADE_PATH] = { .name = "path", .type = BLOBMSG_TYPE_STRING },
+ };
+ 
+ static void
+@@ -368,20 +368,20 @@ procd_spawn_upgraded(char *path)
+   execvp(argv[0], argv);
+ }
+ 
+-static int nand_set(struct ubus_context *ctx, struct ubus_object *obj,
+-  struct ubus_request_data *req, const char *method,
+-  struct blob_attr *msg)
++static int sysupgrade(struct ubus_context *ctx, struct ubus_object *obj,
++struct ubus_request_data *req, const char *method,
++struct blob_attr *msg)
+ {
+-  struct blob_attr *tb[__NAND_MAX];
++  struct blob_attr *tb[__SYSUPGRADE_MAX];
+ 
+   if (!msg)
+   return UBUS_STATUS_INVALID_ARGUMENT;
+ 
+-  blobmsg_parse(nand_policy, __NAND_MAX, tb, blob_data(msg), 
blob_len(msg));
+-  if (!tb[NAND_PATH])
++  blobmsg_parse(sysupgrade_policy, __SYSUPGRADE_MAX, tb, blob_data(msg), 
blob_len(msg));
++  if (!tb[SYSUPGRADE_PATH])
+   return UBUS_STATUS_INVALID_ARGUMENT;
+ 
+-  procd_spawn_upgraded(blobmsg_get_string(tb[NAND_PATH]));
++  procd_spawn_upgraded(blobmsg_get_string(tb[SYSUPGRADE_PATH]));
+   fprintf(stderr, "Yikees, something went wrong. no /sbin/upgraded ?\n");
+   return 0;
+ }
+@@ -400,9 +400,8 @@ static const struct ubus_method system_methods[] = {
+   UBUS_METHOD_NOARG("reboot", system_reboot),
+   UBUS_METHOD("watchdog", watchdog_set, watchdog_policy),
+   UBUS_METHOD("signal", proc_signal, signal_policy),
+-
+-  /* must remain at the end as it ia not always loaded */
+-  UBUS_METHOD("nandupgrade", nand_set, nand_policy),
++  UBUS_METHOD("nandupgrade", sysupgrade, sysupgrade_policy),
++  UBUS_METHOD("sysupgrade", sysupgrade, sysupgrade_policy),
+ };
+ 
+ static struct ubus_object_type system_object_type =
+@@ -431,12 +430,8 @@ procd_bcast_event(char *event, struct blob_attr *msg)
+ 
+ void ubus_init_system(struct ubus_context *ctx)
+ {
+-  struct stat s;
+   int ret;
+ 
+-  if (stat("/sbin/upgraded", &s))
+-  system_object.n_methods -= 1;
+-
+   _ctx = ctx;
+   ret = ubus_add_object(ctx, &system

[LEDE-DEV] [RFC 01/13] procd: prepare NAND sysupgrade for making upgraded dynamically linked

2017-04-23 Thread Matthias Schiffer
Use install_bin to copy upgraded with all dependencies. The old name
/tmp/upgraded is temporarily retained as a symlink to avoid breaking
things.

Signed-off-by: Matthias Schiffer 
---
 package/system/procd/files/nand.sh | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/package/system/procd/files/nand.sh 
b/package/system/procd/files/nand.sh
index 01dba61644..9c831df3b4 100644
--- a/package/system/procd/files/nand.sh
+++ b/package/system/procd/files/nand.sh
@@ -194,7 +194,7 @@ nand_upgrade_prepare_ubi() {
 
 nand_do_upgrade_success() {
local conf_tar="/tmp/sysupgrade.tgz"
-   
+
sync
[ -f "$conf_tar" ] && nand_restore_config "$conf_tar"
echo "sysupgrade successful"
@@ -231,7 +231,7 @@ nand_upgrade_ubifs() {
local rootfs_length=`(cat $1 | wc -c) 2> /dev/null`
 
nand_upgrade_prepare_ubi "$rootfs_length" "ubifs" "0" "0"
-   
+
local ubidev="$( nand_find_ubi "$CI_UBIPART" )"
local root_ubivol="$(nand_find_volume $ubidev rootfs)"
ubiupdatevol /dev/$root_ubivol -s $rootfs_length $1
@@ -333,7 +333,7 @@ nand_upgrade_stage1() {
[ "$SAVE_CONFIG" != 1 -a -f "$CONF_TAR" ] &&
rm $CONF_TAR
 
-   ubus call system nandupgrade "{\"path\": \"$path\" }"
+   ubus call system nandupgrade "{\"prefix\": \"$RAM_ROOT\", 
\"path\": \"$path\" }"
exit 0
}
 }
@@ -370,6 +370,7 @@ nand_do_platform_check() {
 # $(1): file to be used for upgrade
 nand_do_upgrade() {
echo -n $1 > /tmp/sysupgrade-nand-path
-   cp /sbin/upgraded /tmp/
+   install_bin /sbin/upgraded
+   ln -s "$RAM_ROOT"/sbin/upgraded /tmp/upgraded
nand_upgrade_stage1
 }
-- 
2.12.2


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [RFC 03/13] procd: upgraded: link dynamically, chroot during exec

2017-04-23 Thread Matthias Schiffer
The chroot ensures we don't reference anything on the rootfs and is
reverted after the upgraded exec. While we're at it, also improve error
handling and logging at bit.

This change also required changes to sysupgrade, as the dynamically linked
version is expected at a different location, and libraries need to be made
available.

Signed-off-by: Matthias Schiffer 

Note: obviously, this patch should go into the procd repo when it is
actually applied; it is provided like this for now to allow easier testing
of the whole series.
---
 ...raded-link-dynamically-chroot-during-exec.patch | 231 +
 1 file changed, 231 insertions(+)
 create mode 100644 
package/system/procd/patches/0002-upgraded-link-dynamically-chroot-during-exec.patch

diff --git 
a/package/system/procd/patches/0002-upgraded-link-dynamically-chroot-during-exec.patch
 
b/package/system/procd/patches/0002-upgraded-link-dynamically-chroot-during-exec.patch
new file mode 100644
index 00..0840540334
--- /dev/null
+++ 
b/package/system/procd/patches/0002-upgraded-link-dynamically-chroot-during-exec.patch
@@ -0,0 +1,231 @@
+From d212b2f6a5e2e2779f3fc4bfb5fe217fdeed03a8 Mon Sep 17 00:00:00 2001
+Message-Id: 

+In-Reply-To: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492975837.git.mschif...@universe-factory.net>
+References: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492975837.git.mschif...@universe-factory.net>
+From: Matthias Schiffer 
+Date: Sun, 23 Apr 2017 02:28:13 +0200
+Subject: [PATCH 2/4] upgraded: link dynamically, chroot during exec
+
+The chroot ensures we don't reference anything on the rootfs and is
+reverted after the upgraded exec. While we're at it, also improve error
+handling a bit.
+
+This change also required changes to sysupgrade, as the dynamically linked
+version is expected at a different location, and libraries need to be made
+available.
+
+Signed-off-by: Matthias Schiffer 
+---
+ system.c| 25 +++--
+ upgraded/CMakeLists.txt | 10 +-
+ upgraded/upgraded.c | 26 +-
+ watchdog.c  |  9 +++--
+ watchdog.h  |  4 ++--
+ 5 files changed, 50 insertions(+), 24 deletions(-)
+
+diff --git a/system.c b/system.c
+index bb2abe5..193c9b0 100644
+--- a/system.c
 b/system.c
+@@ -345,27 +345,40 @@ static int proc_signal(struct ubus_context *ctx, struct 
ubus_object *obj,
+ 
+ enum {
+   SYSUPGRADE_PATH,
++  SYSUPGRADE_PREFIX,
+   __SYSUPGRADE_MAX
+ };
+ 
+ static const struct blobmsg_policy sysupgrade_policy[__SYSUPGRADE_MAX] = {
+   [SYSUPGRADE_PATH] = { .name = "path", .type = BLOBMSG_TYPE_STRING },
++  [SYSUPGRADE_PREFIX] = { .name = "prefix", .type = BLOBMSG_TYPE_STRING },
+ };
+ 
+ static void
+-procd_spawn_upgraded(char *path)
++procd_exec_upgraded(const char *prefix, char *path)
+ {
+   char *wdt_fd = watchdog_fd();
+-  char *argv[] = { "/tmp/upgraded", NULL, NULL};
++  char *argv[] = { "/sbin/upgraded", NULL, NULL};
++
++  if (chroot(prefix)) {
++  fprintf(stderr, "Failed to chroot for upgraded exec.\n");
++  return;
++  }
+ 
+   argv[1] = path;
+ 
+   DEBUG(2, "Exec to upgraded now\n");
+   if (wdt_fd) {
+-  watchdog_no_cloexec();
++  watchdog_set_cloexec(false);
+   setenv("WDTFD", wdt_fd, 1);
+   }
+   execvp(argv[0], argv);
++
++  /* Cleanup on failure */
++  fprintf(stderr, "Failed to exec upgraded.\n");
++  unsetenv("WDTFD");
++  watchdog_set_cloexec(true);
++  chroot(".");
+ }
+ 
+ static int sysupgrade(struct ubus_context *ctx, struct ubus_object *obj,
+@@ -378,11 +391,11 @@ static int sysupgrade(struct ubus_context *ctx, struct 
ubus_object *obj,
+   return UBUS_STATUS_INVALID_ARGUMENT;
+ 
+   blobmsg_parse(sysupgrade_policy, __SYSUPGRADE_MAX, tb, blob_data(msg), 
blob_len(msg));
+-  if (!tb[SYSUPGRADE_PATH])
++  if (!tb[SYSUPGRADE_PATH] || !tb[SYSUPGRADE_PREFIX])
+   return UBUS_STATUS_INVALID_ARGUMENT;
+ 
+-  procd_spawn_upgraded(blobmsg_get_string(tb[SYSUPGRADE_PATH]));
+-  fprintf(stderr, "Yikees, something went wrong. no /sbin/upgraded ?\n");
++  procd_exec_upgraded(blobmsg_get_string(tb[SYSUPGRADE_PREFIX]),
++  blobmsg_get_string(tb[SYSUPGRADE_PATH]));
+   return 0;
+ }
+ 
+diff --git a/upgraded/CMakeLists.txt b/upgraded/CMakeLists.txt
+index 093dba2..00d8ce5 100644
+--- a/upgraded/CMakeLists.txt
 b/upgraded/CMakeLists.txt
+@@ -2,16 +2,8 @@ cmake_minimum_required(VERSION 2.6)
+ 
+ PROJECT(upgraded C)
+ ADD_DEFINITIONS(-Os -ggdb -Wall -Werror --std=gnu99 -Wmissing-declarations)
+-set(CMAKE_EXE_LINKER_FLAGS "-static -fPIC")
+-set(CMAKE_FIND_LIBRARY_SUFFIXES .a)
+-set(CMAKE_EXE_LINK_DYNAMIC_C_FLAGS)
+-set(CMAKE_EXE_LINK_DYNAMIC_CXX_FLAGS)
+-set(CMAKE_SHARED_LIBRARY_C_FLAGS)
+-set(CMAKE_SHARED_LIBRARY_CXX_FLAGS)
+-set(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS)
+-set(CMAKE_SHARED_LIBRARY_LINK_

[LEDE-DEV] [RFC 12/13] ramips: sysupgrade: move nand_do_upgrade call to platform_do_upgrade

2017-04-23 Thread Matthias Schiffer
All targets with NAND support should gradually move their nand_do_upgrade
calls from platform_pre_upgrade to platform_do_upgrade.

Signed-off-by: Matthias Schiffer 
---
 target/linux/ramips/base-files/lib/upgrade/platform.sh | 9 +
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/target/linux/ramips/base-files/lib/upgrade/platform.sh 
b/target/linux/ramips/base-files/lib/upgrade/platform.sh
index 036e19afe3..747e368d02 100755
--- a/target/linux/ramips/base-files/lib/upgrade/platform.sh
+++ b/target/linux/ramips/base-files/lib/upgrade/platform.sh
@@ -275,7 +275,7 @@ platform_nand_pre_upgrade() {
esac
 }
 
-platform_pre_upgrade() {
+platform_do_upgrade() {
local board=$(ramips_board_name)
 
case "$board" in
@@ -284,13 +284,6 @@ platform_pre_upgrade() {
ubnt-erx)
nand_do_upgrade "$ARGV"
;;
-   esac
-}
-
-platform_do_upgrade() {
-   local board=$(ramips_board_name)
-
-   case "$board" in
*)
default_do_upgrade "$ARGV"
;;
-- 
2.12.2


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [RFC 07/13] fstools: snapshot: handle jffs2 conversion using upgraded

2017-04-23 Thread Matthias Schiffer
We can reuse the kill_remaining and run_ramfs facilities of the stage2 run
by upgraded.

Signed-off-by: Matthias Schiffer 
---
 package/system/fstools/Makefile   |  2 +-
 package/system/fstools/files/snapshot | 16 +---
 2 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/package/system/fstools/Makefile b/package/system/fstools/Makefile
index 759ea65978..db234aeecf 100644
--- a/package/system/fstools/Makefile
+++ b/package/system/fstools/Makefile
@@ -8,7 +8,7 @@
 include $(TOPDIR)/rules.mk
 
 PKG_NAME:=fstools
-PKG_RELEASE:=1
+PKG_RELEASE:=2
 
 PKG_SOURCE_PROTO:=git
 PKG_SOURCE_URL=$(LEDE_GIT)/project/fstools.git
diff --git a/package/system/fstools/files/snapshot 
b/package/system/fstools/files/snapshot
index c1a5b733f3..a495e34345 100644
--- a/package/system/fstools/files/snapshot
+++ b/package/system/fstools/files/snapshot
@@ -42,7 +42,7 @@ do_snapshot_upgrade() {
 
opkg list-upgradable
[ $? -eq 0 ] || exit 2
-   
+
UPDATES=`opkg list-upgradable | cut -d" " -f1`
[ -z "${UPDATES}" ] && exit 0
 
@@ -64,14 +64,16 @@ do_convert_jffs2() {
 do_convert() {
. /lib/functions.sh
. /lib/upgrade/common.sh
-   ubus call system upgrade
-   touch /tmp/sysupgrade
+
cd /overlay/upper
tar czf /tmp/snapshot.tar.gz *
-   kill_remaining TERM
-   sleep 3
-   kill_remaining KILL
-   run_ramfs '. /sbin/snapshot; do_convert_jffs2'
+
+   install_bin /sbin/upgraded
+   ubus call system sysupgrade "{
+   \"prefix\": \"$RAM_ROOT\",
+   \"path\": \"\",
+   \"command\": \". /sbin/snapshot; do_convert_jffs2\"
+   }"
 }
 
 [ -n "$(cat /proc/mounts|grep /overlay|grep jffs2)" ] && {
-- 
2.12.2


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [RFC 11/13] base-files: add support for staged sysupgrades from failsafe mode

2017-04-23 Thread Matthias Schiffer
Signed-off-by: Matthias Schiffer 
---
 .../base-files/files/lib/preinit/40_run_failsafe_hook  |  6 +-
 .../base-files/files/lib/preinit/99_10_failsafe_login  | 11 +++
 package/base-files/files/sbin/sysupgrade   | 18 +-
 3 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/package/base-files/files/lib/preinit/40_run_failsafe_hook 
b/package/base-files/files/lib/preinit/40_run_failsafe_hook
index 7301f77349..533b0a9771 100644
--- a/package/base-files/files/lib/preinit/40_run_failsafe_hook
+++ b/package/base-files/files/lib/preinit/40_run_failsafe_hook
@@ -5,8 +5,12 @@
 run_failsafe_hook() {
 [ "$pi_preinit_no_failsafe" = "y" ] && return
 if [ "$FAILSAFE" = "true" ]; then
+   lock /tmp/.failsafe
boot_run_hook failsafe
-   lock -w /tmp/.failsafe
+   while [ ! -e /tmp/sysupgrade ]; do
+   lock -w /tmp/.failsafe
+   done
+   exit
 fi
 }
 
diff --git a/package/base-files/files/lib/preinit/99_10_failsafe_login 
b/package/base-files/files/lib/preinit/99_10_failsafe_login
index 3147cdc5a6..728c63b2e8 100644
--- a/package/base-files/files/lib/preinit/99_10_failsafe_login
+++ b/package/base-files/files/lib/preinit/99_10_failsafe_login
@@ -8,10 +8,13 @@ failsafe_netlogin () {
 }
 
 failsafe_shell() {
-   lock /tmp/.failsafe
-   ash --login
-   echo "Please reboot system when done with failsafe network logins"
-   while true; do sleep 1; done
+   local console="$(sed -e 's/ /\n/g' /proc/cmdline | grep '^console=' | 
head -1 | sed -e 's/^console=//' -e 's/,.*//')"
+   [ -n "$console" ] || console=console
+   [ -c "/dev/$console" ] || return 0
+   while true; do
+   ash --login <"/dev/$console" >"/dev/$console" 2>"/dev/$console"
+   sleep 1
+   done &
 }
 
 boot_hook_add failsafe failsafe_netlogin
diff --git a/package/base-files/files/sbin/sysupgrade 
b/package/base-files/files/sbin/sysupgrade
index 18dfa50e05..5ec8762628 100755
--- a/package/base-files/files/sbin/sysupgrade
+++ b/package/base-files/files/sbin/sysupgrade
@@ -244,9 +244,17 @@ run_hooks "" $sysupgrade_pre_upgrade
 v "Commencing upgrade."
 [ -n "$(rootfs_type)" ] && v "All shell sessions will be closed now."
 
+COMMAND='. /lib/functions.sh; include /lib/upgrade; do_upgrade_stage2'
+
 install_bin /sbin/upgraded
-ubus call system sysupgrade "{
-   \"prefix\": \"$RAM_ROOT\",
-   \"path\": $(json_string "$IMAGE"),
-   \"command\": \". /lib/functions.sh; include /lib/upgrade; 
do_upgrade_stage2\"
-}"
+
+if [ -n "$FAILSAFE" ]; then
+   printf '%s\x00%s\x00%s' "$RAM_ROOT" "$IMAGE" "$COMMAND" >/tmp/sysupgrade
+   lock -u /tmp/.failsafe
+else
+   ubus call system sysupgrade "{
+   \"prefix\": $(json_string "$RAM_ROOT"),
+   \"path\": $(json_string "$IMAGE"),
+   \"command\": $(json_string "$COMMAND")
+   }"
+fi
-- 
2.12.2


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [RFC 08/13] procd: remove code that has become unnecessary after sysupgrade changes

2017-04-23 Thread Matthias Schiffer
Signed-off-by: Matthias Schiffer 

Note: obviously, this patch should go into the procd repo when it is
actually applied; it is provided like this for now to allow easier testing
of the whole series.
---
 ...-that-has-become-unnecessary-after-sysupg.patch | 120 +
 1 file changed, 120 insertions(+)
 create mode 100644 
package/system/procd/patches/0004-Remove-code-that-has-become-unnecessary-after-sysupg.patch

diff --git 
a/package/system/procd/patches/0004-Remove-code-that-has-become-unnecessary-after-sysupg.patch
 
b/package/system/procd/patches/0004-Remove-code-that-has-become-unnecessary-after-sysupg.patch
new file mode 100644
index 00..94cfa8e630
--- /dev/null
+++ 
b/package/system/procd/patches/0004-Remove-code-that-has-become-unnecessary-after-sysupg.patch
@@ -0,0 +1,120 @@
+From b776d938d6eeb0a6c005e1a6ea2fa2c0d18ba986 Mon Sep 17 00:00:00 2001
+Message-Id: 

+In-Reply-To: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492975837.git.mschif...@universe-factory.net>
+References: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492975837.git.mschif...@universe-factory.net>
+From: Matthias Schiffer 
+Date: Sun, 23 Apr 2017 19:06:12 +0200
+Subject: [PATCH 4/4] Remove code that has become unnecessary after sysupgrade
+ changes
+
+Signed-off-by: Matthias Schiffer 
+---
+ procd.h |  1 -
+ service/instance.c  |  2 --
+ system.c| 12 
+ upgraded/upgraded.c | 10 +-
+ 4 files changed, 5 insertions(+), 20 deletions(-)
+
+diff --git a/procd.h b/procd.h
+index 6a3..5aa3aea 100644
+--- a/procd.h
 b/procd.h
+@@ -27,7 +27,6 @@
+ #define __init __attribute__((constructor))
+ 
+ extern char *ubus_socket;
+-extern int upgrade_running;
+ 
+ void procd_connect_ubus(void);
+ void procd_reconnect_ubus(int reconnect);
+diff --git a/service/instance.c b/service/instance.c
+index 677f8eb..e5c4830 100644
+--- a/service/instance.c
 b/service/instance.c
+@@ -525,8 +525,6 @@ instance_exit(struct uloop_process *p, int ret)
+   runtime = tp.tv_sec - in->start.tv_sec;
+ 
+   DEBUG(2, "Instance %s::%s exit with error code %d after %ld seconds\n", 
in->srv->name, in->name, ret, runtime);
+-  if (upgrade_running)
+-  return;
+ 
+   uloop_timeout_cancel(&in->timeout);
+   service_event("instance.stop", in->srv->name, in->name);
+diff --git a/system.c b/system.c
+index 69ba57e..dc509ff 100644
+--- a/system.c
 b/system.c
+@@ -34,8 +34,6 @@ static struct blob_buf b;
+ static int notify;
+ static struct ubus_context *_ctx;
+ 
+-int upgrade_running = 0;
+-
+ static int system_board(struct ubus_context *ctx, struct ubus_object *obj,
+  struct ubus_request_data *req, const char *method,
+  struct blob_attr *msg)
+@@ -235,14 +233,6 @@ static int system_info(struct ubus_context *ctx, struct 
ubus_object *obj,
+   return UBUS_STATUS_OK;
+ }
+ 
+-static int system_upgrade(struct ubus_context *ctx, struct ubus_object *obj,
+-  struct ubus_request_data *req, const char *method,
+-  struct blob_attr *msg)
+-{
+-  upgrade_running = 1;
+-  return 0;
+-}
+-
+ static int system_reboot(struct ubus_context *ctx, struct ubus_object *obj,
+struct ubus_request_data *req, const char *method,
+struct blob_attr *msg)
+@@ -413,11 +403,9 @@ procd_subscribe_cb(struct ubus_context *ctx, struct 
ubus_object *obj)
+ static const struct ubus_method system_methods[] = {
+   UBUS_METHOD_NOARG("board", system_board),
+   UBUS_METHOD_NOARG("info",  system_info),
+-  UBUS_METHOD_NOARG("upgrade", system_upgrade),
+   UBUS_METHOD_NOARG("reboot", system_reboot),
+   UBUS_METHOD("watchdog", watchdog_set, watchdog_policy),
+   UBUS_METHOD("signal", proc_signal, signal_policy),
+-  UBUS_METHOD("nandupgrade", sysupgrade, sysupgrade_policy),
+   UBUS_METHOD("sysupgrade", sysupgrade, sysupgrade_policy),
+ };
+ 
+diff --git a/upgraded/upgraded.c b/upgraded/upgraded.c
+index 303edb7..79ebd37 100644
+--- a/upgraded/upgraded.c
 b/upgraded/upgraded.c
+@@ -41,10 +41,10 @@ static void upgrade_proc_cb(struct uloop_process *proc, 
int ret)
+ 
+ static void sysupgrade(char *path, char *command)
+ {
+-  char *args[] = { "/sbin/sysupgrade", "nand", NULL, NULL, NULL };
++  char *args[] = { "/lib/upgrade/stage2", NULL, NULL, NULL };
+ 
+-  args[2] = path;
+-  args[3] = command;
++  args[1] = path;
++  args[2] = command;
+   upgrade_proc.cb = upgrade_proc_cb;
+   upgrade_proc.pid = fork();
+   if (!upgrade_proc.pid) {
+@@ -81,14 +81,14 @@ int main(int argc, char **argv)
+   }
+   close(fd);
+ 
+-  if (argc != 2 && argc != 3) {
++  if (argc != 3) {
+   fprintf(stderr, "sysupgrade stage 2 failed, invalid command 
line\n");
+   return -1;
+   }
+ 
+   uloop_init();
+   watchdog_init(0);
+-  sysupgrade(argv[1], (argc == 3) ? argv[2] : NUL

[LEDE-DEV] [RFC 09/13] procd: init: add support for sysupgrades triggered from preinit

2017-04-23 Thread Matthias Schiffer
This will allow to add sysupgrade support via upgraded for failsafe mode.

Signed-off-by: Matthias Schiffer 

Note: obviously, this patch should go into the procd repo when it is
actually applied; it is provided like this for now to allow easier testing
of the whole series.
---
 ...pport-for-sysupgrades-triggered-from-prei.patch | 272 +
 1 file changed, 272 insertions(+)
 create mode 100644 
package/system/procd/patches/0005-init-add-support-for-sysupgrades-triggered-from-prei.patch

diff --git 
a/package/system/procd/patches/0005-init-add-support-for-sysupgrades-triggered-from-prei.patch
 
b/package/system/procd/patches/0005-init-add-support-for-sysupgrades-triggered-from-prei.patch
new file mode 100644
index 00..bbd7b6dd92
--- /dev/null
+++ 
b/package/system/procd/patches/0005-init-add-support-for-sysupgrades-triggered-from-prei.patch
@@ -0,0 +1,272 @@
+From 49970e3f3c5c4ba35b6566e52104cacfc9d3e939 Mon Sep 17 00:00:00 2001
+Message-Id: 
<49970e3f3c5c4ba35b6566e52104cacfc9d3e939.1492987636.git.mschif...@universe-factory.net>
+In-Reply-To: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492987636.git.mschif...@universe-factory.net>
+References: 
<3462ccc0c91964ec92c1c61cde803a6504b2fb24.1492987636.git.mschif...@universe-factory.net>
+From: Matthias Schiffer 
+Date: Mon, 24 Apr 2017 00:40:27 +0200
+Subject: [PATCH 5/5] init: add support for sysupgrades triggered from preinit
+
+This will allow to add sysupgrade support via upgraded for failsafe mode.
+
+Signed-off-by: Matthias Schiffer 
+---
+ CMakeLists.txt  |  4 ++--
+ initd/preinit.c | 46 +-
+ system.c| 35 ---
+ sysupgrade.c| 49 +
+ sysupgrade.h| 15 +++
+ watchdog.h  |  2 ++
+ 6 files changed, 113 insertions(+), 38 deletions(-)
+ create mode 100644 sysupgrade.c
+ create mode 100644 sysupgrade.h
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 0729459..7d05e97 100644
+--- a/CMakeLists.txt
 b/CMakeLists.txt
+@@ -18,7 +18,7 @@ INSTALL(TARGETS setlbf
+ )
+ 
+ 
+-SET(SOURCES procd.c signal.c state.c inittab.c rcS.c ubus.c system.c
++SET(SOURCES procd.c signal.c state.c inittab.c rcS.c ubus.c system.c 
sysupgrade.c
+   service/service.c service/instance.c service/validate.c 
service/trigger.c service/watch.c
+   utils/utils.c)
+ IF(NOT DISABLE_INIT)
+@@ -54,7 +54,7 @@ INCLUDE_DIRECTORIES(${ubox_include_dir})
+ IF(DISABLE_INIT)
+ ADD_DEFINITIONS(-DDISABLE_INIT)
+ ELSE()
+-ADD_EXECUTABLE(init initd/init.c initd/early.c initd/preinit.c initd/mkdev.c 
watchdog.c
++ADD_EXECUTABLE(init initd/init.c initd/early.c initd/preinit.c initd/mkdev.c 
sysupgrade.c watchdog.c
+   utils/utils.c ${SOURCES_ZRAM})
+ TARGET_LINK_LIBRARIES(init ${LIBS})
+ INSTALL(TARGETS init
+diff --git a/initd/preinit.c b/initd/preinit.c
+index 729978e..dda6576 100644
+--- a/initd/preinit.c
 b/initd/preinit.c
+@@ -27,6 +27,7 @@
+ 
+ #include "init.h"
+ #include "../watchdog.h"
++#include "../sysupgrade.h"
+ 
+ static struct uloop_process preinit_proc;
+ static struct uloop_process plugd_proc;
+@@ -49,23 +50,58 @@ check_dbglvl(void)
+ }
+ 
+ static void
++check_sysupgrade(void)
++{
++  char *prefix = NULL, *path = NULL, *command = NULL;
++  size_t n;
++
++  if (chdir("/"))
++  return;
++
++  FILE *sysupgrade = fopen("/tmp/sysupgrade", "r");
++  if (!sysupgrade)
++  return;
++
++  n = 0;
++  if (getdelim(&prefix, &n, 0, sysupgrade) < 0)
++  goto fail;
++  n = 0;
++  if (getdelim(&path, &n, 0, sysupgrade) < 0)
++  goto fail;
++  n = 0;
++  if (getdelim(&command, &n, 0, sysupgrade) < 0)
++  goto fail;
++
++  fclose(sysupgrade);
++
++  sysupgrade_exec_upgraded(prefix, path, command);
++
++  while (true)
++  sleep(1);
++
++fail:
++  fclose(sysupgrade);
++  free(prefix);
++  free(path);
++  free(command);
++}
++
++static void
+ spawn_procd(struct uloop_process *proc, int ret)
+ {
+   char *wdt_fd = watchdog_fd();
+   char *argv[] = { "/sbin/procd", NULL};
+-  struct stat s;
+   char dbg[2];
+ 
+   if (plugd_proc.pid > 0)
+   kill(plugd_proc.pid, SIGKILL);
+ 
+-  if (!stat("/tmp/sysupgrade", &s))
+-  while (true)
+-  sleep(1);
+-
+   unsetenv("INITRAMFS");
+   unsetenv("PREINIT");
+   unlink("/tmp/.preinit");
++
++  check_sysupgrade();
++
+   DEBUG(2, "Exec to real procd now\n");
+   if (wdt_fd)
+   setenv("WDTFD", wdt_fd, 1);
+diff --git a/system.c b/system.c
+index dc509ff..8b8cdbc 100644
+--- a/system.c
 b/system.c
+@@ -28,6 +28,7 @@
+ #include 
+ 
+ #include "procd.h"
++#include "sysupgrade.h"
+ #include "watchdog.h"
+ 
+ static struct blob_buf b;
+@@ -346,34 +347,6 @@ static const struct blobmsg_policy 
sysupgrade_policy[__SYSUPGRADE_MAX] = {
+  

[LEDE-DEV] [RFC 05/13] procd: remove procd-nand package

2017-04-23 Thread Matthias Schiffer
We always want to support staged upgrades now, so it's better to include
upgraded into the main package. /lib/upgrade/nand.sh is moved to
base-files.

The procd-nand-firstboot package is removed for now, it may return later
as a separate package.

Signed-off-by: Matthias Schiffer 
---
 package/base-files/Makefile| 13 +--
 .../files => base-files/files/lib/upgrade}/nand.sh |  0
 package/system/procd/Makefile  | 41 ++
 package/system/procd/files/nand-preinit.sh | 21 ---
 4 files changed, 14 insertions(+), 61 deletions(-)
 rename package/{system/procd/files => base-files/files/lib/upgrade}/nand.sh 
(100%)
 delete mode 100644 package/system/procd/files/nand-preinit.sh

diff --git a/package/base-files/Makefile b/package/base-files/Makefile
index 512fa66ca7..912449c85a 100644
--- a/package/base-files/Makefile
+++ b/package/base-files/Makefile
@@ -18,7 +18,9 @@ PKG_FILE_DEPENDS:=$(PLATFORM_DIR)/ 
$(GENERIC_PLATFORM_DIR)/base-files/
 PKG_BUILD_DEPENDS:=usign/host
 PKG_LICENSE:=GPL-2.0
 
-PKG_CONFIG_DEPENDS := CONFIG_SIGNED_PACKAGES CONFIG_TARGET_INIT_PATH 
CONFIG_TARGET_PREINIT_DISABLE_FAILSAFE
+PKG_CONFIG_DEPENDS := \
+   CONFIG_SIGNED_PACKAGES CONFIG_TARGET_INIT_PATH 
CONFIG_TARGET_PREINIT_DISABLE_FAILSAFE \
+   CONFIG_NAND_SUPPORT
 
 include $(INCLUDE_DIR)/package.mk
 
@@ -30,7 +32,7 @@ endif
 define Package/base-files
   SECTION:=base
   CATEGORY:=Base system
-  DEPENDS:=+netifd +libc +procd +jsonfilter +SIGNED_PACKAGES:usign 
+SIGNED_PACKAGES:lede-keyring +fstools +fwtool
+  DEPENDS:=+netifd +libc +procd +jsonfilter +SIGNED_PACKAGES:usign 
+SIGNED_PACKAGES:lede-keyring +NAND_SUPPORT:ubi-utils +fstools +fwtool
   TITLE:=Base filesystem for Lede
   URL:=http://openwrt.org/
   VERSION:=$(PKG_RELEASE)-$(REVISION)
@@ -105,9 +107,16 @@ ifdef CONFIG_SIGNED_PACKAGES
   endef
 endif
 
+ifeq ($(CONFIG_NAND_SUPPORT),)
+  define Package/base-files/nand-support
+   rm -f $(1)/lib/upgrade/nand.sh
+  endef
+endif
+
 define Package/base-files/install
$(CP) ./files/* $(1)/
$(Package/base-files/install-key)
+   $(Package/base-files/nand-support)
if [ -d $(GENERIC_PLATFORM_DIR)/base-files/. ]; then \
$(CP) $(GENERIC_PLATFORM_DIR)/base-files/* $(1)/; \
fi
diff --git a/package/system/procd/files/nand.sh 
b/package/base-files/files/lib/upgrade/nand.sh
similarity index 100%
rename from package/system/procd/files/nand.sh
rename to package/base-files/files/lib/upgrade/nand.sh
diff --git a/package/system/procd/Makefile b/package/system/procd/Makefile
index 43ea23aab0..d7b8a058a9 100644
--- a/package/system/procd/Makefile
+++ b/package/system/procd/Makefile
@@ -22,11 +22,9 @@ PKG_LICENSE_FILES:=
 
 PKG_MAINTAINER:=John Crispin 
 
-PKG_FLAGS:=nonshared
-
 PKG_CONFIG_DEPENDS:= \
CONFIG_TARGET_INIT_PATH CONFIG_KERNEL_SECCOMP \
-   CONFIG_NAND_SUPPORT CONFIG_PROCD_SHOW_BOOT CONFIG_PROCD_ZRAM_TMPFS \
+   CONFIG_PROCD_SHOW_BOOT CONFIG_PROCD_ZRAM_TMPFS \
CONFIG_KERNEL_NAMESPACES CONFIG_PACKAGE_procd-ujail 
CONFIG_PACKAGE_procd-seccomp
 
 include $(INCLUDE_DIR)/package.mk
@@ -42,7 +40,7 @@ TARGET_LDFLAGS += $(if $(CONFIG_USE_GLIBC),-lrt)
 define Package/procd
   SECTION:=base
   CATEGORY:=Base system
-  DEPENDS:=+ubusd +ubus +libjson-script +ubox +USE_GLIBC:librt +libubox 
+libubus +NAND_SUPPORT:procd-nand
+  DEPENDS:=+ubusd +ubus +libjson-script +ubox +USE_GLIBC:librt +libubox 
+libubus
   TITLE:=OpenWrt system process manager
 endef
 
@@ -60,20 +58,6 @@ define Package/procd-seccomp
   TITLE:=OpenWrt process seccomp helper + utrace
 endef
 
-define Package/procd-nand
-  SECTION:=utils
-  CATEGORY:=Utilities
-  DEPENDS:=@NAND_SUPPORT +ubi-utils
-  TITLE:=OpenWrt sysupgrade nand helper
-endef
-
-define Package/procd-nand-firstboot
-  SECTION:=utils
-  CATEGORY:=Utilities
-  DEPENDS:=procd-nand
-  TITLE:=OpenWrt firstboot nand helper
-endef
-
 define Package/procd/config
 menu "Configuration"
depends on PACKAGE_procd
@@ -91,10 +75,6 @@ endmenu
 endef
 
 
-ifeq ($(CONFIG_NAND_SUPPORT),y)
-  CMAKE_OPTIONS += -DBUILD_UPGRADED=1
-endif
-
 ifeq ($(CONFIG_PROCD_SHOW_BOOT),y)
   CMAKE_OPTIONS += -DSHOW_BOOT_ON_CONSOLE=1
 endif
@@ -114,7 +94,7 @@ endif
 define Package/procd/install
$(INSTALL_DIR) $(1)/sbin $(1)/etc $(1)/lib/functions
 
-   $(INSTALL_BIN) 
$(PKG_INSTALL_DIR)/usr/sbin/{init,procd,askfirst,udevtrigger} $(1)/sbin/
+   $(INSTALL_BIN) 
$(PKG_INSTALL_DIR)/usr/sbin/{init,procd,askfirst,udevtrigger,upgraded} 
$(1)/sbin/
$(INSTALL_DATA) $(PKG_INSTALL_DIR)/usr/lib/libsetlbf.so $(1)/lib
$(INSTALL_BIN) ./files/reload_config $(1)/sbin/
$(INSTALL_DATA) ./files/hotplug*.json $(1)/etc/
@@ -133,21 +113,6 @@ define Package/procd-seccomp/install
$(INSTALL_DATA) $(PKG_INSTALL_DIR)/usr/lib/libpreload-trace.so $(1)/lib
 endef
 
-define Package/procd-nand/install
-   $(INSTALL_DIR) $(1)/sbin $(1)/lib/upgrade
-
-   $(INSTALL_BIN)

[LEDE-DEV] [RFC 13/13] x86: sysupgrade: explicitly rescan disk after writing partition table

2017-04-23 Thread Matthias Schiffer
This should ensure that the kernel partition can be mounted in
platform_copy_config when its size has changed. The following "sleep"
becomes unnecessary and is removed (it was skipped anyways when a change of
the partition table was detected).

While we're at it, also remove the ask_bool that can't work anymore with
staged sysupgrade.

Signed-off-by: Matthias Schiffer 
---
 target/linux/x86/base-files/lib/upgrade/platform.sh | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/target/linux/x86/base-files/lib/upgrade/platform.sh 
b/target/linux/x86/base-files/lib/upgrade/platform.sh
index d3e9f360aa..0beef1703c 100644
--- a/target/linux/x86/base-files/lib/upgrade/platform.sh
+++ b/target/linux/x86/base-files/lib/upgrade/platform.sh
@@ -37,9 +37,8 @@ platform_do_upgrade() {
diff="$(grep -F -x -v -f /tmp/partmap.bootdisk 
/tmp/partmap.image)"
if [ -n "$diff" ]; then
echo "Partition layout is changed.  Full image 
will be written."
-   ask_bool 0 "Abort" && exit
-
get_image "$@" | dd of="/dev/$diskdev" bs=4096 
conv=fsync
+   partx -u - "/dev/$diskdev"
return 0
fi
 
@@ -58,8 +57,7 @@ platform_do_upgrade() {
get_image "$@" | dd of="/dev/$diskdev" bs=1 skip=440 
count=4 seek=440 conv=fsync
else
get_image "$@" | dd of="/dev/$diskdev" bs=4096 
conv=fsync
+   partx -u - "/dev/$diskdev"
fi
-
-   sleep 1
fi
 }
-- 
2.12.2


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [RFC 06/13] base-files: always use staged sysupgrade

2017-04-23 Thread Matthias Schiffer
Support for the -d and -p options is dropped; it may be added again at some
point by adding these flags to the ubus sysupgrade call.

A downside of this is that we get a lot less information about the progress
of the upgrade: as soon as the actual upgrade starts, all shell sessions
are killed to allow unmounting the root filesystem.

Signed-off-by: Matthias Schiffer 
---
 package/base-files/files/lib/upgrade/common.sh | 123 +++--
 package/base-files/files/lib/upgrade/nand.sh   |  60 ++--
 package/base-files/files/lib/upgrade/stage2|  50 ++
 package/base-files/files/sbin/sysupgrade   |  75 ---
 4 files changed, 157 insertions(+), 151 deletions(-)
 create mode 100755 package/base-files/files/lib/upgrade/stage2

diff --git a/package/base-files/files/lib/upgrade/common.sh 
b/package/base-files/files/lib/upgrade/common.sh
index e3519ca2cb..17248c2b1d 100644
--- a/package/base-files/files/lib/upgrade/common.sh
+++ b/package/base-files/files/lib/upgrade/common.sh
@@ -56,7 +56,6 @@ run_ramfs() { #  [...]
/bin/rm /usr/bin/basename /bin/kill /bin/chmod /usr/bin/find \
/bin/mknod
 
-   install_bin /bin/uclient-fetch /bin/wget
install_bin /sbin/mtd
install_bin /sbin/mount_root
install_bin /sbin/snapshot
@@ -96,51 +95,37 @@ run_ramfs() { #  [...]
exec /bin/busybox ash -c "$*"
 }
 
-kill_remaining() { # [  ]
+kill_remaining() { # [  [  ] ]
local sig="${1:-TERM}"
+   local loop="${2:-0}"
+   local run=true
+   local stat
+
echo -n "Sending $sig to remaining processes ... "
 
-   local my_pid=$$
-   local my_ppid=$(cut -d' ' -f4  /proc/$my_pid/stat)
-   local my_ppisupgraded=
-   grep -q upgraded /proc/$my_ppid/cmdline >/dev/null && {
-   local my_ppisupgraded=1
-   }
-   
-   local stat
-   for stat in /proc/[0-9]*/stat; do
-   [ -f "$stat" ] || continue
-
-   local pid name state ppid rest
-   read pid name state ppid rest < $stat
-   name="${name#(}"; name="${name%)}"
-
-   local cmdline
-   read cmdline < /proc/$pid/cmdline
-
-   # Skip kernel threads
-   [ -n "$cmdline" ] || continue
-
-   if [ $$ -eq 1 ] || [ $my_ppid -eq 1 ] && [ -n 
"$my_ppisupgraded" ]; then
-   # Running as init process, kill everything except me
-   if [ $pid -ne $$ ] && [ $pid -ne $my_ppid ]; then
-   echo -n "$name "
-   kill -$sig $pid 2>/dev/null
-   fi
-   else 
-   case "$name" in
-   # Skip essential services
-   
*procd*|*ash*|*init*|*watchdog*|*ssh*|*dropbear*|*telnet*|*login*|*hostapd*|*wpa_supplicant*|*nas*|*relayd*)
 : ;;
-
-   # Killable process
-   *)
-   if [ $pid -ne $$ ] && [ $ppid -ne $$ ]; 
then
-   echo -n "$name "
-   kill -$sig $pid 2>/dev/null
-   fi
-   ;;
-   esac
-   fi
+   while $run; do
+   run=false
+   for stat in /proc/[0-9]*/stat; do
+   [ -f "$stat" ] || continue
+
+   local pid name state ppid rest
+   read pid name state ppid rest < $stat
+   name="${name#(}"; name="${name%)}"
+
+   # Skip PID1, ourself and our children
+   [ $pid -ne 1 -a $pid -ne $$ -a $ppid -ne $$ ] || 
continue
+
+   local cmdline
+   read cmdline < /proc/$pid/cmdline
+
+   # Skip kernel threads
+   [ -n "$cmdline" ] || continue
+
+   echo -n "$name "
+   kill -$sig $pid 2>/dev/null
+
+   [ $loop -eq 1 ] && run=true
+   done
done
echo ""
 }
@@ -175,28 +160,31 @@ v() {
[ "$VERBOSE" -ge 1 ] && echo "$@"
 }
 
+json_string() {
+   local v="$1"
+   v="${v//\\/}"
+   v="${v//\"/\\\"}"
+   echo "\"$v\""
+}
+
 rootfs_type() {
/bin/mount | awk '($3 ~ /^\/$/) && ($5 !~ /rootfs/) { print $5 }'
 }
 
 get_image() { #  [  ]
local from="$1"
-   local conc="$2"
-   local cmd
-
-   case "$from" in
-   http://*|ftp://*) cmd="wget -O- -q";;
-   *) cmd="cat";;
-   esac
-   if [ -z "$conc" ]; then
-   local magic="$(eval $cmd \"$from\" 2>/dev/null | dd bs=2 
count=1 2>/dev/null | hexdump -n 2 -e '1/1 "%02x"')"
+   local cat="$2"
+
+   if [ -z "$cat" ]; then
+   lo

[LEDE-DEV] [RFC 10/13] base-files: sysupgrade cleanup

2017-04-23 Thread Matthias Schiffer
Some functions only used by stage2 are moved there from common.sh.

One piece that could still use more cleanup is platform_pre_upgrade: many
targets reference files from there are aren't available in the ramfs, so
we need to evaluate it before the switch; conversely, flash writes happen
in that function on some targets. Targets that do the latter should be
fixed eventually to use platform_do_upgrade for that purpose.

Signed-off-by: Matthias Schiffer 
---
 package/base-files/files/lib/upgrade/common.sh | 100 -
 package/base-files/files/lib/upgrade/stage2| 120 ++---
 package/base-files/files/sbin/sysupgrade   |   6 +-
 3 files changed, 113 insertions(+), 113 deletions(-)

diff --git a/package/base-files/files/lib/upgrade/common.sh 
b/package/base-files/files/lib/upgrade/common.sh
index 17248c2b1d..fc59bf2323 100644
--- a/package/base-files/files/lib/upgrade/common.sh
+++ b/package/base-files/files/lib/upgrade/common.sh
@@ -30,106 +30,6 @@ install_bin() { #  [  ... ]
}; done
 }
 
-supivot() { #  
-   /bin/mount | grep "on $1 type" 2>&- 1>&- || /bin/mount -o bind $1 $1
-   mkdir -p $1$2 $1/proc $1/sys $1/dev $1/tmp $1/overlay && \
-   /bin/mount -o noatime,move /proc $1/proc && \
-   pivot_root $1 $1$2 || {
-   /bin/umount -l $1 $1
-   return 1
-   }
-
-   /bin/mount -o noatime,move $2/sys /sys
-   /bin/mount -o noatime,move $2/dev /dev
-   /bin/mount -o noatime,move $2/tmp /tmp
-   /bin/mount -o noatime,move $2/overlay /overlay 2>&-
-   return 0
-}
-
-run_ramfs() { #  [...]
-   install_bin /bin/busybox /bin/ash /bin/sh /bin/mount /bin/umount
\
-   /sbin/pivot_root /sbin/reboot /bin/sync /bin/dd /bin/grep   
\
-   /bin/cp /bin/mv /bin/tar /usr/bin/md5sum "/usr/bin/[" /bin/dd   
\
-   /bin/vi /bin/ls /bin/cat /usr/bin/awk /usr/bin/hexdump  
\
-   /bin/sleep /bin/zcat /usr/bin/bzcat /usr/bin/printf /usr/bin/wc 
\
-   /bin/cut /usr/bin/printf /bin/sync /bin/mkdir /bin/rmdir
\
-   /bin/rm /usr/bin/basename /bin/kill /bin/chmod /usr/bin/find \
-   /bin/mknod
-
-   install_bin /sbin/mtd
-   install_bin /sbin/mount_root
-   install_bin /sbin/snapshot
-   install_bin /sbin/snapshot_tool
-   install_bin /usr/sbin/ubiupdatevol
-   install_bin /usr/sbin/ubiattach
-   install_bin /usr/sbin/ubiblock
-   install_bin /usr/sbin/ubiformat
-   install_bin /usr/sbin/ubidetach
-   install_bin /usr/sbin/ubirsvol
-   install_bin /usr/sbin/ubirmvol
-   install_bin /usr/sbin/ubimkvol
-   install_bin /usr/sbin/partx
-   install_bin /usr/sbin/losetup
-   install_bin /usr/sbin/mkfs.ext4
-   for file in $RAMFS_COPY_BIN; do
-   install_bin ${file//:/ }
-   done
-   install_file /etc/resolv.conf /lib/*.sh /lib/functions/*.sh 
/lib/upgrade/*.sh $RAMFS_COPY_DATA
-
-   [ -L "/lib64" ] && ln -s /lib $RAM_ROOT/lib64
-
-   supivot $RAM_ROOT /mnt || {
-   echo "Failed to switch over to ramfs. Please reboot."
-   exit 1
-   }
-
-   /bin/mount -o remount,ro /mnt
-   /bin/umount -l /mnt
-
-   grep /overlay /proc/mounts > /dev/null && {
-   /bin/mount -o noatime,remount,ro /overlay
-   /bin/umount -l /overlay
-   }
-
-   # spawn a new shell from ramdisk to reduce the probability of cache 
issues
-   exec /bin/busybox ash -c "$*"
-}
-
-kill_remaining() { # [  [  ] ]
-   local sig="${1:-TERM}"
-   local loop="${2:-0}"
-   local run=true
-   local stat
-
-   echo -n "Sending $sig to remaining processes ... "
-
-   while $run; do
-   run=false
-   for stat in /proc/[0-9]*/stat; do
-   [ -f "$stat" ] || continue
-
-   local pid name state ppid rest
-   read pid name state ppid rest < $stat
-   name="${name#(}"; name="${name%)}"
-
-   # Skip PID1, ourself and our children
-   [ $pid -ne 1 -a $pid -ne $$ -a $ppid -ne $$ ] || 
continue
-
-   local cmdline
-   read cmdline < /proc/$pid/cmdline
-
-   # Skip kernel threads
-   [ -n "$cmdline" ] || continue
-
-   echo -n "$name "
-   kill -$sig $pid 2>/dev/null
-
-   [ $loop -eq 1 ] && run=true
-   done
-   done
-   echo ""
-}
-
 run_hooks() {
local arg="$1"; shift
for func in "$@"; do
diff --git a/package/base-files/files/lib/upgrade/stage2 
b/package/base-files/files/lib/upgrade/stage2
index 25acfffb2d..a53f5caebe 100755
--- a/package/base-files/files/lib/upgrade/stage2
+++ b/package/base-files/files/lib/upgrade/stage2
@@ -24,16 +24,115 @@ export CONF_TAR=/tmp/sysupgrade.tgz

Re: [LEDE-DEV] early printk breaks kernel on the clearfog board

2017-04-23 Thread Syrone Wong
You may want to enable CONFIG_DEBUG_MVEBU_UART0_ALTERNATE or
CONFIG_DEBUG_MVEBU_UART1_ALTERNATE if early printk is being enabled.


Best Regards,
Syrone Wong


On Mon, Apr 24, 2017 at 3:39 AM, Josua Mayer  wrote:
> Hi everybody,
>
> I noticed a serious problem with a Clearfog build that enables
> CONFIG_KERNEL_EARLY_PRINTK=y.
> In short: after Loading Kernel, the serial console stays completely
> silent, and apparently the board does *not* boot.
>
> This came as a serious surprise to me.
> Turns out the reason for this is the usage of the old uart physical address:
> CONFIG_DEBUG_UART_PHYS=0xd0012000
> This is the default for if DEBUG_MVEBU_UART0
> However the Clearfog Pro uses mainline u-boot, and requires:
> CONFIG_DEBUG_UART_PHYS=0xf1012000
>
> Any opinions what should be done about this?
> I personally suggest to switch to what the kernel config calls "new
> bootloaders".
> After all this is quite the trap when somebody is so thoughtful to start
> with a debug build, only to find out that is the reason his builds don't
> boot.
>
> Imre Kaloz: you are the one who committed this setting with config-4.4,
> so I figured you may have some valuable insight here.
>
> br
> Josua Mayer
>
>
> ___
> Lede-dev mailing list
> Lede-dev@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/lede-dev

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] [PATCH netifd] Revert: set prefix indicator flag when IPv6 prefix lifetime, changes

2017-04-23 Thread Eric Luehrsen
This reverts commit a03216660797173fbe67866f75564e3fec9c1e8d. The previous
change causes IFUP events to occur from 30 seconds to 30 minutes depending
on various conditions at the WAN6 (typ.) interface. If a DHCPv6 lease is
24 hours and the connection is stable, then in other linux distro we would
expect IFUP every 24 hours. This is therefore a significant inconsistency
for LEDE. Add on package maintainers are going to be surprised by the
behavior. Use of the interface triggers in procd will experience further
thrashing. For example, procd managed DNS servers like dnsmasq and Unbound
will dump their DNS cache every 30sec to 30min due to reload triggers.
---
 interface-ip.c | 4 
 1 file changed, 4 deletions(-)

diff --git a/interface-ip.c b/interface-ip.c
index 140ef34..dc246f0 100644
--- a/interface-ip.c
+++ b/interface-ip.c
@@ -1047,10 +1047,6 @@ interface_update_prefix(struct vlist_tree *tree,
list_for_each_entry(c, &prefix_new->assignments, head)
if ((iface = vlist_find(&interfaces, c->name, iface, 
node)))
interface_set_prefix_address(c, prefix_new, 
iface, true);
-
-   if (prefix_new->preferred_until != prefix_old->preferred_until 
||
-   prefix_new->valid_until != 
prefix_old->valid_until)
-   ip->iface->updated |= IUF_PREFIX;
} else if (node_new) {
// Set null-route to avoid routing loops
system_add_route(NULL, &route);
-- 
2.11.0
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] [PATCH netifd] Revert: set prefix indicator flag when IPv6 prefix lifetime, changes

2017-04-23 Thread Eric Luehrsen
On 04/23/2017 10:23 PM, Eric Luehrsen wrote:
> This reverts commit a03216660797173fbe67866f75564e3fec9c1e8d. The previous
> change causes IFUP events to occur from 30 seconds to 30 minutes depending
> on various conditions at the WAN6 (typ.) interface. If a DHCPv6 lease is
> 24 hours and the connection is stable, then in other linux distro we would
> expect IFUP every 24 hours. This is therefore a significant inconsistency
> for LEDE. Add on package maintainers are going to be surprised by the
> behavior. Use of the interface triggers in procd will experience further
> thrashing. For example, procd managed DNS servers like dnsmasq and Unbound
> will dump their DNS cache every 30sec to 30min due to reload triggers.

> Original Patch discussion: 
>> Re: [LEDE-DEV] [PATCH netifd 1/2] interface-ip: set address indicator flag 
>> when IPv6 address lifetime changes
>> On Thu, Mar 9, 2017 at 10:20 PM, Matthias Schiffer
>>  wrote:
>> On 03/09/2017 05:32 PM, Hans Dedecker wrote:
>>> Trigger interface update event when IPv6 address lifetime changes by setting
>>> the address indicator flag to inform external subsystems about IPv6 address
>>> lifetime change.
>>>
>>> Signed-off-by: Hans Dedecker 
>>
>> AFAICT, this will cause a lot of ifupdate events in big IPv6 networks (for
>> example, some large mesh networks based on Gluon have more than 4000 client
>> and 5-10 radvds, often causing more than one RA per second, each updating
>> valid/preferred lifetimes). We *really* want to avoid causing lots of
>> reloads for services that set triggers on interface updates; the majority
>> of services is not interested in the lifetimes of addresses at all.
>>
>> Matthias

See Also FS#713
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


[LEDE-DEV] FW: UDP throughput caused kernel panic if configured bridge mode in /etc/config/network

2017-04-23 Thread Y.B. Lu
Hi John and Jo-Philipp,

Have you ever got similar problem, or known any possible reason about this,
or known anyone who probably know this?

I just found much memory would be consumed if I configured board as bridge mode
in /etc/config/network and did UDP throughput test.
But using brctl to configure bridge mode didn't consume memory.

Thank you very much.


Best regards,
Yangbo Lu


> -Original Message-
> From: Y.B. Lu
> Sent: Thursday, April 13, 2017 1:24 PM
> To: 'lede-dev@lists.infradead.org'
> Subject: UDP throughput caused kernel panic if configured bridge mode in
> /etc/config/network
> 
> Hi all,
> 
> Recently I got below kernel panic when did UDP throughput test on NXP
> LS1043ARDB board. I configured the bridge mode in /etc/config/network.
> But if I used 'brctl' to configure the bridge mode, this issue would not
> happen.
> I also noticed almost all memory was consumed(about 700MB) when kernel
> crashed.
> Anyone have any idea about this?  Thank you very much.
> 
> root@LEDE:/etc/fmc/config# [  263.981540] ksoftirqd/3: page allocation
> failure: order:0, mode:0x2080020 [  263.988339] CPU: 3 PID: 19 Comm:
> ksoftirqd/3 Not tainted 4.4.52 #0 [  263.994508] Hardware name: Freescale
> LS1043A [  263.998767] Backtrace:
> [  264.001213] [] (dump_backtrace) from []
> (show_stack+0x18/0x1c) [  264.008771]  r7: r6:
> r5:6013 r4: [  264.014435] [] (show_stack) from
> [] (dump_stack+0x84/0xa4) [  264.021652] []
> (dump_stack) from [] (warn_alloc_failed+0xf0/0x114)
> [  264.029558]  r5:02080020 r4: [  264.033133] []
> (warn_alloc_failed) from [] (__alloc_pages_nodemask+0x68c/0x6c0)
> [  264.042166]  r3: r2:
> [  264.045735]  r7:0030 r6:02080020 r5: r4:02080020
> [  264.051400] [] (__alloc_pages_nodemask) from []
> (_dpa_bp_add_8_bufs+0x40/0x250) [  264.060434]  r10:c001db08 r9:ebc93d48
> r8:ebf61650 r7:0003 r6:0004 r5:eb603c10 [  264.068266]
> r4:8193f040 [  264.070796] [] (_dpa_bp_add_8_bufs) from
> [] (dpaa_eth_refill_bpools+0x28/0x58)
> [  264.079742]  r10:eb09d480 r9:eb604a74 r8:ef1d9400 r7:eb09d000
> r6:ebf61650 r5:ef1e0b44 [  264.087577]  r4:007f [  264.090106]
> [] (dpaa_eth_refill_bpools) from []
> (priv_rx_default_dqrr+0xd8/0x124) [  264.099313]  r7:eb09d000 r6:ebf61650
> r5:ef1e0b44 r4:ef1e15e0
> 
> The /etc/config/network for bridge mode as below.
> config interface 'lan'
> option type 'bridge'
> option ifname 'eth2 eth6'
> option proto 'static'
> option ipaddr '192.168.1.1'
> option netmask '255.255.255.0'
> option ip6assign '60'
> 
> brctl manual config as below.
> ifconfig eth2 up
> ifconfig eth6 up
> brctl addif br-lan eth6
> brctl addif br-lan eth2
> 
> 
> 
> best regards,
> Yangbo Lu

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] FW: UDP throughput caused kernel panic if configured bridge mode in /etc/config/network

2017-04-23 Thread John Crispin

Hi,

this is most certainly a bug in the kernel. either the ethernet driver 
blows up under load or some other memory allocation related bug. it is 
very common for ethernet to kill boards under load by triggering bugs.


John

On 24/04/17 05:49, Y.B. Lu wrote:

Hi John and Jo-Philipp,

Have you ever got similar problem, or known any possible reason about this,
or known anyone who probably know this?

I just found much memory would be consumed if I configured board as bridge mode
in /etc/config/network and did UDP throughput test.
But using brctl to configure bridge mode didn't consume memory.

Thank you very much.


Best regards,
Yangbo Lu



-Original Message-
From: Y.B. Lu
Sent: Thursday, April 13, 2017 1:24 PM
To: 'lede-dev@lists.infradead.org'
Subject: UDP throughput caused kernel panic if configured bridge mode in
/etc/config/network

Hi all,

Recently I got below kernel panic when did UDP throughput test on NXP
LS1043ARDB board. I configured the bridge mode in /etc/config/network.
But if I used 'brctl' to configure the bridge mode, this issue would not
happen.
I also noticed almost all memory was consumed(about 700MB) when kernel
crashed.
Anyone have any idea about this?  Thank you very much.

root@LEDE:/etc/fmc/config# [  263.981540] ksoftirqd/3: page allocation
failure: order:0, mode:0x2080020 [  263.988339] CPU: 3 PID: 19 Comm:
ksoftirqd/3 Not tainted 4.4.52 #0 [  263.994508] Hardware name: Freescale
LS1043A [  263.998767] Backtrace:
[  264.001213] [] (dump_backtrace) from []
(show_stack+0x18/0x1c) [  264.008771]  r7: r6:
r5:6013 r4: [  264.014435] [] (show_stack) from
[] (dump_stack+0x84/0xa4) [  264.021652] []
(dump_stack) from [] (warn_alloc_failed+0xf0/0x114)
[  264.029558]  r5:02080020 r4: [  264.033133] []
(warn_alloc_failed) from [] (__alloc_pages_nodemask+0x68c/0x6c0)
[  264.042166]  r3: r2:
[  264.045735]  r7:0030 r6:02080020 r5: r4:02080020
[  264.051400] [] (__alloc_pages_nodemask) from []
(_dpa_bp_add_8_bufs+0x40/0x250) [  264.060434]  r10:c001db08 r9:ebc93d48
r8:ebf61650 r7:0003 r6:0004 r5:eb603c10 [  264.068266]
r4:8193f040 [  264.070796] [] (_dpa_bp_add_8_bufs) from
[] (dpaa_eth_refill_bpools+0x28/0x58)
[  264.079742]  r10:eb09d480 r9:eb604a74 r8:ef1d9400 r7:eb09d000
r6:ebf61650 r5:ef1e0b44 [  264.087577]  r4:007f [  264.090106]
[] (dpaa_eth_refill_bpools) from []
(priv_rx_default_dqrr+0xd8/0x124) [  264.099313]  r7:eb09d000 r6:ebf61650
r5:ef1e0b44 r4:ef1e15e0

The /etc/config/network for bridge mode as below.
config interface 'lan'
 option type 'bridge'
 option ifname 'eth2 eth6'
 option proto 'static'
 option ipaddr '192.168.1.1'
 option netmask '255.255.255.0'
 option ip6assign '60'

brctl manual config as below.
ifconfig eth2 up
ifconfig eth6 up
brctl addif br-lan eth6
brctl addif br-lan eth2



best regards,
Yangbo Lu

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev



___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev


Re: [LEDE-DEV] [PATCH netifd] Revert: set prefix indicator flag when IPv6 prefix lifetime, changes

2017-04-23 Thread Hans Dedecker
On Mon, Apr 24, 2017 at 4:24 AM, Eric Luehrsen  wrote:
> This reverts commit a03216660797173fbe67866f75564e3fec9c1e8d. The previous
> change causes IFUP events to occur from 30 seconds to 30 minutes depending
> on various conditions at the WAN6 (typ.) interface. If a DHCPv6 lease is
> 24 hours and the connection is stable, then in other linux distro we would
> expect IFUP every 24 hours. This is therefore a significant inconsistency
> for LEDE. Add on package maintainers are going to be surprised by the
> behavior. Use of the interface triggers in procd will experience further
> thrashing. For example, procd managed DNS servers like dnsmasq and Unbound
> will dump their DNS cache every 30sec to 30min due to reload triggers.
Hi

As explained in FS#713 reverting this patch will lead to complaints
from people homenet is
broken (https://github.com/openwrt/openwrt/issues/346[1]); a more
fundamentel fix is
required possible in procd to fix the issue.
Remark the commit message makes a misleading assumption as an IFUP is
fired by netifd
when interface parameters change like IP addresses, routes, etc ...
and this is dependant
on the renew time of a DHCPv4/6 lease and not the leasetime.
Therefore for netifd hotplug events extra flags were introduced in the
past allowing to
start a hotplug script upon a specific change like an IP address
change which is currently
missing in procd.

Hans
> ---
>  interface-ip.c | 4 
>  1 file changed, 4 deletions(-)
>
> diff --git a/interface-ip.c b/interface-ip.c
> index 140ef34..dc246f0 100644
> --- a/interface-ip.c
> +++ b/interface-ip.c
> @@ -1047,10 +1047,6 @@ interface_update_prefix(struct vlist_tree *tree,
> list_for_each_entry(c, &prefix_new->assignments, head)
> if ((iface = vlist_find(&interfaces, c->name, iface, 
> node)))
> interface_set_prefix_address(c, prefix_new, 
> iface, true);
> -
> -   if (prefix_new->preferred_until != 
> prefix_old->preferred_until ||
> -   prefix_new->valid_until != 
> prefix_old->valid_until)
> -   ip->iface->updated |= IUF_PREFIX;
> } else if (node_new) {
> // Set null-route to avoid routing loops
> system_add_route(NULL, &route);
> --
> 2.11.0
> ___
> Lede-dev mailing list
> Lede-dev@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/lede-dev

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev