Re: [OpenWrt-Devel] [PATCH v2] build: add mkrasimage
Hi On 2018-08-21, David Bauer wrote: > On 8/21/18 8:31 AM, Stefan Lippers-Hollmann wrote: > > While this passes the version check, the appended git hash does break > > later on in the flashing process - so at the moment only the numerical > > revision (of DISTRIB_REVISION='r7890-40eb9bda44') seems to be safe: > > > > RAS_VERSION := "V1.99(OWRT.$$(shell echo $(REVISION) | sed -e s/^r// -e > > s/\\-.*//))C0" > > Does the version string contained in the RAS-header get exposed to a > user in any way? If not, i would go for using a static (high enough so > it probably never get's used by ZyXEL) version string and call it a day > before we are over-complicating things here. [...] The OEM firmware doesn't expose any information about the other partition set, OpenWrt could make use of it via luci-app-advanced-reboot (but hasn't been doing that so far, as OpenWrt installs and sysupgrades don't update the header partition so far). But hardcoding RAS_VERSION := "V1.99(OWRT.)C0 should be fine for the forseeable future as well. Regards Stefan Lippers-Hollmann pgp5Pz6xkxtHD.pgp Description: Digitale Signatur von OpenPGP ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH v2] build: add mkrasimage
Hi, On 8/21/18 8:31 AM, Stefan Lippers-Hollmann wrote: > While this passes the version check, the appended git hash does break > later on in the flashing process - so at the moment only the numerical > revision (of DISTRIB_REVISION='r7890-40eb9bda44') seems to be safe: > > RAS_VERSION := "V1.99(OWRT.$$(shell echo $(REVISION) | sed -e s/^r// -e > s/\\-.*//))C0" Does the version string contained in the RAS-header get exposed to a user in any way? If not, i would go for using a static (high enough so it probably never get's used by ZyXEL) version string and call it a day before we are over-complicating things here. The NBG6617 btw. does not have such a validation (Downgrades using older ZyXEL-firmware are also possible using the Web-UI) Best wishes David ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH v2] build: add mkrasimage
Hi On 2018-08-21, Jo-Philipp Wich wrote: > Hi, > > > RAS_VERSION := "V1.99(OWRT.$$(shell echo $(REVISION) | sed s/^r//))C0" While this passes the version check, the appended git hash does break later on in the flashing process - so at the moment only the numerical revision (of DISTRIB_REVISION='r7890-40eb9bda44') seems to be safe: RAS_VERSION := "V1.99(OWRT.$$(shell echo $(REVISION) | sed -e s/^r// -e s/\\-.*//))C0" # hexdump -C /dev/mmcblk0p6 00 00 b4 bc 01 47 18 00 56 31 2e 39 39 28 4f 57 |.G..V1.99(OW| 0010 52 54 2e 37 38 39 30 29 43 30 00 ff ff ff ff ff |RT.7890)C0..| 0020 ff ff ff ff ff ff ff ff 00 00 d4 1d 4e 42 47 36 |NBG6| 0030 38 31 37 00 ff ff ff ff ff ff ff ff ff ff ff ff |817.| 0040 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff || * 0060 ff ff ff ff ff ff ff ff ff ff ff ff 00 00 98 e5 || 0070 00 40 00 00 ff ff ff ff ff ff ff ff ff ff ff ff |.@..| 0080 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff || * 0001 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 || * 0010 > make that RAS_VERSION := "V1.99(OWRT.$(patsubst r%,%,$(REVISION)))C0" > to avoid spawning a shell + sed process. I'll have a look at improving this tonight, thanks. Regards Stefan Lippers-Hollmann ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH v2] build: add mkrasimage
Hi, > RAS_VERSION := "V1.99(OWRT.$$(shell echo $(REVISION) | sed s/^r//))C0" make that RAS_VERSION := "V1.99(OWRT.$(patsubst r%,%,$(REVISION)))C0" to avoid spawning a shell + sed process. ~ Jo signature.asc Description: OpenPGP digital signature ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH v2] build: add mkrasimage
On 2018-08-21, Stefan Lippers-Hollmann wrote: > Hi > > On 2018-08-21, Stefan Lippers-Hollmann wrote: > > On 2018-08-20, David Bauer wrote: > [...] > > For now, I've supplied a valid/ future firmware version for testing > > > > RAS_VERSION := "V1.00(ABCS.9)C0" > > While perhaps not ideal yet, this is accepted by the OEM webinterface > and should be safe to use for the time being. > > RAS_VERSION := "V1.99(OWRT.7890)C0" Sorry, this was meant to read: RAS_VERSION := "V1.99(OWRT.$$(shell echo $(REVISION) | sed s/^r//))C0" # hexdump -C /dev/mmcblk0p3 00 00 c4 38 01 47 18 00 56 31 2e 39 39 28 4f 57 |...8.G..V1.99(OW| 0010 52 54 2e 37 38 39 30 2d 34 30 65 62 39 62 64 61 |RT.7890-40eb9bda| 0020 34 34 29 43 30 00 ff ff 00 00 cb 60 4e 42 47 36 |44)C0..`NBG6| 0030 38 31 37 00 ff ff ff ff ff ff ff ff ff ff ff ff |817.| 0040 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff || * 0060 ff ff ff ff ff ff ff ff ff ff ff ff 00 00 98 e5 || 0070 00 40 00 00 ff ff ff ff ff ff ff ff ff ff ff ff |.@..| 0080 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff || * 0001 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 || * 0010 Regards Stefan Lippers-Hollmann pgpUy1INt3ylP.pgp Description: Digitale Signatur von OpenPGP ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH v2] build: add mkrasimage
Hi On 2018-08-21, Stefan Lippers-Hollmann wrote: > On 2018-08-20, David Bauer wrote: [...] > For now, I've supplied a valid/ future firmware version for testing > > RAS_VERSION := "V1.00(ABCS.9)C0" While perhaps not ideal yet, this is accepted by the OEM webinterface and should be safe to use for the time being. RAS_VERSION := "V1.99(OWRT.7890)C0" # hexdump -C /dev/mmcblk0p6 00 00 b8 49 01 47 18 00 56 31 2e 39 39 28 4f 57 |...I.G..V1.99(OW| 0010 52 54 2e 37 38 39 30 29 43 30 00 ff ff ff ff ff |RT.7890)C0..| 0020 ff ff ff ff ff ff ff ff 00 00 d3 3f 4e 42 47 36 |...?NBG6| 0030 38 31 37 00 ff ff ff ff ff ff ff ff ff ff ff ff |817.| 0040 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff || * 0060 ff ff ff ff ff ff ff ff ff ff ff ff 00 00 98 e5 || 0070 00 40 00 00 ff ff ff ff ff ff ff ff ff ff ff ff |.@..| 0080 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff || * 0001 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 || * 0010 Regards Stefan Lippers-Hollmann ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH v2] build: add mkrasimage
Hi On 2018-08-20, David Bauer wrote: > The current make-ras.sh image generation script for the ZyXEL NBG6617 > has portability issues with bash. Because of this, factory images are > currently not built correctly by the OpenWRT buildbots. > > This commit replaces the make-ras.sh by C-written mkrasimage. The old > script is still kept but can be deleted in the future. > > The new mkrasimage is also compatible with other ZyXEL devices using > the ras image-format. This is not tested with a OpenWRT build but > it correctly builds the header for ZyXEL factory images for all devices > it is added to. > > Signed-off-by: David Bauer with the caveat explained below: Tested-by: Stefan Lippers-Hollmann > v2 changes: > - Rework image-generation code > - Add factory image for NBG6616 > - Add factory image for NBG6817 Thanks a lot for looking into this! I've given this a test on my ZyXEL NBG6817 and it's basically working, ZyXEL just decided to prevent downgrades from the firmware upgrade functionality of their webinterface, which means it parses the RAS_VERSION and rejects the currently chosen version number. Flashing this image from a tftpd avoids the strict version requirements and can flash the image as-is (press WPS key while power-on, have the factory image renamed to "ras.bin" available from a tftpd at 192.168.1.99). [...] > diff --git a/target/linux/ipq806x/image/Makefile > b/target/linux/ipq806x/image/Makefile > index 2902af3231..cbb03272fb 100644 > --- a/target/linux/ipq806x/image/Makefile > +++ b/target/linux/ipq806x/image/Makefile [...] > @@ -245,6 +246,9 @@ define Device/zyxel_nbg6817 > KERNEL_SIZE := 4096k > BLOCKSIZE := 64k > BOARD_NAME := nbg6817 > + RAS_BOARD := NBG6817 > + RAS_ROOTFS_SIZE := 20934k > + RAS_VERSION := "$(VERSION_DIST) $(REVISION)" [...] For now, I've supplied a valid/ future firmware version for testing RAS_VERSION := "V1.00(ABCS.9)C0" and it flashes nicely from the webinterface (until ZyXEL actually releases this version, which is next in line). I'll have a look if I can track down the version check and supply something that's both a bit closer to OpenWrt and more future safe, but I'd suggest to go with "V1.00(ABCS.9)C0" for now, as that has been confirmed to work for the next couple of weeks. Flashed OpenWrt master image via web-interface, with a fake ZyXEL version number: # hexdump -C /dev/mmcblk0p6 | head -n6 00 00 7a 9f 01 47 18 00 56 31 2e 30 30 28 41 42 |..z..G..V1.00(AB| 0010 43 53 2e 39 29 43 30 00 ff ff ff ff ff ff ff ff |CS.9)C0.| 0020 ff ff ff ff ff ff ff ff 00 00 d5 73 4e 42 47 36 |...sNBG6| 0030 38 31 37 00 ff ff ff ff ff ff ff ff ff ff ff ff |817.| 0040 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff || * Flashed via tftpd, using OpenWrt's preliminary version info: # hexdump -C /dev/mmcblk0p3 | head -n6 00 00 7a 9f 01 47 18 00 4f 70 65 6e 57 72 74 20 |..z..G..OpenWrt | 0010 72 37 38 38 33 2d 66 63 39 63 62 66 33 62 63 30 |r7883-fc9cbf3bc0| 0020 00 ff ff ff ff ff ff ff 00 00 d0 e0 4e 42 47 36 |NBG6| 0030 38 31 37 00 ff ff ff ff ff ff ff ff ff ff ff ff |817.| 0040 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff || * Regards, thank you very much! Stefan Lippers-Hollmann pgpF8_kF_K18G.pgp Description: Digitale Signatur von OpenPGP ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH v2] build: add mkrasimage
David Bauer wrote: > The current make-ras.sh image generation script for the ZyXEL > NBG6617 has portability issues with bash. Because of this, > factory images are currently not built correctly by the OpenWRT > buildbots. > > This commit replaces the make-ras.sh by C-written mkrasimage. > The old script is still kept but can be deleted in the future. You need to update your commit message, this is no longer the case. > > The new mkrasimage is also compatible with other ZyXEL devices > using the ras image-format. This is not tested with a OpenWRT > build but it correctly builds the header for ZyXEL factory > images for all devices it is added to. > > Signed-off-by: David Bauer > --- > > v2 changes: > - Rework image-generation code > - Add factory image for NBG6616 > - Add factory image for NBG6817 > > include/image-commands.mk | 18 +- > scripts/make-ras.sh | 196 --- > target/linux/ar71xx/image/generic.mk | 6 +- > target/linux/ipq40xx/image/Makefile | 2 +- > target/linux/ipq806x/image/Makefile | 6 +- > tools/firmware-utils/Makefile | 1 + > tools/firmware-utils/src/mkrasimage.c | 474 ++ > 7 files changed, 495 insertions(+), 208 deletions(-) > delete mode 100755 scripts/make-ras.sh > create mode 100644 tools/firmware-utils/src/mkrasimage.c > > diff --git a/include/image-commands.mk > b/include/image-commands.mk index 3cc5dc21e1..61ba49de51 100644 > --- a/include/image-commands.mk > +++ b/include/image-commands.mk > @@ -49,17 +49,17 @@ define Build/eva-image > mv $@.new $@ > endef > > -define Build/make-ras > +define Build/zyxel-ras-image > let \ > newsize="$(subst k,* 1024,$(RAS_ROOTFS_SIZE))"; \ > - $(TOPDIR)/scripts/make-ras.sh \ > - --board $(RAS_BOARD) \ > - --version $(RAS_VERSION) \ > - --kernel $(call > param_get_default,kernel,$(1),$(IMAGE_KERNEL)) \ > - --rootfs $@ \ > - --rootfssize $$newsize \ > - $@.new > - @mv $@.new $@ > + $(STAGING_DIR_HOST)/bin/mkrasimage \ > + -b $(RAS_BOARD) \ > + -v $(RAS_VERSION) \ > + -r $@ \ > + -s $$newsize \ > + -o $@.new \ > + $(if $(findstring seperate-kernel,$(word 1,$(1))),-k > $(IMAGE_KERNEL)) \ > + && mv $@.new $@ > endef > > define Build/mkbuffaloimg > diff --git a/scripts/make-ras.sh b/scripts/make-ras.sh deleted > file mode 100755 index ccddaa0016..00 > --- a/scripts/make-ras.sh > +++ /dev/null > @@ -1,196 +0,0 @@ > -#!/usr/bin/env bash > -# > -# --- ZyXEL header format --- > -# Original Version by Benjamin Berg > -# > -# The firmware image prefixed with a header (which is written into the MTD > device). > -# The header is one erase block (~64KiB) in size, but the checksum only > convers the > -# first 2KiB. Padding is 0xff. All integers are in big-endian. > -# > -# The checksum is always a 16-Bit System V checksum (sum -s) stored in a > 32-Bit integer. > -# > -# 4 bytes: checksum of the rootfs image > -# 4 bytes: length of the contained rootfs image file (big endian) > -# 32 bytes: Firmware Version string (NUL terminated, 0xff padded) > -# 4 bytes: checksum over the header partition (big endian - see below) > -# 32 bytes: Model (e.g. "NBG6617", NUL termiated, 0xff padded) > -# 4 bytes: checksum of the kernel partition > -# 4 bytes: length of the contained kernel image file (big endian) > -# rest: 0xff padding > -# > -# The checksums are calculated by adding up all bytes and if a 16bit > -# overflow occurs, one is added and the sum is masked to 16 bit: > -# csum = csum + databyte; if (csum > 0x) { csum += 1; csum &= 0x }; > -# Should the file have an odd number of bytes then the byte len-0x800 is > -# used additionally. > -# > -# The checksum for the header is calculated over the first 2048 bytes with > -# the rootfs image checksum as the placeholder during calculation. > -# > -# The header is padded with 0xff to the erase block size of the device. > -# > -board="" > -version="" > -kernel="" > -rootfs="" > -outfile="" > -err="" > - > -while [ "$1" ]; do > - case "$1" in > - "--board") > - board="$2" > - shift > - shift > - continue > - ;; > - "--version") > - version="$2" > - shift > - shift > - continue > - ;; > - "--kernel") > - kernel="$2" > - shift > - shift > - continue > - ;; > - "--rootfs") > - rootfs="$2" > - shift > - shift > - continue > - ;; > - "--rootfssize") > - rootfssize="$2" > - shift > -
[OpenWrt-Devel] [PATCH v2] build: add mkrasimage
The current make-ras.sh image generation script for the ZyXEL NBG6617 has portability issues with bash. Because of this, factory images are currently not built correctly by the OpenWRT buildbots. This commit replaces the make-ras.sh by C-written mkrasimage. The old script is still kept but can be deleted in the future. The new mkrasimage is also compatible with other ZyXEL devices using the ras image-format. This is not tested with a OpenWRT build but it correctly builds the header for ZyXEL factory images for all devices it is added to. Signed-off-by: David Bauer --- v2 changes: - Rework image-generation code - Add factory image for NBG6616 - Add factory image for NBG6817 include/image-commands.mk | 18 +- scripts/make-ras.sh | 196 --- target/linux/ar71xx/image/generic.mk | 6 +- target/linux/ipq40xx/image/Makefile | 2 +- target/linux/ipq806x/image/Makefile | 6 +- tools/firmware-utils/Makefile | 1 + tools/firmware-utils/src/mkrasimage.c | 474 ++ 7 files changed, 495 insertions(+), 208 deletions(-) delete mode 100755 scripts/make-ras.sh create mode 100644 tools/firmware-utils/src/mkrasimage.c diff --git a/include/image-commands.mk b/include/image-commands.mk index 3cc5dc21e1..61ba49de51 100644 --- a/include/image-commands.mk +++ b/include/image-commands.mk @@ -49,17 +49,17 @@ define Build/eva-image mv $@.new $@ endef -define Build/make-ras +define Build/zyxel-ras-image let \ newsize="$(subst k,* 1024,$(RAS_ROOTFS_SIZE))"; \ - $(TOPDIR)/scripts/make-ras.sh \ - --board $(RAS_BOARD) \ - --version $(RAS_VERSION) \ - --kernel $(call param_get_default,kernel,$(1),$(IMAGE_KERNEL)) \ - --rootfs $@ \ - --rootfssize $$newsize \ - $@.new - @mv $@.new $@ + $(STAGING_DIR_HOST)/bin/mkrasimage \ + -b $(RAS_BOARD) \ + -v $(RAS_VERSION) \ + -r $@ \ + -s $$newsize \ + -o $@.new \ + $(if $(findstring seperate-kernel,$(word 1,$(1))),-k $(IMAGE_KERNEL)) \ + && mv $@.new $@ endef define Build/mkbuffaloimg diff --git a/scripts/make-ras.sh b/scripts/make-ras.sh deleted file mode 100755 index ccddaa0016..00 --- a/scripts/make-ras.sh +++ /dev/null @@ -1,196 +0,0 @@ -#!/usr/bin/env bash -# -# --- ZyXEL header format --- -# Original Version by Benjamin Berg -# -# The firmware image prefixed with a header (which is written into the MTD device). -# The header is one erase block (~64KiB) in size, but the checksum only convers the -# first 2KiB. Padding is 0xff. All integers are in big-endian. -# -# The checksum is always a 16-Bit System V checksum (sum -s) stored in a 32-Bit integer. -# -# 4 bytes: checksum of the rootfs image -# 4 bytes: length of the contained rootfs image file (big endian) -# 32 bytes: Firmware Version string (NUL terminated, 0xff padded) -# 4 bytes: checksum over the header partition (big endian - see below) -# 32 bytes: Model (e.g. "NBG6617", NUL termiated, 0xff padded) -# 4 bytes: checksum of the kernel partition -# 4 bytes: length of the contained kernel image file (big endian) -# rest: 0xff padding -# -# The checksums are calculated by adding up all bytes and if a 16bit -# overflow occurs, one is added and the sum is masked to 16 bit: -# csum = csum + databyte; if (csum > 0x) { csum += 1; csum &= 0x }; -# Should the file have an odd number of bytes then the byte len-0x800 is -# used additionally. -# -# The checksum for the header is calculated over the first 2048 bytes with -# the rootfs image checksum as the placeholder during calculation. -# -# The header is padded with 0xff to the erase block size of the device. -# -board="" -version="" -kernel="" -rootfs="" -outfile="" -err="" - -while [ "$1" ]; do - case "$1" in - "--board") - board="$2" - shift - shift - continue - ;; - "--version") - version="$2" - shift - shift - continue - ;; - "--kernel") - kernel="$2" - shift - shift - continue - ;; - "--rootfs") - rootfs="$2" - shift - shift - continue - ;; - "--rootfssize") - rootfssize="$2" - shift - shift - continue - ;; - *) - if [ ! "$outfile" ]; then - outfile=$1 - shift - continue - fi - ;; - esac -done - -if [ ! -n