Hi Hervé,
I also experienced this overlay move the hard way... and fortunately
"simply" renaming the file did the trick, but it took me hours to
figure out that "simple" thing!
I clearly understand Andrei's position but IMHO the current layer's
management of DT files to build and include in sd_img is kind of
hackish (and was already before your patch)...
.dtbo files are required for rpi kernel 4.4+ which isn't the preferred
version to date. Paul Barker asked if there was any objection to move
to 4.4+ : seems no objection arised.
So maybe it's time to default to .dtbo + kernel 4.4 in a single move,
and document the fact that older kernel have to adapt KERNEL_DEVICETREE
variable.
Today I needed to build a WIC image for RPi. DT files aren't currently
included in the boot partition because IMAGE_BOOT_FILES isn't set and
that one needs to be set in (machine) conf files.
With the current RPi's management of DTs, which dynamically change
which DT files have to be included and eventually now there names, it's
impossible to do... or I would maybe have had to extend the hackish
way!
So I've done changes to support WIC populating DT files and also
touched the layer's DT files management: the patch is attached for
comments (to be applied on your v6 patchset).
I tried to simplify things and add warnings/error in case overlay
filenames doesn't suit kernel's version, but for sure the dynamic DT
management has been lost.
If you have time to review, I'd be happy to receive your comments!
Thanks,
François
Le lundi 25 juillet 2016 à 21:30 -0400, Herve Jourdain a écrit :
> Hi François,
>
> > Actually, I tried to use a -overlay.dtb as a device tree overlay with
the 4.4.9, and it didn't load it...
> > > > If you don't use the tools dedicated to debug device trees (like
vcdbg), there is no error message nor complaint, but the device tree
is not loaded. Only when I used vcdbg did I see it complained that it
could not find the .dtbo file.
> > > > I'm not sure if the loader has changed since then, but if it has not
then you probably need to impose .dtbo for kernels > 4.4.6. Or issue
an error if the KERNEL_DEVICETREE variable is not consistent with the
kernel version...
>
> > > The current behaviour is following some remark/proposal from Andrei -
the very first proposal had different versions of KERNEL_DEVICETREE,
depending on the kernel version actually.
> > But I agree with Andrei that it would be easier to just maintain one
set of default values for the KERNEL_DEVICETREE.
> > > One thing I didn't consider is the override of that variable, maybe a
warning message could be added if we have to change the -overlay.dtb
into .dtbo.
> > > But we should probably do the change to 'dtbo regardless, because
it's very likely it would not load properly later, with no message
except with vcdbg...
>
> Would that solution be OK with you?
>
> Andrei, do you have any comment on this?
>
> Herve
>
> > > Le 25 juil. 2016 à 10:45, Francois Muller <forlor...@gmail.com> a
écrit :
>
> > Hi Hervé,
> > > > Thank you for giving us up and running .dtbo overlays functionality
through this patch, it is running fine for me.
> >
> > > > Nevertheless, I have a concern about the processing applied to
overlay filenames.
> >
> > > > > > > > > > According to this post
> > > > > > > > > > https://www.raspberrypi.org/forums/viewtopic
.php?f=107&t=139732, '-overlay.dtb' suffixes have been superseded
by '.dtbo' suffix to reflect an improvement in overlay
functionalities and thus overlay binary format (or simply content,
I don't know).
> > > > Moreover, changing the suffix gives a chance for older and newer
overlay formats to coexist in a boot folder.
> >
> > > > > > > > So, I'm not very fond of a build system changing the inputs I
> > > > > > > > give
to it (i.e. KERNEL_DEVICETREE) depending on another input from my
side (kernel version) without at least notifying me the change
occurred.
> > > > > > > > > > I would highly prefer a bbwarn (or should it even fail?) if
> > > > > > > > > > the
requested overlay filenames don't suit the kernel version being
chosen (btw device trees are theoretically aimed to be kernel
version and even OS independent as it should only contain pure
hardware platform description).
> > > > > > I understand and appreciate the intention of offering a kind of
compatibility for kernels prior to 4.4.6 but I don't think it's
needed here.
> >
> > > > > > If KERNEL_DEVICETREE isn't empty for kernels prior to 3.18 => let's
disable device trees (or simply remove this test, this layer
doesn't provide any)
> > > > If KERNEL_DEVICETREE asks for -overlay.dtb and kernel >= 3.18 =>
let's compile it, it has been requested
> >
> > > > > > If KERNEL_DEVICETREE asks for .dtbo and kernel < 4.4.6 => let's
fail with a comprehensive message (these kernels even don't know
what a .dtbo file is!)
> > > > If KERNEL_DEVICETREE asks for -overlay.dtb and/or .dtbo and kernel
>= 4.4.6 => let's compile it, it has been requested
> >
> > > > The tough thing will be to keep consistency between
KERNEL_DEVICETREE content and kernel version to build.
> > > > > > > > So as long as 4.1 is the layer's kernel preferred version,
KERNEL_DEVICETREE should only contain '-overlay.dtb' overlays,
along with the next to be used KERNEL_DEVICETREE variable
(containing .dtbo) that is commented out.
> > > > > > As soon as 4.4 is chosen, let's default to .dtbo overlays and
comment out KERNEL_DEVICETREE variable containing '-overlay.dtb'
overlays.
> >
> > What do you think about this behavior ?
> >
> > > > > > > > As side notes, you may use some existing utility function to
compare software versions like bb.utils.vercmp_string_op and
overlays filtering may be done in a single pass : I've attached a
proposal.
> >
> > BR,
> > François
> >
> > Le jeudi 21 juillet 2016 à 06:00 +0800, Herve Jourdain a écrit :
> > > > > > Kernel 4.4.6+ on RaspberryPi support .dtbo files for overlays,
instead of .dtb.
> > > > > > Add support for both variants of overlays ("-overlay.dtb" and
".dtbo")
> > > > > > Change which variant needs to be supported based on the kernel
version
> > >
> > > > > > > > > CAUTION: when called from IMAGE_CMD_rpi-sdimg, 'TMPDIR' is not
set, causing 'STAGING_KERNEL_BUILDDIR' to not be expanded,
causing get_kernelversion_file() to fail!
> > > > > > > > > To avoid this problem, get_dts() and split_overlays() MUST be
called with the kernel version parameter set, when called from
IMAGE_CMD_rpi-sdimg!
> > >
> > > > > > Signed-off-by: Herve Jourdain <herve.jourd...@neuf.fr>
> > > ---
> > > classes/linux-raspberrypi-base.bbclass | 19 ++++++++++++-------
> > > 1 file changed, 12 insertions(+), 7 deletions(-)
> > >
> > > > > > diff --git a/classes/linux-raspberrypi-base.bbclass
b/classes/linux-raspberrypi-base.bbclass
> > > index 40beef1..930fc44 100644
> > > --- a/classes/linux-raspberrypi-base.bbclass
> > > +++ b/classes/linux-raspberrypi-base.bbclass
> > > @@ -1,7 +1,8 @@
> > > inherit linux-kernel-base
> > >
> > > -
> > > def get_dts(d, ver):
> > > + import re
> > > +
> > > staging_dir = d.getVar("STAGING_KERNEL_BUILDDIR", True)
> > > dts = d.getVar("KERNEL_DEVICETREE", True)
> > >
> > > @@ -20,20 +21,24 @@ def get_dts(d, ver):
> > >
> > > # Always turn off device tree support for kernel's < 3.18
> > > try:
> > > - if int(min_ver[0]) <= 3:
> > > - if int(min_ver[1]) < 18:
> > > - dts = ""
> > > + if int(min_ver[0]) >= 4:
> > > > > > + if (int(min_ver[1]) < 4) or (int(min_ver[1]) == 4
and int(min_ver[2]) < 6):
> > > > > > + dts = ' '.join([(re.sub(r'(.*)\.dtbo$', r'\1-
overlay.dtb', x)) for x in dts.split()])
> > > + elif int(min_ver[1]) < 18:
> > > + dts = ""
> > > except IndexError:
> > > min_ver = None
> > >
> > > return dts
> > >
> > >
> > > -def split_overlays(d, out):
> > > - dts = get_dts(d, None)
> > > +def split_overlays(d, ver, out):
> > > + dts = get_dts(d, ver)
> > > if out:
> > > > > > overlays = oe.utils.str_filter_out('\S+\-overlay\.dtb$',
dts, d)
> > > > > > + overlays = oe.utils.str_filter_out('\S+\.dtbo$',
overlays, d)
> > > else:
> > > > > > - overlays = oe.utils.str_filter('\S+\-overlay\.dtb$',
dts, d)
> > > > > > + overlays = oe.utils.str_filter('\S+\-overlay\.dtb$',
dts, d) + \
> > > > > > + " " + oe.utils.str_filter('\S+\.dtbo$', dts,
d)
> > >
> > > return overlays
From b07c22faead4dea1f5eb90e5e31ebca6b65195c4 Mon Sep 17 00:00:00 2001
From: Francois Muller <forlor...@gmail.com>
Date: Tue, 26 Jul 2016 20:35:37 +0200
Subject: [PATCH 1/1] Changed DT root/overlays management and populate WIC
images with these DT.
---
classes/linux-raspberrypi-base.bbclass | 44 ------------------------------
classes/sdcard_image-rpi.bbclass | 40 ++++++++++++---------------
conf/machine/include/rpi-base.inc | 25 +++++++++++++++++
recipes-kernel/linux/linux-raspberrypi.inc | 26 +++++++++++++-----
4 files changed, 61 insertions(+), 74 deletions(-)
delete mode 100644 classes/linux-raspberrypi-base.bbclass
diff --git a/classes/linux-raspberrypi-base.bbclass b/classes/linux-raspberrypi-base.bbclass
deleted file mode 100644
index 930fc44..0000000
--- a/classes/linux-raspberrypi-base.bbclass
+++ /dev/null
@@ -1,44 +0,0 @@
-inherit linux-kernel-base
-
-def get_dts(d, ver):
- import re
-
- staging_dir = d.getVar("STAGING_KERNEL_BUILDDIR", True)
- dts = d.getVar("KERNEL_DEVICETREE", True)
-
- # d.getVar() might return 'None' as a normal string
- # leading to 'is None' check isn't enough.
- # TODO: Investigate if this is a bug in bitbake
- if ver is None or ver == "None":
- ''' if 'ver' isn't set try to grab the kernel version
- from the kernel staging '''
- ver = get_kernelversion_file(staging_dir)
-
- if ver is not None:
- min_ver = ver.split('.', 3)
- else:
- return dts
-
- # Always turn off device tree support for kernel's < 3.18
- try:
- if int(min_ver[0]) >= 4:
- if (int(min_ver[1]) < 4) or (int(min_ver[1]) == 4 and int(min_ver[2]) < 6):
- dts = ' '.join([(re.sub(r'(.*)\.dtbo$', r'\1-overlay.dtb', x)) for x in dts.split()])
- elif int(min_ver[1]) < 18:
- dts = ""
- except IndexError:
- min_ver = None
-
- return dts
-
-
-def split_overlays(d, ver, out):
- dts = get_dts(d, ver)
- if out:
- overlays = oe.utils.str_filter_out('\S+\-overlay\.dtb$', dts, d)
- overlays = oe.utils.str_filter_out('\S+\.dtbo$', overlays, d)
- else:
- overlays = oe.utils.str_filter('\S+\-overlay\.dtb$', dts, d) + \
- " " + oe.utils.str_filter('\S+\.dtbo$', dts, d)
-
- return overlays
diff --git a/classes/sdcard_image-rpi.bbclass b/classes/sdcard_image-rpi.bbclass
index a04204a..55faf74 100644
--- a/classes/sdcard_image-rpi.bbclass
+++ b/classes/sdcard_image-rpi.bbclass
@@ -1,5 +1,4 @@
inherit image_types
-inherit linux-raspberrypi-base
#
# Create an image that can by written onto a SD card using dd.
@@ -75,8 +74,6 @@ IMAGEDATESTAMP = "${@time.strftime('%Y.%m.%d',time.gmtime())}"
IMAGE_CMD_rpi-sdimg[vardepsexclude] += "IMAGEDATESTAMP"
IMAGE_CMD_rpi-sdimg[vardepsexclude] += "DATETIME"
-RPI_KERNEL_VERSION := "${@get_kernelversion_file('${STAGING_KERNEL_BUILDDIR}')}"
-
IMAGE_CMD_rpi-sdimg () {
# Align partitions
@@ -86,9 +83,6 @@ IMAGE_CMD_rpi-sdimg () {
echo "Creating filesystem with Boot partition ${BOOT_SPACE_ALIGNED} KiB and RootFS $ROOTFS_SIZE KiB"
- # Check if we are building with device tree support
- DTS="${@get_dts(d, '${RPI_KERNEL_VERSION}')}"
-
# Initialize sdcard image file
dd if=/dev/zero of=${SDIMG} bs=1024 count=0 seek=${SDIMG_SIZE}
@@ -106,27 +100,27 @@ IMAGE_CMD_rpi-sdimg () {
rm -f ${WORKDIR}/boot.img
mkfs.vfat -n "${BOOTDD_VOLUME_ID}" -S 512 -C ${WORKDIR}/boot.img $BOOT_BLOCKS
mcopy -i ${WORKDIR}/boot.img -s ${DEPLOY_DIR_IMAGE}/bcm2835-bootfiles/* ::/
- if test -n "${DTS}"; then
- # Device Tree Overlays are assumed to be suffixed by '-overlay.dtb' (4.1.x) or by '.dtbo' (4.4.9+) string and will be put in a dedicated folder
- DT_OVERLAYS="${@split_overlays(d, '${RPI_KERNEL_VERSION}', 0)}"
- DT_ROOT="${@split_overlays(d, '${RPI_KERNEL_VERSION}', 1)}"
- # Copy board device trees to root folder
- for DTB in ${DT_ROOT}; do
- DTB_BASE_NAME=`basename ${DTB} .dtb`
+ # Device Tree Overlays are assumed to be suffixed by '-overlay.dtb' (4.1.x) or by '.dtbo' (4.4.9+) string and will be put in a dedicated folder
+ DT_OVERLAYS="${@" ".join(d.getVarFlag('KERNEL_DEVICETREE', 'dtoverlays', True))}"
+ DT_ROOT="${@" ".join(d.getVarFlag('KERNEL_DEVICETREE', 'dtroots', True))}"
- mcopy -i ${WORKDIR}/boot.img -s ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}-${DTB_BASE_NAME}.dtb ::${DTB_BASE_NAME}.dtb
- done
+ # Copy board device trees to root folder
+ for DTB in ${DT_ROOT}; do
+ DTB_BASE_NAME=`basename ${DTB} .dtb`
- # Copy device tree overlays to dedicated folder
- mmd -i ${WORKDIR}/boot.img overlays
- for DTB in ${DT_OVERLAYS}; do
- DTB_EXT=${DTB##*.}
- DTB_BASE_NAME=`basename ${DTB} ."${DTB_EXT}"`
+ mcopy -i ${WORKDIR}/boot.img -s ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}-${DTB_BASE_NAME}.dtb ::${DTB_BASE_NAME}.dtb
+ done
+
+ # Copy device tree overlays to dedicated folder
+ mmd -i ${WORKDIR}/boot.img overlays
+ for DTB in ${DT_OVERLAYS}; do
+ DTB_EXT=${DTB##*.}
+ DTB_BASE_NAME=`basename ${DTB} ."${DTB_EXT}"`
+
+ mcopy -i ${WORKDIR}/boot.img -s ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}-${DTB_BASE_NAME}.${DTB_EXT} ::overlays/${DTB_BASE_NAME}.${DTB_EXT}
+ done
- mcopy -i ${WORKDIR}/boot.img -s ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}-${DTB_BASE_NAME}.${DTB_EXT} ::overlays/${DTB_BASE_NAME}.${DTB_EXT}
- done
- fi
case "${KERNEL_IMAGETYPE}" in
"uImage")
mcopy -i ${WORKDIR}/boot.img -s ${DEPLOY_DIR_IMAGE}/u-boot.bin ::${SDIMG_KERNELIMAGE}
diff --git a/conf/machine/include/rpi-base.inc b/conf/machine/include/rpi-base.inc
index 47eb23b..8a0d14a 100644
--- a/conf/machine/include/rpi-base.inc
+++ b/conf/machine/include/rpi-base.inc
@@ -51,3 +51,28 @@ MACHINE_EXTRA_RRECOMMENDS += " kernel-modules"
SPLASH = "psplash-raspberrypi"
IMAGE_BOOT_FILES ?= "bcm2835-bootfiles/* ${KERNEL_IMAGETYPE};${SDIMG_KERNELIMAGE}"
+
+python () {
+ # Process KERNEL_DEVICETREE variable to seperate normal device trees from overlays and
+ # prepare the IMAGE_BOOT_FILES variable required for WIC to populate boot partition with device trees.
+
+ kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
+ kerneldt = d.getVar("KERNEL_DEVICETREE", True)
+
+ boot_files = []
+ dtovrl = []
+ dtroot = []
+
+ for dt in kerneldt.split() :
+ # Device Tree Overlays are assumed to be suffixed by '-overlay.dtb' (4.1.x) or by '.dtbo' (4.4.9+)
+ if ( dt.endswith('-overlay.dtb') or dt.endswith('.dtbo') ) :
+ boot_files.append("{0}-{1};{2}".format(kerneltype, os.path.basename(dt), dt) )
+ dtovrl.append(dt)
+ else :
+ boot_files.append("{0}-{1};{2}".format(kerneltype, os.path.basename(dt), dt) )
+ dtroot.append(dt)
+
+ d.setVarFlag("KERNEL_DEVICETREE", "dtoverlays", dtovrl)
+ d.setVarFlag("KERNEL_DEVICETREE", "dtroots", dtroot)
+ d.appendVar("IMAGE_BOOT_FILES", ' '+' '.join(boot_files))
+}
diff --git a/recipes-kernel/linux/linux-raspberrypi.inc b/recipes-kernel/linux/linux-raspberrypi.inc
index 6184402..ff12cd4 100644
--- a/recipes-kernel/linux/linux-raspberrypi.inc
+++ b/recipes-kernel/linux/linux-raspberrypi.inc
@@ -1,5 +1,4 @@
require linux-rpi.inc
-inherit linux-raspberrypi-base
DESCRIPTION = "Linux Kernel for Raspberry Pi"
SECTION = "kernel"
@@ -29,13 +28,26 @@ UDEV_GE_141 ?= "1"
KERNEL_MODULE_AUTOLOAD += "${@bb.utils.contains("MACHINE_FEATURES", "pitft28r", "stmpe-ts", "", d)}"
-# Set programmatically some variables during recipe parsing
-# See http://www.yoctoproject.org/docs/current/bitbake-user-manual/bitbake-user-manual.html#anonymous-python-functions
-python __anonymous () {
- kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
- kerneldt = get_dts(d, d.getVar('LINUX_VERSION', True))
- d.setVar("KERNEL_DEVICETREE", kerneldt)
+DISABLE_OVERLAYDTB_WARNING ?= "0"
+
+python check_devicetree_overlays() {
+ linux_version = d.getVar('LINUX_VERSION', True)
+
+ if bb.utils.vercmp_string_op(linux_version, '3.18', '<'):
+ if any( d.getVar("KERNEL_DEVICETREE",True).split() ):
+ bb.fatal("RPi kernels prior to 3.18 can't handle devicetrees. Please empty out the KERNEL_DEVICETREE variable in your configuration.")
+
+ elif bb.utils.vercmp_string_op(linux_version, '4.4.6', '<'):
+ if any( [dtb_ovrl.endswith('.dtbo') for dtb_ovrl in d.getVarFlag("KERNEL_DEVICETREE","dtoverlays",True)]):
+ bb.error("RPi kernels prior to 4.4.6 can't handle devicetree overlays with '*.dtbo' extension. Please remove any '*.dtbo' filename listed in KERNEL_DEVICETREE variable in your configuration or rename each with '*-overlay.dtb' suffix should the overlay be generated for the target.")
+
+ else:
+ if d.getVar("DISABLE_OVERLAYDTB_WARNING", True) == "0":
+ if any( [dtb_ovrl.endswith('-overlay.dtb') for dtb_ovrl in d.getVarFlag("KERNEL_DEVICETREE","dtoverlays",True)]):
+ bb.warn("RPi bootloader and kernels starting from 4.4.6 have disabled usage of '*-overlay.dtb' overlays in benefit to '*.dtbo' suffix. Please update the KERNEL_DEVICETREE variable in your configuration to reflect this change as '*-overlay.dtb' files won't be loaded by the target any more!")
}
+do_configure[prefuncs] += "check_devicetree_overlays"
+
do_kernel_configme_prepend() {
install -m 0644 ${S}/arch/${ARCH}/configs/${KERNEL_DEFCONFIG} ${WORKDIR}/defconfig || die "No default configuration for ${MACHINE} / ${KERNEL_DEFCONFIG} available."
--
2.8.1
--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
