[PATCH v6 1/4] of: change overlay apply input data from unflattened to FDT

[frowand . list]

From: Frank Rowand 

Move duplicating and unflattening of an overlay flattened devicetree
(FDT) into the overlay application code.  To accomplish this,
of_overlay_apply() is replaced by of_overlay_fdt_apply().

The copy of the FDT (aka "duplicate FDT") now belongs to devicetree
code, which is thus responsible for freeing the duplicate FDT.  The
caller of of_overlay_fdt_apply() remains responsible for freeing the
original FDT.

The unflattened devicetree now belongs to devicetree code, which is
thus responsible for freeing the unflattened devicetree.

These ownership changes prevent early freeing of the duplicated FDT
or the unflattened devicetree, which could result in use after free
errors.

of_overlay_fdt_apply() is a private function for the anticipated
overlay loader.

Update unittest.c to use of_overlay_fdt_apply() instead of
of_overlay_apply().

Move overlay fragments from artificial locations in
drivers/of/unittest-data/tests-overlay.dtsi into one devicetree
source file per overlay.  This led to changes in
drivers/of/unitest-data/Makefile and drivers/of/unitest.c.

  - Add overlay directives to the overlay devicetree source files so
that dtc will compile them as true overlays into one FDT data
chunk per overlay.

  - Set CFLAGS for drivers/of/unittest-data/testcases.dts so that
symbols will be generated for overlay resolution of overlays
that are no longer artificially contained in testcases.dts

  - Unflatten and apply each unittest overlay FDT using
of_overlay_fdt_apply().

  - Enable the of_resolve_phandles() check for whether the unflattened
overlay is detached.  This check was previously disabled because the
overlays from tests-overlay.dtsi were not unflattened into detached
trees.

  - Other changes to unittest.c infrastructure to manage multiple test
FDTs built into the kernel image (access by name instead of
arbitrary number).

  - of_unittest_overlay_high_level(): previously unused code to add
properties from the overlay_base devicetree to the live tree
was triggered by the restructuring of tests-overlay.dtsi and thus
testcases.dts.  This exposed two bugs: (1) the need to dup a
property before adding it, and (2) property 'name' is
auto-generated in the unflatten code and thus will be a duplicate
in the __symbols__ node - do not treat this duplicate as an error.

Signed-off-by: Frank Rowand 
---

There are checkpatch warnings.  I have reviewed them and feel they
can be ignored.  They are "line over 80 characters" for either
pre-existing long lines, or lines in devicetree source files.

Rob, I have added making unittest a loadable module to my todo list.

Changes from v5:
  - Add __init to of_unittest_apply_revert_overlay_check(),
of_unittest_overlay_5(), and of_unittest_overlay_11() to fix
compile warnings reported by Geert

Changes from v4:
  - move of_overlay_fdt_apply() prototype from of_private.h to of.h
  - of_overlay_apply(): add kfree(fdt) if resolve phandles or
initialize changeset fail
  - of_overlay_fdt_apply(): add parameter size of overlay_fdt and use
it for some additional validation of the overlay fdt
  - update unittest.c with additional parameter to of_overlay_fdt_apply()

Changes from v3:
  - OF_OVERLAY: add select OF_FLATTREE

Changes from v1:
  - rebase on v4.16-rc1

 drivers/of/Kconfig  |   1 +
 drivers/of/overlay.c| 112 ++-
 drivers/of/resolver.c   |   6 -
 drivers/of/unittest-data/Makefile   |  28 ++-
 drivers/of/unittest-data/overlay_0.dts  |  14 ++
 drivers/of/unittest-data/overlay_1.dts  |  14 ++
 drivers/of/unittest-data/overlay_10.dts |  34 
 drivers/of/unittest-data/overlay_11.dts |  34 
 drivers/of/unittest-data/overlay_12.dts |  14 ++
 drivers/of/unittest-data/overlay_13.dts |  14 ++
 drivers/of/unittest-data/overlay_15.dts |  35 
 drivers/of/unittest-data/overlay_2.dts  |  14 ++
 drivers/of/unittest-data/overlay_3.dts  |  14 ++
 drivers/of/unittest-data/overlay_4.dts  |  23 +++
 drivers/of/unittest-data/overlay_5.dts  |  14 ++
 drivers/of/unittest-data/overlay_6.dts  |  15 ++
 drivers/of/unittest-data/overlay_7.dts  |  15 ++
 drivers/of/unittest-data/overlay_8.dts  |  15 ++
 drivers/of/unittest-data/overlay_9.dts  |  15 ++
 drivers/of/unittest-data/tests-overlay.dtsi | 213 
 drivers/of/unittest.c   | 300 ++--
 include/linux/of.h  |   6 +-
 22 files changed, 562 insertions(+), 388 deletions(-)
 create mode 100644 drivers/of/unittest-data/overlay_0.dts
 create mode 100644 drivers/of/unittest-data/overlay_1.dts
 create mode 100644 drivers/of/unittest-data/overlay_10.dts
 create mode 100644 drivers/of/unittest-data/overlay_11.dts
 create mode 100644 drivers/of/unittest-data/overlay_12.dts
 create mode 

[PATCH v6 1/4] of: change overlay apply input data from unflattened to FDT

[frowand . list]

From: Frank Rowand 

Move duplicating and unflattening of an overlay flattened devicetree
(FDT) into the overlay application code.  To accomplish this,
of_overlay_apply() is replaced by of_overlay_fdt_apply().

The copy of the FDT (aka "duplicate FDT") now belongs to devicetree
code, which is thus responsible for freeing the duplicate FDT.  The
caller of of_overlay_fdt_apply() remains responsible for freeing the
original FDT.

The unflattened devicetree now belongs to devicetree code, which is
thus responsible for freeing the unflattened devicetree.

These ownership changes prevent early freeing of the duplicated FDT
or the unflattened devicetree, which could result in use after free
errors.

of_overlay_fdt_apply() is a private function for the anticipated
overlay loader.

Update unittest.c to use of_overlay_fdt_apply() instead of
of_overlay_apply().

Move overlay fragments from artificial locations in
drivers/of/unittest-data/tests-overlay.dtsi into one devicetree
source file per overlay.  This led to changes in
drivers/of/unitest-data/Makefile and drivers/of/unitest.c.

  - Add overlay directives to the overlay devicetree source files so
that dtc will compile them as true overlays into one FDT data
chunk per overlay.

  - Set CFLAGS for drivers/of/unittest-data/testcases.dts so that
symbols will be generated for overlay resolution of overlays
that are no longer artificially contained in testcases.dts

  - Unflatten and apply each unittest overlay FDT using
of_overlay_fdt_apply().

  - Enable the of_resolve_phandles() check for whether the unflattened
overlay is detached.  This check was previously disabled because the
overlays from tests-overlay.dtsi were not unflattened into detached
trees.

  - Other changes to unittest.c infrastructure to manage multiple test
FDTs built into the kernel image (access by name instead of
arbitrary number).

  - of_unittest_overlay_high_level(): previously unused code to add
properties from the overlay_base devicetree to the live tree
was triggered by the restructuring of tests-overlay.dtsi and thus
testcases.dts.  This exposed two bugs: (1) the need to dup a
property before adding it, and (2) property 'name' is
auto-generated in the unflatten code and thus will be a duplicate
in the __symbols__ node - do not treat this duplicate as an error.

Signed-off-by: Frank Rowand 
---

There are checkpatch warnings.  I have reviewed them and feel they
can be ignored.  They are "line over 80 characters" for either
pre-existing long lines, or lines in devicetree source files.

Rob, I have added making unittest a loadable module to my todo list.

Changes from v5:
  - Add __init to of_unittest_apply_revert_overlay_check(),
of_unittest_overlay_5(), and of_unittest_overlay_11() to fix
compile warnings reported by Geert

Changes from v4:
  - move of_overlay_fdt_apply() prototype from of_private.h to of.h
  - of_overlay_apply(): add kfree(fdt) if resolve phandles or
initialize changeset fail
  - of_overlay_fdt_apply(): add parameter size of overlay_fdt and use
it for some additional validation of the overlay fdt
  - update unittest.c with additional parameter to of_overlay_fdt_apply()

Changes from v3:
  - OF_OVERLAY: add select OF_FLATTREE

Changes from v1:
  - rebase on v4.16-rc1

 drivers/of/Kconfig  |   1 +
 drivers/of/overlay.c| 112 ++-
 drivers/of/resolver.c   |   6 -
 drivers/of/unittest-data/Makefile   |  28 ++-
 drivers/of/unittest-data/overlay_0.dts  |  14 ++
 drivers/of/unittest-data/overlay_1.dts  |  14 ++
 drivers/of/unittest-data/overlay_10.dts |  34 
 drivers/of/unittest-data/overlay_11.dts |  34 
 drivers/of/unittest-data/overlay_12.dts |  14 ++
 drivers/of/unittest-data/overlay_13.dts |  14 ++
 drivers/of/unittest-data/overlay_15.dts |  35 
 drivers/of/unittest-data/overlay_2.dts  |  14 ++
 drivers/of/unittest-data/overlay_3.dts  |  14 ++
 drivers/of/unittest-data/overlay_4.dts  |  23 +++
 drivers/of/unittest-data/overlay_5.dts  |  14 ++
 drivers/of/unittest-data/overlay_6.dts  |  15 ++
 drivers/of/unittest-data/overlay_7.dts  |  15 ++
 drivers/of/unittest-data/overlay_8.dts  |  15 ++
 drivers/of/unittest-data/overlay_9.dts  |  15 ++
 drivers/of/unittest-data/tests-overlay.dtsi | 213 
 drivers/of/unittest.c   | 300 ++--
 include/linux/of.h  |   6 +-
 22 files changed, 562 insertions(+), 388 deletions(-)
 create mode 100644 drivers/of/unittest-data/overlay_0.dts
 create mode 100644 drivers/of/unittest-data/overlay_1.dts
 create mode 100644 drivers/of/unittest-data/overlay_10.dts
 create mode 100644 drivers/of/unittest-data/overlay_11.dts
 create mode 100644 drivers/of/unittest-data/overlay_12.dts
 create mode 100644