Re: [U-Boot] [PATCH v2 15/31] binman: Allow entries to expand after packing

2019-07-17 Thread sjg
Add support for detecting entries that change size after they have already
been packed, and re-running packing when it happens.

This removes the limitation that entry size cannot change after
PackEntries() is called.

Signed-off-by: Simon Glass 
---

Changes in v2:
- Avoid dual assignment in ResetForPack()
- Show a message when a section's size changes
- Use 3 packing passes instead of two, and add a comment
- Use bytearray for Python 3

 tools/binman/README   |  3 +-
 tools/binman/bsection.py  | 12 +
 tools/binman/control.py   | 51 +--
 tools/binman/entry.py | 21 +++-
 tools/binman/etype/_testing.py| 11 +++-
 tools/binman/etype/section.py |  5 ++
 tools/binman/etype/u_boot_with_ucode_ptr.py   |  2 +-
 tools/binman/ftest.py | 33 ++--
 tools/binman/image.py |  8 +++
 tools/binman/test/121_entry_expand.dts| 20 
 tools/binman/test/122_entry_expand_twice.dts  | 21 
 .../binman/test/123_entry_expand_section.dts  | 22 
 12 files changed, 184 insertions(+), 25 deletions(-)
 create mode 100644 tools/binman/test/121_entry_expand.dts
 create mode 100644 tools/binman/test/122_entry_expand_twice.dts
 create mode 100644 tools/binman/test/123_entry_expand_section.dts

Applied to u-boot-dm, thanks!
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot


[U-Boot] [PATCH v2 15/31] binman: Allow entries to expand after packing

2019-07-08 Thread Simon Glass
Add support for detecting entries that change size after they have already
been packed, and re-running packing when it happens.

This removes the limitation that entry size cannot change after
PackEntries() is called.

Signed-off-by: Simon Glass 
---

Changes in v2:
- Avoid dual assignment in ResetForPack()
- Show a message when a section's size changes
- Use 3 packing passes instead of two, and add a comment
- Use bytearray for Python 3

 tools/binman/README   |  3 +-
 tools/binman/bsection.py  | 12 +
 tools/binman/control.py   | 51 +--
 tools/binman/entry.py | 21 +++-
 tools/binman/etype/_testing.py| 11 +++-
 tools/binman/etype/section.py |  5 ++
 tools/binman/etype/u_boot_with_ucode_ptr.py   |  2 +-
 tools/binman/ftest.py | 33 ++--
 tools/binman/image.py |  8 +++
 tools/binman/test/121_entry_expand.dts| 20 
 tools/binman/test/122_entry_expand_twice.dts  | 21 
 .../binman/test/123_entry_expand_section.dts  | 22 
 12 files changed, 184 insertions(+), 25 deletions(-)
 create mode 100644 tools/binman/test/121_entry_expand.dts
 create mode 100644 tools/binman/test/122_entry_expand_twice.dts
 create mode 100644 tools/binman/test/123_entry_expand_section.dts

diff --git a/tools/binman/README b/tools/binman/README
index abbf809b823..9860633792f 100644
--- a/tools/binman/README
+++ b/tools/binman/README
@@ -566,7 +566,8 @@ tree. This sets the correct 'offset' and 'size' vaues, for 
example.
 The default implementatoin does nothing. This can be overriden to adjust the
 contents of an entry in some way. For example, it would be possible to create
 an entry containing a hash of the contents of some other entries. At this
-stage the offset and size of entries should not be adjusted.
+stage the offset and size of entries should not be adjusted unless absolutely
+necessary, since it requires a repack (going back to PackEntries()).
 
 10. WriteSymbols() - write the value of symbols into the U-Boot SPL binary.
 See 'Access to binman entry offsets at run time' below for a description of
diff --git a/tools/binman/bsection.py b/tools/binman/bsection.py
index f49a6e93bc7..9047e55a34a 100644
--- a/tools/binman/bsection.py
+++ b/tools/binman/bsection.py
@@ -45,6 +45,8 @@ class Section(object):
 _name_prefix: Prefix to add to the name of all entries within this
 section
 _entries: OrderedDict() of entries
+_orig_offset: Original offset value read from node
+_orig_size: Original size value read from node
 """
 def __init__(self, name, parent_section, node, image, test=False):
 global entry
@@ -76,6 +78,8 @@ class Section(object):
 """Read properties from the section node"""
 self._offset = fdt_util.GetInt(self._node, 'offset')
 self._size = fdt_util.GetInt(self._node, 'size')
+self._orig_offset = self._offset
+self._orig_size = self._size
 self._align_size = fdt_util.GetInt(self._node, 'align-size')
 if tools.NotPowerOfTwo(self._align_size):
 self._Raise("Alignment size %s must be a power of two" %
@@ -257,6 +261,13 @@ class Section(object):
 for name, info in offset_dict.items():
 self._SetEntryOffsetSize(name, *info)
 
+def ResetForPack(self):
+"""Reset offset/size fields so that packing can be done again"""
+self._offset = self._orig_offset
+self._size = self._orig_size
+for entry in self._entries.values():
+entry.ResetForPack()
+
 def PackEntries(self):
 """Pack all entries into the section"""
 offset = self._skip_at_start
@@ -325,6 +336,7 @@ class Section(object):
 for entry in self._entries.values():
 if not entry.ProcessContents():
 sizes_ok = False
+print("Entry '%s' size change" % self._node.path)
 return sizes_ok
 
 def WriteSymbols(self):
diff --git a/tools/binman/control.py b/tools/binman/control.py
index 9022cf76e99..35faf115062 100644
--- a/tools/binman/control.py
+++ b/tools/binman/control.py
@@ -170,21 +170,42 @@ def Binman(args):
 # completed and written, but that does not seem important.
 image.GetEntryContents()
 image.GetEntryOffsets()
-try:
-image.PackEntries()
-image.CheckSize()
-image.CheckEntries()
-except Exception as e:
-if args.map:
-fname = image.WriteMap()
-print("Wrote map file '%s' to show errors"  % fname)
-raise
-image.SetImagePos()
-if args.update_fdt:
-image.SetCalculatedProperties()
-for