[RSB 5 PATCH 6/6] sb/setbuilder: Install the build when stagging or configured to install

[chrisj]

From: Chris Johns 

Updates #4716
---
 source-builder/sb/setbuilder.py | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/source-builder/sb/setbuilder.py b/source-builder/sb/setbuilder.py
index 1452aca..9836e9b 100644
--- a/source-builder/sb/setbuilder.py
+++ b/source-builder/sb/setbuilder.py
@@ -226,6 +226,9 @@ class buildset:
 def installing(self):
 return self.install_mode() == 'installing'
 
+def installable(self):
+return not self.opts.no_install() or self.staging()
+
 def staging(self):
 return not self.installing()
 
@@ -553,14 +556,14 @@ class buildset:
', '.join([b.name() for b in builds])))
 if deps is None and not have_errors:
 for b in builds:
-log.trace('_bset:   : %s: %r' % (self.install_mode(),
- b.installable()))
+log.trace('_bset:   : %s: installable=%r 
build-installable=%r' % \
+  (self.install_mode(), self.installable(), 
b.installable()))
 if b.installable():
 prefix = b.config.expand('%{_prefix}')
 buildroot = path.join(b.config.expand('%{buildroot}'), 
prefix)
 if self.staging():
 prefix = b.config.expand('%{stagingroot}')
-if not self.opts.no_install():
+if self.installable():
 self.install(self.install_mode(), b.name(), 
buildroot, prefix)
 #
 # Sizes ...
-- 
2.37.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[RSB 5 PATCH 5/6] rtems/kernel: Support deployment standard buildset configs

[chrisj]

From: Chris Johns 

- Check and optionally support arch/bsp format 'with_rtems_bsp' defines

Updates #4717
---
 rtems/config/tools/rtems-kernel-common.cfg | 49 +-
 1 file changed, 39 insertions(+), 10 deletions(-)

diff --git a/rtems/config/tools/rtems-kernel-common.cfg 
b/rtems/config/tools/rtems-kernel-common.cfg
index 3a17a28..ea2e0ac 100644
--- a/rtems/config/tools/rtems-kernel-common.cfg
+++ b/rtems/config/tools/rtems-kernel-common.cfg
@@ -19,6 +19,43 @@
  %define rtems_kernel_version %{rsb_version}
 %endif
 
+#
+# The BSP.
+#
+%if !%{defined with_rtems_bsp} && %{rtems_kernel_error}
+ %error No RTEMS BSP specified: --with-rtems-bsp=bsp
+%endif
+%define rtems_bsp %{with_rtems_bsp}
+
+#
+# If the BSP(s) have a '/' it is the arch/bsp notation.
+#
+%define is_arch_bsp %(echo %{rtems_bsp} | sed -e 's/.*\/.*/yes/g')
+%if %{is_arch_bsp} == yes
+ %define rtems_ab_bsps \
+  %(x=''; \
+   for b in %{rtems_bsp}; do x="$x "$(echo $b | sed -e 's/.*\///g'); 
done; \
+   echo $x)
+ %define rtems_ab_archs  \
+  %(x=''; \
+   for b in %{rtems_bsp}; do x="$x "$(echo $b | sed -e 's/\/.*//g'); 
done; \
+   echo $x | tr ' ' '\n' | sort | uniq)
+ %define rtems_arch_count %(echo %{rtems_ab_archs} | tr ' ' '\n' | wc -l)
+ %if %{rtems_arch_count} != 1
+  %error Invalid BSP architecture count
+ %endif
+ %define rtems_bsp%{rtems_ab_bsps}
+ %define rtems_target %{rtems_ab_archs}-rtems%{rtems_version}
+ %define rtems_host   %{rtems_target}
+%endif
+
+%define rtems_bsp_count %(echo %{rtems_bsp} | tr ' ' '\n' | wc -l)
+%if %{rtems_bsp_count} == 1
+ %define rtems_bsp_pkgname %{rtems_bsp}
+%else
+ %define rtems_bsp_pkgname bsps
+%endif
+
 #
 # The target. It could be set in rtems_target.
 #
@@ -29,14 +66,6 @@
  %error No RTEMS target specified: --target=-rtems.
 %endif
 
-#
-# The BSP.
-#
-%if !%{defined with_rtems_bsp} && %{rtems_kernel_error}
- %error No RTEMS BSP specified: --with-rtems-bsp=bsp
-%endif
-%define rtems_bsp %{with_rtems_bsp}
-
 #
 # If no tools are provided use the prefix.
 #
@@ -52,12 +81,12 @@
 #
 # Define the package.
 #
-package: rtems-%{rtems_version}-%{_target}-%{rtems_bsp}-%{_host}-%{release}
+package: 
rtems-%{rtems_version}-%{_target}-%{rtems_bsp_pkgname}-%{_host}-%{release}
 
 #
 # Package details.
 #
-Name:%{_target}-kernel-%{rtems_bsp}-%{release}
+Name:%{_target}-kernel-%{rtems_bsp_pkgname}-%{release}
 Summary: RTEMS v%{rtems_kernel_version} for target %{_target} BSP %{rtems_bsp}
 Version: %{rtems_kernel_version}
 Release: %{release}
-- 
2.37.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[RSB 5 PATCH 4/6] sb/setbuilder: Do not install if --no-install option is used

[chrisj]

From: Chris Johns 

- This is a bug introduced in the recent bset tar file changes

Updates #4716
---
 source-builder/sb/setbuilder.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/source-builder/sb/setbuilder.py b/source-builder/sb/setbuilder.py
index 36fca46..1452aca 100644
--- a/source-builder/sb/setbuilder.py
+++ b/source-builder/sb/setbuilder.py
@@ -560,7 +560,8 @@ class buildset:
 buildroot = path.join(b.config.expand('%{buildroot}'), 
prefix)
 if self.staging():
 prefix = b.config.expand('%{stagingroot}')
-self.install(self.install_mode(), b.name(), buildroot, 
prefix)
+if not self.opts.no_install():
+self.install(self.install_mode(), b.name(), 
buildroot, prefix)
 #
 # Sizes ...
 #
-- 
2.37.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[RSB 5 PATCH 2/6] sb/config: Correctly handle multiple config paths

[chrisj]

From: Chris Johns 

- Add rtems/config to the config directories searched to better
  support deployment

- Correctly expand the configdir and path searchs

Updates #4716
---
 source-builder/defaults.mc  |  2 +-
 source-builder/sb/config.py | 52 ++---
 2 files changed, 26 insertions(+), 28 deletions(-)

diff --git a/source-builder/defaults.mc b/source-builder/defaults.mc
index 8ed7003..4658277 100644
--- a/source-builder/defaults.mc
+++ b/source-builder/defaults.mc
@@ -98,7 +98,7 @@ _host_cc:none,none, 'gcc'
 _host_cxx:   none,none, 'g++'
 _arch:   none,none, '%{_host_arch}'
 _topdir: dir, required, '%{_cwd}'
-_configdir:  dir, optional, 
'%{_topdir}/config:%{_sbdir}/config:%{_sbtop}/bare/config'
+_configdir:  dir, optional, 
'%{_topdir}/config:%{_sbdir}/config:%{_sbtop}/bare/config:%{_sbtop}/rtems/config'
 _tardir: dir, optional, '%{_topdir}/tar'
 _sourcedir:  dir, optional, '%{_topdir}/sources'
 _patchdir:   dir, optional, '%{_topdir}/patches:%{_sbdir}/patches'
diff --git a/source-builder/sb/config.py b/source-builder/sb/config.py
index df2aa31..9250896 100644
--- a/source-builder/sb/config.py
+++ b/source-builder/sb/config.py
@@ -785,7 +785,16 @@ class file:
 mn = '%{nil}'
 if mn:
 if mn.lower() in self.macros:
-s = s.replace(m, self.macros[mn.lower()])
+em = self.macros[mn.lower()]
+if self.macros.get_type(mn) == 'dir' and ':' in em:
+ss = []
+for sp in s.split():
+if m in sp:
+sp = ':'.join([sp.replace(mn, ps) for ps 
in em.split(':')])
+ss += [sp]
+s = ' '.join(ss)
+else:
+s = s.replace(m, self.macros[mn.lower()])
 expanded = True
 elif show_warning:
 self._error("macro '%s' not found" % (mn))
@@ -1345,6 +1354,14 @@ class file:
 right = right[:-1]
 return end
 
+def search_path(confignames):
+for configname in confignames.split(':'):
+if not configname.endswith('.cfg'):
+configname = '%s.cfg' % (configname)
+if path.exists(configname):
+return configname
+return None
+
 if self.load_depth == 0:
 self._packages[self.package] = package(self.package,
self.define('%{_arch}'),
@@ -1358,7 +1375,7 @@ class file:
 
 #
 # Locate the config file. Expand any macros then add the
-# extension. Check if the file exists, therefore directly
+# extension. Check if the file exists then it is directly
 # referenced. If not see if the file contains ':' or the path
 # separator. If it does split the path else use the standard config dir
 # path in the defaults.
@@ -1366,32 +1383,13 @@ class file:
 
 exname = self.expand(name)
 
-#
-# Macro could add an extension.
-#
-if exname.endswith('.cfg'):
-configname = exname
-else:
-configname = '%s.cfg' % (exname)
-name = '%s.cfg' % (name)
-
-if ':' in configname:
-cfgname = path.basename(configname)
-else:
-cfgname = common_end(configname, name)
+configname = search_path(exname)
+if configname is None:
+configname = search_path(self.expand(path.join('%{_configdir}', 
exname)))
+if configname is None:
+raise error.general('no config file found: %s' % 
(','.join(exname.split(':'
 
-if not path.exists(configname):
-if ':' in configname:
-configdirs = path.dirname(configname).split(':')
-else:
-configdirs = self.define('_configdir').split(':')
-for cp in configdirs:
-configname = path.join(path.abspath(cp), cfgname)
-if path.exists(configname):
-break
-configname = None
-if configname is None:
-raise error.general('no config file found: %s' % (cfgname))
+name = path.basename(configname)
 
 try:
 log.trace('config: %s:  _open: %s' % (self.name, 
path.host(configname)))
-- 
2.37.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[RSB 5 PATCH 3/6] sb/setbuilder: Correctly create build set tar files

[chrisj]

From: Chris Johns 

- Make a single tarfile for all buildsets built

- Use the staging tree as the tarfile source

- Use python's tarfile module

- Create a config.file object without loading a .cfg file

Updates #4716
---
 source-builder/sb/config.py |  5 +--
 source-builder/sb/setbuilder.py | 58 -
 2 files changed, 39 insertions(+), 24 deletions(-)

diff --git a/source-builder/sb/config.py b/source-builder/sb/config.py
index 9250896..24590eb 100644
--- a/source-builder/sb/config.py
+++ b/source-builder/sb/config.py
@@ -258,7 +258,7 @@ class file:
 re.compile('%select'),
 re.compile('%disable') ]
 
-def __init__(self, name, opts, macros = None):
+def __init__(self, name, opts, macros = None, load = True):
 log.trace('config: %s: initialising' % (name))
 self.opts = opts
 self.init_name = name
@@ -267,7 +267,8 @@ class file:
 self.sf = re.compile(r'%\([^\)]+\)')
 self.set_macros(macros)
 self._reset(name)
-self.load(name)
+if load:
+self.load(name)
 
 def __str__(self):
 
diff --git a/source-builder/sb/setbuilder.py b/source-builder/sb/setbuilder.py
index 55bd86f..36fca46 100644
--- a/source-builder/sb/setbuilder.py
+++ b/source-builder/sb/setbuilder.py
@@ -30,11 +30,14 @@ import glob
 import operator
 import os
 import sys
+import tarfile
+
 import textwrap
 
 try:
 import build
 import check
+import config
 import error
 import log
 import mailer
@@ -259,21 +262,31 @@ class buildset:
 self.root_copy(_build.config.expand('%{buildroot}'),
_build.config.expand('%{_tmproot}'))
 
-def bset_tar(self, _build):
-tardir = _build.config.expand('%{_tardir}')
-if (self.opts.get_arg('--bset-tar-file') or 
self.opts.canadian_cross()) \
-   and not _build.macros.get('%{_disable_packaging}'):
+def bset_tar(self, stagingroot):
+if self.opts.get_arg('--bset-tar-file') or self.opts.canadian_cross():
+# Use a config to expand the macros because it supports all
+# expansions, ie %{_cwd}
+cfg = config.file(self.bset, self.opts, self.macros, load=False)
+prefix = cfg.expand('%{_prefix}')
+tardir = cfg.expand('%{_tardir}')
 path.mkdir(tardir)
-tar = path.join(tardir,
-_build.config.expand('%s.tar.bz2' % \
- 
(_build.main_package().name(
-log.notice('tarball: %s' % (os.path.relpath(path.host(tar
+tarname = path.join(tardir,
+path.basename('%s.tar.bz2' % (self.bset)))
+log.notice('tarfile: %s' % (os.path.relpath(path.host(tarname
 if not self.opts.dry_run():
-tmproot = _build.config.expand('%{_tmproot}')
-cmd = _build.config.expand('"cd ' + tmproot + \
-   ' && %{__tar} -cf - . | %{__bzip2} 
> ' + \
-   tar + '"')
-_build.run(cmd, shell_opts = '-c', cwd = tmproot)
+tar = None
+try:
+tar = tarfile.open(tarname, 'w:bz2')
+for filedir in sorted(path.listdir(stagingroot)):
+src = path.join(stagingroot, filedir)
+dst = path.join(prefix, filedir)
+log.trace('tar: %s -> %s' % (src, dst))
+tar.add(src, dst)
+except OSError as oe:
+raise error.general('tarfile: %s: %s' % (self.bset, oe))
+finally:
+if tar is not None:
+tar.close()
 
 def parse(self, bset):
 
@@ -500,8 +513,6 @@ class buildset:
 copy.copy(self.macros),
 format = 'xml',
 mail = mail)
-if s == len(configs) - 1 and not have_errors:
-self.bset_tar(b)
 else:
 deps += b.config.includes()
 builds += [b]
@@ -540,7 +551,7 @@ class buildset:
 log.trace('_bset: %2d: %s: builds: %s' % \
   (nesting_count, self.install_mode(),
', '.join([b.name() for b in builds])))
-if deps is None and not self.opts.no_install() and not have_errors:
+if deps is None and not have_errors:
 for b in builds:
 log.trace('_bset:   : %s: %r' % (self.install_mode(),
  b.installable()))
@@ -550,7 +561,6 @@ class buildset:
 if self.staging():
 

[RSB 5 PATCH 1/6] sb/setbuilder: Support line continuation

[chrisj]

From: Chris Johns 

Updates #4716
---
 source-builder/sb/setbuilder.py | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/source-builder/sb/setbuilder.py b/source-builder/sb/setbuilder.py
index 16e8cc8..55bd86f 100644
--- a/source-builder/sb/setbuilder.py
+++ b/source-builder/sb/setbuilder.py
@@ -284,6 +284,13 @@ class buildset:
 line = line[1:b]
 return line.strip()
 
+def _clean_and_pack(line, last_line):
+leading_ws = ' ' if len(line) > 0 and line[0].isspace() else ''
+line = _clean(line)
+if len(last_line) > 0:
+line = last_line + leading_ws + line
+return line
+
 bset = macro_expand(self.macros, bset)
 bsetname = bset
 
@@ -306,11 +313,16 @@ class buildset:
 
 try:
 lc = 0
+ll = ''
 for l in bset:
 lc += 1
-l = _clean(l)
+l = _clean_and_pack(l, ll)
 if len(l) == 0:
 continue
+if l[-1] == '\\':
+ll = l[0:-1]
+continue
+ll = ''
 log.trace('_bset:   : %s: %03d: %s' % (self.bset, lc, l))
 ls = l.split()
 if ls[0][-1] == ':' and ls[0][:-1] == 'package':
-- 
2.37.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[RSB 5] Deployment and backporting of fixes

[chrisj]

Hi

These patches are a mix of back ported fixed from our devel branch
and deployment support for RTEMS 5.

I have a personal repo I will upload soon that tests RTEMS 5 as well
as RTEMS 6 (devel branch).

Chris

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] c-user: Add application config info directives

[Chris Johns]

Hi,

Sorry about not reviewing this before now.

Thanks for this, it looks good and most welcome. I have raised some general
questions that apply to a number of entries. I did not tag every case.

On 10/9/22 12:33 am, Sebastian Huber wrote:
> Close #4267.
> Close #4269.
> ---
>  c-user/config/directives.rst   | 1197 
>  c-user/config/index.rst|2 +
>  c-user/config/intro.rst|   43 --
>  c-user/config/introduction.rst |  221 ++
>  4 files changed, 1420 insertions(+), 43 deletions(-)
>  create mode 100644 c-user/config/directives.rst
>  create mode 100644 c-user/config/introduction.rst
> 
> diff --git a/c-user/config/directives.rst b/c-user/config/directives.rst
> new file mode 100644
> index 000..7a73d77
> --- /dev/null
> +++ b/c-user/config/directives.rst
> @@ -0,0 +1,1197 @@
> +.. SPDX-License-Identifier: CC-BY-SA-4.0
> +
> +.. Copyright (C) 2009, 2021 embedded brains GmbH 
> (http://www.embedded-brains.de)
> +.. Copyright (C) 1988, 2021 On-Line Applications Research Corporation (OAR)
> +
> +.. This file is part of the RTEMS quality process and was automatically
> +.. generated.  If you find something that needs to be fixed or
> +.. worded better please post a report or patch to an RTEMS mailing list
> +.. or raise a bug report:
> +..
> +.. https://www.rtems.org/bugs.html
> +..
> +.. For information on updating and regenerating please refer to the How-To
> +.. section in the Software Requirements Engineering chapter of the
> +.. RTEMS Software Engineering manual.  The manual is provided as a part of
> +.. a release.  For development sources please refer to the online
> +.. documentation at:
> +..
> +.. https://docs.rtems.org
> +
> +.. _ApplicationConfigurationInformationDirectives:
> +
> +Directives
> +==
> +
> +This section details the directives of the Application Configuration
> +Information. A subsection is dedicated to each of this manager's directives 
> and
> +lists the calling sequence, parameters, description, return values, and notes
> +of the directive.
> +
> +.. Generated from spec:/rtems/config/if/get-build-label
> +
> +.. raw:: latex
> +
> +\clearpage
> +
> +.. index:: rtems_get_build_label()
> +
> +.. _InterfaceRtemsGetBuildLabel:
> +
> +rtems_get_build_label()
> +---
> +
> +Gets the RTEMS build label.
> +
> +.. rubric:: CALLING SEQUENCE:
> +
> +.. code-block:: c
> +
> +const char *rtems_get_build_label( void );
> +
> +.. rubric:: DESCRIPTION:
> +
> +The build label is a user-provided string defined by the build configuration.

Is the way a user sets this label documented? What can a user put in the build
label?

> +
> +.. rubric:: RETURN VALUES:
> +
> +Returns the pointer to the RTEMS build label.
> +
> +.. rubric:: NOTES:
> +
> +The build label can be used to distinguish test suite results obtained from
> +different build configurations.  A use case is to record test results with
> +performance data to track performance regressions.  For this a database of
> +performance limits is required.  The build label and the target hash obtained
> +from :ref:`InterfaceRtemsGetTargetHash` can be used as a key to obtain
> +performance limits.
> +
> +.. rubric:: CONSTRAINTS:
> +
> +The following constraints apply to this directive:
> +
> +* The directive may be called from within any runtime context.
> +
> +* The directive will not cause the calling task to be preempted.
> +

What can a user expect from the format of the build label? ASCII? White space?

> +.. Generated from spec:/rtems/config/if/get-copyright-notice
> +
> +.. raw:: latex
> +
> +\clearpage
> +
> +.. index:: rtems_get_copyright_notice()
> +
> +.. _InterfaceRtemsGetCopyrightNotice:
> +
> +rtems_get_copyright_notice()
> +
> +
> +Gets the RTEMS copyright notice.

Subject to format changes, ie users should decode and expect backwards
compatibility?

> +
> +.. rubric:: CALLING SEQUENCE:
> +
> +.. code-block:: c
> +
> +const char *rtems_get_copyright_notice( void );
> +
> +.. rubric:: RETURN VALUES:
> +
> +Returns the pointer to the RTEMS copyright notice.
> +
> +.. Generated from spec:/rtems/config/if/get-target-hash
> +
> +.. raw:: latex
> +
> +\clearpage
> +
> +.. index:: rtems_get_target_hash()
> +
> +.. _InterfaceRtemsGetTargetHash:
> +
> +rtems_get_target_hash()
> +---
> +
> +Gets the RTEMS target hash.
> +
> +.. rubric:: CALLING SEQUENCE:
> +
> +.. code-block:: c
> +
> +const char *rtems_get_target_hash( void );
> +
> +.. rubric:: DESCRIPTION:
> +
> +The target hash is calculated from BSP-specific values which characterize a
> +target system.

What is the format of the hash, ie hex ascii digits?

Is the size set or can it vary?

> +
> +.. rubric:: RETURN VALUES:
> +
> +Returns the pointer to the RTEMS target hash.
> +
> +.. rubric:: NOTES:
> +
> +For example, the device tree, settings of the memory controller, processor 
> and
> +bus frequencies, a serial number of a chip may