Re: [PATCH 1/1] RSB: Mitigate too short error reports

[Frank Kühndel]

Hello Chris,
Hello Joel,

On 1/16/23 18:27, Joel Sherrill wrote:

Subject:
Re: [PATCH 1/1] RSB: Mitigate too short error reports
From:
Joel Sherrill 
Date:
1/16/23, 18:27

To:
Frank Kühndel 
CC:
Chris Johns , devel@rtems.org


On Mon, Jan 16, 2023 at 8:46 AM Frank Kühndel <
frank.kuehn...@embedded-brains.de> wrote:


Hi Chris,

On 1/16/23 01:02, Chris Johns wrote:

Subject:
Re: [PATCH 1/1] RSB: Mitigate too short error reports
From:
Chris Johns
Date:
1/16/23, 01:02

To:
Frank Kühndel,devel@rtems.org


On 22/12/2022 9:09 pm, Frank Kühndel wrote:

On 12/21/22 00:06, Chris Johns wrote:

On 21/12/2022 3:44 am, Frank Kuehndel wrote:

From: Frank Kühndel

Close #4642
---
source-builder/sb/ereport.py | 4 
1 file changed, 4 insertions(+)

diff --git a/source-builder/sb/ereport.py

b/source-builder/sb/ereport.py

index d8fb5f6..d391917 100755
--- a/source-builder/sb/ereport.py
+++ b/source-builder/sb/ereport.py
@@ -55,6 +55,10 @@ def generate(name, opts, header = None, footer =

None):

with open(name, 'w') as l:
l.write(os.linesep.join(r))
log.notice('  See error report: %s' % (name))
+log.notice('  (Hint: The first error may be in front of

a '

+'line containing\n'
+'  "Error 1" [GNU make] and may be only in the whole

log '

Is this too specific to GNU make? What ifs a package uses cmake or

something

else?

As the text indicates, this is specific to GNU make. For those using

something

else reading this text will still hint that the first error may not be

in the

end of the report "and may be only in the whole log".

Another weak point in this text is that by far not all errors

originating from

"make". Yet, the true trouble is the original "See error report: %s"

where it

can sometimes happen that the error is not in this "error report" at

all.

I found it difficult to find a wording which is short, clear and

helpful to the

reader and at the same time not confusing. I am not perfectly happy

with the

notice above. I just found it a reasonable compromise.

If you prefer more generic texts - such as the examples below - I will

send a

new patch with the suggested test.

  "Hint: The first error may be far way from the end of the
  report and in cases can only be found in the whole build log."

  "Hint: The error is most likely in the error report otherwise
  see the whole build log [--log option]."

If you find any such change might cause more confusion than it might be

helpful,

I think it better to close this bug than to try to fix it.


I think all you have written is valid and I have found the wording

difficult.

There will never be a robust error message scanner or a simple full

proof way to

find errors. The parallel builds makes tracking the errors difficult and

the

point of error and end of the build a long distance apart.

I usually search the logs for "rror:" and that's the first time something
is reported
whether by make or gcc or whatever. It may not be the root cause but it gets
me to the first report.

Cutting any of these long reports down is always going to be possible to
cut out the real issue. It's ok because it it's more than just an odd setup
issue on the host, someone will have to build locally to reproduce the
issue.
And then they will get the full output.



As a result I question the value of the report and wonder if it should be
removed. The report adds overhead to the build as the logging process

needs to

maintain a buffer of lines that is always updating. Your attention and

interest

around this feature highlights how problematic it is so maybe it is

simpler and

better to remove it and we leave users to find the error in the log file.

I am happy to accept the report has not worked as a feature, remove it

and in

the process we recover some overheads in the logging area of the RSB?


I am not against the error report and I do not say it is a useless
feature. It is just that I found the message '  See error report: %s'
confusing in those cases where the report does not contain the error
at all because it is too short (the error report consists simply of the
last 400 lines of the build log).

To answer your question, I believe there is always a build log - no
matter whether the `--log` option is used or not. In this case, removing
the error report and pointing to the build log in case of error (for
example like '  See build log: %s') would certainly solve all my concerns.


But on the build@ reports, it is nice to have something. Many times it
is possible to diagnose the issue. Just in the past fifteen minutes, there
was one which having the log made it clear that CentOS 7 and other older
distributions need to use a newer GCC. Having the info in the build@
message was more than enough to diagnose that.


On the other hand, implementing the error report took time and was
certainly done with good reason. I do not feel like I should be the one
de

Re: [PATCH 1/1] RSB: Mitigate too short error reports

[Joel Sherrill]

On Thu, Jan 19, 2023 at 6:47 AM Frank Kühndel <
frank.kuehn...@embedded-brains.de> wrote:

> Hello Chris,
> Hello Joel,
>
> On 1/16/23 18:27, Joel Sherrill wrote:
> > Subject:
> > Re: [PATCH 1/1] RSB: Mitigate too short error reports
> > From:
> > Joel Sherrill 
> > Date:
> > 1/16/23, 18:27
> >
> > To:
> > Frank Kühndel 
> > CC:
> > Chris Johns , devel@rtems.org
> >
> >
> > On Mon, Jan 16, 2023 at 8:46 AM Frank Kühndel <
> > frank.kuehn...@embedded-brains.de> wrote:
> >
> >> Hi Chris,
> >>
> >> On 1/16/23 01:02, Chris Johns wrote:
> >>> Subject:
> >>> Re: [PATCH 1/1] RSB: Mitigate too short error reports
> >>> From:
> >>> Chris Johns
> >>> Date:
> >>> 1/16/23, 01:02
> >>>
> >>> To:
> >>> Frank Kühndel,devel@rtems.org
> >>>
> >>>
> >>> On 22/12/2022 9:09 pm, Frank Kühndel wrote:
>  On 12/21/22 00:06, Chris Johns wrote:
> > On 21/12/2022 3:44 am, Frank Kuehndel wrote:
> >> From: Frank Kühndel
> >>
> >> Close #4642
> >> ---
> >> source-builder/sb/ereport.py | 4 
> >> 1 file changed, 4 insertions(+)
> >>
> >> diff --git a/source-builder/sb/ereport.py
> >> b/source-builder/sb/ereport.py
> >> index d8fb5f6..d391917 100755
> >> --- a/source-builder/sb/ereport.py
> >> +++ b/source-builder/sb/ereport.py
> >> @@ -55,6 +55,10 @@ def generate(name, opts, header = None, footer =
> >> None):
> >> with open(name, 'w') as l:
> >> l.write(os.linesep.join(r))
> >> log.notice('  See error report: %s' % (name))
> >> +log.notice('  (Hint: The first error may be in front of
> >> a '
> >> +'line containing\n'
> >> +'  "Error 1" [GNU make] and may be only in the
> whole
> >> log '
> > Is this too specific to GNU make? What ifs a package uses cmake or
> >> something
> > else?
>  As the text indicates, this is specific to GNU make. For those using
> >> something
>  else reading this text will still hint that the first error may not be
> >> in the
>  end of the report "and may be only in the whole log".
> 
>  Another weak point in this text is that by far not all errors
> >> originating from
>  "make". Yet, the true trouble is the original "See error report: %s"
> >> where it
>  can sometimes happen that the error is not in this "error report" at
> >> all.
>  I found it difficult to find a wording which is short, clear and
> >> helpful to the
>  reader and at the same time not confusing. I am not perfectly happy
> >> with the
>  notice above. I just found it a reasonable compromise.
> 
>  If you prefer more generic texts - such as the examples below - I will
> >> send a
>  new patch with the suggested test.
> 
>    "Hint: The first error may be far way from the end of the
>    report and in cases can only be found in the whole build log."
> 
>    "Hint: The error is most likely in the error report otherwise
>    see the whole build log [--log option]."
> 
>  If you find any such change might cause more confusion than it might
> be
> >> helpful,
>  I think it better to close this bug than to try to fix it.
> 
> >>> I think all you have written is valid and I have found the wording
> >> difficult.
> >>> There will never be a robust error message scanner or a simple full
> >> proof way to
> >>> find errors. The parallel builds makes tracking the errors difficult
> and
> >> the
> >>> point of error and end of the build a long distance apart.
> > I usually search the logs for "rror:" and that's the first time something
> > is reported
> > whether by make or gcc or whatever. It may not be the root cause but it
> gets
> > me to the first report.
> >
> > Cutting any of these long reports down is always going to be possible to
> > cut out the real issue. It's ok because it it's more than just an odd
> setup
> > issue on the host, someone will have to build locally to reproduce the
> > issue.
> > And then they will get the full output.
> >
> >
> >>> As a result I question the value of the report and wonder if it should
> be
> >>> removed. The report adds overhead to the build as the logging process
> >> needs to
> >>> maintain a buffer of lines that is always updating. Your attention and
> >> interest
> >>> around this feature highlights how problematic it is so maybe it is
> >> simpler and
> >>> better to remove it and we leave users to find the error in the log
> file.
> >>>
> >>> I am happy to accept the report has not worked as a feature, remove it
> >> and in
> >>> the process we recover some overheads in the logging area of the RSB?
> >>>
> >> I am not against the error report and I do not say it is a useless
> >> feature. It is just that I found the message '  See error report: %s'
> >> confusing in those cases where the report does not contain the error
> >> at all because it is too short (the error report consists simply of the
> >> la

Re: [PATCHES rtems, source-builder] Add GitHub Actions scripts

[Gedare Bloom]

Nice. I would like some time to look at this and think about it a
little more. What would be the plan for removing this capability? Will
it leave any artifacts behind in the RTEMS github mirror?

On Mon, Jan 16, 2023 at 6:42 AM Christian Mauderer
 wrote:
>
> Hello,
>
> some weeks ago I created a GitHub Actions based CI script that we
> (embedded brains) wanted to use to test patches (see
> https://github.com/embedded-brains/rtems/tree/ci). I don't think much of
> the RTEMS community noted these. I would like to suggest adding the
> scripts to the official RTEMS repositories so that the actions are
> executed in the official GitHub RTEMS mirrors.
>
> To make sure that GitHub pull requests are not perceived at the official
> way to make RTEMS contributions, an auto-responder action notifies the
> pull request user that the current way to make contributions is sending
> patch sets to devel@rtems.org.
>
> This step will allow users to easily test patches on a number of
> simulators before they send them to the mailing list. No one is forced
> to do it, but everyone can try it. For RTEMS, it has the advantage that
> the patches are at least guaranteed to be compile-clean on a selected
> number of BSPs and that they survived a test run on a simulator.
>
> Please note: With this patch I do not intent to push GitHub as the RTEMS
> CI or to move from mailing list patches to push-requests. My idea is to
> allow everyone to experiment with a proof of concept prototype. Based on
> your experiences in this test phase, I would suggest that we have a
> review discussion in a month or two to select a suitable way forward for
> RTEMS CI. I think after that test phase we all know better what we want
> or expect which helps selecting the best CI system that then can replace
> this proof of concept system with GitHub.
>
> But now to make make it more clear what we will get with merging these
> patches:
>
> You can find a (not yet cleaned up) version of the patches in these
> repositories:
>
> https://github.com/embedded-brains/rtems
> https://github.com/embedded-brains/rtems-source-builder
>
> The results from a current run on RTEMS are here:
>
> https://github.com/embedded-brains/rtems/actions/runs/3901364934
>
> If you scroll down on that page, you get a summary that shows which
> tests fail on three (mostly) randomly selected simulator BSPs. GR740
> usually can run all tests but currently jffs2_fsrdwr fails. The full
> output of the rtems tester is in the Artifacts in case you want to take
> a look at the test output.
>
> If you want to try the CI with some of your patches before we merge this
> to the official repositories, feel free to create a pull request to the
> ci branches of the embedded-brains/rtems repositories. See the github
> manual for guidance how to create a pull request:
>
>   https://docs.github.com/articles/creating-a-pull-request
>
> Note: It is important that you somewhen forked from the official RTEMS
> repositories or from one of the forks using (for example) the fork
> button in the github web interface. If you just pushed a repo to an
> empty one, github doesn't recognize the link and won't allow you to
> create a pull-request towards the embedded-brains repository.
>
> Best regards
>
> Christian
>
>
> ___
> devel mailing list
> devel@rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCHES rtems, source-builder] Add GitHub Actions scripts

[Sebastian Huber]

On 19.01.23 15:42, Gedare Bloom wrote:

Nice. I would like some time to look at this and think about it a
little more. What would be the plan for removing this capability? Will
it leave any artifacts behind in the RTEMS github mirror?


It seems by default stuff is kept for about 90 days:

https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts

--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCHES rtems, source-builder] Add GitHub Actions scripts

[Christian MAUDERER]

Am 19.01.23 um 15:42 schrieb Gedare Bloom:

Nice. I would like some time to look at this and think about it a
little more. What would be the plan for removing this capability? Will
it leave any artifacts behind in the RTEMS github mirror?


As soon as we want to get rid of the scripts again (because we have 
found and implemented a proper CI/CD that we officially want to use and 
not only have in a test phase), we can just remove the scripts with a 
new commit.


If we use both commits, we will have a bot that adds a comment to pull 
requests, that patches should be sent to the mailing list as soon as 
they are tested. It will close pull requests after 30 days. That can be 
disabled again with removing that action.


All artefacts can be removed by everyone with enough rights in the RTEMS 
GitHub organization. That should be most maintainers.


Best regards

Christian



On Mon, Jan 16, 2023 at 6:42 AM Christian Mauderer
 wrote:


Hello,

some weeks ago I created a GitHub Actions based CI script that we
(embedded brains) wanted to use to test patches (see
https://github.com/embedded-brains/rtems/tree/ci). I don't think much of
the RTEMS community noted these. I would like to suggest adding the
scripts to the official RTEMS repositories so that the actions are
executed in the official GitHub RTEMS mirrors.

To make sure that GitHub pull requests are not perceived at the official
way to make RTEMS contributions, an auto-responder action notifies the
pull request user that the current way to make contributions is sending
patch sets to devel@rtems.org.

This step will allow users to easily test patches on a number of
simulators before they send them to the mailing list. No one is forced
to do it, but everyone can try it. For RTEMS, it has the advantage that
the patches are at least guaranteed to be compile-clean on a selected
number of BSPs and that they survived a test run on a simulator.

Please note: With this patch I do not intent to push GitHub as the RTEMS
CI or to move from mailing list patches to push-requests. My idea is to
allow everyone to experiment with a proof of concept prototype. Based on
your experiences in this test phase, I would suggest that we have a
review discussion in a month or two to select a suitable way forward for
RTEMS CI. I think after that test phase we all know better what we want
or expect which helps selecting the best CI system that then can replace
this proof of concept system with GitHub.

But now to make make it more clear what we will get with merging these
patches:

You can find a (not yet cleaned up) version of the patches in these
repositories:

https://github.com/embedded-brains/rtems
https://github.com/embedded-brains/rtems-source-builder

The results from a current run on RTEMS are here:

https://github.com/embedded-brains/rtems/actions/runs/3901364934

If you scroll down on that page, you get a summary that shows which
tests fail on three (mostly) randomly selected simulator BSPs. GR740
usually can run all tests but currently jffs2_fsrdwr fails. The full
output of the rtems tester is in the Artifacts in case you want to take
a look at the test output.

If you want to try the CI with some of your patches before we merge this
to the official repositories, feel free to create a pull request to the
ci branches of the embedded-brains/rtems repositories. See the github
manual for guidance how to create a pull request:

   https://docs.github.com/articles/creating-a-pull-request

Note: It is important that you somewhen forked from the official RTEMS
repositories or from one of the forks using (for example) the fork
button in the github web interface. If you just pushed a repo to an
empty one, github doesn't recognize the link and won't allow you to
create a pull-request towards the embedded-brains repository.

Best regards

Christian


___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel


--

embedded brains GmbH
Herr Christian MAUDERER
Dornierstr. 4
82178 Puchheim
Germany
email:  christian.maude...@embedded-brains.de
phone:  +49-89-18 94 741 - 18
mobile: +49-176-152 206 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH] build: Update PyYAML to 5.4.1

[Sebastian Huber]

---
 yaml/LICENSE  |  2 +-
 yaml/PKG-INFO | 14 ++
 yaml/lib/yaml/__init__.py |  2 +-
 yaml/lib/yaml/constructor.py  | 30 ++
 yaml/lib/yaml/cyaml.py|  2 +-
 yaml/lib/yaml/reader.py   |  9 +++--
 yaml/lib/yaml/resolver.py |  4 ++--
 yaml/lib3/yaml/__init__.py|  2 +-
 yaml/lib3/yaml/constructor.py | 24 
 yaml/lib3/yaml/cyaml.py   |  2 +-
 yaml/lib3/yaml/resolver.py|  4 ++--
 11 files changed, 56 insertions(+), 39 deletions(-)

diff --git a/yaml/LICENSE b/yaml/LICENSE
index 3d82c281ee..2f1b8e15e5 100644
--- a/yaml/LICENSE
+++ b/yaml/LICENSE
@@ -1,4 +1,4 @@
-Copyright (c) 2017-2020 Ingy döt Net
+Copyright (c) 2017-2021 Ingy döt Net
 Copyright (c) 2006-2016 Kirill Simonov
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
diff --git a/yaml/PKG-INFO b/yaml/PKG-INFO
index bf47ef4b35..04d0abf6e5 100644
--- a/yaml/PKG-INFO
+++ b/yaml/PKG-INFO
@@ -1,12 +1,17 @@
-Metadata-Version: 1.1
+Metadata-Version: 1.2
 Name: PyYAML
-Version: 5.3.1
+Version: 5.4.1
 Summary: YAML parser and emitter for Python
-Home-page: https://github.com/yaml/pyyaml
+Home-page: https://pyyaml.org/
 Author: Kirill Simonov
 Author-email: x...@resolvent.net
 License: MIT
 Download-URL: https://pypi.org/project/PyYAML/
+Project-URL: Bug Tracker, https://github.com/yaml/pyyaml/issues
+Project-URL: CI, https://github.com/yaml/pyyaml/actions
+Project-URL: Documentation, https://pyyaml.org/wiki/PyYAMLDocumentation
+Project-URL: Mailing lists, 
http://lists.sourceforge.net/lists/listinfo/yaml-core
+Project-URL: Source Code, https://github.com/yaml/pyyaml
 Description: YAML is a data serialization format designed for human readability
 and interaction with scripting languages.  PyYAML is a YAML parser
 and emitter for Python.
@@ -28,11 +33,12 @@ Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 2
 Classifier: Programming Language :: Python :: 2.7
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.5
 Classifier: Programming Language :: Python :: 3.6
 Classifier: Programming Language :: Python :: 3.7
 Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: Implementation :: CPython
 Classifier: Programming Language :: Python :: Implementation :: PyPy
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Topic :: Text Processing :: Markup
+Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*
diff --git a/yaml/lib/yaml/__init__.py b/yaml/lib/yaml/__init__.py
index 211fc8665e..3c988198d5 100644
--- a/yaml/lib/yaml/__init__.py
+++ b/yaml/lib/yaml/__init__.py
@@ -8,7 +8,7 @@ from nodes import *
 from loader import *
 from dumper import *
 
-__version__ = '5.3.1'
+__version__ = '5.4.1'
 
 try:
 from cyaml import *
diff --git a/yaml/lib/yaml/constructor.py b/yaml/lib/yaml/constructor.py
index 794681cb7f..ff4e36828e 100644
--- a/yaml/lib/yaml/constructor.py
+++ b/yaml/lib/yaml/constructor.py
@@ -38,6 +38,12 @@ class timezone(datetime.tzinfo):
 def dst(self, dt=None):
 return datetime.timedelta(0)
 
+def __copy__(self):
+return self.__deepcopy__()
+
+def __deepcopy__(self, memodict={}):
+return self.__class__(self.utcoffset())
+
 __repr__ = __str__ = tzname
 
 
@@ -722,18 +728,6 @@ FullConstructor.add_multi_constructor(
 u'tag:yaml.org,2002:python/name:',
 FullConstructor.construct_python_name)
 
-FullConstructor.add_multi_constructor(
-u'tag:yaml.org,2002:python/module:',
-FullConstructor.construct_python_module)
-
-FullConstructor.add_multi_constructor(
-u'tag:yaml.org,2002:python/object:',
-FullConstructor.construct_python_object)
-
-FullConstructor.add_multi_constructor(
-u'tag:yaml.org,2002:python/object/new:',
-FullConstructor.construct_python_object_new)
-
 class UnsafeConstructor(FullConstructor):
 
 def find_python_module(self, name, mark):
@@ -750,6 +744,18 @@ class UnsafeConstructor(FullConstructor):
 return super(UnsafeConstructor, self).set_python_instance_state(
 instance, state, unsafe=True)
 
+UnsafeConstructor.add_multi_constructor(
+u'tag:yaml.org,2002:python/module:',
+UnsafeConstructor.construct_python_module)
+
+UnsafeConstructor.add_multi_constructor(
+u'tag:yaml.org,2002:python/object:',
+UnsafeConstructor.construct_python_object)
+
+UnsafeConstructor.add_multi_constructor(
+u'tag:yaml.org,2002:python/object/new:',
+UnsafeConstructor.construct_python_object_new)
+
 UnsafeConstructor.add_multi_constructor(
 u'tag:yaml.org,2002:python/object/apply:',
 UnsafeConstructor.construct_python_object_apply)
diff --git a/yaml/lib/yaml/cyaml.py b/yaml/lib/yaml/cyaml.py
index ebb8959352..768b49d6b9 100644
--- a/yaml/li

Re: [PATCH] build: Update PyYAML to 5.4.1

[Sebastian Huber]

The latest version is 6.0 which dropped support for Python 2.7.

--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 1/1] RSB: Mitigate too short error reports

[Frank Kühndel]

Hi Joel,

On 1/19/23 15:08, Joel Sherrill wrote:

Subject:
Re: [PATCH 1/1] RSB: Mitigate too short error reports
From:
Joel Sherrill 
Date:
1/19/23, 15:08

To:
Frank Kühndel 
CC:
Chris Johns , devel@rtems.org


On Thu, Jan 19, 2023 at 6:47 AM Frank Kühndel <
frank.kuehn...@embedded-brains.de> wrote:


Hello Chris,
Hello Joel,

On 1/16/23 18:27, Joel Sherrill wrote:

Subject:
Re: [PATCH 1/1] RSB: Mitigate too short error reports
From:
Joel Sherrill
Date:
1/16/23, 18:27

To:
Frank Kühndel
CC:
Chris Johns,devel@rtems.org


On Mon, Jan 16, 2023 at 8:46 AM Frank Kühndel <
frank.kuehn...@embedded-brains.de> wrote:


Hi Chris,

On 1/16/23 01:02, Chris Johns wrote:

Subject:
Re: [PATCH 1/1] RSB: Mitigate too short error reports
From:
Chris Johns
Date:
1/16/23, 01:02

To:
Frank Kühndel,devel@rtems.org


On 22/12/2022 9:09 pm, Frank Kühndel wrote:

On 12/21/22 00:06, Chris Johns wrote:

On 21/12/2022 3:44 am, Frank Kuehndel wrote:

From: Frank Kühndel

Close #4642
---
 source-builder/sb/ereport.py | 4 
 1 file changed, 4 insertions(+)

diff --git a/source-builder/sb/ereport.py

b/source-builder/sb/ereport.py

index d8fb5f6..d391917 100755
--- a/source-builder/sb/ereport.py
+++ b/source-builder/sb/ereport.py
@@ -55,6 +55,10 @@ def generate(name, opts, header = None, footer =

None):

 with open(name, 'w') as l:
 l.write(os.linesep.join(r))
 log.notice('  See error report: %s' % (name))
+log.notice('  (Hint: The first error may be in front of

a '

+'line containing\n'
+'  "Error 1" [GNU make] and may be only in the

whole

log '

Is this too specific to GNU make? What ifs a package uses cmake or

something

else?

As the text indicates, this is specific to GNU make. For those using

something

else reading this text will still hint that the first error may not be

in the

end of the report "and may be only in the whole log".

Another weak point in this text is that by far not all errors

originating from

"make". Yet, the true trouble is the original "See error report: %s"

where it

can sometimes happen that the error is not in this "error report" at

all.

I found it difficult to find a wording which is short, clear and

helpful to the

reader and at the same time not confusing. I am not perfectly happy

with the

notice above. I just found it a reasonable compromise.

If you prefer more generic texts - such as the examples below - I will

send a

new patch with the suggested test.

   "Hint: The first error may be far way from the end of the
   report and in cases can only be found in the whole build log."

   "Hint: The error is most likely in the error report otherwise
   see the whole build log [--log option]."

If you find any such change might cause more confusion than it might

be

helpful,

I think it better to close this bug than to try to fix it.


I think all you have written is valid and I have found the wording

difficult.

There will never be a robust error message scanner or a simple full

proof way to

find errors. The parallel builds makes tracking the errors difficult

and

the

point of error and end of the build a long distance apart.

I usually search the logs for "rror:" and that's the first time something
is reported
whether by make or gcc or whatever. It may not be the root cause but it

gets

me to the first report.

Cutting any of these long reports down is always going to be possible to
cut out the real issue. It's ok because it it's more than just an odd

setup

issue on the host, someone will have to build locally to reproduce the
issue.
And then they will get the full output.



As a result I question the value of the report and wonder if it should

be

removed. The report adds overhead to the build as the logging process

needs to

maintain a buffer of lines that is always updating. Your attention and

interest

around this feature highlights how problematic it is so maybe it is

simpler and

better to remove it and we leave users to find the error in the log

file.

I am happy to accept the report has not worked as a feature, remove it

and in

the process we recover some overheads in the logging area of the RSB?


I am not against the error report and I do not say it is a useless
feature. It is just that I found the message '  See error report: %s'
confusing in those cases where the report does not contain the error
at all because it is too short (the error report consists simply of the
last 400 lines of the build log).

To answer your question, I believe there is always a build log - no
matter whether the `--log` option is used or not. In this case, removing
the error report and pointing to the build log in case of error (for
example like '  See build log: %s') would certainly solve all my

concerns.

But on the build@ reports, it is nice to have something. Many times it
is possible to diagnose the issue. Just in the past fifteen minutes,

there

was one whic

Re: [PATCH 1/3] build: Format build items

[Amar Takhar]

On 2023-01-19 08:21 +0100, Sebastian Huber wrote:
> 
> In rtems-central.git there are Python modules and scripts which generate 
> source, header, and documentation files from specification items. This 
> repository contains the pre-qualification support for RTEMS. By 
> accident, a part of the build system uses also specification items. So, 
> if you want to change the format of the items, you can use the tooling 
> available in rtems-central.git to do this. An example is the recent 
> merge of the "default" and "default-by-variant" attributes.

Why are these in 'rtems-central' and not the RTEMS repository?  Why are we 
using 
tools with unpinned versioning generating files in RTEMS?


> The changes were made by a 10 liner throw away script. I could have done 
> the changes also manually it would be just more work.

This does not alter my above point.  I understand that in this case it was a 
script but the rest wasn't generated by a throw-away script.


> There is nothing to regenerate. The patch set contains simple format 
> changes and a transformation of attributes.
> 
> However, your concern is still valid for the generated files, for example
> 
> https://github.com/RTEMS/rtems/blob/master/cpukit/include/rtems.h
> 
> which is generated from
> 
> https://github.com/RTEMS/rtems-central/blob/master/spec/rtems/if/header.yml

Yes I do not like this at all both the software and header.yml should be 
sitting 
in our main repository.


> Most of the Classic API header files are generated from specification 
> items. If nobody takes care that rtems-central.git is used and stays up 
> to date the specification will get out of synchronization with rtems.git.

Okay but why is this sitting in rtems-central.git and not in rtems.git.  You've 
just made the argument that it's very important we don't want anything to be 
out 
of sync.


> > I'm not against the purpose of the tool or the work has been done but this 
> > is
> > not the correct way to handle generated files within a repository if we 
> > want to
> > maintain the ability to make changes or debug years down the line.
> 
> I would use a single repository for RTEMS just like the FreeBSD base 
> system, but this is another discussion.

FreeBSD also uses a vendor system which brings all this code under a central 
repository.  They aren't maintaining code outside and then requiring FreeBSD to 
be built with it.  They even bring the compiler in this has an astounding 
impact 
on the usability of FreeBSD.

Recently I took a FreeBSD 3.1 image and rebuilt it for a customer who needed a 
change it was used in an appliance.  It just worked.

If I want to make a chance to rtems.h it's now generated from a header.yml with 
no version pinning how am I going to find the first header.yml to make a single 
change in 20 years?

rtems.h is a small file where we can get rid of it and move to another tool.  
It's integral and anything that works with it is integral this is arguably 
different.


Amar.
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH v1 4/4] testsuites/libtest/dl11: Add DL test for TLS

[Kinsey Moore]

On 1/15/2023 6:07 PM, Chris Johns wrote:

On 13/1/2023 12:51 pm, Kinsey Moore wrote:

On Thu, Jan 12, 2023 at 5:11 PM Joel Sherrill mailto:j...@rtems.org>> wrote:

 Will this need to be added as an expected fail on other architectures?

 Just wondering how many bsp test configuration files need touching


I suppose I could set it to enabled for only AArch64 for now. It would
definitely fail on other architectures.

Should we enable it for archs we would like supported for 6 and then make sure
we have support for the release?

The support is a side effect of the switch to TLS in newlib and not related to
libdl.


That sounds reasonable. I'll leave it as-is.

Assuming that's commit approval, I'll open a ticket for the missing support.


Kinsey

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH] spec: Install NandPsu headers correctly

[Kinsey Moore]

These headers were mistakenly placed in the root of the installed BSP
include directory. They should be in dev/nand.
---
 spec/build/bsps/objnandpsu.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/spec/build/bsps/objnandpsu.yml b/spec/build/bsps/objnandpsu.yml
index 0258417bd7..a0ff1b0b9e 100644
--- a/spec/build/bsps/objnandpsu.yml
+++ b/spec/build/bsps/objnandpsu.yml
@@ -11,7 +11,7 @@ includes:
 - bsps/include/xil/
 - bsps/include/xil/${XIL_SUPPORT_PATH}/
 install:
-- destination: ${BSP_INCLUDEDIR}
+- destination: ${BSP_INCLUDEDIR}/dev/nand
   source:
   - bsps/include/dev/nand/xnandpsu_bbm.h
   - bsps/include/dev/nand/xnandpsu.h
-- 
2.30.2

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 2/3] build: Replace variant patterns with a list

[Chris Johns]

On 19/1/2023 6:06 pm, Sebastian Huber wrote:
> On 19.01.23 01:17, Chris Johns wrote:
 Why has this been done?
>>> The enabled-by expressions used in patch 3 do not support regular 
>>> expressions.
>> I did not pick that up. Why was that regx functionality removed? Is it 
>> related
>> to scripting and rtems-central?
> 
> The enabled-by expressions supported by the wscript don't support regular
> expressions:
> 
> https://github.com/RTEMS/rtems/blob/master/wscript#L144
> 
> The interesting part of the patch set is patch 3:
> 
> Merge the "default" and "default-by-variant" attributes.  Use an
> "enabled-by" expression to select the default value based on the enabled
> set.  This makes it possible to select default values depending on other
> options.  For example you could choose memory settings based on whether
> RTEMS_SMP is enabled or disabled.

Thanks, this make sense. I had not made the connections.

 Why the noise in the patch of only lines being moved? Where has this come 
 from?
 Is there a new requirement spec fields be in some osrt of order?
>>> I did the change with a script which sorted the lists. In general, the lists
>>> should be sorted.
>> This is a good example of problems a lack of scripting support along side the
>> build spec files creates. It makes manual changes difficult when rules get
>> layered like this plus it means you are the only one who can make generated
>> changes without there being a clash of scripting specifics, ie my script vs 
>> your
>> script or something like that.
> 
> I don't understand what is the problem with the sorted lists. If you have a 
> list
> of unordered phrases, it is not unusual to still sort them in lexicographic 
> order.

I welcome sorted entries. Manual edits may or may not be sorted so we end up
with a mix and when a "magic" script runs again it generates more of these white
space changes. If I can run:

./waf formatspec

the "magic" happens it is easy to document and for any user to update these
files, run a command and know the results are OK to post. It is no different to
the work Gedare is doing to get the code formatting sorted so none us have to
audit formatting of the code.

So let me turn this around ... I am do not understand your resistance to there
being importable modules of python in rtems.git to read and write these files in
rtems.git?

>> And I have no intention in cloning rtems-central and learning how to use it.
>> There was a primary agreement for the separation of all qual work and the
>> ability for it to be removed from the project without any side effects.
> 
> You don't have to use the stuff in rtems-central, it could be just more
> convenient and efficient. It would be possible to write a Github action which
> checks that the build items match the expected format for pull requests.

Lets not bring more github stuff into our workflows. There is no agreement that
github is what this project wants to use. I am OK with automation if it is
something we can manage.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 1/3] build: Format build items

[Chris Johns]

On 20/1/2023 6:01 am, Amar Takhar wrote:
> On 2023-01-19 08:21 +0100, Sebastian Huber wrote:
>>
>> In rtems-central.git there are Python modules and scripts which generate 
>> source, header, and documentation files from specification items. This 
>> repository contains the pre-qualification support for RTEMS. By 
>> accident, a part of the build system uses also specification items. So, 
>> if you want to change the format of the items, you can use the tooling 
>> available in rtems-central.git to do this. An example is the recent 
>> merge of the "default" and "default-by-variant" attributes.
> 
> Why are these in 'rtems-central' and not the RTEMS repository?  Why are we 
> using 
> tools with unpinned versioning generating files in RTEMS?

My view is below.

>> The changes were made by a 10 liner throw away script. I could have done 
>> the changes also manually it would be just more work.
> 
> This does not alter my above point.  I understand that in this case it was a 
> script but the rest wasn't generated by a throw-away script.

It is good to know the amount of python is small. It should be easy to add :)

>> There is nothing to regenerate. The patch set contains simple format 
>> changes and a transformation of attributes.
>>
>> However, your concern is still valid for the generated files, for example
>>
>> https://github.com/RTEMS/rtems/blob/master/cpukit/include/rtems.h
>>
>> which is generated from
>>
>> https://github.com/RTEMS/rtems-central/blob/master/spec/rtems/if/header.yml
> 
> Yes I do not like this at all both the software and header.yml should be 
> sitting 
> in our main repository.

I have been OK with the headers and tests being generated this way because the
agreement is files in rtems.git can be manually edited and rtems-central has to
track those changes. The sources and code to generate the files need to be
located together.

>> Most of the Classic API header files are generated from specification 
>> items. If nobody takes care that rtems-central.git is used and stays up 
>> to date the specification will get out of synchronization with rtems.git.
> 
> Okay but why is this sitting in rtems-central.git and not in rtems.git.  
> You've 
> just made the argument that it's very important we don't want anything to be 
> out 
> of sync.

The specification maintenance is a separate responsibility to the function and
role of rtems.git and related repos. The bar to make changes in rtems.git is at
the source level because it is the norm for open source projects and those
working with the code. The separation also helps make it clear how funding of
the qual related work happens. If we bring those specifications items down into
rtems.git the responsibility moves to that repo and that raises the bar on being
able to make a change. Our view on this may change as we learn now the pre-qual
effort effects what we do.

We have YAML files to run our build system and it is awesome. It also allows
automation so I think it makes sense to provide an interface to read and write
these files to aid automation rather than a disjointed set of personal scripts,
tools etc that may break or clash.

>>> I'm not against the purpose of the tool or the work has been done but this 
>>> is
>>> not the correct way to handle generated files within a repository if we 
>>> want to
>>> maintain the ability to make changes or debug years down the line.
>>
>> I would use a single repository for RTEMS just like the FreeBSD base 
>> system, but this is another discussion.
> 
> FreeBSD also uses a vendor system which brings all this code under a central 
> repository.  They aren't maintaining code outside and then requiring FreeBSD 
> to 
> be built with it.  They even bring the compiler in this has an astounding 
> impact 
> on the usability of FreeBSD.
> 
> Recently I took a FreeBSD 3.1 image and rebuilt it for a customer who needed 
> a 
> change it was used in an appliance.  It just worked.
> 
> If I want to make a chance to rtems.h it's now generated from a header.yml 
> with 
> no version pinning how am I going to find the first header.yml to make a 
> single 
> change in 20 years?
> 
> rtems.h is a small file where we can get rid of it and move to another tool.  
> It's integral and anything that works with it is integral this is arguably 
> different.

The FreeBSD single repo is about the kernel and base runtime. The ports are not
part of this so the analogy breaks down. Bringing the RSB, RTEMS tools and
documentation into a single repo creates new complexities I am not sure we are
prepared enough to handle those. Maybe patch review and CI would help here and
how they effect a single repo is something I have not given much consideration 
too.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH v1 4/4] testsuites/libtest/dl11: Add DL test for TLS

[Chris Johns]

On 20/1/2023 8:40 am, Kinsey Moore wrote:
> On 1/15/2023 6:07 PM, Chris Johns wrote:
>> On 13/1/2023 12:51 pm, Kinsey Moore wrote:
>>> On Thu, Jan 12, 2023 at 5:11 PM Joel Sherrill >> > wrote:
>>>
>>>  Will this need to be added as an expected fail on other architectures?
>>>
>>>  Just wondering how many bsp test configuration files need touching
>>>
>>>
>>> I suppose I could set it to enabled for only AArch64 for now. It would
>>> definitely fail on other architectures.
>> Should we enable it for archs we would like supported for 6 and then make 
>> sure
>> we have support for the release?
>>
>> The support is a side effect of the switch to TLS in newlib and not related 
>> to
>> libdl.
>>
> That sounds reasonable. I'll leave it as-is.
> 
> Assuming that's commit approval, I'll open a ticket for the missing support.

Yes and sorry for not making this clear.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 1/1] RSB: Mitigate too short error reports

[Chris Johns]

On 20/1/2023 3:13 am, Frank Kühndel wrote:
> Hi Joel,
> 
> On 1/19/23 15:08, Joel Sherrill wrote:
>> Subject:
>> Re: [PATCH 1/1] RSB: Mitigate too short error reports
>> From:
>> Joel Sherrill 
>> Date:
>> 1/19/23, 15:08
>>
>> To:
>> Frank Kühndel 
>> CC:
>> Chris Johns , devel@rtems.org
>>
>>
>> On Thu, Jan 19, 2023 at 6:47 AM Frank Kühndel <
>> frank.kuehn...@embedded-brains.de> wrote:
>>
>>> Hello Chris,
>>> Hello Joel,
>>>
>>> On 1/16/23 18:27, Joel Sherrill wrote:
 Subject:
 Re: [PATCH 1/1] RSB: Mitigate too short error reports
 From:
 Joel Sherrill
 Date:
 1/16/23, 18:27

 To:
 Frank Kühndel
 CC:
 Chris Johns,devel@rtems.org


 On Mon, Jan 16, 2023 at 8:46 AM Frank Kühndel <
 frank.kuehn...@embedded-brains.de> wrote:

> Hi Chris,
>
> On 1/16/23 01:02, Chris Johns wrote:
>> Subject:
>> Re: [PATCH 1/1] RSB: Mitigate too short error reports
>> From:
>> Chris Johns
>> Date:
>> 1/16/23, 01:02
>>
>> To:
>> Frank Kühndel,devel@rtems.org
>>
>>
>> On 22/12/2022 9:09 pm, Frank Kühndel wrote:
>>> On 12/21/22 00:06, Chris Johns wrote:
 On 21/12/2022 3:44 am, Frank Kuehndel wrote:
> From: Frank Kühndel
>
> Close #4642
> ---
>  source-builder/sb/ereport.py | 4 
>  1 file changed, 4 insertions(+)
>
> diff --git a/source-builder/sb/ereport.py
> b/source-builder/sb/ereport.py
> index d8fb5f6..d391917 100755
> --- a/source-builder/sb/ereport.py
> +++ b/source-builder/sb/ereport.py
> @@ -55,6 +55,10 @@ def generate(name, opts, header = None, footer =
> None):
>  with open(name, 'w') as l:
>  l.write(os.linesep.join(r))
>  log.notice('  See error report: %s' % (name))
> +    log.notice('  (Hint: The first error may be in front of
> a '
> +    'line containing\n'
> +    '  "Error 1" [GNU make] and may be only in the
>>> whole
> log '
 Is this too specific to GNU make? What ifs a package uses cmake or
> something
 else?
>>> As the text indicates, this is specific to GNU make. For those using
> something
>>> else reading this text will still hint that the first error may not be
> in the
>>> end of the report "and may be only in the whole log".
>>>
>>> Another weak point in this text is that by far not all errors
> originating from
>>> "make". Yet, the true trouble is the original "See error report: %s"
> where it
>>> can sometimes happen that the error is not in this "error report" at
> all.
>>> I found it difficult to find a wording which is short, clear and
> helpful to the
>>> reader and at the same time not confusing. I am not perfectly happy
> with the
>>> notice above. I just found it a reasonable compromise.
>>>
>>> If you prefer more generic texts - such as the examples below - I will
> send a
>>> new patch with the suggested test.
>>>
>>>    "Hint: The first error may be far way from the end of the
>>>    report and in cases can only be found in the whole build log."
>>>
>>>    "Hint: The error is most likely in the error report otherwise
>>>    see the whole build log [--log option]."
>>>
>>> If you find any such change might cause more confusion than it might
>>> be
> helpful,
>>> I think it better to close this bug than to try to fix it.
>>>
>> I think all you have written is valid and I have found the wording
> difficult.
>> There will never be a robust error message scanner or a simple full
> proof way to
>> find errors. The parallel builds makes tracking the errors difficult
>>> and
> the
>> point of error and end of the build a long distance apart.
 I usually search the logs for "rror:" and that's the first time something
 is reported
 whether by make or gcc or whatever. It may not be the root cause but it
>>> gets
 me to the first report.

 Cutting any of these long reports down is always going to be possible to
 cut out the real issue. It's ok because it it's more than just an odd
>>> setup
 issue on the host, someone will have to build locally to reproduce the
 issue.
 And then they will get the full output.


>> As a result I question the value of the report and wonder if it should
>>> be
>> removed. The report adds overhead to the build as the logging process
> needs to
>> maintain a buffer of lines that is always updating. Your attention and
> interest
>> around this feature highlights how problematic it is so maybe it is
> simpler and
>> better to remove it and we leave users to find the error in the log
>>> file.
>> I am happy to accept the r

Re: [PATCH 1/3] build: Format build items

[Karel Gardas]

Sorry to hijack that thread, but correction is needed here.

On 1/20/23 01:03, Chris Johns wrote:

The FreeBSD single repo is about the kernel and base runtime. The ports are not
part of this so the analogy breaks down.


Certainly all BSDs have separated ports repos, but AFAIK all of them 
maintain so called base and in base there is kernel, libs, bin/sbin 
commands and most importantly *all* tool-chain required to build *all* 
this code to form whole OS. IMHO this rebuild whole OS distro is 
discussed here, isn't it? If not, I'm sorry for noise here...


Karel


___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] build: Update PyYAML to 5.4.1

[Chris Johns]

On 20/1/2023 2:19 am, Sebastian Huber wrote:
> The latest version is 6.0 which dropped support for Python 2.7.

Was 6 our last version to support python 2 and 3?

Is making this change this close to a release wise?

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH v1 4/4] testsuites/libtest/dl11: Add DL test for TLS

[Kinsey Moore]

On 1/19/2023 6:16 PM, Chris Johns wrote:

On 20/1/2023 8:40 am, Kinsey Moore wrote:

On 1/15/2023 6:07 PM, Chris Johns wrote:

On 13/1/2023 12:51 pm, Kinsey Moore wrote:

On Thu, Jan 12, 2023 at 5:11 PM Joel Sherrill mailto:j...@rtems.org>> wrote:

  Will this need to be added as an expected fail on other architectures?

  Just wondering how many bsp test configuration files need touching


I suppose I could set it to enabled for only AArch64 for now. It would
definitely fail on other architectures.

Should we enable it for archs we would like supported for 6 and then make sure
we have support for the release?

The support is a side effect of the switch to TLS in newlib and not related to
libdl.


That sounds reasonable. I'll leave it as-is.

Assuming that's commit approval, I'll open a ticket for the missing support.

Yes and sorry for not making this clear.


Thanks! The ticket has been created: https://devel.rtems.org/ticket/4802


Kinsey

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 1/3] build: Format build items

[Amar Takhar]

On 2023-01-20 11:03 +1100, Chris Johns wrote:
 
> It is good to know the amount of python is small. It should be easy to add :)

Agreed.


> I have been OK with the headers and tests being generated this way because the
> agreement is files in rtems.git can be manually edited and rtems-central has 
> to
> track those changes. The sources and code to generate the files need to be
> located together.

Yes.  To be clear in this case if we have  in rtems.git and 
 in rtems-central.git where you need to edit  to 
generate  then both the tool *and* somefile.yml need to be in 
rtems.git.

If what you are saying is the requirement is reversed.  rtems-central.git needs 
rtems.git to work that's fine and the order it should be.


> The specification maintenance is a separate responsibility to the function and
> role of rtems.git and related repos. The bar to make changes in rtems.git is 
> at
> the source level because it is the norm for open source projects and those
> working with the code. The separation also helps make it clear how funding of
> the qual related work happens. If we bring those specifications items down 
> into
> rtems.git the responsibility moves to that repo and that raises the bar on 
> being
> able to make a change. Our view on this may change as we learn now the 
> pre-qual
> effort effects what we do.

I think the important part here is it's evolving.  Change happens sometimes you 
do one thing one way to discover you really want or need to do it another.


> We have YAML files to run our build system and it is awesome. It also allows
> automation so I think it makes sense to provide an interface to read and write
> these files to aid automation rather than a disjointed set of personal 
> scripts,
> tools etc that may break or clash.

Agreed on this.


> The FreeBSD single repo is about the kernel and base runtime. The ports are 
> not
> part of this so the analogy breaks down. Bringing the RSB, RTEMS tools and
> documentation into a single repo creates new complexities I am not sure we are
> prepared enough to handle those. Maybe patch review and CI would help here and
> how they effect a single repo is something I have not given much 
> consideration too.

In theory I'm not against bringing everything together into one repository it 
allows for a lot of inter connectivity we can't do right now.  Regardless even 
if we had been doing it this way I would still advocate for the change being in 
the /rtems/ directory versus /rtems-central/


Amar.
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] build: Update PyYAML to 5.4.1

[Gedare Bloom]

On Thu, Jan 19, 2023 at 5:23 PM Chris Johns  wrote:
>
> On 20/1/2023 2:19 am, Sebastian Huber wrote:
> > The latest version is 6.0 which dropped support for Python 2.7.
>
> Was 6 our last version to support python 2 and 3?
>
> Is making this change this close to a release wise?
>
I sense confusion here. Are we referring to Pyyaml 6.0, or RTEMS 6.0?

I understand the comment above to mean we can't update to pyyaml 6.0
yet, because it doesn't support Python 2.7 which we still require for
the user-facing tools.

Gedare

> Chris
> ___
> devel mailing list
> devel@rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] build: Update PyYAML to 5.4.1

[Chris Johns]

On 20/1/2023 1:51 pm, Gedare Bloom wrote:
> On Thu, Jan 19, 2023 at 5:23 PM Chris Johns  wrote:
>>
>> On 20/1/2023 2:19 am, Sebastian Huber wrote:
>>> The latest version is 6.0 which dropped support for Python 2.7.
>>
>> Was 6 our last version to support python 2 and 3?
>>
>> Is making this change this close to a release wise?
>>
> I sense confusion here. Are we referring to Pyyaml 6.0, or RTEMS 6.0?

I assumed it referenced RTEMS 6.

> I understand the comment above to mean we can't update to pyyaml 6.0
> yet, because it doesn't support Python 2.7 which we still require for
> the user-facing tools.

Thanks, that confirms my understanding. So this means it is not OK to apply.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 1/3] build: Format build items

[Chris Johns]

On 20/1/2023 11:22 am, Karel Gardas wrote:
> 
> Sorry to hijack that thread, but correction is needed here.
> 
> On 1/20/23 01:03, Chris Johns wrote:
>> The FreeBSD single repo is about the kernel and base runtime. The ports are 
>> not
>> part of this so the analogy breaks down.
> 
> Certainly all BSDs have separated ports repos, but AFAIK all of them maintain 
> so
> called base and in base there is kernel, libs, bin/sbin commands and most
> importantly *all* tool-chain required to build *all* this code to form whole 
> OS.

Yes the ports place executables in /usr/local. The kernel and base is considered
a working set so the user commands under /bin /usr/bin /sbin etc match the
kernel and it is tested as a whole.

We will never have a single repo of all sources such as gcc etc like FreeBSD 
has.

> IMHO this rebuild whole OS distro is discussed here, isn't it? If not, I'm 
> sorry
> for noise here...

Ah ok, the discussion is about the how we manage over a long period of time
scripts, tools etc that read and write the spec files in rtems.git. This patch
adds a format option to aid automation via scripting however there is no script
support included so is that left as an exercise for those in the know on how to
do it or should we require support being brought into rtems.git and maintained
along with the file format? If there is no provided scripting support who does
this patch serve or benefit and what value does it bring other than overheads in
maintaining these spec files? [1]

The spec files are close or the same as the YAML spec files rtems-central has
and that repo has scripting support so those working with rtems-central are ok
because that single repo has consistent support. The history and agreement for
the qual effort is these repos remain separated and nothing has changed here.
The rtems.git repo will always take precedence over rtems-central.git.

In relation to a single RTEMS repo it helps make deployment easier for some use
cases and for others it makes no difference. It all depends on how you want to
tool up for deployment but it is not essential. Many repos or a single repo does
not change what is in a release and that is where the project's efforts are 
focused.

Chris

[1] the patch in this thread was pushed while I am still questioning it (hmm)
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 1/3] build: Format build items

[Chris Johns]

On 20/1/2023 11:53 am, Amar Takhar wrote:
> On 2023-01-20 11:03 +1100, Chris Johns wrote:
> 
>> I have been OK with the headers and tests being generated this way because 
>> the
>> agreement is files in rtems.git can be manually edited and rtems-central has 
>> to
>> track those changes. The sources and code to generate the files need to be
>> located together.
> 
> Yes.  To be clear in this case if we have  in rtems.git and 
>  in rtems-central.git where you need to edit  to 
> generate  then both the tool *and* somefile.yml need to be in 
> rtems.git.

Placing the API header spec files into rtems.git complicates being able to
update or add a new API by editing the header files. With the spec files to
generate the headers in the same repo they would need to be updated and that
means learning about the whole qual workflow, spec formats etc. This is why the
separation is done this way. As things are the rtems-central could be left as is
and rtems.git is not effected. We just continue to maintain the headers
manually. Those interested in qualifications can fund support for rtems-central
independently of the other parts of RTEMS.

> If what you are saying is the requirement is reversed.  rtems-central.git 
> needs 
> rtems.git to work that's fine and the order it should be.

The rtems-central specs files are cleverly put together so the headers,
documentation and other pieces are all handled from one place. This means all
pieces would need to be brought together. The rtems-central repo is the central
clearing house for the specifications.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] build: Update PyYAML to 5.4.1

[Chris Johns]

On 20/1/2023 1:59 pm, Chris Johns wrote:
> On 20/1/2023 1:51 pm, Gedare Bloom wrote:
>> On Thu, Jan 19, 2023 at 5:23 PM Chris Johns  wrote:
>>>
>>> On 20/1/2023 2:19 am, Sebastian Huber wrote:
 The latest version is 6.0 which dropped support for Python 2.7.
>>>
>>> Was 6 our last version to support python 2 and 3?
>>>
>>> Is making this change this close to a release wise?
>>>
>> I sense confusion here. Are we referring to Pyyaml 6.0, or RTEMS 6.0?
> 
> I assumed it referenced RTEMS 6.
> 
>> I understand the comment above to mean we can't update to pyyaml 6.0
>> yet, because it doesn't support Python 2.7 which we still require for
>> the user-facing tools.
> 
> Thanks, that confirms my understanding. So this means it is not OK to apply.
> 

Oh I see and yes I have misunderstood the patch twice. Sorry about that.

We can drop user facing python 2 support after RTEMS 6 branches. :)

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCHES rtems, source-builder] Add GitHub Actions scripts

[Chris Johns]

On 20/1/2023 1:50 am, Christian MAUDERER wrote:
> Am 19.01.23 um 15:42 schrieb Gedare Bloom:
>> Nice. I would like some time to look at this and think about it a
>> little more. What would be the plan for removing this capability? Will
>> it leave any artifacts behind in the RTEMS github mirror?
> 
> As soon as we want to get rid of the scripts again (because we have found and
> implemented a proper CI/CD that we officially want to use and not only have 
> in a
> test phase), we can just remove the scripts with a new commit.
> 
> If we use both commits, we will have a bot that adds a comment to pull 
> requests,
> that patches should be sent to the mailing list as soon as they are tested. It
> will close pull requests after 30 days. That can be disabled again with 
> removing
> that action.
> 
> All artefacts can be removed by everyone with enough rights in the RTEMS 
> GitHub
> organization. That should be most maintainers.

What github account does all this happen on?

Chris

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCHES rtems, source-builder] Add GitHub Actions scripts

[Christian MAUDERER]

Am 20.01.23 um 06:21 schrieb Chris Johns:

On 20/1/2023 1:50 am, Christian MAUDERER wrote:

Am 19.01.23 um 15:42 schrieb Gedare Bloom:

Nice. I would like some time to look at this and think about it a
little more. What would be the plan for removing this capability? Will
it leave any artifacts behind in the RTEMS github mirror?


As soon as we want to get rid of the scripts again (because we have found and
implemented a proper CI/CD that we officially want to use and not only have in a
test phase), we can just remove the scripts with a new commit.

If we use both commits, we will have a bot that adds a comment to pull requests,
that patches should be sent to the mailing list as soon as they are tested. It
will close pull requests after 30 days. That can be disabled again with removing
that action.

All artefacts can be removed by everyone with enough rights in the RTEMS GitHub
organization. That should be most maintainers.


What github account does all this happen on?

Chris


Hello Chris,

at the moment, I have a prototype at an embedded-brains account (see 
links in my first mail). My suggestion is to add the scripts to the 
official RTEMS GitHub mirror repositories to get some feedback from 
users and developers what is good or bad in this prototype CI system. 
Based on this we should discuss in one or two months what features we 
need and select and implement a long-term CI system for RTEMS.


Best regards

Christian
--

embedded brains GmbH
Herr Christian MAUDERER
Dornierstr. 4
82178 Puchheim
Germany
email:  christian.maude...@embedded-brains.de
phone:  +49-89-18 94 741 - 18
mobile: +49-176-152 206 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] build: Update PyYAML to 5.4.1

[Sebastian Huber]

On 20.01.23 03:59, Chris Johns wrote:

On 20/1/2023 1:51 pm, Gedare Bloom wrote:

On Thu, Jan 19, 2023 at 5:23 PM Chris Johns  wrote:

On 20/1/2023 2:19 am, Sebastian Huber wrote:

The latest version is 6.0 which dropped support for Python 2.7.

Was 6 our last version to support python 2 and 3?

Is making this change this close to a release wise?


I sense confusion here. Are we referring to Pyyaml 6.0, or RTEMS 6.0?

I assumed it referenced RTEMS 6.


I understand the comment above to mean we can't update to pyyaml 6.0
yet, because it doesn't support Python 2.7 which we still require for
the user-facing tools.

Thanks, that confirms my understanding. So this means it is not OK to apply.


What is the problem with updating PyYAML to 5.4.1? This version still 
supports Python 2.7.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 1/3] build: Format build items

[Sebastian Huber]

On 17.01.23 22:52, Chris Johns wrote:

On 17/1/2023 6:39 pm, Sebastian Huber wrote:

On 17.01.23 03:48, Chris Johns wrote:

On 16/1/2023 6:56 pm, Sebastian Huber wrote:

On 16.01.23 01:35, Chris Johns wrote:

On 13/1/2023 1:54 am, Sebastian Huber wrote:

On 12.01.23 15:44, Kinsey Moore wrote:

The other two patches look fine to me. The use of dump() that results in this
patch does several things:
* Removal of whitespace
This is fine for whitespace at the base level of indentation. Whitespace
within an indented block may be more important for readability.

* Removal of comments
This is not good as they are exclusively used to annotate manually ordered
blocks of test result expectations

* Rearrangement of items in alphabetical order
In general, rearrangement of top-level sections is good. For indented
sections
specifically in tst*.yml, this is bad for the above reaso

One goal of the new build system was to be able to alter the data through
scripts. This requires that the build items are human and machine readable and
writable. The Python YAML import/export does not preserve white space and
comments.

Can someone edit the file and add a hex number?

Yes, you can manually use whatever format is understood by the YAML loader. When
you write the file with the YAML dumper, then the standard representation is
used.

Are there python modules in rtems.git someone could import that reads and writes
the YAML spec files? If not should we provide a module to do this? It could be
`spec` so a user can use `import spec` after setting the import path.

That is, if external scripts (and I encourage this) all used a common read and
write type functionality the format consistency is relative to specific version
of rtems.git being used. Updates become read then write.

The Python modules to work with specification items are in rtems-central.git.
This repository contains also a format specification of the build items.

And how does that help here with this repo? I suggest you reconsider the
accessibility of those modules before pushing scripted generation changes like
this into rtems.git.


It was not my idea to use multiple repositories. Do you prefer now to 
move everything back into rtems.git (tools, documentation)?


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 2/3] build: Replace variant patterns with a list

[Sebastian Huber]

On 20.01.23 00:36, Chris Johns wrote:

So let me turn this around ... I am do not understand your resistance to there
being importable modules of python in rtems.git to read and write these files in
rtems.git?


These modules are in rtems-central.git (which has rtems.git as a 
submodule) together with the format specification. How can I use them in 
rtems.git?


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

libbsd updates

[Christian MAUDERER]

Hello,

recently an internal discussion about updates in the libbsd started. All 
who participated in the discussion agreed that we should move the 
discussion to a public one. Below you can find the mail thread. It's 
oldest mail first. Mails are split with lines of = signs. I removed some 
duplicated mail text in case there were no inline comments. The 
date/time of the mails are in my current locale which is UTC+1.


Best regards

Christian

=
On 2023-01-03 16:52, Thomas DOERFLER wrote:

Hello,

first of all I wish you all a healthy, happy and successful new year.

Unfortunately I already have to re-raise an issue: One of our customers
has asked us about the RTEMS libbsd update policy. Since we still use a
rather old libbsd version, it contains "OpenSSL 1.1.1d-freebsd  10 Sep
2019", there are many openSSL fixes missing when using RTEMS, and this
most likely is true for other parts of libbsd.

IMHO we should find a way to overcome the current libbsd update blocking
points and try to stay close to the FreeBSD code (maybe in a 1-3 month
interval).

AFAIK the big blocking point are the different views around changes of
the file descriptor handling between RTEMS and BSD (this may be a bit
off the real topic, I am not yet into the details). In the next week I
would like to get things rolling to see the pros and cons of both
possible implementations and I would be happy if those closely involved
would support this.

Apart from the current blocking point, can we agree that staying close
the the FreeBSD code (with a 1-3 month update/sync interval) would be
desirable?

Kind regards to you all,

Thomas.



=
On 2023-01-12 01:24, Gedare Bloom wrote:

Hi Thomas,

I think the goal you have stated is laudable and fits with the initial
design goals of libbsd. I will take a closer look at this topic and
report back soon, I hope. I have remained (purposefully) ignorant
about libbsd evolution over time, but I guess it is time for me to
learn a bit more and catch up.

 From what I have understood about the current blocking issue, it has
to do with the changes that were made to use FreeBSD File Descriptors
and the VFS. However, one of the main problems that was noted actually
appears to be just related to the size increase caused by the syscall
glue implemented in a single .c file. So, I will plan to start my
libbsd learning adventure by trying to split that .c file apart into
separate build components to see how/if that alleviates the linked
image size. If successful, that should get us closer.

I think we can then revisit whether or not the FreeBSD File
Descriptors themselves are in fact problematic.

It would also be helpful to decide if we want to clarify any
requirements for libbsd maintenance. So, for example, the ability to
keep it updated on a rolling basis would require good automation and
validation that changes/updates remain usable. It would appear that
there are some minimum resource requirements that some users of libbsd
would like to remain under if possible. To ensure that would require
establishing what platform(s) have resource requirements for using
libbsd, and adding configurations/testing to check for when those
resources are exceeded by an update. Because it is entirely plausible
that an update pulls in changes from upstream that causes the
resources to grow.

Gedare


=
On 2023-01-12 08:46, Sebastian Huber wrote:

Hello Gedare,

On 12.01.23 01:24, Gedare Bloom wrote:

Hi Thomas,

I think the goal you have stated is laudable and fits with the initial
design goals of libbsd. I will take a closer look at this topic and
report back soon, I hope. I have remained (purposefully) ignorant
about libbsd evolution over time, but I guess it is time for me to
learn a bit more and catch up.


yes, one of the goals of libbsd was to follow the FreeBSD upstream,
however, I was the only one doing this work in the past and I was busy
with other things in the last three years (mainly the RTEMS
pre-qualification). We need a more automated way to keep libbsd up to
date. My proposal is to focus on the master branch since the FreeBSD EOL
is 31. December 2023. We could use FreeBSD 14 for the next production
branch. The current libbsd synchronization support was designed to do
updates from Subversion. With Git we can do things more efficiently. I
would replace the libbsd build system with the one from RTEMS (waf +
spec items). With the spec items we can more easily read the data in
other scripts. For continuous and automatic updates I would do the
following:

1. Check out the FreeBSD X branch.

2. Move all FreeBSD top-level directories under the new "freebsd"
directory and merge in all updates from FreeBSD.

3. Gather the set of files imported from FreeBSD. Name the set F.

4. Pick a start FreeBSD commit C.

5. Make a patch f