Re: Update to GDB 11.2?

[Anders Montonen]

Hi,

> On 25 Feb 2022, at 7:53, Sebastian Huber  
> wrote:
> 
> On 24/02/2022 14:53, Sebastian Huber wrote:
>> On 24/02/2022 14:40, Joel Sherrill wrote:
>>> On Thu, Feb 24, 2022, 5:50 AM Sebastian Huber 
>>> >> > wrote:
>>> 
>>> Hello,
>>> 
>>> we still use GDB 10 for the RTEMS 6 tool suite. I noticed some build
>>> problems for this version. Should we update to the latest GDB 11.2
>>> release?
>>> 
>>> 
>>> If it solves build problems and doesn't introduce others, sure. I tend to 
>>> want to bump to the latest  versions.
>> Ok, I started a build on Linux using GDB 11.2. If this works, I will update 
>> the RSB.
> 
> It didn't help to fix the build issues on macOS. Now all GDB builds seem to 
> fail with:
> 
> checking whether to use MPFR... auto
> checking for libmpfr... no
> configure: WARNING: MPFR is missing or unusable; some features may be 
> unavailable.
> checking whether to use python... /usr/bin/python2
> checking for python... no
> configure: error: no usable python found at /usr/bin/python2

You may be running into this long-standing issue with Apple’s Python 2 
framework builds: https://bugs.python.org/issue3588
See also this thread: 
https://lists.rtems.org/pipermail/users/2020-November/067962.html

-a
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH rtems-docs v2] Update package list for Ubuntu

[Anders Montonen]

Hi,

> On 20 Dec 2021, at 17:22, Ryan Long  wrote:
> 
> Updated the latest version referenced of Ubuntu to 21.04, updated
> package list, and identified what packages need to still be installed
> afterwards to pass sb-check.

This should be 20.04, to match the patch

> -The latest version is Ubuntu 18.04.1 LTS 64-bit. This section also includes
> +The latest version is Ubuntu 20.04.3 LTS 64-bit. This section also includes

Regards,
Anders Montonen

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Build fail on debian - Toolset build error

[Anders Montonen]

Hi,

> On 12 Oct 2020, at 15:30, Karel Gardas  wrote:
> 
> 
> Sure, but you have to install header files whatever this means on
> macosx. On Ubuntu this means 'apt install python2.7-dev’

System frameworks on macOs usually include the development libraries and 
headers. Running “python-config —cflags”, and “python-config 
—ldflags” returns the compiler and linker flags needed.

Regards,
Anders Montonen___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 0/2] Source Builder: Fix UnicodeDecodeError build hang

[Anders Montonen]

On 8 May 2020, at 9:12, Chris Johns  wrote:
> On 8/5/20 3:50 pm, Anders Montonen wrote:
> 
>> On a not completely unrelated note, have there been any thoughts about 
>> setting a timetable for deprecating Python 2 support now that it is EOL, eg. 
>> in RTEMS6? Are there any specific host platforms you want to support that 
>> don’t have Python 3?
> We cannot just yet because we have users who run long life operating systems 
> and cannot move. This is one of the roles RTEMS provides. We maintain tools 
> and the OS longer than others and on older target hardware. Large orgs cannot 
> retool just because it exists.

Right, which is why you set a timetable to give sufficient advance warning, and 
coincide it with a major release. Though I would have assumed that these kinds 
of users don’t really switch to newer releases anyway.

Regards,
Anders___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 0/2] Source Builder: Fix UnicodeDecodeError build hang

[Anders Montonen]

> On 8 May 2020, at 7:36, Chris Johns  wrote:
> 
> On 8/5/20 2:27 pm, Anders Montonen wrote:
>> Hi,
>>> On 7 May 2020, at 5:10, Chris Johns  wrote:
>>> 
>>> On 7/5/20 9:37 am, Anders Montonen wrote:
>>>> This patch fixes the UnicodeDecodeError that randomly caused source
>>>> builder to hang.
>>>> 
>>>> In my testing, there was no difference in performance at least under
>>>> Python 3.
>>> 
>>> Have you tested python 2?
>> Unfortunately the patch caused the log.py module to trigger 
>> UnicodeEncodeErrors under Python 2, so consider it withdrawn.
> 
> Oh ok. Is it worth looking at fixing log.py? Our fix seems to me to important.

I’m looking into it, I just wanted to give a heads-up.

On a not completely unrelated note, have there been any thoughts about setting 
a timetable for deprecating Python 2 support now that it is EOL, eg. in RTEMS6? 
Are there any specific host platforms you want to support that don’t have 
Python 3?

Regards,
Anders
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 0/2] Source Builder: Fix UnicodeDecodeError build hang

[Anders Montonen]

Hi,

> On 7 May 2020, at 5:10, Chris Johns  wrote:
> 
> On 7/5/20 9:37 am, Anders Montonen wrote:
>> This patch fixes the UnicodeDecodeError that randomly caused source
>> builder to hang.
>> 
>> In my testing, there was no difference in performance at least under
>> Python 3.
> 
> Have you tested python 2?

Unfortunately the patch caused the log.py module to trigger UnicodeEncodeErrors 
under Python 2, so consider it withdrawn.

Regards,
Anders
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH 0/2] Source Builder: Fix UnicodeDecodeError build hang

[Anders Montonen]

Hi,

> On 7 May 2020, at 5:10, Chris Johns  wrote:
> 
> On 7/5/20 9:37 am, Anders Montonen wrote:
>> This patch fixes the UnicodeDecodeError that randomly caused source
>> builder to hang.
>> 
>> In my testing, there was no difference in performance at least under
>> Python 3.
> 
> Have you tested python 2?

Not beyond verifying that the tests in the file pass. I have deliberately not 
installed Python 2 on the systems I’ve been testing on, but I suppose I could 
spin up a sacrificial VM.

> I would like to explore this change on master after we branch for 5 and if the
> testing on master proves stable we can look at merging it on the 5 branch for
> 5.2. Is this approach OK with you?

Of course.

Regards,
Anders
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH 0/2] Source Builder: Fix UnicodeDecodeError build hang

[Anders Montonen]

This patch fixes the UnicodeDecodeError that randomly caused source
builder to hang.

In my testing, there was no difference in performance at least under
Python 3.

Anders Montonen (2):
  sb/execute.py: Open subprocess I/O handles in text mode
  sb/execute.py: Add test for _readthread() code point splitting

 source-builder/sb/execute.py | 27 ---
 1 file changed, 8 insertions(+), 19 deletions(-)

-- 
2.25.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH 1/2] sb/execute.py: Open subprocess I/O handles in text mode

[Anders Montonen]

With a UTF-8 locale, _readthread() would occasionally split a multibyte
code point. When running under Python 3, this would trigger an
UnicodeDecodeError, which resulted in the build hanging. Solve this
issue by opening the subprocess streams in text mode, and let Python's
runtime deal with the nitty-gritty of returning complete code points.

Fixes:
Exception in thread _stderr[]:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
  File "/home/anders/work/rtems/rsb/source-builder/sb/execute.py", line 204, in 
_readthread
data = data.decode(sys.stdout.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4094-4095: 
unexpected end of data

Signed-off-by: Anders Montonen 
---
 source-builder/sb/execute.py | 24 ++--
 1 file changed, 6 insertions(+), 18 deletions(-)

diff --git a/source-builder/sb/execute.py b/source-builder/sb/execute.py
index 06f9b7d..092267b 100755
--- a/source-builder/sb/execute.py
+++ b/source-builder/sb/execute.py
@@ -123,11 +123,6 @@ class execute(object):
 is a timeout check."""
 if trace_threads:
 print('execute:_writethread: start')
-encoding = True
-try:
-tmp = bytes('temp', sys.stdin.encoding)
-except:
-encoding = False
 input_types = [str, bytes]
 try:
 # Unicode is not valid in python3, not added to the list
@@ -143,8 +138,6 @@ class execute(object):
 print('execute:_writethread: input returned:', 
type(lines))
 if type(lines) in input_types:
 try:
-if encoding:
-lines = bytes(lines, sys.stdin.encoding)
 fh.write(lines)
 fh.flush()
 except:
@@ -192,16 +185,13 @@ class execute(object):
 # and the process is shutting down.
 #
 try:
-data = fh.read1(4096)
+data = fh.read(4096)
 except:
 data = ''
 if len(data) == 0:
 if len(line) > 0:
 _output_line(line + '\n', exe, prefix, out, count)
 break
-# str and bytes are the same type in Python2
-if type(data) is not str and type(data) is bytes:
-data = data.decode(sys.stdout.encoding)
 last_ch = data[-1]
 sd = (line + data).split('\n')
 if last_ch != '\n':
@@ -267,7 +257,7 @@ class execute(object):
  name = '_stdout[%s]' % (name),
  args = (self,
  
io.open(proc.stdout.fileno(),
- mode = 'rb',
+ mode = 'r',
  closefd = False),
  self.output,
  ''))
@@ -278,7 +268,7 @@ class execute(object):
  name = '_stderr[%s]' % (name),
  args = (self,
  
io.open(proc.stderr.fileno(),
- mode = 'rb',
+ mode = 'r',
  closefd = False),
  self.output,
  self.error_prefix))
@@ -381,7 +371,8 @@ class execute(object):
 cwd = cwd, env = env,
 stdin = stdin, stdout = stdout,
 stderr = stderr,
-close_fds = False)
+close_fds = False,
+universal_newlines=True)
 if not capture:
 return (0, proc)
 if self.output is None:
@@ -563,10 +554,7 @@ if __name__ == "__main__":
 if ec == 0:
 print('piping input into ' + commands['pipe'][0] + ': ' + \
   commands['pipe'][2])
-try:
-out = bytes(commands['pipe'][2], sys.stdin.encoding)
-   

[PATCH 2/2] sb/execute.py: Add test for _readthread() code point splitting

[Anders Montonen]

Signed-off-by: Anders Montonen 
---
 source-builder/sb/execute.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/source-builder/sb/execute.py b/source-builder/sb/execute.py
index 092267b..fc379ae 100755
--- a/source-builder/sb/execute.py
+++ b/source-builder/sb/execute.py
@@ -565,6 +565,7 @@ if __name__ == "__main__":
 
 cmd_shell_test = 'if "%OS%" == "Windows_NT" (echo It is WinNT) else echo 
Is is not WinNT'
 sh_shell_test = 'x="me"; if [ $x = "me" ]; then echo "It was me"; else "It 
was him"; fi'
+long_output_test = 'echo ' + 4095*'a' + u'\U0001'
 
 commands = {}
 commands['windows'] = {}
@@ -575,7 +576,7 @@ if __name__ == "__main__":
 commands['windows']['csubsts'] = [('netstat %0', ['-a']),
   ('netstat %0 %1', ['-a', '-n'])]
 commands['windows']['pipe'] = ('ftp', None, 'help\nquit')
-commands['unix']['shell'] = ['pwd', 'ls -las', './xyz', sh_shell_test]
+commands['unix']['shell'] = ['pwd', 'ls -las', './xyz', sh_shell_test, 
long_output_test]
 commands['unix']['spawn'] = ['ls', 'execute.pyc', ['ls', '-i']]
 commands['unix']['cmd'] = [('date'), ('date', '-R'), ('date', ['-u', '+%d 
%D']),
('date', '-u "+%d %D %S"')]
-- 
2.25.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: m2005 untar error on msys2 (tar01 directory)

[Anders Montonen]

Hi,

> On 5 May 2020, at 8:44, Sebastian Huber  
> wrote:
> On 05/05/2020 07:41, Chris Johns wrote:
> 
>> On 5/5/20 3:34 pm, Sebastian Huber wrote:
>> 
>> What does `os.uname()` return?
> 
> In the msys shell:
> 
> $ python
> Python 3.7.4 (default, Jul 11 2019, 09:35:14)
> [GCC 9.1.0] on msys
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import os
> >>> os.uname()
> posix.uname_result(sysname='MSYS_NT-6.1-7601', nodename='Blub', 
> release='3.0.7-338.x86_64', version='2019-07-11 10:58 UTC', machine='x86_64')
> 
> In the mingw64 shell:
> 
> $ python
> Python 3.8.2 (default, Feb 27 2020, 05:27:33)  [GCC 9.2.0 64 bit (AMD64)] on 
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import os
> >>> os.uname()
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: module 'os' has no attribute ‘uname'

Python’s own documentation only says “recent flavors of Unix” under 
availability for the os.uname() method. platform.uname() or sys,platform might 
be more portable?


Regards,
Anders___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Build Linux: FAILED devel/qemu4 on x86_64-linux-gnu (qemu-v4.1.0-x86_64-linux-gnu-1)

[Anders Montonen]

On 3 May 2020, at 8:37, Joel Sherrill  wrote:
> 
> This appears to fail from the m2005 release snapshot but not when building 
> git master. This is on CentOS 7.
> 
> Hopefully there is something useful in what was mailed out. If not, I'll dig 
> into the saved stuff on the machine tomorrow.

> download: 
> https://ftp.rtems.org/pub/rtems/releases/5/5.0.0/5.0.0-m2005-1/sources/qemu-git-42d58e7.tar.xz
>  
> :
>  error: HTTP Error 404: Not Found

This looks to be the source of the failure. The file name in 
bare/config/devel/qemu4-git-1.cfg doesn’t match what’s on the FTP. Looking at 
, the 
correct file name should be qemu-git-42d58e7c67.tar.xz.

Looking in the log, there were a couple of other 404s:

> download: 
> https://ftp.rtems.org/pub/rtems/releases/4.12/4.12/sources/qemu-git-42d58e7.tar.xz
>  
> :
>  error: HTTP Error 404: Not Found

I guess this is used as a fallback location, but there is no release 4.12 
directory: 

> download: 
> https://ftp.rtems.org/pub/rtems/releases/5/5.0.0/5.0.0-m2005-1/sources/glib-2.46.2.tar.xz
>  
> :
>  error: HTTP Error 404: Not Found
> download: 
> https://ftp.rtems.org/pub/rtems/releases/5/5.0.0/5.0.0-m2005-1/sources/glib-2.46.2-werror.patch
>  
> :
>  error: HTTP Error 404: Not Found

bare/config/devel/qemu4.bset specifies glib-2.46.2, but the 5.0.0 snapshot 
contains glib-2.48.2.tar.xz. The older version and the patch are eventually 
found from upstream locations.

Regards,
Anders___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH 2/3] sb/linux.py: Workaround for Python >= 3.8

[Anders Montonen]

The platform.dist() interface has been deprecated since Python 3.5, and
has been removed in Python 3.8.

Fixes:
Traceback (most recent call last):
  File "/home/anders/work/rtems/rsb/source-builder/sb/options.py", line 682, in 
load
overrides = linux.load()
  File "/home/anders/work/rtems/rsb/source-builder/sb/linux.py", line 60, in 
load
distro = platform.dist()[0]
AttributeError: module 'platform' has no attribute 'dist'

Signed-off-by: Anders Montonen 
---
 source-builder/sb/linux.py | 18 +++---
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/source-builder/sb/linux.py b/source-builder/sb/linux.py
index c3059a9..791bbd6 100644
--- a/source-builder/sb/linux.py
+++ b/source-builder/sb/linux.py
@@ -55,13 +55,17 @@ def load():
 '__tar':('exe', 'required', '/bin/tar')
 }
 
-# Works for LSB distros
-try:
-distro = platform.dist()[0]
-distro_ver = float(platform.dist()[1])
-except ValueError:
-# Non LSB distro found, use failover"
-pass
+# platform.dist() was removed in Python 3.8
+if hasattr(platform, 'dist'):
+# Works for LSB distros
+try:
+distro = platform.dist()[0]
+distro_ver = float(platform.dist()[1])
+except ValueError:
+ # Non LSB distro found, use failover"
+ pass
+else:
+ distro = ''
 
 # Non LSB - fail over to issue
 if distro == '':
-- 
2.25.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH 3/3] source-builder/config/gdb-common-1.cfg: Fix building with Python 3.8

[Anders Montonen]

Starting with Python 3.8, python3-config requires the "--embed" flag to
print the flags required for embedding Python in an application.

Fixes:
error: shell macro failed: 
/home/anders/work/rtems/rsb/source-builder/sb/rtems-build-dep -c gcc  -l : 2: 
error: no library (-l) provided

Signed-off-by: Anders Montonen 
---
 source-builder/config/gdb-common-1.cfg | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/source-builder/config/gdb-common-1.cfg 
b/source-builder/config/gdb-common-1.cfg
index 5847f11..397d44d 100644
--- a/source-builder/config/gdb-common-1.cfg
+++ b/source-builder/config/gdb-common-1.cfg
@@ -109,7 +109,12 @@
   %endif
   %if %{gdb-python-config} != %{nil}
 %define gdb-python-lib-filter awk 'BEGIN{FS=" 
"}/python/{for(i=1;ihttp://lists.rtems.org/mailman/listinfo/devel

[PATCH 0/3] Source Builder: Fixes for Python 3.8

[Anders Montonen]

These patches fix some issues with Python 3.8 compatibility, which I
encountered when trying out the new Ubuntu 20.04 release. There's also
fixes for a number of syntax warnings which are flagged even with older
Python versions.

The patches should not break compatibility with older Python versions,
but I have not actually tested.

I'm not sure if the Linux distro check fix is the most elegant, but I
think the whole thing should be changed to check if the required
executables are found in the runtime path, instead of assuming some
hard-coded locations.

Anders Montonen (3):
  sb: Fix Python literal comparison warnings
  sb/linux.py: Workaround for Python >= 3.8
  source-builder/config/gdb-common-1.cfg: Fix building with Python 3.8

 source-builder/config/gdb-common-1.cfg |  7 +++-
 source-builder/sb/config.py|  2 +-
 source-builder/sb/linux.py | 18 
 source-builder/sb/macros.py| 58 +-
 4 files changed, 47 insertions(+), 38 deletions(-)

-- 
2.25.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH 1/3] sb: Fix Python literal comparison warnings

[Anders Montonen]

Fixes:
SyntaxWarning: "is" with a literal. Did you mean "=="?

Signed-off-by: Anders Montonen 
---
 source-builder/sb/config.py |  2 +-
 source-builder/sb/macros.py | 58 ++---
 2 files changed, 30 insertions(+), 30 deletions(-)

diff --git a/source-builder/sb/config.py b/source-builder/sb/config.py
index 4856088..afcac84 100644
--- a/source-builder/sb/config.py
+++ b/source-builder/sb/config.py
@@ -338,7 +338,7 @@ class file:
 raise error.general(self._name_line_msg(msg))
 
 def _label(self, name):
-if name.startswith('%{') and name[-1] is '}':
+if name.startswith('%{') and name[-1] == '}':
 return name
 return '%{' + name.lower() + '}'
 
diff --git a/source-builder/sb/macros.py b/source-builder/sb/macros.py
index 9ba6cac..5a639b8 100644
--- a/source-builder/sb/macros.py
+++ b/source-builder/sb/macros.py
@@ -233,7 +233,7 @@ class macros:
 return [rm[5:] for rm in self.read_maps]
 
 def key_filter(self, key):
-if key.startswith('%{') and key[-1] is '}':
+if key.startswith('%{') and key[-1] == '}':
 key = key[2:-1]
 return key.lower()
 
@@ -269,28 +269,28 @@ class macros:
 print(' c:%s(%d) s:%s t:"%s" m:%r M:%s' % \
 (c, ord(c), state, token, macro, map))
 l_remaining = l_remaining[1:]
-if c is '#' and not state.startswith('value'):
+if c == '#' and not state.startswith('value'):
 break
 if c == '\n' or c == '\r':
-if not (state is 'key' and len(token) == 0) and \
+if not (state == 'key' and len(token) == 0) and \
 not state.startswith('value-multiline'):
 raise error.general('malformed macro line:%d: %s' % 
(lc, l))
-if state is 'key':
+if state == 'key':
 if c not in string.whitespace:
-if c is '[':
+if c == '[':
 state = 'map'
-elif c is '%':
+elif c == '%':
 state = 'directive'
-elif c is ':':
+elif c == ':':
 macro += [token]
 token = ''
 state = 'attribs'
-elif c is '#':
+elif c == '#':
 break
 else:
 token += c
-elif state is 'map':
-if c is ']':
+elif state == 'map':
+if c == ']':
 if token not in macros:
 macros[token] = {}
 map = token
@@ -300,7 +300,7 @@ class macros:
 token += c
 else:
 raise error.general('invalid macro map:%d: %s' % (lc, 
l))
-elif state is 'directive':
+elif state == 'directive':
 if c in string.whitespace:
 if token == 'include':
 self.load(_clean(l_remaining))
@@ -311,7 +311,7 @@ class macros:
 token += c
 else:
 raise error.general('invalid macro directive:%d: %s' % 
(lc, l))
-elif state is 'include':
+elif state == 'include':
 if c is string.whitespace:
 if token == 'include':
 state = 'include'
@@ -319,49 +319,49 @@ class macros:
 token += c
 else:
 raise error.general('invalid macro directive:%d: %s' % 
(lc, l))
-elif state is 'attribs':
+elif state == 'attribs':
 if c not in string.whitespace:
-if c is ',':
+if c == ',':
 macro += [token]
 token = ''
 if len(macro) == 3:
 state = 'value-start'
 else:
 token += c
-elif state is 'value-start':
-if c is "'":
+elif state == 'value-start':
+if c == "'":
 state = 'value-line-start'
-elif state is 'value-line-start':
-if c is "'":
+elif state == 'value-line-start':
+if c == "'":
 state = 'value-multiline-start'
 else: