[PATCH] libdl: Add support to import base image TLS symbols

[chrisj]

From: Chris Johns 

This change requires an rtems-tools update for symbol generation.

Working architectures:
 - aarch64
 - arm
 - powerpc
 - sparc

Updates #4920
---
 cpukit/include/rtems/rtl/rtl-sym.h |  23 -
 cpukit/include/rtems/rtl/rtl.h |   6 +-
 cpukit/libdl/rtl-elf.c |  20 +++-
 cpukit/libdl/rtl-mdreloc-aarch64.c |   6 +-
 cpukit/libdl/rtl-mdreloc-arm.c |   3 +-
 cpukit/libdl/rtl-mdreloc-powerpc.c |  18 +++-
 cpukit/libdl/rtl-mdreloc-sparc.c   | 152 -
 cpukit/libdl/rtl-sym.c |  36 ++-
 cpukit/libdl/rtl-tls.c | 111 +
 cpukit/libdl/rtl-tls.h |  51 ++
 cpukit/libdl/rtl.c |   6 +-
 spec/build/cpukit/objdl.yml|   1 +
 testsuites/libtests/dl11/dl-load.c |  27 -
 13 files changed, 408 insertions(+), 52 deletions(-)
 create mode 100644 cpukit/libdl/rtl-tls.c
 create mode 100644 cpukit/libdl/rtl-tls.h

diff --git a/cpukit/include/rtems/rtl/rtl-sym.h 
b/cpukit/include/rtems/rtl/rtl-sym.h
index 0d29a6ae40..9eef9a1b7b 100644
--- a/cpukit/include/rtems/rtl/rtl-sym.h
+++ b/cpukit/include/rtems/rtl/rtl-sym.h
@@ -62,6 +62,22 @@ typedef struct rtems_rtl_symbols
   size_t   nbuckets;
 } rtems_rtl_symbols;
 
+/**
+ * A TLS variable offset call. There is one per base image TLS
+ * variable.
+ */
+typedef size_t (*rtems_tls_offset_func)(void);
+
+/**
+ * A TLS symbol offset entry. It is used with an exported symbol table
+ * to find a TSL table offset for a variable at runtime.
+ */
+typedef struct rtems_tls_offset
+{
+  size_tindex;  /** exported symbol table index */
+  rtems_tls_offset_func offset; /** TLS offset function */
+} rtems_tls_offset;
+
 /**
  * Open a symbol table with the specified number of buckets.
  *
@@ -101,10 +117,15 @@ void rtems_rtl_symbol_table_close (rtems_rtl_symbols* 
symbols);
  * @param obj The object table the symbols are for.
  * @param esyms The exported symbol table.
  * @param size The size of the table in bytes.
+ * @param tls_offsets The TLS offsets table. If NULL none provided.
+ * @param tls_size The number TLS offset entries in the table..
+
  */
 bool rtems_rtl_symbol_global_add (rtems_rtl_obj*   obj,
   const unsigned char* esyms,
-  unsigned int size);
+  unsigned int size,
+  rtems_tls_offset*tls_offsets,
+  unsigned int tls_size);
 
 /**
  * Find a symbol given the symbol label in the global symbol table.
diff --git a/cpukit/include/rtems/rtl/rtl.h b/cpukit/include/rtems/rtl/rtl.h
index 0fd4e74cdf..aaaeb7b592 100644
--- a/cpukit/include/rtems/rtl/rtl.h
+++ b/cpukit/include/rtems/rtl/rtl.h
@@ -393,9 +393,13 @@ bool rtems_rtl_path_prepend (const char* path);
  *
  * @param esyms The exported symbol table.
  * @param count The size of the exported symbol table.
+ * @param tls_offsets The TLS offsets table. If NULL none provided.
+ * @param tls_size The number TLS offset entries in the table..
  */
 void rtems_rtl_base_sym_global_add (const unsigned char* esyms,
-unsigned int count);
+unsigned int count,
+rtems_tls_offset*tls_offsets,
+unsigned int tls_size);
 
 /**
  * Return the object file descriptor for the base image. The object file
diff --git a/cpukit/libdl/rtl-elf.c b/cpukit/libdl/rtl-elf.c
index 5754070518..b46d2ac3a0 100644
--- a/cpukit/libdl/rtl-elf.c
+++ b/cpukit/libdl/rtl-elf.c
@@ -178,12 +178,19 @@ rtems_rtl_elf_find_symbol (rtems_rtl_obj*  obj,
 
   /*
* If the symbol type is STT_NOTYPE the symbol references a global
-   * symbol. The gobal symbol table is searched to find it and that value
+   * symbol. The global symbol table is searched to find it and that value
* returned. If the symbol is local to the object module the section for the
* symbol is located and it's base added to the symbol's value giving an
* absolute location.
+   *
+   * If the symbols type of TLS return the symbols value. It is the
+   * offset from the thread's TLS area base. The offset is set by the
+   * linker for the base image and by the TLS allocator for loaded
+   * modules. There is no section and no absolute base.
*/
-  if (ELF_ST_TYPE(sym->st_info) == STT_NOTYPE || sym->st_shndx == SHN_COMMON)
+  if (ELF_ST_TYPE (sym->st_info) == STT_NOTYPE ||
+  sym->st_shndx == SHN_COMMON ||
+  ELF_ST_TYPE (sym->st_info) == STT_TLS)
   {
 /*
  * Search the object file then the global table for the symbol.
@@ -246,6 +253,13 @@ rtems_rtl_elf_reloc_parser (rtems_rtl_obj*  obj,
   rtems_rtl_wordrel_words[3];
   rtems_rtl_elf_rel_status  rs;
 
+  /*
+   * TLS are not parsed.
+   */
+  if (ELF_ST_TYPE (sym->st_info) 

[rtems-tools PATCH] linker: Add TLS support to the symbol table generator

[chrisj]

From: Chris Johns 

This is not complete as it has aarch64 code that should be in the
kernel.
---
 linkers/rtems-syms.cpp | 179 ++---
 1 file changed, 134 insertions(+), 45 deletions(-)

diff --git a/linkers/rtems-syms.cpp b/linkers/rtems-syms.cpp
index e5170e1..1c54ffc 100644
--- a/linkers/rtems-syms.cpp
+++ b/linkers/rtems-syms.cpp
@@ -60,10 +60,22 @@ static const char* c_header[] =
   " *  Automatically generated. Do not edit..",
   " */",
   "",
+  "#include ",
+  "#include ",
+  "",
+  "extern void* rtems_rtl_tls_get_base (void);",
+  "",
   "extern const unsigned char rtems__rtl_base_globals[];",
   "extern const unsigned int rtems__rtl_base_globals_size[];",
   "",
-  "void rtems_rtl_base_sym_global_add (const unsigned char* , unsigned int );",
+  "typedef size_t (*rtems_tls_offset_func)(void);",
+  "typedef struct rtems_tls_offset {",
+  "  size_t index;",
+  "  rtems_tls_offset_func offset;",
+  "} rtems_tls_offset;",
+  "",
+  "void rtems_rtl_base_sym_global_add (const unsigned char* , unsigned int,",
+  "rtems_tls_offset*, size_t );",
   "",
   "asm(\".section \\\".rodata\\\"\");",
   "",
@@ -78,35 +90,60 @@ static const char* c_header[] =
   0
 };
 
-static const char* c_trailer[] =
+static const char* c_sym_table_end[] =
 {
   "asm(\"  .byte0\");",
   "asm(\"  .ascii   \\\"\\xde\\xad\\xbe\\xef\\\"\");",
-#if BROKEN_ON_SOME_ASSEMBLERS
-  "asm(\"  .typertems__rtl_base_globals, #object\");",
-  "asm(\"  .sizertems__rtl_base_globals, . - rtems__rtl_base_globals\");",
-#endif
   "",
+  0
+};
+
+static const char* c_tls_call_table_start[] =
+{
+  "rtems_tls_offset rtems_tls_offsets[] = {",
+  0
+};
+
+static const char* c_tls_call_table_end[] =
+{
+  "};",
+  "#define RTEMS_TLS_OFFSETS_NUM " \
+  "(sizeof(rtems_tls_offsets) / (sizeof(rtems_tls_offsets[0])))",
+  "",
+  0
+};
+
+static const char* c_trailer[] =
+{
   "/*",
   " * Symbol table size.",
   " */",
   "asm(\"  .align   4\");",
   "asm(\"  .local   rtems__rtl_base_globals_size\");",
-#if BROKEN_ON_SOME_ASSEMBLERS
-  "asm(\"  .typertems__rtl_base_globals_size, #object\");",
-  "asm(\"  .sizertems__rtl_base_globals_size, 4\");",
-#endif
   "asm(\"rtems__rtl_base_globals_size:\");",
   "asm(\"  .long rtems__rtl_base_globals_size - rtems__rtl_base_globals\");",
   "",
   0
 };
 
+static const char* c_rtl_call_body_embeded[] =
+{
+  "{",
+  "  rtems_rtl_base_sym_global_add (__rtl_base_globals[0],",
+  " rtems__rtl_base_globals_size[0],",
+  " _tls_offsets[0],",
+  " RTEMS_TLS_OFFSETS_NUM);",
+  "}",
+  0
+};
+
 static const char* c_rtl_call_body[] =
 {
   "{",
   "  rtems_rtl_base_sym_global_add (__rtl_base_globals[0],",
-  " rtems__rtl_base_globals_size[0]);",
+  " rtems__rtl_base_globals_size[0],",
+  " NULL",
+  " 0);",
   "}",
   0
 };
@@ -140,7 +177,7 @@ c_embedded_trailer (rld::process::tempfile& c)
 {
   c.write_line ("void rtems_rtl_base_global_syms_init(void);");
   c.write_line ("void rtems_rtl_base_global_syms_init(void)");
-  temporary_file_paint (c, c_rtl_call_body);
+  temporary_file_paint (c, c_rtl_call_body_embeded);
 }
 
 /**
@@ -214,16 +251,27 @@ symbol_filter::filter (const rld::symbols::symtab& 
symbols,
 
 struct output_sym
 {
+  enum struct output_mode {
+symbol,
+tls_func,
+tls_call_table
+  };
   rld::process::tempfile& c;
   const bool  embed;
   const bool  weak;
-
-  output_sym(rld::process::tempfile& c,
- boolembed,
- boolweak)
-: c (c),
-  embed (embed),
-  weak (weak) {
+  const output_mode   mode;
+  size_t& index;
+
+  output_sym(rld::process::tempfile& c_,
+ boolembed_,
+ boolweak_,
+ output_mode mode_,
+ size_t& index_)
+: c (c_),
+  embed (embed_),
+  weak (weak_),
+  mode (mode_),
+  index (index_) {
   }
 
   void operator ()(const rld::symbols::symtab::value_type& value);
@@ -240,34 +288,56 @@ output_sym::operator ()(const 
rld::symbols::symtab::value_type& value)
   if (weak && sym.value () == 0)
 return;
 
-  c.write_line ("asm(\"  .asciz \\\"" + sym.name () + "\\\"\");");
-
-  if (sym.type() == STT_TLS)
-  {
-c.write_line ("asm(\"  .type \\\"" + sym.name () + "\\\", 
%tls_object\");");
-  }
-
-  if (embed)
-  {
-c.write_line ("#if __SIZEOF_POINTER__ == 8");
-c.write_line ("asm(\"  .quad " + sym.name () + "\");");
-c.write_line ("#else");
-c.write_line ("asm(\"  .long " + sym.name () + "\");");
-c.write_line ("#endif");
-  }
-  else
-  {
-std::stringstream oss;
-oss << std::hex << std::setfill 

libdl support for base image TLS variables

[chrisj]

These patches add support to rtems.git and rtems-tools.git to 
provide access to TLS variables in the base from loaded code.
The changes contain the ground work to resolving #4920 which is 
a blocker for the release of 6.

This change is required due to the regression in libdl due to the
change in newlib to use TLS variables for reneterant data.

The rtems.git patch can be applied before the rtems-tools.git
patch. If the tools are updated and the libdl code is not present
the tests fail with linker errors. If the tools change is
not present the test will link but it will continue to fail.

The currently supported archs that build are:

 - aarch64
 - arm
 - powerpc
 - sparc

The follow are to be completed and do not build:

 - bfin
 - i386
 - lm32
 - m68k
 - mips
 - moxie
 - riscv
 - v850  

We need to add support for these architectures if they support TLS.

Chris


___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH] fixes 4942 Regulator warnings

[zack]

diff --git a/cpukit/libmisc/regulator/regulator.c 
b/cpukit/libmisc/regulator/regulator.c
index 2071c468fd..97a48be4f5 100644
--- a/cpukit/libmisc/regulator/regulator.c
+++ b/cpukit/libmisc/regulator/regulator.c
@@ -35,6 +35,7 @@
 
 #include 
 #include 
+#include 
 
 #include 
 
-- 
2.34.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH rtems-docs] user/testing: add testing using renode docs

[Muhammad Sulthan Mazaya]

Testing using renode docs. Consists of how to install it and how to add
new configuration to test BSP using renode.

---
 user/testing/index.rst  |  1 +
 user/testing/renode.rst | 84 +
 2 files changed, 85 insertions(+)
 create mode 100644 user/testing/renode.rst

diff --git a/user/testing/index.rst b/user/testing/index.rst
index 3d36fab..75e22a3 100644
--- a/user/testing/index.rst
+++ b/user/testing/index.rst
@@ -48,3 +48,4 @@ extension.
simulation
gdb-jtag
tftp
+   renode
diff --git a/user/testing/renode.rst b/user/testing/renode.rst
new file mode 100644
index 000..72b1946
--- /dev/null
+++ b/user/testing/renode.rst
@@ -0,0 +1,84 @@
+Testing Using Renode
+
+
+`Renode `_ is one of the simulators supported for testing
+BSPs using rtems-test. Currently, two BSPs are supported
+for testing via Renode: RISCV Kendryte K210 and SPARC Leon3. To use it,
+the host computer needs to have the necessary dependencies installed.
+
+1. Mono
+   Renode requires Mono >= 5.20 (Linux, macOS) or .NET >= 4.7 (Windows).
+
+   .. csv-table::
+   :delim: |
+
+   **Linux** | Install the ``mono-complete`` package as per the 
installation instructions for various Linux distributions, which can be found 
on `the Mono project website 
`_.
+   **macOS** | On macOS, the Mono package can be downloaded directly from 
`the Mono project website 
`_.
+   **Windows** | On Windows 7, download and install `.NET Framework 4.7 
`_. Windows 10 
ships with .NET by default, so no action is required.
+
+2. Other dependencies (Linux only)
+   On Ubuntu 20.04, you can install the remaining dependencies with the 
following command::
+
+  $ sudo apt-get install policykit-1 libgtk2.0-0 screen uml-utilities 
gtk-sharp2 libc6-dev gcc python3 python3-pip
+
+   If you are running a different distribution, you will need to install an 
analogous list of packages using your package manager; note that the package 
names may differ slightly.
+
+Installing Renode
+^
+
+All the dependencies are installed, you can go install Renode using the RTEMS 
Source Builder (the RSB).
+First, ``cd`` into the place where you put your RSB directory. Then you 
install Renode as follows::
+
+$ cd source-builder
+$ ../source-builder/sb-set-builder --prefix=$YOUR_PREFIX --trace 
--bset-tar-file renode
+
+Adding New BSP Test Config
+^^
+
+The implementation of Renode testing in ``rtems-test`` can be found in the 
rtems-tools repository
+at ``tester/rtems/renode`` folder. This folder contains all the ``resc`` 
scripts used to configure
+BSP testing with Renode. To add a new test configuration, you first need to 
check if the BSP is
+supported by Renode. You can do this by visiting `Renode's page that lists all 
supported boards 
+`_.
+
+If the board is listed there, you'll likely find an example ``resc`` script 
for a simple 
+testing configuration for the BSP in `Renode's GitHub repository 
+`_. You can 
then copy the 
+configuration to a new resc file in ``tester/rtems/renode``. The additional 
configuration that
+you need to add for it to work is the following::
+
+  showAnalyzer "uartAnalyzer" uart 
Antmicro.Renode.Analyzers.LoggingUartAnalyzer
+  uartAnalyzer TimestampFormat None
+
+  set report_repeating_line """
+  from Antmicro.Renode.Logging import ConsoleBackend 
+  ConsoleBackend.Instance.ReportRepeatingLines = True
+  """
+
+  set add_hook """
+  def my_match(line):
+  ok_to_kill_lines = [
+  '*** TEST STATE: USER_INPUT',
+  '*** TEST STATE: BENCHMARK',
+  '*** END OF TEST ',
+  '*** FATAL ***'
+  ]
+  return any(l in line for l in ok_to_kill_lines)
+
+  def my_hook(line):
+  print line
+  monitor.Parse("q")
+
+  
Antmicro.Renode.Hooks.UartHooksExtensions.AddLineHook(monitor.Machine["sysbus.uart"],
 my_match, my_hook)
+  """
+
+  python $add_hook
+
+  python $report_repeating_line 
+
+You need to add the above script configuration just before the ELF is loaded. 
In many cases,
+this is before the ``sysbus LoadELF`` line. The additional configuration is 
used to terminate the
+test according to the configuration of ``rtems-test`` and to map the test 
output to stdout.
+
+The next step is to delete the ``$bin`` variable definition from the example 
script. 
+This is because the ``$bin`` variable will be supplied via the ``renode.cfg`` 
file as the test binary.
-- 
2.34.1

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] build: Make gzip archives reproducible

[Chris Johns]

On 14/8/2023 4:24 pm, Sebastian Huber wrote:
> On 12.08.23 18:29, Joel Sherrill wrote:
>> Piling on Chris' comments about where is the requirement for this. This is a
>> topic which must be addressed in the Software Engineering Guide and accepted
>> before it can be put into practices. A software development process of any
>> quality cannot be changed on a whim without discussion and modifying all the
>> artifacts necessary.
> 
> Sorry, it was not my intention to open another can of worms. This reproducible
> builds stuff appeared to be some small thing to do.

I think it would be good to understand what you intend and why? At the moment we
are wondering or guessing based on some patches and that is never good. Our main
focus is the long term liability change have for us. Anything you can add to
explain things helps.

I think reproducible builds is a great thing to attempt but in the past there
have been issues. For example at one point the register allocator in gcc for the
m68k used the libc sort and Windows and Linux differed when sorting "all the
elements are the same" so the code built on Windows did not match the code built
on Linux even though it was equivalent code, ie register ordering for push and
pops differed.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] build: Make gzip archives reproducible

[Sebastian Huber]

On 12.08.23 18:29, Joel Sherrill wrote:

This looks ok.


Thanks.



Piling on Chris' comments about where is the requirement for this. This 
is a topic which must be addressed in the Software Engineering Guide and 
accepted before it can be put into practices. A software development 
process of any quality cannot be changed on a whim without discussion 
and modifying all the artifacts necessary.


Sorry, it was not my intention to open another can of worms. This 
reproducible builds stuff appeared to be some small thing to do.


--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel