On 01.06.23 22:06, Gedare Bloom wrote:
On Thu, Jun 1, 2023 at 2:00 PM Gedare Bloom<ged...@rtems.org> wrote:
---------- Forwarded message ---------
From: Sebastian Huber<sebastian.hu...@embedded-brains.de>
Date: Wed, May 31, 2023 at 10:31 AM
The existing tests in the RTEMS test suite are basically BSP
independent. This patch set introduces BSP-specific validation tests.
These tests are disabled for other BSPs through the build system, for
example:
spec/build/testsuites/validation/bsp-sparc-leon3-gr712rc.yml
[...]
cxxflags: []
enabled-by: sparc/gr712rc
features: c cprogram
[...]
The use of the enabled-by: field to control the BSP-specific tests
looks reasonable. However, we should decide where/how any BSP-specific
tests should reside. It looks to me like the current approach is to
dump all test files in a single monolithic 'validation' directory, and
let the user/script sort it out. This results in a mix of tests
intended for all targets, and some for specific targets. This is
pretty much non-maintainable from my point-of-view without some
additional tool support. Correct me if I'm wrong.
Yes, this is the current approach. There is no strict one-to-one
relationship of test cases and test suites. The file names are somewhat
descriptive, for example:
ls -l *leon3*
-rw-r--r-- 1 sebastian_h domain users 3739 31. Mai 10:44
tc-bsp-sparc-leon3-gr712rc.c
-rw-r--r-- 1 sebastian_h domain users 6138 31. Mai 10:44
tc-fatal-bsp-sparc-leon3-shutdown.c
-rw-r--r-- 1 sebastian_h domain users 5135 31. Mai 10:44
tr-fatal-bsp-sparc-leon3-cache-snooping-disabled-boot.c
-rw-r--r-- 1 sebastian_h domain users 2723 31. Mai 10:44
tr-fatal-bsp-sparc-leon3-cache-snooping-disabled-boot.h
-rw-r--r-- 1 sebastian_h domain users 5406 31. Mai 10:44
tr-fatal-bsp-sparc-leon3-cache-snooping-disabled-secondary.c
-rw-r--r-- 1 sebastian_h domain users 2753 31. Mai 10:44
tr-fatal-bsp-sparc-leon3-cache-snooping-disabled-secondary.h
-rw-r--r-- 1 sebastian_h domain users 5270 31. Mai 10:44
tr-fatal-bsp-sparc-leon3-clock-initialization.c
-rw-r--r-- 1 sebastian_h domain users 2681 31. Mai 10:44
tr-fatal-bsp-sparc-leon3-clock-initialization.h
-rw-r--r-- 1 sebastian_h domain users 2488 31. Mai 10:44
ts-bsp-sparc-leon3-gr712rc.c
-rw-r--r-- 1 sebastian_h domain users 2845 31. Mai 10:44
ts-fatal-bsp-sparc-leon3-cache-snooping-disabled-boot.c
-rw-r--r-- 1 sebastian_h domain users 2919 31. Mai 10:44
ts-fatal-bsp-sparc-leon3-cache-snooping-disabled-secondary.c
-rw-r--r-- 1 sebastian_h domain users 2797 31. Mai 10:44
ts-fatal-bsp-sparc-leon3-clock-initialization.c
-rw-r--r-- 1 sebastian_h domain users 3148 31. Mai 10:45
ts-fatal-bsp-sparc-leon3-shutdown-response.c
-rw-r--r-- 1 sebastian_h domain users 4909 31. Mai 10:44
ts-fatal-bsp-sparc-leon3-shutdown.c
We could also introduce subdirectories to organize things. The test
framework prints out the file name in messages, so it would be nice if
they remain unique. With subdirectories this would lead to longer path
names, for example
testsuites/validation/sparc/leon3/tc-bsp-sparc-leon3-gr712rc.c
I would like to discuss possible ways to manage the integration of
tests that are conditionally-built based on the arch/bsp tuple. We
should have clear guidance for others who want to add such tests in
the future, or who would modify existing tests.
Yes, this makes sense. We could add a new section for BSP-specific tests to:
https://docs.rtems.org/branches/master/eng/req/howto.html
For a pre-qualified BSP you have to specify the fatal errors and write
validation tests for it. Other BSP-specific specification and validation
may be necessary for the kernel IO device driver, cache controller
support, memory management unit initialization, memory protection unit
initialization, etc.
Even for normal BSPs, some unit tests would be helpful. For example, I
had to fix _AArch32_PMSA_Initialize() twice:
https://github.com/RTEMS/rtems/commits/master/cpukit/score/cpu/arm/aarch32-psma-init.c
Unit tests would have probably found the errors before applications hit
the special cases.
--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.hu...@embedded-brains.de
phone: +49-89-18 94 741 - 16
fax: +49-89-18 94 741 - 08
Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
