Hello,
in the long run we need a framework to write tests. At the moment I have
absolutely no time to work on this. I need a short term solution to get
profiling data for the tests.
Why is this profiling important? Currently we don't know how good RTEMS is in
terms of thread dispatch latency for example (an important characteristic for
real-time systems). We have some timing tests, but they yield only some
artificial values (except tmcontext01). The timing tests neglect cache effects
and load from other processors on an SMP system. The tmcontext01 is a template
test that addresses this issue. With the profiling data obtained for all tests
of the RTEMS test suite we get a larger sample size. The profiling data is in
XML format which makes it very easy to post-process, e.g.
http://git.rtems.org/rtems/tree/testsuites/tmtests/tmcontext01/plot.py
Supporting SMP will lead to massive changes in the critical sections if we want
to support fine grained locking. Thus we have to show somehow that things
don't get too bad for the single-processor systems. This is why I work on the
profiling first.
With the first profiling data available (smpload01) it is obvious that we need
fine grained locking:
<SMPLockProfilingReport name="Giant">
<MaxAcquireTime unit="ns">245180</MaxAcquireTime>
<MaxSectionTime unit="ns">232130</MaxSectionTime>
<UsageCount>10102</UsageCount>
<TotalAcquireTime unit="ns">416556635</TotalAcquireTime>
<TotalSectionTime unit="ns">529718895</TotalSectionTime>
<ContentionCount initialQueueLength="0">1950</ContentionCount>
<ContentionCount initialQueueLength="1">4735</ContentionCount>
<ContentionCount initialQueueLength="2">2441</ContentionCount>
<ContentionCount initialQueueLength="3">976</ContentionCount>
</SMPLockProfilingReport>
Now back to the tests. The only practically available way to determine if a
test was successful is to look at the "*** END OF TEST" pattern in the test
output. It makes sense to use a common function for test begin and end
messages. Now where should this function be located?
We have a testsuites/support directory with a bunch of files. I would like to
move away from the existing test support. The problem is that you have to add
at least this in every test Makefile.am:
test_SOURCES += ../../support/src/test_support.c
AM_CPPFLAGS += -I$(top_srcdir)/../support/include
I consider ../ paths in Makefiles as hacks.
You have to compile test_support.c again for each tests which increases the
build times.
You have also hacks like this in a header file (buffer_test_io.h):
#if defined(TEST_INIT) || defined(CONFIGURE_INIT)
char _test_output_buffer[_TEST_OUTPUT_BUFFER_SIZE];
int _test_output_buffer_index = 0;
void _test_output_append(char *_buffer)
{
char *p;
for ( p=_buffer ; *p ; p++ ) {
_test_output_buffer[_test_output_buffer_index++] = *p;
_test_output_buffer[_test_output_buffer_index] = '\0';
#if 0
if ( *p == '\n' ) {
fprintf( stderr, "BUFFER -- %s", _test_output_buffer );
_test_output_buffer_index = 0;
_test_output_buffer[0] = '\0';
}
#endif
if ( _test_output_buffer_index >= (_TEST_OUTPUT_BUFFER_SIZE - 80) )
_test_output_flush();
}
}
#include <termios.h>
#include <unistd.h>
void _test_output_flush(void)
{
fprintf( stderr, "%s", _test_output_buffer );
_test_output_buffer_index = 0;
tcdrain( 2 );
}
#endif
Profiling data can be output in a test fatal extension handler.
I propose to move test support code to the cpukit. Attached is an example
header file.
--
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
/*
* Copyright (c) 2014 embedded brains GmbH. All rights reserved.
*
* embedded brains GmbH
* Dornierstr. 4
* 82178 Puchheim
* Germany
* <rt...@embedded-brains.de>
*
* The license and distribution terms for this file may be
* found in the file LICENSE in this distribution or at
* http://www.rtems.com/license/LICENSE.
*/
#ifndef _RTEMS_TEST_H
#define _RTEMS_TEST_H
#include <rtems.h>
#include <rtems/bspIo.h>
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @defgroup RTEMSTest Test Support
*
* @brief Test support functions.
*
* @{
*/
/**
* @brief Each test must define a test name string.
*/
extern const char rtems_test_name[];
/**
* @brief Fatal extension for tests.
*/
void rtems_test_fatal_extension(
rtems_fatal_source source,
bool is_internal,
rtems_fatal_code code
);
/**
* @brief Initial extension for tests.
*/
#define RTEMS_TEST_INITIAL_EXTENSION \
{ NULL, NULL, NULL, NULL, NULL, NULL, NULL, rtems_test_fatal_extension }
/**
* @brief Prints a begin of test message.
*
* @param[in] printf_func The formatted output function.
* @param[in, out] printf_arg The formatted output function argument.
*
* @returns As specified by printf().
*/
int rtems_test_begin_with_plugin(
rtems_printk_plugin_t printf_func,
void *printf_arg
);
/**
* @brief Prints a begin of test message using printf().
*
* @returns As specified by printf().
*/
static inline int rtems_test_begin(void)
{
return rtems_test_begin_with_plugin(rtems_printf_plugin, NULL);
}
/**
* @brief Prints a begin of test message using printk().
*
* @returns As specified by printf().
*/
static inline int rtems_test_begink(void)
{
return rtems_test_begin_with_plugin(printk_plugin, NULL);
}
/**
* @brief Prints an end of test message.
*
* @param[in] printf_func The formatted output function.
* @param[in, out] printf_arg The formatted output function argument.
*
* @returns As specified by printf().
*/
int rtems_test_end_with_plugin(
rtems_printk_plugin_t printf_func,
void *printf_arg
);
/**
* @brief Prints an end of test message using printf().
*
* @returns As specified by printf().
*/
static inline int rtems_test_end(void)
{
return rtems_test_end_with_plugin(rtems_printf_plugin, NULL);
}
/**
* @brief Prints an end of test message using printk().
*
* @returns As specified by printf().
*/
static inline int rtems_test_endk(void)
{
return rtems_test_end_with_plugin(printk_plugin, NULL);
}
/** @} */
#ifdef __cplusplus
}
#endif /* __cplusplus */
#endif /* _RTEMS_TEST_H */
_______________________________________________
rtems-devel mailing list
rtems-devel@rtems.org
http://www.rtems.org/mailman/listinfo/rtems-devel
