Hmmm...I must have gotten the wrong patch as this now applies to master for me. Spelling issues noted.
On Wed, Nov 18, 2015 at 12:09 PM, Mike Holmes <mike.hol...@linaro.org> wrote: > Signed-off-by: Mike Holmes <mike.hol...@linaro.org> > --- > doc/implementers-guide/implementers-guide.adoc | 265 > +++++++++++++++++-------- > 1 file changed, 185 insertions(+), 80 deletions(-) > > diff --git a/doc/implementers-guide/implementers-guide.adoc > b/doc/implementers-guide/implementers-guide.adoc > index 0033ba3..8b9c729 100644 > --- a/doc/implementers-guide/implementers-guide.adoc > +++ b/doc/implementers-guide/implementers-guide.adoc > @@ -11,9 +11,11 @@ Further details about ODP may be found at > http://opendataplane.org[ODP homepage] > > > :numbered: > -The include structure > ---------------------- > -The implementers view of the include source tree allows the common API > definitions and documentation to be reused by all the platforms defined in > the tree, but leave the actual definitions to be defined by the specific > platform. > +== The include structure == > + > +The implementers view of the include source tree allows the common API > +definitions and documentation to be reused by all the platforms defined > in the > +tree, but leave the actual definitions to be defined by the specific > platform. > > .Implementers include structure > ---- > @@ -29,94 +31,145 @@ The implementers view of the include source tree > allows the common API definitio > │ ├── <implementation name>/ > │ │ ├── include/ > │ │ │ ├── odp/ > -│ │ │ │ ├── In-line function definitions of the public API for > this platform > -│ │ │ │ │ seen by the applicationx. > +│ │ │ │ ├── In-line function definitions of the public API for > this > +│ │ │ │ │ platform seen by the application. > │ │ │ │ │ > │ │ │ │ └── plat/ > -│ │ │ │ └── Platform specific types, enums etc as seen by the > application > -│ │ │ │ but require overriding by the implementation. > +│ │ │ │ └── Platform specific types, enums etc as seen by the > +│ │ │ │ application but require overriding by the > +│ │ │ │ implementation. > │ │ │ │ > │ │ │ └── Internal header files seen only by the implementation. > ---- > > -The doxygen description of the API definition is held in the public api > file 'include/odp/api'. > -This file is included by a counterpart in 'platform/<implementation > name>/include/odp'. > -The include of the public API is AFTER the platform specific definitions > to allow the platform to provide definitions that match the underlying > hardware. > -The implementation code includes 'platform/<implementation > name>/include/plat' and this then provides the source files with a complete > definition the ODP API to be implemented. > -Applications in turn include the include/odp.h file which includes the > 'platform/<implementation name>/include/plat' files to provide a complete > definition of the API. > +The doxygen description of the API definition is held in the public api > file > +'include/odp/api'. > +This file is included by a counterpart in > +'platform/<implementation name>/include/odp'. > +The include of the public API is AFTER the platform specific definitions > to > +allow the platform to provide definitions that match the underlying > hardware. > +The implementation code includes 'platform/<implementation > name>/include/plat' > +and this then provides the source files with a complete definition the > ODP API > +to be implemented. > +Applications in turn include the include/odp.h file which includes the > +'platform/<implementation name>/include/plat' files to provide a complete > +definition of the API. > > -The validation Suite > --------------------- > -ODP provides a comprehensive set of API validation tests that are > intended to be used by implementers during development and by application > developers to verify that a particular implementation meets their > requirements. > +== The validation Suite == > + > +ODP provides a comprehensive set of API validation tests that are > intended to be > +used by implementers during development and by application developers to > verify > +that a particular implementation meets their requirements. > > The list of these tests is expected to grow as ODP grows. > > -The list of test executables is run by the automake test harness, when > running "make check". > -Therefore, as required by this harness, each executable should return 0 > on success (tests passed), 77 on inconclusive, or any other values on > failure. > -The automake functionality shows a status line (PASSED/FAIL...) for each > of the ran test executables. > +The list of test executables is run by the automake test harness, when > running > +"make check". > +Therefore, as required by this harness, each executable should return 0 on > +success (tests passed), 77 on inconclusive, or any other values on > failure. > +The automake functionality shows a status line (PASSED/FAIL...) for each > of the > +ran test executables. > > -It is expected that ODP developers will need to run tests as early as > possible in the development cycle, before all APIs have been implemented. > -Besides, although there are no APIs that are formally listed as optional, > it is also expected that there may be cases where a subset of APIs remain > unimplemented on a particular platform. > -Moreover, some platforms may require specific initialization/termination > code prior/after running the standard tests. > +It is expected that ODP developers will need to run tests as early as > possible > +in the development cycle, before all APIs have been implemented. > +Besides, although there are no APIs that are formally listed as optional, > it is > +also expected that there may be cases where a subset of APIs remain > +unimplemented on a particular platform. > +Moreover, some platforms may require specific initialization/termination > code > +prior/after running the standard tests. > > -To accommodate with these platform disparities, the ODP validation has > been divided in two distinct areas: > +To accommodate with these platform disparities, the ODP validation has > been > +divided in two distinct areas: > > * The platform agnostic area, > * A platform dependent area (one per platform). > > -Platform agnostic > -~~~~~~~~~~~~~~~~~ > -This grouping defines tests that are expected to be executable and > succeed on any platform, though possibly with very different performances, > depending on the underlying platform. > -They are written in plain C code, and may only use functions defined in > the standard libC library (besides the ODP functions being tested, of > course). > -No other languages (like scripting) are allowed as their usage would make > assumptions on the platform capability. > +=== Platform agnostic === > + > +This grouping defines tests that are expected to be executable and > succeed on > +any platform, though possibly with very different performances, depending > on the > +underlying platform. > +They are written in plain C code, and may only use functions defined in > the > +standard libC library (besides the ODP functions being tested, of course). > +No other languages (like scripting) are allowed as their usage would make > +assumptions on the platform capability. > > This area is located at: 'test/validation/' > > -The ODP API itself is ordered by module, where each module groups the set > of ODP API functions related to the same "topic". > -Examples of modules includes "classification" (API functions dealing with > ingres packets classification), time (functions dealing with time, > excluding timers which have their own module), timer,... > -The complete module list can be seen at: > http://docs.opendataplane.org/linux-generic-doxygen-html/modules.html[ODP > Modules] + > -Within the platform agnostic area, the tests are also grouped by modules, > matching the ODP API modules: 'test/validation/' mainly contains a list of > directories matching each module name (as defined by the doxygen > "@defgroup" or "@ingroup" statement present in each API ".h" file). > +The ODP API itself is ordered by module, where each module groups the set > of ODP > +API functions related to the same "topic". > +Examples of modules includes "classification" (API functions dealing with > ingres > ingress > +packets classification), time (functions dealing with time, excluding > timers > +which have their own module), timer,... > +The complete module list can be seen at: > +http://docs.opendataplane.org/linux-generic-doxygen-html/modules.html[ODP > Modules] + > +Within the platform agnostic area, the tests are also grouped by modules, > +matching the ODP API modules: 'test/validation/' mainly contains a list of > +directories matching each module name (as defined by the doxygen > "@defgroup" or > +"@ingroup" statement present in each API ".h" file). > > -Within each of these directories, a library (called "libtest<module>.la") > and its associated ".h" file (called "<module>.h") defines all the test > functions for this module as well as few other functions to initialize, > terminate, and group the tests. > -An executable called "<module>_main*", is also built. It is permissible > to generate more than one executable to cover the functionality in the test > library for the module. > +Within each of these directories, a library (called "libtest<module>.la") > and > +its associated ".h" file (called "<module>.h") defines all the test > functions > +for this module as well as few other functions to initialize, terminate, > and > +group the tests. > +An executable called "<module>_main*", is also built. It is permissible to > +generate more than one executable to cover the functionality in the test > library > +for the module. > These executable(s) shall call all the tests for this module. + > See <<anchor-1, Module test and naming convention>> for more details. > > -It is important to be aware that the tests defined for a given module > (defined in 'test/validation/<module>') are focused to test the ODP > functions belonging to this module, but are not limited to use this > module's ODP functions only: many modules needs some interaction with some > other module to be tested. > -The obvious illustration of this is for module "init" whose functions are > required by all tests of all other modules (as ODP needs to be initialized > to test anything else). + > +It is important to be aware that the tests defined for a given module > +(defined in 'test/validation/<module>') are focused to test the ODP > functions > +belonging to this module, but are not limited to use this module's ODP > functions > +only: many modules needs some interaction with some other module to be > tested. > +The obvious illustration of this is for module "init" whose functions are > +required by all tests of all other modules (as ODP needs to be > initialized to > +test anything else). + > > -There is a "Makefile.am" located at the top of the platform agnostic > area. Its role is limited to the construction of the different test > libraries and the "<module>_main*" executables. No tests are run from this > area when "make check" is performed. > +There is a "Makefile.am" located at the top of the platform agnostic > area. Its > +role is limited to the construction of the different test libraries and > the > +"<module>_main*" executables. No tests are run from this area when "make > check" > +is performed. > > -CUnit > -^^^^^^ > -Within a given test executable CUnit is used to run the different tests. > The usage of CUnit implies the following structure: > +==== CUnit ==== > + > +Within a given test executable CUnit is used to run the different tests. > The > +usage of CUnit implies the following structure: > > * Tests are simple C functions. > -* Tests are grouped in arrays called test suites. Each test suite can be > associated with a suite initialization/termination function(s), called by > CUnit before and after the whole suite is run. > -* An array of test suites (and associated init/term functions) defines > the test registry run by the test executable. > +* Tests are grouped in arrays called test suites. Each test suite can be > +associated with a suite initialization/termination function(s), called by > CUnit > +before and after the whole suite is run. > +* An array of test suites (and associated init/term functions) defines > the test > +registry run by the test executable. > > -Moreover, two extra functions can be used to initialize/terminate the > test executable (these are not part of CUnit). + > +Moreover, two extra functions can be used to initialize/terminate the test > +executable (these are not part of CUnit). + > A test executable return success (0) if every test of each suite succeed. > > -More details about http://cunit.sourceforge.net/doc/index.html[CUnit > users guide] > +More details about > +http://cunit.sourceforge.net/doc/index.html[CUnit users guide] > > [[anchor-1]] > -Module test and naming convention > -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > +==== Module test and naming convention ==== > + > > * Tests, i.e. C functions which are used in CUnit test suites are named: > *<Module>_test_+++*+++* + > where the suffix idendify the test. > identifies > > -* Test arrays, i.e. arrays of odp_testinfo_t, listing the test functions > belonging to a suite, are called: > +* Test arrays, i.e. arrays of odp_testinfo_t, listing the test functions > + belonging to a suite, are called: > *<Module>_suite+++[_*]+++* + > where the possible suffix can be used if many suites are declared. > > * CUnit suite init and termination functions are called: > *<Module>+++_suite[_*]_init()+++* and > *<Module>+++_suite[_*]_term()+++* respectively. + > - where the possible extra middle pattern can be used if many suites are > declared. > + where the possible extra middle pattern can be used if many suites are > + declared. > > -* Suite arrays, i.e. arrays of odp_suiteinfo_t used in executables (CUnit > registry) are called: > +* Suite arrays, i.e. arrays of odp_suiteinfo_t used in executables > + (CUnit registry) are called: > *<Module>+++_suites[_*]+++* + > where the possible suffix identifies the executable using it, if many. > > @@ -128,18 +181,28 @@ Module test and naming convention > *<Module>_init* > *<Module>_term* > > -All the above symbols are part of the generated libtest<Module>.la > libraries. The generated main executable(s) (named <module>_+++main[_*]+++, > where the optional suffix is used to distinguish the executables belonging > to the same module, if many) simply call(s) the related > <Module>_main+++[_*]+++ from the library. > +All the above symbols are part of the generated libtest<Module>.la > libraries. > +The generated main executable(s) (named <module>_+++main[_*]+++, where the > +optional suffix is used to distinguish the executables belonging to the > same > +module, if many) simply call(s) the related <Module>_main+++[_*]+++ from > the > +library. > > -Platform specific > -~~~~~~~~~~~~~~~~~ > -These tests are located under 'platform/<platform>/test'. There is one > such area for each platform implementing ODP. > -This location will be referred as <PLATFORM_SPECIFIC> in the rest of this > document. > +=== Platform specific === > > -The normal case > -^^^^^^^^^^^^^^^ > -If the considered platform needs no platform specific tests, this > directory simply needs to contain a single Makefile.am listing each of the > executables (named <module>_main) built from the platform agnostic area. > The executables are listed in the automake TEST variable and will therefore > be run on "make check". > +These tests are located under 'platform/<platform>/test'. There is one > such area > +for each platform implementing ODP. > +This location will be referred as <PLATFORM_SPECIFIC> in the rest of this > +document. > > -For the linux-generic platform, most tested modules fall into this > category: currently, the 'platform/linux-generic/test/Makefile.am' looks as > follows: > +==== The normal case ==== > + > +If the considered platform needs no platform specific tests, this > directory > +simply needs to contain a single Makefile.am listing each of the > executables > +(named <module>_main) built from the platform agnostic area. The > executables are > +listed in the automake TEST variable and will therefore be run on "make > check". > + > +For the linux-generic platform, most tested modules fall into this > category: > +currently, the 'platform/linux-generic/test/Makefile.am' looks as follows: > > [source,am] > ---- > @@ -175,22 +238,39 @@ endif > > ---- > > -With the exception for module pktio, all other modules testing just > involves calling the platform agnostic <module>_main executables (in > test/validation). > +With the exception for module pktio, all other modules testing just > involves > +calling the platform agnostic <module>_main executables (in > test/validation). > > -Using other languages > -^^^^^^^^^^^^^^^^^^^^^ > -The pktio module, above, is actually tested using a bash script. This > script is needed to set up the interfaces used by the tests. The pktio_run > script eventually calls the platform agnostic > 'test/validation/pktio/pktio_main' after setting up the interfaces needed > by the tests. > -Notice that the path to the script, 'pktio/pktio_run', is pointing to a > file within the <PLATFORM_SPECIFIC> tree so is private to this platform. > Any languages supported by the tested platform can be used there, as it > will not impact other platforms. > -The platform "private" executables (such as this script), of course, must > also return one of the return code expected by the automake test harness (0 > for success, 77 for skipped, other values for errors). > +==== Using other languages ==== > > -Defining test wrappers > -^^^^^^^^^^^^^^^^^^^^^^ > -The pktio case above is actually using a script as wrapper around the > "standard" (platform independent) test executable. Wrappers can also be > defined by using the LOG_COMPILER variable of automake. > -This is applicable in cases where the same wrapper should be used for > more then one test, as the test name is passed has parameter to the > wrapper. A wrapper is just a program expecting one argument: the test name. > +The pktio module, above, is actually tested using a bash script. This > script is > +needed to set up the interfaces used by the tests. The pktio_run script > +eventually calls the platform agnostic 'test/validation/pktio/pktio_main' > after > +setting up the interfaces needed by the tests. > +Notice that the path to the script, 'pktio/pktio_run', is pointing to a > file > +within the <PLATFORM_SPECIFIC> tree so is private to this platform. Any > +languages supported by the tested platform can be used there, as it will > not > +impact other platforms. > +The platform "private" executables (such as this script), of course, must > also > +return one of the return code expected by the automake test harness > +(0 for success, 77 for skipped, other values for errors). > > -Automake also supports the usage different wrappers based of the > executable filename suffix. See > https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html[Parallel-Test-Harness] > for more information. > +==== Defining test wrappers ==== > > -To add a wrapper around the executed test, just add the following > LOG_COMPILER definition line in the '<PLATFORM_SPECIFIC>/Makefile.am': > +The pktio case above is actually using a script as wrapper around the > "standard" > +(platform independent) test executable. Wrappers can also be defined by > using > +the LOG_COMPILER variable of automake. > +This is applicable in cases where the same wrapper should be used for > more then > +one test, as the test name is passed has parameter to the wrapper. A > wrapper is > +just a program expecting one argument: the test name. > + > +Automake also supports the usage different wrappers based of the > executable > +filename suffix. See > + > https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html[Parallel-Test-Harness] > +for more information. > + > +To add a wrapper around the executed test, just add the following > LOG_COMPILER > +definition line in the '<PLATFORM_SPECIFIC>/Makefile.am': > > [source,am] > ---- > @@ -221,18 +301,40 @@ echo "Do something to clean up the mess here :-)" > exit $res > ---- > > -Note how the above script stores the return code of the test executable > to return it properly to the automake test harness. > +Note how the above script stores the return code of the test executable to > +return it properly to the automake test harness. > > -Defining platform specific tests > -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > -Sometimes, it may be necessary to call platform specific system calls to > check some functionality: For instance, testing odp_cpumask_* could involve > checking the underlying system CPU mask. On linux, such a test would > require using the CPU_ISSET macro, which is linux specific. Such a test > would be written in '<PLATFORM_SPECIFIC>/cpumask/...' The contents of this > directory would be very similar to the contents of the platform agnostic > side cpu_mask tests (including a Makefile.am...), but platform specific > test would be written there. > -'<PLATFORM_SPECIFIC>/Makefile.am' would then trigger the building of the > platform specific tests (by listing their module name in SUBDIRS and > therefore calling the appropriate Makefile.am) and then it would call both > the platform agnostic executable(s) and the platform specific test > executable. > +==== Defining platform specific tests ==== > > -Marking tests as inactive > -^^^^^^^^^^^^^^^^^^^^^^^^^ > -The general policy is that a full run of the validation suite (a "make > check") must pass at all times. However a particular platform may have one > or more test cases that are known to be unimplemented either during > development or permanently, so to avoid these test cases being reported as > failures it's useful to be able to skip them. This can be achieved by > creating a new test executable (still on the platform side), giving the > platform specific initialisation code the opportunity to modify the > registered tests in order to mark unwanted tests as inactive while leaving > the remaining tests active. It's important that the unwanted tests are > still registered with the test framework to allow the fact that they're not > being tested to be recorded. > +Sometimes, it may be necessary to call platform specific system calls to > check > +some functionality: For instance, testing odp_cpumask_* could involve > checking > +the underlying system CPU mask. On linux, such a test would require using > the > +CPU_ISSET macro, which is linux specific. Such a test would be written in > +'<PLATFORM_SPECIFIC>/cpumask/...' The contents of this directory would be > very > +similar to the contents of the platform agnostic side cpu_mask tests > +(including a Makefile.am...), but platform specific test would be written > there. > +'<PLATFORM_SPECIFIC>/Makefile.am' would then trigger the building of the > +platform specific tests (by listing their module name in SUBDIRS and > therefore > +calling the appropriate Makefile.am) and then it would call both the > platform > +agnostic executable(s) and the platform specific test executable. > > -The odp_cunit_update() function is intended for this purpose, it is used > to modify the properties of previously registered tests, for example to > mark them as inactive. Inactive tests are registered with the test > framework but aren't executed and will be recorded as inactive in test > reports. > +==== Marking tests as inactive ==== > + > +The general policy is that a full run of the validation suite (a "make > check") > +must pass at all times. However a particular platform may have one or > more test > +cases that are known to be unimplemented either during development or > +permanently, so to avoid these test cases being reported as failures it's > useful > +to be able to skip them. This can be achieved by creating a new test > executable > +(still on the platform side), giving the platform specific initialisation > code > initialization > +the opportunity to modify the registered tests in order to mark unwanted > tests > +as inactive while leaving the remaining tests active. It's important that > the > +unwanted tests are still registered with the test framework to allow the > fact > +that they're not being tested to be recorded. > + > +The odp_cunit_update() function is intended for this purpose, it is used > to > +modify the properties of previously registered tests, for example to mark > them > +as inactive. Inactive tests are registered with the test framework but > aren't > +executed and will be recorded as inactive in test reports. > > In 'test/validation/foo/foo.c', define all tests for the 'foo' module: > > @@ -250,7 +352,8 @@ odp_suiteinfo_t foo_suites[] = { > }; > ------------------ > > -In 'platform/<platform>/test/foo/foo_main.c', register all the tests > defined in the 'foo' module, then mark a single specific test case as > inactive: > +In 'platform/<platform>/test/foo/foo_main.c', register all the tests > defined in > +the 'foo' module, then mark a single specific test case as inactive: > > [source,c] > ------------------ > @@ -280,4 +383,6 @@ int foo_main(void) > > So 'foo_test_a' will be executed and 'foo_test_b' is inactive. > > -It's expected that early in the development cycle of a new implementation > the inactive list will be quite long, but it should shrink over time as > more parts of the API are implemented. > +It's expected that early in the development cycle of a new implementation > the > +inactive list will be quite long, but it should shrink over time as more > parts > +of the API are implemented. > -- > 2.5.0 > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > https://lists.linaro.org/mailman/listinfo/lng-odp >
_______________________________________________ lng-odp mailing list lng-odp@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lng-odp
