Re: [PATCH net-next 7/7] tools: tc-testing: Update README and TODO
On Wed, Feb 14, 2018 at 2:09 PM, Brenda J. Butler wrote: > Signed-off-by: Brenda J. Butler Acked-by: Lucas Bates > --- > tools/testing/selftests/tc-testing/README | 173 > +--- > tools/testing/selftests/tc-testing/TODO.txt | 25 +++- > 2 files changed, 179 insertions(+), 19 deletions(-) > > diff --git a/tools/testing/selftests/tc-testing/README > b/tools/testing/selftests/tc-testing/README > index 970ff294fec8..3a0336782d2d 100644 > --- a/tools/testing/selftests/tc-testing/README > +++ b/tools/testing/selftests/tc-testing/README > @@ -14,11 +14,11 @@ REQUIREMENTS > > * The kernel must have network namespace support > > -* The kernel must have veth support available, as a veth pair is created > +* The kernel must have veth support available, as a veth pair is created > prior to running the tests. > > -* All tc-related features must be built in or available as modules. > - To check what is required in current setup run: > +* All tc-related features being tested must be built in or available as > + modules. To check what is required in current setup run: > ./tdc.py -c > > Note: > @@ -44,10 +44,13 @@ using the -p option when running tdc: > RUNNING TDC > --- > > -To use tdc, root privileges are required. tdc will not run otherwise. > +To use tdc, root privileges are required. This is because the > +commands being tested must be run as root. The code that enforces > +execution by root uid has been moved into a plugin (see PLUGIN > +ARCHITECTURE, below). > > -All tests are executed inside a network namespace to prevent conflicts > -within the host. > +If nsPlugin is linked, all tests are executed inside a network > +namespace to prevent conflicts within the host. > > Running tdc without any arguments will run all tests. Refer to the section > on command line arguments for more information, or run: > @@ -59,6 +62,33 @@ output captured from the failing test will be printed > immediately following > the failed test in the TAP output. > > > +OVERVIEW OF TDC EXECUTION > +- > + > +One run of tests is considered a "test suite" (this will be refined in the > +future). A test suite has one or more test cases in it. > + > +A test case has four stages: > + > + - setup > + - execute > + - verify > + - teardown > + > +The setup and teardown stages can run zero or more commands. The setup > +stage does some setup if the test needs it. The teardown stage undoes > +the setup and returns the system to a "neutral" state so any other test > +can be run next. These two stages require any commands run to return > +success, but do not otherwise verify the results. > + > +The execute and verify stages each run one command. The execute stage > +tests the return code against one or more acceptable values. The > +verify stage checks the return code for success, and also compares > +the stdout with a regular expression. > + > +Each of the commands in any stage will run in a shell instance. > + > + > USER-DEFINED CONSTANTS > -- > > @@ -70,23 +100,132 @@ executed as part of the test. More will be added as > test cases require. > Example: > $TC qdisc add dev $DEV1 ingress > > +The NAMES values are used to substitute into the commands in the test cases. > + > > COMMAND LINE ARGUMENTS > -- > > Run tdc.py -h to see the full list of available arguments. > > --p PATH Specify the tc executable located at PATH to be used on > this > - test run > --cShow the available test case categories in this test file > --c CATEGORY Run only tests that belong to CATEGORY > --f FILE Read test cases from the JSON file named FILE > --l [CATEGORY] List all test cases in the JSON file. If CATEGORY is > - specified, list test cases matching that category. > --s ID Show the test case matching ID > --e ID Execute the test case identified by ID > --iGenerate unique ID numbers for test cases with no existing > - ID number > +usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]] > + [-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v] > + [-d DEVICE] [-n NS] [-V] > + > +Linux TC unit tests > + > +optional arguments: > + -h, --helpshow this help message and exit > + -p PATH, --path PATH The full path to the tc executable to use > + -v, --verbose Show the commands that are being run > + -d DEVICE, --device DEVICE > +Execute the test case in flower category > + > +selection: > + select which test cases: files plus directories; filtered by categories > + plus testids > + > + -D DIR [DIR ...], --directory DIR [DIR ...] > +Collect tests from the specified directory(ies) > +(default [tc-tests]) > + -f FILE [FILE ...], --file FILE [FILE ..
[PATCH net-next 7/7] tools: tc-testing: Update README and TODO
Signed-off-by: Brenda J. Butler --- tools/testing/selftests/tc-testing/README | 173 +--- tools/testing/selftests/tc-testing/TODO.txt | 25 +++- 2 files changed, 179 insertions(+), 19 deletions(-) diff --git a/tools/testing/selftests/tc-testing/README b/tools/testing/selftests/tc-testing/README index 970ff294fec8..3a0336782d2d 100644 --- a/tools/testing/selftests/tc-testing/README +++ b/tools/testing/selftests/tc-testing/README @@ -14,11 +14,11 @@ REQUIREMENTS * The kernel must have network namespace support -* The kernel must have veth support available, as a veth pair is created +* The kernel must have veth support available, as a veth pair is created prior to running the tests. -* All tc-related features must be built in or available as modules. - To check what is required in current setup run: +* All tc-related features being tested must be built in or available as + modules. To check what is required in current setup run: ./tdc.py -c Note: @@ -44,10 +44,13 @@ using the -p option when running tdc: RUNNING TDC --- -To use tdc, root privileges are required. tdc will not run otherwise. +To use tdc, root privileges are required. This is because the +commands being tested must be run as root. The code that enforces +execution by root uid has been moved into a plugin (see PLUGIN +ARCHITECTURE, below). -All tests are executed inside a network namespace to prevent conflicts -within the host. +If nsPlugin is linked, all tests are executed inside a network +namespace to prevent conflicts within the host. Running tdc without any arguments will run all tests. Refer to the section on command line arguments for more information, or run: @@ -59,6 +62,33 @@ output captured from the failing test will be printed immediately following the failed test in the TAP output. +OVERVIEW OF TDC EXECUTION +- + +One run of tests is considered a "test suite" (this will be refined in the +future). A test suite has one or more test cases in it. + +A test case has four stages: + + - setup + - execute + - verify + - teardown + +The setup and teardown stages can run zero or more commands. The setup +stage does some setup if the test needs it. The teardown stage undoes +the setup and returns the system to a "neutral" state so any other test +can be run next. These two stages require any commands run to return +success, but do not otherwise verify the results. + +The execute and verify stages each run one command. The execute stage +tests the return code against one or more acceptable values. The +verify stage checks the return code for success, and also compares +the stdout with a regular expression. + +Each of the commands in any stage will run in a shell instance. + + USER-DEFINED CONSTANTS -- @@ -70,23 +100,132 @@ executed as part of the test. More will be added as test cases require. Example: $TC qdisc add dev $DEV1 ingress +The NAMES values are used to substitute into the commands in the test cases. + COMMAND LINE ARGUMENTS -- Run tdc.py -h to see the full list of available arguments. --p PATH Specify the tc executable located at PATH to be used on this - test run --cShow the available test case categories in this test file --c CATEGORY Run only tests that belong to CATEGORY --f FILE Read test cases from the JSON file named FILE --l [CATEGORY] List all test cases in the JSON file. If CATEGORY is - specified, list test cases matching that category. --s ID Show the test case matching ID --e ID Execute the test case identified by ID --iGenerate unique ID numbers for test cases with no existing - ID number +usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]] + [-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v] + [-d DEVICE] [-n NS] [-V] + +Linux TC unit tests + +optional arguments: + -h, --helpshow this help message and exit + -p PATH, --path PATH The full path to the tc executable to use + -v, --verbose Show the commands that are being run + -d DEVICE, --device DEVICE +Execute the test case in flower category + +selection: + select which test cases: files plus directories; filtered by categories + plus testids + + -D DIR [DIR ...], --directory DIR [DIR ...] +Collect tests from the specified directory(ies) +(default [tc-tests]) + -f FILE [FILE ...], --file FILE [FILE ...] +Run tests from the specified file(s) + -c [CATG [CATG ...]], --category [CATG [CATG ...]] +Run tests only from the specified category/ies, or if +no category/ies is/are specified, list known +categories.
