Re: [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable
On Thu, Oct 15, 2020 at 02:57:34PM +0530, Naresh Kamboju wrote: > On Tue, 29 Sep 2020 at 01:56, Kees Cook wrote: > > > > Currently with run_kselftest.sh there is no way to choose which test > > we could run. All the tests listed in kselftest-list.txt are all run > > every time. This patch enhanced the run_kselftest.sh to make the test > > collections (or tests) individually selectable. e.g.: > > > > $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep > > > > Additionally adds a way to list all known tests with "-l", usage > > with "-h", and perform a dry run without running tests with "-n". > > > While testing this patch set on LAVA the skip test functionality is not > working. > We may have to revisit test definitions kselftest skip logic > or else > may add one more option to skip a given test on run_kselftest.sh script. > > ref: > https://github.com/Linaro/test-definitions/blob/master/automated/linux/kselftest/kselftest.sh#L196 Yes, LAVA's hack to skip tests needs to be adjusted. Here's what it should probably look like: https://github.com/Linaro/test-definitions/pull/231 -- Kees Cook
Re: [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable
On Tue, 29 Sep 2020 at 01:56, Kees Cook wrote: > > Currently with run_kselftest.sh there is no way to choose which test > we could run. All the tests listed in kselftest-list.txt are all run > every time. This patch enhanced the run_kselftest.sh to make the test > collections (or tests) individually selectable. e.g.: > > $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep > > Additionally adds a way to list all known tests with "-l", usage > with "-h", and perform a dry run without running tests with "-n". While testing this patch set on LAVA the skip test functionality is not working. We may have to revisit test definitions kselftest skip logic or else may add one more option to skip a given test on run_kselftest.sh script. ref: https://github.com/Linaro/test-definitions/blob/master/automated/linux/kselftest/kselftest.sh#L196 - Naresh
Re: [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable
On Tue, 29 Sep 2020 at 01:56, Kees Cook wrote: > > Currently with run_kselftest.sh there is no way to choose which test > we could run. All the tests listed in kselftest-list.txt are all run > every time. This patch enhanced the run_kselftest.sh to make the test > collections (or tests) individually selectable. e.g.: > > $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep > > Additionally adds a way to list all known tests with "-l", usage > with "-h", and perform a dry run without running tests with "-n". > > Co-developed-by: Hangbin Liu > Signed-off-by: Hangbin Liu > Signed-off-by: Kees Cook Tested-by: Naresh Kamboju - Naresh
[PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable
Currently with run_kselftest.sh there is no way to choose which test we could run. All the tests listed in kselftest-list.txt are all run every time. This patch enhanced the run_kselftest.sh to make the test collections (or tests) individually selectable. e.g.: $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep Additionally adds a way to list all known tests with "-l", usage with "-h", and perform a dry run without running tests with "-n". Co-developed-by: Hangbin Liu Signed-off-by: Hangbin Liu Signed-off-by: Kees Cook --- tools/testing/selftests/run_kselftest.sh | 77 ++-- 1 file changed, 71 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh index 8b0ad4766d78..609a4ef9300e 100755 --- a/tools/testing/selftests/run_kselftest.sh +++ b/tools/testing/selftests/run_kselftest.sh @@ -8,21 +8,86 @@ cd $BASE_DIR TESTS="$BASE_DIR"/kselftest-list.txt if [ ! -r "$TESTS" ] ; then echo "$0: Could not find list of tests to run ($TESTS)" >&2 - exit 1 + available="" +else + available="$(cat "$TESTS")" fi -available="$(cat "$TESTS")" . ./kselftest/runner.sh ROOT=$PWD -if [ "$1" = "--summary" ] ; then - logfile="$BASE_DIR"/output.log - cat /dev/null > $logfile +usage() +{ + cat < $logfile + shift ;; + -t | --test) + TESTS="$TESTS $2" + shift 2 ;; + -c | --collection) + COLLECTIONS="$COLLECTIONS $2" + shift 2 ;; + -l | --list) + echo "$available" + exit 0 ;; + -n | --dry-run) + dryrun="echo" + shift ;; + -h | --help) + usage 0 ;; + "") + break ;; + *) + usage 1 ;; + esac +done + +# Add all selected collections to the explicit test list. +if [ -n "$COLLECTIONS" ]; then + for collection in $COLLECTIONS ; do + found="$(echo "$available" | grep "^$collection:")" + if [ -z "$found" ] ; then + echo "No such collection '$collection'" >&2 + exit 1 + fi + TESTS="$TESTS $found" + done +fi +# Replace available test list with explicitly selected tests. +if [ -n "$TESTS" ]; then + valid="" + for test in $TESTS ; do + found="$(echo "$available" | grep "^${test}$")" + if [ -z "$found" ] ; then + echo "No such test '$test'" >&2 + exit 1 + fi + valid="$valid $found" + done + available="$(echo "$valid" | sed -e 's/ /\n/g')" fi collections=$(echo "$available" | cut -d: -f1 | uniq) for collection in $collections ; do [ -w /dev/kmsg ] && echo "kselftest: Running tests in $collection" >> /dev/kmsg tests=$(echo "$available" | grep "^$collection:" | cut -d: -f2) - (cd "$collection" && run_many $tests) + ($dryrun cd "$collection" && $dryrun run_many $tests) done -- 2.25.1