Re: [PATCH] Documentation: kunit: Update kunit_tool page

2021-04-16 Thread Daniel Latypov
lltests > > Note that this does overlap a little with the new running_tips page. I > don't think it's a problem having both: this page is supposed to be a > bit more of a reference, rather than a list of useful tips, so the fact > that they both describe the same features isn't a probl

[PATCH v2] kunit: add unit test for filtering suites by names

2021-04-16 Thread Daniel Latypov
o includes a clever `kfree_at_end()` helper that makes this test easier to write than it otherwise would have been. Tested by running just the new tests using itself $ ./tools/testing/kunit/kunit.py run '*exec*' Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- v1 -> v2: * Fix

Re: [PATCH v5] lib: add basic KUnit test for lib/math

2021-04-16 Thread Daniel Latypov
On Tue, Apr 13, 2021 at 5:33 PM Daniel Latypov wrote: > > On Mon, Apr 12, 2021 at 11:41 PM David Gow wrote: > > > > On Tue, Apr 13, 2021 at 3:07 AM Daniel Latypov wrote: > > > > > > Add basic test coverage for files that don't require any config option

[PATCH v6] lib: add basic KUnit test for lib/math

2021-04-16 Thread Daniel Latypov
, but they * provide short and simple examples of parameterized tests * provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy * looking at code coverage, we hit all the branches in the .c files Signed-off-by: Daniel Latypov

[PATCH v4] Documentation: kunit: add tips for running KUnit

2021-04-14 Thread Daniel Latypov
ml_abort() in os_dump_core(). I've documented these hacks in "Notes" but left TODOs for brendanhigg...@google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov Reviewed-by: David Gow ---

Re: [PATCH v3] Documentation: kunit: add tips for running KUnit

2021-04-14 Thread Daniel Latypov
On Tue, Apr 13, 2021 at 9:22 PM David Gow wrote: > > On Wed, Apr 14, 2021 at 8:45 AM Daniel Latypov wrote: > > > > This is long overdue. > > > > There are several things that aren't nailed down (in-tree > > .kunitconfig's), or partially broken (GCOV on UML), b

Re: [PATCH v2] Documentation: dev-tools: Add Testing Overview

2021-04-14 Thread Daniel Latypov
worth at least having something. Hopefully > this can form the basis for more detailed documentation later. > > Signed-off-by: David Gow Reviewed-by: Daniel Latypov Looks good to me. Some minor typos and nits about wording here and there. > --- > Thanks, everyone, for the comment

[PATCH v3] Documentation: kunit: add tips for running KUnit

2021-04-13 Thread Daniel Latypov
ml_abort() in os_dump_core(). I've documented these hacks in "Notes" but left TODOs for brendanhigg...@google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov --- v2 -> v3: * Suggest --mak

Re: [PATCH v5] lib: add basic KUnit test for lib/math

2021-04-13 Thread Daniel Latypov
On Mon, Apr 12, 2021 at 11:41 PM David Gow wrote: > > On Tue, Apr 13, 2021 at 3:07 AM Daniel Latypov wrote: > > > > Add basic test coverage for files that don't require any config options: > > * part of math.h (what seem to be the most commonly used macros)

Re: [PATCH] kunit: add unit test for filtering suites by names

2021-04-13 Thread Daniel Latypov
On Mon, Apr 12, 2021 at 10:00 PM David Gow wrote: > > On Tue, Apr 13, 2021 at 8:08 AM Daniel Latypov wrote: > > > > This adds unit tests for kunit_filter_subsuite() and > > kunit_filter_suites(). > > > > Note: what the executor means by "subsuite" is

Re: [PATCH v4 2/3] mm/slub, kunit: add a KUnit test for SLUB debugging functionality

2021-04-13 Thread Daniel Latypov
nt error whenever slab_bug() or slab_fix() is called or when > the count of pages is wrong. > > Signed-off-by: Oliver Glitta Acked-by: Daniel Latypov Looks good to me! My one minor suggestion: perhaps let's log a summary of the error or the func name in slab_add_kunit_errors(). > --

[PATCH v2] Documentation: kunit: add tips for running KUnit

2021-04-12 Thread Daniel Latypov
ml_abort() in os_dump_core(). I've documented these hacks in "Notes" but left TODOs for brendanhigg...@google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov --- Documentation/dev-too

[PATCH] kunit: add unit test for filtering suites by names

2021-04-12 Thread Daniel Latypov
o includes a clever `kfree_at_end()` helper that makes this test easier to write than it otherwise would have been. Tested by running just the new tests using itself $ ./tools/testing/kunit/kunit.py run '*exec*' Signed-off-by: Daniel Latypov --- lib/kunit/executor.c | 26

Re: [PATCH] Documentation: kunit: add tips for running KUnit

2021-04-12 Thread Daniel Latypov
On Mon, Apr 12, 2021 at 1:42 PM Brendan Higgins wrote: > > On Mon, Apr 12, 2021 at 10:27 AM Daniel Latypov wrote: > > > > hOn Fri, Apr 9, 2021 at 9:10 PM David Gow wrote: > > > > > > Thanks for writing this: it's good to have th

[PATCH v5] lib: add basic KUnit test for lib/math

2021-04-12 Thread Daniel Latypov
, but they * provide short and simple examples of parameterized tests * provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy * looking at code coverage, we hit all the branches in the .c files Signed-off-by: Daniel Latypov

Re: [PATCH] Documentation: kunit: add tips for running KUnit

2021-04-12 Thread Daniel Latypov
do it here, so I'm not against removing it. > > Otherwise, a few minor comments and nitpicks: > > -- David > > On Sat, Apr 10, 2021 at 2:01 AM Daniel Latypov wrote: > > > > This is long overdue. > > > > There are several things that aren't nailed down (in

Re: [PATCH] Documentation: dev-tools: Add Testing Overview

2021-04-10 Thread Daniel Latypov
On Sat, Apr 10, 2021 at 12:05 AM David Gow wrote: > > The kernel now has a number of testing and debugging tools, and we've > seen a bit of confusion about what the differences between them are. > > Add a basic documentation outlining the testing tools, when to use each, > and how they interact.

[PATCH] Documentation: kunit: add tips for running KUnit

2021-04-09 Thread Daniel Latypov
m who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/index.rst | 1 + .../dev-tools/kunit/running_tips.rst | 278 ++ Documentation/dev-too

Re: [PATCH v4] lib: add basic KUnit test for lib/math

2021-04-09 Thread Daniel Latypov
On Fri, Apr 9, 2021 at 8:30 AM Andy Shevchenko wrote: > > On Thu, Apr 08, 2021 at 06:40:01PM -0700, Daniel Latypov wrote: > > Add basic test coverage for files that don't require any config options: > > * gcd.c > > * lcm.c > > * int_sqrt.c > > * reciprocal_div

[PATCH v4] lib: add basic KUnit test for lib/math

2021-04-08 Thread Daniel Latypov
* provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy Signed-off-by: Daniel Latypov --- Changes since v3: * fix `checkpatch.pl --strict` warnings * add test cases for gcd(0,0) and lcm(0,0) * minor: don't test both gcd

Re: [PATCH v3 1/2] kunit: add a KUnit test for SLUB debugging functionality

2021-04-08 Thread Daniel Latypov
On Thu, Apr 8, 2021 at 3:30 AM Marco Elver wrote: > > On Tue, 6 Apr 2021 at 12:57, Vlastimil Babka wrote: > > > > > > On 4/1/21 11:24 PM, Marco Elver wrote: > > > On Thu, 1 Apr 2021 at 21:04, Daniel Latypov wrote: > > >> > } > >

[PATCH] Documentation: kunit: add tips for using current->kunit_test

2021-04-06 Thread Daniel Latypov
of kunit_fail_current_test(). Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/tips.rst | 78 +- 1 file changed, 76 insertions(+), 2 deletions(-) diff --git a/Documentation/dev-tools/kunit/tips.rst b/Documentation/dev-tools/kunit/tips.rst index a6ca0af

Re: linux-next: build warning after merge of the kunit-next tree

2021-04-06 Thread Daniel Latypov
Thanks for the catch. Should be addressed by https://lore.kernel.org/linux-kselftest/20210406172901.1729216-1-dlaty...@google.com/ When I was testing the CONFIG_KUNIT=n case, I added it to a file that wasn't being compiled (CONFIG_UBSAN=y is not sufficient for lib/ubsan.c to be compiled...). On

[PATCH] kunit: fix -Wunused-function warning for __kunit_fail_current_test

2021-04-06 Thread Daniel Latypov
When CONFIG_KUNIT is not enabled, __kunit_fail_current_test() an empty static function. But GCC complains about unused static functions, *unless* they're static inline. So add inline to make GCC happy. Signed-off-by: Daniel Latypov Fixes: 359a376081d4 ("kunit: support failure from dy

Re: [PATCH v4 1/2] kunit: support failure from dynamic analysis tools

2021-04-02 Thread Daniel Latypov
On Fri, Apr 2, 2021 at 10:53 AM Shuah Khan wrote: > > On 4/2/21 2:55 AM, Brendan Higgins wrote: > > On Thu, Mar 11, 2021 at 7:23 AM Daniel Latypov wrote: > >> > >> From: Uriel Guajardo > >> > >> Add a kunit_fail_current_test() function

[PATCH v5 1/2] kunit: support failure from dynamic analysis tools

2021-04-02 Thread Daniel Latypov
[15:19:34] not ok 1 - example_simple_test Co-developed-by: Daniel Latypov Signed-off-by: Daniel Latypov Signed-off-by: Uriel Guajardo Reviewed-by: Alan Maguire Reviewed-by: Brendan Higgins --- include/kunit/test-bug.h | 29 + lib/kunit/test.c | 39 +++

[PATCH v5 2/2] kunit: ubsan integration

2021-04-02 Thread Daniel Latypov
() as well, so there's some slight duplication, but it also ensures an error is recorded in the debugfs entry for the running KUnit test. Print a shorter version of the message to make it less spammy. Co-developed-by: Daniel Latypov Signed-off-by: Daniel Latypov Signed-off-by: Uriel Guajardo

[PATCH v5 0/2] kunit: fail tests on UBSAN errors

2021-04-02 Thread Daniel Latypov
v1 by Uriel is here: [1]. Since it's been a while, I've dropped the Reviewed-By's. It depended on commit 83c4e7a0363b ("KUnit: KASAN Integration") which hadn't been merged yet, so that caused some kerfuffle with applying them previously and the series was reverted. This revives the series but

Re: [PATCH] kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals

2021-04-02 Thread Daniel Latypov
On Fri, Apr 2, 2021 at 12:19 PM Shuah Khan wrote: > > On 4/2/21 1:09 PM, Daniel Latypov wrote: > > On Fri, Apr 2, 2021 at 10:47 AM Shuah Khan > > wrote: > >> > >> On 4/2/21 3:35 AM, Brendan Higgins wrote: > >>> On Fri, Feb 5, 2021 at 2:1

[PATCH v2] kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals

2021-04-02 Thread Daniel Latypov
ECT_STREQ(test, "hello", "world") since we don't expect it to realistically happen in checked in tests. (If you really wanted a test to fail, KUNIT_FAIL("msg") exists) In that case, you'd get: > Expected "hello" == "world", but Signed-off-by: Danie

Re: [PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-04-02 Thread Daniel Latypov
that as an > >> example more close to home (and thus less likely to be accidentally > >> broken). > >> > >> [1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/ > >> > >> Signed-off-by: Daniel Latypov > > > > Reviewed-by: B

Re: [PATCH] kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals

2021-04-02 Thread Daniel Latypov
On Fri, Apr 2, 2021 at 10:47 AM Shuah Khan wrote: > > On 4/2/21 3:35 AM, Brendan Higgins wrote: > > On Fri, Feb 5, 2021 at 2:18 PM Daniel Latypov wrote: > >> > >> Before: > >>> Expected str == "world", but > >>> str == hell

Re: [PATCH v3 1/2] kunit: add a KUnit test for SLUB debugging functionality

2021-04-01 Thread Daniel Latypov
On Thu, Apr 1, 2021 at 2:16 AM 'Marco Elver' via KUnit Development wrote: > > [Note, if you'd like me to see future versions, please Cc me, otherwise > it's unlikely I see it in time. Also add kunit-...@googlegroups.com if > perhaps a KUnit dev should have another look, too.] > > On Wed, Mar 31,

[PATCH v4 1/2] kunit: support failure from dynamic analysis tools

2021-03-11 Thread Daniel Latypov
[15:19:34] not ok 1 - example_simple_test Co-developed-by: Daniel Latypov Signed-off-by: Daniel Latypov Signed-off-by: Uriel Guajardo Reviewed-by: Alan Maguire --- include/kunit/test-bug.h | 30 ++ lib/kunit/test.c | 39 +++ 2 files

[PATCH v4 2/2] kunit: ubsan integration

2021-03-11 Thread Daniel Latypov
() as well, so there's some slight duplication, but it also ensures an error is recorded in the debugfs entry for the running KUnit test. Print a shorter version of the message to make it less spammy. Co-developed-by: Daniel Latypov Signed-off-by: Daniel Latypov Signed-off-by: Uriel Guajardo

[PATCH v4 0/2] kunit: fail tests on UBSAN errors

2021-03-11 Thread Daniel Latypov
v1 by Uriel is here: [1]. Since it's been a while, I've dropped the Reviewed-By's. It depended on commit 83c4e7a0363b ("KUnit: KASAN Integration") which hadn't been merged yet, so that caused some kerfuffle with applying them previously and the series was reverted. This revives the series but

Re: [PATCH] kunit: tool: Fix a python tuple typing error

2021-02-22 Thread Daniel Latypov
, this fix does make the code more stylistically correct and should definitely go in. > > Fixes 97752c39bd ("kunit: kunit_tool: Allow .kunitconfig to disable config > items") > Signed-off-by: David Gow Reviewed-by: Daniel Latypov > --- > tools/testing/kunit/

Re: [PATCH] ext4: add .kunitconfig fragment to enable ext4-specific tests

2021-02-22 Thread Daniel Latypov
On Tue, Feb 9, 2021 at 6:33 PM Theodore Ts'o wrote: > > On Tue, Feb 09, 2021 at 05:32:06PM -0800, Daniel Latypov wrote: > > > > After [2]: > > $ ./tools/testing/kunit.py run --kunitconfig=fs/ext4/.kunitconfig > > Any chance that in the future this might become:

[PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-02-22 Thread Daniel Latypov
add in such a fragment for kunit itself so we can give that as an example more close to home (and thus less likely to be accidentally broken). [1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/ Signed-off-by: Daniel Latypov --- lib/kunit/.kunitconfig | 3 +++ tools

Re: [PATCH v3 1/2] kunit: support failure from dynamic analysis tools

2021-02-16 Thread Daniel Latypov
On Thu, Feb 11, 2021 at 1:33 PM 'Brendan Higgins' via KUnit Development wrote: > > On Thu, Feb 11, 2021 at 12:58 PM Daniel Latypov wrote: > > > > On Thu, Feb 11, 2021 at 7:40 AM Alan Maguire > > wrote: > > > > > > On Thu, 11 Feb 2021, David Gow wrote:

Re: [PATCH v3 1/2] kunit: support failure from dynamic analysis tools

2021-02-11 Thread Daniel Latypov
On Thu, Feb 11, 2021 at 7:40 AM Alan Maguire wrote: > > On Thu, 11 Feb 2021, David Gow wrote: > > > On Wed, Feb 10, 2021 at 6:14 AM Daniel Latypov wrote: > > > > > > From: Uriel Guajardo > > > > > > Add a kunit_fail_current_test() functi

Re: [PATCH] ext4: add .kunitconfig fragment to enable ext4-specific tests

2021-02-09 Thread Daniel Latypov
On Tue, Feb 9, 2021 at 6:33 PM Theodore Ts'o wrote: > > On Tue, Feb 09, 2021 at 05:32:06PM -0800, Daniel Latypov wrote: > > > > After [2]: > > $ ./tools/testing/kunit.py run --kunitconfig=fs/ext4/.kunitconfig > > Any chance that in the future this might become:

[PATCH] ext4: add .kunitconfig fragment to enable ext4-specific tests

2021-02-09 Thread Daniel Latypov
3180f5924ed27ea417db39feb7f9691777688e * 372/5556 directories isn't too much, but still not a small number: $ find -type f -name 'k*' | xargs dirname | sort -u | wc -l 372 Signed-off-by: Daniel Latypov --- fs/ext4/.kunitconfig | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 fs/ext4/.k

[PATCH v3 1/2] kunit: support failure from dynamic analysis tools

2021-02-09 Thread Daniel Latypov
[15:19:34] not ok 1 - example_simple_test Co-developed-by: Daniel Latypov Signed-off-by: Uriel Guajardo Signed-off-by: Daniel Latypov --- include/kunit/test-bug.h | 30 ++ lib/kunit/test.c | 37 + 2 files changed, 63 insertions(+), 4 de

[PATCH v3 0/2] kunit: fail tests on UBSAN errors

2021-02-09 Thread Daniel Latypov
v1 by Uriel is here: [1]. Since it's been a while, I've dropped the Reviewed-By's. It depended on commit 83c4e7a0363b ("KUnit: KASAN Integration") which hadn't been merged yet, so that caused some kerfuffle with applying them previously and the series was reverted. This revives the series but

Re: [PATCH v2 1/2] kunit: support failure from dynamic analysis tools

2021-02-09 Thread Daniel Latypov
On Tue, Feb 9, 2021 at 2:12 PM Alan Maguire wrote: > > On Tue, 9 Feb 2021, Daniel Latypov wrote: > > > On Tue, Feb 9, 2021 at 9:26 AM Alan Maguire wrote: > > > > > > On Fri, 5 Feb 2021, Daniel Latypov wrote: > > > > > > > From: Uriel G

[PATCH v3 2/2] kunit: ubsan integration

2021-02-09 Thread Daniel Latypov
() as well, so there's some slight duplication, but it also ensures an error is recorded in the debugfs entry for the running KUnit test. Print a shorter version of the message to make it less spammy. Co-developed-by: Daniel Latypov Signed-off-by: Uriel Guajardo Signed-off-by: Daniel Latypov

Re: [PATCH v2 1/2] kunit: support failure from dynamic analysis tools

2021-02-09 Thread Daniel Latypov
On Tue, Feb 9, 2021 at 9:26 AM Alan Maguire wrote: > > On Fri, 5 Feb 2021, Daniel Latypov wrote: > > > From: Uriel Guajardo > > > > Add a kunit_fail_current_test() function to fail the currently running > > test, if any, with an error message. > > &

[PATCH v2] kunit: don't show `1 == 1` in failed assertion messages

2021-02-05 Thread Daniel Latypov
ns to an integer literal as the right hand side. Signed-off-by: Daniel Latypov Tested-by: David Gow Reviewed-by: Brendan Higgins --- lib/kunit/assert.c | 39 +-- 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/lib/kunit/assert.c b/lib/kunit/asse

[PATCH v4 2/3] kunit: tool: add support for filtering suites by glob

2021-02-05 Thread Daniel Latypov
y supports filtering by suite name. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins --- tools/testing/kunit/kunit.py | 21 - tools/testing/kunit/kunit_kernel.py| 4 +++- tools/testing/kunit/kunit_tool_test.py | 15 +-- 3 files changed, 28 inserti

[PATCH v4 3/3] kunit: tool: fix unintentional statefulness in run_kernel()

2021-02-05 Thread Daniel Latypov
o trigger any negative side-effects right now. Fixes: 6ebf5866f2e8 ("kunit: tool: add Python wrappers for running KUnit tests") Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins --- tools/testing/kunit/kunit_kernel.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion

[PATCH v4 0/3] kunit: support running subsets of test suites from kunit.py

2021-02-05 Thread Daniel Latypov
ray in suite_set. Found by Dan Carpenter and kernel test robot. v2 -> v3: Add MODULE_PARM_DESC() for kunit.filter_glob. v3 -> v4: Rebase on top of kunit_tool_test.py and typing fixes for merging. Daniel Latypov (3): kunit: add kunit.filter_glob cmdline option to filter suites kunit:

[PATCH v2 2/2] kunit: ubsan integration

2021-02-05 Thread Daniel Latypov
() as well, so there's some slight duplication, but it also ensures an error is recorded in the debugfs entry for the running KUnit test. Print a shorter version of the message to make it less spammy. Co-developed-by: Daniel Latypov Signed-off-by: Uriel Guajardo Signed-off-by: Daniel Latypov

[PATCH v4 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-05 Thread Daniel Latypov
-test.*del* But at the moment, it's far easier to manually comment out test cases in test files as opposed to messing with sets of Kconfig entries to select specific suites. So even just doing this makes using kunit far less annoying. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins ---

[PATCH v2 1/2] kunit: support failure from dynamic analysis tools

2021-02-05 Thread Daniel Latypov
1 - example_simple_test Co-developed-by: Daniel Latypov Signed-off-by: Uriel Guajardo Signed-off-by: Daniel Latypov --- include/kunit/test-bug.h | 30 ++ lib/kunit/test.c | 36 2 files changed, 62 insertions(+), 4 de

[PATCH] kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals

2021-02-05 Thread Daniel Latypov
ECT_STREQ(test, "hello", "world") since we don't expect it to realistically happen in checked in tests. (If you really wanted a test to fail, KUNIT_FAIL("msg") exists) In that case, you'd get: > Expected "hello" == "world", but Signed-off-by: Dani

[PATCH v2 0/2] kunit: fail tests on UBSAN errors

2021-02-05 Thread Daniel Latypov
v1 by Uriel is here: [1]. Since it's been a while, I've dropped the Reviewed-By's. It depended on commit 83c4e7a0363b ("KUnit: KASAN Integration") which hadn't been merged yet, so that caused some kerfuffle with applying them previously and the series was reverted. This revives the series but

[PATCH v3 3/3] kunit: tool: fix unintentional statefulness in run_kernel()

2021-02-04 Thread Daniel Latypov
o trigger any negative side-effects right now. Fixes: 6ebf5866f2e8 ("kunit: tool: add Python wrappers for running KUnit tests") Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins --- tools/testing/kunit/kunit_kernel.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion

[PATCH v3 2/3] kunit: tool: add support for filtering suites by glob

2021-02-04 Thread Daniel Latypov
y supports filtering by suite name. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins --- tools/testing/kunit/kunit.py| 21 - tools/testing/kunit/kunit_kernel.py | 4 +++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/tools/testing/kunit/kunit.

[PATCH v3 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-04 Thread Daniel Latypov
-test.*del* But at the moment, it's far easier to manually comment out test cases in test files as opposed to messing with sets of Kconfig entries to select specific suites. So even just doing this makes using kunit far less annoying. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins ---

[PATCH v3 0/3] kunit: support running subsets of test suites from kunit.py

2021-02-04 Thread Daniel Latypov
ray in suite_set. Found by Dan Carpenter and kernel test robot. v2 -> v3: Add MODULE_PARM_DESC() for kunit.filter_glob. Daniel Latypov (3): kunit: add kunit.filter_glob cmdline option to filter suites kunit: tool: add support for filtering suites by glob kunit: tool: fix unintentional statef

Re: [PATCH 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-04 Thread Daniel Latypov
On Wed, Feb 3, 2021 at 11:13 PM Dan Carpenter wrote: > > Hi Daniel, > > url: > https://github.com/0day-ci/linux/commits/Daniel-Latypov/kunit-support-running-subsets-of-test-suites-from/20210204-074405 > base: 88bb507a74ea7d75fa49edd421eaa710a7d80598 > config: x86_64-ran

[PATCH v2 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-04 Thread Daniel Latypov
-test.*del* But at the moment, it's far easier to manually comment out test cases in test files as opposed to messing with sets of Kconfig entries to select specific suites. So even just doing this makes using kunit far less annoying. Signed-off-by: Daniel Latypov --- lib/kunit/Kconfig| 1 +

[PATCH v2 3/3] kunit: tool: fix unintentional statefulness in run_kernel()

2021-02-04 Thread Daniel Latypov
o trigger any negative side-effects right now. Fixes: 6ebf5866f2e8 ("kunit: tool: add Python wrappers for running KUnit tests") Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_kernel.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/testing/kunit/k

[PATCH v2 2/3] kunit: tool: add support for filtering suites by glob

2021-02-04 Thread Daniel Latypov
y supports filtering by suite name. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 21 - tools/testing/kunit/kunit_kernel.py | 4 +++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kuni

[PATCH v2 0/3] kunit: support running subsets of test suites from kunit.py

2021-02-04 Thread Daniel Latypov
ray in suite_set. Found by Dan Carpenter and kernel test robot. Daniel Latypov (3): kunit: add kunit.filter_glob cmdline option to filter suites kunit: tool: add support for filtering suites by glob kunit: tool: fix unintentional statefulness in run_kernel() lib/kunit/Kcon

[PATCH 3/3] kunit: tool: fix unintentional statefulness in run_kernel()

2021-02-03 Thread Daniel Latypov
o trigger any negative side-effects right now. Fixes: 6ebf5866f2e8 ("kunit: tool: add Python wrappers for running KUnit tests") Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_kernel.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/testing/kunit/k

[PATCH 2/3] kunit: tool: add support for filtering suites by glob

2021-02-03 Thread Daniel Latypov
y supports filtering by suite name. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 21 - tools/testing/kunit/kunit_kernel.py | 4 +++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kuni

[PATCH 0/3] kunit: support running subsets of test suites from

2021-02-03 Thread Daniel Latypov
work. This patch series also includes a bugfix for a latent bug that can't be triggered right now but has worse consequences as a result of the changes needed to plumb in this suite name glob. [1] https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlaty...@google.com/ Daniel

[PATCH 1/3] kunit: add kunit.filter_glob cmdline option to filter suites

2021-02-03 Thread Daniel Latypov
-test.*del* But at the moment, it's far easier to manually comment out test cases in test files as opposed to messing with sets of Kconfig entries to select specific suites. So even just doing this makes using kunit far less annoying. Signed-off-by: Daniel Latypov --- lib/kunit/Kconfig| 1 +

[PATCH v2] kunit: make kunit_tool accept optional path to .kunitconfig fragment

2021-02-01 Thread Daniel Latypov
in the same directory as the code-under-test, so it feels more natural to allow the kunitconfig fragments to live anywhere. (Though, people could create a separate directory if wanted; this patch imposes no restrictions on the path). Signed-off-by: Daniel Latypov --- Changes since v1: change from

Re: [PATCH] kunit: don't show `1 == 1` in failed assertion messages

2021-01-29 Thread Daniel Latypov
On Thu, Jan 28, 2021 at 8:51 PM David Gow wrote: > > On Fri, Jan 29, 2021 at 10:26 AM Daniel Latypov wrote: > > > > Currently, given something (fairly dystopian) like > > > KUNIT_EXPECT_EQ(test, 2 + 2, 5) > > > > KUnit will prints a failure messag

Re: [PATCH] kunit: make kunit_tool accept optional path to .kunitconfig fragment

2021-01-29 Thread Daniel Latypov
On Thu, Jan 28, 2021 at 10:33 PM David Gow wrote: > > On Sat, Jan 23, 2021 at 8:17 AM Daniel Latypov wrote: > > > > Currently running tests via KUnit tool means tweaking a .kunitconfig > > file, which you'd keep around locally and never commit. > > This chan

[PATCH] kunit: don't show `1 == 1` in failed assertion messages

2021-01-28 Thread Daniel Latypov
ns to an integer literal as the right hand side. Signed-off-by: Daniel Latypov --- lib/kunit/assert.c | 39 +-- 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c index 33acdaa28a7d..e0ec7d6fed6f 100644 ---

[PATCH] Documentation: kunit: add tips.rst for small examples

2021-01-26 Thread Daniel Latypov
alloc() instead of kzalloc() so you don't have to worry about calling kfree() yourself" and the like. Change start.rst to point users to this new page first and let them know that usage.rst is more of optional further reading. Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/index

[PATCH] kunit: make kunit_tool accept optional path to .kunitconfig fragment

2021-01-22 Thread Daniel Latypov
directory as the code-under-test, so it feels more natural to allow the kunitconfig fragments to live anywhere. (Though, people could create a separate directory if wanted; this patch imposes no restrictions on the path). Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py | 9

[PATCH] KUnit: Docs: make start.rst example Kconfig follow style.rst

2021-01-19 Thread Daniel Latypov
The primary change is that we want to encourage people to respect KUNIT_ALL_TESTS to make it easy to run all the relevant tests for a given config. Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/start.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git

[PATCH v3] lib: add basic KUnit test for lib/math

2021-01-19 Thread Daniel Latypov
* provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy Signed-off-by: Daniel Latypov --- Changes since v2: mv math_test.c => math_kunit.c Changes since v1: * Rebase and rewrite to use the new parameterized test

Re: [PATCH v2] lib: add basic KUnit test for lib/math

2021-01-19 Thread Daniel Latypov
On Mon, Jan 18, 2021 at 2:36 AM Andy Shevchenko wrote: > > On Thu, Jan 14, 2021 at 10:27:00AM -0800, Daniel Latypov wrote: > > Add basic test coverage for files that don't require any config options: > > * gcd.c > > * lcm.c > > * int_sqrt.c > > * reciprocal_div

[PATCH v4 2/3] kunit: tool: fix minor typing issue with None status

2021-01-14 Thread Daniel Latypov
empty"). Also slightly simplify the code and add type annotations. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit_parser.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/testing/kunit/kunit_parser.py b/too

[PATCH v4 3/3] kunit: tool: move kunitconfig parsing into __init__, make it optional

2021-01-14 Thread Daniel Latypov
/want to load the config. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins --- tools/testing/kunit/kunit.py| 20 tools/testing/kunit/kunit_kernel.py | 25 + 2 files changed, 17 insertions(+), 28 deletions(-) diff --git a/tools/testing

[PATCH v4 1/3] kunit: tool: surface and address more typing issues

2021-01-14 Thread Daniel Latypov
ypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- Changes since v1: none here, reworked last patch. Changes since v2: rebased onto torvalds/master. Changes since v3: rebased again torva

[PATCH v2] lib: add basic KUnit test for lib/math

2021-01-14 Thread Daniel Latypov
* are written so adding new test cases to cover edge cases should be easy Signed-off-by: Daniel Latypov --- Changes since v1: * Rebase and rewrite to use the new parameterized testing support. * misc: fix overflow in literal and inline int_sqrt format string. * related: commit 1f0e943df68a

[PATCH v3 2/3] kunit: tool: fix minor typing issue with None status

2021-01-07 Thread Daniel Latypov
empty"). Also slightly simplify the code and add type annotations. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit_parser.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/testing/kunit/kunit_parser.py b/too

[PATCH v3 3/3] kunit: tool: move kunitconfig parsing into __init__, make it optional

2021-01-07 Thread Daniel Latypov
/want to load the config. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 20 tools/testing/kunit/kunit_kernel.py | 25 + 2 files changed, 17 insertions(+), 28 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing

[PATCH v3 1/3] kunit: tool: surface and address more typing issues

2021-01-07 Thread Daniel Latypov
ypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit.py| 14 - tools/testing/kunit/kunit_config.py | 7 +++-- tools/testing/kunit/kunit_json.py |

[PATCH] Documentation: kunit: include example of a parameterized test

2020-12-15 Thread Daniel Latypov
feature. Signed-off-by: Daniel Latypov --- Documentation/dev-tools/kunit/usage.rst | 57 + 1 file changed, 57 insertions(+) diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst index d9fdc14f0677..650f99590df5 100644 --- a/Documenta

Re: [PATCH] kunit: Print test statistics on failure

2020-12-11 Thread Daniel Latypov
On Thu, Dec 10, 2020 at 11:23 PM David Gow wrote: > > When a number of tests fail, it can be useful to get higher-level > statistics of how many tests are failing (or how many parameters are > failing in parameterised tests), and in what cases or suites. This is > already done by some non-KUnit

[PATCH v2 3/3] kunit: tool: move kunitconfig parsing into __init__, make it optional

2020-12-11 Thread Daniel Latypov
/want to load the config. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 20 tools/testing/kunit/kunit_kernel.py | 25 + 2 files changed, 17 insertions(+), 28 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing

[PATCH v2 1/3] kunit: tool: surface and address more typing issues

2020-12-11 Thread Daniel Latypov
ypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit.py| 14 - tools/testing/kunit/kunit_config.py | 7 +++-- tools/testing/kunit/kunit_json.py |

[PATCH v2 2/3] kunit: tool: fix minor typing issue with None status

2020-12-11 Thread Daniel Latypov
empty"). Also slightly simplify the code and add type annotations. Signed-off-by: Daniel Latypov Reviewed-by: David Gow --- tools/testing/kunit/kunit_parser.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/testing/kunit/kunit_parser.py b/too

[PATCH] kunit: tool: simplify kconfig is_subset_of() logic

2020-12-08 Thread Daniel Latypov
Don't use an O(nm) algorithm* and make it more readable by using a dict. *Most obviously, it does a nested for-loop over the entire other config. A bit more subtle, it calls .entries(), which constructs a set from the list for _every_ outer iteration. Signed-off-by: Daniel Latypov --- tools

Re: [PATCH 3/3] kunit: tool: move kunitconfig parsing into __init__

2020-12-04 Thread Daniel Latypov
On Thu, Dec 3, 2020 at 7:57 PM David Gow wrote: > > On Fri, Dec 4, 2020 at 3:41 AM Daniel Latypov wrote: > > > > LinuxSourceTree will unceremoniously crash if the user doesn't call > > read_kunitconfig() first in a number of functions. > > This patch seems to partly

Re: [PATCH 2/3] kunit: tool: fix minor typing issue with None status

2020-12-04 Thread Daniel Latypov
On Thu, Dec 3, 2020 at 8:17 PM David Gow wrote: > > On Fri, Dec 4, 2020 at 3:41 AM Daniel Latypov wrote: > > > > This seems good to me, but I have a few questions, particularly around > the description. > > > The code to handle aggregating statuses didn't check th

[PATCH 2/3] kunit: tool: fix minor typing issue with None status

2020-12-03 Thread Daniel Latypov
nnotations. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_parser.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 24954bbc9baf..97e070506c31 100644 --- a/tools/tes

[PATCH 1/3] kunit: tool: surface and address more typing issues

2020-12-03 Thread Daniel Latypov
ypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 14 - tools/testing/kunit/kunit_config.py | 7 +++-- tools/testing/kunit/kunit_json.py | 2 +- tools/testing/ku

[PATCH 3/3] kunit: tool: move kunitconfig parsing into __init__

2020-12-03 Thread Daniel Latypov
and to reduce copy-paste. The https://github.com/google/pytype type-checker complained that _config wasn't initialized. With this, kunit_tool now type checks under both pytype and mypy. Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit.py| 20 tools

Re: [PATCH v2 1/4] kunit: tool: fix unit test cleanup handling

2020-12-02 Thread Daniel Latypov
On Wed, Dec 2, 2020 at 7:05 PM David Gow wrote: > > On Thu, Dec 3, 2020 at 3:09 AM Daniel Latypov wrote: > > > > * Stop leaking file objects. > > * Use self.addCleanup() to ensure we call cleanup functions even if > > setUp() fails. > > * use mock.patch.stopal

Re: [PATCH 2/5] kunit: tool: fix unit test so it can run from non-root dir

2020-12-02 Thread Daniel Latypov
On Tue, Dec 1, 2020 at 8:41 PM David Gow wrote: > > On Wed, Dec 2, 2020 at 3:00 AM Daniel Latypov wrote: > > > > On Mon, Nov 30, 2020 at 11:33 PM David Gow wrote: > > > > > > On Tue, Dec 1, 2020 at 7:33 AM Daniel Latypov wrote: > > > > >

[PATCH v2 1/4] kunit: tool: fix unit test cleanup handling

2020-12-02 Thread Daniel Latypov
* Stop leaking file objects. * Use self.addCleanup() to ensure we call cleanup functions even if setUp() fails. * use mock.patch.stopall instead of more error-prone manual approach Signed-off-by: Daniel Latypov --- tools/testing/kunit/kunit_tool_test.py | 14 ++ 1 file changed, 6

  1   2   >