Re: [PATCH 2/5] patman: test_util: Handle nonexistent tests while loading tests

2022-06-28 Thread Simon Glass
On Sat, 2 Apr 2022 at 11:06, Alper Nebi Yasak  wrote:
>
> It's possible to request a specific test to run when trying to run a
> python tool's tests. If we request a nonexistent test, the unittest
> loaders generate a fake test that reports this as an error. However, we
> get these fake tests even when the test exists, because test_util can
> load tests from multiple places one by one and the test we want only
> exists in one.
>
> The test_util helpers currently remove these fake tests when printing
> test results, but that's more of a workaround than a proper solution.
> Instead, don't even try to load the missing tests.
>
> Signed-off-by: Alper Nebi Yasak 
> ---
>
>  tools/patman/test_util.py | 21 +
>  1 file changed, 5 insertions(+), 16 deletions(-)

Reviewed-by: Simon Glass 

Applied to u-boot-dm, thanks!


Re: [PATCH 2/5] patman: test_util: Handle nonexistent tests while loading tests

2022-04-19 Thread Simon Glass
On Sat, 2 Apr 2022 at 11:06, Alper Nebi Yasak  wrote:
>
> It's possible to request a specific test to run when trying to run a
> python tool's tests. If we request a nonexistent test, the unittest
> loaders generate a fake test that reports this as an error. However, we
> get these fake tests even when the test exists, because test_util can
> load tests from multiple places one by one and the test we want only
> exists in one.
>
> The test_util helpers currently remove these fake tests when printing
> test results, but that's more of a workaround than a proper solution.
> Instead, don't even try to load the missing tests.
>
> Signed-off-by: Alper Nebi Yasak 
> ---
>
>  tools/patman/test_util.py | 21 +
>  1 file changed, 5 insertions(+), 16 deletions(-)

Reviewed-by: Simon Glass 


[PATCH 2/5] patman: test_util: Handle nonexistent tests while loading tests

2022-04-02 Thread Alper Nebi Yasak
It's possible to request a specific test to run when trying to run a
python tool's tests. If we request a nonexistent test, the unittest
loaders generate a fake test that reports this as an error. However, we
get these fake tests even when the test exists, because test_util can
load tests from multiple places one by one and the test we want only
exists in one.

The test_util helpers currently remove these fake tests when printing
test results, but that's more of a workaround than a proper solution.
Instead, don't even try to load the missing tests.

Signed-off-by: Alper Nebi Yasak 
---

 tools/patman/test_util.py | 21 +
 1 file changed, 5 insertions(+), 16 deletions(-)

diff --git a/tools/patman/test_util.py b/tools/patman/test_util.py
index 8b2220dbbafc..a4c2a2c3c0b2 100644
--- a/tools/patman/test_util.py
+++ b/tools/patman/test_util.py
@@ -110,19 +110,6 @@ def report_result(toolname:str, test_name: str, result: 
unittest.TestResult):
 test_name: Name of test that was run, or None for all
 result: A unittest.TestResult object containing the results
 """
-# Remove errors which just indicate a missing test. Since Python v3.5 If an
-# ImportError or AttributeError occurs while traversing name then a
-# synthetic test that raises that error when run will be returned. These
-# errors are included in the errors accumulated by result.errors.
-if test_name:
-errors = []
-
-for test, err in result.errors:
-if ("has no attribute '%s'" % test_name) not in err:
-errors.append((test, err))
-result.testsRun -= 1
-result.errors = errors
-
 print(result)
 for test, err in result.errors:
 print(test.id(), err)
@@ -184,10 +171,12 @@ def run_test_suites(result, debug, verbosity, 
test_preserve_dirs, processes,
 preserve_outdirs=test_preserve_dirs and test_name is not None,
 toolpath=toolpath, verbosity=verbosity)
 if test_name:
-try:
+# Since Python v3.5 If an ImportError or AttributeError occurs
+# while traversing a name then a synthetic test that raises that
+# error when run will be returned. Check that the requested test
+# exists, otherwise these errors are included in the results.
+if test_name in loader.getTestCaseNames(module):
 suite.addTests(loader.loadTestsFromName(test_name, module))
-except AttributeError:
-continue
 else:
 suite.addTests(loader.loadTestsFromTestCase(module))
 if use_concurrent and processes != 1:
-- 
2.35.1