Okay, the implementation makes sense and I see that it works.
However, isn't the whole issue stemming from the fact that the currently
existing get_overall_result() function calls max() on self.children, and
self.children (the result objects) are IntEnum enums which are ordered in
ascending order, starting with PASS, then SKIP, then BLOCK, then FAIL, then
ERROR? If we just swap PASS and SKIP, doesn't that make the existing logic
(using the max() function call) work correctly?
I.e. re-order this class:
class Result(IntEnum):
"""The possible states that a setup, a teardown or a test case may end
up in."""
#:
PASS = auto()
#:
SKIP = auto()
#:
BLOCK = auto()
#:
FAIL = auto()
#:
ERROR = auto()
On Fri, Mar 13, 2026 at 2:22 PM Dean Marx <[email protected]> wrote:
> Update overall result of test suites such that
> when some cases skip and at least one passes,
> the result is a pass instead of a skip. Only
> when all cases skip is the result a skip.
>
> Bugzilla ID: 1899
>
> Signed-off-by: Dean Marx <[email protected]>
> Tested-by: Andrew Bailey <[email protected]>
> Reviewed-by: Andrew Bailey <[email protected]>
> ---
> dts/framework/test_result.py | 29 +++++++++++++++++------------
> 1 file changed, 17 insertions(+), 12 deletions(-)
>
> diff --git a/dts/framework/test_result.py b/dts/framework/test_result.py
> index c6bddc55a9..e05663f90e 100644
> --- a/dts/framework/test_result.py
> +++ b/dts/framework/test_result.py
> @@ -187,18 +187,23 @@ def serialize_model(self) -> dict[str, Any]:
>
> def get_overall_result(self) -> ResultLeaf:
> """The overall result of the underlying results."""
> -
> - def extract_result(value: ResultNode | ResultLeaf) -> ResultLeaf:
> - match value:
> - case ResultNode():
> - return value.get_overall_result()
> - case ResultLeaf():
> - return value
> -
> - return max(
> - (extract_result(child) for child in self.children),
> - default=ResultLeaf(result=Result.PASS),
> - )
> + results = [
> + child.get_overall_result() if isinstance(child, ResultNode)
> else child
> + for child in self.children
> + ]
> + max_result = max(results, default=ResultLeaf(result=Result.PASS))
> +
> + if max_result.result != Result.SKIP:
> + return max_result
> +
> + if any(
> + r.result == Result.PASS
> + for child, r in zip(self.children, results)
> + if not (isinstance(child, ResultNode) and child.label in
> self.__ignore_steps)
> + ):
> + return ResultLeaf(result=Result.PASS)
> +
> + return max_result
>
> def make_summary(self) -> Counter[Result]:
> """Make the summary of the underlying results while ignoring
> special nodes."""
> --
> 2.52.0
>
>
