This doesn't answer the question I asked: does the pytest command
return a zero exit code when some of the tests fail? If so, that is
incorrect behaviour.
It's expected that we can catch the overall failiure from the exit
code even if there's no output that can point to specific test cases
failing (because upstreams change the output without telling us, sed
processing in run-ptest quietly regress etc).
Alex
On Thu, 14 Dec 2023 at 04:35, Yu, Mingli <mingli...@windriver.com> wrote:
>
> Hi Alex,
>
> The single command is enough to indicate the build failure such as:
> root@qemux86-64:/usr/lib64/python3-license-expression/ptest# ./run-ptest
> [snip]
> PASS:
> tests/test_license_expression.py::CombineExpressionTest::test_combine_expressions_with_empty_input
> PASS:
> tests/test_license_expression.py::CombineExpressionTest::test_combine_expressions_with_or_relationship
> PASS:
> tests/test_license_expression.py::CombineExpressionTest::test_combine_expressions_with_regular
> FAIL: tests/test_skeleton_codestyle.py::BaseTests::test_skeleton_codestyle
>
> =================================== FAILURES
> ===================================
> ______________________ BaseTests.test_skeleton_codestyle
> _______________________
>
> self = <test_skeleton_codestyle.BaseTests testMethod=test_skeleton_codestyle>
>
> def test_skeleton_codestyle(self):
> """
> This test shouldn't run in proliferated repositories.
> """
> setup_cfg = configparser.ConfigParser()
> setup_cfg.read("setup.cfg")
> > if setup_cfg["metadata"]["name"] != "skeleton":
>
> tests/test_skeleton_codestyle.py:22:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
>
> self = <configparser.ConfigParser object at 0x7f04c0473610>, key = 'metadata'
>
> def __getitem__(self, key):
> if key != self.default_section and not self.has_section(key):
> > raise KeyError(key)
> E KeyError: 'metadata'
>
> ../../python3.11/configparser.py:979: KeyError
> =========================== short test summary info
> ============================
> FAIL tests/test_skeleton_codestyle.py::BaseTests::test_skeleton_codestyle
> ======================== 1 failed, 174 passed in 7.08s
> =========================
>
> Thanks,
> ________________________________
> From: Alexander Kanavin <alex.kana...@gmail.com>
> Sent: Thursday, December 14, 2023 00:39
> To: Yu, Mingli <mingli...@windriver.com>
> Cc: openembedded-core@lists.openembedded.org
> <openembedded-core@lists.openembedded.org>
> Subject: Re: [OE-core] [PATCH] python3-license-expression: Fix the ptest
> failure
>
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and
> know the content is safe.
>
> I see that run-ptest consists of a single pytest command; does that
> command return a zero exit code even it fails?
>
> Alex
>
> On Wed, 13 Dec 2023 at 17:38, Alexander Kanavin <alex.kana...@gmail.com>
> wrote:
> >
> > Why isn't the ptest failure reported by the autobuilder? Shouldn't we
> > first fix the ptest so that the failure is reported correctly, if it
> > starts occurring again?
> >
> > Alex
> >
> > On Wed, 13 Dec 2023 at 11:21, Yu, Mingli <mingli...@eng.windriver.com>
> > wrote:
> > >
> > > From: Mingli Yu <mingli...@windriver.com>
> > >
> > > Fix the below ptest failure:
> > > self = <test_skeleton_codestyle.BaseTests
> > > testMethod=test_skeleton_codestyle>
> > > def test_skeleton_codestyle(self):
> > > """
> > > This test shouldn't run in proliferated repositories.
> > > """
> > > setup_cfg = configparser.ConfigParser()
> > > setup_cfg.read("setup.cfg")
> > > > if setup_cfg["metadata"]["name"] != "skeleton":
> > > tests/test_skeleton_codestyle.py:22:
> > > self = <configparser.ConfigParser object at 0x7fd87cf55110>, key =
> > > 'metadata'
> > > def _getitem_(self, key):
> > > if key != self.default_section and not self.has_section(key):
> > > > raise KeyError(key)
> > > E KeyError: 'metadata'
> > >
> > > After the patch:
> > > # ./run-ptest
> > > [snip]
> > > PASS:
> > > tests/test_license_expression.py::LicensingValidateTest::test_validation_bad_syntax
> > > PASS:
> > > tests/test_license_expression.py::LicensingValidateTest::test_validation_exception_as_regular_key
> > > PASS:
> > > tests/test_license_expression.py::LicensingValidateTest::test_validation_exception_with_choice
> > > PASS:
> > > tests/test_license_expression.py::LicensingValidateTest::test_validation_invalid_license_exception
> > > PASS:
> > > tests/test_license_expression.py::LicensingValidateTest::test_validation_invalid_license_exception_strict_false
> > > PASS:
> > > tests/test_license_expression.py::LicensingValidateTest::test_validation_invalid_license_key
> > > PASS: tests/test_license_expression.py::UtilTest::test_build_licensing
> > > PASS:
> > > tests/test_license_expression.py::UtilTest::test_build_spdx_licensing
> > > PASS:
> > > tests/test_license_expression.py::UtilTest::test_get_license_key_info
> > > PASS:
> > > tests/test_license_expression.py::UtilTest::test_get_license_key_info_vendored
> > > PASS:
> > > tests/test_license_expression.py::CombineExpressionTest::test_combine_expressions_with_duplicated_elements
> > > PASS:
> > > tests/test_license_expression.py::CombineExpressionTest::test_combine_expressions_with_empty_input
> > > PASS:
> > > tests/test_license_expression.py::CombineExpressionTest::test_combine_expressions_with_or_relationship
> > > PASS:
> > > tests/test_license_expression.py::CombineExpressionTest::test_combine_expressions_with_regular
> > > PASS:
> > > tests/test_skeleton_codestyle.py::BaseTests::test_skeleton_codestyle
> > >
> > > ============================= 175 passed in 10.36s
> > > =============================
> > >
> > > Signed-off-by: Mingli Yu <mingli...@windriver.com>
> > > ---
> > > .../python/python3-license-expression/run-ptest | 2 +-
> > > .../python/python3-license-expression_30.1.1.bb | 1 +
> > > 2 files changed, 2 insertions(+), 1 deletion(-)
> > >
> > > diff --git
> > > a/meta/recipes-devtools/python/python3-license-expression/run-ptest
> > > b/meta/recipes-devtools/python/python3-license-expression/run-ptest
> > > index 5cec711696..3385d68939 100644
> > > --- a/meta/recipes-devtools/python/python3-license-expression/run-ptest
> > > +++ b/meta/recipes-devtools/python/python3-license-expression/run-ptest
> > > @@ -1,3 +1,3 @@
> > > #!/bin/sh
> > >
> > > -pytest
> > > +pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'|
> > > sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e
> > > 's/SKIPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" ||
> > > $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'|
> > > awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" ||
> > > $NF=="XPASS") {$NF="";print $0}else{print}}'
> > > diff --git
> > > a/meta/recipes-devtools/python/python3-license-expression_30.1.1.bb
> > > b/meta/recipes-devtools/python/python3-license-expression_30.1.1.bb
> > > index 31fb88d6e5..5d3923d487 100644
> > > --- a/meta/recipes-devtools/python/python3-license-expression_30.1.1.bb
> > > +++ b/meta/recipes-devtools/python/python3-license-expression_30.1.1.bb
> > > @@ -33,4 +33,5 @@ do_install_ptest() {
> > > install -d ${D}${PTEST_PATH}/src
> > > cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/
> > > cp -rf ${S}/src/* ${D}${PTEST_PATH}/src/
> > > + cp -rf ${S}/setup.cfg ${D}${PTEST_PATH}/
> > > }
> > > --
> > > 2.25.1
> > >
> > >
> > >
> > >
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192338):
https://lists.openembedded.org/g/openembedded-core/message/192338
Mute This Topic: https://lists.openembedded.org/mt/103146950/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
