Hi Heinrich, On Mon, 26 Sept 2022 at 00:56, Heinrich Schuchardt <xypron.g...@gmx.de> wrote: > > On 9/25/22 17:02, Simon Glass wrote: > > At present we normally write tests either in Python or in C. But most > > Python tests end up doing a lot of checks which would be better done in C. > > Checks done in C are orders of magnitude faster and it is possible to get > > full access to U-Boot's internal workings, rather than just relying on > > the command line. > > > > The model is to have a Python test set up some things and then use C code > > (in a unit test) to check that they were done correctly. But we don't want > > those checks to happen as part of normal test running, since each C unit > > tests is dependent on the associate Python tests, so cannot run without > > it. > > > > To acheive this, add a new UT_TESTF_MANUAL flag to use with the C 'check' > > tests, so that they can be skipped by default when the 'ut' command is > > used. Require that tests have a name ending with '_norun', so that pytest > > knows to skip them. > > > > Signed-off-by: Simon Glass <s...@chromium.org> > > --- > > > > arch/sandbox/cpu/spl.c | 2 +- > > doc/develop/tests_writing.rst | 22 ++++++++++++++++++++++ > > include/test/test.h | 8 ++++++++ > > include/test/ut.h | 4 +++- > > test/cmd_ut.c | 16 +++++++++++++--- > > test/dm/test-dm.c | 2 +- > > test/py/conftest.py | 8 +++++++- > > test/test-main.c | 27 ++++++++++++++++++++++++++- > > 8 files changed, 81 insertions(+), 8 deletions(-) > > > > diff --git a/arch/sandbox/cpu/spl.c b/arch/sandbox/cpu/spl.c > > index 6d4981152bb..2678370481a 100644 > > --- a/arch/sandbox/cpu/spl.c > > +++ b/arch/sandbox/cpu/spl.c > > @@ -90,7 +90,7 @@ void spl_board_init(void) > > int ret; > > > > ret = ut_run_list("spl", NULL, tests, count, > > - state->select_unittests, 1); > > + state->select_unittests, 1, false); > > /* continue execution into U-Boot */ > > } > > } > > diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst > > index 1ddf7a353a7..d3ffc550300 100644 > > --- a/doc/develop/tests_writing.rst > > +++ b/doc/develop/tests_writing.rst > > @@ -74,6 +74,28 @@ NOT rely on running with sandbox, but instead should > > function correctly on any > > board supported by U-Boot. > > > > > > +Mixing Python and C > > +------------------- > > + > > +The best of both worlds is sometimes to have a Python test set things up > > and > > +perform some operations, with a 'checker' C unit test doing the checks > > +afterwards. This can be achieved with these steps: > > + > > +- Add the `UT_TESTF_MANUAL` flag to the checker test so that the `ut` > > command > > + does not run it by default > > +- Add a `_norun` suffix to the name so that pytest knows to skip it too > > + > > +In your Python test use the `-f` flag to the `ut` command to force the > > checker > > +test to run it, e.g.:: > > + > > + # Do the Python part > > + host load ... > > + bootm ... > > + > > + # Run the checker to make sure that everything worked > > + ut -f bootstd vbe_test_fixup_norun > > Shouldn't the documentation describes what a test looks like that can > only be run manually?
It is just a normal test and I figure there is an example in the code. What sort of info is missing here? Regards, Simon