Hi Heinrich, On Mon, 8 Feb 2021 at 00:25, Heinrich Schuchardt <xypron.g...@gmx.de> wrote: > > On 2/8/21 5:05 AM, Simon Glass wrote: > > Tests are supposed to be independent. With driver model tests, the > > environment is reset before each test, which ensures that. > > > > With Python tests there is no reset of the board between tests, since we > > want to run all the tests as quickly as possible and without needing the > > external scripts running constantly. > > > > In principle the Python tests can be independent if they each put the > > world back the way they found it, but it turns out that some are not. > > This means that some tests cannot be run unless another test is run > > first. It also means that tests cannot be run in parallel, e.g. on > > sandbox. > > > > This series fixes some of them. Those that remain: > > > > test_gpt_swap_partitions - not sure? > > test_pinmux_status - not sure? > > test_sqfs_load - cannot be run more than once! > > test_bind_unbind_with_uclass - relies on previous test > > > > The last one would be much better done as a C test, so it doesn't have > > to deal with the changing driver tree. There isn't a lot of value in > > running the test on a real board, since sandbox should find any bugs > > in driver model or the 'bind' command. > > > > If the above can be resolved we can enable parallel tests. On my test > > machine (32 threads) it reduces the time from 38 seconds to 7.5s > > > > To use this feature: > > pip3 install pytest-xdist > > > > test/py/test.py -B sandbox --build-dir /tmp/xx -q -k 'not slow' -n32 > > Thanks for looking into parallelization. > > What I am missing in this series is a patch for > doc/develop/py_testing.rst describing how parallelization of Python > tests is controlled.
I don't think we can mention that yet, as it doesn't actually work. > > I have seen that test/py/tests/test_fs/test_basic.py test_fs1() is > always failing on my machine because the test file 2.5GB.file is > truncated. It is not truncated if I add some waiting time. > > Could this be caused by parallelization? Not at present since we don't use it. I seldom run those tests. I wonder whether they work once and then not again, i.e. something needs to be reset at the start of that test? Also I have not even tried to parallelise those tests. For me they use too much memory. > > Package 'python3-pytest-xdist' is not installed on my system. OK. Then the -n flag is not available. Regards, Simon