On 03.11.18 08:54, Craig de Stigter wrote: > However, I note the excellent pytest-xdist plugin is good at spawning worker > subprocesses and sending tests to them, and rather than re-invent the wheel I > wondered if I could make my plugin depend on xdist, and /control/ the xdist > workers (e.g. run one worker with all the tests; if it crashes, run a new > worker with half the tests, etc).
I do not think xdist is a good fit here, since you do not want to run the tests distributed (like you correctly said). I think the order tests is important here, so you might want to look into pytest-randomly (https://github.com/pytest-dev/pytest-randomly). A new plugin could make sense here, where you would record if the test run failed completely in the end, or due to some specific test(s), while running them with pytest-randomly on CI as normal (where builds then get restarted, but the information should be recorded). Then you could build some tree(s) based on this information. I think using pytest-randomly from the start is good to not get into a situation where you appear to be now already.. ;) For debugging this locally `pytest -x` with pytest-randomly might be good already - start looking at fast failures. btw: it reminded me of https://github.com/sixty-north/cosmic-ray (Mutation testing for Python), where you can use celery workers to run mutations of your tests. Maybe this could be useful to just run mutations of the test order? (just my quick 2 cents, not having looked at the PR or the project, but am a user of gdal, so that's great to have covered by pytest soon.. :)) Cheers, Daniel. _______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
