[issue22813] No facility for test randomisation
Changes by Ezio Melotti ezio.melo...@gmail.com: -- stage: - needs patch type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22813 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22813] No facility for test randomisation
Michael Foord added the comment: The point is that it is easy to have unintentional dependencies between tests. Test a sets up some state that test b relies on. This means that test b passes, so long as test a has already run. This is bad, tests should be isolated - it also means you can break test b when you change test a. Randomising test run order means you discover these unintentional dependencies earlier. With test randomisation you ideally need the seed to be displayed as part of the test run, and you need to be able to run with a particular seed. This enables you to reproduce failures, or odd results, from any particular test run. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22813 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22813] No facility for test randomisation
sbspider added the comment: Right makes sense. I'll see what I can do. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22813 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22813] No facility for test randomisation
sbspider added the comment: And what do you think would it be useful for? I mean, it shouldn't be too hard to implement, a couple of hours at most, but I can't seem to see what would be the purpose. However, if you think that there is enough of a use case behind it, then I would be happy to have a crack at it :). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22813 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22813] No facility for test randomisation
New submission from Robert Collins: Unittest doesn't support a test randomisation feature. Such a feature should support: - passing in a seed (to allow reproducing the order for debugging) - preserving the suite hierarchy, to preserve class and module setUp performance optimisations - and randomising globally with cloned suite hierarchies, for more comprehensive randomisation - allowing some scopes to opt out of randomisation (where tests really have dependencies on execution order and thats what the test author wants) From https://code.google.com/p/unittest-ext/issues/detail?id=6 -- components: Library (Lib) messages: 230778 nosy: rbcollins priority: normal severity: normal status: open title: No facility for test randomisation versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22813 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22813] No facility for test randomisation
Changes by Terry J. Reedy tjre...@udel.edu: -- nosy: +ezio.melotti, michael.foord ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22813 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22813] No facility for test randomisation
sbspider added the comment: To clarify - are you querying about a) Randomiser for tests, so that tests can have random variables or b) That the order in which tests are called should have the option to be randomized ? -- nosy: +sbspider ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22813 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22813] No facility for test randomisation
Robert Collins added the comment: b) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22813 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
