This would be something I'd really like to see. Also it would be nice to make it the default.
I suggest we add the flag as "--random" taking an integer that is the seed, or 0 to disable. Do check out my project pytest-randomly for a battle-tested project randomly shuffling tests. One thing pytest-randomly also does is it resets the random seeds in some well-known libraries, for example factory boy. I don't think that's something we'd want in Django since it's a framework to be built upon. Instead we could just make sure it would be possible to override a DiscoverRunner method in order to use the seed value to reset other libraries' randomization. On Wed, 10 Mar 2021 at 10:13, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: > I'd like to ask folks if they would support reopening ticket #24522, which > is to add a --random option to Django's test runner: > https://code.djangoproject.com/ticket/24522 > > It was opened 6 years ago and closed as won't fix. > > I've been doing some simplifications of the test-suite related code in > Django's DiscoverRunner, and I think a --random option could be added > pretty simply. It seems like every project could benefit by running their > tests in a random order. It would help to unearth hidden dependencies > between tests. > > If I were to implement this, the --random option would accept an optional > integer "seed" argument. If --random is passed with no seed argument, the > seed would be generated randomly by choosing a random integer between 0 and > some large integer. Whether or not an explicit seed value is passed, Django > can log the seed value being used before running tests (if --random was > passed), and DiscoverRunner can store the seed as an attribute like > self.random_order_seed. That would give callers the option of doing > something with the seed programmatically, if logging to stderr isn't enough. > > When --random is used, the order of classes can be randomized within their > groups: > > https://docs.djangoproject.com/en/3.1/topics/testing/overview/#order-in-which-tests-are-executed > and the order of methods within their class can also be randomized. > > The seed can be used by instantiating a Python random.Random() object with > it: > https://docs.python.org/3/library/random.html#random.Random > and the random orders could be generated by calling methods of that > now-deterministic object. > > --Chris > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/CAOTb1wd7YnjmLwS_Gvvj3iGOx_Oodp7Xv%2BwZb_tCJ2Qmy%2BHO5g%40mail.gmail.com > <https://groups.google.com/d/msgid/django-developers/CAOTb1wd7YnjmLwS_Gvvj3iGOx_Oodp7Xv%2BwZb_tCJ2Qmy%2BHO5g%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- Adam -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM2BiFoGPbmOjC9_iEP7_09c-fxOBcV0gnMGHR48_Z1MYA%40mail.gmail.com.
