#32769: Fix for sporadically crashing parallel test runner
-----------------------------------+--------------------------------------
Reporter: quinox | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: 2.2
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+--------------------------------------
Changes (by Mariusz Felisiak):
* cc: Chris Jerdonek (added)
* status: new => closed
* resolution: => duplicate
Comment:
Thanks for the report. I think it's a duplicate of #27734, it's not an
issue with the number of workers, but an issue with a shared counter (see
[https://code.djangoproject.com/ticket/27734#comment:5 comment]). Reducing
the number of workers just make it less likely. Can you reproduce it with
Python 3.8+ and on the main branch? Django 2.2 is in extended support so
it doesn't receive bugfixes anymore (except security patches).
I agree that we could reduce the number of workers but it's only a small
cleanup that will not fix the main issue described in #27734. I would
reduce the number of processes directly in the `ParallelTestSuite`, e.g.
{{{
diff --git a/django/test/runner.py b/django/test/runner.py
index ab5512e628..50f66d3d87 100644
--- a/django/test/runner.py
+++ b/django/test/runner.py
@@ -405,7 +405,9 @@ class ParallelTestSuite(unittest.TestSuite):
def __init__(self, suite, processes, failfast=False, buffer=False):
self.subsuites = partition_suite_by_case(suite)
- self.processes = processes
+ # Since tests are distributed across processes on a per-TestCase
basis,
+ # there's no need for more processes than TestCases.
+ self.processes = min(processes, len(self.subsuites))
self.failfast = failfast
self.buffer = buffer
super().__init__()
@@ -662,14 +664,8 @@ class DiscoverRunner:
self.failfast,
self.buffer,
)
-
- # Since tests are distributed across processes on a per-
TestCase
- # basis, there's no need for more processes than TestCases.
- parallel_units = len(parallel_suite.subsuites)
- self.parallel = min(self.parallel, parallel_units)
-
# If there's only one TestCase, parallelization isn't needed.
- if self.parallel > 1:
+ if parallel_suite.processes > 1:
suite = parallel_suite
return suite
}}}
Feel-free to prepare a patch with `Refs #27734 -- ...`.
Duplicate of #27734.
--
Ticket URL: <https://code.djangoproject.com/ticket/32769#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/064.5e243e562a749f8cde93959ded93ca19%40djangoproject.com.
