#36770: SQLite threading tests are flaky when parallel test suite runs in
forkserver/spawn
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Type:
| Cleanup/optimization
Status: new | Component: Testing
| framework
Version: 5.2 | Severity: Normal
Keywords: 3.14, forkserver, | Triage Stage:
spawn, parallel | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
We have two tests often failing on GitHub Actions CI runs under the
parallel test runner having to do with threading and sqlite in-memory
databases.
- `backends.sqlite.tests.ThreadSharing.test_database_sharing_in_threads`
-
`servers.tests.LiveServerInMemoryDatabaseLockTest.test_in_memory_database_lock`
As of now, the failures are most common on the byte-compiled Django
workflow, but we've at least seen the `test_in_memory_database_lock`
failure on other workflows.
Tracebacks:
{{{#!py
======================================================================
FAIL: test_database_sharing_in_threads
(backends.sqlite.tests.ThreadSharing.test_database_sharing_in_threads)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/opt/hostedtoolcache/Python/3.14.0/x64/lib/python3.14/unittest/case.py",
line 58, in testPartExecutor
yield
File
"/opt/hostedtoolcache/Python/3.14.0/x64/lib/python3.14/unittest/case.py",
line 669, in run
self._callTestMethod(testMethod)
File
"/opt/hostedtoolcache/Python/3.14.0/x64/lib/python3.14/unittest/case.py",
line 615, in _callTestMethod
result = method()
^^^^^^^^^^^^^^^
File "/home/runner/work/django/django/tests/backends/sqlite/tests.py",
line 282, in test_database_sharing_in_threads
self.assertEqual(Object.objects.count(), 2)
^^^^^^^^^^^
File
"/opt/hostedtoolcache/Python/3.14.0/x64/lib/python3.14/unittest/case.py",
line 925, in assertEqual
assertion_func(first, second, msg=msg)
^^^^^^^^^^^^^^^
File
"/opt/hostedtoolcache/Python/3.14.0/x64/lib/python3.14/unittest/case.py",
line 918, in _baseAssertEqual
raise self.failureException(msg)
^^^^^^^^^^^
AssertionError: 1 != 2
----------------------------------------------------------------------
}}}
{{{#!py
test_in_memory_database_lock
(servers.tests.LiveServerInMemoryDatabaseLockTest.test_in_memory_database_lock)
failed:
AssertionError('Unexpected error due to a database lock.')
}}}
Other times, the workflow deadlocks, so we don't know which test failed,
which caused us to add `timeout-minutes: 60` everywhere defensively in
e48527f91d341c85a652499a5baaf725d36ae54f.
This failure started manifesting after we upgraded more CI jobs to Python
3.14, which defaults POSIX systems to the `forkserver` multiprocessing
mode. See #36531.
I haven't been successful reproducing locally.
I have a low-confidence hypothesis that it might be something do with
calls to `setup_worker_connection` inside `_init_worker` that occur during
the middle of the test runs when there are resource contentions. Is it
possible that a late worker init is clobbering some of these particular
tests' setup where database connections are being overwritten to be the
same (?)
--
Ticket URL: <https://code.djangoproject.com/ticket/36770>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019ae6669f8a-b7ee5a9e-e076-413a-91f9-3eae7e71025f-000000%40eu-central-1.amazonses.com.
