#36674: select_related() leaks memory
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Type:
| Cleanup/optimization
Status: new | Component: Database
| layer (models, ORM)
Version: dev | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Calls to `select_related()` create nested functions that aren't cleaned up
until garbage collection runs:
Borrowed testing strategy from #34865:
{{{#!diff
diff --git a/tests/select_related/tests.py b/tests/select_related/tests.py
index 41ed350cf3..252e884d7a 100644
--- a/tests/select_related/tests.py
+++ b/tests/select_related/tests.py
@@ -242,6 +242,30 @@ class SelectRelatedTests(TestCase):
FETCH_PEERS,
)
+ def test_memory_leak(self):
+ # todo: factor out a test util
+ import gc
+
+ # Schedule the restore of the garbage collection settings.
+ self.addCleanup(gc.set_debug, 0)
+ self.addCleanup(gc.enable)
+
+ # Disable automatic garbage collection to control when it's
triggered,
+ # then run a full collection cycle to ensure `gc.garbage` is
empty.
+ gc.disable()
+ gc.collect()
+
+ # The garbage list isn't automatically populated to avoid CPU
overhead,
+ # so debugging needs to be enabled to track all unreachable items
and
+ # have them stored in `gc.garbage`.
+ gc.set_debug(gc.DEBUG_SAVEALL)
+
+ list(Species.objects.select_related("genus"))
+
+ # Enforce garbage collection to populate `gc.garbage` for
inspection.
+ gc.collect()
+ self.assertEqual(gc.garbage, [])
+
class SelectRelatedValidationTests(SimpleTestCase):
"""
}}}
Gives:
{{{#!diff
AssertionError: Lists differ: [<cell at 0x10630c940: function object at
[148 chars]040>] != []
First list contains 3 additional elements.
First extra element 0:
<cell at 0x10630c940: function object at 0x1079b8040>
+ []
- [<cell at 0x10630c940: function object at 0x1079b8040>,
- (<cell at 0x10630c940: function object at 0x1079b8040>,),
- <function SQLCompiler.get_select.<locals>.get_select_from_parent at
0x1079b8040>]
}}}
At a glance, it looks like we can lift `get_select_from_parent` to a class
method or all the way to module-level without a loss of functionality to
address this.
--
Ticket URL: <https://code.djangoproject.com/ticket/36674>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019a03f7e367-c10f0ecd-45c4-467a-8a0c-b65496ecf849-000000%40eu-central-1.amazonses.com.
