#16759: Expensive sql.Query cloning
-------------------------------------+-------------------------------------
Reporter: Suor | Owner: Suor
Type: Bug | Status: new
Component: Database layer | Version: 1.3
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: orm performance | Needs documentation: 0
cloning | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by akaariai):
* needs_better_patch: 1 => 0
Comment:
I cleaned up the patch, I think it is good to go. Aggregates are handled
by copy.copy now, and there is an assert that subtree_parents is empty
when cloning. I removed all deepcopy references from sql/query.py
There is one bad abstraction leak: tree.clone() must know the internal
structure of queryset's lookup condition tuples. The fix would be to
create a class for these tuples also. This would be a quite big change,
but I think it would make the code much cleaner in sql/query.py and
sql/where.py, too.
The speed is from 30% to 60% faster for three simple test-cases, query
construction only (not compiling it to sql, or executing the sql):
{{{
Test 1, 1000x TestModel.objects.all().filter(pk=1)
Test 2, 1000x TestModel.objects.all().filter(pk=1).filter(pk=1)
Test 3, 1000x
TestModel.objects.all().filter(pk=1).filter(pk=1).filter(pk=1)
}}}
I would expect 70%+ speedup is possible for complex realworld queries.
All tests passed on sqlite3.
I think there is still much room for improvement, but this is the low-
hanging fruit. I do think this would be a good candidate for 1.4. I have
seen reports of 200+ms time used for queryset construction for single page
load. So the problem is real.
--
Ticket URL: <https://code.djangoproject.com/ticket/16759#comment:23>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
