#18757: Examine ways to get rid of DB backend convert_values()
-------------------------------------+-------------------------------------
Reporter: akaariai | Owner: mjtamlyn
Type: | Status: assigned
Cleanup/optimization | Version: master
Component: Database layer | Resolution:
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 0
Has patch: 0 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by akaariai):
I have tested a bit in this area and we really can't call
field.from_db_value for every field. It is just plain too expensive. We
are talking about 2-3x performance slowdown for models with ten fields. A
single method call is somewhere around 5% slowdown in model initialization
speed. For 10 fields the above field.from_db_value() definition will cause
3 calls per field, so that is 5% * 3 * 10 = 150% slowdown.
Instead we need to do the following:
1) Collect converters for each field before we start iterating the
result set.
2) Not all fields define a converter - in fact almost all fields in core
do not need a converter at all.
3) At the same time we also collect converters from the database
backend.
4) When we have collected the converters, start iterating through the
result set. Pass the values through the found converters.
This is needed so that if a field doesn't define any converter, then there
is no overhead for that field. For this reason I would also try to avoid
doing backend specific conversion in the field's from_db method - this
means that if any backend needs from_db support, then all backends need to
pay the overhead of calling the method for no benefit at all.
I don't think the new way needs to be backwards compatible to the old
convert_values way. We just need to ensure the old way works for backwards
compatibility period.
So, alternate proposal:
1) Add field.get_db_value_converters(connection) method. The method
returns a list of converters or None if no converters are needed.
2) Add backend.get_value_converter(field) method. By default this method
returns [backend.convert_values] if that is defined for the backend,
otherwise it returns None.
3) Collect all converters before iterating results.
4) When iterating through the results, the converters will be called
with just value as argument (we could likely also add connection here)
5) Deprecate SubfieldBase, deprecate backend.convert_values.
As for how to do the convert_values collection, look for
https://github.com/akaariai/django/tree/custom_lookups for one
implementation. Unfortunately that implementation is mixed with unrelated
changes.
--
Ticket URL: <https://code.djangoproject.com/ticket/18757#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/066.78762739f3801a83cedf9801c11b07cd%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
