#16052: MySQL: CharField's with utf8_bin collation return as str, should return
as
unicode.
-------------------------------------+-------------------------------------
Reporter: | Owner: graham_king
graham_king | Status: new
Type: Bug | Component: Database layer
Milestone: | (models, ORM)
Version: SVN | Severity: Normal
Resolution: | Keywords: mysql utf_bin
Triage Stage: Design | unicode
decision needed | Has patch: 1
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
-------------------------------------+-------------------------------------
Changes (by kmtracey):
* stage: Unreviewed => Design decision needed
Comment:
We tried to do something like this before, see r8318 and r8329. You'll
notice we were trying to specify `FIELD_TYPE.VARCHAR`, not
`FIELD_TYPE.VAR_STRING` and upon further investigation we found that the
VARCHAR variant was never in fact returned by the MySQL C API, so the code
added to attempt to convert to unicode wasn't ever called. The attempt was
reverted in r8380.
It was reverted rather than switched to specify VAR_STRING because
VAR_STRING is also the type reported for a MySQL column with type
VARBINARY: truly binary data. Attempting to convert truly binary data to
unicode (assuming utf-8 encoding) could generate an exception or could
corrupt the data; it didn't seem appropriate to be attempting the
conversion for all types MySQL reported as VAR_STRING types.
Now, Django doesn't actually support binary fields (never generates a
(VAR)BINARY column), but back then there was some hope it might in the
future. (Though I've not heard any mention of binary field support in a
while.) However, the ability to pretty easily create/use custom fields
means there might be people out there with custom fields that use the
MySQL VARBINARY type...I fear this proposed change would break such
fields.
Also note that this deviation from the norm of returning unicode for
character data is documented in the database notes:
http://docs.djangoproject.com/en/1.3/ref/databases/#collation-settings.
If/when we do find a way to safely change the low-level behavior here
those notes will need to be revised as well.
--
Ticket URL: <https://code.djangoproject.com/ticket/16052#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
