#16315: FileSystemStorage.listdir returns names with unicode normalization form
that is different from names in database
-------------------------------------+-------------------------------------
Reporter: philomat | Owner: nobody
Type: Bug | Status: reopened
Component: File | Version: 1.3
uploads/storage | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: storage unicode | Needs documentation: 0
normalization | Patch needs improvement: 0
Has patch: 0 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by julien):
* status: closed => reopened
* resolution: needsinfo =>
* stage: Unreviewed => Accepted
Comment:
I'm getting some test failures for the django staticfiles tests on my
Ubuntu 10.04.4 LTS box due to unicode issues similar to those as reported
in this ticket.
The root of the problem is that my box's filesystem apparently uses
[http://en.wikipedia.org/wiki/Combining_character#Unicode_ranges combining
diacritical marks] for encoding certain characters when creating
filenames. In the particular case of these failing tests, my box encodes
the character u'ş' (u'\u015f') from the filename fişier.txt as:
u's\u0327', that is with a 's' followed by a combining cedilla.
Here's one way of illustrating the problem:
{{{#!python
>>> u'ş'
u'\u015f'
>>> print u'\u015f'
ş
>>> print u's\u0327' # Combining cedilla
ş
}}}
It seems like the right approach would be to make Django normalize
filenames:
{{{#!python
>>> import unicodedata
>>> unicodedata.normalize("NFC", u"s\u0327")
u'\u015f'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/16315#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
