#16315: FileSystemStorage.listdir returns names with unicode normalization form
that is different from names in database
-------------------------------------+-------------------------------------
Reporter: philomat | Owner: nobody
Type: Bug | Status: new
Milestone: | Component: File
Version: 1.3 | uploads/storage
Keywords: storage unicode | Severity: Normal
normalization | Triage Stage: Unreviewed
Has patch: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
When you want to write a function that finds files on disk that are not
stored in the database anymore, and use FileSystemStorage.listdir to
compare what's returned with what's in the database: You will not be able
to compare strings without normalizing them first since unicode characters
can be encoded using different normalization forms.
This problem is best demonstrated with some example code:
{{{
# Assuming that my storage root contains one folder named u'ä'
>>> import os
>>> from django.core.files.storage import FileSystemStorage
>>> import unicodedata
>>>
# listdir returns u'a' followed by 'COMBINING DIAERESIS' (U+0308)
>>> FileSystemStorage().listdir('')[0][0]
u'a\u0308'
# in the database, this character is stored using a different
normalization form:
>>> os.path.basename(FileSystemStorage().path(u'ä'))
u'\xe4'
# the values should be normalized:
>>> unicodedata.normalize('NFC', FileSystemStorage().listdir('')[0][0])
u'\xe4'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/16315>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
