Re: Problem with Django 1.0 admin on Russian language
Everybody, thank you for feedback. Karen, you were absolutely right! I've changed collation to utf8_unicode_ci and Django is converting strings to unicode! This is fine! Thanks a lot! RTFM :-) On Sep 18, 6:12 pm, "Karen Tracey" <[EMAIL PROTECTED]> wrote: > Since you say you are using UTF8_BIN collation on MySQL, be sure to read: > > http://docs.djangoproject.com/en/dev/ref/databases/#collation-settings > > When you specify a binary collation in MySQL, your application code has to > take on the responsibility of transforming bytestrings returned by the > database to unicode objects. As noted in the docs, you can use > django.utils.encoding.smart_unicode() to do this. You should use this in > your __unicode__ method to coerce the bytestrings coming from the database > to unicode. Failure to do this results in many problems like what you have > been reporting, as the admin app relies on the model's __unicode__ method to > display/log information about objects. > > Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Problem with Django 1.0 admin on Russian language
Since you say you are using UTF8_BIN collation on MySQL, be sure to read: http://docs.djangoproject.com/en/dev/ref/databases/#collation-settings When you specify a binary collation in MySQL, your application code has to take on the responsibility of transforming bytestrings returned by the database to unicode objects. As noted in the docs, you can use django.utils.encoding.smart_unicode() to do this. You should use this in your __unicode__ method to coerce the bytestrings coming from the database to unicode. Failure to do this results in many problems like what you have been reporting, as the admin app relies on the model's __unicode__ method to display/log information about objects. Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Problem with Django 1.0 admin on Russian language
Hello Fedya! On Thu, 18 Sep 2008 07:24:16 -0700 (PDT) you wrote: > > Unfortunately I'm dummy in python... so it's twice harder to find the > reason... But I'm still trying :) Maybe this helps? # grep -i utf8 /usr/lib/python2.4/site.py encoding = "utf8" # Default value set by _PyUnicode_Init() -- Всего наилучшего! Григорий greg [at] anastasia [dot] ru Письмо отправлено: 2008/09/18 19:04 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Problem with Django 1.0 admin on Russian language
Unfortunately I'm dummy in python... so it's twice harder to find the
reason... But I'm still trying :)
I wrote very simple view:
def main_menu(request):
html = ''
menu = Item.root('main').children()
for s in menu[0].title:
html = '%s %s' % (html, ord(s))
html = '%s%s' % (menu[0].title, html)
return HttpResponse(html)
And http://test.legalsw.ru/menu/ outputs
АБВ
208 144 208 145 208 146
when my.cnf contains default-character-set = cp1251
and
���
192 193 194
when my.cnf contains default-character-set = utf8
208 144 is 0xd090 which is Cyrillic A code in UTF8
192 is Cyrillic A in CP1251...
"???" is АБВ if browser's character encoding is cp1251
But then if i use python's unicode string, so u'%s%s' instead of
'%s%s', I get :
UnicodeDecodeError at /menu/
'ascii' codec can't decode byte 0xc0 in position 0: ordinal not in
range(128)
Request Method: GET
Request URL:http://test.legalsw.ru/menu/
Exception Type: UnicodeDecodeError
Exception Value:
'ascii' codec can't decode byte 0xc0 in position 0: ordinal not in
range(128)
Exception Location: /home/l/legalswru/test/public_html/menu/views.py
in main_menu, line 11
Python Executable: /usr/bin/python
Python Version: 2.4.4
Python Path:['/home/l/legalswru/test/public_html', '/home/l/
legalswru/test/public_html', '/usr/lib/python24.zip', '/usr/lib/
python2.4', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib-
tk', '/usr/lib/python2.4/lib-dynload', '/usr/lib/portage/pym', '/usr/
lib/python2.4/site-packages', '/usr/lib/python2.4/site-packages/PIL']
Server time:Чет, 18 Сен 2008 09:17:58 -0500
As I understand python supposes that menu[0].title is ASCII string and
tries to convert it to unicode string... I guess the same happens with
admin templates... Do you have any ideas how to solve this? I've tried
to set use_unicode to True, but nothing changed.
On Sep 18, 3:25 pm, Fedya <[EMAIL PROTECTED]> wrote:
> Still something wrong with character set.
>
> When I use Russian characters inside value of a field in one table,
> Django admin interface can't show list of object corresponding to that
> table. Like I'm trying to accesshttp://test.legalsw.ru/admin/menu/item/
> and getting:
> TemplateSyntaxError at /admin/menu/item/
>
> Caught an exception while rendering: 'ascii' codec can't decode byte
> 0xd0 in position 0: ordinal not in range(128). You passed in
>
> ()
>
> Original Traceback (most recent call last):
> File "/home/l/legalswru/test/public_html/django/template/debug.py",
> line 71, in render_node
> result = node.render(context)
> File "/home/l/legalswru/test/public_html/django/template/
> loader_tags.py", line 24, in render
> result = self.nodelist.render(context)
> File "/home/l/legalswru/test/public_html/django/template/
> __init__.py", line 768, in render
> bits.append(self.render_node(node, context))
> File "/home/l/legalswru/test/public_html/django/template/debug.py",
> line 78, in render_node
> wrapped = TemplateSyntaxError(u'Caught an exception while
> rendering: %s' % force_unicode(e, errors='replace'))
> File "/home/l/legalswru/test/public_html/django/utils/encoding.py",
> line 70, in force_unicode
> raise DjangoUnicodeDecodeError(s, *e.args)
> DjangoUnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in
> position 0: ordinal not in range(128). You passed in
>
> ()
>
> Request Method: GET
> Request URL:http://test.legalsw.ru/admin/menu/item/
> Exception Type: TemplateSyntaxError
> Exception Value:
>
> Caught an exception while rendering: 'ascii' codec can't decode byte
> 0xd0 in position 0: ordinal not in range(128). You passed in
>
> ()
>
> Original Traceback (most recent call last):
> File "/home/l/legalswru/test/public_html/django/template/debug.py",
> line 71, in render_node
> result = node.render(context)
> File "/home/l/legalswru/test/public_html/django/template/
> loader_tags.py", line 24, in render
> result = self.nodelist.render(context)
> File "/home/l/legalswru/test/public_html/django/template/
> __init__.py", line 768, in render
> bits.append(self.render_node(node, context))
> File "/home/l/legalswru/test/public_html/django/template/debug.py",
> line 78, in render_node
> wrapped = TemplateSyntaxError(u'Caught an exception while
> rendering: %s' % force_unicode(e, errors='replace'))
> File "/home/l/legalswru/test/public_html/django/utils/encoding.py",
> line 70, in force_unicode
> raise DjangoUnicodeDecodeError(s, *e.args)
> DjangoUnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in
> position 0: ordinal not in range(128). You passed in
>
> ()
>
> Exception Location: /home/l/legalswru/test/public_html/django/
> template/debug.py in render_node, line 81
> Python Executable: /usr/bin/python
> Python Version: 2.4.4
> Python Path:['/home/l/legalswru/test/public_html', '/home/l/
> legalswru/test/public_html', '/usr/lib/python24.zip', '/usr/lib/
> python2
Re: Problem with Django 1.0 admin on Russian language
Still something wrong with character set. When I use Russian characters inside value of a field in one table, Django admin interface can't show list of object corresponding to that table. Like I'm trying to access http://test.legalsw.ru/admin/menu/item/ and getting: TemplateSyntaxError at /admin/menu/item/ Caught an exception while rendering: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128). You passed in () Original Traceback (most recent call last): File "/home/l/legalswru/test/public_html/django/template/debug.py", line 71, in render_node result = node.render(context) File "/home/l/legalswru/test/public_html/django/template/ loader_tags.py", line 24, in render result = self.nodelist.render(context) File "/home/l/legalswru/test/public_html/django/template/ __init__.py", line 768, in render bits.append(self.render_node(node, context)) File "/home/l/legalswru/test/public_html/django/template/debug.py", line 78, in render_node wrapped = TemplateSyntaxError(u'Caught an exception while rendering: %s' % force_unicode(e, errors='replace')) File "/home/l/legalswru/test/public_html/django/utils/encoding.py", line 70, in force_unicode raise DjangoUnicodeDecodeError(s, *e.args) DjangoUnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128). You passed in () Request Method: GET Request URL:http://test.legalsw.ru/admin/menu/item/ Exception Type: TemplateSyntaxError Exception Value: Caught an exception while rendering: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128). You passed in () Original Traceback (most recent call last): File "/home/l/legalswru/test/public_html/django/template/debug.py", line 71, in render_node result = node.render(context) File "/home/l/legalswru/test/public_html/django/template/ loader_tags.py", line 24, in render result = self.nodelist.render(context) File "/home/l/legalswru/test/public_html/django/template/ __init__.py", line 768, in render bits.append(self.render_node(node, context)) File "/home/l/legalswru/test/public_html/django/template/debug.py", line 78, in render_node wrapped = TemplateSyntaxError(u'Caught an exception while rendering: %s' % force_unicode(e, errors='replace')) File "/home/l/legalswru/test/public_html/django/utils/encoding.py", line 70, in force_unicode raise DjangoUnicodeDecodeError(s, *e.args) DjangoUnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128). You passed in () Exception Location: /home/l/legalswru/test/public_html/django/ template/debug.py in render_node, line 81 Python Executable: /usr/bin/python Python Version: 2.4.4 Python Path:['/home/l/legalswru/test/public_html', '/home/l/ legalswru/test/public_html', '/usr/lib/python24.zip', '/usr/lib/ python2.4', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib- tk', '/usr/lib/python2.4/lib-dynload', '/usr/lib/portage/pym', '/usr/ lib/python2.4/site-packages', '/usr/lib/python2.4/site-packages/PIL'] Server time:Чет, 18 Сен 2008 07:10:34 -0500 I understand that this is most probably misconfiguration of mysql, but I've tried a lot of different configurations using init_command, charset, use_unicode and read_default_file parameters... I've also tried to reconfigure database to use cp1251 charset, because this is default encoding for mysql server, but without positive effect... The only difference I can see is a modification of the byte which is 0xd0 in above error report. Sorry for my bad English, but I really need any fresh ideas! On Sep 18, 12:29 pm, Fedya <[EMAIL PROTECTED]> wrote: > hm... I've tried with very simple model from the scratch and I got the > same problem without init_command... > > On Sep 18, 2:30 am, Malcolm Tredinnick <[EMAIL PROTECTED]> > wrote: > > > On Thu, 2008-09-18 at 01:57 +0300, Fedor Tyurin wrote: > > > Another possible solution (and probably better one) is to use > > > init_command option set to 'SET NAMES utf8'. > > > Django and MySQLdb already do this by default. Django passes the > > "charset='utf-8'" parameter to the connection and MySQLdb turns this > > into a "SET NAMES" call. So you're doing something unusual if this isn't > > working by default. > > > Regards, > > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Problem with Django 1.0 admin on Russian language
hm... I've tried with very simple model from the scratch and I got the same problem without init_command... On Sep 18, 2:30 am, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Thu, 2008-09-18 at 01:57 +0300, Fedor Tyurin wrote: > > Another possible solution (and probably better one) is to use > > init_command option set to 'SET NAMES utf8'. > > Django and MySQLdb already do this by default. Django passes the > "charset='utf-8'" parameter to the connection and MySQLdb turns this > into a "SET NAMES" call. So you're doing something unusual if this isn't > working by default. > > Regards, > Malcolm > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Problem with Django 1.0 admin on Russian language
On Thu, 2008-09-18 at 01:57 +0300, Fedor Tyurin wrote: > Another possible solution (and probably better one) is to use > init_command option set to 'SET NAMES utf8'. Django and MySQLdb already do this by default. Django passes the "charset='utf-8'" parameter to the connection and MySQLdb turns this into a "SET NAMES" call. So you're doing something unusual if this isn't working by default. Regards, Malcolm > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Problem with Django 1.0 admin on Russian language
Another possible solution (and probably better one) is to use init_command option set to 'SET NAMES utf8'. 2008/9/18 Fedor Tyurin <[EMAIL PROTECTED]> > I set default-character-set to cp1251 (default server encoding) and now > admin seems to work correctly > > 2008/9/18 Fedor Tyurin <[EMAIL PROTECTED]> > > It seems to be related with database connection properties. I've tried with >> sqlite3 and problem disappeared. >> >> If I run mysql console and request status (\s command) mysql response the >> following information about character set: >> Server characterset:cp1251 >> Db characterset:utf8 >> Client characterset:cp1251 >> Conn. characterset:cp1251 >> >> I've also tried to set DATABASE_OPTIONS, but I don't know how to change >> connection and results character set from configuration file (only by using >> SET NAMES 'utf8'). >> >> Any help will be useful. Unfortunately I can't change mysqld settings, >> because it is on the web hosting provider's side. >> >> 2008/9/17 Fedya <[EMAIL PROTECTED]> >> >> Hi everybody, >>> >>> I'm trying to port my site to Django 1.0... And it works fine if >>> LANGUAGE_CODE is set to en-us, but if I set it to ru the following >>> problem appears while I'm trying add new object using admin interface: >>> >>> Warning at /admin/storage/file/add/ >>> >>> Data truncated for column 'message' at row 1 >>> >>> Request Method: POST >>> Request URL:http://test.legalsw.ru/admin/storage/file/add/ >>> Exception Type: Warning >>> Exception Value: >>> >>> Data truncated for column 'message' at row 1 >>> >>> Exception Location: /usr/lib/python2.4/warnings.py in warn_explicit, >>> line 96 >>> Python Executable: /usr/bin/python >>> Python Version: 2.4.4 >>> Python Path:['/home/l/legalswru/test/public_html', '/home/l/ >>> legalswru/test/public_html', '/usr/lib/python24.zip', '/usr/lib/ >>> python2.4', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib- >>> tk', '/usr/lib/python2.4/lib-dynload', '/usr/lib/portage/pym', '/usr/ >>> lib/python2.4/site-packages', '/usr/lib/python2.4/site-packages/PIL'] >>> Server time:Ср, 17 Сен 2008 14:56:13 -0500 >>> >>> I'm using MySQL database and it configured to use UTF8_BIN collation. >>> >> >> > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Problem with Django 1.0 admin on Russian language
I set default-character-set to cp1251 (default server encoding) and now admin seems to work correctly 2008/9/18 Fedor Tyurin <[EMAIL PROTECTED]> > It seems to be related with database connection properties. I've tried with > sqlite3 and problem disappeared. > > If I run mysql console and request status (\s command) mysql response the > following information about character set: > Server characterset:cp1251 > Db characterset:utf8 > Client characterset:cp1251 > Conn. characterset:cp1251 > > I've also tried to set DATABASE_OPTIONS, but I don't know how to change > connection and results character set from configuration file (only by using > SET NAMES 'utf8'). > > Any help will be useful. Unfortunately I can't change mysqld settings, > because it is on the web hosting provider's side. > > 2008/9/17 Fedya <[EMAIL PROTECTED]> > > Hi everybody, >> >> I'm trying to port my site to Django 1.0... And it works fine if >> LANGUAGE_CODE is set to en-us, but if I set it to ru the following >> problem appears while I'm trying add new object using admin interface: >> >> Warning at /admin/storage/file/add/ >> >> Data truncated for column 'message' at row 1 >> >> Request Method: POST >> Request URL:http://test.legalsw.ru/admin/storage/file/add/ >> Exception Type: Warning >> Exception Value: >> >> Data truncated for column 'message' at row 1 >> >> Exception Location: /usr/lib/python2.4/warnings.py in warn_explicit, >> line 96 >> Python Executable: /usr/bin/python >> Python Version: 2.4.4 >> Python Path:['/home/l/legalswru/test/public_html', '/home/l/ >> legalswru/test/public_html', '/usr/lib/python24.zip', '/usr/lib/ >> python2.4', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib- >> tk', '/usr/lib/python2.4/lib-dynload', '/usr/lib/portage/pym', '/usr/ >> lib/python2.4/site-packages', '/usr/lib/python2.4/site-packages/PIL'] >> Server time:Ср, 17 Сен 2008 14:56:13 -0500 >> >> I'm using MySQL database and it configured to use UTF8_BIN collation. >> > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Problem with Django 1.0 admin on Russian language
It seems to be related with database connection properties. I've tried with sqlite3 and problem disappeared. If I run mysql console and request status (\s command) mysql response the following information about character set: Server characterset:cp1251 Db characterset:utf8 Client characterset:cp1251 Conn. characterset:cp1251 I've also tried to set DATABASE_OPTIONS, but I don't know how to change connection and results character set from configuration file (only by using SET NAMES 'utf8'). Any help will be useful. Unfortunately I can't change mysqld settings, because it is on the web hosting provider's side. 2008/9/17 Fedya <[EMAIL PROTECTED]> > Hi everybody, > > I'm trying to port my site to Django 1.0... And it works fine if > LANGUAGE_CODE is set to en-us, but if I set it to ru the following > problem appears while I'm trying add new object using admin interface: > > Warning at /admin/storage/file/add/ > > Data truncated for column 'message' at row 1 > > Request Method: POST > Request URL:http://test.legalsw.ru/admin/storage/file/add/ > Exception Type: Warning > Exception Value: > > Data truncated for column 'message' at row 1 > > Exception Location: /usr/lib/python2.4/warnings.py in warn_explicit, > line 96 > Python Executable: /usr/bin/python > Python Version: 2.4.4 > Python Path:['/home/l/legalswru/test/public_html', '/home/l/ > legalswru/test/public_html', '/usr/lib/python24.zip', '/usr/lib/ > python2.4', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib- > tk', '/usr/lib/python2.4/lib-dynload', '/usr/lib/portage/pym', '/usr/ > lib/python2.4/site-packages', '/usr/lib/python2.4/site-packages/PIL'] > Server time:Ср, 17 Сен 2008 14:56:13 -0500 > > I'm using MySQL database and it configured to use UTF8_BIN collation. > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
