[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Michael McFadden added the comment: Also fixes 19907? -- nosy: +flipmcf ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Serhiy Storchaka added the comment: Default encoding on Python 3 is UTF-8. You should declare your encoding at the top of file if it differs from UTF-8 or ASCII (i.e. # -*- coding: euc-jp -*-). Otherwise Python will reject your file (for Shift_JIS and EUC-JP) or produce incorrect result (for ISO-2022-JP). $ python3 konnichiha.Shift_JIS.py File konnichiha.Shift_JIS.py, line 5 SyntaxError: Non-UTF-8 code starting with '\x82' in file konnichiha.Shift_JIS.py on line 5, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details $ python3 konnichiha.ISO-2022-JP.py konnichiha B$3$s$K$A$O -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
umedoblock added the comment: python3 output translate Japanese with pygettext.install(). EVERYTHING IS OK! please check to use a konnichiha.2.tar.gz. == please do below shell command. $ for f in `find . -name 'konnichiha.*.py'` ; do echo f=$f ; python3 $f ; echo -- ; done f=./konnichiha.Shift_JIS.py HELLO ハローで、今日は 日本語をＵＴＦ８にしてコンニチハ -- f=./konnichiha.UTF-8.py HELLO ハローで、今日は 日本語をＵＴＦ８にしてコンニチハ -- f=./konnichiha.ISO-2022-JP.py HELLO ハローで、今日は 日本語をＵＴＦ８にしてコンニチハ -- f=./konnichiha.EUC-JP.py HELLO ハローで、今日は 日本語をＵＴＦ８にしてコンニチハ -- == konnichiha script encoding is OK! $ nkf -g ./konnichiha.*.py ./konnichiha.EUC-JP.py: EUC-JP ./konnichiha.ISO-2022-JP.py: ISO-2022-JP ./konnichiha.Shift_JIS.py: Shift_JIS ./konnichiha.UTF-8.py: UTF-8 == also coding: is OK! $ head -2 konnichiha.*.py == konnichiha.EUC-JP.py == # coding: euc-jp import gettext == konnichiha.ISO-2022-JP.py == # coding: iso-2022-jp import gettext == konnichiha.Shift_JIS.py == # coding: shift-jis import gettext == konnichiha.UTF-8.py == # coding: utf-8 import gettext == THANK YOU serhiy.storchaka ! -- Added file: http://bugs.python.org/file29018/konnichiha.2.tar.gz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Roundup Robot added the comment: New changeset 49b1fde510a6 by Serhiy Storchaka in branch '2.7': Issue #17156: pygettext.py now correctly escapes non-ascii characters. http://hg.python.org/cpython/rev/49b1fde510a6 New changeset cd59b398907d by Serhiy Storchaka in branch '3.2': Issue #17156: pygettext.py now uses an encoding of source file and correctly http://hg.python.org/cpython/rev/cd59b398907d New changeset 062406c06cc1 by Serhiy Storchaka in branch '3.3': Issue #17156: pygettext.py now uses an encoding of source file and correctly http://hg.python.org/cpython/rev/062406c06cc1 New changeset 99795d711a40 by Serhiy Storchaka in branch 'default': Issue #17156: pygettext.py now uses an encoding of source file and correctly http://hg.python.org/cpython/rev/99795d711a40 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Changes by Serhiy Storchaka storch...@gmail.com: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Changes by Serhiy Storchaka storch...@gmail.com: Removed file: http://bugs.python.org/file28992/pygettext.py.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Changes by Serhiy Storchaka storch...@gmail.com: -- assignee: - serhiy.storchaka nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Serhiy Storchaka added the comment: Here is a patch for 3.x, which correctly detects input file encoding and correctly escapes non-ascii output files if -E specified (and only if it specified). For 2.7 we should just negate an argument for make_escapes. -- components: +Unicode nosy: +ezio.melotti, loewis stage: - patch review versions: +Python 3.3, Python 3.4 Added file: http://bugs.python.org/file29001/pygettext_unicode.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Serhiy Storchaka added the comment: Here is a patch for 2.7. pygettext doesn't try to detect input encoding and transparently works with bytes, but it no longer escapes non-ascii bytes if -E is not specified. -- versions: +Python 2.7 Added file: http://bugs.python.org/file29002/pygettext_unicode-2.7.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
umedoblock added the comment: thanks serhiy.storchaka. I try to use Shift_JIS, UTF-8, ISO-2022-JP and EUC-JP. your patch detects UTF-8. However it doesn't detect Shift_JIS, ISO-2022-JP and EUC-JP. it misunderstand ISO-2022-JP charset is UTF-8. it raised UnicodeDecodeError when Shift_JIS, EUC-JP. Please check my test to use konnichiha.sh. -- Added file: http://bugs.python.org/file29009/konnichiha.tar.gz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
umedoblock added the comment: I use just a pygettext_unicode.patch. don't use a pygettext_unicode-2.7.patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
New submission from umedoblock:
I'd like to parse _('こんにちは').
However pygettext.py doesn't parse _('こんにちは').
pygettext.py said me 'IndexError'.
now I attached pygettext.py.patch to fix a bug.
I show you command history.
$ pygettext.py -o - --verbose konnichiha.py
...
#: konnichiha.py:6
msgid konnichiha
msgstr
#: konnichiha.py:7
Traceback (most recent call last):
File /home/umetaro/local/bin/pygettext.py, line 664, in module
main()
File /home/umetaro/local/bin/pygettext.py, line 657, in main
eater.write(fp)
File /home/umetaro/local/bin/pygettext.py, line 497, in write
print('msgid', normalize(k), file=fp)
File /home/umetaro/local/bin/pygettext.py, line 250, in normalize
s = '' + escape(s) + ''
File /home/umetaro/local/bin/pygettext.py, line 236, in escape
s[i] = escapes[ord(s[i])]
IndexError: list index out of range
please use pygettext.py.patch.
$ pygettext.py -o - --verbose konnichiha.py
...
#: konnichiha.py:6
msgid konnichiha
msgstr
#: konnichiha.py:7
msgid こんにちは
msgstr
--
components: Demos and Tools
files: konnichiha.py
messages: 181651
nosy: umedoblock
priority: normal
severity: normal
status: open
title: Tools/i18n/pygettext.py doesn't parse unicode string.
type: behavior
versions: Python 3.2
Added file: http://bugs.python.org/file28991/konnichiha.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17156
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
Changes by umedoblock umedobl...@gmail.com: -- keywords: +patch Added file: http://bugs.python.org/file28992/pygettext.py.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17156] Tools/i18n/pygettext.py doesn't parse unicode string.
umedoblock added the comment: TOO SORRY. pygettext.py.patch umedoblock, 2013-02-08 10:32 is wrong a patch. please forget it. -- Added file: http://bugs.python.org/file28993/pygettext.py.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17156 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
