[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Andrew Svetlov added the comment: Closing as won't fix. -- assignee: kbk -> asvetlov nosy: +asvetlov resolution: -> wont fix stage: test needed -> committed/rejected status: open -> closed ___ Python tracker __

[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Changes by Andreas Stührk : -- nosy: +Trundle ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Martin v. Löwis added the comment: For 2.7, I don't think it's possible to really fix this. I see the following options: A. current status. Byte strings are compiled correctly, Unicode strings are not. B. compile source as a Unicode string, as proposed in msg85886. Unicode strings are compile

[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Naoki INADA added the comment: How to use locale.getpreferredencoding() instead of locale.nl_langinfo(locale.CODESET). --- IOBinding.py.back Sun Apr 12 19:54:52 2009 +++ IOBinding.pySun Apr 12 20:02:58 2009 @@ -35,40 +35,16 @@ # Encoding for file names filesystemencoding = sys.getf

[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Naoki INADA added the comment: utf-8 is not locale encoding. >>> f = open('á.txt') If this line compiled into utf-8 and locale encoding is not utf-8, can't open 'á.txt'. IMHO, in case of Python 2.x, correct approach is fix IOBindings.encoding and compile() with pep0263. -- __

[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Santiago Gala added the comment: Updating the components as the error surfaces in the compile builtin. the compile builtin works when given unicode, but fails when using a utf8 (local input encoding) string. Rather than adding a "coding" string to compile, my guess is that compile should be fix

[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Naoki INADA added the comment: This patch is for iplib/PyShell.py#ModifiedInterpreter.runsource. if isinstance(source, types.UnicodeType): import IOBinding try: source = source.encode(IOBinding.encoding) +source = "# coding: %s

[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Naoki INADA added the comment: This issue is caused by compile() behavior. Following sample is in codepage 932. >>> 'あ' '\x82\xa0' # OK - 'あ' is '\x82\xa0' in cp932 >>> u'あ' u'\u3042' # OK - u'あ' is '\u3042' in UCS-2 compile as byte string. >>> c = compile("'あ'", 'test', 'single') >>> exec

[issue1542677] IDLE shell gives different len() of unicode strings compared to Python shell

Daniel Diniz added the comment: This is about a disparity between IDLE and the python shell. I'm guessing different encodings are to blame here and that this is invalid. The disparity is present in an UCS4 build (IDLE shows UCS2-like behavior[1], maybe because it's using UTF8?). [1] http://mai