[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-31 Thread Andrew Svetlov
Andrew Svetlov andrew.svet...@gmail.com added the comment: Closing again. Now IDLE works fine both in subprocess and inprocess mode. Future support of non-BMP can be continues after implementing codec for that — #14304 Now I like to close that as «good enough for now». At least IDLE doesn't

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-25 Thread Roundup Robot
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 89878808f4ce by Andrew Svetlov in branch 'default': Issue #14200 — now displayhook for IDLE works in non-subprocess mode as well as subprecess. http://hg.python.org/cpython/rev/89878808f4ce --

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-25 Thread Andrew Svetlov
Andrew Svetlov andrew.svet...@gmail.com added the comment: After experiments with non-BMP characters I figured out: — non-bmp symbols processed by Tk text widgets (Entry, Text etc.) differently. For example Entry can display non-bmp with spaces after glyph, Text reduces symbol to BMP. Editing

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-15 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: On Windows Vista, I do see that print() behaves differently than evaluating the expression. An exception is raised for: print('\N{GOTHIC LETTER AHSA}') As is for most other characters not supported in your OEM code page, e.g. (likely)

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-15 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: I stand corrected. Thank you for the information. The behavior of the console depends on its locale. IDLE has no facility for changing the locale of the PyShell window. Should this option be included somewhere? --

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-15 Thread Andrew Svetlov
Andrew Svetlov andrew.svet...@gmail.com added the comment: I think that doesn't make sense. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-15 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: The Tkinter Text widget is the output for the IDLE shell and it has the limitation imposed by Tcl/Tk of not handling non-BMP unicode characters. Is the following reasonable: The IDLE shell console has a locale of non-BMP utf8? If so,

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-15 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: The behavior of the console depends on its locale. IDLE has no facility for changing the locale of the PyShell window. Should this option be included somewhere? It may be remotely desirable to be able to set the terminal encoding in IDLE

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-15 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: Is the following reasonable: The IDLE shell console has a locale of non-BMP utf8? [BMP utf8] That's indeed the approach that Andrew and I were discussing. Unfortunately, there is no codec for it yet. We were discussing to add a utf8bom

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-15 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Martin, you are right. I created a separate issue #14326. Let me know what I can do to help. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Roundup Robot
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset c06b94c5c609 by Andrew Svetlov in branch 'default': Issue #14200: Idle shell crash on printing non-BMP unicode character. http://hg.python.org/cpython/rev/c06b94c5c609 --

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Andrew Svetlov
Andrew Svetlov andrew.svet...@gmail.com added the comment: Patch escapes avery non-ascii char while better to escape only non-BMP. Will be done after #14304 -- resolution: - fixed ___ Python tracker rep...@bugs.python.org

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Andrew Svetlov
Changes by Andrew Svetlov andrew.svet...@gmail.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___ ___

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Andrew, please reopen this issue. Your committed patch does not work if IDLE is not using the subprocess. got_ahsa = \N{GOTHIC LETTER AHSA} got_ahsa Traceback (most recent call last): File pyshell#1, line 1, in module

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Attached is a patch to undo Andrew's and fixes the issue in a simple manner. The tcl_unicode_range.patch from Issue12342 has already been applied, so catching ValueError within IDLE is all that is now needed. -- Added file:

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Attached is a better implementation of the patch. The Percolator which ultimately handles writing to the Text widget should intercept the ValueError due to non-BMP characters. The issue14200_rev1.patch fixes this issue and Issue13153.

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Andrew Svetlov
Andrew Svetlov andrew.svet...@gmail.com added the comment: Roger, you are missing the difference between calling print() and evaluating expression in python interactive mode. While later should be unicode escaped the former should to raise error — we need to follow the same way as console

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Andrew, I do admit that I have a lot to learn about Unicode support in Python, for instance with its error-handling and its corner cases. On Windows Vista, I do see that print() behaves differently than evaluating the expression. An

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-14 Thread Andrew Svetlov
Andrew Svetlov andrew.svet...@gmail.com added the comment: I consulted with Martin at PyCon sprint and he suggested sulution which I'm following — to split `print` and REPL (read-eval-print loop). Output passed to print() function encoded with sys.stdout.encoding UTF has been invented to

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-13 Thread Andrew Svetlov
Changes by Andrew Svetlov andrew.svet...@gmail.com: -- nosy: +asvetlov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___ ___

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-11 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Attached is a patch to have the rpc marshal exceptions. When used with Martin's patch, IDLE returns '\U00010330' Traceback (most recent call last): File pyshell#3, line 1, in module '\U00010330' ValueError: character U+10330 is above

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-11 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: I made a mistake in msg155410. The results in the message are WITHOUT unicodeerror.diff applied. When it is applied, the IDLE shell gives: '\U00010330' Traceback (most recent call last): File pyshell#1, line 1, in module '\U00010330'

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-11 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: Martin: I disagree with the approach of raising a UnicodeEncodeError if IDLE can't render the output of a user's program, especially when the program would otherwise run without error if ran from outside of IDLE. This is really an

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-11 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Having had some time to work on it, the bug is in the unicodeerror.diff patch. If the string is empty then max(s) will raise a ValueError. This is easy to trigger by generating an exception at the python prompt, like 1/0. Attached is a

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-11 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Martin, I got your message after I submitted the last one. This issue does involve IDLE crashing, but it's not crashing due to non-BMP characters. That is a side-effect of a bigger issue with pythonw.exe. See Issue13582 for more

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-06 Thread Vlastimil Brom
Vlastimil Brom vlastimil.b...@gmail.com added the comment: Sorry for mixing the different problems, these were somehow things I noticed at once in the new python version, but I should have noticed the different domains myself. I still might not understand the term crash properly - I just meant

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-06 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: That pythonw suddenly closes is a separate issue: if pythonw attempts to write to stderr, it crashes. To get your example to run in pythonw.exe, try pythonw.exe Lib\idlelib\idle.py 2 out.txt I think the behavior of pythonw terminating when

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Vlastimil Brom
New submission from Vlastimil Brom vlastimil.b...@gmail.com: Hi, while testing python 3.3a1 a bit, especially the new string handling of non-BMP characters, I noticed a problem in Idle in this regard: Python 3.3.0a1 (default, Mar 4 2012, 17:27:59) [MSC v.1500 32 bit (Intel)] on win32 ...

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Ezio Melotti
Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +haypo, loewis, ned.deily, terry.reedy type: - crash ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Terry J. Reedy
Changes by Terry J. Reedy tjre...@udel.edu: -- nosy: +serwy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___ ___ Python-bugs-list mailing

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Roger Serwy
Roger Serwy roger.se...@gmail.com added the comment: Hi Vlastimil, Can you repeat your test case while running IDLE from the command prompt and report the error you see? python -m idlelib.idle IDLE closes suddenly on Windows because IDLE uses pythonw.exe which has no stdout or stderr.

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Vlastimil Brom
Vlastimil Brom vlastimil.b...@gmail.com added the comment: Hi, thanks for the pointer, after invoking idle using python.exe, I don't see the crash mentioned in the report: Python 3.3.0a1 (default, Mar 4 2012, 17:27:59) [MSC v.1500 32 bit (Intel)] on win32 Type copyright, credits or license()

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Terry J. Reedy
Terry J. Reedy tjre...@udel.edu added the comment: On 3.2.2, Win7, the length is 2 and printing in Idle prints a square, as it usually does for chars it cannot print. I presume Tk recognizes surrogate pairs. Printing to the screen should not raise an exception, so the square would be better.

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Vlastimil Brom
Vlastimil Brom vlastimil.b...@gmail.com added the comment: I'd like to add some further observations to the mentioned issue; it seems, that the crash is indeed not specific to idle. In a sample tkinter app, where I just display e.g. chr(66352) in an Entry widget, I also get the same immediate

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: Vlastimil: you are mixing issues. Some of your observations are actually correct behaviour; please don't clutter the report with that, but report each separate behavior in a separate report. In Python 3.3, surrogate pairs do *not*

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Martin v . Löwis
Changes by Martin v. Löwis mar...@v.loewis.de: -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Martin v . Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: Oops, wrong issue. -- resolution: fixed - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Martin v . Löwis
Changes by Martin v. Löwis mar...@v.loewis.de: -- Removed message: http://bugs.python.org/msg155005 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___

[issue14200] Idle shell crash on printing non-BMP unicode character

2012-03-05 Thread Martin v . Löwis
Changes by Martin v. Löwis mar...@v.loewis.de: -- Removed message: http://bugs.python.org/msg155006 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14200 ___