[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by David-Sarah Hopwood : Added file: http://bugs.python.org/file20362/doc-patch.diff ___ Python tracker ___ ___ Python-bugs-list mailin

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by David-Sarah Hopwood : -- nosy: +BreamoreBoy versions: +Python 3.1, Python 3.2 -Python 3.3 Added file: http://bugs.python.org/file20361/doc-patch.diff ___ Python tracker ___

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by David-Sarah Hopwood : -- nosy: +BreamoreBoy versions: +Python 3.1, Python 3.2 -Python 3.3 Added file: http://bugs.python.org/file20360/doc-patch.diff ___ Python tracker ___

[issue1602] windows console doesn't print utf8 (Py30a2)

David-Sarah Hopwood added the comment: "... os.dup2() ..." Good point, thanks. It would work to change os.dup2 so that if its second argument is 0, 1, or 2, it calls _get_osfhandle to get the Windows handle for that fd, and then reruns the console-detection logic. That would even allow Unico

[issue1602] windows console doesn't print utf8 (Py30a2)

Amaury Forgeot d'Arc added the comment: Even if python.exe starts normally, py.test for example uses os.dup2() to redirect the file descriptors 1 and 2 to temporary files. sys.stdout.fileno() is still 1, the STD_OUTPUT_HANDLE did not change, but normal print() now goes to a file; but the prop

[issue1602] windows console doesn't print utf8 (Py30a2)

David-Sarah Hopwood added the comment: Note: Michael Kaplan's code checks whether GetConsoleMode failed due to ERROR_INVALID_HANDLE. My code intentionally doesn't do that, because it is correct and conservative to fall back to the non-console behaviour when there is *any* error from GetConsol

[issue1602] windows console doesn't print utf8 (Py30a2)

David-Sarah Hopwood added the comment: > The script unicode2.py uses the console STD_OUTPUT_HANDLE iff > sys.stdout.fileno()==1. You may have missed "if not_a_console(hStdout): real_stdout = False". not_a_console uses GetFileType and GetConsoleMode to check whether that handle is directed to

[issue1602] windows console doesn't print utf8 (Py30a2)

STINNER Victor added the comment: amaury> The script unicode2.py uses the console STD_OUTPUT_HANDLE iff amaury> sys.stdout.fileno()==1 Interesting article about the Windows console: http://blogs.msdn.com/b/michkap/archive/2008/03/18/8306597.aspx There is an example which has many tests to chec

[issue1602] windows console doesn't print utf8 (Py30a2)

Amaury Forgeot d'Arc added the comment: The script unicode2.py uses the console STD_OUTPUT_HANDLE iff sys.stdout.fileno()==1. But is it always the case? What about pythonw.exe? Also some applications may redirect fd=1: I'm sure that py.test does this http://pytest.org/capture.html#setting-cap

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by Tim Golden : -- versions: +Python 3.3 -Python 3.1, Python 3.2 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsub

[issue1602] windows console doesn't print utf8 (Py30a2)

Tim Golden added the comment: Reopening as there seems to be some possibility of progress -- nosy: -BreamoreBoy resolution: invalid -> status: closed -> open ___ Python tracker ___

[issue1602] windows console doesn't print utf8 (Py30a2)

David-Sarah Hopwood added the comment: I'll have a look at the Py3k I/O internals and see what I can do. (Reopening a bug appears to need Coordinator permissions.) -- ___ Python tracker

[issue1602] windows console doesn't print utf8 (Py30a2)

Glenn Linderman added the comment: I would certainly be delighted if someone would reopen this issue, and figure out how to translate unicode2.py to Python internals so that Python's console I/O on Windows would support Unicode "out of the box". Otherwise, I'll have to include the equivalent

[issue1602] windows console doesn't print utf8 (Py30a2)

David-Sarah Hopwood added the comment: haypo wrote: > davidsarah wrote: >> It is certainly possible to write Unicode to the console >> successfully using WriteConsoleW > > Did you tried with characters not encodable to the code page and with > character that cannot be rendeded by the font? Ye

[issue1602] windows console doesn't print utf8 (Py30a2)

STINNER Victor added the comment: > It is certainly possible to write Unicode to the console > successfully using WriteConsoleW Did you tried with characters not encodable to the code page and with character that cannot be rendeded by the font? See msg120414 for my tests with WriteConsoleOut

[issue1602] windows console doesn't print utf8 (Py30a2)

David-Sarah Hopwood added the comment: Glenn Linderman wrote: > I skipped the unmangling of command-line arguments, because it produced an > error I didn't understand, about needing a buffer protocol. If I understand correctly, that part isn't needed on Python 3 because issue2128 is already f

[issue1602] windows console doesn't print utf8 (Py30a2)

Glenn Linderman added the comment: Interesting! I was able to tweak David-Sarah's code to work with Python 3.x, mostly doing things that 2to3 would probably do: changing unicode() to str(), dropping u from u'...', etc. I skipped the unmangling of command-line arguments, because it produced

[issue1602] windows console doesn't print utf8 (Py30a2)

David-Sarah Hopwood added the comment: It is certainly possible to write Unicode to the console successfully using WriteConsoleW. This works regardless of the console code page, including 65001. The code http://tahoe-lafs.org/trac/tahoe-lafs/browser/src/allmydata/windows/fixups.py";>here does

[issue1602] windows console doesn't print utf8 (Py30a2)

STINNER Victor added the comment: I don't understand exactly the goal of this issue. Different people described various bugs of the Windows console, but I don't see any problem with Python here. It looks like it's just not possible to display correctly unicode with the Windows console (the wh

[issue1602] windows console doesn't print utf8 (Py30a2)

Χρήστος Γεωργίου (Christos Georgiou) added the comment: http://blogs.msdn.com/b/michkap/archive/2008/03/18/8306597.aspx If you want any kind of Unicode output in the console, the font must be an “official” MS console TTF (“official” as defined by the Windows version); I believe only Lucida C

[issue1602] windows console doesn't print utf8 (Py30a2)

STINNER Victor added the comment: sys_write_stdtout.patch: Create sys.write_stdout() test function to call WriteConsoleOutputA() or WriteConsoleOutputW() depending on the input types (bytes or str). -- keywords: +patch Added file: http://bugs.python.org/file19493/sys_write_stdout.patc

[issue1602] windows console doesn't print utf8 (Py30a2)

STINNER Victor added the comment: I wrote a small function to call WriteConsoleOutputA() and WriteConsoleOutputW() in Python to do some tests. It works correclty, except if I change the code page using chcp command. It looks like the problem is that the chcp command changes the console code

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by David Sankel : -- nosy: +David.Sankel ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue1602] windows console doesn't print utf8 (Py30a2)

Mark Lawrence added the comment: @Brian/Tim what's your take on this? -- nosy: +BreamoreBoy, brian.curtin, tim.golden ___ Python tracker ___ _

[issue1602] windows console doesn't print utf8 (Py30a2)

STINNER Victor added the comment: > Maybe the solution is to use the win32 console API directly... Yes, it is the best solution because it avoids the horrible mbcs encoding. About cp65001: it is not *exactly* the same encoding than utf-8 and so it cannot be used as an alias to utf-8: see issu

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by Antoine Pitrou : -- stage: unit test needed -> needs patch versions: +Python 3.2 -Python 3.0 ___ Python tracker ___ ___ Pyth

[issue1602] windows console doesn't print utf8 (Py30a2)

Christoph Burgmer added the comment: Will this bug be tackled or Python2.7? And is there a way to get hold of the access denied error? Here are my steps to reproduce: I started the console with "cmd /u /k chcp 65001" ___ Akti

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by sorin : -- nosy: +sorin ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailma

[issue1602] windows console doesn't print utf8 (Py30a2)

Glenn Linderman added the comment: The choice of the Lucida Consola or the Consolas font cures most of the rectangle problems. Those are just a limitation of the selected font for the console window. -- ___ Python tracker

[issue1602] windows console doesn't print utf8 (Py30a2)

Marc-Andre Lemburg added the comment: Mark Summerfield wrote: > > Mark Summerfield added the comment: > > Glenn Linderman's fix pretty well works for me on XP Home. I can print > every Unicode character up to and including U+D7FF (although most just > come out as rectangles, at least I don't

[issue1602] windows console doesn't print utf8 (Py30a2)

Mark Summerfield added the comment: Glenn Linderman's fix pretty well works for me on XP Home. I can print every Unicode character up to and including U+D7FF (although most just come out as rectangles, at least I don't get encoding errors). It fails at U+D800 with message: UnicodeEncodeError:

[issue1602] windows console doesn't print utf8 (Py30a2)

Glenn Linderman added the comment: With Python 3.1.1, the following batch file seems to be necessary to use UTF-8 successfully from an XP console: set PYTHONIOENCODING=UTF-8 cmd /u /k chcp 65001 set PYTHONIOENCODING= exit the cmd line seems to be necessary because of Windows having compatibili

[issue1602] windows console doesn't print utf8 (Py30a2)

Χρήστος Γεωργίου (Christos Georgiou) added the comment: Another note: if one creates a dummy Stream object (having a softspace attribute and a write method that writes using os.write, as in http://stackoverflow.com/questions/878972/windows-cmd-encoding-change-causes-python-crash/1432462#1432462

[issue1602] windows console doesn't print utf8 (Py30a2)

Χρήστος Γεωργίου (Christos Georgiou) added the comment: Just in case it helps, this behaviour is on Win XP Pro, Python 2.5.1: First, I added an alias for 'cp65001' to 'utf_8' in Lib/encodings/aliases.py . Then, I opened a command prompt with a bitmap font. c:\windows\system32>python Python 2

[issue1602] windows console doesn't print utf8 (Py30a2)

Amaury Forgeot d'Arc added the comment: an immediate thing to do is to declare cp65001 as an encoding: Index: Lib/encodings/aliases.py === --- Lib/encodings/aliases.py(revision 72757) +++ Lib/encodings/aliases.py(working co

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by Antoine Pitrou : -- nosy: +amaury.forgeotdarc ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue1602] windows console doesn't print utf8 (Py30a2)

Antoine Pitrou added the comment: Yes, it is a Windows problem. There simply doesn't seem to be a true Unicode codepage for command-line apps. Recommend closing. -- nosy: +pitrou ___ Python tracker

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by STINNER Victor : -- components: +Windows ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by STINNER Victor : -- components: -Windows ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue1602] windows console doesn't print utf8 (Py30a2)

Changes by Daniel Diniz : -- nosy: +ezio.melotti, haypo stage: -> test needed versions: +Python 3.1 ___ Python tracker ___ ___ Python-

[issue1602] windows console doesn't print utf8 (Py30a2)

Christian Heimes added the comment: We are aware of multiple Windows related problems. We are planing to rewrite parts of the Windows specific API to use the widechar variants. Maybe that will help. -- keywords: +py3k nosy: +tiran priority: -> low __ Tra

[issue1602] windows console doesn't print utf8 (Py30a2)

Mark Summerfield added the comment: I've looked into this a bit more, and from what I can see, code page 65001 just doesn't work---so it is a Windows problem not a Python problem. A possible solution might be to read/write UTF16 which "managed" Windows applications can do. __

[issue1602] windows console doesn't print utf8 (Py30a2)

New submission from Mark Summerfield: I am not sure if this is a Python bug or simply a limitation of cmd.exe. I am using Windows XP Home. I run cmd.exe with the /u option and I have set my console font to "Lucida Console" (the only TrueType font offered), and I run chcp 65001 to set the utf8 co