[issue11395] print(s) fails on Windows with long strings

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue11395] print(s) fails on Windows with long strings

Eryk Sun added the comment: > the problem (or at least with these numbers) occurs only when > the code is saved in a script, and this is run by double- > clicking the file The .py file association is probably using a different version of Python, or it's associated with the py launcher and

[issue11395] print(s) fails on Windows with long strings

Adam Bartoš added the comment: I've been hit by this issue recently. On my configuration, print("a" * 10215) fails with an infinite loop of OSErrors (WinError 8). This even cannot by interrupted with Ctrl-C nor the exception can be catched. - print("a" * 10214) is fine - print("a" * 10215)

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: This issue is closed. You should reopen it or open a new one. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395 ___

[issue11395] print(s) fails on Windows with long strings

eryksun added the comment: The buffer size only needs to be capped if WINVER 0x602. This issue doesn't apply to Windows 8 since it uses the ConDrv device driver instead of LPC. Prior to Windows 8, WriteFile redirects to WriteConsoleA when passed a console handle. This makes an LPC call to

[issue11395] print(s) fails on Windows with long strings

David-Sarah Hopwood david-sa...@jacaranda.org added the comment: If I understand the bug in the Windows console functions correctly, a limit of 32767 bytes might not always be small enough. The problem is that if two or more threads are concurrently using any console functions (which all use

[issue11395] print(s) fails on Windows with long strings

Roundup Robot devnull@devnull added the comment: New changeset 8939a21bdb94 by Victor Stinner in branch '3.2': Issue #11395: io.FileIO().write() clamps the data length to 32,767 bytes on http://hg.python.org/cpython/rev/8939a21bdb94 New changeset 4b3472169493 by Victor Stinner in branch

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: I realized that it was a little more difficult to port the fix on 3.1 because 3.1 doesn't have the fix for Windows 64 bits. So I only fixed Python 3.2 and 3.3, also because nobody reported failure for Python 3.1 on Windows with -u

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: I tried to commit io_write.patch, but I had problems with Mercurial :-) I will commit it later. -- Added file: http://bugs.python.org/file21030/io_write.patch ___ Python tracker

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: This last patch looks good, except that the comments if stdout mode is binary (python -u) are incorrect: since r87824, all files are opened in binary mode. -- ___ Python tracker

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: This last patch looks good, except that the comments if stdout mode is binary (python -u) are incorrect: since r87824, all files are opened in binary mode. I plan to commit the patch to 3.1 and then forward port to 3.2 and 3.3.

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: I did some tests: os.write(1, b'X'*length) does always fail with length = 63842. It does sometimes fail with length 35000. The maximum looks completly random: as written in Microsoft documentation, The maximum size of the buffer

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: Remarks about test_wconsole_binlarge.patch: - I don't know if isatty() is cheap or not. Is it a system call? If it might be slow, it should be only be called once in the constructor. On Windows, I don't think that isatty(fd)

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: Other remarks about test_wconsole_binlarge.patch: - the patch doesn't apply on Python 3.3 - I would prefer 32767 instead of 32000 for the maximum length Suggestion for the comment in fileio.c: * Issue #11395: not enough space

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya santoso.wij...@gmail.com added the comment: Thanks for the comment. It's my first patch. :-) - the patch doesn't apply on Python 3.3 That latest patch file I generated against the tip of 3.1 branch. Should I create two separate patches for 3.1 and 3.2+ (which will apply on

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya santoso.wij...@gmail.com added the comment: Attached a modified patch that should work against 3.2+ heads: - Added `isatty` bit field in isatty that's evaluated during its construction. This should eliminate the need to call `isatty()` on every write. - Cap buffer length to

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: On Windows, isatty() is a cheap call: a simple lookup in the _ioinfo structure. And dup2() can still change the destination of a file descriptor, so the new attribute can be out of sync... I suggest to call isatty() on every write.

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya santoso.wij...@gmail.com added the comment: FWIW, here's the Microsoft's source for isatty (in VC\crt\src\isatty.c): /*** *int _isatty(handle) - check if handle is a device * *Purpose: * Checks if the given handle is associated with a character device * (terminal,

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya santoso.wij...@gmail.com added the comment: Attached a version of the last patch without `.isatty` caching. -- Added file: http://bugs.python.org/file21025/winconsole_large_py33_direct.patch ___ Python tracker rep...@bugs.python.org

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya santoso.wij...@gmail.com added the comment: And a patch for the test + fix. -- Added file: http://bugs.python.org/file21012/wconsole_large.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya santoso.wij...@gmail.com added the comment: Indeed, Python3.1 fails with the -u option. I'm also attaching another test to reproduce the crash with '-u' option. -- Added file: http://bugs.python.org/file21013/test_wconsole_binlarge.patch

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Likewise, this fails with 3.2:: import os os.write(1, ba * 66000) -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395

[issue11395] print(s) fails on Windows with long strings

Changes by Santoso Wijaya santoso.wij...@gmail.com: -- nosy: +santa4nt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395 ___ ___

[issue11395] print(s) fails on Windows with long strings

Charles-Francois Natali neolo...@free.fr added the comment: It's probably a Windows limitation regarding the number of bytes that can be written to stdout in one write. As for the difference between python versions, what does python -c import sys; print(sys.getsizeof('a')) return ? --

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: no, it works with 3.2b2 (r32b2:87398), and fails with 3.2 final (r32:88445) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: Extract of issue #1602: WriteConsoleW has one bug that I know of, which is that it a href=http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1232;fails when writing more than 26608 characters at once/a. That's easy to work around by

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: Extract of the WriteConsole Function: The storage for this buffer is allocated from a shared heap for the process that is 64 KB in size. The maximum size of the buffer will depend on heap usage.

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: This changed with r87824 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395 ___ ___

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: This changed with r87824 Yes, I changed Python to open all files in binary mode. With Python 3.2, you can open sys.std* streams in binary mode using -u command line option (u like unbuffered, not Unicode ;-)). --

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Indeed, Python3.1 fails with the -u option. Before r87824, the C call to write() performed CRLF conversion. In the implementation of MSVCRT, a local buffer is used (1025 chars in vs8.0, 5*1024 in vs10.0), so WriteFile is called with

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: Anyway, use os.write() to write unicode into the Windows console is not the right thing to do. We should use WriteConsoleW(): #1602 is the correct fix for this issue. -- ___ Python

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: I'm writing bytes here: os.write(1, bb * 66000) And WriteConsole has the same issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395

[issue11395] print(s) fails on Windows with long strings

STINNER Victor victor.stin...@haypocalc.com added the comment: And WriteConsole has the same issue. print() (sys.stdout and sys.stderr) should use WriteConsoleW() and use small chunks (smaller than 64 KB, I don't know the safest size). -- ___

[issue11395] print(s) fails on Windows with long strings

Antoine Pitrou pit...@free.fr added the comment: IIUC, this is a Windows bug? Is there any easy workaround for us? -- versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: It may be a windows bug, but it's also an a python regression! A fix is to limit the number of chars: === --- D:/py3k/Modules/_io/fileio.c (revision 87824) +++

[issue11395] print(s) fails on Windows with long strings

Changes by Santoso Wijaya santoso.wij...@gmail.com: -- components: +Windows ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395 ___ ___

[issue11395] print(s) fails on Windows with long strings

Terry J. Reedy tjre...@udel.edu added the comment: print(a*66000) works (after some delay) running from an IDLE edit window (but see #144249). Works means that I get a working prompt back with no errors. Unlike IDLE, the Command Prompt Windows keeps a limited number of lines in its buffers

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya santoso.wij...@gmail.com added the comment: I'm adding a test that will reproduce the crash. -- keywords: +patch Added file: http://bugs.python.org/file21003/writeconsole.patch ___ Python tracker rep...@bugs.python.org

[issue11395] print(s) fails on Windows with long strings

New submission from Case Van Horsen cas...@gmail.com: Python 3.2 fails when printing long strings. C:\Python32python Python 3.2 (r32:88445, Feb 20 2011, 21:30:00) [MSC v.1500 64 bit (AMD64)] on win32 Type help, copyright, credits or license for more information. print(a*66000) Traceback (most

[issue11395] print(s) fails on Windows with long strings

Changes by Amaury Forgeot d'Arc amaur...@gmail.com: -- nosy: +pitrou priority: normal - high ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11395 ___