[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Eryk Sun]

Change by Eryk Sun :


--
resolution:  -> third party
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Segev Finer]

Segev Finer  added the comment:

This should be fixed by the Windows 10 April Update (build 1803) according to 
https://github.com/Microsoft/vscode/issues/36630.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Segev Finer]

Segev Finer  added the comment:

Upstream issue https://github.com/Microsoft/console/issues/40, also has links 
to similar issues this caused in other projects.

--
nosy: +Segev Finer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Eryk Sun]

Eryk Sun  added the comment:

I was able to reproduce this problem in 3.6 in Windows 10 (1709), but only for 
code paths that call WriteFile, i.e. os.write and legacy standard I/O mode.

Writing to the console will block if there's an active text selection. The 
operation completes once the user copies the selection to the clipboard (e.g. 
by pressing enter). In the case of WriteFile, when the blocked call finally 
completes, the console mistakenly returns the number of internally written 
UTF-16 bytes. 

This bug is not present with WriteConsoleA, which means there's a simple 
workaround. _Py_write could detect a console handle and call WriteConsoleA 
instead of _write. 

The following test executes a write on a separate thread, after a timed delay 
that allows selecting text beforehand. Text can be selected via Ctrl+A, 
Shift+Up, or the mouse (the latter requires quick-edit mode or toggling mark 
mode). 

import os
import sys
import ctypes
import msvcrt
import threading

kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
fd_stdout = sys.stdout.fileno()
h_stdout = msvcrt.get_osfhandle(fd_stdout)
n = ctypes.c_ulong()

def test_write():
n.value = os.write(fd_stdout, b'spam\r\n')

def test_WriteFile():
kernel32.WriteFile(h_stdout, b'spam\r\n', 6, ctypes.byref(n), None)

def test_WriteConsoleA():
kernel32.WriteConsoleA(h_stdout, b'spam\r\n', 6, ctypes.byref(n), None)


For example, given manual text selection after starting the timer:

>>> threading.Timer(5, test_write).start()
>>> spam

>>> n
c_ulong(12)

>>> threading.Timer(5, test_WriteFile).start()
>>> spam

>>> n
c_ulong(12)

>>> threading.Timer(5, test_WriteConsoleA).start()
>>> spam

>>> n
c_ulong(6)

This test could be completely automated by combining SendInput (to set the 
session control-key state), GetConsoleWindow, and PostMessageW (WM_KEYDOWN, 
WM_KEYUP).

--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Simon Depiets]

Simon Depiets  added the comment:

The issue doesn't seem to happen on either 3.6 (with the new stdio mode) or 
with win_unicode_console enabled.

I was able to reproduce it on 3.6 with the PYTHONLEGACYWINDOWSSTDIO flag 
enabled, it's easier to trigger when using directional keys or mouse clicks 
within the console.

chcp : Active code page : 437

System locale : en-us

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Larry Hastings]

Change by Larry Hastings :


--
nosy:  -larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Eryk Sun]

Eryk Sun  added the comment:

We need a test that reproduces this problem on a vanilla installation of Python 
3.5. Include the system locale context as well, i.e. the ANSI codepage, OEM 
codepage, and active console output codepage. A reliable test will help 
determine whether this problem also affects legacy console I/O in Python 3.6. 
However, even if the problem affects 3.6, that doesn't mean there's anything we 
can reasonably do about it if the fault is the console host process (i.e. 
conhost.exe, running either ConhostV1.dll or the new ConhostV2.dll 
implementation). 

A possible workaround in Python 3.5 would be to install and enable the 
win_unicode_console package. This package uses the console's native Unicode API 
instead of the legacy codepage API.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Larry Hastings]

Larry Hastings  added the comment:

To confirm what Steve said: we no longer accept bug fixes for Python 3.5 (or 
3.4).  We only accept security fixes for 3.5 (and 3.4).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Steve Dower]

Steve Dower  added the comment:

Does this only affect Python 3.5? We're highly unlikely to take a fix for that 
version.

This code was rewritten for 3.6, so it wouldn't surprise me if there is no 
longer an issue.

--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Eryk Sun]

Change by Eryk Sun :


--
components: +Windows
nosy: +paul.moore, tim.golden, zach.ware
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Berker Peksag]

Change by Berker Peksag :


--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32245] OSError: raw write() returned invalid length on latest Win 10 Consoles

[Simon Depiets]

New submission from Simon Depiets :

A couple of users have been having issues on console output since the Fall 2017 
Creator Update on Windows 10

An OSError is triggered randomly when rewriting data on the console (typically 
with progress bars, for instance when you install a module with pip), this only 
happens with the Microsoft Console (within Powershell or cmd.exe).

It seems the windows stdout console stream returns a length double what python 
expects. I don't have the skills to go deeper than the bufferedio.c method 
_bufferedwriter_raw_write to diagnostic the issue, so I've made a very dirty 
fix (which I do not recommend) 
https://github.com/python/cpython/compare/3.5...LlianeFR:patch-1

Different unrelated use cases where an error is triggered :

https://stackoverflow.com/questions/47356993/oserror-raw-write-returned-invalid-length-when-using-print-in-python

https://github.com/Microsoft/vscode/issues/39149

--
components: IO
messages: 307811
nosy: Simon Depiets
priority: normal
severity: normal
status: open
title: OSError: raw write() returned invalid length on latest Win 10 Consoles
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com