New submission from Eryk Sun <[email protected]>:
io._WindowsConsoleIO reads from the console via ReadConsoleW in line-input
mode, which limits the line length to the maximum of 256 and the size of the
client buffer, including the trailing CRLF (or just CR if processed-input mode
is disabled). Text that's typed or pasted beyond the length limit is ignored.
The call returns when a carriage return ("\r") is read or the user types the
enter key anywhere on the line.
Currently the buffer that _WindowsConsoleIO passes to ReadConsoleW is capped at
512 wide characters, based on the C runtime's BUFSIZ (512) macro. This is too
small. Legacy mode (i.e. PYTHONLEGACYWINDOWSSTDIO) uses io.FileIO with an 8 KiB
buffer, which is 8K characters when the console input codepage is a legacy
single-byte encoding. _WindowsConsoleIO should support at least this much.
I'd prefer that it allowed up to 32K characters, which is the upper limit for a
process command line or for a long filepath. By way of comparison, input(),
which calls _PyOS_WindowsConsoleReadline if stdin is a console file, is
currently capped at 16K characters.
To be able to read up to 32K characters also requires increasing the
BufferedReader default buffer size and TextIOWrapper chunk size to 96 KiB (BMP
codes in the range 0x0800-0xFFFF encode as a 3-byte UTF-8 sequence) in order to
ensure that f.readline() and f.buffer.readline() request the maximum size. This
would require changing _io__WindowsConsoleIO___init___impl to set
`self->blksize = 96 * 1024` when `console_type == 'r'`, as well as changing
_io_open_impl to manually set the _CHUNK_SIZE of the TextIOWrapper to 96 KiB
for console input (type 'r'). Maybe TextIOWrapper should just query the raw
_blksize as the initial chunk size. That would remove the need to manually set
_CHUNK_SIZE.
----------
components: IO, Unicode, Windows
messages: 377436
nosy: eryksun, ezio.melotti, paul.moore, steve.dower, tim.golden, vstinner,
zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Support reading long lines with io._WindowsConsoleIO
type: enhancement
versions: Python 3.10, Python 3.9
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue41849>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
