Dāvis added the comment:
There is right encoding, it's encoding that's actually used. Here we're inside
subprocess.Popen which does the actual winapi.CreateProcess call and thus we
can check for any creationflags and adjust encoding logic accordingly. I would
say almost all Windows console programs does use console's encoding for
input/output because otherwise user wouldn't be able to read it. And programs
which use different encoding it would be caller's responsibly to set used
encoding because only it could know which encoding to use for that program.
So I think basically Popen should accept encoding parameter which would be then
passed to TextIOWrapper. Preferably way to set different encoding for
stdin/out/err
and then if there's no encoding specified, we use our logic to determine
default encoding which would be by using _Py_device_encoding(fd) and this would
be right for almost all if not all cases. And if some program changes console's
encoding after we got consoles encoding, we could get encoding again after
program's execution and then use this new set console's encoding.
Anyway while looking more into this I found why we get wrong encoding.
looking at subprocess.check_output can see
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
**kwargs).stdout
that stdout is set to PIPE
and then in subprocess.__init__
if c2pread != -1:
self.stdout = io.open(c2pread, 'rb', bufsize)
if universal_newlines:
self.stdout = io.TextIOWrapper(self.stdout)
there c2pread will be fd for pipe (3)
when looking inside _io_TextIOWrapper___init___impl
fileno = _PyObject_CallMethodId(buffer, &PyId_fileno, NULL);
[...]
int fd = _PyLong_AsInt(fileno);
[...]
self->encoding = _Py_device_encoding(fd);
[...]
we'll set encoding with _Py_device_encoding(3);
but there
if (fd == 0)
cp = GetConsoleCP();
else if (fd == 1 || fd == 2)
cp = GetConsoleOutputCP();
else
cp = 0;
so encoding would be correct for stdin/stdout/stderr but not for pipe and
that's why this issue.
I see 2 ways to fix this and I've added patches for both options.
----------
Added file: http://bugs.python.org/file43100/subprocess_fix_encoding_v2_a.patch
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue27179>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
