Checked Python 3.4 (latest hg checkout) and this area of the code hasn't changed: pythonrun,c initstdio()
reported the issue: http://bugs.python.org/issue16129 On Thu, Oct 4, 2012 at 2:51 PM, Guido van Rossum <gu...@python.org> wrote: > This seems more fit for the tracker; can you file there? (Then post the > issue link here.) I do think you have a legitimate use case to set the > default encoding to utf-8. (Though there may be a way already.) Does Python > 3.3 have te same bug? > > > On Wednesday, October 3, 2012, Campbell Barton wrote: >> >> On Thu, Oct 4, 2012 at 1:35 PM, Campbell Barton <ideasma...@gmail.com> >> wrote: >> > Hi, >> > >> > We've run into an issue recently with blender3d on ms-windows where we >> > want to enforce the encoding is UTF-8 with the embedded python >> > interpreter. >> > (the encoding defaults to cp437). >> > >> > I naively thought setting the environment variable before calling >> > Py_Initialize() would work, but the way python DLL loads, it gets its >> > own environment variables that cant be modified directly [1]. >> > eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape"); >> > >> > We had bug reports by windows users not able to export files because >> > the stdout errors on printing paths with unsupported encoding. [2],[3] >> > >> > --- >> > >> > Of course we could distribute blender with a bat file launcher that >> > sets env variables, or ask the user to set their env variable - but I >> > dont think this is really a good option. >> > >> > I tried overriding the stderr & stdout, but this caused another bug in >> > a part of out code that catches exception messages from the stderr. >> > [4] >> > import sys, io >> > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', >> > errors='surrogateescape', line_buffering=True) >> > sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', >> > errors='surrogateescape', line_buffering=True) >> > >> > >> > >> > IMHO either of these solutions would be fine. >> > >> > * have a PyOS_PutEnv() function, gettext has gettext_putenv() to >> > workaround this problem. >> > >> > * manage this the same as Py_GetPythonHome(), which can be defined by >> > the embedding application to override the default. >> > >> > >> > Id like to know if there is some known solution to workaround this >> > issue, if not - would either of these would be acceptable in python >> > (can write up a patch if it helps) >> > >> > Regards, >> > Campbell >> > >> > --- >> > >> > [1] >> > http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe >> > [2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750 >> > [3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555 >> > [4] http://projects.blender.org/tracker/?func=detail&aid=32720 >> >> To follow up and give a correction to overwriting sys.stdout/stderr, >> The issue seemed to be that __stderr__/__stdout__ was later >> overwritten, loosing the original reference to the buffer (maybe >> refcount issue here?), either way it would silence the output. >> >> >> import sys, io >> sys.__stdout__ = sys.stdout = >> io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1), >> encoding='utf-8', errors='surrogateescape', newline="\n", >> line_buffering=True) >> sys.__stderr__ = sys.stderr = >> io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1), >> encoding='utf-8', errors='surrogateescape', newline="\n", >> line_buffering=True) >> >> This all works as expected without bug [4] (above), however on exit I >> get an assert in MSVCR90.DLL's write.c (called from python32_d.dll): >> _VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1); >> >> I'd rather not loose more time debugging why this assert happens, >> IMHO this is too low-level a way to change the encoing of stdio/stderr >> and error-prone too, so some way to reliably set PYTHONIOENCODING from >> a program embedding python is still needed. >> >> -- >> - Campbell >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > -- > --Guido van Rossum (python.org/~guido) -- - Campbell _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
