[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: boost-python was using the removed private _Py_fopen() function, I proposed https://github.com/boostorg/python/pull/344 to replace _Py_fopen() with fopen(). -- ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset a12491681f08a33abcca843f5150330740c9 by Victor Stinner in branch 'master': bpo-32381: pymain_run_command() uses PyCF_IGNORE_COOKIE (GH-23724) https://github.com/python/cpython/commit/a12491681f08a33abcca843f5150330740c9 --

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset ceb420251c1d635520049fbb7b5269a73d63fb58 by Victor Stinner in branch 'master': bpo-32381: pymain_run_file() uses PySys_FormatStderr() (GH-23778) https://github.com/python/cpython/commit/ceb420251c1d635520049fbb7b5269a73d63fb58 --

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: My PR 23778 fix the encoding/error handler when writing the filename into stderr, when the file does not exist: $ LANG= PYTHONCOERCECLOCALE=0 ./python -X utf8=0 héllo.py ./python: can't open file '/home/vstinner/python/master/h\udcc3\udca9llo.py': [Errno 2]

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22636 pull_request: https://github.com/python/cpython/pull/23778 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22585 pull_request: https://github.com/python/cpython/pull/23724 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset a82f63f5af027a0eab0f0812d750b804368cbd25 by Victor Stinner in branch 'master': bpo-32381: Add _PyRun_AnyFileObject() (GH-23723) https://github.com/python/cpython/commit/a82f63f5af027a0eab0f0812d750b804368cbd25 --

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22584 pull_request: https://github.com/python/cpython/pull/23723 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset ca064402079f889226cb107b26b329891431c319 by Victor Stinner in branch 'master': bpo-32381: Remove unused _Py_fopen() function (GH-23711) https://github.com/python/cpython/commit/ca064402079f889226cb107b26b329891431c319 --

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22574 pull_request: https://github.com/python/cpython/pull/23711 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset 550e4673be538d98b6ddf5550b3922539cf5c4b2 by Victor Stinner in branch 'master': bpo-32381: Add _PyRun_SimpleFileObject() (GH-23709) https://github.com/python/cpython/commit/550e4673be538d98b6ddf5550b3922539cf5c4b2 --

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22573 pull_request: https://github.com/python/cpython/pull/23709 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset 815506d852daabc40e14ff0987c1142c0205fbe7 by Victor Stinner in branch 'master': bpo-32381: Rewrite PyErr_ProgramText() (GH-23700) https://github.com/python/cpython/commit/815506d852daabc40e14ff0987c1142c0205fbe7 --

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by Alexey Izbyshev : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: -Python 3.7 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Alexey Izbyshev added the comment: Thanks for the fix and backports! -- resolution: fixed -> stage: resolved -> patch review status: closed -> open versions: +Python 3.7 ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22567 pull_request: https://github.com/python/cpython/pull/23700 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: It's now fixed in 3.8, 3.9 and master branches. Thanks for the bug report tianjg. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: -Python 3.7 ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset b5cf308de8b19bf8f77053013f7e8a944159e1aa by Victor Stinner in branch '3.8': bpo-32381: Fix PyRun_SimpleFileExFlags() encoding (GH-23642) (GH-23692) (GH-23696) https://github.com/python/cpython/commit/b5cf308de8b19bf8f77053013f7e8a944159e1aa

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22563 pull_request: https://github.com/python/cpython/pull/23696 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset f0e42ae03c41ec32fcb3064772f46ff7f2c5ff3b by Victor Stinner in branch '3.9': bpo-32381: Fix PyRun_SimpleFileExFlags() encoding (GH-23642) (GH-23692) https://github.com/python/cpython/commit/f0e42ae03c41ec32fcb3064772f46ff7f2c5ff3b --

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22561 pull_request: https://github.com/python/cpython/pull/23692 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: New changeset b6d98c10fff6f320f8fdf595c3f9a05d8be4e31d by Victor Stinner in branch 'master': bpo-32381: Fix PyRun_SimpleFileExFlags() encoding (GH-23642) https://github.com/python/cpython/commit/b6d98c10fff6f320f8fdf595c3f9a05d8be4e31d --

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: > Isn't it a part of the public API? I can't find it in the docs, but it seems > to be declared in the public header. The Python C API has a strange history... -- ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Alexey Izbyshev added the comment: > It seems like PyErr_ProgramText() is no longer used in Python. Isn't it a part of the public API? I can't find it in the docs, but it seems to be declared in the public header. -- ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: > Just so it doesn't get lost: the problem with the contract of > PyErr_ProgramText() which I mentioned in my dup 42568 is still there. It seems like PyErr_ProgramText() is no longer used in Python. PyErr_ProgramTextObject() is used and it pass the filename

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Alexey Izbyshev added the comment: Thanks for the patch, Victor, it looks good. Just so it doesn't get lost: the problem with the contract of PyErr_ProgramText() which I mentioned in my dup 42568 is still there. -- ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: bpo-42568 is marked as a duplicate of this issue. -- ___ Python tracker ___ ___ Python-bugs-list

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: I can reproduce the issue on Python 3.10 with a script called 北京市.py which contains: print("hello"). c:\> python 北京市.py hello c:\>python __pycache__\北京市.cpython-310.pyc python: Can't reopen .pyc file And with my PR 23642 fix, it works as expected:

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by STINNER Victor : -- pull_requests: +22510 pull_request: https://github.com/python/cpython/pull/23642 ___ Python tracker ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Alexey Izbyshev added the comment: Thanks, Eryk, for catching the dup, I missed it somehow. @ZackerySpytz: do you plan to proceed with your PR? If not, I can pick it up -- this issue broke the software I develop after upgrade to 3.8. I filed issue 42569 to hopefully clarify the status of

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by Eryk Sun : -- versions: +Python 3.10 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: Hum. In fact, this problem can be fixed differently: modify PyRun_xxx() functions to pass the filename as an Unicode string. Maybe pass it as a wchar_t* string or even a Python str object. -- ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by Zackery Spytz : -- nosy: +ZackerySpytz versions: +Python 3.8, Python 3.9 -Python 3.6 ___ Python tracker ___ ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by Zackery Spytz : -- keywords: +patch pull_requests: +14498 stage: test needed -> patch review pull_request: https://github.com/python/cpython/pull/14699 ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: > I would make _Py_fopen() more compatible with the PEP 529. Typo: It* would -- ___ Python tracker

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

STINNER Victor added the comment: run_file() gets a wchar_t* string which comes from wmain() argv. run_file() encodes the wchar_t* using PyUnicode_EncodeFSDefault(). Later, PyRun_SimpleFileExFlags() calls indirectly fopen() with the encoded filename. > This could

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Change by R. David Murray : -- keywords: +3.6regression ___ Python tracker ___ ___

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Eryk Sun added the comment: Workarounds: (1) force 3.6 to use the legacy ANSI filesystem encoding by setting the environment variable PYTHONLEGACYWINDOWSFSENCODING. (2) Use 8.3 DOS names, if creating them is enabled on the volume. You can check their value in CMD via `dir

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Tianjg added the comment: Thanks a lot. What should I do to reopen .pyc file with non-ASCII path use python3.6 in cmd？Could you give me* some **code examples*.Thank you again, and I look forward to hearing from you 2017-12-20 15:35 GMT+08:00 Eryk Sun

[issue32381] Python 3.6 cannot reopen .pyc file with non-ASCII path

Eryk Sun added the comment: run_file encodes the file path via PyUnicode_EncodeFSDefault, which encodes as UTF-8 in Windows, starting with 3.6. PyRun_SimpleFileExFlags subsequently tries to open this encoded path via _Py_fopen, which calls fopen. The CRT expects an ANSI