[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: > bpo-42846: Convert CJK codec extensions to multiphase init (GH-24157) I added a new test and new test spotted a reference leak, likely an existing one: bpo-42866 "test test_multibytecodec: Test_IncrementalEncoder.test_subinterp() leaks references". --

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: > 1) python -m test --verbose test_threading > 2) python -m test --verbose test_embed I ran manually these two tests with cp932 ANSI code page: they now pass with my fix. I also added a regression test to test_multibytecodec.py. Thanks for your quick bug re

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: New changeset 07f2cee93f1b619650403981c455f47bfed8d818 by Victor Stinner in branch 'master': bpo-42846: Convert CJK codec extensions to multiphase init (GH-24157) https://github.com/python/cpython/commit/07f2cee93f1b619650403981c455f47bfed8d818 --

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: Ah, if you don't want to change the ANSI code page to cp932 (Japanese language) just to reproduce the issue, you can just set the stdio encoding: - C:\> set PYTHONIOENCODING=cp932 C:\> python t.py|more sys.stdout.encoding='cp1250' TypeError: codec is unex

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: Simpler way to reproduce the issue with t.py script: --- import test.support import sys import _testcapi print(f"{sys.stdout.encoding=}", file=sys.stderr) with test.support.SuppressCrashReport(): _testcapi.run_in_subinterp("pass") --- By default, UTF-8

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: Attached PR 24157 should fix the issue. > FAIL: test_daemon_threads_fatal_error > (test.test_threading.SubinterpThreadingTests) This test runs code in a subinterpreter which is run in a subprocess. The problem is not in the code run in the subinterpreter, b

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

Change by STINNER Victor : -- keywords: +patch pull_requests: +22985 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24157 ___ Python tracker ___ _

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: I'm working on a fix. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: h

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

Erlend Egeberg Aasland added the comment: It should be sufficient to convert cjkcodecs.h to multi-phase init then? From what I can see, the support modules are state less, right? -- ___ Python tracker _

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: It took me a while to understand it, the _multibytecodec module itself is fine. The issue comes from the _codecs_jp module which uses the legacy module API: codec = _codecs_jp.getcodec('cp932') -- ___ Python t

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

STINNER Victor added the comment: I can reproduce the issue on Windows configured in Japanese language: ANSI code page cp932. I managed to reproduce the bug on Linux with attached bug.py -- Added file: https://bugs.python.org/file49727/bug.py ___

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

Erlend Egeberg Aasland added the comment: I'm unable to reproduce this on Windows 10 (amd64). What's your exact locale setting? Are you compiling with HEAD at 0b858cdd5d114f0890b11b6c4d6559d0ceb468ab? -- ___ Python tracker

[issue42846] Using _multibytecodec module on Windows, test_threading/embed get failure

New submission from neonene : After https://github.com/python/cpython/commit/0b858cdd5d114f0890b11b6c4d6559d0ceb468ab (bpo-1635741: Convert _multibytecodec to multi-phase init), On Windows x64/x86 with chinese/japanese/korean system-locale, MultibyteCodec_Check() in multibytecodec.c returns fa