[issue45793] execute shell command ./configure with python subprocess.popen, os.system ... checking build system type... is wrong
junyixie added the comment: > What's the architecture for the python binary (run "file $(which python3)" in > a shell to get this information, excluding the quotes)? yeah, is x86_64 python executable .../opt/anaconda3/bin/python3: Mach-O 64-bit executable x86_64 -- ___ Python tracker <https://bugs.python.org/issue45793> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45793] execute shell command ./configure with python subprocess.popen, os.system ... checking build system type... is wrong
junyixie added the comment: when run pyperformance compile in apple silicon also have this problem -- ___ Python tracker <https://bugs.python.org/issue45793> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45793] execute shell command ./configure with python subprocess.popen, os.system ... checking build system type... is wrong
junyixie added the comment: subprocess.Popen([], executable=os.path.join(cpython_dir, "configure"), cwd=cpython_dir).wait() -- ___ Python tracker <https://bugs.python.org/issue45793> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45793] execute shell command ./configure with python subprocess.popen, os.system ... checking build system type... is wrong
New submission from junyixie : wrong: checking build system type... x86_64-apple-darwin20.5.0 checking host system type... x86_64-apple-darwin20.5.0 expect: checking build system type... arm-apple-darwin20.5.0 checking host system type... arm-apple-darwin20.5.0 -- components: macOS messages: 406194 nosy: JunyiXie, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: execute shell command ./configure with python subprocess.popen, os.system ... checking build system type... is wrong type: behavior versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue45793> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
junyixie added the comment: About PEP How is the progress? Where can I track it? Is there any relevant plan? thanks -- ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44532] multi subinterpreters use _PyStructSequence_InitType failed.
junyixie added the comment:
Include/internal/pycore_structseq.h:
10
11
12: PyAPI_FUNC(int) _PyStructSequence_InitType(
13 PyTypeObject *type,
14 PyStructSequence_Desc *desc,
Modules/_cursesmodule.c:
4794 /* ncurses_version */
4795 if (NcursesVersionType.tp_name == NULL) {
4796: if (_PyStructSequence_InitType(,
4797 _version_desc,
4798
Py_TPFLAGS_DISALLOW_INSTANTIATION) < 0) {
Objects/structseq.c:
463
464 int
465: _PyStructSequence_InitType(PyTypeObject *type, PyStructSequence_Desc
*desc,
466 unsigned long tp_flags)
467 {
...
527 PyStructSequence_InitType2(PyTypeObject *type, PyStructSequence_Desc
*desc)
528 {
529: return _PyStructSequence_InitType(type, desc, 0);
530 }
531
Python/sysmodule.c:
2813 /* version_info */
2814 if (VersionInfoType.tp_name == NULL) {
2815: if (_PyStructSequence_InitType(,
2816 _info_desc,
2817
Py_TPFLAGS_DISALLOW_INSTANTIATION) < 0) {
2827 // sys.flags: updated in-place later by _PySys_UpdateConfig()
2828 if (FlagsType.tp_name == 0) {
2829: if (_PyStructSequence_InitType(, _desc,
2830
Py_TPFLAGS_DISALLOW_INSTANTIATION) < 0) {
2831 goto type_init_failed;
2837 /* getwindowsversion */
2838 if (WindowsVersionType.tp_name == 0) {
2839: if (_PyStructSequence_InitType(,
2840 _version_desc,
2841
Py_TPFLAGS_DISALLOW_INSTANTIATION) < 0) {
--
___
Python tracker
<https://bugs.python.org/issue44532>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45196] macOS. ./configure --with-address-sanitizer; make test; cause test case crash.
junyixie added the comment: ignore test_decimal.py crash backtrace. -- ___ Python tracker <https://bugs.python.org/issue45196> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45196] macOS. ./configure --with-address-sanitizer; make test; cause test case crash.
Change by junyixie : -- keywords: +patch pull_requests: +26742 stage: -> patch review pull_request: https://github.com/python/cpython/pull/28331 ___ Python tracker <https://bugs.python.org/issue45196> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45196] macOS. ./configure --with-address-sanitizer; make test; cause test case crash.
Change by junyixie : -- type: -> crash ___ Python tracker <https://bugs.python.org/issue45196> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45196] macOS. ./configure --with-address-sanitizer; make test; cause test case crash.
New submission from junyixie : test_io.py ``` = ==54932==ERROR: AddressSanitizer: requested allocation size 0x7fff (0x80001000 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x100 (thread T0) #0 0x102f1fa6c in wrap_malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3fa6c) #1 0x102565fcc in _buffered_init bufferedio.c:730 #2 0x10255bba4 in _io_BufferedReader___init__ bufferedio.c.h:435 #3 0x10226c8c8 in wrap_init typeobject.c:6941 #4 0x10216d3f8 in _PyObject_Call call.c:305 #5 0x102387a6c in _PyEval_EvalFrameDefault ceval.c:4285 #6 0x10237eaa8 in _PyEval_Vector ceval.c:5073 #7 0x102396860 in call_function ceval.c:5888 #8 0x102385444 in _PyEval_EvalFrameDefault ceval.c:4206 #9 0x10237eaa8 in _PyEval_Vector ceval.c:5073 #10 0x102396860 in call_function ceval.c:5888 #11 0x102385444 in _PyEval_EvalFrameDefault ceval.c:4206 #12 0x10237eaa8 in _PyEval_Vector ceval.c:5073 #13 0x102172bec in method_vectorcall classobject.c:53 #14 0x102396860 in call_function ceval.c:5888 #15 0x1023885e4 in _PyEval_EvalFrameDefault ceval.c:4221 #16 0x10237eaa8 in _PyEval_Vector ceval.c:5073 #17 0x102396860 in call_function ceval.c:5888 #18 0x102385444 in _PyEval_EvalFrameDefault ceval.c:4206 #19 0x10237eaa8 in _PyEval_Vector ceval.c:5073 #20 0x102172af4 in method_vectorcall classobject.c:83 #21 0x10216d0a8 in PyVectorcall_Call call.c:255 #22 0x102387a6c in _PyEval_EvalFrameDefault ceval.c:4285 #23 0x10237eaa8 in _PyEval_Vector ceval.c:5073 #24 0x10216c248 in _PyObject_FastCallDictTstate call.c:142 #25 0x10216dc00 in _PyObject_Call_Prepend call.c:431 #26 0x102268740 in slot_tp_call typeobject.c:7481 #27 0x10216c5d4 in _PyObject_MakeTpCall call.c:215 #28 0x102396b88 in call_function ceval.c #29 0x1023885e4 in _PyEval_EvalFrameDefault ceval.c:4221 ==54932==HINT: if you don't care about these errors you may set allocator_may_return_null=1 SUMMARY: AddressSanitizer: allocation-size-too-big (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3fa6c) in wrap_malloc+0x94 ==54932==ABORTING Fatal Python error: Aborted Current thread 0x000102e93d40 (most recent call first): File "/Users/xiejunyi/github-cpython/Lib/unittest/case.py", line 201 in handle File "/Users/xiejunyi/github-cpython/Lib/unittest/case.py", line 730 in assertRaises File "/Users/xiejunyi/github-cpython/Lib/test/test_io.py", line 1558 in test_constructor File "/Users/xiejunyi/github-cpython/Lib/unittest/case.py", line 549 in _callTestMethod File "/Users/xiejunyi/github-cpython/Lib/unittest/case.py", line 591 in run File "/Users/xiejunyi/github-cpython/Lib/unittest/case.py", line 650 in __call__ File "/Users/xiejunyi/github-cpython/Lib/unittest/suite.py", line 122 in run File "/Users/xiejunyi/github-cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/xiejunyi/github-cpython/Lib/unittest/suite.py", line 122 in run File "/Users/xiejunyi/github-cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/xiejunyi/github-cpython/Lib/unittest/suite.py", line 122 in run File "/Users/xiejunyi/github-cpython/Lib/unittest/suite.py", line 84 in __call__ File "/Users/xiejunyi/github-cpython/Lib/test/support/testresult.py", line 140 in run File "/Users/xiejunyi/github-cpython/Lib/test/support/__init__.py", line 990 in _run_suite File "/Users/xiejunyi/github-cpython/Lib/test/support/__init__.py", line 1115 in run_unittest File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/runtest.py", line 261 in _test_module File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/runtest.py", line 297 in _runtest_inner2 File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/runtest.py", line 335 in _runtest_inner File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/runtest.py", line 215 in _runtest File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/runtest.py", line 245 in runtest File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/runtest_mp.py", line 83 in run_tests_worker File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/main.py", line 678 in _main File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/main.py", line 658 in main File "/Users/xiejunyi/github-cpython/Lib/test/libregrtest/main.py", line 736 in main File "/Users/xiejunyi/github-cpython/Lib/test/regrtest.py", line 43 in _main File "/Users/xiejunyi/github-cpython/Lib/test/regrtest.py", line 47 in File "/Users/xiejunyi/github-cpython/Lib/runpy.py", line 86 in _run_code File "/Users/xiejunyi/github-cpython/Lib/runpy.py", line 196 in _run_module_as_main ``` test_d
[issue44532] multi subinterpreters use _PyStructSequence_InitType failed.
Change by junyixie : -- keywords: +patch pull_requests: +25516 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26949 ___ Python tracker <https://bugs.python.org/issue44532> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44532] multi subinterpreters use _PyStructSequence_InitType failed.
New submission from junyixie :
In _PyStructSequence_InitType, it will check type is initialized.
but when we have multi subinterpreters, type may be initialized expected.
when type already been initialized, should return 0 rather than throw exception.
```c
/* PyTypeObject has already been initialized */
if (Py_REFCNT(type) != 0) {
return 0;
}
```
```c
int
_PyStructSequence_InitType(PyTypeObject *type, PyStructSequence_Desc *desc,
unsigned long tp_flags)
{
PyMemberDef *members;
Py_ssize_t n_members, n_unnamed_members;
#ifdef Py_TRACE_REFS
/* if the type object was chained, unchain it first
before overwriting its storage */
if (type->ob_base.ob_base._ob_next) {
_Py_ForgetReference((PyObject *)type);
}
#endif
/* PyTypeObject has already been initialized */
if (Py_REFCNT(type) != 0) {
PyErr_BadInternalCall();
return -1;
}
```
--
components: Subinterpreters
messages: 396703
nosy: JunyiXie
priority: normal
severity: normal
status: open
title: multi subinterpreters use _PyStructSequence_InitType failed.
type: crash
___
Python tracker
<https://bugs.python.org/issue44532>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44374] PyThreadState_IsCurrent bug under building Python with --with-experimental-isolated-subinterpreters
New submission from junyixie :
under building Python with --with-experimental-isolated-subinterpreters
PyThreadState_IsCurrent use _PyRuntime.gilstate. is shared by multi sub
interpreters.
Use interpreter `gil->last_holder == state` can fix it?
```
static int
PyThreadState_IsCurrent(PyThreadState *tstate)
{
/* Must be the tstate for this thread */
struct _gilstate_runtime_state *gilstate = &_PyRuntime.gilstate;
assert(_PyGILState_GetThisThreadState(gilstate) == tstate);
return tstate == _PyRuntimeGILState_GetThreadState(gilstate);
}
```
```
static int
PyThreadState_IsCurrent(PyThreadState *tstate)
{
#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS
PyInterpreterState *interp = tstate->interp;
struct _ceval_state *ceval2 = >ceval;
struct _gil_runtime_state *gil = >gil;
return tstate == (PyThreadState*)_Py_atomic_load_relaxed(>last_holder);
#else
/* Must be the tstate for this thread */
struct _gilstate_runtime_state *gilstate = &_PyRuntime.gilstate;
assert(_PyGILState_GetThisThreadState(gilstate) == tstate);
return tstate == _PyRuntimeGILState_GetThreadState(gilstate);
#endif
}
```
--
components: Subinterpreters
messages: 395517
nosy: JunyiXie, vstinner
priority: normal
severity: normal
status: open
title: PyThreadState_IsCurrent bug under building Python with
--with-experimental-isolated-subinterpreters
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue44374>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43687] use unicode_state empty string before unicode_init. without define WITH_DOC_STRINGS
Change by junyixie : -- keywords: +patch pull_requests: +23876 stage: -> patch review pull_request: https://github.com/python/cpython/pull/25129 ___ Python tracker <https://bugs.python.org/issue43687> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43687] use unicode_state empty string before unicode_init. without define WITH_DOC_STRINGS
New submission from junyixie : use unicode_state empty string before unicode_init. without define WITH_DOC_STRINGS. PyType_Ready call PyUnicode_FromString, if doc string striped, cause crash. unicode_get_empty() must not be called before _PyUnicode_Init() or after _PyUnicode_Fini() PyType_Ready ``` const char *old_doc = _PyType_DocWithoutSignature(type->tp_name,type->tp_doc); PyObject *doc = PyUnicode_FromString(old_doc); ``` -- messages: 389950 nosy: JunyiXie priority: normal severity: normal status: open title: use unicode_state empty string before unicode_init. without define WITH_DOC_STRINGS type: crash versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43687> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43630] unittest use dir(module) to load test cases. Run unittest file by _PyRun_SimpleFileObject, have bug
New submission from junyixie :
_PyRun_SimpleFileObject load __main__ module at cache. Call this function
multiple times, the attributes stored in the module dict will affect eachother.
for example.
if we run test_A.py, call _PyRun_SimpleFileObject will create __main__ module,
test_A.py add some attribute in __main__ module dict.
now we run test_B.py. call _PyRun_SimpleFileObject will load cached __main__
module. now in __main__ module dict, we can get test_A's attribute.
in unittest, if we execute test, and don't exit. (unittest main.py
TestProgram), set exit=False.
```
def __init__(self, module='__main__', defaultTest=None, argv=None,
testRunner=None, testLoader=loader.defaultTestLoader,
exit=True, verbosity=1, failfast=None, catchbreak=None,
buffer=None, warnings=None, *, tb_locals=False):
```
dir(module), We got unexpected results
```
for name in dir(module):
...
```
then when unittest load tests. if we use _PyRun_SimpleFileObject to run
unittest, it will Repeated load test cases
```
for name in dir(module):
obj = getattr(module, name)
if isinstance(obj, type) and issubclass(obj, case.TestCase):
tests.append(self.loadTestsFromTestCase(obj))
```
```
int
_PyRun_SimpleFileObject(FILE *fp, PyObject *filename, int closeit,
PyCompilerFlags *flags)
{
PyObject *m, *d, *v;
int set_file_name = 0, ret = -1;
m = PyImport_AddModule("__main__");
if (m == NULL)
return -1;
Py_INCREF(m);
d = PyModule_GetDict(m);
```
--
messages: 389540
nosy: JunyiXie
priority: normal
severity: normal
status: open
title: unittest use dir(module) to load test cases. Run unittest file by
_PyRun_SimpleFileObject, have bug
___
Python tracker
<https://bugs.python.org/issue43630>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43629] fix _PyRun_SimpleFileObject create __main__ module and cache. Call this function multiple times, the attributes stored in the module dict will affect eachother.
Change by junyixie : -- resolution: -> wont fix ___ Python tracker <https://bugs.python.org/issue43629> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43629] fix _PyRun_SimpleFileObject create __main__ module and cache. Call this function multiple times, the attributes stored in the module dict will affect eachother.
Change by junyixie : -- keywords: +patch pull_requests: +23773 stage: -> patch review pull_request: https://github.com/python/cpython/pull/25023 ___ Python tracker <https://bugs.python.org/issue43629> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43629] fix _PyRun_SimpleFileObject create __main__ module and cache. Call this function multiple times, the attributes stored in the module dict will affect eachother.
New submission from junyixie :
fix _PyRun_SimpleFileObject create __main__ module and cache. Call this
function multiple times, the attributes stored in the module dict will affect
eachother.
create __main__ module, and cache it.
for example.
if we run fileA, call _PyRun_SimpleFileObject will create __main__ module,
fileA add some attribute in __main__ module dict.
now we run fileB. call _PyRun_SimpleFileObject will load cached __main__
module. now in __main__ module dict, we can get fileA's attribute.
dir(module), We got unexpected results
```
for name in dir(module):
...
```
in unittest, if we execute test, and don't exit. (unittest main.py
TestProgram), set exit=False.
```
def __init__(self, module='__main__', defaultTest=None, argv=None,
testRunner=None, testLoader=loader.defaultTestLoader,
exit=True, verbosity=1, failfast=None, catchbreak=None,
buffer=None, warnings=None, *, tb_locals=False):
```
then when unittest load tests. if we use _PyRun_SimpleFileObject to run
unittest, it will Repeated load test cases
```
for name in dir(module):
obj = getattr(module, name)
if isinstance(obj, type) and issubclass(obj, case.TestCase):
tests.append(self.loadTestsFromTestCase(obj))
```
```
int
_PyRun_SimpleFileObject(FILE *fp, PyObject *filename, int closeit,
PyCompilerFlags *flags)
{
PyObject *m, *d, *v;
int set_file_name = 0, ret = -1;
m = PyImport_AddModule("__main__");
if (m == NULL)
return -1;
Py_INCREF(m);
d = PyModule_GetDict(m);
```
--
components: C API
messages: 389538
nosy: JunyiXie
priority: normal
severity: normal
status: open
title: fix _PyRun_SimpleFileObject create __main__ module and cache. Call this
function multiple times, the attributes stored in the module dict will affect
eachother.
type: behavior
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue43629>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43601] Tools/c-analyzer/check-c-globals.py run throw exception err
junyixie added the comment:
found in must-reslove.sh
function run_capi() {
./python Tools/c-analyzer/c-analyzer.py capi \
--no-progress \
--group-by kind \
--func --inline --macro \
--no-show-empty \
--ignore '' \
$@
}
--
resolution: -> not a bug
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/issue43601>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43601] Tools/c-analyzer/check-c-globals.py run throw exception err
New submission from junyixie : how to use Tools/c-analyzer/check-c-globals.py? in readme, python3 Tools/c-analyzer/check-c-globals.py /Users/xiejunyi/cpython/Tools/c-analyzer/c_common/tables.py:236: FutureWarning: Possible nested set at position 12 _COLSPEC_RE = re.compile(textwrap.dedent(r''' Traceback (most recent call last): File "Tools/c-analyzer/check-c-globals.py", line 33, in (cmd, cmd_kwargs, verbosity, traceback_cm) = parse_args() File "Tools/c-analyzer/check-c-globals.py", line 16, in parse_args _cli_check(parser, checks=''), File "/Users/xiejunyi/cpython/Tools/c-analyzer/cpython/__main__.py", line 119, in _cli_check return c_analyzer._cli_check(parser, CHECKS, **kwargs, **FILES_KWARGS) TypeError: _cli_check() got multiple values for argument 'checks' -- components: Demos and Tools messages: 389370 nosy: JunyiXie priority: normal severity: normal status: open title: Tools/c-analyzer/check-c-globals.py run throw exception err type: crash versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43601> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
Change by junyixie : -- pull_requests: +23739 pull_request: https://github.com/python/cpython/pull/24972 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
Change by junyixie : -- pull_requests: +23738 pull_request: https://github.com/python/cpython/pull/24971 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
junyixie added the comment: I will fix the static variable problem of frequently used code -- ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43558] The dataclasses documentation should mention how to call super().__init__
Change by junyixie : -- pull_requests: +23731 pull_request: https://github.com/python/cpython/pull/24972 ___ Python tracker <https://bugs.python.org/issue43558> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43558] The dataclasses documentation should mention how to call super().__init__
Change by junyixie : -- keywords: +patch nosy: +JunyiXie nosy_count: 1.0 -> 2.0 pull_requests: +23730 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/24971 ___ Python tracker <https://bugs.python.org/issue43558> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
Change by junyixie : -- pull_requests: +23729 pull_request: https://github.com/python/cpython/pull/24970 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
Change by junyixie : -- pull_requests: +23728 pull_request: https://github.com/python/cpython/pull/24969 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
junyixie added the comment: Okay, I will fix these problems first. (Need for your own project) -- ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
junyixie added the comment: -> Okay, I will fix these problems first. (Need for my own project) -- ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
Change by junyixie : -- pull_requests: +23727 pull_request: https://github.com/python/cpython/pull/24968 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
Change by junyixie : -- pull_requests: +23726 pull_request: https://github.com/python/cpython/pull/24966 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
junyixie added the comment: fix ./Objects/exceptions.c:static PyObject *print_prefix = NULL; ./Objects/exceptions.c:static PyObject *exec_prefix = NULL; https://github.com/python/cpython/pull/24965 -- ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
Change by junyixie : -- pull_requests: +23724 pull_request: https://github.com/python/cpython/pull/24965 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
junyixie added the comment: grep -E 'static (\w+) \*(\w+) =' $(find . -name "*.c") | wc -l 67 static variable may need fix ➜ cpython git:(master) grep -E 'static (\w+) \*(\w+) =' $(find . -name "*.c") ./PC/winreg.c:static char *failMsg = "bad operand type"; ./Python/codecs.c:static _PyUnicode_Name_CAPI *ucnhash_capi = NULL; ./Python/initconfig.c:static char *_Py_StandardStreamEncoding = NULL; ./Python/initconfig.c:static char *_Py_StandardStreamErrors = NULL; ./Python/import.c:static PyObject *extensions = NULL; ./Python/import.c:static PyObject *builtins_str = NULL; ./Python/import.c:static PyObject *import_str = NULL; ./Python/sysmodule.c:static PyObject *newline = NULL; ./Python/_warnings.c:static PyObject *importlib_string = NULL; ./Python/_warnings.c:static PyObject *bootstrap_string = NULL; ./Objects/unicodeobject.c:static _PyUnicode_Name_CAPI *ucnhash_capi = NULL; ./Objects/codeobject.c:static PyObject *emptystring = NULL; ./Objects/codeobject.c:static PyObject *nulltuple = NULL; ./Objects/listobject.c:static PyObject *indexerr = NULL; ./Objects/exceptions.c:static PyObject *print_prefix = NULL; ./Objects/exceptions.c:static PyObject *exec_prefix = NULL; ./Objects/boolobject.c:static PyObject *false_str = NULL; ./Objects/boolobject.c:static PyObject *true_str = NULL; ./Parser/pegen.c:static void *cache = NULL; ./Modules/_cursesmodule.c:static char *screen_encoding = NULL; ./Modules/arraymodule.c:static PyObject *array_reconstructor = NULL; ./Modules/_tkinter.c:static PyObject *tcl_library_path = NULL; ./Modules/_tkinter.c:static PyThreadState *tcl_tstate = NULL; ./Modules/_tkinter.c:static PyThreadState *event_tstate = NULL; ./Modules/syslogmodule.c:static PyObject *S_ident_o = NULL; /* identifier, held by openlog() */ ./Modules/_tracemalloc.c:static PyObject *unknown_filename = NULL; ./Modules/_tracemalloc.c:static _Py_hashtable_t *tracemalloc_filenames = NULL; ./Modules/_tracemalloc.c:static traceback_t *tracemalloc_traceback = NULL; ./Modules/_tracemalloc.c:static _Py_hashtable_t *tracemalloc_tracebacks = NULL; ./Modules/_tracemalloc.c:static _Py_hashtable_t *tracemalloc_traces = NULL; ./Modules/_tracemalloc.c:static _Py_hashtable_t *tracemalloc_domains = NULL; ./Modules/_ssl.c:static PyObject *x509_asn = NULL; ./Modules/_ssl.c:static PyObject *pkcs_7_asn = NULL; ./Modules/_ssl.c:static PyThread_type_lock *_ssl_locks = NULL; ./Modules/_testbuffer.c:static PyObject *structmodule = NULL; ./Modules/_testbuffer.c:static PyObject *Struct = NULL; ./Modules/_testbuffer.c:static PyObject *calcsize = NULL; ./Modules/_testbuffer.c:static PyObject *simple_format = NULL; ./Modules/_testbuffer.c:static char *infobuf = NULL; ./Modules/_io/bufferedio.c:static PyObject *eintr_int = NULL; ./Modules/_asynciomodule.c:static futureiterobject *fi_freelist = NULL; ./Modules/_zoneinfo.c:static PyObject *io_open = NULL; ./Modules/_zoneinfo.c:static PyObject *_tzpath_find_tzfile = NULL; ./Modules/_zoneinfo.c:static PyObject *_common_mod = NULL; ./Modules/_zoneinfo.c:static PyObject *TIMEDELTA_CACHE = NULL; ./Modules/_zoneinfo.c:static PyObject *ZONEINFO_WEAK_CACHE = NULL; ./Modules/_zoneinfo.c:static StrongCacheNode *ZONEINFO_STRONG_CACHE = NULL; ./Modules/_sqlite/microprotocols.c:static PyObject *psyco_adapters = NULL; ./Modules/_ssl/debughelpers.c:static PyThread_type_lock *lock = NULL; ./Modules/_datetimemodule.c:static PyObject *us_per_ms = NULL; /* 1000 */ ./Modules/_datetimemodule.c:static PyObject *us_per_second = NULL; /* 100 */ ./Modules/_datetimemodule.c:static PyObject *us_per_minute = NULL; /* 1e6 * 60 as Python int */ ./Modules/_datetimemodule.c:static PyObject *us_per_hour = NULL;/* 1e6 * 3600 as Python int */ ./Modules/_datetimemodule.c:static PyObject *us_per_day = NULL; /* 1e6 * 3600 * 24 as Python int */ ./Modules/_datetimemodule.c:static PyObject *us_per_week = NULL;/* 1e6*3600*24*7 as Python int */ ./Modules/_datetimemodule.c:static PyObject *seconds_per_day = NULL; /* 3600*24 as Python int */ ./Modules/_datetimemodule.c:static PyObject *module = NULL; ./Modules/_decimal/_decimal.c:static PyObject *tls_context_key = NULL; ./Modules/_decimal/_decimal.c:static PyDecContextObject *cached_context = NULL; ./Modules/_decimal/_decimal.c:static PyObject *current_context_var = NULL; ./Modules/_decimal/_decimal.c:static PyObject *default_context_template = NULL; ./Modules/_decimal/_decimal.c:static PyObject *basic_context_template = NULL; ./Modules/_decimal/_decimal.c:static PyObject *extended_context_template = NULL; ./Modules/_decimal/_decimal.c:static PyObject *DecimalException = NULL; ./Modules/_decimal/_decimal.c:static PyObject *Rational = NULL; ./Modules/_decimal/_decimal.c:static PyTypeObject *DecimalTuple = NULL; ./Modules/_decimal/_decimal.c:static PyObject *capsule = NULL; -- ___ Python tra
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
junyixie added the comment: 43551 [Subinterpreters]: PyImport_Import use static silly_list under building Python with --with-experimental-isolated-subinterpreters share silly_list in multi subinterpreters cause crash. https://bugs.python.org/issue43551 -- message_count: 1.0 -> 2.0 pull_requests: +23722 pull_request: https://github.com/python/cpython/pull/24929 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
Change by junyixie : -- keywords: +patch pull_requests: +23720 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24963 ___ Python tracker <https://bugs.python.org/issue43588> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43588] [Subinterpreters]: use static variable under building Python with --with-experimental-isolated-subinterpreters cause crash.
New submission from junyixie :
use static module variable under building Python with
--with-experimental-isolated-subinterpreters cause crash.
compiler_mod(struct compiler *c, mod_ty mod)
{
PyCodeObject *co;
int addNone = 1;
static PyObject *module;
if (!module) {
module = PyUnicode_InternFromString("");
if (!module)
return NULL;
}
...
}
--
components: Subinterpreters
messages: 389282
nosy: JunyiXie, vstinner
priority: normal
severity: normal
status: open
title: [Subinterpreters]: use static variable under building Python with
--with-experimental-isolated-subinterpreters cause crash.
type: crash
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue43588>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43551] [Subinterpreters]: PyImport_Import use static silly_list under building Python with --with-experimental-isolated-subinterpreters share silly_list in multi subinterpreters cause crash.
Change by junyixie : -- keywords: +patch pull_requests: +23691 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24929 ___ Python tracker <https://bugs.python.org/issue43551> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43551] [Subinterpreters]: PyImport_Import use static silly_list under building Python with --with-experimental-isolated-subinterpreters share silly_list in multi subinterpreters cause crash.
New submission from junyixie :
fix PyImport_Import use static silly_list under building Python with
--with-experimental-isolated-subinterpreters share silly_list in multi
subinterpreters cause crash.
Under the sub interpreters parallel, PyObject_CallFunction clean stack,
Py_DECREF(stack[i]), Py_DECREF silly_list is not thread safe. cause crash
```
PyObject *
PyImport_Import(PyObject *module_name)
{
PyThreadState *tstate = _PyThreadState_GET();
static PyObject *silly_list = NULL;
...
/* Initialize constant string objects */
if (silly_list == NULL) {
import_str = PyUnicode_InternFromString("__import__");
if (import_str == NULL)
return NULL;
builtins_str = PyUnicode_InternFromString("__builtins__");
if (builtins_str == NULL)
return NULL;
silly_list = PyList_New(0);
if (silly_list == NULL)
return NULL;
}
...
/* Call the __import__ function with the proper argument list
Always use absolute import here.
Calling for side-effect of import. */
r = PyObject_CallFunction(import, "i", module_name, globals,
globals, silly_list, 0, NULL);
```
--
messages: 389056
nosy: JunyiXie, vstinner
priority: normal
severity: normal
status: open
title: [Subinterpreters]: PyImport_Import use static silly_list under building
Python with --with-experimental-isolated-subinterpreters share silly_list in
multi subinterpreters cause crash.
___
Python tracker
<https://bugs.python.org/issue43551>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
junyixie added the comment: > 2. when free memory address, find this address in all interpreter pymalloc > pool. and free it.(but it need add lock to pymalloc) when finalize_interp_delete, we need keep interpreter pymalloc pool in linked list.It will be used when search memory in pymalloc pools. -- ___ Python tracker <https://bugs.python.org/issue43313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
junyixie added the comment: by the way, There is no operation to destroy the memory pool in the cpython code. Repeated creation of the pymalloc pool will cause memory leaks. -- ___ Python tracker <https://bugs.python.org/issue43313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
junyixie added the comment: > malloc pointer in interpreterA and free pointer is usual. malloc pointer in interpreterA and free pointer in interpreterB is usual. -- ___ Python tracker <https://bugs.python.org/issue43313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
junyixie added the comment: There is a problem: if we bound pymalloc state with a interpreter. malloc pointer in interpreterA and free pointer is usual. it's cause a problem. when we use PyObject_Free, 1. we look up address in pymalloc pool. 2. if not find, current code will call PyMem_RawFree(p) to free. it will cause crash.(address is pymalloc_alloc from another interpreter) I think it has two way to slove this problem: 1. free/alloc memory in one interpreter. Frequent switch interpreter affects performance 2. when free memory address, find this address in all interpreter pymalloc pool. and free it.(but it need add lock to pymalloc) -- ___ Python tracker <https://bugs.python.org/issue43313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
junyixie added the comment: https://github.com/python/cpython/pull/24857 -- ___ Python tracker <https://bugs.python.org/issue43313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
junyixie added the comment: github pr -- keywords: +patch message_count: 2.0 -> 3.0 pull_requests: +23617 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24857 ___ Python tracker <https://bugs.python.org/issue43313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
junyixie added the comment:
Made two changes:
1. support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
2. _copy_raw_string api alloc memory use PyMem_RawFree and PyMem_RawMalloc.
I extend _xxsubinterpretermodule.c to support call any function in sub
interpreter.
when i need return result from sub interpreter call.
1. i need create item->name in shared item. will use pymem_xxx api to manage
memory. when with_pymalloc macro defined, it will create memory and bound to
interpreter(iterp1) pymalloc state.
2. after switch interpreter state, now in iterp2 state, get return value from
shareditem, and i need free shared item. but item->name memory managed by
interp1 pymalloc state. if i want to free them, i need switch to interpreter
state 1. it's complicated. to implementation it, we need save interpid in
shared item.
so i think, in _sharednsitem_init _copy_raw_string, need malloc by
PyMem_RawAPI. easy to management.
```
static int
_sharednsitem_init(struct _sharednsitem *item, PyObject *key, PyObject *value)
{
item->name = _copy_raw_string(key);
```
```
_sharedns *result_shread = _sharedns_new(1);
#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS
// Switch to interpreter.
PyThreadState *new_tstate = PyInterpreterState_ThreadHead(interp);
PyThreadState *save1 = PyEval_SaveThread();
(void)PyThreadState_Swap(new_tstate);
#else
// Switch to interpreter.
PyThreadState *save_tstate = NULL;
if (interp != PyInterpreterState_Get()) {
// XXX Using the "head" thread isn't strictly correct.
PyThreadState *tstate = PyInterpreterState_ThreadHead(interp);
// XXX Possible GILState issues?
save_tstate = PyThreadState_Swap(tstate);
}
#endif
PyObject *module = PyImport_ImportModule(PyUnicode_AsUTF8(module_name));
PyObject *function = PyObject_GetAttr(module, function_name);
result = PyObject_Call(function, args, kwargs);
if (result == NULL) {
// exception handler
...
}
if (result && _sharednsitem_init(_shread->items[0],
PyUnicode_FromString("result"), result) != 0) {
PyErr_Format(RunFailedError, "interp_call_function result convert to
shared failed");
return NULL;;
}
#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS
// Switch back.
PyEval_RestoreThread(save1);
#else
// Switch back.
if (save_tstate != NULL) {
PyThreadState_Swap(save_tstate);
}
#endif
// ...
if (result) {
result =
_PyCrossInterpreterData_NewObject(_shread->items[0].data);
_sharedns_free(result_shread);
}
```
--
nosy: +vstinner
___
Python tracker
<https://bugs.python.org/issue43313>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23835] configparser does not convert defaults to strings
Change by junyixie : -- nosy: +JunyiXie nosy_count: 7.0 -> 8.0 pull_requests: +23600 pull_request: https://github.com/python/cpython/pull/24821 ___ Python tracker <https://bugs.python.org/issue23835> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40601] [C API] Hide static types from the limited C API
junyixie added the comment: It seems that there is no continued progress for move static type in heap.This will make it impossible to continue to achieve sub interpreters parallel. Are there any plans to try other solutions to the problem? In my project, i try to slove this problem, It can work, we verify on millions of devices. 1. In typeobject.c add lock to ensure that some functions that modification type are thread-safe 2. and make the PyCFunction and descri object of the Type will never be released. （Frequently used when load method/attributed, locking affects performance） Can this change be submitted to cpython？ -- nosy: +JunyiXie ___ Python tracker <https://bugs.python.org/issue40601> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43441] [Subinterpreters]: global variable next_version_tag cause method cache bug
junyixie added the comment: under building Python with --with-experimental-isolated-subinterpreters -- ___ Python tracker <https://bugs.python.org/issue43441> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43441] [Subinterpreters]: global variable next_version_tag cause method cache bug
junyixie added the comment: This is a simple fix. https://github.com/python/cpython/pull/24822/commits/e61ce1dd28a48534ee497aaacb4439193bedfd42 -- ___ Python tracker <https://bugs.python.org/issue43441> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43441] [Subinterpreters]: global variable next_version_tag cause method cache bug
Change by junyixie : -- keywords: +patch pull_requests: +23588 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24822 ___ Python tracker <https://bugs.python.org/issue43441> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43441] [Subinterpreters]: global variable next_version_tag cause method cache bug
Change by junyixie : -- components: +Subinterpreters -Interpreter Core title: mutilcorevm: global variable next_version_tag cause method cache bug -> [Subinterpreters]: global variable next_version_tag cause method cache bug ___ Python tracker <https://bugs.python.org/issue43441> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter
junyixie added the comment: https://github.com/python/cpython/pull/24821/commits/9d7681dbd273b5025fd9b19d1be0a1f978a0b12e -- ___ Python tracker <https://bugs.python.org/issue40521> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter
junyixie added the comment:
Should Make dtoa bigint free list per-interpreter.
static Bigint *bigint_freelist[Kmax+1]; -> _is { Bigint
*bigint_freelist[Kmax+1]; }
--
message_count: 43.0 -> 44.0
nosy: +JunyiXie
nosy_count: 5.0 -> 6.0
pull_requests: +23587
pull_request: https://github.com/python/cpython/pull/24821
___
Python tracker
<https://bugs.python.org/issue40521>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43311] bpo-43311: PyInterpreterState_New use thread-specific data tstate before key create .
junyixie added the comment: Yes, this is an issue under building Python with --with-experimental-isolated-subinterpreters -- ___ Python tracker <https://bugs.python.org/issue43311> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39511] [subinterpreters] Per-interpreter singletons (None, True, False, etc.)
junyixie added the comment:
> Which API should be used in C extensions to be "subinterpreter-safe"? ?>
> Currently, Py_None is a singleton shared by multiple interpreters. > > >
> Should suddenly all C extensions use a new Py_GetNone() function which >
> returns the per-interpreter singleton? If yes, that's basically what my > PR
> 18301 does:
> #define Py_None Py_GetNone()
after read you [WIP] bpo-39511: Add Py_GetNone() and Py_GetNoneRef() functions
#18301.
Actually, interp->none shared _Py_NoneStruct variable.
when two interperter modify interp->none refcount,will modify _Py_NoneStruct
variable.
> the CPU cacheline of common singletons like None, True and False can quickly
> become a performance bottleneck.
even if add Py_INCREF(none);.
In the scenario of parallel interpreter, will also have thread safety issues.
> PyStatus
> _Py_InitSingletons(PyThreadState *tstate)
> {
> PyObject *none = &_Py_NoneStruct;
> Py_INCREF(none);
> tstate->interp->none = none;
> return _PyStatus_OK();
> }
--
nosy: +JunyiXie -Mark.Shannon, corona10, eric.snow, jeremy.kloth, jkloth,
larry, maciej.szulik, nanjekyejoannah, ncoghlan, phsilva, rhettinger,
shihai1991, steve.dower
___
Python tracker
<https://bugs.python.org/issue39511>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43441] mutilcorevm: global variable next_version_tag cause method cache bug
Change by junyixie : -- type: -> crash ___ Python tracker <https://bugs.python.org/issue43441> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43442] multicorevm: guarantee type multi sub interpreters safe
New submission from junyixie : in multi core cpython project. when use multi sub interpreters, Type is not safe. Type shared by interpreters. but isolate type may cause python abi/api change. python 4.0? temporary solution: 1. add a type lock to guarantee type object safe in multi subinterpreters. 2. some thing like pycmethod object and descr in pytype, set their refcount to INT MAX.It is guaranteed that these objects will not be released. and not cause memory leaks, only one type exist in memory. -- messages: 388333 nosy: JunyiXie priority: normal severity: normal status: open title: multicorevm: guarantee type multi sub interpreters safe ___ Python tracker <https://bugs.python.org/issue43442> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43441] mutilcorevm: global variable next_version_tag cause method cache bug
junyixie added the comment:
fix:
only main interpreter fini, clear method cache.
void
_PyType_Fini(PyInterpreterState *interp)
{
if (_Py_IsMainInterpreter(interp)) {
clear_slotdefs();
_PyType_ClearCache(>type_cache);
}
}
when python4.0? type isolate, each interpreter dealloc should clear method
cache.
--
___
Python tracker
<https://bugs.python.org/issue43441>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43441] mutilcorevm: global variable next_version_tag cause method cache bug
junyixie added the comment:
when sub interpreter finalize.
_PyType_ClearCache set next_version_tag = 0.
Type shared between interpreters.
another interpreter assign_version_tag "1" for a type, the type is first assign.
the dealloc interpreter had assign_version_tag "1" for another type.
now, two different type has same version tag. it cause method cache wrong.
static unsigned int
_PyType_ClearCache(struct type_cache *cache)
{
#if MCACHE_STATS
size_t total = cache->hits + cache->collisions + cache->misses;
fprintf(stderr, "-- Method cache hits= %zd (%d%%)\n",
cache->hits, (int) (100.0 * cache->hits / total));
fprintf(stderr, "-- Method cache true misses = %zd (%d%%)\n",
cache->misses, (int) (100.0 * cache->misses / total));
fprintf(stderr, "-- Method cache collisions = %zd (%d%%)\n",
cache->collisions, (int) (100.0 * cache->collisions / total));
fprintf(stderr, "-- Method cache size= %zd KiB\n",
sizeof(cache->hashtable) / 1024);
#endif
unsigned int cur_version_tag = next_version_tag - 1;
next_version_tag = 0;
type_cache_clear(cache, 0);
return cur_version_tag;
}
--
___
Python tracker
<https://bugs.python.org/issue43441>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43441] mutilcorevm: global variable next_version_tag cause method cache bug
New submission from junyixie :
type->tp_version_tag = next_version_tag++;
when sub interpreters parallel, next_version_tag++ is thread-unsafe. may cause
different type has same tp_version_tag.
cause method cache bug in _PyType_Lookup
#define MCACHE_HASH_METHOD(type, name) \
MCACHE_HASH((type)->tp_version_tag, \
((PyASCIIObject *)(name))->hash)
if (MCACHE_CACHEABLE_NAME(name) &&
_PyType_HasFeature(type, Py_TPFLAGS_VALID_VERSION_TAG)) {
/* fast path */
unsigned int h = MCACHE_HASH_METHOD(type, name);
struct type_cache *cache = get_type_cache();
struct type_cache_entry *entry = >hashtable[h];
if (entry->version == type->tp_version_tag && entry->name == name) {
#if MCACHE_STATS
cache->hits++;
#endif
return entry->value;
}
}
static int
assign_version_tag(struct type_cache *cache, PyTypeObject *type)
{
...
type->tp_version_tag = next_version_tag++;
...
}
--
components: Interpreter Core
messages: 388327
nosy: JunyiXie
priority: normal
severity: normal
status: open
title: mutilcorevm: global variable next_version_tag cause method cache bug
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue43441>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43311] bpo-43311: PyInterpreterState_New use thread-specific data tstate before key create .
junyixie added the comment: PyInterpreterState_New call and use PyThreadState *tstate = _PyThreadState_GET(); _PyRuntime.gilstate.autoTSSkey has to be initialized before pthread_getspecific() or pthread_setspecific() can be used. _PyRuntime.gilstate.autoTSSkey create in _PyGILState_Init. PyInterpreterState_New called before _PyGILState_Init. -- ___ Python tracker <https://bugs.python.org/issue43311> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43311] bpo-43311: PyInterpreterState_New use thread-specific data tstate before key create .
Change by junyixie : -- title: PyInterpreterState_New use thread tstate before set. -> bpo-43311: PyInterpreterState_New use thread-specific data tstate before key create . ___ Python tracker <https://bugs.python.org/issue43311> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
New submission from junyixie : https://github.com/ericsnowcurrently/multi-core-python/issues/73 https://github.com/JunyiXie/cpython/commit/820954879fd546fcb29b654d10c424bd47da70ce changes: move pymalloc state in obmalloc.h _is add pymalloc_state pymalloc_allocxx api use subinterpreter pymalloc_state -- components: Subinterpreters messages: 387614 nosy: JunyiXie priority: normal severity: normal status: open title: feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state ___ Python tracker <https://bugs.python.org/issue43313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43313] feature: support pymalloc for subinterpreters. each subinterpreter has pymalloc_state
Change by junyixie : -- versions: +Python 3.10 ___ Python tracker <https://bugs.python.org/issue43313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43311] PyInterpreterState_New use thread tstate before set.
Change by junyixie : -- keywords: +patch pull_requests: +23420 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24636 ___ Python tracker <https://bugs.python.org/issue43311> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43311] PyInterpreterState_New use thread tstate before set.
New submission from junyixie :
PyInterpreterState_New use thread tstate before set.
PyInterpreterState_New use tstate. but tstate is not set.
tstate will set after PyInterpreterState_New.
PyInterpreterState *interp = PyInterpreterState_New();
if (interp == NULL) {
*tstate_p = NULL;
return _PyStatus_OK();
}
PyThreadState *tstate = PyThreadState_New(interp);
if (tstate == NULL) {
PyInterpreterState_Delete(interp);
*tstate_p = NULL;
return _PyStatus_OK();
}
PyThreadState *save_tstate = PyThreadState_Swap(tstate);
--
components: Subinterpreters
messages: 387610
nosy: JunyiXie
priority: normal
severity: normal
status: open
title: PyInterpreterState_New use thread tstate before set.
type: crash
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue43311>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
