[issue32604] Expose the subinterpreters C-API in Python for testing use.

STINNER Victor added the comment: The revert fixed the test: $ ./python -m test -R 3:3 test__xxsubinterpreters (...) Tests result: SUCCESS -- ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

STINNER Victor added the comment: New changeset f2c3b6823bc4777d4a14eb0c3615b719521f763a by Victor Stinner in branch 'master': Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)" (GH-20089)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

STINNER Victor added the comment: The buildbots are broken for one whole week and nobody is available to investigate the reference leak regression. Following the CI policy, I wrote PR 20089 to revert the change which broke the CI: https://pythondev.readthedocs.io/ci.html#revert-on-fail If

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by STINNER Victor : -- pull_requests: +19393 pull_request: https://github.com/python/cpython/pull/20089 ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

STINNER Victor added the comment: The commit a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57 introduced reference leaks: $ ./python -m test -R 3:3 test__xxsubinterpreters -m test.test__xxsubinterpreters.RunFailedTests.test_traceback_propagated (...) test__xxsubinterpreters leaked [1863, 1861, 1863]

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57 by Eric Snow in branch 'master': bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768) https://github.com/python/cpython/commit/a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57 --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +19149 pull_request: https://github.com/python/cpython/pull/19829 ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

miss-islington added the comment: New changeset 5e8c691594d68925213d36296ce7c4b3e90bcb1d by Eric Snow in branch 'master': bpo-32604: Add support for a "default" arg in channel_recv(). (GH-19770) https://github.com/python/cpython/commit/5e8c691594d68925213d36296ce7c4b3e90bcb1d --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +19090 pull_request: https://github.com/python/cpython/pull/19770 ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +19088 pull_request: https://github.com/python/cpython/pull/19768 ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Jamie Stribling : -- components: +Documentation -Interpreter Core ___ Python tracker ___ ___ Python-bugs-list mailing

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: Also see #24553. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 63799136e6c0491bb5d6f4a234d5a775db3458db by Eric Snow in branch 'master': bpo-33615: Re-enable a subinterpreter test. (gh-7251) https://github.com/python/cpython/commit/63799136e6c0491bb5d6f4a234d5a775db3458db --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +6960 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 110bc01407ac8c75545d0386577c6e17254d97d9 by Eric Snow in branch 'master': bpo-33615: Temporarily disable a test that is triggering crashes on a few buildbots. (gh-7288)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +6915 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +6877 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Emily Morehouse : -- nosy: +emilyemorehouse ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Yury Selivanov : -- nosy: -yselivanov ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 3ab0136ac5d6059ce96d4debca89c5f5ab0356f5 by Eric Snow in branch 'master': bpo-32604: Implement force-closing channels. (gh-6937) https://github.com/python/cpython/commit/3ab0136ac5d6059ce96d4debca89c5f5ab0356f5

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +6607 ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Alexander Belopolsky : -- nosy: +belopolsky ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 6d2cd9036c0ab78a83de43d1511befb7a7fc0ade by Eric Snow in branch 'master': bpo-32604: Improve subinterpreter tests. (#6914) https://github.com/python/cpython/commit/6d2cd9036c0ab78a83de43d1511befb7a7fc0ade --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +6582 ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Ned Deily added the comment: As agreed upon at the PyCon dev sprints, 6bd0c476c58ca0046969f70dc2a4e4dfb3268062 removes _xxsubinterpreters from the 3.7 branch and 3.7.0 and retargets for 3.8. Since it was intended for internal use, no user documentation or APIs should be

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 6bd0c476c58ca0046969f70dc2a4e4dfb3268062 by Eric Snow in branch '3.7': bpo-32604: Remove xid registry. (#6813) https://github.com/python/cpython/commit/6bd0c476c58ca0046969f70dc2a4e4dfb3268062 --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +6499 stage: needs patch -> patch review ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: Sounds good, Ned. Sorry for my confusion. -- ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Ned Deily : -- priority: normal -> deferred blocker ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Ned Deily added the comment: > Still planned for 3.7: [...] Eric, please try to get everything you want done and into master as soon as possible, then produce a PR for 3.7, and then we can review it and make a decision if it is appropriate for 3.7. As I've mentioned in

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: Still planned for 3.7: 1. add more tests in Lib/test/test__xxsubinterpreters.py 2. add test.support.interpreters (high-level module) 3. add more tests in Lib/test/test_interpreters.py At that point I'll start using the high-level

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: As far as I know, everything related to this issue is in a stable state. I have not been able to determine any way in which the reported memory leak (in test_multiprocessing_fork) could be related. Sorry, I didn't notice the release

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Nick Coghlan added the comment: _xxsubinterpreters has been added, and we'll use PEP 554 as the interim documentation while it's only exposed for testing purposes. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Ned Deily added the comment: What's the status of this issue? 3.7.0b2 is tagging in 48 hours or so. -- versions: +Python 3.8 ___ Python tracker

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset eed3c7af4ec520d324a55e0357f41589a66906f7 by Eric Snow (Miss Islington (bot)) in branch '3.7': bpo-32604: Swap threads only if the interpreter is different. (gh-5783)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset f53d9f2778a87bdd48eb9030f782a4ebf9e7622f by Eric Snow in branch 'master': bpo-32604: Swap threads only if the interpreter is different. (gh-5778)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by miss-islington : -- pull_requests: +5564 ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +5559 ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by pmpp : -- nosy: +pmpp ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Zachary Ware added the comment: I'm pretty sure that report is still running on the last changeset on hg.python.org; can you confirm that Antoine? -- nosy: +pitrou ___ Python tracker

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Yury Selivanov added the comment: FYI I found out about this refleak from https://mail.python.org/pipermail/python-checkins/2018-February/153907.html So it's definitely not Mac OS X specific thing. -- ___ Python tracker

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Zachary Ware added the comment: Looks like macOS rather than Windows, and I can't reproduce it locally with current master: 14:28 $ ./python.exe -m test -R3:3 test_multiprocessing_fork Run tests sequentially 0:00:00 load avg: 2.68 [1/1] test_multiprocessing_fork

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: Davin, any thoughts on how my most recent commit (for this issue) might be causing the leaks Yury found? -- nosy: +davin ___ Python tracker

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: I'm not seeing any refleak (on linux/clang). I'm guessing this is Windows-specific (based on use of "./python.exe"). How does test_multiprocessing_fork even run on Windows? I thought "fork" is an unsupported start method on Windows

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: @Yury, thanks! I thought I had checked before I made the PR, but apparently not. I'll git it fixed. -- ___ Python tracker

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Yury Selivanov added the comment: Eric, it looks like your recent commit introduced a refleak. We need to fix it before beta2. ~/d/p/cpython (master $) » ./python.exe -m test -R3:3 test_multiprocessing_fork Run tests sequentially 0:00:00 load avg: 2.52 [1/1]

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 3db05a3a9c109f49d54b4965852e41c657e9b117 by Eric Snow (Miss Islington (bot)) in branch '3.7': bpo-32604: Clean up created subinterpreters before runtime finalization. (gh-5710)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by miss-islington : -- pull_requests: +5499 ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 4c6955e2b0ccf88c705f8d1fac685a8e65f9699e by Eric Snow in branch 'master': bpo-32604: Clean up created subinterpreters before runtime finalization. (gh-5709)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +5498 ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Zachary Ware added the comment: New changeset fe61e8d8c7cd1f6fb0ce7e9b8f0460b47b5f29a5 by Zachary Ware (Miss Islington (bot)) in branch '3.7': bpo-32604: Make _xxsubinterpreters build on Windows (GH-5624)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by miss-islington : -- pull_requests: +5432 ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Zachary Ware added the comment: New changeset 310b05289b5d9550040f469e60b5e8e77f1022b6 by Zachary Ware in branch 'master': bpo-32604: Make _xxsubinterpreters build on Windows (GH-5516) https://github.com/python/cpython/commit/310b05289b5d9550040f469e60b5e8e77f1022b6

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Zachary Ware : -- pull_requests: +5348 ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: FYI, out of 2389 source lines in the C extension, 1563 are the channel-related code. That means the non-channel code is 826 lines (about a third). That non-channel code does not depend on the channel code at all and I considered

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset f33ecedcad5a001735fa4ded5d21caa2cbf27911 by Eric Snow (Miss Islington (bot)) in branch '3.7': bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (GH-5507)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by miss-islington : -- pull_requests: +5340 ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: I've landed a PR that fixes all the memory leaks in the module. It also fixes the 2 defects reported by coverity. The only thing left here is to get the module building under Windows. --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 4e9da0d163731caa79811c723c703ee416c31826 by Eric Snow in branch 'master': bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (#5507)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +5338 ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Yury Selivanov added the comment: A couple defects reported by coverity: ** CID 1428758: Integer handling issues (CONSTANT_EXPRESSION_RESULT) /Modules/_xxsubinterpretersmodule.c: 45 in _coerce_id()

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: The buildbots should be happier now. I'll keep an eye on them. -- ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 83e64c8a544028ae677af2a0bc268dbe1c11cc3a by Eric Snow in branch 'master': bpo-32604: NULL-terminate kwlist in channel_drop_interpreter(). (gh-5437)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: I just put up a PR that should fix the 4 buildbots. -- ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +5270 ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: On the PPC64 AIX 3.x buildbot: == FAIL: test_repr (test.test__xxsubinterpreters.ChannelIDTests) --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: On 4 of the buildbots: == ERROR: test_drop_multiple_times (test.test__xxsubinterpreters.ChannelTests)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: Yeah, I'm looking into it. Also, I noticed some refleaks that I'll be sorting out. -- ___ Python tracker

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Ned Deily added the comment: Eric, looks like some buildbots are unhappy, for instance: http://buildbot.python.org/all/#builders/13/builds/648 -- ___ Python tracker

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: I've merged the patch without Windows support, which shouldn't be a problem given the purpose of the extension module. I've also added a PR for get the module building under Windows. I'd like to get that resolved ASAP. --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- pull_requests: +5269 ___ Python tracker ___ ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: New changeset 7f8bfc9b9a8381ddb768421b5dd5cbd970266190 by Eric Snow in branch 'master': bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. (gh-1748)

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: Sounds good, Ned. Thanks for taking a look. I should have everything finished up by Friday, so I'm hopeful for landing the change before the deadline. I may have a few minor tweaks to make after that, but I'll discuss that with you

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Ned Deily added the comment: @Eric, given the breadth of change introduced in the PR (including adding a new extension), I think it would be best if at all possible to get it in for beta 1 if we can resolve the review comments in time. If necessary and if there are no

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: FYI, there are a few things I need to clean up in the PR. However, I expect that those changes will be minor relative to the the whole patch, so I wanted to get the ball rolling on a review. :) --

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Change by Eric Snow : -- keywords: +patch pull_requests: +5094 stage: needs patch -> patch review ___ Python tracker ___

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: @Nick, I may make the name change you suggested in issue #30439 ("_subinterpreters"). -- ___ Python tracker

[issue32604] Expose the subinterpreters C-API in Python for testing use.

Eric Snow added the comment: @Ned, it may be a little tight to land this given the time left before beta 1. However, this is meant as a tool for us to use in the test suite (particularly to test the subinterpreter C-API). So I'm arguing that, if necessary, it

[issue32604] Expose the subinterpreters C-API in Python for testing use.

New submission from Eric Snow : In the interest of getting something landed for 3.7, so we can start using it in tests, I'm putting up a patch for a low-level interpreters module. In some ways this is a precursor for issue #30439, which will add a proper public