[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Martin Panter added the comment: FYI the change here to remember all the thread objects ever created looks to be the cause of the memory leak reported in Issue 37193 -- nosy: +martin.panter ___ Python tracker

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: The bug has been fixed in Python 3.7. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: I created bpo-33540: "socketserver: Add an opt-in option to get Python 3.6 behaviour on server_close()". I set the release blocker priority on this new issue. -- priority: deferred blocker ->

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: New changeset db8189bb8db609ca3993dec31fb95fdbe134469a by Victor Stinner in branch 'master': bpo-31233, bpo-31151: Document socketserver changes (#5417)

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Change by STINNER Victor : -- pull_requests: +5251 ___ Python tracker ___ ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Ned Deily added the comment: If we're going to leave this open until 3.7.0b2, can you at least add a brief warning in the "Porting section" for b1 that the issue is unresolved? -- ___ Python tracker

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: > What's the status of this? The status is bad. The commit b8f4163da30e16c7cd58fe04f4b17e38d53cd57e is backward incompatible, it's not documented in the "Porting the Python 3.7" section of What's New in Python 3.7, and it's not

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Ned Deily added the comment: What's the status of this? Feature freeze is here. -- ___ Python tracker ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: Klon666 added https://github.com/python/cpython/commit/96a6cbc5b0aee267f47e5efb50fba183b52cd8c6 repository. But this URL is unrelated. It looks like a bot spamming the bug tracker and I don't see how to remove the URL... --

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Change by Berker Peksag : -- components: -Build nosy: -Tu madre type: security -> resource usage ___ Python tracker ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Change by Klon666 : -- components: +Build nosy: +Tu madre type: resource usage -> security ___ Python tracker ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Change by Klon666 : -- hgrepos: +376 ___ Python tracker ___ ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: The current code leaks memory since it never clears threads which completed. We need a cleanup function similar to ForkingMixIn.collect_children() which is called by handle_timeout() and service_actions(). We can check if a thread

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Changes by Ned Deily : -- priority: release blocker -> deferred blocker ___ Python tracker ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: New changeset b8f4163da30e16c7cd58fe04f4b17e38d53cd57e by Victor Stinner in branch 'master': bpo-31233: socketserver.ThreadingMixIn.server_close() (#3523) https://github.com/python/cpython/commit/b8f4163da30e16c7cd58fe04f4b17e38d53cd57e --

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: New changeset 97d7e65dfed1d42d40d9bc2f630af56240555f02 by Victor Stinner in branch 'master': bpo-30830: logging.config.listen() calls server_close() (#3524) https://github.com/python/cpython/commit/97d7e65dfed1d42d40d9bc2f630af56240555f02 --

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: Again, PR 3523 is a temporary fix to unblock the situation. For a proper fix, I have a question: Should server_close() calls self.close_request(request) on thread requests? -- ___ Python tracker

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Changes by STINNER Victor : -- pull_requests: +3524 ___ Python tracker ___ ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: Attached PR 3523 is a temporary solution until we agree how to handle these threads and child processes in socketserver. I wrote the PR to fix "dangling threads" warnings to fix random buildbot failures. The PR fixes warnings but also reenable test_logging

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Changes by STINNER Victor : -- keywords: +patch pull_requests: +3521 stage: -> patch review ___ Python tracker ___

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: Multiple test_logging tests have been skipped until this issue is fixed: see bpo-30830 and commit 6966960468327c958b03391f71f24986bd697307. -- ___ Python tracker

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: Another recent example while testing bpo-31234: test_ThreadingUDPServer (test.test_socketserver.SocketServerTest) ... (...) done Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok --

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: New changeset 6966960468327c958b03391f71f24986bd697307 by Victor Stinner in branch 'master': bpo-30830: test_logging uses threading_setup/cleanup (#3137) https://github.com/python/cpython/commit/6966960468327c958b03391f71f24986bd697307 --

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

STINNER Victor added the comment: Oh, test_socketserver just failed with ENV_CHANGED on AMD64 Windows7 SP1 3.x: http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/865/steps/test/logs/stdio test_TCPServer (test.test_socketserver.SocketServerTest) ... creating server

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

Antoine Pitrou added the comment: The difference is that letting a thread run doesn't create a zombie thread, so I don't think the issue is really similar. -- nosy: +pitrou ___ Python tracker

[issue31233] socketserver.ThreadingMixIn leaks running threads after server_close()

New submission from STINNER Victor: test_logging has a race condition: sometimes, it leaks dangling threads on FreeBSD: bpo-30830. I identified the root issue: socketserver.ThreadingMixIn spawns threads without waiting for their completion in server_close(). This issue is very similar to