[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

STINNER Victor added the comment:


New changeset b65cb8a35641675d44af84c9b18c40c6094f03ef by Victor Stinner in 
branch '2.7':
bpo-31019: Fix multiprocessing.Process.is_alive() (#2875) (#2882)
https://github.com/python/cpython/commit/b65cb8a35641675d44af84c9b18c40c6094f03ef


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

STINNER Victor added the comment:

I fixed multiprocessing.Process.is_alive() in 2.7, 3.6 and master. I close the 
issue.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions: +Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2934

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

STINNER Victor added the comment:


New changeset d0adfb25c5082046133a18fd185375508c1c334f by Victor Stinner in 
branch '3.6':
[3.6] bpo-26762, bpo-31019: Backport multiprocessing fixes from master to 3.6 
(#2879)
https://github.com/python/cpython/commit/d0adfb25c5082046133a18fd185375508c1c334f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2931

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

STINNER Victor added the comment:


New changeset 2db64823c20538a6cfc6033661fab5711d2d4585 by Victor Stinner in 
branch 'master':
bpo-31019: Fix multiprocessing.Process.is_alive() (#2875)
https://github.com/python/cpython/commit/2db64823c20538a6cfc6033661fab5711d2d4585


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

STINNER Victor added the comment:

Hum, I don't think that https://github.com/python/cpython/pull/2849 fixes the 
root issue.

The root issue is that calling Process.is_alive() doesn't update the _children 
list if the process completes. I wrote a fix for that:
https://github.com/python/cpython/pull/2875

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2926

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2898

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31019] multiprocessing.Pool should join "dead" processes

[STINNER Victor]

New submission from STINNER Victor:

With debug patches for bpo-26762, I noticed that some unit tests of 
test_multiprocessing_spawn leaks "dangling" processes:
---
haypo@selma$ ./python -m test --fail-env-changed test_multiprocessing_spawn -v 
--match test.test_multiprocessing_spawn.WithProcessesTestPool.test_context
== CPython 3.7.0a0 (heads/master:b364d9f, Jul 24 2017, 11:06:33) [GCC 6.3.1 
20161221 (Red Hat 6.3.1-1)]
== Linux-4.11.9-200.fc25.x86_64-x86_64-with-fedora-25-Twenty_Five little-endian
== hash algorithm: siphash24 64bit
== cwd: /home/haypo/prog/python/master/build/test_python_20982
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, 
dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, 
verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 load avg: 0.16 [1/1] test_multiprocessing_spawn
test_context (test.test_multiprocessing_spawn.WithProcessesTestPool) ... ok
Warning -- Dangling processes: {}
Dangling processes: {}

--
Ran 1 test in 1.342s

OK
test_multiprocessing_spawn failed (env changed)

1 test altered the execution environment:
test_multiprocessing_spawn

Total duration: 1 sec
Tests result: ENV CHANGED
---

multiprocessing.Pool.terminate() doesn't call the join() method of a Process 
object if its is_alive() method returns false. But in practice, avoid the 
join() creates "dangling" processes.

Attached pull request fixes the warning: Pool.terminate() now calls join() on 
all processes including "dead" processes.

--
components: Library (Lib)
messages: 299002
nosy: haypo
priority: normal
severity: normal
status: open
title: multiprocessing.Pool should join "dead" processes
type: resource usage
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com