[issue35267] reproducible deadlock with multiprocessing.Pool

[Davin Potts]

Davin Potts  added the comment:

I second what @vstinner already said in the comments for PR11143, that this 
should not merely be documented.

--
nosy: +davin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Windson Yang]

Change by Windson Yang :


--
keywords: +patch
pull_requests: +10374
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Jonathan Gossage]

Jonathan Gossage  added the comment:

I think documentation is sufficient but I would like it to state the pitfalls 
available if apply_async is not synchronized correctly which will happen 
whenever the output does not fit the pipe buffer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Windson Yang]

Windson Yang  added the comment:

As Jonathan Gossage said, I think it may break some code to fix this issue, 
maybe we could just add a warning on the document?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Jonathan Gossage]

Change by Jonathan Gossage :


Added file: https://bugs.python.org/file47980/lock1.result.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Jonathan Gossage]

Jonathan Gossage  added the comment:

This is a great example of abusing the multi-processing API and thus
creating timing errors that lead to locks not being released. What is
happening is that the example attempts to transmit data that is too
big for the underlying pipe and this creates the timing errors since
the pipe that returns the result of invoking apply_async does not have
the capacity of transmitting the result of the worker process in a single
operation and accordingly the worker process returns prematurely while
still holding the pipe lock. This can be seen by testing the ready status
of the result returned by apply_async which will show whether the complete
result of the worker process has been received.
  
 The solution to this situation is simple, simply invoke get() on
 the asynchronous result returned by apply_async. Using this call has the
 effect of correctly synchronizing the pipe used by the low-level queues
 used by the Pool Manager and there will be no lock left active when it
 should'nt be. The script lock1.py contains code that implements this fix
 as well as verifying that everything has worked properly. The output from
 the script is found in the file lock1.result.txt.
  
 Because there is an API based solution to the problem plus the
 behavior of apply_async makes sense in that the process of transferring
 multi-buffer data is very CPU intensive and should be delegated to the
 worker process rather than to the main-line process, I do not recommend that
 any changes be made to the multiprocessing code in Python, rather the
 solution should use the available multiprocessing API correctly.

--
nosy: +Jonathan.Gossage
Added file: https://bugs.python.org/file47979/lock1.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[dzhu]

dzhu  added the comment:

Given the hairiness of the deadlock, I think I would rather let someone who has 
more experience with the codebase in general handle it, but I can come back to 
it if it doesn't get addressed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Windson Yang]

Windson Yang  added the comment:

I will work on it if no one wants to create a PR for this next week.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Neil Conway]

Change by Neil Conway :


--
nosy: +nconway

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Ned Deily]

Change by Ned Deily :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[Windson Yang]

Windson Yang  added the comment:

Hello, dzhu. I can reproduce on my OSX, since you already dive into the code, 
do you have any idea to fix or improve it?

--
nosy: +Windson Yang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35267] reproducible deadlock with multiprocessing.Pool

[dzhu]

New submission from dzhu :

The attached snippet causes a deadlock just about every time it's run (tested 
with 3.6.7/Ubuntu, 3.7.1/Arch, 3.6.7/OSX, and 3.7.1/OSX -- deadlock seems to be 
less frequent on the last, but still common). The issue appears to be something 
like the following sequence of events:

1. The main thread calls pool.__exit__, eventually entering 
Pool._terminate_pool.
2. result_handler's state is set to TERMINATE, causing it to stop reading from 
outqueue.
3. The main thread, in _terminate_pool, joins on worker_handler, which is 
(usually) in the middle of sleeping for 0.1 seconds, opening a window for the 
next two steps to occur.
4. The worker process finishes its task and acquires the shared outqueue._wlock.
5. The worker attempts to put the result into outqueue, but its pickled form is 
too big to fit into the buffer of os.pipe, and it blocks here with the lock 
held.
6. worker_handler wakes up and exits, freeing _terminate_pool to continue.
7. _terminate_pool terminates the worker.
8. task_handler tries to put None into outqueue, but blocks, since the lock was 
acquired by the terminated worker.
9. _terminate_pool joins on task_handler, and everything is deadlocked.

--
components: Library (Lib)
files: lock.py
messages: 330017
nosy: dzhu
priority: normal
severity: normal
status: open
title: reproducible deadlock with multiprocessing.Pool
type: behavior
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47937/lock.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com