New submission from Sergei Zobov <ser...@szobov.ru>:
In case if the process target function put something unpickable in the `multiprocessing.Queue` and finish before `ForkingPickler`'s exception is handled we get two weird situations: * Internal queue semaphore stay acquired and never will be released, so we get `qsize()` always returns 1(or more if more then one process) even if there is nothing in queue's buffer and `empty()` returns `True`. * An exception's traceback will be silently suppressed. To reproduce this behavior I wrote a test in cpython repo (be aware sometime it pass sometime no): https://github.com/szobov/cpython/commit/95524f10a7b6510e9ab4647223cc67af85ebff86 I suppose it happens because of this: * The function `util._exit_function()` calls every time process target function finished. Inside this function the global (for current process) flag `utils._exiting` marks as `True`: https://github.com/python/cpython/blob/master/Lib/multiprocessing/process.py#L300 https://github.com/python/cpython/blob/master/Lib/multiprocessing/util.py#L294-L295 * When exception while pickling object happens in `Queue._feed` function we are ignore it because of the clause `is_exiting()` that checks the flag `utils._exiting` described above. https://github.com/python/cpython/blob/master/Lib/multiprocessing/queues.py#L257-L259 So we have unreleased semaphore and ignored exceptions. I think it's quite disappointing for user because they have broken queue and nothing about why it happened. I'm not quite sure that it's a bag but I found nothing in documentation about it. ---------- messages: 329889 nosy: szobov priority: normal severity: normal status: open title: multiprocessing.Queue in an inconsistent state and a traceback silently suppressed if put an unpickable object and process's target function is finished type: behavior versions: Python 3.7 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue35242> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
