[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

Changes by Antoine Pitrou : -- status: pending -> closed ___ Python tracker ___ ___

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

Antoine Pitrou added the comment: I'm willing to close this issue, as I don't think a ResourceWarning is appropriate here. -- resolution: -> not a bug stage: -> resolved status: open -> pending ___ Python tracker

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

Antoine Pitrou added the comment: I don't think a ResourceWarning should be emitted. There is no risk of data loss or resource leak if you don't close a multiprocessing Queue explicitly. Actually, in the real world, I don't think I've ever seen code that closes queues explicitly.

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

STINNER Victor added the comment: Let's discuss created_by_this_process in bpo-30886. This issue is more about adding or not a ResourceWarning. -- ___ Python tracker

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

STINNER Victor added the comment: > Specifically "the thread wasn't started by a subprocess"... I'm talking about this check in Queue._start_thread() of multiprocessing.queues: created_by_this_process = (self._opid == os.getpid()) if not self._joincancelled and not

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

Antoine Pitrou added the comment: Specifically "the thread wasn't started by a subprocess"... -- ___ Python tracker ___

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

Antoine Pitrou added the comment: > The problem is that multiprocessing.Queue.join_thread() does nothing since > the thread wasn't started by a subprocess. I don't understand what this means. Can you clarify a bit? -- ___ Python tracker

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

STINNER Victor added the comment: Another example: http://bugs.python.org/issue30886#msg298014 "The problem is that multiprocessing.Queue.join_thread() does nothing since the thread wasn't started by a subprocess." -- ___ Python tracker

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

Antoine Pitrou added the comment: The thread seems to be stopped when the Queue object is finalized: # Send sentinel to the thread queue object when garbage collected self._close = Finalize( self, Queue._finalize_close, [self._buffer, self._notempty],

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

STINNER Victor added the comment: See also issue #30244: Emit a ResourceWarning in concurrent.futures executor destructors. -- ___ Python tracker ___

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

Changes by Raymond Hettinger : -- assignee: -> davin ___ Python tracker ___ ___

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

STINNER Victor added the comment: Oh, I forgot that I hitted this issue while analyzing issue #30131: test_logging leaks a "dangling" thread. It took me a while to find multiprocessing queues in the big test_logging file! -- ___ Python tracker

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

STINNER Victor added the comment: Example: haypo@selma$ ./python queue_leak.py number of thread diff: +1 dangling threads! before: [<_MainThread(MainThread, started 139814961067072)>] after: [<_MainThread(MainThread, started 139814961067072)>] Note: queue_leak.py resource check is based on

[issue30171] Emit ResourceWarning in multiprocessing Queue destructor

New submission from STINNER Victor: A multiprocessing Queue object managers multiple resources: * a multiprocessing Pipe * a thread * (a lock and a semaphore) If a Queue is not cleaned up properly, your application may leak many resources. Try attached queue_leak.py to see an example "leaking