[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-18 Thread Henrique Andrade
Henrique Andrade <h...@unscrambl.com> added the comment: You're a missing the fact that in C++, there is no garbage collector. The destructor both releases resources and deallocates memory. There is no window between releasing resources and object destruction. Python, while not exactl

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-18 Thread Henrique Andrade
Henrique Andrade <h...@unscrambl.com> added the comment: Your comparison is not correct. RAII in C++ ensures that, on object destruction, resources that have been acquired will be closed and deallocated. The closest analogy in Python is the use of a context manager, which, btw, a Queu

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-18 Thread Henrique Andrade
Henrique Andrade <h...@unscrambl.com> added the comment: I don't want to "close the pipes but maintain the queue alive" - I want to terminate the queue and make sure that no resources are leaked. It's that simple. When one closes a file or a socket, there is no underlying OS res

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-18 Thread Henrique Andrade
Henrique Andrade <h...@unscrambl.com> added the comment: @pablo: I am using Python 2.7.12 (distributed with Ubuntu 16), what are you using? This might explain the difference between what we see. Yet, irrespective of this difference, imho, it would be a better design to have "clos

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-17 Thread Henrique Andrade
Henrique Andrade <h...@unscrambl.com> added the comment: Here is the repro (I am running this on Ubuntu 16 with the stock Python version 2.7.12): #!/usr/bin/env python imp

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-15 Thread Henrique Andrade
Henrique Andrade <h...@unscrambl.com> added the comment: Unfortunately this is not the case. I will shrink my repro down to a more manageable size and post it here. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-15 Thread Henrique Andrade
Henrique Andrade <h...@unscrambl.com> added the comment: Pablo, but there is no way to close the other side. Indeed, if you look in the implementation, you will see that the writer file descriptor can't be closed. -- ___ Python tracke

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-15 Thread Henrique Andrade
Henrique Andrade <h...@unscrambl.com> added the comment: Correcting my original - after close(): > ll /proc/8096/fd total 0 dr-x-- 2 hcma hcma 0 2018-03-15 14:03:23.210089578 -0400 . dr-xr-xr-x 9 hcma hcma 0 2018-03-15 14:03:23.190089760 -0400 .. lrwx-- 1 hcma hcma 64 2018-03

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

2018-03-15 Thread Henrique Andrade
New submission from Henrique Andrade <h...@unscrambl.com>: A simple example like such demonstrates that one of the file descriptors associated with the underlying pipe will be leaked: >>> from multiprocessing.queues import Queue >>> x = Queue() >>> x.close()

[issue25459] EAGAIN errors in Python logging module

2016-03-08 Thread Henrique Andrade
Henrique Andrade added the comment: Vinay, apologies for the long time you took me to respond (the notification was stuck in my Spam folder, thanks Gmail!). Indeed, both of your comments are spot on. In my particular case, I ended up implementing what your first suggestion alludes to (a retry

[issue25459] EAGAIN errors in Python logging module

2015-10-22 Thread Henrique Andrade
New submission from Henrique Andrade: There is a particular bug we hit when using the Python logging module very consistently under the particular settings in which we run one of our applications. We are using Python 2.7.10 on RHEL7/RHEL6/Ubuntu14.04. Anyways, here is the symptom: Traceback

[issue25459] EAGAIN errors in Python logging module

2015-10-22 Thread Henrique Andrade
Henrique Andrade added the comment: The stream in this case (where I hit the bug) is just the console, but I suspect the same issue will affect other streams too. A key piece of information is that this is probably triggered by having a custom SIGPIPE handler, which my particular application