[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

Glen Walker added the comment: Correct me if I'm wrong, the change released in Python 2.7.15 doesn't actually fix this race condition. The race is: * T1: gc_was_enabled = gc.isenabled() # True * T1: gc.disable() * T2: gc_was_enabled = gc.isenabled() # False * T1: gc.enable() * T2: gc.disable(

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

Gregory P. Smith added the comment: the recommendation to use subprocess32 still stands, but this bug has been "fixed" in as much as it can be without a complete rewrite within 2.7. -- resolution: -> fixed stage: -> resolved status: open -> closed

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

Gregory P. Smith added the comment: New changeset 5e8e371364ee58dadb9a4e4e51c7e9cf6bedbfae by Gregory P. Smith in branch '2.7': bpo-27448: Work around a gc.disable race condition in subprocess. (#1932) https://github.com/python/cpython/commit/5e8e371364ee58dadb9a4e4e51c7e9cf6bedbfae -

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

Changes by Gregory P. Smith : -- pull_requests: +2009 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://m

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

Gregory P. Smith added the comment: ah, okay, that makes much more sense and is indeed a bug. Recommendation given that most people won't be using a python 2.7 version that has this and tons of other subprocess from a threaded process bugs fixed: Use subprocess32 where this isn't a problem. -

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

Andrew added the comment: Gregory P. Smith, "if you are seeing a memory leak and the statements that fail before the gc.enable call are more than" The problem in my case was different. It was race condition between gc.isenabled and gc.disable. Isenabled was called when gc was disabled by a

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

Gregory P. Smith added the comment: FWIW at this point i'm not willing to "fix" issues in Python 2.7's subprocess module for POSIX platforms as subprocess32 is a superior drop in alternative without the issues. inspecting the 2.7 code, if you are seeing a memory leak and the statements that f

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

Andrew added the comment: pppery, I don't think I am breaking gc functionality with my code. The code line I gave just meant to give the basic idea of what was helping to workaround this. If you are actually interested in the code I use, it is below: def fix_subprocess_racecondition(): """

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

ppperry added the comment: What are you doing that creates so many circular references that the not collecting them causes a massive memory leak? Or are you using an alternative implementation of Python? In addition, your monkeypatching is incorrect. `subprocess.gc` is the same object as the

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

R. David Murray added the comment: I presume we don't re-enable gc in the child because if there's an exception it is exiting anyway. FYI subprocess32, which fixes some other race conditions, shouldn't have this problem, since it is a backport of subprocess from python3, which no longer manip

[issue27448] Race condition in subprocess.Popen which causes a huge memory leak

New submission from Andrew: We had problem where at some point python start consuming RAM. Until it ends. The reason was race condition in subprocess.Popen. Which triggered gc.disable() and never gc.enable(). The workaround we use is: subprocess.gc.isenabled = lambda: True The scenario for r