[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Benjamin Peterson added the comment: Yes On Wed, Oct 31, 2018, at 22:40, Mike Frysinger wrote: > > Mike Frysinger added the comment: > > that's highlighting the SemLock._make_name func which doesn't > have retry> logic in it. did you mean to highlight SemLock.__init__ which > has a > retry

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Mike Frysinger added the comment: that's highlighting the SemLock._make_name func which doesn't have retry logic in it. did you mean to highlight SemLock.__init__ which has a retry loop that looks similar to the C code ?

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Benjamin Peterson added the comment: The retry logic is implemented at a different layer in Python 3: https://github.com/python/cpython/blob/a1c249c40517917d2e0971d55aea8d14a44b2cc8/Lib/multiprocessing/synchronize.py#L115-L117 -- ___ Python

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Change by Benjamin Peterson : -- pull_requests: +9590 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Mike Frysinger added the comment: it does seem like the patch was never applied to any python 3 branch :/ -- ___ Python tracker ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Ryan Petrello added the comment: Any chance this patch was every applied to Python3? It looks to me like 3.6 has the old code: https://github.com/python/cpython/blob/3.6/Modules/_multiprocessing/semaphore.c#L448 -- nosy: +ryan.petrello ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Davin Potts added the comment: @neologix: I second your proposed patch -- looks like a winner to me. Apologies for not following up earlier. -- ___ Python tracker

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Roundup Robot added the comment: New changeset d3662c088db8 by Charles-François Natali in branch '2.7': Issue #24303: Fix random EEXIST upon multiprocessing semaphores creation with https://hg.python.org/cpython/rev/d3662c088db8 -- nosy: +python-dev

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Mike Frysinger added the comment: i don't feel strongly about either version -- ___ Python tracker ___ ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid -> process mapping.

Charles-François Natali added the comment: Anyone opposed to me committing the patch I submitted? It slves a real problem, and is fairly straight-forward (and conceptually more correct). -- ___ Python tracker

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Changes by Charles-François Natali cf.nat...@gmail.com: -- keywords: +needs review nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Charles-François Natali added the comment: Here's a patch against 2.7 using _PyOS_URandom(): it should apply as-is to 3.3. -- keywords: +patch nosy: +neologix versions: +Python 3.3 Added file: http://bugs.python.org/file39679/mp_sem_race.diff ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Mike Frysinger added the comment: also, it looks like this bug is in python-3.3. not sure what the status of that branch is, but maybe the backport from 3.4 would be easy ? -- ___ Python tracker rep...@bugs.python.org

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Mike Frysinger added the comment: the original report on our side w/bunches of tracebacks: http://crbug.com/481223 with that traceback in hand, it's pretty trivial to write a reproduction :). attached! i couldn't figure out how to make it work w/out completely execing a new python instance.

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Davin Potts added the comment: Triggering it regularly in a build farm indeed sounds like genuine pain. @Paul or @vapier: In tracking down this issue, did you already create a convenient way to repro the misbehavior that could be used in testing? Any finalized patch that we make will need

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Changes by Yury Selivanov yseliva...@gmail.com: -- nosy: +jnoller ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___ ___ Python-bugs-list

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Davin Potts added the comment: At first blush it does appear there is potential for conflict because of how the semaphore filename template was implemented -- that's a cool find. In practice, I wonder how often this has actually bitten anyone in the real world. The Linux world's use of

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Mike Frysinger added the comment: we hit this problem daily in Chromium OS's build farm because we use pid namespaces heavily -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Changes by Dirkjan Ochtman dirk...@ochtman.nl: -- nosy: +djc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___ ___ Python-bugs-list mailing

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

New submission from Paul Hobbs: Using pid namespacing it is possible to have multiple processes with the same pid. semlock_new creates a semaphore file with the template /dev/shm/mp{pid}-{counter}. This can conflict if the same semaphore file already exists due to another Python process

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Changes by Ned Deily n...@acm.org: -- nosy: +devin, sbt stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___ ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Changes by Ned Deily n...@acm.org: -- nosy: +davin -devin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___ ___ Python-bugs-list mailing

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Changes by Mike Frysinger vap...@users.sourceforge.net: -- nosy: +vapier ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___ ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___

[issue24303] OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid - process mapping.

Changes by Mike Gilbert floppymas...@gmail.com: -- nosy: +floppymaster ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24303 ___ ___