[issue38106] [2.7] Race in PyThread_release_lock on macOS - can lead to memory corruption and deadlock

[Kirill Smelkov]

Kirill Smelkov  added the comment:

:) Yes and no. PyPy did not make a new release with the fix yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38106] [2.7] Race in PyThread_release_lock on macOS - can lead to memory corruption and deadlock

[STINNER Victor]

STINNER Victor  added the comment:

> PyPy people fixed the bug the same day it was reported: ...

Oh ok, so the CPython commit message is already outdated :-)

"PyPy has the same bug for both pypy2 and pypy3 (...)"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38106] [2.7] Race in PyThread_release_lock on macOS - can lead to memory corruption and deadlock

[Kirill Smelkov]

Kirill Smelkov  added the comment:

Victor, thanks for merging.

> I'm surprised that we still find new bugs in this code which is supposed to
> be battle tested! Maybe recent Darwin changed made the bug more likely.

As discussed above (https://bugs.python.org/issue38106#msg351917,
https://bugs.python.org/issue38106#msg351970), due to the GIL, the bug is not
possible to trigger from pure-python code, and it can be hit only via using CAPI
directly. I indeed started to observe the problem after moving locks and
other code that implement channels in Pygolang from Python to C level (see
"0.0.3" in https://pypi.org/project/pygolang/#pygolang-change-history)

The bug was there since 1994 and, from my point of view, it was not discovered
because locking functionality was not enough hammer-tested. The bug was also
not possible to explain without taking lock lifetime into account, as, without
create/destroy, just lock/unlock sequence was race free. 
https://bugs.python.org/issue433625 confirms that.

I cannot say about whether recent macOS changes are relevant. My primary
platform is Linux and I only recently started to use macOS under QEMU for
testing. However from my brief study of 
https://github.com/apple/darwin-libpthread
I believe the difference in scheduling related to pthread condition variable
signalling under macOS and Linux is there already for a long time.

> PyPy: it's now your turn to fix it ;-)

PyPy people fixed the bug the same day it was reported:

https://bitbucket.org/pypy/pypy/issues/3072

:)

Kirill

P.S. Mariusz, thanks also for your feedback.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38106] [2.7] Race in PyThread_release_lock on macOS - can lead to memory corruption and deadlock

[STINNER Victor]

STINNER Victor  added the comment:

I merged Kirill Smelkov's fix. Thanks. I close the issue.

I'm surprised that we still find new bugs in this code which is supposed to be 
battle tested! Maybe recent Darwin changed made the bug more likely.

PyPy: it's now your turn to fix it ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38106] [2.7] Race in PyThread_release_lock on macOS - can lead to memory corruption and deadlock

[STINNER Victor]

STINNER Victor  added the comment:


New changeset c5abd63e94fcea10bdfa7a20842c6af3248bbd74 by Victor Stinner 
(Kirill Smelkov) in branch '2.7':
bpo-38106: Fix race in pthread PyThread_release_lock() (GH-16047)
https://github.com/python/cpython/commit/c5abd63e94fcea10bdfa7a20842c6af3248bbd74


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38106] [2.7] Race in PyThread_release_lock on macOS - can lead to memory corruption and deadlock

[Mariusz Wawrzonkowski]

Mariusz Wawrzonkowski  added the comment:

I can confirm that I have the deadlock because of the reason described above. 
When I change the PyThread_release_lock source locally to signal before 
releasing the mutex, then my deadlock does not occur anymore. I have tested it 
on a heave multithreading model.

--
nosy: +mariosoft

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38106] [2.7] Race in PyThread_release_lock on macOS - can lead to memory corruption and deadlock

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38106] [2.7] Race in PyThread_release_lock on macOS - can lead to memory corruption and deadlock

[STINNER Victor]

Change by STINNER Victor :


--
title: Race in PyThread_release_lock - can lead to memory corruption and 
deadlock -> [2.7] Race in PyThread_release_lock on macOS - can lead to memory 
corruption and deadlock

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com