[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 6318e45bda6c37d5497f33a6039cdb65aa494c93 by Miss Islington (bot) 
in branch '3.8':
bpo-40089: Fix threading._after_fork() (GH-19191) (GH-19194)
https://github.com/python/cpython/commit/6318e45bda6c37d5497f33a6039cdb65aa494c93


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a514ccb3ea6f01fef850d9465b82a1670d5ace44 by Miss Islington (bot) 
in branch '3.7':
bpo-40089: Fix threading._after_fork() (GH-19191) (GH-19193)
https://github.com/python/cpython/commit/a514ccb3ea6f01fef850d9465b82a1670d5ace44


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 87255be6964979b5abdc4b9dcf81cdcfdad6e753 by Victor Stinner in 
branch 'master':
bpo-40089: Add _at_fork_reinit() method to locks (GH-19195)
https://github.com/python/cpython/commit/87255be6964979b5abdc4b9dcf81cdcfdad6e753


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-40092: "Crash in _PyThreadState_DeleteExcept() at fork in the 
process child".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-40091 "Crash in logging._after_at_fork_child_reinit_locks()" that 
I just created.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18557
pull_request: https://github.com/python/cpython/pull/19196

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[miss-islington]

Change by miss-islington :


--
pull_requests: +18554
pull_request: https://github.com/python/cpython/pull/19194

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18555
pull_request: https://github.com/python/cpython/pull/19195

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:


New changeset d8ff44ce4cd6f3ec0fab5fccda6bf14afcb25c30 by Victor Stinner in 
branch 'master':
bpo-40089: Fix threading._after_fork() (GH-19191)
https://github.com/python/cpython/commit/d8ff44ce4cd6f3ec0fab5fccda6bf14afcb25c30


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 2.0 -> 3.0
pull_requests: +18553
pull_request: https://github.com/python/cpython/pull/19193

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:

_at_fork() has a bug: it creates a _DummyThread instead of a _MainThread. 
Example:
---
import os, _thread, threading, time

def fork_in_thread():
pid = os.fork()
if pid:
# parent
os._exit(0)

# child process
print(f"child process: {threading.current_thread()=}")
print(f"child process: {threading._main_thread=}")

print(f"parent process: {threading.current_thread()=}")
print(f"parent process: {threading._main_thread=}")

_thread.start_new_thread(fork_in_thread, ())
# block the main thread: fork_in_thread() exit the process
time.sleep(60)
---

Output:
---
parent process: threading.current_thread()=<_MainThread(MainThread, started 
139879200077632)>
parent process: threading._main_thread=<_MainThread(MainThread, started 
139879200077632)>
child process: threading.current_thread()=<_DummyThread(Dummy-1, started daemon 
139878980245248)>
child process: threading._main_thread=<_DummyThread(Dummy-1, started daemon 
139878980245248)>
---

My PR 19191 fix the issue:
---
parent process: threading.current_thread()=<_MainThread(MainThread, started 
140583665170240)>
parent process: threading._main_thread=<_MainThread(MainThread, started 
140583665170240)>
child process: threading.current_thread()=<_MainThread(MainThread, started 
140583445075712)>
child process: threading._main_thread=<_MainThread(MainThread, started 
140583445075712)>
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +18551
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19191

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
nosy: +Batuhan Taskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

New submission from STINNER Victor :

Using a lock after fork() is unsafe and can crash.

Example of a crash in logging after a fork on AIX:
https://bugs.python.org/issue40068#msg365028

This problem is explained in length in bpo-6721: "Locks in the standard library 
should be sanitized on fork".

The threading module registers an "at fork" callback: 
Thread._reset_internal_locks() is called to reset self._started 
(threading.Event) and self._tstate_lock. The current implementation creates new 
Python lock objects and forgets about the old ones.

I propose to add a new _at_fork_reinit() method to Python lock objects which 
reinitializes the native lock internally without having to create a new Python 
object.

Currently, my implementation basically creates a new native lock object and 
forgets about the old new (don't call PyThread_free_lock()).

Tomorrow, we can imagine a more efficient impementation using platform specific 
function to handle this case without having to forget about the old lock.

--
components: Library (Lib)
messages: 365157
nosy: vstinner
priority: normal
severity: normal
status: open
title: Add _at_fork_reinit() method to locks
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com