[issue42763] Exposing a race in the "_warnings" resulting Python parser crash

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.11 -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42763] Exposing a race in the "_warnings" resulting Python parser crash

[Xinmeng Xia]

Xinmeng Xia  added the comment:

Now I see. By the way, I think this case should be moved back to 
"cpython/Lib/test/crashers/" since the bug still exists.  It is not fixed 
completely, the old case is outdated. I suggest we can put the new case into 
directory "crashers".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42763] Exposing a race in the "_warnings" resulting Python parser crash

[Steve Stagg]

Steve Stagg  added the comment:

If we take your minimal example (that /sometimes/ crashes), and look at what 
python is doing:

import threading
t = threading.Thread(target=1, daemon=True).start()

---

The main thread does the following:

M1. Startup interpreter, parse code
M2. Import & execute threading module
M3. Create thread `T`, set target=1, set daemon=True
M4. Start thread
M5. Finalize local variables & state
M6. Terminate daemon threads
M7. Acquire IO lock to clean up IO
M8. Shutdown interpreter

Whereas the thread `T` does this:

T1. Startup
T2. Try to call target
T3. Realise that target is not callable & raise exception ('int' object is not 
callable)
T4. Acquire IO lock to print traceback
T5. Print traceback line 1
T6. Print traceback line n
T7. Release IO lock
T8. Shutdown thread


Now steps T1->T8 can happen *at any time* between M4 and M6.  But the moment M6 
runs, the thread disappears without warning.

If the T4 step is run, then the daemon thread owns the IO lock, and must 
release it, but if step T7 doesn't run, then that lock is never released.

So in the case where you get a crash, the order is something like:

M1. M2. M3. M4.  T1. T2. T3. T4. T5. M5. [thread killed] M6. M7. <- Crash, 
because M7 can't acquire lock

But when there is no crash, the order is probably:

M1. M2. M3. M4.  T1. T2. T3. T4. T5. T6. T7. T8. M5. M6. M7. M8. <- No Crash 
because IO lock released in T7.

So you can see that this is the same fundamental thing as issue42717  (Where 
the shutdown during output was caused by spamming output repeatedly), but 
caused by a different route.

It might be possible to add handlers to the daemonic thread shutdown to clean 
up IO locks more cleanly?  But it's tricky, it seems like daemonic threads 
shutdown when they attempt to re-acquire the GIL after the interpreter has 
shutdown, so we're in pretty complex state management territory here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42763] Exposing a race in the "_warnings" resulting Python parser crash

[Xinmeng Xia]

Xinmeng Xia  added the comment:

Thank you, but I don't think this is a duplicate of issue42717. This crash is 
probably caused by the parameter "target" of Thread. "Target" accept the 
"callable object". Defaults to None. In this program, it's assigned to a "Int 
object". I think a pre-checking of parameter for such error should be added. 



Minimal test case should be:
===
import threading
for i in range(10):
t = threading.Thread(target=1, daemon=True).start()  
= 



In fact, I try this one. No loop, It sometime crashes the parser.(not always, 
twice in ten times)
==
import threading
t = threading.Thread(target=1, daemon=True).start()
==

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42763] Exposing a race in the "_warnings" resulting Python parser crash

[Steve Stagg]

Steve Stagg  added the comment:

Looks like a duplicate of issue42717.  

Causing a daemonic thread to terminate while doing IO will trigger the above 
abort

--
nosy: +stestagg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42763] Exposing a race in the "_warnings" resulting Python parser crash

[Xinmeng Xia]

New submission from Xinmeng Xia :

This program is initially from 
"cpython/Lib/test/crashers/warnings_del_crasher.py" in Python 2.7. The original 
case is fixed for all version of Python and removed from "crashers" directory. 
However, if we replace the statement  "for i in range(10):" of original program 
 with the statement "for do_work in range(10):" . The race will happen again, 
and it will crash Python 3.7 - 3.10.
==
import threading
import warnings

class WarnOnDel(object):
def __del__(self):
warnings.warn("oh no something went wrong", UserWarning)

def do_work():
while True:
w = WarnOnDel()

-for i in range(10):
+for do_work in range(10):
t = threading.Thread(target=do_work)
t.setDaemon(1)
t.start()
=


Error messages on Python 3.7-3.10:
---
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/python310/lib/python3.10/threading.py", line 960, in 
_bootstrap_inner
Exception in thread Thread-3:
Traceback (most recent call last):
Exception in thread Thread-4:
  File "/usr/local/python310/lib/python3.10/threading.py", line 960, in 
_bootstrap_inner
Exception in thread Thread-5:
Traceback (most recent call last):
self.run()
Traceback (most recent call last):
self.run()
Exception in thread Thread-6:
Exception in thread Thread-8:
Exception in thread Thread-9:
Exception in thread Thread-10:
Fatal Python error: _enter_buffered_busy: could not acquire lock for 
<_io.BufferedWriter name=''> at interpreter shutdown, possibly due to 
daemon threads
Python runtime state: finalizing (tstate=0x2679180)

Current thread 0x7f3481d3a700 (most recent call first):

Aborted (core dumped)

--
components: Interpreter Core
messages: 383883
nosy: xxm
priority: normal
severity: normal
status: open
title: Exposing a race in the "_warnings" resulting Python parser crash
type: crash
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com