New submission from Oleksiy Markovets: INTRODUCTION
While embedding python script in c++ application I faced random crashes with error: Fatal Python error: Py_EndInterpreter: not the last thread Aborted Even though I explicitly join each thread created with *threading* module. Please see attachment, this is simplest python script + c++ code which demonstrates problem. By using threading.setprofiler I was able to make problem appear each run. If you uncomment time.sleep(1) problem won't reproduce. Note: In main.cpp I manually create separate interpreter because in real-life application is multi-threaded and different thread uses different interpreters for at least some sand-boxing. INVESTIGATION I did some investigation, and here is what I found out: * Each new thread is started by calling *thread_PyThread_start_new_thread* from threadmodule.c. Basically this function creates new thread which executes *t_bootstrap* and adds this thread to interpreter's thread list. * *t_bootstrap* executes python function *Thread.__bootstrap* and when it's done, removes thread from interpreter's thread list. * *Thread.__bootstrap* runs *Thread.run()* (actually python code which should be executed in separate thread) and when it's done calls *Thread.__stop* (which by mean of condition variable sets Boolean flag Thread.__stopped) * *Thread.join* doesn't wait thread to exit, it only waits for *Thread.__stopped* flag to be set. So here is race condition: When main thread finished *Thread.join* it's only guaranteed that *Thread.run* is finished, but *t_bootstrap* still may be running (interpreter's thread list is not cleared). POSSIBLE SOLUTION To call Thread.__stop from *t_bootstrap* (instead of Thread.__bootstrap) after removing thread from interpreter's thread list. Or do not use detached threads and call something like *pthread_join* in *Thread.join*. Which is IMHO much clearer approach but also requires much more efforts. ---------- components: Extension Modules files: main.cpp messages: 246035 nosy: Oleksiy Markovets priority: normal severity: normal status: open title: Race condition and crash when embedding multi-thread script type: crash versions: Python 2.7 Added file: http://bugs.python.org/file39840/main.cpp _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24544> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com