[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: New changeset 852aee69f49c654a03ad1f64d90a78ba8848e2c6 by Stefan Krah in branch '3.7': bpo-39776: Lock ++interp->tstate_next_unique_id (GH-18746) https://github.com/python/cpython/commit/852aee69f49c654a03ad1f64d90a78ba8848e2c6 --

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: New changeset 5a92f42d8723ee865be80f028d402204649da15d by Stefan Krah in branch '3.8': bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746) (#18752) https://github.com/python/cpython/commit/5a92f42d8723ee865be80f028d402204649da15d --

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Change by Stefan Krah : -- pull_requests: +18108 pull_request: https://github.com/python/cpython/pull/18753 ___ Python tracker ___

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Change by Stefan Krah : -- pull_requests: +18107 pull_request: https://github.com/python/cpython/pull/18752 ___ Python tracker ___

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: New changeset b3b9ade4a3d3fe00d933bcd8fc5c5c755d1024f9 by Stefan Krah in branch 'master': bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746) https://github.com/python/cpython/commit/b3b9ade4a3d3fe00d933bcd8fc5c5c755d1024f9 --

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: I think the PR fixes the issue but I have to run longer tests still. Threads created by PyGILState_Ensure() could have a duplicate tstate->id, which confused the ContextVar caching machinery. -- ___ Python tracker

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Change by Stefan Krah : -- keywords: +patch pull_requests: +18101 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/18746 ___ Python tracker

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Change by STINNER Victor : -- nosy: +vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Change by Antoine Pitrou : -- priority: normal -> critical versions: +Python 3.9 ___ Python tracker ___ ___ Python-bugs-list

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Change by Raymond Hettinger : -- nosy: +yselivanov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: > With python 3.7.3 without https://github.com/python/cpython/pull/5278 works > just fine. Thanks, I'm now getting the same results as you. Looking at the smaller test case, I also agree that it should work (as it did in 3.6). -- keywords:

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Evgeny Boytsov added the comment: Also I understood the source of your crash with my initial example. Since you haven't used CMake to configure project, pybind didn't setup required macroses to enable threading support. So no issues in pybind. --

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Evgeny Boytsov added the comment: I rewrote my example without pybind and eliminated C++ module (I realized that time.sleep() also releases the GIL, so we achieve the same effect). Still the same results: with python 3.7.3 app crashes with attached ASAN output, with python 3.7.3 without

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: Regarding *my* issue, it could be anything, e.g. a missing call to PyEval_InitThreads() in 3.6: "Changed in version 3.7: This function is now called by Py_Initialize(), so you don’t have to call it yourself anymore." This is why we need to eliminate pybind11

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Evgeny Boytsov added the comment: Your callstack is very strange. At line 30 of main.cpp GIL is obviously locked: // importing module in this tread gstate = PyGILState_Ensure(); py::module crash_test = py::module::import( "crash_test" ); <-- import PyGILState_Release( gstate );

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Change by Stefan Krah : -- resolution: not a bug -> stage: resolved -> ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: Note that my pybind11 is from GitHub master, it can also be a pybind11 issue. It is interesting that you cannot reproduce your original issue with 3.6, so I'm reopening this issue. I think we need a reproducer without pybind11 though, could you tweak

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: This is 3.6.7, compiled --with-pydebug: $ ./main Aborted (core dumped) (gdb) bt #0 0x7f9974077428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 #1 0x7f997407902a in __GI_abort () at abort.c:89 #2 0x0056e2d1 in

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Evgeny Boytsov added the comment: I'am unable to reproduce neither my or your issues with python 3.6. The program runs infinitely as it meant to be. Can you please give me C++ traceback from the core dump, which was created when you ran my program? --

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Evgeny Boytsov added the comment: Thank you for feedback. I will try to reproduce the issue with 3.6. By the way, haven't you used gdb with python pretty-printers enabled to examine the state of the program? I've got the same error message, then I breaked the execution in debugger and tried

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: I built your example with 3.6: git clone https://github.com/pybind/pybind11 wget https://bugs.python.org/file48923/decimal_crash.zip unzip decimal_crash.zip git checkout v3.6.7 ./configure --with-pydebug make g++ -std=c++11 -pthread -Wno-unused-result

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Evgeny Boytsov added the comment: Please note, that UnlockGILandSleep takes GIL back before returning. In a real production code there is a database query. In this example I emulate them with random sleep. So I don't see any problems here. -- ___

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Change by Stefan Krah : -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker ___ ___

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: I've briefly looked at the zip archive. Without going much into the C++ module as a whole, this should not be done: gil_unlocker.UnlockGILAndSleep() self.val = decimal.Decimal(1) / decimal.Decimal(7) gil_unlocker.UnlockGILAndSleep() If you want

[issue39776] Crash in decimal module in heavy-multithreaded scenario

Stefan Krah added the comment: Before I look at the example code: Can you also reproduce this with Python 3.6? The threading code in _decimal was changed to a ContextVar in 3.7. There's a high chance though that the problem is in the c++ module. -- nosy: +skrah

[issue39776] Crash in decimal module in heavy-multithreaded scenario

New submission from Evgeny Boytsov : Hello everybody! We are using Python 3.7 running at CentOS 7 x64. Python is used as a library to create dynamic extensions for our app server. Some time ago we began to experience crashes in decimal module in some heavy-multithreaded scenarios. After