On Tue, Apr 21, 2020 at 10:49 AM Antoine Pitrou <[email protected]> wrote:
> On Tue, 21 Apr 2020 18:46:04 +0200 > Petr Viktorin <[email protected]> wrote: > > On 2020-04-21 11:01, Antoine Pitrou wrote: > > > On Mon, 20 Apr 2020 19:21:21 -0600 > > > Eric Snow <[email protected]> wrote: > > >> Honest question: how many C extensions have process-global state that > > >> will cause problems under subinterpreters? In other words, how many > > >> already break in mod_wsgi? > > > > > > A slightly tricky question is what happens if a PyObject survives > > > longer than the subinterpreter that created it. > > > > > > For example, in PyArrow, we allow passing a Python buffer-like object > > > as a C++ buffer to C++ APIs. The C++ buffer could conceivably be kept > > > around by C++ code for some time. When the C++ buffer is destroyed, > > > Py_DECREF() is called on the Python object (I figure that we would have > > > to switch to the future interpreter-aware PyGILState API -- when will > > > it be available?). > In this scenario given that the PyObject owning the buffer passed to C++ by PyArrow was Py_INCREF'd, an option during Py_Finalize is to not release anything who's refcount never made it to 0. Which also implies leaving enough of the interpreter state around so that Py_DECREF could be called and trigger the free(). Basically (sub)interpreter finalization should have the right to return failure. Everything ever allocated by an interpreter holds a reference (implied today) to its associated interpreter state. -gps > > > > > > > But what happens if the interpreter which created > > > the Python object is already destroyed at this point? > > > > That's a good question. What happens today if someone calls Py_Finalize > > while the buffer is still around? > > > > I don't think you need to treat *sub*interpreters specially. > > The difference is that subinterpreters may have a shorter lifetime than > the main interpreter. Also, we're currently using a global memory > allocator, so simple operations may work until the process dies. > Regards > > Antoine. > > _______________________________________________ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/UWWO3RS3UOYCZXVFORGTJAEXXOCUQQXD/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/LJJVHHK7OG6ZRISCT6G2NWBBIYDGFHVB/ Code of Conduct: http://python.org/psf/codeofconduct/
