[issue32312] Create Py_AtExitRegister C API

[STINNER Victor]

Change by STINNER Victor :


--
components: +C API

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[STINNER Victor]

STINNER Victor  added the comment:

Joannah Nanjekye asked me to have a look at this issue.

The existing Py_AtExit() C function is very low-level: it's called very late in 
Py_Finalize(), when the Python runtime is already destroyed.

I understand that the need here is to call C code which would still like to 
access the Python C API.

I propose the following C API:

   void Py_AtExitRegister(PyObject* (*func) (void *data), void *data)

which would call func(data).

The return type is void since I don't see any practical way to log an useful 
warning if a function fails: we only gets its memory address, we cannot log 
that. The callback function ('func') should be responsible to log errors itself 
(ex: using PySys_WriteStderr which writes into sys.stderr).


Neil:
> It would be handy to have a C API that registered an atexit function, similar 
> to what calling atexit.register does.  This API could be used by C extension 
> modules to register atexit functions.


The PyModuleDef already has 2 slots to call code at Python exit:
  void m_clear(PyObject *module)
and:
  void m_free(void *data).

But these callbacks can be late in Py_Finalize(), while Python runtime is being 
destroyed: when all modules are unloaded.

atexit.register() callbacks are different: they are called "early" in 
Py_Finalize(), when Python is still fully working.

I guess that Py_AtExitRegister() callbacks should also be called when Python is 
still fully working, right?

Apart C extensions, Py_AtExitRegister() also sounds interesting when Python is 
embedded in an application. For example, if you call Py_Main() or Py_RunMain(), 
you cannot easily execute arbitrary C code just before Py_Finalize().



Antoine:
> Calling "atexit.register" using the C API isn't very difficult. The annoying 
> part is to wrap a simple C function pointer in a callable PyObject (I don't 
> think there is a simple C API for that).

We could store C callbacks as C function pointers. We can have 2 lists of 
functions in the atexit module: one for Python function, one for C functions 
(registered using the C API). I would suggest to call Python functions first, 
and then call C functions.

--

I changed my mind, and I now consider that adding a public function would be 
useful. Sorry, first I misunderstood the use cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This may be especially useful to make sure that extension modules that have 
threads that were not created by Python calling into Python (registering with 
the interpreter and picking up the GIL) are stopped before the interpreter 
starts shutting down to avoid callbacks in the middle of the tear-down 
procedure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[Eric Snow]

Eric Snow  added the comment:

> Neil Schemenauer  added the comment:
> Regarding m_traverse, maybe the list of finalizers should be stored somewhere 
> in the interpreter
> state, not in the atexit module state.  That would make more sense to me.  
> atexit could merely be
> a way to manage it rather than actually holding it.

+1

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[Cheryl Sabella]

Change by Cheryl Sabella :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[Neil Schemenauer]

Neil Schemenauer  added the comment:

Private is fine.  We want to get the design correct before making it part of 
the official API.  My thought is that providing a handy atexit hook would be a 
good thing in that it could be an alternative to 3rd party code using __del__ 
to do cleanup.

One downside of atexit is that there is still a question of ordering the calls 
during shutdown.  I had the idea that maybe register() should take some sort of 
argument to specify when the function should be called.  That seems like 
leading down the road of madness maybe.  It would be useful to look at what 
other languages do, e.g. Java.  My understanding is that new versions of Java 
encourage finalizers like weakref callbacks and discourage or disallow 
finalizers like __del__.  I.e. you are not allowed to see the state of the 
object being finalized and cannot resurrect it.

Regarding m_traverse, maybe the list of finalizers should be stored somewhere 
in the interpreter state, not in the atexit module state.  That would make more 
sense to me.  atexit could merely be a way to manage it rather than actually 
holding it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[STINNER Victor]

STINNER Victor  added the comment:

Oh by the way, if we decide to add a function, I would prefer to start with a 
private function. The C API is big enough. We can decide in the next version if 
the function becomes useful enough to justify to be added to the public API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Calling "atexit.register" using the C API isn't very difficult. The annoying 
part is to wrap a simple C function pointer in a callable PyObject (I don't 
think there is a simple C API for that).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[STINNER Victor]

STINNER Victor  added the comment:

See also:
[Python-Dev] PEP 489: module m_traverse called with NULL module state
https://mail.python.org/pipermail/python-dev/2017-December/151238.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +pitrou, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32312] Create Py_AtExitRegister C API

[Neil Schemenauer]

New submission from Neil Schemenauer :

It would be handy to have a C API that registered an atexit function, similar 
to what calling atexit.register does.  This API could be used by C extension 
modules to register atexit functions.

I think the implementation would be fairly simple.  We need a function that 
calls atexit_register().  The tricky bit is that we need to make sure the 
atexit module has been initialized as atexit_register uses the module state.

This API is different from Py_AtExit in that atexit.register() calls the 
function early in the interpreter shutdown whereas Py_AtExit calls functions 
very late in the shutdown.

--
messages: 308246
nosy: nascheme
priority: low
severity: normal
stage: needs patch
status: open
title: Create Py_AtExitRegister C API
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com