New submission from Gregory Szorc <gregory.sz...@gmail.com>:
https://bugs.python.org/issue41994 / commit 4db8988420e0a122d617df741381b0c385af032c removed _PyImport_FindExtensionObject() from the C API and it is no longer available in CPython 3.10 after alpha 5. At least py2exe and PyOxidizer rely on this API for implementing a custom module loader for extension modules. Essentially, both want to implement a custom Loader.create_module() so they can use a custom mechanism for injecting a shared library into the process. While the details shouldn't be important beyond "they can't use imp.create_dynamic()," both use a similar technique that hooks LoadLibrary() on Windows to enable them to load a DLL from memory (as opposed to a file). While I don't have the extension module loading mechanism fully paged in to my head at the moment, I believe the reason that _PyImport_FindExtensionObject() (now effectively import_find_extension()) is important for py2exe and PyOxidizer is because they need to support at most once initialization, including honoring the multi-phase initialization semantics. Since the state of extension modules is stored in `static PyObject *extensions` and the thread state (which are opaque to the C API), the loss of _PyImport_FindExtensionObject() means there is no way to check for and use an existing extension module module object from the bowels of the importer machinery. And I think this means it isn't possible to implement well-behaved alternate extension module loaders any more. I'm aware the deleted API was "private" and probably shouldn't have been used in the first place. And what py2exe and PyOxidizer are doing here is rather unorthodox. In my mind the simplest path forward is restoring _PyImport_FindExtensionObject(). But a properly designed public API is probably a better solution. Until 3.10 makes equivalent functionality available or another workaround is supported, PyOxidizer won't be able to support loading extension modules from memory on Windows on Python 3.10. This is unfortunate. But probably not a deal breaker and I can probably go several months shipping PyOxidizer with this regression without too many complaints. ---------- components: C API messages: 402754 nosy: indygreg, petr.viktorin, serhiy.storchaka, vstinner priority: normal severity: normal status: open title: Removal of _PyImport_FindExtensionObject() in 3.10 limits custom extension module loaders type: enhancement versions: Python 3.10 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue45307> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
