New submission from Daniel Shaulov: Hi,
I was trying to build a python interpreter that has the cpp part of msgpack as a built-in module. I thought that I could just copy the msgpack folder to the Modules folder and add this 2 lines to Modules/Setup.local: msgpack._packer msgpack/_packer.cpp msgpack._unpacker msgpack/_unpacker.cpp I had a few obstacles, the attached patch fixes them all. The first - makesetup has a list of regexes to match and it has *.* after going through all known extensions to throw a "bad word" error. I removed the check. All those things will now be assumed to be a module with a package. Now the actual init function name is PyInit__packer and doesn't have msgpack in it, so I also added code in makesetup to use the full name as the name and only the module name for the PyInit function. The second is that in Lib/importlib/_bootsrap.py in BuiltinImporter.find_spec there is a specific case to ignore modules that are part of a package. Is there a reason to forbid it? I removed that check. There were also unit tests that checked this behavior which I deleted. I added tests that check module in package instead of them. Changing _bootsrap.py also changes Python/importlib.h (it is the frozen importlib), I added a separate patch with that change, to not clutter the main patch. The third - the __name__ didn't have the package prefix. Digging around I found a comment in PyModule_Create2 that says that the shared library loader stores the full name _Py_PackageContext before loading the module, so I did the same in _imp_create_builtin that is done in _PyImport_LoadDynamicModuleWithSpec to set _Py_PackageContext and the __name__ was fixed too. (If anyone tries to do this with msgpack and wants to see that it works - you also need to copy the msgpack directory to the Lib directory, and in __init__.py, where it catches the import error and goes to the fallback, just reraise the exception instead of letting it go to the fallback) Do note that this does not allow for built-in packages, only build-it module in package. If we want to allow built-in packages, we will need to decide on some syntax to distinguish it in the Setup files and some way to distinguish them in PyImport_Inittab (for example - an asterix before the name of the package?) Thanks, Daniel. ---------- components: Build, Interpreter Core files: builtin_package.patch keywords: patch messages: 262495 nosy: Daniel Shaulov, brett.cannon, eric.snow, ncoghlan priority: normal severity: normal status: open title: Allow built-in module in package type: enhancement versions: Python 3.6 Added file: http://bugs.python.org/file42297/builtin_package.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26646> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
