[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: Yeah, I'm pretty sure that TODO is out of date. -- ___ Python tracker ___ ___

[issue19944] Make importlib.find_spec load packages as needed

Terry J. Reedy added the comment: pyclbr.py has this comment # XXX This will change once issue19944 lands. above the line that was changed by the patch applied. Am I correct in thinking that the comment is obsolete and should be removed? -- nosy: +terry.reedy

[issue19944] Make importlib.find_spec load packages as needed

Roundup Robot added the comment: New changeset 665f1ba77b57 by Eric Snow in branch 'default': Issue 19944: Fix importlib.find_spec() so it imports parents as needed. http://hg.python.org/cpython/rev/665f1ba77b57 -- nosy: +python-dev ___ Python

[issue19944] Make importlib.find_spec load packages as needed

Changes by Eric Snow ericsnowcurren...@gmail.com: -- resolution: - fixed stage: - committed/rejected status: open - pending type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944

[issue19944] Make importlib.find_spec load packages as needed

Changes by Eric Snow ericsnowcurren...@gmail.com: -- status: pending - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944 ___ ___

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: Here's an update to the patch. It includes doc and test changes. It also moves find_spec() into importlib.util. (I'm removing the other patches since I don't consider them valid approaches any longer). -- Added file:

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: (...and to reduce any confusion on which patch is under consideration.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944 ___

[issue19944] Make importlib.find_spec load packages as needed

Changes by Eric Snow ericsnowcurren...@gmail.com: Removed file: http://bugs.python.org/file33112/issue19944-find-spec-mirror-import-module-simple.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944

[issue19944] Make importlib.find_spec load packages as needed

Changes by Eric Snow ericsnowcurren...@gmail.com: Removed file: http://bugs.python.org/file33111/issue19944-find-spec-mirror-import-module.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944

[issue19944] Make importlib.find_spec load packages as needed

Changes by Eric Snow ericsnowcurren...@gmail.com: Removed file: http://bugs.python.org/file33087/issue19944-find-spec-parent-module.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944 ___

[issue19944] Make importlib.find_spec load packages as needed

Changes by Eric Snow ericsnowcurren...@gmail.com: Removed file: http://bugs.python.org/file33299/issue19944-find-spec-mirror-import-module-direct.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: If nothing else comes up, this will be the last PEP-451 functional change for 3.4. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944 ___

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: Brett: What do you think about moving importlib.find_spec() to importlib.util? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944 ___

[issue19944] Make importlib.find_spec load packages as needed

Brett Cannon added the comment: Moving to importlib.util is fine by me. I put find_loader() to mirror import_module(), but honestly the former is much less used compared to the latter, so moving it to importlib.util makes sense. -- ___ Python

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: find_spec() is at package level because find_module() is and for no other good reason I'm aware of. I'd be just fine with moving it to util. I don't expect it to be used enough to warrant that top-level placement. Regarding builtins.__import__(), I'm using it in

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: Good points, Nick. I was trying to limit touches on _bootstrap.py for 3.4, but that simple patch is mostly just a hacky band-aid. Here's a patch that hopefully stands on its own and still limits touches. (The patch is the bare-bones changes only.) There are 2

[issue19944] Make importlib.find_spec load packages as needed

Nick Coghlan added the comment: Actually, why *is* find_spec at package level, rather than in util with resolve_name? I know we said it was at package level in the PEP, but we never really gave the question serious thought. Also, why use builtins.__import__ rather than using __import__

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: Any thoughts on the latest patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944 ___ ___ Python-bugs-list

[issue19944] Make importlib.find_spec load packages as needed

Nick Coghlan added the comment: The simple patch actually looks like a good way to end up with find_spec specific bugs because it diverges from the more thoroughly tested main import path (e.g. it looks to me like it doesn't release the import lock properly) So the _FoundSpec version actually

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: I've closed #16492 in favor of this ticket. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944 ___ ___

[issue19944] Make importlib.find_spec load packages as needed

Nick Coghlan added the comment: Adding a dependency on issue 19700, as I'm about to commit that fix and runpy should be updated to use whatever we come up with here. -- dependencies: +Update runpy for PEP 451 ___ Python tracker

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: Nick: that sounds good to me. I like the idea of find_spec() being the same as import_module(), minus actually loading the module. In that spirit, here's a rough patch that accomplishes that. It refactors things a bit to get there. Considering where we are in

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: Here's a version that I'd feel more comfortable with for 3.4 (with the intent of refactoring in 3.5). -- Added file: http://bugs.python.org/file33112/issue19944-find-spec-mirror-import-module-simple.diff ___ Python

[issue19944] Make importlib.find_spec load packages as needed

Nick Coghlan added the comment: One function (the current one) promises not to import anything. The other (the new one) may have side effects, even if it fails to find the module (any successfully imported parent modules will remain imported). IIRC, that no side effects guarantee was the

[issue19944] Make importlib.find_spec load packages as needed

Brett Cannon added the comment: I'm not saying get rid of the ability to guarantee no side-effects. All I'm saying is that the point of the function is to find a spec, whether that includes importing or implicitly using a parent package from sys.modules is just technical detail (and thus

[issue19944] Make importlib.find_spec load packages as needed

Nick Coghlan added the comment: Look at it from a conceptual point of view, though, there are two quite different operations: - find a spec at the current level, using the specified path (which may be empty, implying a top level import) - find a spec according to the full import cycle,

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: It looks like there are two concepts at play though: 1. side-effect-free vs. may-have-side-effects 2. just-find-the-spec-dangit vs. find-the-spec-relative-to-submodule-search-locations-I-already-know In the case of #1, providing the path is just a means to an end.

[issue19944] Make importlib.find_spec load packages as needed

Nick Coghlan added the comment: Actually, I think you make a good point. importlib.find_spec should use the *import_module* signature (no path parameter, but with support for relative imports). If it is exposed at all, the one level only single step version should appear in importlib.machinery,

[issue19944] Make importlib.find_spec load packages as needed

New submission from Nick Coghlan: In implementing the runpy module updates for PEP 451 I ran into the fact that importlib.find_spec had copied the find_loader behaviour where it doesn't handle dotted names automatically - you have to explicitly pass in the __path__ from the parent module for

[issue19944] Make importlib.find_spec load packages as needed

Brett Cannon added the comment: Another option would be to add a keyword-only use_parent_path flag to importlib.find_spec() which is True by default. If use_parent is True, 'path' is provided, and there is no parent, then 'path' can act as a fallback. Which makes me wonder if also adding a

[issue19944] Make importlib.find_spec load packages as needed

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19944 ___

[issue19944] Make importlib.find_spec load packages as needed

Nick Coghlan added the comment: They're fundamentally different operations though, with one being a single step of the other. I just think do the full lookup should have the more obvious name, with the do a single level lookup still as a public API, but with the less obvious name. --

[issue19944] Make importlib.find_spec load packages as needed

Eric Snow added the comment: As a testament to counter-intuitive API, I did not even realize this about either find_loader() or find_spec() until the bug the other day with reloading submodules. So I'm on board! As to the best approach, I'll argue for keeping just 1 function. I tried out