[issue21235] importlib's spec module create algorithm is not exposed

[Brett Cannon]

Brett Cannon added the comment:

Due to lack of response, I'm assuming all issues are now addressed.

--
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Brett Cannon]

Brett Cannon added the comment:

Is there anything left in this issue that hasn't been addressed in Python 3.5?

--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Nikolay Bryskin]

Changes by Nikolay Bryskin :


--
nosy: +nikicat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Brett Cannon]

Brett Cannon added the comment:

I'm not torn so let that settle your torment. =) Considering we are talking 
about the standard library for a language that has a mantra of "explicit is 
better than implicit" I think worrying about an added line to very little code 
since so few people muck with this stuff is enough to not worry about it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Eric Snow]

Eric Snow added the comment:

I'm just considering current usage:

  mod = loader.load_module(name)

becomes:

  spec = spec_from_loader(name, loader)
  mod = load(spec)

vs.

  mod = load(name, loader=loader)

I guess I'm torn.  I like how the former forces you to consider specs when 
dealing with the import system.  On the other hand, I don't want it to be 
annoying to have to move to this brave new world (that's exactly why we toned 
down the deprecations).  And on the third hand, perhaps it would be annoying to 
just a handful of people so we shouldn't sweat it.

So if we end up simplifying, load() could look like this:

def load(spec):  # or "load_from_spec"
return _SpecMethods(spec).load()

It already takes care of everything we need, including the import lock stuff.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Brett Cannon]

Brett Cannon added the comment:

Opened issue #21581 to discuss Armin's point about 
importlib.abc.Loader.create_module() being there but not being much use since 
the method is entirely optional.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Brett Cannon]

Changes by Brett Cannon :


--
dependencies: +Consider dropping importlib.abc.Loader.create_module()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Brett Cannon]

Brett Cannon added the comment:

I think that's the wrong abstraction(it would be fine in a third-party library, 
though, that's trying to smooth over 3.3->3.4 transitions). Since 
importlib.util.find_spec() always returns a spec, then you want something more 
like::

  def load(spec):
loader = spec.loader
if hasattr(loader, 'exec_module'):
module = importlib.util.whatever_issue_20383_leads_to()
loader.exec_module(module)
return module
else:
loader.load_module(spec.name)
return sys.modules[spec.name]

Since this is Python 3.5 code we are talking about you don't have to worry 
about the find_loader/find_module case as find_spec wraps both of those and 
normalizes it all to just using specs. You could even tweak it to pass the 
module in explicitly and in the load_module branch make sure that the module is 
placed in sys.modules first so it essentially becomes a reload (but as both 
know that's not exactly the same nor pleasant in certain cases so probably best 
not exposed that way =).

If this exists in importlib.util then you make import_module() be 
(essentially)::

  def import_module(name, package):
fullname = resolve_name(name, package)
spec = find_spec(fullname)
return load(spec)

--
dependencies: +Add a keyword-only spec argument to types.ModuleType

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Eric Snow]

Eric Snow added the comment:

How about this replacement for direct use of Loader.load_module():

# in importlib.util
def load(spec_or_name, /, **kwargs):  # or "load_from_spec"
if isinstance(spec_or_name, str):
name = spec_or_name
if not kwargs:
raise TypeError('missing loader')
spec = spec_from_loader(name, **kwargs)
else:
if kwargs:
raise TypeError('got unexpected keyword arguments')
spec = spec_or_name
return _SpecMethods(spec).load()

(See a similar proposal for new_module() in msg219135, issue #20383).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Brett Cannon]

Brett Cannon added the comment:

Issue #20383 is tracking adding an API to simply getting the proper module and 
having it be initialized.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Éric Araujo]

Changes by Éric Araujo :


--
nosy: +eric.araujo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Tshepang Lekhonkhobe]

Changes by Tshepang Lekhonkhobe :


--
nosy: +tshepang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Florent Xicluna]

Changes by Florent Xicluna :


--
nosy: +flox
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Eric Snow]

Eric Snow added the comment:

I've opened up #21240 to address the the docs concern.  Thanks for bringing it 
up. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Yury Selivanov]

Changes by Yury Selivanov :


--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Armin Ronacher]

Armin Ronacher added the comment:

Also mostly unrelated importlib now does something I have never seen an ABC do: 
the ABC has create_module but concrete implementations mostly have that 
function entirely absent.  That should probably be reconsidered as it's super 
confusing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Armin Ronacher]

Armin Ronacher added the comment:

I'm not sure myself what I need right now.  I personally have avoided 
importlib/imp entirely for my code and I roll with manual module creation 
because it is most stable between 2.6 - 3.4 but it's getting more complicated 
to work because of all the new attributes (__package__, __spec__ etc.).

This particular case came from SQLAlchemy I believe (I tried to help Mike Bayer 
transition his code).

It's true that create() is the wrong function, load() is actually what should 
have been used there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Brett Cannon]

Brett Cannon added the comment:

Are you after just the module creation/initialization code so you can call 
exec_module() yourself, Armin, or do you want even more of the algorithm 
exposed (e.g. the lock stuff)? There are not tons of importlib users to the 
level of wanting to re-implement import to the level of wanting to control 
module creation, setting sys.modules manually, etc., so we are constantly 
trying to figure out how far to go in exposing things without going so far as 
to make it impossible to make changes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Eric Snow]

Eric Snow added the comment:

I agree that this is something we need to address in 3.5.  Adding this to 3.4 
won't be an option since it would require a new feature.  However, 
Loader.load_module() is only deprecated (and won't be removed in 3.X), so the 
current approach will still work until we provide a new approach in 3.5.  The 
only gap is that now it is possible (even if very unlikely) that in 3.4 you 
would run into a loader that does not implement load_module(), which would 
obviously break direct calls to the method. :(

For 3.4 such direct calls in the stdlib to loader.load_module() were replaced 
with this (mostly):

  spec = importlib.util.spec_from_loader(name, loader)
  # or importlib.util.spec_from_file_location(...)
  methods = _SpecMethods(spec)
  mod = methods.load()

As you already noted, this relies on a current implementation detail of 
importlib._bootstrap.  We'd rather not encourage use of such private API so 
providing a simple replacement makes sense.

Futhermore, for 3.5 (in the "soon" timeframe) I'm planning on working on 
improving the import system abstractions*.  My expectation is that part of that 
will be exposing most or all of the functionality of _SpecMethods directly.  At 
the same time I don't want that API to be a distraction.  I think accomplishing 
both is doable.

So you shouldn't need to worry about create() or anything else.

* Suggestions welcome. :)  You can email me directly 
(ericsnowcurren...@gmail.com).

--
components: +Library (Lib)
nosy: +brett.cannon, eric.snow, ncoghlan
versions: +Python 3.5 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Armin Ronacher]

Armin Ronacher added the comment:

On further investigation that is not even enough yet due to the new locking 
mechanism.  I'm not even sure if exposing _SpecMethods would be enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21235] importlib's spec module create algorithm is not exposed

[Armin Ronacher]

New submission from Armin Ronacher:

3.4 deprecates load_module on the loaders and now proposes to use create_module 
(optionally) and exec_module.  Unfortunately for external callers these 
interfaces are not useful because you need to reimplement _SpecMethods.create 
and a whole bunch of other stuff for it.

Right now a caller can get hold of _SpecMethods by importing from 
_frozen_importlib but that seems very unsupported and is obviously entirely not 
exposed.

Is there a reason those methods are not on the ModuleSpec directly but on a 
private wrapper?

Example usage:

from types import ModuleType
from importlib.machinery import ModuleSpec, SourceFileLoader
from _frozen_importlib import _SpecMethods

loader = SourceFileLoader('plugin_base', 'my_file.py')
spec = ModuleSpec(name=loader.name, loader=loader)
mod = _SpecMethods(spec).create()

In the absence of _SpecMethods a caller needs to do this:

from importlib.machinery import ModuleSpec, SourceFileLoader

loader = SourceFileLoader('plugin_base', 'my_file.py')
spec = ModuleSpec(name=loader.name, loader=loader)
if not hasattr(loader, 'create_module'):
module = types.ModuleType(loader.name)
else:
module = loader.create_module(spec)
module.__loader__ = loader
module.__spec__ = spec
try:
module.__package__ = spec.parent
except AttributeError:
pass
loader.exec_module(module)

(And that is not the whole logic yet)

--
keywords: 3.4regression
messages: 216295
nosy: aronacher
priority: normal
severity: normal
status: open
title: importlib's spec module create algorithm is not exposed
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com