[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Oct 20, 2021, at 11:27, Brett Cannon  wrote:
> 
>> That should probably be a separate issue/PR in either case.
> 
> https://bugs.python.org/issue38782 and I was trying to rope you into doing 
> the work. 

Ha!  You should have nosied me then :D - but anyway I’m following that ticket 
now so we’ll see.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:


New changeset 876fc7fcec9a79a11546b7588d3683a5ccb4d31c by Barry Warsaw in 
branch 'main':
bpo-35673: Add a public alias for namespace package __loader__ attribute 
(#29049)
https://github.com/python/cpython/commit/876fc7fcec9a79a11546b7588d3683a5ccb4d31c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Eric Snow]

Eric Snow  added the comment:

On Mon, Jan 7, 2019 at 11:41 PM Eric V. Smith  wrote:
> Namespace packages (PEP 420) predate ModuleSpec (PEP 451). So, I think this 
> probably happened when 451 was implemented. Maybe Eric Snow recalls?

PEP 451 talks about this a little
(https://www.python.org/dev/peps/pep-0451/#namespace-packages):

"""
Currently a path entry finder may return (None, portions) from
find_loader() to indicate it found part
of a possible namespace package. To achieve the same effect,
find_spec() must return a spec with
"loader" set to None (a.k.a. not set) and with
submodule_search_locations set to the same portions
as would have been provided by find_loader(). It's up to PathFinder
how to handle such specs.
"""

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Brett Cannon]

Brett Cannon  added the comment:

> What benefit would be gained?

The ABCs are broader than what the import system actually requires due to their 
helper methods. So for typing purposes they are actually not a perfect fit.

> That should probably be a separate issue/PR in either case.

https://bugs.python.org/issue38782 and I was trying to rope you into doing the 
work. 

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

I don't know.  What benefit would be gained?  That should probably be a 
separate issue/PR in either case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Brett Cannon]

Brett Cannon  added the comment:

Should we register with the ABC or is it time to do proper typing.Protocol 
classes and have the ABCs inherit from those?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Since the documentation problem reported here has since been fixed, and really 
all that's left is to expose NamespaceLoader publicly and register it with the 
abc, this is technically a new feature so it can't be backported.  Thus, 
targeting only 3.11.

--
versions:  -Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

First crack at a PR for this issue.

--
keywords: +needs review -patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
keywords: +patch
pull_requests: +27320
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29049

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 3.6, Python 3.7, Python 
3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

I'm going to take a look at this during the Python core sprint.

--
assignee:  -> barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Filipe Laíns]

Change by Filipe Laíns :


--
nosy: +FFY00

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Eric V. Smith]

Eric V. Smith  added the comment:

I think a PR with tests would be a good first step.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Isaac]

Isaac  added the comment:

Not sure if it's proper etiquette to bump issues on the tracker, but is there 
any interest in this issue for 3.11?

--
nosy: +fwahhab

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Eric V. Smith]

Eric V. Smith  added the comment:

I think exposing _NamespaceLoader as NamespaceLoader and registering the ABC 
make sense. That would make this in to a feature request for 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The background for all of this: I'm currently rewriting modulegraph 
(https://pypi.org/project/modulegraph/) to use importlib instead of its own 
implementation of the import mechanism.

I currently detect PEP420 style namespace packages, but I'm not sure if I 
really need that information.  With the current behaviour of the namespace 
loader I probably do, because I'd otherwise end up with creating an __init__.py 
for these packages when I use the generated module graph in py2app to copy 
modules into an application bundle.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Eric V. Smith]

Eric V. Smith  added the comment:

Namespace packages (PEP 420) predate ModuleSpec (PEP 451). So, I think this 
probably happened when 451 was implemented. Maybe Eric Snow recalls?

I say this without having looked at it very deeply.

As to why the namespace package loader is a private class: it never occurred to 
me someone would care about inspecting it.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Jan 7, 2019, at 03:16, Ronald Oussoren  wrote:
> 
> Do you know why the namespace package loader lies about the source and code? 
> Both .get_source() and .get_code() return a value that isn't None.

> And likewise: Why is the namespace package loader a private class, other 
> loaders are exposed in importlib.machinery?  This makes it hard to detect 
> PEP420 style namespace packages without relying on private APIs, esp. 
> combined with the behaviour of .get_source() and .get_code().

I don’t remember the history of this.  I wonder if Brett or Eric have any 
additional insights.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Ronald Oussoren]

Ronald Oussoren  added the comment:

@barry: I agree on both.

Do you know why the namespace package loader lies about the source and code? 
Both .get_source() and .get_code() return a value that isn't None.

And likewise: Why is the namespace package loader a private class, other 
loaders are exposed in importlib.machinery?  This makes it hard to detect 
PEP420 style namespace packages without relying on private APIs, esp. combined 
with the behaviour of .get_source() and .get_code().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On the first point, I'd categorize this as a documentation bug, and in fact, 
it's inconsistent with the language reference, which doesn't have the same 
language:

https://docs.python.org/3/reference/import.html#__loader__

On the second point, it probably does make sense to register the ABC.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35673] Loader for namespace packages

[Ronald Oussoren]

New submission from Ronald Oussoren :

The documentation for import lib.machinery.ModuleSpec says that the attribute 
"loader" should be None for namespace packages (see 
)

In reality the loader for namespace packages is an instance of a private class, 
and that class does not conform to the importlib.abc.Loader ABC.

To reproduce:

* Create and empty directory "namespace"
* (Optionally) create an empty "module.py" in that directory
* Start a python shell and follow along:

Python 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import namespace
>>> namespace.__loader__
<_frozen_importlib_external._NamespaceLoader object at 0x104c7bdd8>
>>> import importlib.abc
>>> isinstance(namespace.__loader__, importlib.abc.Loader)
False
>>> import importlib.util
>>> importlib.util.find_spec('namespace')
ModuleSpec(name='namespace', 
loader=<_frozen_importlib_external._NamespaceLoader object at 0x104c7bdd8>, 
submodule_search_locations=_NamespacePath(['/Users/ronald/Projects/pyobjc-hg/modulegraph2/namespace']))

Note how "namespace" has an attribute named "__loader__" that is not None, and 
the same is true for the ModuleSpec found using importlib.util.find_spec.  The 
loader does not claim to conform to any Loader ABC (but provides all methods 
required for conformance to the InspectLoader ABC)

I'm not sure if this should be two issues:

1) Documentation doesn't match behaviour

2) The loader for namespace packages isn't registered with the relevant ABCs

P.S. the latter is also true for zipimport.zipimporter.

--
components: Library (Lib)
messages: 333111
nosy: brett.cannon, ronaldoussoren
priority: normal
severity: normal
status: open
title: Loader for namespace packages
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com