[issue33210] pkgutil.walk_packages gives incomplete results

[Nick Coghlan]

Nick Coghlan  added the comment:

I think this is actually two distinct problems, one documentation one (which 
should be addressed in the online docs for all currently maintained versions), 
and one actual functional issue.

The documentation issue is the one you've reported: in order for the recursive 
descent to work in walk_packages given the current algorithm, then the 
combination of the given prefix, and the current global import configuration 
must allow that package to actually be imported. While there is a note about 
that limitation, it's currently thoroughly unclear.

The functional issue is two-fold:

1. pkgutil.iter_modules() doesn't identify PEP 420 namespace packages correctly 
(it ignores them as not being potential packages)

2. The recursive import to check pkg.__path__ uses a name based global 
__import__ rather than the more state independent 
https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly
 technique

It's that second problem that introduces the "prefix must be set to get useful 
output" behaviour that you're currently seeing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33210] pkgutil.walk_packages gives incomplete results

[Cyker Way]

Cyker Way  added the comment:

Update test program.

--
Added file: https://bugs.python.org/file47519/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33210] pkgutil.walk_packages gives incomplete results

[Cyker Way]

Change by Cyker Way :


Removed file: https://bugs.python.org/file47516/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33210] pkgutil.walk_packages gives incomplete results

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33210] pkgutil.walk_packages gives incomplete results

[Ned Deily]

Change by Ned Deily :


--
nosy: +brett.cannon, eric.snow, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33210] pkgutil.walk_packages gives incomplete results

[Cyker Way]

New submission from Cyker Way :

The current implementation of `pkgutil.walk_packages()` is confusing. Users may 
be given incomplete results while no exception is raised if they don't 
explicitly provide the `prefix` parameter. The doc says:

>   prefix is a string to output on the front of every module name on output.

But the fact is, `prefix` is not merely an output formatter at all. This 
function cannot handle an empty prefix (which is the default) and will often 
encounter import errors which are then simply ignored without an `onerror` 
function (which is default again).

See test program for details.

Doc:

https://docs.python.org/3.6/library/pkgutil.html#pkgutil.walk_packages

Source:

https://github.com/python/cpython/blob/3.6/Lib/pkgutil.py

--
components: Library (Lib)
files: test.py
messages: 314850
nosy: cykerway
priority: normal
severity: normal
status: open
title: pkgutil.walk_packages gives incomplete results
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47516/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com