On 10.11.2016 01:02, Steve D'Aprano wrote:
On Thu, 10 Nov 2016 08:08 am, Wolfgang Maier wrote:
Hi,
I just used the stdlib's modulefinder.ModuleFinder (intended to find
modules used by a script) for the first time in my life and it just
doesn't seem to work like documented at all.
Not sure what is going on, but if I try the usage example from
https://docs.python.org/3/library/modulefinder.html
it's reporting every single module from the stdlib whether imported or
not!
I see what you mean, but it's not quite *every* module. After I run the
example from the docs, I get 197 modules, and here's at least two that
aren't included:
py> len(finder.modules)
197
py> 'statistics' in finder.modules
False
py> 'cmath' in finder.modules
False
Curiously, it includes modules that aren't cached in sys.modules:
py> len(finder.modules.keys() - sys.modules.keys())
107
So I'm not sure how that's possible.
According to the example module, it imports re and itertools. Both of those
have already been imported, and so will be cached in sys.modules, as will
all their dependencies. So I don't see how it is possible that ModuleFinder
can find dependencies that aren't cached.
Theoretically, of course some dependency might import a bunch of modules,
then delete them from sys.modules. If it were only one or two, I'd believe
that. But not 107 of them.
After running the module finder on the sample file, I ran the report()
method to get a nicer display of the imported modules:
py> finder = ModuleFinder()
py> finder.run_script('/tmp/bacon.py')
py> finder.report()
Name File
---- ----
m __future__ /usr/local/lib/python3.5/__future__.py
m __main__ /tmp/bacon.py
m _ast
m _bootlocale /usr/local/lib/python3.5/_bootlocale.py
m _bz2
/usr/local/lib/python3.5/lib-dynload/_bz2.cpython-35m-i386-linux-gnu.so
[...]
which shows the unittest package being loaded, which is pretty dubious.
On the other hand, here's a simpler example which seems to work fine:
py> with open('/tmp/do_little.py', 'w') as f:
... f.write('import math\n')
...
12
py> finder = ModuleFinder()
py> finder.run_script('/tmp/do_little.py')
py> finder.report()
Name File
---- ----
m __main__ /tmp/do_little.py
m math
/usr/local/lib/python3.5/lib-dynload/math.cpython-35m-i386-linux-gnu.so
So I'm not really sure what's going on.
Thanks for the detailed analysis.
You are right that it seems to work with certain imports. I verified
your example with import math and found a few additional examples.
Things seem to work with imports of any of the following (alone or in
combination):
io, itertools, math, sys
OTOH, I doubt that it reports correctly with:
os, heapq, operator, re
Personally, I fail to see a pattern here, but maybe somebody else can
explain that behaviour.
I'd really like to report this to the bug tracker, but at the moment I'm
not sure as what exactly. It seems that at least the documentation is
incorrect since the usage example there is not working.
Best,
Wolfgang
--
https://mail.python.org/mailman/listinfo/python-list
