Re: [Python-Dev] importlib quest
On Mon, Feb 6, 2012 at 14:49, Antoine Pitrou wrote: > On Mon, 6 Feb 2012 09:57:56 -0500 > Brett Cannon wrote: > > Thanks for any help people can provide me on this now 5 year quest to get > > this work finished. > > Do you have any plan to solve the performance issue? > I have not even looked at performance or attempted to profile the code, so I suspect there is room for improvement. > > $ ./python -m timeit -s "import sys; mod='struct'" \ > "__import__(mod); del sys.modules[mod]" > 1 loops, best of 3: 75.3 usec per loop > $ ./python -m timeit -s "import sys; mod='struct'; from importlib import > __import__" \ > "__import__(mod); del sys.modules[mod]" > 1000 loops, best of 3: 421 usec per loop > > Startup time is already much worse in 3.3 than in 2.7. With such a > slowdown in importing fresh modules, applications using many batteries > (third-party or not) will be heavily impacted. > I have a benchmark suite for importing modules directly at importlib.test.benchmark, but it doesn't explicitly cover searching far down sys.path. I will see if any of the existing tests implicitly do that and if not add it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] importlib quest
On Mon, 6 Feb 2012 20:49:48 +0100
Antoine Pitrou wrote:
> On Mon, 6 Feb 2012 09:57:56 -0500
> Brett Cannon wrote:
> > Thanks for any help people can provide me on this now 5 year quest to get
> > this work finished.
>
> Do you have any plan to solve the performance issue?
>
> $ ./python -m timeit -s "import sys; mod='struct'" \
> "__import__(mod); del sys.modules[mod]"
> 1 loops, best of 3: 75.3 usec per loop
> $ ./python -m timeit -s "import sys; mod='struct'; from importlib import
> __import__" \
> "__import__(mod); del sys.modules[mod]"
> 1000 loops, best of 3: 421 usec per loop
The culprit for the overhead is likely to be PathFinder.find_module:
$ ./python -m timeit -s "import sys; mod='struct'; from importlib._bootstrap
import _DefaultPathFinder; finder=_DefaultPathFinder"
"finder.find_module('struct')"
1000 loops, best of 3: 355 usec per loop
$ ./python -S -m timeit -s "import sys; mod='struct'; from importlib._bootstrap
import _DefaultPathFinder; finder=_DefaultPathFinder"
"finder.find_module('struct')"
1 loops, best of 3: 176 usec per loop
Note how it's dependent on sys.path length. On an installed Python with
many additional sys.path entries (e.g. because of distribute-based
module installs), import times will be much worse.
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] importlib quest
On Mon, 6 Feb 2012 09:57:56 -0500 Brett Cannon wrote: > Thanks for any help people can provide me on this now 5 year quest to get > this work finished. Do you have any plan to solve the performance issue? $ ./python -m timeit -s "import sys; mod='struct'" \ "__import__(mod); del sys.modules[mod]" 1 loops, best of 3: 75.3 usec per loop $ ./python -m timeit -s "import sys; mod='struct'; from importlib import __import__" \ "__import__(mod); del sys.modules[mod]" 1000 loops, best of 3: 421 usec per loop Startup time is already much worse in 3.3 than in 2.7. With such a slowdown in importing fresh modules, applications using many batteries (third-party or not) will be heavily impacted. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
