[issue25159] Regression in time to import a module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Yes, importing 4 additional modules can be a cause of the slowdown. Thanks 
Victor.

--
resolution:  -> works for me
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25159] Regression in time to import a module

[STINNER Victor]

STINNER Victor added the comment:

$ 3.4/python -c "import sys; sys.modules.clear(); import enum; 
print(len(sys.modules))"
18

$ 3.5/python -c "import sys; sys.modules.clear(); import enum; 
print(len(sys.modules))"
22

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25159] Regression in time to import a module

[STINNER Victor]

STINNER Victor added the comment:

Importing a module looks slow to me. Importing a module which is a an *empty* 
.py file takes ~160 us.

$ python3 -I -m timeit -s "open('slow.py', 'wb').close(); import sys; import 
slow" "del sys.modules['slow']; import slow"
1 loops, best of 3: 156 usec per loop

The difference betwen Python 3.4 and 3.5 is that enum.py imports indirectly 
more modules. The following two imports were added to types.py in Python 3.5:

import functools as _functools 
import collections.abc as _collections_abc

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25159] Regression in time to import a module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I just have rebuilt my Python's in development brances after made distclean to 
avoid possible effects of incremental building, and got the same results.

(1) for i in `seq 5`; do ./python -I -m timeit -n1 -r1 -s "import sys; 
sys.modules.clear()" -- "import enum"; done

Python 3.4: 29 msec
Python 3.5: 43.4 msec (+50%)
Python 3.6: 44.4 msec (+2%, +53%)


(2) for i in `seq 5`; do ./python -I -m timeit "import enum"; done

Python 3.4: 3.02 usec
Python 3.5: 3.14 usec (+4%)
Python 3.6: 3.34 usec (+6%, +10%)


(3) for i in `seq 5`; do ./python -I -m timeit -n1 -r1 "import enum"; done

Python 3.4: 3.49 msec
Python 3.5: 4.19 msec (+20%)
Python 3.6: 4.45 msec (+6%, +28%)


32-bit Ubuntu 14.04, Linux 3.16, gcc 4.8.4, Intel(R) Atom(TM) CPU N570   @ 
1.66GH.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25159] Regression in time to import a module

[STINNER Victor]

STINNER Victor added the comment:

I tested on Linux.

(1) for i in `seq 5`; do ./python -I -m timeit -n1 -r1 -s "import sys; 
sys.modules.clear()" -- "import enum"; done

(shortest timing)

Python 3.4: 6.93 msec
Python 3.6: 7.05 msec (+2%)


(2) for i in `seq 5`; do ./python -I -m timeit "import enum"; done

Python 3.4: 0.331 usec
Python 3.6: 0.341 usec (+%3)

These numbers are nanoseconds, it's too short to run a real benchmark.


(3) for i in `seq 5`; do ./python -I -m timeit -n1 -r1 "import enum"; done

Python 3.4: 801 usec
Python 3.6: 774 usec (-3%)


Sorry, I don't see major differences like you showed. Can you explain exactly 
how to reproduce them? Exact Python version? OS?

I used the development branches (branch 3.4 and branch default).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25159] Regression in time to import a module

[Terry J. Reedy]

Terry J. Reedy added the comment:

I did not find any significant different between 3.4 and 3.5 on Windows running 
the following:

import os, time
from importlib import import_module

files = os.listdir("C:/Programs/Python34/Lib")
excludes = {'antigravity.py', 'compileall.py', 'crypt.py', 'pty.py',
'this.py', 'tty.py', '__phello__.foo.py'}
start = time.monotonic()
for name in files:
if name.endswith('.py') and name not in excludes:
try:
import_module(name[:-3])
except:
print(name)
print(time.monotonic()-start)

Other than a couple of outliers, the time was consintly about .29 seconds with 
both 3.4 and 3.5:
  F:\Python\dev>py -3.x  c:/programs/python34/tem.py
The 3.5 run is a bit odd since it runs both 3.4 code and 3.5 code, but there 
were no extra exceptions once I added the needed exclusions (pyt and tty are 
also unix only) for 3.4.  The same code runs equally faster with Idle (.17 
seconds), but that must be because Idle has already imported so many more /Lib 
modules (not including idlelib/*) than python itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25159] Regression in time to import a module

[Terry J. Reedy]

Terry J. Reedy added the comment:

Clarify title; regression is not in 'import time'. I am working on timing the 
import of Lib/*.py on windows.  I already discovered that unix-only crypt is 
present on windows (but should not be).

--
nosy: +terry.reedy
title: Import time regression -> Regression in time to import a module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com