[issue31555] Windows pyd slower when not loaded via load_dynamic

[Steve Dower]

Steve Dower  added the comment:

I don't really care, since you're comparing multiple unsupported technologies, 
but my guess is that the egg install injects the library much earlier on 
sys.path, which avoids most of the relatively expensive file system scan.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Safihre]

Safihre  added the comment:

If you know the problem, would you also know the solution?
What is the difference in how they generate sys.path entries and how does this 
affect performance during execution of the module function?
Just want to understand what is going on :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Steve Dower]

Steve Dower  added the comment:

The difference is on startup in how they generate sys.path entries.

Skip the install step and directly run the library in a clean virtual 
environment, so that no .pth or .egg files are in use. If the difference still 
occurs, show that it also occurs on a recent version of Python as well and we 
can look at it

I promise you, there is only one way that extension modules get loaded. The 
performance difference is not because of anything in core.

--
resolution:  -> third party
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Safihre]

Safihre  added the comment:

Very good question!

5000 times via imp.load_dynamic:
 yEnc C New  took  5870 ms
 yEnc C New  took  5878 ms
 yEnc C New  took  5835 ms
5000 times via "pip: having the .pyd in site-packages"
 yEnc C New  took  6489 ms
 yEnc C New  took  6369 ms
 yEnc C New  took  6390 ms

Difference ~450 ms

1 times via imp.load_dynamic:
 yEnc C New  took  11775 ms
 yEnc C New  took  11720 ms
 yEnc C New  took  11695 ms

1 times via "pip: having the .pyd in site-packages"
 yEnc C New  took  12338 ms
 yEnc C New  took  12281 ms
 yEnc C New  took  12345 ms

Difference ~500 ms

1 times via imp.load_dynamic:
 yEnc C New  took  23431 ms
 yEnc C New  took  23406 ms
 yEnc C New  took  23283 ms

1 times via "pip: having the .pyd in site-packages"
 yEnc C New  took  24401 ms
 yEnc C New  took  24177 ms
 yEnc C New  took  24482 ms

Difference ~1100 ms

So not very linearly scaling but still increasing. Odd.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Stefan Krah]

Stefan Krah  added the comment:

Does the difference stay at 10-15% if you run it 5 times or is it
a one time cost of around 100 ms?

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Safihre]

Change by Safihre :


--
resolution: third party -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Safihre]

Safihre  added the comment:

No, the difference is not wheel vs setuptools. They both generate the identical 
pyd file. 
The difference is python: if the module is loaded by just being available on 
the path or if the module is loaded via  imp.load_dynamic

I understand if it's closed because it's not python 3, but it's not external 
tool related.

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Steve Dower]

Steve Dower  added the comment:

The only difference between your two tests is the install tool, and neither 
setuptools nor wheel bugs are tracked here.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Terry J. Reedy]

Terry J. Reedy  added the comment:

This issue is more likely to get attention if there is a difference in 3.6, or 
even better, 3.7.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[S Safihre]

New submission from S Safihre:

I have a Python 2.7 C-extension that was made for performance, however, I 
noticed something strange on Windows:
When I run locally python setup.py install and it installs it the "egg" way, 
the performance of calling the function in the module 500x in a loop is:

 yEnc C New  took   579 ms
 yEnc C New  took   580 ms
 yEnc C New  took   580 ms

But when I install it as a wheel locally or via pip without a single change to 
the C-code, this is the result:

 yEnc C New  took   702 ms
 yEnc C New  took   694 ms
 yEnc C New  took   691 ms

That's a 10-15% difference. I also checked macOS and Ubuntu, here it does not 
seem to be the case.
By investigating (https://github.com/pypa/setuptools/issues/1154) I found that 
the difference is only that the "egg" way the module gets loaded via 
imp.load_dynamic 

I cannot test under Python 3, since the module is (not yet) converted to Python 
3-style.


To reproduce run and look at the yEnc C New score between the 2 ways (on 
Windows only of course):

git clone https://github.com/sabnzbd/sabyenc.git
cd sabyenc
python setup.py install
python .\tests\speed_compare.py; python .\tests\speed_compare.py;python 
.\tests\speed_compare.py;
pip uninstall sabyenc -y

pip install sabyenc
# Or this:
# python setup.py install bdist_wheel
# pip install .\dist\sabyenc-3.3.1-cp27-cp27m-win_amd64.whl
python .\tests\speed_compare.py; python .\tests\speed_compare.py;python 
.\tests\speed_compare.py;

--
components: Extension Modules, Windows
messages: 302768
nosy: S Safihre, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows pyd slower when not loaded via load_dynamic
type: performance
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com