[Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread David Genest
Hi, I was wondering what is the recommended approach to bundling runtime dll dependencies when using wheels. We are migrating from egg to wheels for environment installation and of various python dependencies. Some of those have extension modules, and some have extension modules that depend

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread Paul Moore
On 30 September 2014 14:32, David Genest wrote: > But the only way to get a dependent dll found on windows is to have it on > PATH, and the scripts directory on > windows is on path when a virtualenv is activated. This is not true. Python loads DLLs with LOAD_WITH_ALTERED_SEARCH_PATH, to allow t

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread David Genest
> This is not true. Python loads DLLs with LOAD_WITH_ALTERED_SEARCH_PATH, to > allow them to be located alongside the pyd file. You should therefore be able > to ship the > dependent dll in the package directory (which wheels support fine). > Paul Ok, so what if the dll is shared in a given e

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread Paul Moore
On 30 September 2014 15:31, David Genest wrote: > Ok, so what if the dll is shared in a given environment (multiple extensions > use it)?, the shared dll should be copied to every package? Won't that cause > multiple loads by the system? I honestly don't know in that case, sorry. You might get

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread Nick Coghlan
On 1 October 2014 00:37, Paul Moore wrote: > On 30 September 2014 15:31, David Genest wrote: >> Ok, so what if the dll is shared in a given environment (multiple extensions >> use it)?, the shared dll should be copied to every package? Won't that >> cause multiple loads by the system? > > I ho

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread Daniel Holth
Or you could just create a Python package that only contains the dll, and depend on it from your others. On Tue, Sep 30, 2014 at 10:44 AM, Nick Coghlan wrote: > On 1 October 2014 00:37, Paul Moore wrote: >> On 30 September 2014 15:31, David Genest wrote: >>> Ok, so what if the dll is shared in

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread Paul Moore
On 30 September 2014 15:45, Daniel Holth wrote: > Or you could just create a Python package that only contains the dll, > and depend on it from your others. The problem is getting the DLL on PATH. What you could do is distribute a package containing: 1. The dll 2. An __init__.py that adds the p

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread Chris Barker
On Tue, Sep 30, 2014 at 7:45 AM, Daniel Holth wrote: > Or you could just create a Python package that only contains the dll, > and depend on it from your others. but it won't be on the dll search path. Paul Moore wrote: > What you could do is distribute a package containing: > 1. The dll > 2

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread Steve Dower
David Genest wrote: > Subject: Re: [Distutils] Wheels and dependent third party dlls on windows > > >> This is not true. Python loads DLLs with >> LOAD_WITH_ALTERED_SEARCH_PATH, to allow them to be located alongside the pyd > file. You should therefore be able to ship

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread David Cournapeau
On Tue, Sep 30, 2014 at 3:31 PM, David Genest wrote: > > > This is not true. Python loads DLLs with LOAD_WITH_ALTERED_SEARCH_PATH, > to allow them to be located alongside the pyd file. You should therefore be > able to ship the > > dependent dll in the package directory (which wheels support fine

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-09-30 Thread David Cournapeau
On Tue, Sep 30, 2014 at 3:44 PM, Nick Coghlan wrote: > On 1 October 2014 00:37, Paul Moore wrote: > > On 30 September 2014 15:31, David Genest > wrote: > >> Ok, so what if the dll is shared in a given environment (multiple > extensions use it)?, the shared dll should be copied to every package

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread David Genest
Thank you all for the precious info. Here are my observations: - We are merely writing extension modules with third party dependant code packaged in a dll. In my mind, this use case is not the exception, and would not necessarily warrant the use of a full blown solution like conda. Our deploy

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread Steve Dower
David Genest wrote: > 1) add the dependent dlls to every package that needs it (Steve's answer > https://mail.python.org/pipermail/distutils-sig/2014-September/024982.html > concurs that the dependent dll would be loaded only once) This is the best approach regardless of what else works/doesn't wo

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread Paul Moore
On 1 October 2014 17:44, David Genest wrote: > - If you run python setup.py bdist_wheel, the dlls specified in the scripts > parameter end up in the wheel archive and does what is needed for our setup. > (the dlls are copied to the scripts directory which is on PATH for the > activated environm

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread David Cournapeau
On Wed, Oct 1, 2014 at 5:44 PM, David Genest wrote: > Thank you all for the precious info. > > Here are my observations: > > - We are merely writing extension modules with third party dependant code > packaged in a dll. In my mind, this use case is not the exception, and > would not necessarily w

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread Chris Barker
On Wed, Oct 1, 2014 at 9:44 AM, David Genest wrote: > - We are merely writing extension modules with third party dependent code > packaged in a dll. In my mind, this use case is not the exception, and > would not necessarily warrant the use of a full blown solution like conda. agreed -- it is n

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread Daniel Holth
On Wed, Oct 1, 2014 at 1:35 PM, Paul Moore wrote: > On 1 October 2014 17:44, David Genest wrote: >> - If you run python setup.py bdist_wheel, the dlls specified in the scripts >> parameter end up in the wheel archive and does what is needed for our setup. >> (the dlls are copied to the scripts

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread Paul Moore
On 1 October 2014 21:06, Daniel Holth wrote: > You are confusing generated entry_points script wrappers with the > setup(scripts=...) scripts. The scripts=... scripts should never be > skipped, even with --skip-scripts, they should work the same as they > always have. Sorry, you're right. But the

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread Nick Coghlan
On 2 Oct 2014 06:12, "Paul Moore" wrote: > > On 1 October 2014 21:06, Daniel Holth wrote: > > You are confusing generated entry_points script wrappers with the > > setup(scripts=...) scripts. The scripts=... scripts should never be > > skipped, even with --skip-scripts, they should work the same

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread Paul Moore
On 1 October 2014 23:10, Nick Coghlan wrote: >> Sorry, you're right. But the legacy (non entry-point) scripts are >> certainly fragile, and I'd recommend avoiding them. Even for actual >> scripts, and *certainly* as a hack to get things in the "Scripts" >> directory... > > Note that PEP 459 curren

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-01 Thread David Genest
> Note that PEP 459 currently proposes preserving this capability as > "python.commands.prebuilt", so I personally consider it reasonable as a way > of packaging arbitrary >executables and non-entry-point based scripts. Yes, this will prove valuable (for other things than dlls, admittedly).

Re: [Distutils] Wheels and dependent third party dlls on windows

2014-10-02 Thread Chris Barker
On Wed, Oct 1, 2014 at 5:44 PM, David Genest wrote: > We control our environment and package only what is needed in it. This > makes a micro system in which everything is controlled and isolated, even > the global dlls (to the virtual env) I wanted to install. If that is your use case, you may