Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Paul Moore
On 30 January 2014 05:33, Evgeny Sazhin eug...@sazhin.us wrote: Eh, I think both 1 and 3 are things that are possibly reasonable to happen and they are both things that I've contemplated as things to bring forward in using xz as an alternative compression format. Even if #1 would need a major

Re: [Distutils] Using Wheel with zipimport

2014-01-30 Thread Ralf Gommers
On Wed, Jan 29, 2014 at 5:22 PM, Vinay Sajip vinay_sa...@yahoo.co.ukwrote: On Wed, 29/1/14, Paul Moore p.f.mo...@gmail.com wrote: That package installation utilities should not dabble in sys.path manipulation. The import path is the user's

Re: [Distutils] Using Wheel with zipimport

2014-01-30 Thread Vinay Sajip
On Thu, 30/1/14, Ralf Gommers ralf.gomm...@gmail.com wrote: Also end user. If, as a user, I want to use inplace builds and PYTHONPATH instead of virtualenvs for whatever reason, that should be supported. Setuptools inserting stuff to sys.path that

Re: [Distutils] pip on windows experience

2014-01-30 Thread David Cournapeau
On Thu, Jan 30, 2014 at 7:50 AM, Paul Moore p.f.mo...@gmail.com wrote: On 29 January 2014 22:50, David Cournapeau courn...@gmail.com wrote: i.e. it would be nice if anyone setup to build C extensions could just build numpy. This has always been possible, and if not, that's certainly

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Vinay Sajip
On Thu, 30/1/14, Paul Moore p.f.mo...@gmail.com wrote: This is the biggest concern I see with promoting wheels being directly importable via zipimport (I say promoting and not specifying deliberately, but I don't want to get back into the process

Re: [Distutils] pip on windows experience

2014-01-30 Thread Paul Moore
On 30 January 2014 09:12, David Cournapeau courn...@gmail.com wrote: On Thu, Jan 30, 2014 at 7:50 AM, Paul Moore p.f.mo...@gmail.com wrote: On 29 January 2014 22:50, David Cournapeau courn...@gmail.com wrote: i.e. it would be nice if anyone setup to build C extensions could just build

Re: [Distutils] pip on windows experience

2014-01-30 Thread David Cournapeau
On Thu, Jan 30, 2014 at 10:20 AM, Paul Moore p.f.mo...@gmail.com wrote: On 30 January 2014 09:12, David Cournapeau courn...@gmail.com wrote: On Thu, Jan 30, 2014 at 7:50 AM, Paul Moore p.f.mo...@gmail.com wrote: On 29 January 2014 22:50, David Cournapeau courn...@gmail.com wrote: i.e.

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Paul Moore
On 30 January 2014 09:45, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: So much heat over process, but so little light over exactly why *appropriately designed* software deployed in wheels shouldn't be importable. Good point. In my view, *appropriately designed* software is fine, and a metadata

Re: [Distutils] pip on windows experience

2014-01-30 Thread Paul Moore
Thanks. I've added the installation details and output from a test run. Paul On 30 January 2014 10:25, David Cournapeau courn...@gmail.com wrote: On Thu, Jan 30, 2014 at 10:20 AM, Paul Moore p.f.mo...@gmail.com wrote: On 30 January 2014 09:12, David Cournapeau courn...@gmail.com wrote:

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Nick Coghlan
On 30 Jan 2014 15:27, Donald Stufft don...@stufft.io wrote: I can't believe folks are unable to differentiate between the difference of It's possible to do so because we didn't actively attempt to prevent it and This is a documented goal of the format and thus must be considered as part of the

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Vinay Sajip
My one technical issue is with going beyond zipimport behaviour to the point of extracting DLLs to the filesystem. I remain -1 on that feature, and I believe I have explained why I think there are issues (and why I think that any solution should be part of zipimport and not added on in

[Distutils] distlib.mount API design (was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Nick Coghlan
On 30 January 2014 21:57, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: My one technical issue is with going beyond zipimport behaviour to the point of extracting DLLs to the filesystem. I remain -1 on that feature, and I believe I have explained why I think there are issues (and why I think that

[Distutils] Extracting C extensions from zipfiles on sys.path (Was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Paul Moore
Changing the subject to clearly focus the discussion. On 30 January 2014 11:57, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: If you have other reasons for your -1, I'd like to hear them. OK. Note that this is not, in my view, an issue with wheels, but rather about zipfiles on sys.path, and

Re: [Distutils] distlib.mount API design (was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Paul Moore
On 30 January 2014 12:29, Nick Coghlan ncogh...@gmail.com wrote: I actually think this is a useful thing to experiment with, I'm just not sure distlib is the best place for that experiment. With appropriately secure tempfile handling and the right sys.path (and module __path__) manipulation

Re: [Distutils] pip on windows experience

2014-01-30 Thread Paul Moore
On 30 January 2014 10:56, Paul Moore p.f.mo...@gmail.com wrote: Here we go: https://github.com/numpy/numpy/issues/4245 Thanks. I've added the installation details and output from a test run. That bug report was just closed blaming a distutils issue which apparently numpy aren't going to work

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Paul Moore
On 30 January 2014 11:57, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: ISTM the reason why it can't go in there just yet is also the reason why one might have some reservations about the feature: binary compatibility Sorry, I didn't comment on this point. To me, binary compatibility is *not* the

Re: [Distutils] PEP 427

2014-01-30 Thread Donald Stufft
On Jan 29, 2014, at 11:29 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Jan 29, 2014, at 11:17 PM, Donald Stufft don...@stufft.io wrote: On Jan 29, 2014, at 11:16 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Jan 29, 2014, at 10:49 PM, Donald Stufft don...@stufft.io wrote: On Jan

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Donald Stufft
On Jan 30, 2014, at 9:30 AM, Donald Stufft don...@stufft.io wrote: On Jan 30, 2014, at 6:43 AM, Nick Coghlan ncogh...@gmail.com wrote: On 30 Jan 2014 15:27, Donald Stufft don...@stufft.io wrote: I can't believe folks are unable to differentiate between the difference of It's possible

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Paul Moore
On 30 January 2014 15:03, Donald Stufft don...@stufft.io wrote: For those following along at home who have no interest in supporting zipped Wheels who want to disclaim support for it, here's how you do zip_safe=False for Wheels: https://gist.github.com/dstufft/8710270. Note that this will also

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Donald Stufft
On Jan 30, 2014, at 10:12 AM, Paul Moore p.f.mo...@gmail.com wrote: On 30 January 2014 15:03, Donald Stufft don...@stufft.io wrote: For those following along at home who have no interest in supporting zipped Wheels who want to disclaim support for it, here's how you do zip_safe=False for

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Eric V. Smith
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/30/2014 10:23 AM, Donald Stufft wrote: On Jan 30, 2014, at 10:12 AM, Paul Moore p.f.mo...@gmail.com wrote: On 30 January 2014 15:03, Donald Stufft don...@stufft.io wrote: For those following along at home who have no interest in

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Daniel Holth
I see you've noticed wheel was released in an imperfect state. Let's add a Zip-Safe flag to the WHEEL file, values of true and false same as Root-Is-Purelib. Daniel On Thu, Jan 30, 2014 at 10:44 AM, Eric V. Smith e...@trueblade.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On

Re: [Distutils] Extracting C extensions from zipfiles on sys.path (Was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Vinay Sajip
On Thu, 30/1/14, Paul Moore p.f.mo...@gmail.com wrote: Subject: Extracting C extensions from zipfiles on sys.path (Was: wheels on sys.path clarification (reboot)) To: Vinay Sajip vinay_sa...@yahoo.co.uk Cc: Distutils distutils-sig@python.org Date:

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Donald Stufft
Just an FYI, I've begun compiling notes and plan on proposing PEPs that will be aimed at supplanting PEP 425 and PEP 427 with the goal of nailing down undefined behavior, including missing functionality, and removing misfeatures. On Jan 30, 2014, at 10:56 AM, Daniel Holth dho...@gmail.com wrote:

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Vinay Sajip
On Thu, 30/1/14, Eric V. Smith e...@trueblade.com wrote: The flag really needs to convey 2 meanings: - - There are some C extensions here that can't be loaded unless they live on a real filesystem path. - - There is some code (maybe in

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Evgeny Sazhin
On Thu, Jan 30, 2014 at 3:09 AM, Paul Moore p.f.mo...@gmail.com wrote: On 30 January 2014 05:33, Evgeny Sazhin eug...@sazhin.us wrote: Eh, I think both 1 and 3 are things that are possibly reasonable to happen and they are both things that I've contemplated as things to bring forward in using xz

Re: [Distutils] PEP 427

2014-01-30 Thread Evgeny Sazhin
On Thu, Jan 30, 2014 at 9:38 AM, Donald Stufft don...@stufft.io wrote: On Jan 29, 2014, at 11:29 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Jan 29, 2014, at 11:17 PM, Donald Stufft don...@stufft.io wrote: On Jan 29, 2014, at 11:16 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Jan 29,

Re: [Distutils] PEP 427

2014-01-30 Thread Donald Stufft
On Jan 30, 2014, at 11:44 AM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 9:38 AM, Donald Stufft don...@stufft.io wrote: On Jan 29, 2014, at 11:29 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Jan 29, 2014, at 11:17 PM, Donald Stufft don...@stufft.io wrote: On Jan

Re: [Distutils] PEP 427

2014-01-30 Thread Evgeny Sazhin
On Thu, Jan 30, 2014 at 11:48 AM, Donald Stufft don...@stufft.io wrote: On Jan 30, 2014, at 11:44 AM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 9:38 AM, Donald Stufft don...@stufft.io wrote: On Jan 29, 2014, at 11:29 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Jan 29,

Re: [Distutils] Extracting C extensions from zipfiles on sys.path (Was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Thomas Heller
Am 30.01.2014 14:23, schrieb Paul Moore: First of all, it is not possible to load a DLL into a process' memory [2, 3] unless it is stored as a file in the filesystem. [...] [2] I'm talking from a Windows perspective here. I do not have sufficient low-level knowledge of Unix to comment on

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Evgeny Sazhin
On Thu, Jan 30, 2014 at 11:59 AM, Donald Stufft don...@stufft.io wrote: On Jan 30, 2014, at 11:33 AM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 3:09 AM, Paul Moore p.f.mo...@gmail.com wrote: On 30 January 2014 05:33, Evgeny Sazhin eug...@sazhin.us wrote: Eh, I think both

Re: [Distutils] PEP 427

2014-01-30 Thread Daniel Holth
On Thu, Jan 30, 2014 at 12:23 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 11:48 AM, Donald Stufft don...@stufft.io wrote: On Jan 30, 2014, at 11:44 AM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 9:38 AM, Donald Stufft don...@stufft.io wrote: On Jan

[Distutils] PEX - Twitter's multi-platform executable archive format for Python

2014-01-30 Thread Vinay Sajip
I searched this list for PEX and didn't find any hits, so I thought some people here might be interested in the PEX software from Twitter: https://www.youtube.com/watch?v=NmpnGhRwsu0 is a 15-minute presentation by Brian Wickman about it and other topics. It uses zipimport and is used to do

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Paul Moore
On 30 January 2014 16:33, Evgeny Sazhin eug...@sazhin.us wrote: I don't have good references or pointers to good Python development or deployment practices, but you may want to ask around on python-list. And that is my biggest concern (deployment). I believe these questions should be

Re: [Distutils] Extracting C extensions from zipfiles on sys.path (Was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Paul Moore
On 30 January 2014 17:51, Thomas Heller thel...@ctypes.org wrote: I wanted to post this to clarify the current state. Thanks. I had no idea (or I'd forgotten - one or the other!) that this was actually being done. Paul ___ Distutils-SIG maillist -

Re: [Distutils] platform specific python only wheels

2014-01-30 Thread Daniel Holth
Your best bet currently is to execute the mv command to change the filename of the wheel. I think bdist_wheel should probably accept a setup() argument to set the tags. Currently you can just set a universal flag which means pure 2+3 python. On Thu, Jan 30, 2014 at 1:14 PM, Ronald Oussoren

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Evgeny Sazhin
On Thu, Jan 30, 2014 at 12:56 PM, Paul Moore p.f.mo...@gmail.com wrote: On 30 January 2014 16:33, Evgeny Sazhin eug...@sazhin.us wrote: I don't have good references or pointers to good Python development or deployment practices, but you may want to ask around on python-list. And that is my

[Distutils] platform specific python only wheels

2014-01-30 Thread Ronald Oussoren
Hi, Is there a way to create a wheel that contains only python code, but can only be installed on particular platforms? In particular, I’m looking for a way to create a wheel for py2app that can only be installed on OSX because py2app cannot “cross-compile” application bundles. Ronald

Re: [Distutils] Using Wheel with zipimport

2014-01-30 Thread Noah Kantrowitz
On Jan 30, 2014, at 1:09 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: On Thu, 30/1/14, Ralf Gommers ralf.gomm...@gmail.com wrote: Also end user. If, as a user, I want to use inplace builds and PYTHONPATH instead of virtualenvs for whatever

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Daniel Holth
One thing that might be useful would be to develop the unpacked wheel which is currently undefined but would be deliberately identical to a site-packages with just one wheel extracted into it. You wouldn't have to argue or worry about the zip issue. I like the way npm puts everything into a

Re: [Distutils] platform specific python only wheels

2014-01-30 Thread Vinay Sajip
On Thu, 30/1/14, Ronald Oussoren ronaldousso...@mac.com wrote: Is there a way to create a wheel that contains only python code, but can only be installed on particular platforms? Perhaps not yet, but the idea is there in the

Re: [Distutils] platform specific python only wheels

2014-01-30 Thread Ronald Oussoren
On 30 Jan 2014, at 19:19, Daniel Holth dho...@gmail.com wrote: Your best bet currently is to execute the mv command to change the filename of the wheel. I think bdist_wheel should probably accept a setup() argument to set the tags. Currently you can just set a universal flag which means

Re: [Distutils] platform specific python only wheels

2014-01-30 Thread Ronald Oussoren
On 30 Jan 2014, at 19:23, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: On Thu, 30/1/14, Ronald Oussoren ronaldousso...@mac.com wrote: Is there a way to create a wheel that contains only python code, but can only be installed on particular

Re: [Distutils] platform specific python only wheels

2014-01-30 Thread Daniel Holth
On Thu, Jan 30, 2014 at 1:29 PM, Ronald Oussoren ronaldousso...@mac.com wrote: On 30 Jan 2014, at 19:19, Daniel Holth dho...@gmail.com wrote: Your best bet currently is to execute the mv command to change the filename of the wheel. I think bdist_wheel should probably accept a setup()

Re: [Distutils] platform specific python only wheels

2014-01-30 Thread Daniel Holth
You are probably a perfect customer for supports_environment On Thu, Jan 30, 2014 at 1:40 PM, Ronald Oussoren ronaldousso...@mac.com wrote: On 30 Jan 2014, at 19:36, Daniel Holth dho...@gmail.com wrote: On Thu, Jan 30, 2014 at 1:29 PM, Ronald Oussoren ronaldousso...@mac.com wrote: On 30

Re: [Distutils] platform specific python only wheels

2014-01-30 Thread Ronald Oussoren
On 30 Jan 2014, at 19:36, Daniel Holth dho...@gmail.com wrote: On Thu, Jan 30, 2014 at 1:29 PM, Ronald Oussoren ronaldousso...@mac.com wrote: On 30 Jan 2014, at 19:19, Daniel Holth dho...@gmail.com wrote: Your best bet currently is to execute the mv command to change the filename of

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Evgeny Sazhin
On Thu, Jan 30, 2014 at 1:24 PM, Daniel Holth dho...@gmail.com wrote: One thing that might be useful would be to develop the unpacked wheel which is currently undefined but would be deliberately identical to a site-packages with just one wheel extracted into it. You wouldn't have to argue or

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Ronald Oussoren
On 30 Jan 2014, at 17:27, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: On Thu, 30/1/14, Eric V. Smith e...@trueblade.com wrote: The flag really needs to convey 2 meanings: - - There are some C extensions here that can't be loaded unless

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Daniel Holth
On Thu, Jan 30, 2014 at 2:15 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 1:24 PM, Daniel Holth dho...@gmail.com wrote: One thing that might be useful would be to develop the unpacked wheel which is currently undefined but would be deliberately identical to a site-packages

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Evgeny Sazhin
On Thu, Jan 30, 2014 at 2:32 PM, Daniel Holth dho...@gmail.com wrote: On Thu, Jan 30, 2014 at 2:15 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 1:24 PM, Daniel Holth dho...@gmail.com wrote: One thing that might be useful would be to develop the unpacked wheel which is

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Brett Cannon
On Thu, Jan 30, 2014 at 12:46 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 11:59 AM, Donald Stufft don...@stufft.io wrote: On Jan 30, 2014, at 11:33 AM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 3:09 AM, Paul Moore p.f.mo...@gmail.com wrote: On

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Evgeny Sazhin
On Thu, Jan 30, 2014 at 2:52 PM, Brett Cannon br...@python.org wrote: still far from seeing good reasoning. Please, correct me, but so far i saw 2: 1. It is impossible to load C extensions 2. Error handling is bad None of these reasons seems to be good enough to lead to a conclusion

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Daniel Holth
Wow, it really can install from a directory-named-like-a-wheel with the --no-index flag passed. I'm sad to say that is totally unintentional, but bizarrely interesting. On Thu, Jan 30, 2014 at 2:52 PM, Evgeny Sazhin eug...@sazhin.us wrote: On Thu, Jan 30, 2014 at 2:32 PM, Daniel Holth

Re: [Distutils] wheels on sys.path clarification (reboot)

2014-01-30 Thread Vinay Sajip
On Thu, 30/1/14, Daniel Holth dho...@gmail.com wrote: Wow, it really can install from a directory-named-like-a-wheel with the --no-index flag passed. I'm sad to say that is totally unintentional, but bizarrely interesting. What, *more*

Re: [Distutils] Using Wheel with zipimport

2014-01-30 Thread Donald Stufft
On Jan 30, 2014, at 1:21 PM, Noah Kantrowitz n...@coderanger.net wrote: On Jan 30, 2014, at 1:09 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: On Thu, 30/1/14, Ralf Gommers ralf.gomm...@gmail.com wrote: Also end user. If, as a user, I

Re: [Distutils] PEP 427

2014-01-30 Thread Nick Coghlan
On 31 Jan 2014 02:48, Donald Stufft don...@stufft.io wrote: How am i supposed to manage that using pip and virtual envs in production? The same way you'd use them in development? Hell I believe you can even do: $ virtualenv my_virtualenv $ my_virtualenv/bin/pip install

Re: [Distutils] Using Wheel with zipimport

2014-01-30 Thread Nick Coghlan
Thanks for pointing that out Noah - I was planning to come back and check if that wording was considered more acceptable. On 31 Jan 2014 07:12, Donald Stufft don...@stufft.io wrote: The updated text is fine with me as it at least accurately documents the fact that using that feature is fraught

Re: [Distutils] distlib.mount API design (was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Nick Coghlan
On 30 Jan 2014 23:26, Paul Moore p.f.mo...@gmail.com wrote: On 30 January 2014 12:29, Nick Coghlan ncogh...@gmail.com wrote: I actually think this is a useful thing to experiment with, I'm just not sure distlib is the best place for that experiment. With appropriately secure tempfile

Re: [Distutils] distlib.mount API design (was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Paul Moore
On 30 January 2014 22:38, Nick Coghlan ncogh...@gmail.com wrote: The advantage of wheels over plain zipfiles for this use case is the structured metadata. distlib.mount doesn't try to guess the package structure for the extensions, you have to provide an EXTENSIONS file in the metadata that

Re: [Distutils] distlib.mount API design (was: wheels on sys.path clarification (reboot))

2014-01-30 Thread Vinay Sajip
On Thu, 30/1/14, Paul Moore p.f.mo...@gmail.com wrote: I'm still not comfortable with the temp directory clutter that unpacking leaves (in particular on Windows where deletion isn't even possible in an atexit routine) but I'll survive. It's up to