Re: [Distutils] window 64bit madness
On 03/16/2011 11:47 PM, P.J. Eby wrote: At 11:28 AM 3/16/2011 +0100, Adam GROSZER wrote: On 03/14/2011 07:50 PM, P.J. Eby wrote: Run python -c 'import pkg_resources;print pkg_resources.get_build_platform()' (with the Python interpreter you're using. D:\installc:\Python26_64\python.exe Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit (AMD64)] on win32 Type help, copyright, credits or license for more information. import pkg_resources;print pkg_resources.get_build_platform() win-amd64 Hm. What's sys.platform? Is it win32? Exactly. I suspect the problem has to do with all the win32-specificness scattered through setuptools; I think I know what I need to do to fix it all, but it'll be a fairly substantial patch; can you help with the testing if I email you such a patch? Sure thing. -- Best regards, Adam GROSZER -- Quote of the day: Don't knock President Fillmore. He kept us out of Vietnam. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv, take two
Carl Meyer carl at oddbird.net writes: work well. Please try it and tell me what I missed! To try it out, create a directory somewhere with a bin/ subdirectory, symlink or copy (it works either way) the python3 binary from an install of the pythonv2 branch into bin/, and create lib/python3.3/site-packages It seems to work with a symlink but not with a copy. With an empty pythonv.conf file in ~/projects/vptest: vinay@eta-natty:~/projects/vptest/bin$ ln -s ~/tools/cpythonv/python vinay@eta-natty:~/projects/vptest/bin$ ./python Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) [GCC 4.5.2] on linux2 Type help, copyright, credits or license for more information. import sys sys.path ['', '/usr/local/lib/python33.zip', '/home/vinay/tools/cpythonv/Lib', '/home/vinay/tools/cpythonv/Lib/plat-linux2', '/home/vinay/tools/cpythonv/build/lib.linux-i686-3.3', '/home/vinay/projects/vptest/lib/python3.3/site-packages'] vinay@eta-natty:~/projects/vptest/bin$ rm python vinay@eta-natty:~/projects/vptest/bin$ cp ~/tools/cpythonv/python . vinay@eta-natty:~/projects/vptest/bin$ ./python Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) [GCC 4.5.2] on linux2 Type help, copyright, credits or license for more information. import sys sys.path ['', '/usr/local/lib/python33.zip', '/usr/local/lib/python3.3', '/usr/local/lib/python3.3/plat-linux2', '/usr/local/lib/python3.3/lib-dynload', '/usr/local/lib/python3.3/site-packages'] Still, very promising! Nice work. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
Whatever mechanism we end up with, I suggest that a standard python install include an isolated configuration. This is a common use case and should be available without having to create a virtualenv (or whatever) for each project or working directory. I'm very happy to see this work taking place. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] reservations about pythonv
On Wed, Mar 16, 2011 at 10:39 PM, Barry Warsaw ba...@python.org wrote: I vaguely recall that while this *should* work, it actually doesn't because once the executable has started, $LD_LIBRARY_PATH isn't consulted again. I recall less vaguely, since we've had to deal with this problem more recently than your Solaris antics. :-) The loader consults LD_LIBRARY_PATH before main() is called, initializing it's own state, and doesn't pick up changes. This makes a re-exec necessary if the new value is different from the original (worth checking to avoid an exec). -Fred -- Fred L. Drake, Jr. fdrake at acm.org A storm broke loose in my mind. --Albert Einstein ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] reservations about pythonv
On Mar 17, 2011, at 08:36 AM, Fred Drake wrote: On Wed, Mar 16, 2011 at 10:39 PM, Barry Warsaw ba...@python.org wrote: I vaguely recall that while this *should* work, it actually doesn't because once the executable has started, $LD_LIBRARY_PATH isn't consulted again. I recall less vaguely, since we've had to deal with this problem more recently than your Solaris antics. :-) The loader consults LD_LIBRARY_PATH before main() is called, initializing it's own state, and doesn't pick up changes. This makes a re-exec necessary if the new value is different from the original (worth checking to avoid an exec). Thanks for helping bolster my memory! I still think setting $LD_LIBRARY_PATH won't be necessary in the majority of cases, so generally no re-exec should happen. I can imagine that if it *were* necessary, an appropriate section in the pythonv.conf file would trigger it. E.g.: [env] LD_LIBRARY_PATH: /path/to/add [pythonv] re-exec: true Cheers, -Barry signature.asc Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote: Whatever mechanism we end up with, I suggest that a standard python install include an isolated configuration. This is a common use case and should be available without having to create a virtualenv (or whatever) for each project or working directory. Could you elaborate on what this means? I don't quite understand what you mean by include an isolated configuration. I do think that Python should include a script to create the small amount of set up needed to trigger a (built-in) virtual environment. E.g. create the bin and lib/... directories, populate bin, and drop a minimal pythonv.conf file. -Barry signature.asc Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] distutils2 Forward Compatibility module
Hi all, This is my first post specifically to distutils-sig, though I've had an interest in packaging for a while (having come up with some fairly arcane schemes in the past, where no better alternatives were apparent). At any rate: I'm currently working on a plan to overhaul how a number of my company's projects are packaged and distributed. Right now they all rely on a monkey-patched, hacked up distutils that needs to go away. I'm already on top of that. But seeing as how distutils2 is going to be the new hotness I want to plan for at as part of my overhaul. I realize that distutils2 is still in flux, and anything I do now will have to be tweaked as development on it continues. I am fine with this, as I still intend to use Distribute as the primary installation mechanism. But I really like how distutils2 keeps all metadata in the setup.cfg file, and want to start doing that now, so that I don't have to keep two copies of everything. It should be no problem to just have my setup.py read everything it needs out of setup.cfg, but what I'm wondering is if there is already an extension to do this, or will I have to roll my own? It just seems like an obvious thing to have for transitioning to distutils2, and if it doesn't already exist it should (I will of course be happy to contribute). I should note that I don't want distutils2 itself to be a dependency for installing my packages, as it is too unstable, so directly using any machinery built into it is out of the question. Thanks for any comments, Erik ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] New buildout options: checksums and allow-omitted-checksums
Hi, two weeks ago I asked about your opinions on a buildout option that enforces specifying (MD5) checksums for all files downloaded through buildout's download utility API. I've been discussing the subject with Christian Theune in the meantime and would like to describe a more concrete proposal now that deals with the questions raised in my previous post: In analogy with version pinning for eggs, two new options could be introduced to the buildout section: - checksums: This option would name a config section that contains checksums for any number of resources by URL. I suggest a default value of checksum for it. Listing some URL with an empty checksum would explicitly express that the checksum for this resource should never be checked. I'm not sure how to structure the contents of the checksums section: URLs are not valid config keys in general (they can contain = characters) but I'd still like to be able to rely on the existing mechanism for overriding single options to override single checksums. Mapping arbitrary keys to whitespace-separated pairs of URL and checksum would work but sounds inelegant. - allow-omitted-checksums: This option would specify whether resources should be downloaded that are not listed in the checksums section. I'd like to use False as this option's default value, meaning that buildout should raise a UserError if a resource is omitted from the checksums section. (Intentionally empty checksums would still be allowed.) In fact, I'm not completely happy about inventing an option with negative semantics but doing it this way is at least consistent with allow-picked-versions. I'd like to hear other people's opinion on both the general idea and the details. Unless the whole thing gets shot down, I plan to start implementing it on a branch of zc.buildout next week. Thank you. -- Thomas ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distutils2 Forward Compatibility module
Hi Erik, On Thu, Mar 17, 2011 at 8:42 AM, Erik Bray erik.m.b...@gmail.com wrote: Hi all, This is my first post specifically to distutils-sig, though I've had an interest in packaging for a while (having come up with some fairly arcane schemes in the past, where no better alternatives were apparent). At any rate: I'm currently working on a plan to overhaul how a number of my company's projects are packaged and distributed. Right now they all rely on a monkey-patched, hacked up distutils that needs to go away. I'm already on top of that. But seeing as how distutils2 is going to be the new hotness I want to plan for at as part of my overhaul. I realize that distutils2 is still in flux, and anything I do now will have to be tweaked as development on it continues. I am fine with this, as I still intend to use Distribute as the primary installation mechanism. But I really like how distutils2 keeps all metadata in the setup.cfg file, and want to start doing that now, so that I don't have to keep two copies of everything. It should be no problem to just have my setup.py read everything it needs out of setup.cfg, but what I'm wondering is if there is already an extension to do this, or will I have to roll my own? It just seems like an obvious thing to have for transitioning to distutils2, and if it doesn't already exist it should (I will of course be happy to contribute). I should note that I don't want distutils2 itself to be a dependency for installing my packages, as it is too unstable, so directly using any machinery built into it is out of the question. There's such a thing, look at this function here: http://hg.python.org/distutils2/file/6fca65ff60ad/distutils2/util.py#l1098 It's not hooked yet to a command-line tool like mkcfg, but should be soon. Let us know how it works out for you. Thanks for any comments, Erik ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig -- Tarek Ziadé | http://ziade.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv, take two
Hi Vinay, On 03/17/2011 06:25 AM, Vinay Sajip wrote: It seems to work with a symlink but not with a copy. With an empty pythonv.conf file in ~/projects/vptest: vinay@eta-natty:~/projects/vptest/bin$ ln -s ~/tools/cpythonv/python vinay@eta-natty:~/projects/vptest/bin$ ./python Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) [GCC 4.5.2] on linux2 Type help, copyright, credits or license for more information. import sys sys.path ['', '/usr/local/lib/python33.zip', '/home/vinay/tools/cpythonv/Lib', '/home/vinay/tools/cpythonv/Lib/plat-linux2', '/home/vinay/tools/cpythonv/build/lib.linux-i686-3.3', '/home/vinay/projects/vptest/lib/python3.3/site-packages'] vinay@eta-natty:~/projects/vptest/bin$ rm python vinay@eta-natty:~/projects/vptest/bin$ cp ~/tools/cpythonv/python . vinay@eta-natty:~/projects/vptest/bin$ ./python Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) [GCC 4.5.2] on linux2 Type help, copyright, credits or license for more information. import sys sys.path ['', '/usr/local/lib/python33.zip', '/usr/local/lib/python3.3', '/usr/local/lib/python3.3/plat-linux2', '/usr/local/lib/python3.3/lib-dynload', '/usr/local/lib/python3.3/site-packages'] This is odd, as the same setup works fine for me with a copied binary (I'm on Ubuntu 10.10). If you're willing to do any more debugging on this, here's what would be helpful: 1. Check the values of sys.executable, sys.prefix, and sys.exec_prefix. 2. Stick a pdb.set_trace() into site.py near the top of the virtualize() method and step through it, observing where something goes awry. In particular: does it find pythonv.conf? Does it modify PREFIXES? Thanks for testing it out! Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] reservations about pythonv
On 03/17/2011 10:56 AM, Barry Warsaw wrote: Thanks for helping bolster my memory! I still think setting $LD_LIBRARY_PATH won't be necessary in the majority of cases, so generally no re-exec should happen. I can imagine that if it *were* necessary, an appropriate section in the pythonv.conf file would trigger it. E.g.: [env] LD_LIBRARY_PATH: /path/to/add [pythonv] re-exec: true I also realized last night that if the need for LD_LIBRARY_PATH is as rare as it seems to be, people could just as well set it themselves before running stuff in their virtualenv. We could even have our shell activate script set it, so you'd only have to set it yourself if using the virtualenv's binary directly without activate. I'm a bit tempted in this direction, as the whole auto-re-exec business feels like it introduces an entirely different level of magic. But if it would make life easier, it shouldn't be hard to do. Since I'm finding and parsing pythonv.conf in site.py, would it be acceptable to do the exec there? Or are there good reasons for it to happen earlier in C code? (I'm a bit loath to have to find and parse pythonv.conf twice). If we do this, I'd also appreciate (for the PEP) a more concrete real-life example of what you might try do in a a virtualenv that would require this. Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On 03/17/2011 10:59 AM, Barry Warsaw wrote: On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote: Whatever mechanism we end up with, I suggest that a standard python install include an isolated configuration. This is a common use case and should be available without having to create a virtualenv (or whatever) for each project or working directory. Could you elaborate on what this means? I don't quite understand what you mean by include an isolated configuration. I'm also not entirely clear what this means, but I think perhaps python -S already covers it? That will start up a python interpreter without importing site.py, so it will have no site-packages at all; nothing but the stdlib. Of course, then you'd have to take care of fixing up sys.path yourself to include your project and its dependencies: this may be reasonable for buildout. I do think that Python should include a script to create the small amount of set up needed to trigger a (built-in) virtual environment. E.g. create the bin and lib/... directories, populate bin, and drop a minimal pythonv.conf file. Definitely. It looks like this'll end up being the bulk of the added code; I'm planning to do it (borrowing liberally from virtualenv.py) as soon as I'm confident of the basic approach. Any opinions on the commandline UI for this? I was thinking of just adding a pythonv.py to the stdlib that you could execute with python -m pythonv path/to/new/env (and would also export appropriate API to create environments programmatically). Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] reservations about pythonv
On Mar 17, 2011, at 12:32 PM, Carl Meyer wrote: I also realized last night that if the need for LD_LIBRARY_PATH is as rare as it seems to be, people could just as well set it themselves before running stuff in their virtualenv. We could even have our shell activate script set it, so you'd only have to set it yourself if using the virtualenv's binary directly without activate. +1. Keeps Python nice and simple and AFAICT, it should work because it'll set the environment variable before Python starts executing. OTOH, if no actual use case for LD_LIBRARY_PATH gets raised, then even this won't be necessary. -Barry signature.asc Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] New buildout options: checksums and allow-omitted-checksums
Hi Thomas, I like your idea in general. I'd like to point to the following suggestion with patch+test (though it might need some cleanup) that is not exactly related to what you're proposing, but has to do with the same thing (relationship between files and checksums): https://bugs.launchpad.net/zc.buildout/+bug/692600 It suggests (and implements) automatically redownloading files when checksums don't match (which could happen when you update your config file with a new checksum for a file that has changed upstream). Other comments below: On Thu, Mar 17, 2011 at 16:55, Thomas Lotze tho...@thomas-lotze.de wrote: Hi, [...] - checksums: This option would name a config section that contains checksums for any number of resources by URL. I suggest a default value of checksum for it. Won't 'checksums' (plural) as the value be better? It would keep with the tradition of matching the name of the buildout option and the name of the section by default. Listing some URL with an empty checksum would explicitly express that the checksum for this resource should never be checked. I'm not sure how to structure the contents of the checksums section: URLs are not valid config keys in general (they can contain = characters) but I'd still like to be able to rely on the existing mechanism for overriding single options to override single checksums. Mapping arbitrary keys to whitespace-separated pairs of URL and checksum would work but sounds inelegant. I suggest mapping the checksums (which are valid keys) to URLs, and having a special key with line-break separated values for explicitly not checking: [checksums] 080d2c6a849ebd6b7fd49049c21b910a = http://example.com/foo/bar.tgz no-check = http://example.com/foo/baz.tgz http://example.com/foo/fred.tgz This will not be so elegant when you want to extend another configuration and override some decisions, but it works somewhat: [buildout] extends = config-file-above.cfg [checksums] 080d2c6a849ebd6b7fd49049c21b910a = no-check += http://example.com/foo/bar.tgz [...] Cheers, Leo ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Thu, Mar 17, 2011 at 12:44 PM, Carl Meyer c...@oddbird.net wrote: On 03/17/2011 10:59 AM, Barry Warsaw wrote: On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote: Whatever mechanism we end up with, I suggest that a standard python install include an isolated configuration. This is a common use case and should be available without having to create a virtualenv (or whatever) for each project or working directory. Could you elaborate on what this means? I don't quite understand what you mean by include an isolated configuration. As I mentioned in an earlier thread, I want to get complete isolation from local additions relative to the standard Python distribution. I want python executable I can use to bootstrap a buildout that doesn't include site packages or it's equivalent. My understanding of how this will work was that I could created this myself by creating some sort of configuration file, say clean.cfg and then link a Python executable to the name clean. Reading pythonv, take two more carefully, I see that it is a lot more virtualenv-oriented than I'd hoped. shrug I'm also not entirely clear what this means, but I think perhaps python -S already covers it? I thought so too, but people keep telling me that's not enough. One issue was that maybe site.py might be imported accidentally, for example to get at one of the helper functions it contains. I don't know if this is a significant danger. I'd like to have some defined way to express my need for isolation that's more rubust that simply not importing site.py. That will start up a python interpreter without importing site.py, so it will have no site-packages at all; nothing but the stdlib. Of course, then you'd have to take care of fixing up sys.path yourself to include your project and its dependencies: this may be reasonable for buildout. Yup. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On 03/17/2011 01:23 PM, Jim Fulton wrote: My understanding of how this will work was that I could created this myself by creating some sort of configuration file, say clean.cfg and then link a Python executable to the name clean. Reading pythonv, take two more carefully, I see that it is a lot more virtualenv-oriented than I'd hoped. shrug I'm also not entirely clear what this means, but I think perhaps python -S already covers it? I thought so too, but people keep telling me that's not enough. One issue was that maybe site.py might be imported accidentally, for example to get at one of the helper functions it contains. I don't know if this is a significant danger. I'd like to have some defined way to express my need for isolation that's more rubust that simply not importing site.py. Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up, and you simply _don't_ create any lib/python3.3/site-packages relative to pythonv.conf, what you'll end up with is identical to python -S but robust against import of site.py (it will already have been imported, but it won't have found any site-packages directories). If the possibility of someone accidentally creating lib/python3.3/site-packages is an issue, we could easily add a no-site=True option to pythonv.conf that would prevent it from even checking for the existence of that directory. And in either case, we could add a --no-site option to python -m pythonv that would create a virtualenv without the site-packages directory (and with no-site=True, if we decide that's worth having). Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Thu, Mar 17, 2011 at 1:41 PM, Carl Meyer c...@oddbird.net wrote: Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up, and you simply _don't_ create any lib/python3.3/site-packages relative to pythonv.conf, what you'll end up with is identical to python -S but robust against import of site.py (it will already have been imported, but it won't have found any site-packages directories). Cool. If the possibility of someone accidentally creating lib/python3.3/site-packages is an issue, we could easily add a no-site=True option to pythonv.conf that would prevent it from even checking for the existence of that directory. I don't think this is an issue. And in either case, we could add a --no-site option to python -m pythonv that would create a virtualenv without the site-packages directory (and with no-site=True, if we decide that's worth having). Cool (for virtualenv). It occurs to me that it would be nice if site.py could grow knowledge of whether -S was used and not automatically mutate the path if -S was used. That would allow -S to work robustly without having to link anything or create a config file. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv, take two
On 03/16/2011 11:04 PM, Barry Warsaw wrote: +1. Time for a PEP? Working on a draft PEP. I'll push it to bitbucket to make collaboration easier - then the next step would be to present the draft on python-ideas, if I'm reading PEP 2 correctly? Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote: Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up, Is there any reason why the configuration file has to be one directory up, intead of adjacent to the executable? ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On 03/17/2011 03:00 PM, P.J. Eby wrote: At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote: Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up, Is there any reason why the configuration file has to be one directory up, intead of adjacent to the executable? Not a particularly good one. I inherited this from the standard layout of virtualenv, which includes a bin/ directory for the python binary and scripts, and it made more sense to me in that layout for pythonv.conf to live at the root of the virtualenv rather than in bin/. I'd like to continue to support it being one level up, but I can't think of any reason not to first check adjacent, and then one level up. I suppose it could just keep checking up the tree, but that introduces a bunch more filesystem checks and could give surprising results. I won't do that unless someone steps up to give convincing arguments for it. Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv, take two
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/17/2011 02:50 PM, Carl Meyer wrote: On 03/16/2011 11:04 PM, Barry Warsaw wrote: +1. Time for a PEP? Working on a draft PEP. I'll push it to bitbucket to make collaboration easier - then the next step would be to present the draft on python-ideas, if I'm reading PEP 2 correctly? Correct. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2CXVgACgkQ+gerLs4ltQ7UhwCdHAa2pk5ia3Ls9w4O6SADdeJ7 ehgAnibKo+/zwJ7aydUAYYbuk/xkmvxk =K0bJ -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv, take two
On Mar 17, 2011, at 02:50 PM, Carl Meyer wrote: On 03/16/2011 11:04 PM, Barry Warsaw wrote: +1. Time for a PEP? Working on a draft PEP. I'll push it to bitbucket to make collaboration easier - then the next step would be to present the draft on python-ideas, if I'm reading PEP 2 correctly? I dunno. I tend to think python-ideas are where ideas go to die. ;) I think we have general consensus that this is a good idea, and we just need to hash out the details. So I personally think python-dev is fine at this point. (Has nothing to do with me not actually being subscribed to python-ideas. Nope, not at all. :) -Barry signature.asc Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On 03/17/2011 01:53 PM, Jim Fulton wrote: It occurs to me that it would be nice if site.py could grow knowledge of whether -S was used and not automatically mutate the path if -S was used. That would allow -S to work robustly without having to link anything or create a config file. This seems entirely sensible to me (and quite easy to implement). I can't imagine why anyone would be relying on being able to do python -S and later import site and get the path modifications automatically, so it seems pretty much like a straightforward bugfix. And the fix is a one-liner. Issue filed with patch: http://bugs.python.org/issue11591 Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Thu, Mar 17, 2011 at 3:46 PM, Carl Meyer c...@oddbird.net wrote: On 03/17/2011 01:53 PM, Jim Fulton wrote: It occurs to me that it would be nice if site.py could grow knowledge of whether -S was used and not automatically mutate the path if -S was used. That would allow -S to work robustly without having to link anything or create a config file. This seems entirely sensible to me (and quite easy to implement). I can't imagine why anyone would be relying on being able to do python -S and later import site and get the path modifications automatically, so it seems pretty much like a straightforward bugfix. And the fix is a one-liner. Issue filed with patch: http://bugs.python.org/issue11591 Thanks! Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Thu, Mar 17, 2011 at 3:04 PM, Carl Meyer c...@oddbird.net wrote: On 03/17/2011 03:00 PM, P.J. Eby wrote: At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote: Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up, Is there any reason why the configuration file has to be one directory up, intead of adjacent to the executable? Not a particularly good one. I inherited this from the standard layout of virtualenv, which includes a bin/ directory for the python binary and scripts, and it made more sense to me in that layout for pythonv.conf to live at the root of the virtualenv rather than in bin/. I'd like to continue to support it being one level up, but I can't think of any reason not to first check adjacent, and then one level up. I suppose it could just keep checking up the tree, but that introduces a bunch more filesystem checks and could give surprising results. I won't do that unless someone steps up to give convincing arguments for it. I suggest the following: Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'. So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On 03/17/2011 05:13 PM, Jim Fulton wrote: I suggest the following: Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'. So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg. Nice - I like the ability to have multiple interpreters side-by-side with different pythonv configurations. Is .cfg generally preferred to .conf for some good reason? I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-) And I kind of dislike the inconsistency in extension; would clean.pythonv.cfg be acceptable? To simplify documentation and allow more flexibility, I might just check for all four: first the executable-specific one in both directories, then the general one in both directories. Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Thu, Mar 17, 2011 at 5:33 PM, Carl Meyer c...@oddbird.net wrote: On 03/17/2011 05:13 PM, Jim Fulton wrote: I suggest the following: Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'. So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg. Nice - I like the ability to have multiple interpreters side-by-side with different pythonv configurations. Is .cfg generally preferred to .conf for some good reason? I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-) And I kind of dislike the inconsistency in extension; would clean.pythonv.cfg be acceptable? To simplify documentation and allow more flexibility, I might just check for all four: first the executable-specific one in both directories, then the general one in both directories. Too many variants will get annoying. Along those lines, if it can be in multiple places please make having more than one an error. -- Benji York ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Thu, Mar 17, 2011 at 5:33 PM, Carl Meyer c...@oddbird.net wrote: Is .cfg generally preferred to .conf for some good reason? I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-) That all depends on who you ask; I tend to prefer .conf myself (but only by a hair). Jim likely has his own opinion. And I kind of dislike the inconsistency in extension; would clean.pythonv.cfg be acceptable? I'd accept that, so long as we pick only one of .conf or .cfg. -Fred -- Fred L. Drake, Jr. fdrake at acm.org A storm broke loose in my mind. --Albert Einstein ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
At 05:13 PM 3/17/2011 -0400, Jim Fulton wrote: I suggest the following: Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'. So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg. And on Windows, presumably remove the .exe part? or are you saying 'python.exe.pythonv'? ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Mar 17, 2011, at 12:44 PM, Carl Meyer wrote: Any opinions on the commandline UI for this? I was thinking of just adding a pythonv.py to the stdlib that you could execute with python -m pythonv path/to/new/env (and would also export appropriate API to create environments programmatically). Bikeshedding time: how about something a little more descriptive? $ python -m virtualize path/to/new/env -Barry signature.asc Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On 03/17/2011 04:39 PM, Barry Warsaw wrote: Bikeshedding time: how about something a little more descriptive? $ python -m virtualize path/to/new/env Ok, that color will do nicely. Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
On Mar 17, 2011, at 05:33 PM, Carl Meyer wrote: On 03/17/2011 05:13 PM, Jim Fulton wrote: I suggest the following: Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'. So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg. Nice - I like the ability to have multiple interpreters side-by-side with different pythonv configurations. Indeed. In that case though, wouldn't ./bin/clean.cfg be fine? Even if the executable were named ./bin/python a sibling ./bin/python.cfg would be fine too I think. I agree with Carl that I dislike the inconsistency in the extension, but I think argv[0] + '.cfg' would be fine. Having a non-.cfg or -.conf extension makes it harder to do stuff like automatically set the major mode in Emacs (although I guess it wouldn't be that hard to add a mapping for *.pythonv). Is .cfg generally preferred to .conf for some good reason? I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-) I guess I'm on the .cfg side, but wouldn't care too much if .conf is the consensus. And I kind of dislike the inconsistency in extension; would clean.pythonv.cfg be acceptable? I'm not sure you need the .pythonv. part. To simplify documentation and allow more flexibility, I might just check for all four: first the executable-specific one in both directories, then the general one in both directories. Definitely add a --verbose option to print out exactly which one got chosen. :) -Barry signature.asc Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
FWIW, I've pushed a reworking of the config-file-finding, with the following shed-paint color choices: * File is named executable-name.virtual.conf (I kept the .virtual, despite it being longer and not strictly necessary, because I think it more clearly expresses the function of the file. The existence or absence of this file changes Python's behavior significantly, so I think its name should be quite explicit.) * executable-name has the extension stripped on Windows, but not otherwise. * Config file can be located adjacent to the binary or one level up. The default sys.prefix is always the directory in which the configuration file is found, but can be overridden with a new prefix setting in the config file (which can be a path relative to the config file's location). * I eliminated any form of generically-named config file, as I think the extra possible names (and thus possible sources of confusion) is not really worth the benefit. The only use case I can think of is if you have multiple python binaries or symlinks next to each other and want them all to use the same virtual config - and I can't really think why you'd want multiple binaries in that case. Substantive comments on these choices is fine, especially in the form of presenting real use-cases I'm not currently allowing for. Testing that reveals a flaw in the basic operation of this style of virtual environment is much more useful. The bikeshed colors are easy to change, but I want to be really sure this thing actually does what it's supposed to reliably before I go presenting a PEP for it. Thanks! Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv, take two
Carl Meyer carl at oddbird.net writes: This is odd, as the same setup works fine for me with a copied binary (I'm on Ubuntu 10.10). If you're willing to do any more debugging on this, here's what would be helpful: I think I know what the problem is: the python executable checks to see where it was run from. If it looks as if it was run from a source build, it looks for site.py in the Lib folder relative to the executable; otherwise it looks for site.py in sys.prefix/lib/pythonX.Y. I've copied ~/tools/cpythonv/python to ~/projects/vptest/bin, and here's what happens: vinay@eta-natty:~/projects/vptest/bin$ ~/tools/cpythonv/python Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) [GCC 4.5.2] on linux2 Type help, copyright, credits or license for more information. import sys; sys.prefix '/usr/local' sys.path ['', '/usr/local/lib/python33.zip', '/home/vinay/tools/cpythonv/Lib', '/home/vinay/tools/cpythonv/Lib/plat-linux2', '/home/vinay/tools/cpythonv/build/lib.linux-i686-3.3', '/usr/local/lib/python3.3/site-packages'] vinay@eta-natty:~/projects/vptest/bin$ ./python Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) [GCC 4.5.2] on linux2 Type help, copyright, credits or license for more information. import sys; sys.prefix '/usr/local' sys.path ['', '/usr/local/lib/python33.zip', '/usr/local/lib/python3.3', '/usr/local/lib/python3.3/plat-linux2', '/usr/local/lib/python3.3/lib-dynload', '/usr/local/lib/python3.3/site-packages'] Notice how the sys.prefix is the same in both cases, but the path is different. Your site.py is never run with the copy, because it gets the site.py from /usr/local/lib/python3.3: vinay@eta-natty:~/projects/vptest/bin$ ~/tools/cpythonv/python Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) [GCC 4.5.2] on linux2 Type help, copyright, credits or license for more information. import site; site.__file__ '/home/vinay/tools/cpythonv/Lib/site.py' vinay@eta-natty:~/projects/vptest/bin$ ./python Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) [GCC 4.5.2] on linux2 Type help, copyright, credits or license for more information. import site; site.__file__ '/usr/local/lib/python3.3/site.py' I suppose this won't be a problem when the actual Python 3.3 is installed - the virtualizing site.py will be in /usr/lib/python3.3 or /usr/local/lib/python3.3. On this system I have a local build of Python made from the official default branch, which is installed to /usr/local/lib since the system Python is 3.2 (Ubuntu 11.04 - Natty). Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv, take two
Vinay Sajip vinay_sajip at yahoo.co.uk writes: On this system I have a local build of Python made from the official default branch, which is installed to /usr/local/lib since the system Python is 3.2 (Ubuntu 11.04 - Natty). Sorry, I meant to say that the system Python is 2.7 (in /usr). I also have an official Ubuntu build of 3.2 in /usr, but the source build installs by default to /usr/local. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
At 07:53 PM 3/17/2011 -0400, Carl Meyer wrote: FWIW, I've pushed a reworking of the config-file-finding, with the following shed-paint color choices: * File is named executable-name.virtual.conf (I kept the .virtual, despite it being longer and not strictly necessary, because I think it more clearly expresses the function of the file. The existence or absence of this file changes Python's behavior significantly, so I think its name should be quite explicit.) * executable-name has the extension stripped on Windows, but not otherwise. This combination should work well on Windows, where the default explorer settings will show e.g. 'python.virtual' next to 'python' in the directory. (Windows strips suffixes from display by default, and moves them to a type column.) * I eliminated any form of generically-named config file, as I think the extra possible names (and thus possible sources of confusion) is not really worth the benefit. The only use case I can think of is if you have multiple python binaries or symlinks next to each other and want them all to use the same virtual config - and I can't really think why you'd want multiple binaries in that case. The main reason I'd use differently-named binaries would be if I were shipping multiple runnable applications that I wanted to look to users like true .exe's. I don't see a reason why I wouldn't use separate .virtual.conf files, though, especially if their contents are minimal. (Awesomeness bonus: if the executable put *itself* on sys.path, and ran __main__, you could simply tack a zipfile on the end of the .exe and have a ready-to-run application.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python
Sounds good to me. On Mar 17, 2011, at 07:53 PM, Carl Meyer wrote: * executable-name has the extension stripped on Windows, but not otherwise. It should probably also have the extension stripped on OS X too. -Barry signature.asc Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
