Okay, so I've been thinking about all the recent complaints/requests regarding the perils of PYTHONPATH, site-packages, --prefix, and all that. And I'm thinking, is there some way I can work around all this stuff so that installation is totally DWIMmish -- that is, that the install can just "do what I mean", while still allowing a default version of a package to be set.
This already works perfectly with root access or a virtual Python, so we can pretty much ignore those setups for this discussion (except to make sure we don't break them). So, mostly what's left to worry about is being able to select an arbitrary installation location, and still have it work. If we assume you're smart enough to pick a location that will already be on your PYTHONPATH (and we could verify that by checking that it already is), then all we have to worry about is making sure that the right version of a package gets added to sys.path at runtime. For scripts this isn't that big of a deal. I could create a specialized bootstrap loader version of pkg_resources.py and dump it alongside any scripts installed by easy_install. It'd be a pain to implement, but I could probably make it work such that sys.path was searched for a setuptools egg, and then have that get used to bootstrap the rest. What this doesn't work for is the interactive interpreter, or code that doesn't use require(). Even if the relevant directory is on sys.path, Python doesn't normally read .pth files in PYTHONPATH directories. I've also learned that many Linux distros appear to hack sys.path in odd ways, and I'm not positive that all of them process .pth files in the directories they add. Certainly they don't support .pth for PYTHONPATH directories. Since it's not just the interactive interpreter, hacking .pythonrc isn't going to help much. That leaves the 'site' and 'sitecustomize' modules. The current PYTHONPATH install support hijacks the 'site' module with a special .pth-honoring feature, so as long as you have the setuptools egg listed directly in PYTHONPATH, you're golden. But alas, that isn't DWIM, since you now have to manipulate the PYTHONPATH *before* you can reasonably do anything. Part of the essence of what we desire is that any required steps be done for you *during* the installation rather than requiring an extra step before or afterwards. Unfortunately, this means that hacking 'site' can't work unless we actually install a site.py. Will that work? Well, yes and no. It appears Python will recognize an added site.py that appears in a PYTHONPATH, but many platforms will not recognize a site.py that's in the script directory but not PYTHONPATH. Windows seems to be the only platform where putting a site.py in the script directory has any effect. This is really good news, however. It appears to mean that slapping a copy of my hacked 'site.py' (i.e., the one that honors .pth files along PYTHONPATH) into the easy_install target directory would make PYTHONPATH-based installs essentially DWIMmish. To complete the illusion, I'd need to add some code to script wrappers to process the setuptools.pth file in the script directory if pkg_resources can't be otherwise imported. We could then lobby for the Python 2.5 site.py to behave the same way as my hacked one does: i.e., process .pth files found in PYTHONPATH directories, inserting them ahead of site-packages and its ilk, so that it wouldn't be necessary to install the hacked site.py in such cases. (It's also not necessary to install it in known "site" directories, such as site-packages and any directories listed in --site-dirs. But even if it's installed unnecessarily, it shouldn't hurt anything.) Since 'site' is a stdlib module and isn't normally intended for customization, it shouldn't conflict with any user-installed modules. However, easy_install could detect an existing site.py and warn if its content is different from the one easy_install wants to use. I'm trying to remember why I didn't do this around the time I implemented the original PYTHONPATH support (which first required a hacked 'site' module). I think I was thrown by the lack of cross-platform script directory support for importing 'site', and it didn't occur to me that I could easily work around that from within scripts. Also, it's easy to mentally confuse the script directory and the install directory while thinking about some of this stuff. But as long as the install directory (irrespective of the script directory) is on PYTHONPATH and has a hacked site.py, what's in the script directory doesn't matter. The only case where the script directory matters is if: 1. it's the same directory the libraries were installed to, 2. it's not on PYTHONPATH, and 3. the user puts a script of their own in there and expects it to work anywhere but Windows This edge case could be handled, however, by refusing to install libraries to non-PYTHONPATH directories unless you're on Windows or you use --multi-version. I'm somewhat uncertain about this, though, because it doesn't seem to let you create a self-sufficient application directory. Right now, you can use "easy_install -ad somedir ..." to copy all needed eggs and scripts to the target directory, making a self-contained application directory that has everything it needs to run. OTOH, I suppose if you just made that -mad instead of -ad, it would work just fine for that purpose under the proposed new regime. So, to sum up, here are the changes that would take place: * easy_install would treat all PYTHONPATH-specified directories as though they were listed in --site-dirs, but it would also copy and compile a hacked site.py into them during installation, if they aren't *actually* listed in --site-dirs or in setuptools' default computation of "site" directories. * Currently, easy_install defaults to -m if you install to a non-"site" directory; it would now instead stop and refuse to install to such directories unless you explicitly specify -m. * easy_install could now meaningfully grow a --prefix option, since it will now DWIM as long as the target dir is recognizable as (or convertible to) a "site" directory. And, in the cases where DWIM can't be guaranteed, it will stop and inform the user that they need to fix PYTHONPATH, --site-dirs, or use -m/--multi-version mode. What do y'all think? Do we have a winner yet? _______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
