Hi Vinay, On 03/29/2011 09:45 AM, Vinay Sajip wrote: > I've been thinking about the C-level configuration some more (cpythonv issue > #6). Suppose we have a virtual env at absolute path <testenv>, and in > <testenv>/bin we have a copied, non-installed Python, obtained from absolute > path <pythonsrc>. Say we provide the location of the source in the > configuration, and that this is read in getpath.c. Logically, it should just > replace the argv0_path value, and if the subsequent logic is unchanged, at the > point that site.py's main() runs, you would have (on Linux i686): > > sys.prefix, sys.exec_prefix would have values set from the configure script > ('/usr/local' by default). > sys.path would be ['/usr/local/lib/python33.zip', '<pythonsrc>/Lib', > '<pythonsrc>/Lib/plat-linux2', '<pythonsrc>/build/lib.linux-i686-3.3'] > > which seems OK. > > If a virtual environment is in absolute path <testenv>, then after the call to > virtualize() in site.py, we would have sys.prefix and sys.exec_prefix both set > to <testenv>. Since sys.executable still points to <testenv>/bin, the code in > sysconfig.py appears not to be doing the right thing, e.g. > sysconfig._PROJECT_BASE would be <testenv>/bin, which seems wrong. > > So it seems as if site.py and/or sysconfig.py might need further changes, as > well as changes to getpath.c. What do you think?
In general, I think the "copied binary" case should be as similar as possible to the "symlinked binary" case. Python doesn't dereference symlinks in setting sys.executable, so in either case sys.executable will point to the virtual environment's binary, which I think is clearly what we want (think code using sys.executable to decide how to re-exec itself). The changes I made to sysconfig thus far were the minimum necessary to be able to successfully install stuff in a pythonv virtualenv. On a brief glance, it looks to me like you are right; we'll need to modify the setting of _PROJECT_BASE in sysconfig.py to be virtual-aware (this is part -- maybe all -- of what's needed to fix issue #1). Also, see issue #10 - I'm wavering on whether changing sys.prefix and sys.exec_prefix to point to the virtualenv is actually the right approach. We're already forced to modify sysconfig.py to be virtual-aware in order to avoid having to hackily copy/symlink bits of the stdlib and include directories and the Makefile and config.h and whatnot into the virtualenv. Once we've taken the step of making sysconfig virtual-aware, it seems to me that perhaps sys.prefix is better left pointing to the actual Python installation, and we add a sys.virtual_prefix or some such to tell site.py and sysconfig.py where "virtualized" stuff belongs. There are a variety of possible meanings of sys.prefix, and the only one we really want to modify is "where should I install third-party packages?" Looking at sys.prefix references in the stdlib, I see them in trace, tkinter, and idlelib. It appears to me that in all of those cases, having sys.prefix pointing to the virtualenv would be wrong. If we switch to sys.virtual_prefix and leave sys.prefix alone, the danger would be that some installer that's using sys.prefix directly, and ignoring sysconfig.py, would try to install stuff to the global Python installation. But I'm not aware of any such installer, and I think we could safely call any installer of Python code that ignores the layout schemes in sysconfig thoroughly broken anyway. Your thoughts? (Or anyone else?) Carl P.S. The elephant in the room here is the fact that if we want pythonv to be backwards-compatible with distutils/distribute, which we do, we can't get away with just modifying sysconfig.py, we also have to modify the old distutils/sysconfig.py. In general distutils is frozen; I'm hoping that Tarek will let me get away with a few changes for pythonv if they aren't changing APIs at all, just modifying some install-scheme paths to be virtual-aware. _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig