At 12:11 AM 3/4/2009 -0600, s...@pobox.com wrote:
So I have rather casually installed a bunch of packages from PyPI using
easy_install. It now contains 23 eggs, all of which get used a bit, but
nowhere near as often as the standard lib or our internal stuff.
Unsuccessfully stat()ing in all those egg directories was really starting to
hammer our NFS server (we often start up hundreds of Python processes at a
time), so I commented out the second import line at the end which serves to
leave the eggs at the end of sys.path. Now every time I install a package I
have to quickly edit easy-install.path to remove the import line the install
process adds.
I wonder though... Is there some alternate way to install stuff from PyPI
which reduces the number of directories on sys.path? Maybe some
post-processing step which takes the current set of eggs, creates a single
directory into which they are all unfolded? That single directory could
then be added to sys.path.
You could use Ian Bicking's 'pip' installer. It can't install from
eggs, but it does do source installs this way.
There is a very straightforward mapping from zipped eggs to "flat"
files, btw. For a given .egg file or directory, the mapping is
simply to unzip it, renaming its EGG-INFO subdirectory to the
original egg's filename with '-info' appended. E.g.
"foobar-1.2-py2.5.egg/EGG-INFO/" becomes
"foobar-1.2-py2.5.egg-info/". Everything else remains exactly the same.
It wouldn't be hard to make a tool that does this; in fact, the
setuptools.archive_util module has an 'extract_archive' function that
works on directories as well as zipfiles, and can be passed a filter
function that can change the extraction location for any given
file. Something like:
def flatten_egg(egg_filename, extract_dir):
eggbase = os.path.filename(egg_filename)+'-info'
def file_filter(src, dst):
if src.startswith('EGG-INFO/'):
src = eggbase+s[8:]
dst = os.path.join(extract_dir, *src.split('/'))
return dst
return extract_archive(egg_filename, extract_dir, file_filter)
Of course, this will only work if 'egg_filename' is a valid .egg file
or directory; the results are otherwise pretty unpredictable. This
also doesn't check for overwriting other files, deal with
permissions, etc. Ergo, making this production-ready is left as an
exercise for the implementer. ;-)
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
