Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Joachim König]

Phillip J. Eby wrote:
> It would be, if .eggs were a packaging format, rather than a binary 
> distribution/runtime format.
>
> Remember "eggs are to Python as jars are to Java" -- a Java .jar 
> doesn't contain documentation either, unless it's needed at 
> runtime.  Same for configuration files.
>   
But there's generally no need to easily have a look into a .class file 
with a tool the user
is used to whereas for python, we're often interested in knowing the 
details. And having
a zip file in my way to the source has left me frustrated often enough.

If you want to be consequent and consistent leave out the .py files from 
eggs, a jar file
normally doesn't contain .java sources files either.

> They're not system packages, in other words.  The assumption that 
> they are is another marketing failure, due to conflation of "package 
> == distribution of python code" and "package == thing you manage with 
> a system packager".  People see, "I put my package in an .egg" and 
> think it's the latter definition, when it's barely even the former.  :)
>   
I agree that they are not system packages, but I would have prefered to 
install multiple versions
of a package to separate "site-packages" directories, something that is 
really well understood by
most unsofisticated python programmers. The selection of the version 
could then be made at
runtime by a PYTHONPATH setting and not by fiddling with .pth files 
(something that could
be autmated by a tool and persisted in batch files).

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Stephen J. Turnbull]

Ben Finney writes:
 > "Stanley A. Klein" <[EMAIL PROTECTED]> writes:
 > 
 > > IMHO, the main system without a package manager is Windows.
 > 
 > AFAICT the MacOS platform also lacks in this area.

Actually, they both have them.  Windows has Cygwin (rpm-based), while
for MacOS Fink (deb-based), MacPorts (FreeBSD ports-like), and
NetBSD's pkgsrc are all viable options if you want packaging support
for 3rd-party packages.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Alexander Michael]

On Wed, Apr 9, 2008 at 3:40 PM, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
>  That is indeed a problem -- but it's a social one, not a technical
>  one.  It's trivial for the publisher of an egg to change their
>  command line from "setup.py bdist_egg upload" to "setup.py sdist
>  bdist_egg upload", as soon as their users (politely) request that they do so.

I know you say it in
,
but perhaps you should be more explicit about the best practice being
to distribute an sdist as well as one or more eggs and highlight this
with some standout text like:

"""
When uploading your project to PyPI, always upload your sdist in
addition to any eggs unless you have a good idiosyncratic reason not
to. The easy_install tool can also download and install sdist's on all
platforms (universally for pure-python packages and provided compilers
are available for distributions containing python extensions), so
uploading them gives your project the widest possible audience.
"""

Perhaps this will help some the social challenges.

Regards,
Alex
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Greg Ewing]

Paul Moore wrote:
> And I would say that Windows doesn't have a problem. Are any Windows
> users proposing building a package management system for Windows
> (Python-specific or otherwise)? It's a genuine question - is this
> something that Windows users are after, or is it just Linux users
> trying to show Windows users what they are missing?

I think the requirements for a package manager are different
on different platforms.

On Linux, you need to be able to cope with files scattered
all over the system, and complex webs of dependencies between
packages.

On Windows, you need to be able to cope with scattered files
and multiple applications sharing a file, but not usually
with dependencies, because each application typically comes
with all the files it needs (even if some of them might not
get installed because they're already there for another
application).

On MacOSX, applications are usually completely self-contained,
include all their dependencies and are not spread around,
so there's really nothing for a package manager to do.

What all this means for Python package management, I really
don't know. Whatever is done, I'd like to see it kept as
dirt-simple as possible. Ideal would be the MacOSX situation
where the package is just a directory of files that you
put somewhere obvious, and you can tell what it is just
by looking at it, and get rid of it by dragging it to the
trash -- so you don't need a package manager.

--
Greg
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[zooko]

On Apr 9, 2008, at 4:12 PM, Phillip J. Eby wrote:

>> http://allmydata.org/trac/setuptools/ticket/5 # binary eggs should
>> come with .py files by default, rather than .pyc files
>
> Filling your tracker with already-rejected proposals isn't likely  
> to encourage me to look at it, especially when I've personally  
> rejected them to you in IRC.  That goes for your ticket #4 as well.

Part of my motivation in maintaining this tracker is to take issue  
discussions from IRC, and from mailing lists, and make them more  
permanent and structured.  This part is useful even for rejected  
proposals, as an historical record that other people interested in  
those issues can consult.

I will mark those two tickets as "rejected by PJE".  Could you please  
repeat (so that I don't misrepresent you due to my faulty memory of  
our IRC discussion from more than a year ago) your reason for  
rejecting these two:

http://allmydata.org/trac/setuptools/ticket/4 (when considering  
whether to zip, err on the side of safety rather than performance)

http://allmydata.org/trac/setuptools/ticket/5 (binary eggs should  
come with .py files by default, rather than .pyc files)

You are of course welcome to log in to that trac and update those  
tickets yourself, but if you prefer not to then I will paste your  
reasons into those tickets.

Regards,

Zooko
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Phillip J. Eby]

At 11:48 PM 4/9/2008 +0100, Paul Moore wrote:
>On 09/04/2008, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> > It would be, if .eggs were a packaging format, rather than a binary
> >  distribution/runtime format.
> >
> >  Remember "eggs are to Python as jars are to Java" -- a Java .jar
> >  doesn't contain documentation either, unless it's needed at
> >  runtime.  Same for configuration files.
>
>And yet, Java doesn't have an equivalent of easy_install for jar
>files, to my knowledge.

Actually, OSGi and Eclipse plugins and "feature sites" come quite 
close, and setuptools rips off many of its features from them.  OSGi 
is basically a standard for additional .jar metadata to encompass 
dependencies and other info.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Phillip J. Eby]

At 03:20 PM 4/9/2008 -0700, zooko wrote:
>I've opened a ticket on my setuptools trac about this proposal:
>
>http://allmydata.org/trac/setuptools/ticket/5 # binary eggs should
>come with .py files by default, rather than .pyc files

Filling your tracker with already-rejected proposals isn't likely to 
encourage me to look at it, especially when I've personally rejected 
them to you in IRC.  That goes for your ticket #4 as well.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Paul Moore]

On 09/04/2008, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> It would be, if .eggs were a packaging format, rather than a binary
>  distribution/runtime format.
>
>  Remember "eggs are to Python as jars are to Java" -- a Java .jar
>  doesn't contain documentation either, unless it's needed at
>  runtime.  Same for configuration files.

And yet, Java doesn't have an equivalent of easy_install for jar
files, to my knowledge. If Python had eggs but no easy_install, maybe
this whole discussion wouldn't be taking place.

(I know I personally like the idea of egg-as-jar-file, but *hate* the
idea of egg-as-dependency-handling-tool-and-everything-else).

Paul.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Phillip J. Eby]

At 09:42 PM 4/9/2008 +0100, Floris Bruynooghe wrote:
>On Wed, Apr 09, 2008 at 03:57:13PM -0400, Phillip J. Eby wrote:
> > At 10:30 AM 4/9/2008 -0700, zooko wrote:
> > >where I would prefer either:
> > >a.  Doing away with database files entirely and relying on the
> > >filesystem alone to hold that information, or
> >
> > ...which is what PEP 262 *does*.
> >
> > Unfortunately, PEP 262's title is bad for marketing, as you've
> > effectively pointed out.  It would be better titled something
> > "package installation manifests" or "package contents files", or
> > something of that sort.
>
>Heh, *package* appears in both of your suggestions.  Surely this means
>tools that want to support installing script or data files need to
>keep their own manifests separately.  This would greatly simplify any
>PEP 262 style implementation.

Sadly, "package" has the additional meaning of "thing you distribute" 
in a distutils "distribution".

At this point, btw, I've pretty much come to the conclusion that the 
simplest way to solve the installdb problem is just going to be for 
setuptools to do its own thing, yet again.  :(

The only piece of the puzzle left, then, is the namespace package 
bit, but at least there the problem space has narrowed 
considerably.  The only issue there will be how to make the new 
(non-egg) installation method interoperate with the existing 
nspkg.pth solution.

And even there, the only problem to resolve is if you install 
something to a PYTHONPATH directory that shares a namespace with a 
system package.  Currently, the only way I know of to fix that would 
be to ensure that the PYTHONPATH directory supports .pth files, and 
to install an appropriate .pth there as well.  (Where "appropriate" 
means, an even hairier hack than today's nspkg.pth files.)

I'd really like to avoid that, if possible.

It would be best if we could solve the virtual-package thing in 
co-operation with the major system packagers (meaning the people, not 
just the tools).

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Floris Bruynooghe]

On Wed, Apr 09, 2008 at 03:57:13PM -0400, Phillip J. Eby wrote:
> At 10:30 AM 4/9/2008 -0700, zooko wrote:
> >where I would prefer either:
> >a.  Doing away with database files entirely and relying on the
> >filesystem alone to hold that information, or
> 
> ...which is what PEP 262 *does*.
> 
> Unfortunately, PEP 262's title is bad for marketing, as you've 
> effectively pointed out.  It would be better titled something 
> "package installation manifests" or "package contents files", or 
> something of that sort.

Heh, *package* appears in both of your suggestions.  Surely this means
tools that want to support installing script or data files need to
keep their own manifests separately.  This would greatly simplify any
PEP 262 style implementation.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Phillip J. Eby]

At 10:30 AM 4/9/2008 -0700, zooko wrote:
>PEP 262 sounds like a non-starter to me because
>
>1.  It appears to be backwards-incompatible with setuptools/ 
>easy_install/eggs, thus losing all the recently gained cooperation
>that I mentioned in the previous paragraph, and

No.  It provides a forward-compatibility path for the distutils, so 
that easy_install doesn't have to install things in .egg format in 
the future.  There's no compatibility breakage at all.


>2.  It defines a new database file,

No, it defines *files*.  One file per installed distribution, 
containing (among other things) an installation manifest.


>where I would prefer either:
>a.  Doing away with database files entirely and relying on the
>filesystem alone to hold that information, or

...which is what PEP 262 *does*.

Unfortunately, PEP 262's title is bad for marketing, as you've 
effectively pointed out.  It would be better titled something 
"package installation manifests" or "package contents files", or 
something of that sort.

>b.  Continuing to use the current ".pth" database file format,
>possibly improved by having native support for .pth files in the
>Python import machinery.

These mechanisms are orthogonal to this issue.


>3.  Because of #2, it triggers programmers to exclaim "They are
>planning to reinvent apt!", thus making it unlikely that the new
>proposal will recapture the cooperation that setuptools has already
>(slowly) gained.

Yeah, we need a new name.  Everybody is going off of "database of 
installed packages" and thinking "apt", because they aren't paying 
any closer attention.  However, given that we are discussing this on 
Python-Dev and distutils-sig, I do think it's reasonable to expect 
(if perhaps not reasonable to receive) that people discussing the PEP 
have *read* the freaking PEP first, prior to trashing it or offering 
alternatives.

And it's not like I'm personally offended or anything -- I didn't 
even write the PEP in question.  But what's the point of having PEPs 
if people read nothing but their titles?  We could just delete 
everything but PEP 0.  :)


>Perhaps PEP 262 and my proposal are not actually alternatives, but
>are complementary.

As I've already pointed out, your proposal does not address multiple 
installed versions of a package, and I see no sane way to modify it to do so.


>What I want is for the already implemented, tested, and deployed 
>code- re-use features of setuptools/easy_install to be more widely
>accepted.  This is best and most easily achieved by fixing the two
>most frequent objections to setuptools/easy_install: 1.  That you
>can't conveniently install into an arbitrary directory, and 2. that
>it subverts the meaning of your PYTHONPATH.

As I've already stated, the only way for these problems to be fixed 
is for easy_install to not install files in .egg form -- which also 
solves the general objection to using .eggs in the first place.

And the only way to do that, is to have a way to keep track of what 
files are installed.  Rather than have easy_install come up with its 
own way of doing that, I would prefer to share a standard with the 
distutils.  Hence, the PEP discussion.

For earlier versions of Python, it will still be possible to install 
and uninstall with setuptools using this approach.  You just won't be 
able to uninstall pure distutils-based packages, unless you installed 
them using easy_install.

Meanwhile, it has occurred to me that the easiest way of handling 
compatibility is not to require that other packaging tools mark their 
files for non-removability, but simply not allow easy_install to 
remove or overwrite anything that *isn't* claimed by a manifest.  In 
that way, easy_install would be immediately usable in the new mode, 
without any updates to Python or to system packaging tools.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[zooko]

On Apr 8, 2008, at 9:41 PM, Phillip J. Eby wrote:
>
> I'm curious.  Have any of you actually read PEP 262 in any detail?

I read it, though not in fine detail.

I didn't write that you are planning to reinvent apt.  I wrote that  
when programmers hear about this PEP they exclaim "They are planning  
to reinvent apt!".

That is a matter of perception and marketing -- the value that I want  
to gain from Python packages is the value of a critical mass of good  
programmers using compatible tools for code re-use.  If a lot of  
programmers hate an idea, then it doesn't matter what the details are  
-- it isn't going to provide this value to me.

I think part of our disagreement is that we are talking about two  
overlapping use cases: programmer and sysadmin (and "end user" which  
is much like sysadmin).  I am not, at this time, interested in the  
sysadmin use case.  As I've mentioned, my sysadmin needs are  
currently well satisfied by apt (and sometimes by GNU stow), and my  
fellow sysadmins with whom I work are absolutely not going to relax  
their "apt-only policy" for the forseeable future, so I cannot use  
such a tool unless it is named "apt" and written by Debian/Ubuntu.

On the other hand I am very interested in the programmer use case,  
because setuptools/easy_install already works pretty well for that,  
and we are already very close to achieving a critical mass of good  
programmers.  Recently several more packages that my project [1]  
relies on have become easy_installable -- Twisted, pywin32 (thanks to  
you, PJE), and foolscap -- and several more have had bugfixes which  
make them work better with easy_install/setuptools -- Nevow and  
zope.interface -- and there are some patches in the queue to make  
another one compatible with setuptools -- pyflakes [2, 3].

So setuptools/easy_install is already (slowly) winning.  I want to  
accelerate that process by reducing the degree to which it is  
incompatible, inconvenient, or objectionable to other programmers.

PEP 262 sounds like a non-starter to me because

1.  It appears to be backwards-incompatible with setuptools/ 
easy_install/eggs, thus losing all the recently gained cooperation  
that I mentioned in the previous paragraph, and

2.  It defines a new database file, where I would prefer either:
a.  Doing away with database files entirely and relying on the  
filesystem alone to hold that information, or
b.  Continuing to use the current ".pth" database file format,  
possibly improved by having native support for .pth files in the  
Python import machinery.

3.  Because of #2, it triggers programmers to exclaim "They are  
planning to reinvent apt!", thus making it unlikely that the new  
proposal will recapture the cooperation that setuptools has already  
(slowly) gained.

I'm sorry, PJE -- I know it must be frustrating to you to have your  
proposal criticized on social rather than technical grounds -- but  
social benefits are what I am seeking right now.

Perhaps PEP 262 and my proposal are not actually alternatives, but  
are complementary.  I do not object to Python maintaining a database  
of installed packages for itself (although I cannot *rely* upon such  
behavior, not least because I will be maintaining backwards  
compatibility with Python 2.4 for at least the next several years,  
and with Python 2.5 for at least the next several years after that).  
What I want is for the already implemented, tested, and deployed code- 
re-use features of setuptools/easy_install to be more widely  
accepted.  This is best and most easily achieved by fixing the two  
most frequent objections to setuptools/easy_install: 1.  That you  
can't conveniently install into an arbitrary directory, and 2. that  
it subverts the meaning of your PYTHONPATH.

Regards,

Zooko

[1] http://allmydata.org/source/tahoe/trunk/docs/install.html
[2] http://divmod.org/trac/ticket/2535
[3] http://divmod.org/trac/ticket/2048
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[zooko]

On Apr 8, 2008, at 11:27 AM, Lloyd Kvam wrote:
>
> When I wear my sysadmin hat, eggs become a nuisance.
...
> As a developer, eggs are great.
...
> Fortunately, distutils includes tools like bdist_rpm so that python
> modules can be packaged for easy processing by the system package
> manager.  So once I need to switch back to a sysadmin role, I can use
> the system tools to install and track packages.

This is the same experience I have.  I rely on setuptools and eggs  
extensively in developing our software, and I use setuptools and eggs  
as the primary method of giving our source code to other programmers.

But no software is ever installed on our production servers unless  
that software is in .deb form in an apt-gettable repository, and this  
requirement is unlikely to change for the forseeable future.

Regards,

Zooko

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Lloyd Kvam]

On Tue, 2008-04-08 at 10:01 -0700, zooko wrote:
> They both agreed that it made perfect sense.  I told one of them  
> about the alternate proposal to define a new database file to
> contain  
> a list of installed packages, and he sighed and rolled his eyes and  
> said "So they are planning to reinvent apt!".

When I wear my sysadmin hat, eggs become a nuisance.  They are not
listed in the system packages; if zipped they won't work when the apache
user tries to import them; easy_install can produce unexpected upgrades.
The system package manager (apt or yum) is much preferred.

As a developer, eggs are great.  If a python module is not already
available from my system packagers, easy_install will find it, get it,
and install it.  I waste almost no time with system administration
issues while developing.

Fortunately, distutils includes tools like bdist_rpm so that python
modules can be packaged for easy processing by the system package
manager.  So once I need to switch back to a sysadmin role, I can use
the system tools to install and track packages.

-- 
Lloyd Kvam
Venix Corp
DLSLUG/GNHLUG library
http://www.librarything.com/catalog/dlslug
http://www.librarything.com/profile/dlslug
http://www.librarything.com/rsshtml/recent/dlslug

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Chris McDonough]

zooko wrote:
> 
> On Mar 26, 2008, at 7:34 PM, Chris McDonough wrote:
>> zooko wrote:
> 
> http://mail.python.org/pipermail/python-dev/2008-March/078243.html
> 
>>> Here is a simple proposal:  make the standard Python "import"  
>>> mechanism notice eggs on the PYTHONPATH and insert them (into the  
>>> *same* location) on the sys.path.
>>> This eliminates the #1 problem with eggs -- that they don't easily  
>>> work when installing them into places other than your site-packages  
>>> and that if you allow any of them to be installed on your system 
>>> then  they take precedence over your non-egg packages even you 
>>> explicitly  put those other packages earlier in your PYTHONPATH.  
>>> (That latter  behavior is very disagreeable to more than a few 
>>> prorgammers.)
>>
>> Sorry if I'm out of the loop and there's some subtlety here that I'm 
>> disregarding, but it doesn't appear that either of the issues you 
>> mention is a actually problem with eggs.  These are instead problems 
>> with how eggs get installed by easy_install (which uses a .pth file to 
>> extend sys.path).  It's reasonable to put eggs on the PYTHONPATH 
>> manually (e.g. sys.path.append('/path/to/some.egg')) instead of using 
>> easy_install to install them.
> 
> Yes, you are missing something.  While many programmers, such as 
> yourself and Lennart Regebro (who posted to this thread) find the 
> current eggs system to be perfectly convenient and to Just Work, many 
> others, such as Glyph Lefkowitz (who posted to a related thread) find 
> them to be so annoying that they actively ensure that no eggs are ever 
> allowed to touch their system.
> 
> The reasons for this latter problem are two:
> 
> 1.  You can't conveniently install eggs into a non-system directory, 
> such as ~/my-python-stuff.

That's just not true.

$ export PYTHONPATH=/home/you/my-python-stuff/foo-1.3.egg
$ python
 >>> import foo

Eggs are directories (or zipfiles) that contain packages.  They happen to 
contain other metadata directories too, but these can be ignored if you just 
want to *use* them (as opposed to wanting to introspect metadata about them).

> 2.  If you allow even a single egg to be installed into your PYTHONPATH, 
> it will change the semantics of your PYTHONPATH.

I think you are mistaken.  The use of the .pth file that changes sys.path is a 
feature of easy_install, not of eggs.  You don't need to use any .pth file to 
put eggs on your PYTHONPATH.

Note that zc.buildout is a framework that installs eggs, and it doesn't rely at 
all on .pth files to automatically hack sys.path.  Instead, it munges console 
scripts to add each egg dir to sys.path.  This is pretty nasty too, but it does 
prove the point.

It is however true that you need to change sys.path to use an egg.  Is that 
what 
you're objecting to fundamentally?  You just don't want to have to change 
sys.path at all to use an egg?  Maybe you're objecting to the notion that an 
egg 
can contain more than one package.  There is functionally no difference between 
an egg and a directory full of packages.

> Both of these problems are directly caused by the need for eggs to hack 
> your site.py.  If Python automatically added eggs found in the 
> PYTHONPATH to the sys.path, both of these problems would go away.

I'm not sure what you mean.  Eggs don't hack site.py.  Eggs are just a 
packaging 
format.  easy_install doesn't hack site.py either.  It just puts a .pth file 
(the parsing of which is a feature of "core" Python itself, not any setuptools 
magic) in site packages and manages it.

It seems like you're advocating adding magic that you can't turn off (magical 
detection of eggs in an already site.py-approved packages directory) to defeat 
magic that you can turn off (by not using easy_install or .pth files).  At some 
level the magic you want to see built in to Python would almost certainly wind 
up doing what you hate by hacking sys.path unless you wanted to restrict eggs 
to 
containing a single package only.  And you wouldn't be able to turn it off 
except through some obscure environment variable setting.

> By the way, since I posted my proposal two weeks ago I have pointed a 
> couple of Python hackers who currently refuse to use eggs at the URL:
> 
> http://mail.python.org/pipermail/python-dev/2008-March/078243.html
> 
> They both agreed that it made perfect sense.  I told one of them about 
> the alternate proposal to define a new database file to contain a list 
> of installed packages, and he sighed and rolled his eyes and said "So 
> they are planning to reinvent apt!".

I think changing the Python core is the worst possible answer to this problem. 
"Don't use easy_install" is currently the best, AFAICT.

- C

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Paul Moore]

On 08/04/2008, zooko <[EMAIL PROTECTED]> wrote:
>  By the way, since I posted my proposal two weeks ago I have pointed a
>  couple of Python hackers who currently refuse to use eggs at the URL:
>
>  http://mail.python.org/pipermail/python-dev/2008-March/078243.html
>
>  They both agreed that it made perfect sense.  I told one of them
>  about the alternate proposal to define a new database file to contain
>  a list of installed packages, and he sighed and rolled his eyes and
>  said "So they are planning to reinvent apt!".

I do think that a simple solution like that has some merit. It has two
significant downsides, however:

1. It requires that core Python "bless" the egg format to some extent
- something Guido has said he is unwilling to do.
2. It ignores the issue of package management completely. Personally,
I avoid anything that doesn't integrate with a unified package manager
(whether that be the Windows add/remove feature, or an
as-yet-to-be-built custom Python package manager is not important).

Filesystem commands do not a package manager make...

Paul.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Phillip J. Eby]

At 10:01 AM 4/8/2008 -0700, zooko wrote:

>On Mar 26, 2008, at 7:34 PM, Chris McDonough wrote:
> > zooko wrote:
>
>http://mail.python.org/pipermail/python-dev/2008-March/078243.html
>
> >> Here is a simple proposal:  make the standard Python "import"
> >> mechanism notice eggs on the PYTHONPATH and insert them (into the
> >> *same* location) on the sys.path.
> >> This eliminates the #1 problem with eggs -- that they don't
> >> easily  work when installing them into places other than your site-
> >> packages  and that if you allow any of them to be installed on
> >> your system then  they take precedence over your non-egg packages
> >> even you explicitly  put those other packages earlier in your
> >> PYTHONPATH.  (That latter  behavior is very disagreeable to more
> >> than a few prorgammers.)
> >
> > Sorry if I'm out of the loop and there's some subtlety here that
> > I'm disregarding, but it doesn't appear that either of the issues
> > you mention is a actually problem with eggs.  These are instead
> > problems with how eggs get installed by easy_install (which uses
> > a .pth file to extend sys.path).  It's reasonable to put eggs on
> > the PYTHONPATH manually (e.g. sys.path.append('/path/to/some.egg'))
> > instead of using easy_install to install them.
>
>Yes, you are missing something.  While many programmers, such as
>yourself and Lennart Regebro (who posted to this thread) find the
>current eggs system to be perfectly convenient and to Just Work, many
>others, such as Glyph Lefkowitz (who posted to a related thread) find
>them to be so annoying that they actively ensure that no eggs are
>ever allowed to touch their system.
>
>The reasons for this latter problem are two:
>
>1.  You can't conveniently install eggs into a non-system directory,
>such as ~/my-python-stuff.

Wha?

>2.  If you allow even a single egg to be installed into your
>PYTHONPATH, it will change the semantics of your PYTHONPATH.

Only in the same way that manually putting an egg on the front of 
PYTHONPATH can be considered to "change the semantics" of your PYTHONPATH.


>Both of these problems are directly caused by the need for eggs to
>hack your site.py.  If Python automatically added eggs found in the
>PYTHONPATH to the sys.path, both of these problems would go away.

And add new ones.


>I am skeptical that the current proposals to define a new database
>for installed packages will fare any better than the current eggs
>scheme does in this respect.

The purpose for the installation database is to allow easy_install to 
eschew the use of .egg files or directories for anything other than 
multi-version installs.  Thus, no need to add those .egg files or 
directories to the head of the PYTHONPATH.  Conflicts would be 
handled at install time rather than runtime, in other words.


>I am skeptical that prorgammers are going to be willing to use a new
>database format.  They already have a database -- their filesystem --
>and they already have the tools to control it -- mv, rm, and
>PYTHONPATH.  Many of them already hate the existence the
>"easy_instlal.pth" database file, and I don't see why a new database
>file would be any different.

PEP 262 does not propose a database file -- it proposes the inclusion 
of a metadata file for each installed distribution.


>My proposal makes the current benefits of eggs -- clean, easy code re-
>use among programmers -- more compatible with their current tools --
>mv, rm, and PYTHONPATH.  It is also forward-compatible with more
>sophisticated proposals to add features like uninstall and operating
>system integration.

Actually, your current proposal doesn't work, unless you at least 
have some way to indicate which *version* of an egg should be 
automatically added to sys.path -- and some way to change 
that.  Otherwise, you might as well use the -m option to 
easy_install, and require() the eggs at runtime.  (Which needs 
neither .pth files nor site.py hacking.)

Meanwhile, my understanding is that the people who dislike eggs, 
dislike them because when they install a setuptools-based package, 
it's installed as an egg by default.  The installation database 
proposal (and by the way, people really should read and understand 
PEP 262, including the open issues, before trying to compete with 
it), will allow setuptools-based packages to install the 
"old-fashioned" way by default.  That is, not as eggs.  Similarly, 
easy_install would be able to skip installing .eggs unless you wanted 
multi-version support.

So, people who don't like eggs would never see them, since the only 
way you'd ever get them would be via easy_install -m, and they would 
never use it.


>By the way, since I posted my proposal two weeks ago I have pointed a
>couple of Python hackers who currently refuse to use eggs at the URL:
>
>http://mail.python.org/pipermail/python-dev/2008-March/078243.html
>
>They both agreed that it made perfect sense.  I told one of them
>about the alternate proposal to define a new database file to

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[zooko]

On Mar 26, 2008, at 7:34 PM, Chris McDonough wrote:
> zooko wrote:

http://mail.python.org/pipermail/python-dev/2008-March/078243.html

>> Here is a simple proposal:  make the standard Python "import"   
>> mechanism notice eggs on the PYTHONPATH and insert them (into the   
>> *same* location) on the sys.path.
>> This eliminates the #1 problem with eggs -- that they don't  
>> easily  work when installing them into places other than your site- 
>> packages  and that if you allow any of them to be installed on  
>> your system then  they take precedence over your non-egg packages  
>> even you explicitly  put those other packages earlier in your  
>> PYTHONPATH.  (That latter  behavior is very disagreeable to more  
>> than a few prorgammers.)
>
> Sorry if I'm out of the loop and there's some subtlety here that  
> I'm disregarding, but it doesn't appear that either of the issues  
> you mention is a actually problem with eggs.  These are instead  
> problems with how eggs get installed by easy_install (which uses  
> a .pth file to extend sys.path).  It's reasonable to put eggs on  
> the PYTHONPATH manually (e.g. sys.path.append('/path/to/some.egg'))  
> instead of using easy_install to install them.

Yes, you are missing something.  While many programmers, such as  
yourself and Lennart Regebro (who posted to this thread) find the  
current eggs system to be perfectly convenient and to Just Work, many  
others, such as Glyph Lefkowitz (who posted to a related thread) find  
them to be so annoying that they actively ensure that no eggs are  
ever allowed to touch their system.

The reasons for this latter problem are two:

1.  You can't conveniently install eggs into a non-system directory,  
such as ~/my-python-stuff.

2.  If you allow even a single egg to be installed into your  
PYTHONPATH, it will change the semantics of your PYTHONPATH.

Both of these problems are directly caused by the need for eggs to  
hack your site.py.  If Python automatically added eggs found in the  
PYTHONPATH to the sys.path, both of these problems would go away.

I am skeptical that the current proposals to define a new database  
for installed packages will fare any better than the current eggs  
scheme does in this respect.

This issue is important to me, because the benefits of eggs grow  
superlinearly with the number of good programmers who use them.  They  
are a tool for re-using source code -- a tool for cooperation between  
programmers.  To gain the greatest benefits at this point we do not  
need to add new features to eggs, we need to make them more palatable  
to more good programmers.

I am skeptical that prorgammers are going to be willing to use a new  
database format.  They already have a database -- their filesystem --  
and they already have the tools to control it -- mv, rm, and  
PYTHONPATH.  Many of them already hate the existence the  
"easy_instlal.pth" database file, and I don't see why a new database  
file would be any different.

My proposal makes the current benefits of eggs -- clean, easy code re- 
use among programmers -- more compatible with their current tools --  
mv, rm, and PYTHONPATH.  It is also forward-compatible with more  
sophisticated proposals to add features like uninstall and operating  
system integration.

By the way, since I posted my proposal two weeks ago I have pointed a  
couple of Python hackers who currently refuse to use eggs at the URL:

http://mail.python.org/pipermail/python-dev/2008-March/078243.html

They both agreed that it made perfect sense.  I told one of them  
about the alternate proposal to define a new database file to contain  
a list of installed packages, and he sighed and rolled his eyes and  
said "So they are planning to reinvent apt!".

Regards,

Zooko

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Chris McDonough]

zooko wrote:
> Folks:
> 
> Here is a simple proposal:  make the standard Python "import"  
> mechanism notice eggs on the PYTHONPATH and insert them (into the  
> *same* location) on the sys.path.
> 
> This eliminates the #1 problem with eggs -- that they don't easily  
> work when installing them into places other than your site-packages  
> and that if you allow any of them to be installed on your system then  
> they take precedence over your non-egg packages even you explicitly  
> put those other packages earlier in your PYTHONPATH.  (That latter  
> behavior is very disagreeable to more than a few prorgammers.)

Sorry if I'm out of the loop and there's some subtlety here that I'm 
disregarding, but it doesn't appear that either of the issues you mention is a 
actually problem with eggs.  These are instead problems with how eggs get 
installed by easy_install (which uses a .pth file to extend sys.path).  It's 
reasonable to put eggs on the PYTHONPATH manually (e.g. 
sys.path.append('/path/to/some.egg')) instead of using easy_install to install 
them.

I don't think there would be any benefit to changing Python's import machinery 
to deal with them; they are essentially just directories (or zipfiles) that 
contain packages.

- C

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Lennart Regebro]

Has somebody made a list of the problems with eggs? Because I use them
all the time and hasn't encountered any problems whatsoever, myself...
:) So I am a bit surprised at the various discussions about them.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig