Re: [Distutils] formencode as .egg in Debian ??
At 12:54 PM 11/25/2005 +1100, David Arnold wrote: So, if a system package, shipped by the upstream developer as an egg, is "unpacked" into a directory structure, and its metadata is maintained in a .egg-info file somewhere in sys.path, non-system eggs will have all they need to operate correctly? Yes, with a few clarifications. The internal structure of an egg, let's say foobar-1.2-py23.egg, would look something like: foobar/ __init__.py baz.py # plus .pyc files, etc. EGG-INFO/ PKG-INFO # distutils metadata like description/version requires.txt # optional and required dependencies # plus other metadata files, either setuptools-defined or # project specific If you unpack this as-is, but rename EGG-INFO to foobar.egg-info (today) or foobar-1.2.egg-info (when I release 0.6a9 of setuptools), and the whole tree above is in a directory on sys.path, this egg is good to go. I would like to clarify the phrase "shipped as an egg", though. To me, that would mean that the developer is distributing a binary .egg file, and I'm assuming that Debian is primarily interested in *source* packages, being a Free Software distribution. (A binary .egg doesn't have to contain source code at all; you can specifically build it with the source stripped if you desire.) The plan for setuptools 0.6a9 is to provide an option to "setup.py install" that will basically install the layout described above, with the correctly named .egg-info directory automatically created. (Normally, the whole tree above is instead nested in an .egg file or directory.) I think I should also clarify that whether the upstream developer sets out to package their project as an egg or not, it's possible to create an .egg-info directory and PKG-INFO file to identify that distribution, using setuptools' "easy_install" program and the source distribution. So if the developer of 'foobar' did not choose to create an egg or use setuptools, this doesn't stop a developer who wants to *use* foobar from simply running easy_install to create an .egg file for it. So, this is what I mean when I say there's no such thing as a non-egg package for an egg developer. Someone who depends on a package can simply say they depend on it, and when they build their package, they'll get eggs for their dependencies as a side effect. So that's another goal of eggs? To provide information to a package maintainer to assist in determining if it's the user's PYTHONPATH or .pth files that are causing a bug? More specifically, what versions of what packages they're *actually* using, as opposed to what they think they installed or have on their system. PYTHONPATH and .pth files can of course be a factor in that, but also just people thinking they installed something, or not knowing that a bug is fixed in a particular version. Part of it too is finding out whether they're reporting a regression or whether they're just still using a version that has a bug that's been fixed. In the case of the TurboGears mailing list, it's often been the case that TurboGears users flush out a bug in a dependency, which then gets fixed, but then a new TurboGears user maybe reports the same problem, and then it's obvious from their error message whether or not they upgraded. I realize this is stuff you guys probably do all day for system packages, but eggs make the support job easier upstream too. I can see that this is *nice*; I'd debate "need". But I'm happy to accept that for egg-based stuff, this is a nice feature. Well, need is relative. A project like TurboGears "needs" this, because otherwise it would be uneconomical to provide the current level of support on as many platforms. So, one project's "nice to have" may be another project's lifeblood, depending on available resources. They've also made it easier for the authors of TurboGears' dependencies to assist in support as well. For me, I'm glad that these features have helped to make something like TurboGears possible and practical. I'm not going to try to assert "Unix values" here. My observation is that historically, Unix has installed things into one of a couple of directory hierarchies (/usr, /usr/local, /opt). Within those hierarchies, there has been scope for only one version of any given thing. Um, sure. Not sure what this has to do with the present discussion. As a practical matter, only *one* version of an egg can be *active* (i.e. importable) on sys.path within a given process anyway. It's also clearly not going to be the case on a Debian system that somebody would have multiple versions of something living in /usr/lib, although they might do it for /usr/local or in a user-private directory. So, I think maybe I lost the train of thought on this point here. I was under the impression that the consensus of the Debian-Python folks so far was that of any egg f
Re: [Distutils] formencode as .egg in Debian ??
-->"Phillip" == Phillip J Eby <[EMAIL PROTECTED]> writes: Phillip> Python developers would *love* to have Debian manage their Phillip> packages, they would simply like to be able to verify at Phillip> runtime that the management has in fact been done. It's not Phillip> that we don't trust you, it's just that we're paranoid. :) >> From a Debian perspective, that's strictly your own affair. >> Debian packages "just work", and if they don't it's a bug. There's >> no need for anyone other than the packager to worry about this. Phillip> And if we were Debian developers rather than Python Phillip> developers, this would be a sensible approach to take. As a Phillip> practical matter, we have to worry, because our users aren't Phillip> strictly Debian users, and asking them all to move to Debian Phillip> isn't practical for most of us. ;) Yep, I understand that totally. Phillip> Let me ask you this: does Debian ban people from putting $id$ Phillip> strings in C code to allow identifying the version of the Phillip> source that was used to build a library or executable? No. Phillip> Because that's what *exactly* what we're talking about here Phillip> in respect to dependencies, just for Python code instead of Phillip> C. So, if a system package, shipped by the upstream developer as an egg, is "unpacked" into a directory structure, and its metadata is maintained in a .egg-info file somewhere in sys.path, non-system eggs will have all they need to operate correctly? Phillip> We'd like for Debian to advertise to our packages, precisely Phillip> what versions of which of our dependencies are installed. >> In general, I don't see why packages should care. Either it works, >> in which case there's no problem, or it doesn't, in which case it's >> a packaging bug, and it will be fixed. Phillip> But we can't tell if it's a packaging bug, or a bug against Phillip> what version of our dependencies, unless we know what version Phillip> the user is running. So that's another goal of eggs? To provide information to a package maintainer to assist in determining if it's the user's PYTHONPATH or .pth files that are causing a bug? I can see this would be helpful. But Debian won't tell you that (as you said you'd like, above). Phillip> Given that sys.path can include *both* Debian system packages Phillip> and local user-installed packages, we need a more Phillip> comprehensive way of dealing with the issue, which is what Phillip> eggs provide. I can see that this is *nice*; I'd debate "need". But I'm happy to accept that for egg-based stuff, this is a nice feature. >> Supporting the installation and simultaneous use of multiple >> versions seems to be a goal of eggs? Phillip> Absolutely. >> FWIW, that's kinda heretical on Unix (which might explain some of >> the antipathy). Phillip> The Python approach is "Practicality Beats Purity", which I Phillip> thought was also a Unix value. I'm not going to try to assert "Unix values" here. My observation is that historically, Unix has installed things into one of a couple of directory hierarchies (/usr, /usr/local, /opt). Within those hierarchies, there has been scope for only one version of any given thing. This is ok because developers take care to maintain API compatibility at the source level, at least, and ABI compatibility if they're well behaved. If you need to break compatibility, it's normally done by renaming the new verion (ie. gtk vs gtk2). Without going into the merits of this, it's my observation that this has been the in-practice approach (and you can draw whatever philosophical "Unix values" you like from that). Phillip> And we'd like all this to cleanly work with any Phillip> locally-installed non-Debian eggs that might be in the mix, Phillip> since we need to do development, beta testing, etc. >> And non-egg packages as well, right? Phillip> There isn't any such thing, from an egg developer's Phillip> perspective. Really? So if I use one egg, everything has to be an egg? Phillip> Any distutils package can be made into an egg, because all of Phillip> the metadata needed is supplied by the standard distutils Phillip> setup script. So, if you have the source, you can make it an Phillip> egg. What if I don't have the source (or setup.py) ? Phillip> That doesn't mean you should change Debian to eggs, because Phillip> eggs aren't trying to solve the problems that you guys solve Phillip> either. It seems to me that it's likely Debian will need to cope with eggs. Phillip> But if you'd like for your system packages of Python project Phillip> *not* to have to be duplicated as separately distributed Phillip> eggs, I personally would appreciate that not being the case Phillip> then we need to have some dependency info, so that your Phillip> packages can play in those areas where you *aren't* providing Phillip> a complete solu
Re: [Distutils] formencode as .egg in Debian ??
At 09:30 AM 11/25/2005 +1100, David Arnold wrote: -->"Phillip" == Phillip J Eby <[EMAIL PROTECTED]> writes: Phillip> Python developers would *love* to have Debian manage their Phillip> packages, they would simply like to be able to verify at Phillip> runtime that the management has in fact been done. It's not Phillip> that we don't trust you, it's just that we're paranoid. :) From a Debian perspective, that's strictly your own affair. Debian packages "just work", and if they don't it's a bug. There's no need for anyone other than the packager to worry about this. And if we were Debian developers rather than Python developers, this would be a sensible approach to take. As a practical matter, we have to worry, because our users aren't strictly Debian users, and asking them all to move to Debian isn't practical for most of us. ;) Let me ask you this: does Debian ban people from putting $id$ strings in C code to allow identifying the version of the source that was used to build a library or executable? Because that's what *exactly* what we're talking about here in respect to dependencies, just for Python code instead of C. Phillip> We'd like for Debian to advertise to our packages, precisely Phillip> what versions of which of our dependencies are installed. In general, I don't see why packages should care. Either it works, in which case there's no problem, or it doesn't, in which case it's a packaging bug, and it will be fixed. But we can't tell if it's a packaging bug, or a bug against what version of our dependencies, unless we know what version the user is running. Given that sys.path can include *both* Debian system packages and local user-installed packages, we need a more comprehensive way of dealing with the issue, which is what eggs provide. Supporting the installation and simultaneous use of multiple versions seems to be a goal of eggs? Absolutely. FWIW, that's kinda heretical on Unix (which might explain some of the antipathy). The Python approach is "Practicality Beats Purity", which I thought was also a Unix value. Phillip> We'd also like for Debian to include the metadata we provided Phillip> with our packages, when it installs them. It think that's a reasonable expectation. So long as Python packages are location-agnostic, and allow Debian to put such things where it thinks is reasonable? As long as it can be found by the package, without needing any special configuration. Python packages are looked up relative to sys.path, so it's reasonable to expect to find the metadata on sys.path as well, preferably at a fixed location relative to the code. Phillip> And we'd like all this to cleanly work with any Phillip> locally-installed non-Debian eggs that might be in the mix, Phillip> since we need to do development, beta testing, etc. And non-egg packages as well, right? There isn't any such thing, from an egg developer's perspective. Any distutils package can be made into an egg, because all of the metadata needed is supplied by the standard distutils setup script. So, if you have the source, you can make it an egg. With respect to the whole "Dependencies are a solved problem" thing, you're beating a dead horse. I freely admit you can manage *system* dependencies better. But that's only *part* of an application developer's problem. That's no critique of your efforts, only a recognition of their scope. For example, it doesn't help somebody developing a package while using Subversion-head versions of three other packages. It doesn't help Windows users. There's a whole bunch of things it just doesn't help with, that eggs do. That doesn't mean you should change Debian to eggs, because eggs aren't trying to solve the problems that you guys solve either. But if you'd like for your system packages of Python project *not* to have to be duplicated as separately distributed eggs, then we need to have some dependency info, so that your packages can play in those areas where you *aren't* providing a complete solution, but *can* provide a part of it. Thus, maybe the issue for some is that supplying the dependency information would be an admission that there really *are* problems Debian doesn't solve, as opposed to "merely" helping with. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
-->"Phillip" == Phillip J Eby <[EMAIL PROTECTED]> writes: Phillip> Python developers would *love* to have Debian manage their Phillip> packages, they would simply like to be able to verify at Phillip> runtime that the management has in fact been done. It's not Phillip> that we don't trust you, it's just that we're paranoid. :) >From a Debian perspective, that's strictly your own affair. Debian packages "just work", and if they don't it's a bug. There's no need for anyone other than the packager to worry about this. Of course, the application is free to do its own verification of this, but it would need to use an internal mechanism: since there's no *need* for it, Debian doesn't expose this information from its internal dependency database. Phillip> We'd like for Debian to advertise to our packages, precisely Phillip> what versions of which of our dependencies are installed. In general, I don't see why packages should care. Either it works, in which case there's no problem, or it doesn't, in which case it's a packaging bug, and it will be fixed. Of course, when supporting packages fail to provide a stable API, this means you end up needing to install multiple versions, or have the application adapt internally. Supporting the installation and simultaneous use of multiple versions seems to be a goal of eggs? FWIW, that's kinda heretical on Unix (which might explain some of the antipathy). Phillip> We'd also like for Debian to include the metadata we provided Phillip> with our packages, when it installs them. It think that's a reasonable expectation. So long as Python packages are location-agnostic, and allow Debian to put such things where it thinks is reasonable? Phillip> And we'd like all this to cleanly work with any Phillip> locally-installed non-Debian eggs that might be in the mix, Phillip> since we need to do development, beta testing, etc. And non-egg packages as well, right? But again, I don't see why that would be a problem. Perhaps it's worth pointing out: Debian users typically don't recognise dependency management as a problem, because it's not a problem for them. If you can't apt-get something, the choice is to wait a month for it to arrive in unstable, or to enter a world of pain where you're dealing with other (often conflicting) dependency management systems or no dependency management at all (and btw, this is the commonality I think eggs have with CPAN and PEAR). Debian developers take their role of banishing this problem seriously, which I suppose is why this is a contentious topic. d -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
At 12:39 PM 11/24/2005 -0800, Robert Kern wrote: I'm not suggesting that /usr/share/.../ should be the only place to find .egg-info directories. Simply that pkg_resources would scan sys.path+['/usr/share/.../'] and treat the ones found in /usr/share/.../ as if they were in /usr/lib/pythonX.Y/site-packages/. Everywhere else that a user could put Python packages (e.g. /usr/local/lib/pythonX.Y/site-packages/) is fine to put whatever the user wants. Eggs, .egg-info/; it doesn't matter. It doesn't need an alternate location. Debian packages need to be more disciplined, but all Debianized Python packages are installed to /usr/lib/pythonX.Y/site-packages/ so we only need the one alias, not many. It's possible that only the DD in charge of packaging setuptools will have to concern himself with coding this. Or they can just put the /usr/share/whatever directory on sys.path just after site-packages, with no need to modify setuptools then at all. This solution would work today even with a locally-installed (including user-home-directory-installed) version of setuptools. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
At 07:54 AM 11/25/2005 +1100, David Arnold wrote:
From a Python-centric viewpoint, Debian's (and RedHat, Gentoo, Solaris,
etc) packaging mechanism, however great, covers only one of the many
possible platforms that an application might need to support.
From a Debian-centric viewpoint, dependency management is a solved
problem, with a decade of experience behind it, and proposing that
Python applications should have their dependencies managed separately is
blatantly foolhardy.
I prefer a slightly different spin:
Python developers would *love* to have Debian manage their packages, they
would simply like to be able to verify at runtime that the management has
in fact been done. It's not that we don't trust you, it's just that we're
paranoid. :) We'd like for Debian to advertise to our packages, precisely
what versions of which of our dependencies are installed. We'd also like
for Debian to include the metadata we provided with our packages, when it
installs them. And we'd like all this to cleanly work with any
locally-installed non-Debian eggs that might be in the mix, since we need
to do development, beta testing, etc.
I'm not convinced that's the case (I'm not sure how reasonable it is to
automatically convert 'require("FooBar>=1.2")' into the equivalent
Debian package names and versions, but my expectation is that it's no
more difficult than it currently is to package a "product" for Debian.
The 'FooBar' name generally refers to a well-known name registered with
PyPI, and both the name and version come from metadata that is already
supplied by setup.py for all distributions packaged with the distutils
today. So you actually have this information *already*, for any
distutils-based distribution you package with Debian.
For the Python developer's purposes (i.e., verifying whether something is
there), you do not *need* to match this up with Debian package naming,
however. You only need to do that when you are packaging something that
uses such dependencies, and as you say, it's probably not any harder than
the dependency mapping you do now. (Except for the part where egg-based
projects tend to have more dependencies simply because they *can*,
encouraging reuse instead of reinvention.)
Anyway, for advertising the presence of dependencies, we only need a
standardized way for distributors to say, "FooBar 1.2 is installed
here". If a developer's project needs FooBar 1.2, he's not really worried
about what Debian package provides it, and the egg runtime isn't in the
business of asking system packagers to install things. It just "properly
complains" (as one person put it) about the absence of things that should
be there.
Finally, this is ignoring other possible uses of egg metadata, which
might end up being far more problematic?
I don't see why they would be any more problematic than any other kind of
data that comes with Python packages. Again, we just need to be able to
read this information at runtime to discover services, plugins, etc.
without requiring user configuration besides just installing things.
Here's a motivating example of what this kind of data can be used
for. Suppose I said, "okay, I don't see how we can satisfy the Debian
requirements cleanly, so suppose I make setuptools allow plugins to
redefine how the metadata gets found?"
I would then define an entry point group, let's say
"setuptools.dependency_resolvers", and add some code to query these entry
points when scanning for egg metadata. Debian could then create a separate
Python package, and include in its metadata an entry point for this entry
point group, which would tell setuptools where to import the plugin code
from. Debian would then make sure this package and its metadata are
discoverable to setuptools, and setuptools would then automatically do the
right thing on any Debian installation, even if the user was running with a
new, locally-installed version of setuptools.
Of course, this scenario wouldn't actually work due to the bootstrapping
issue. I just mention it so you can see an example of why packages want to
be able to discover metadata besides the mere presence of a dependency. In
this example, the hypothetical Debian-written plugin need not be known
about by the provider of the plugin point, i.e. setuptools.
Anyway, the point is that we simply need a way to link that kind of data to
sys.path entries, so we can be sure that the things we need are there, and
not shadowed by things that are before them on sys.path.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
At 09:10 PM 11/24/2005 +0100, Josselin Mouette wrote: A sane way, first of all, means a consistent way. Having two sorts of Debian python packages is a no-go. Therefore, if we want to switch to a new way of distributing packages, there has to be some serious grounds for it. Currently, the picture I have of eggs doesn't show any advantages over the plain old distutils we are currently using. What advantage does the internet have? It lets you be connected to the internet. Eggs are a network, packaging them lets your packages be part of that network. That's the only *possible* benefit to eggs from a system packager's point of view, which is why it has never been my policy to go to system packagers and ask them to include eggs. Instead, I leave it to their users to ask, because it's really none of my business whether a distributor packages eggs. That's between you and your users. I only get involved if someone asks for help in packaging the eggs, and I help anybody who asks. Which is why I find this discussion so frustrating; somehow or other, people keep spinning this back to some kind of advocacy thing. If you want to provide eggs, great. You don't want to provide them, great. It really makes no difference to me; it's an issue between you and your users. I'm only here to help if you *do* want to provide eggs and have issues with *how* to do it in a way that works for you. [snip items already discussed that have technical solutions already arrived at] A sane way means compliance with standards, especially the filesystem hierarchy standard. When some people are trying to separate .py and .pyc/.pyo files to respect it, you're asking to put them in a single archive. PEP 304 has been languishing for years, apparently due to lack of interest. In any case, its motivation was never FHS conformance, as far as I can tell. More immediately relevant, I'm 1) not asking anybody to do anything, and 2) an archive is only one of three already-valid ways to lay out the installation of an egg. > 1. Egg-based projects need to install their published metadata, in a > well-known location relative to the installation location of their code, so > that it can be found by searching sys.path, so that it and other projects > can locate the metadata for currently-importable projects, *without* > needing to first import the project's code. > > 2. Egg-based projects need to be able to identify whether another Python > project package is installed and what version it is, without requiring > modification to that other project's code or needing to import it. (And > this is independent of whether the depended-on project was packaged as an > egg by its author.) > > As far as I'm aware, those are the irreducible technical minimum > requirements for making an egg-based project work. *How* these > requirements are met is quite flexible, as there are already three working > layouts that achieve this. As I said before, I'm quite willing to > implement a fourth. But nobody has been proposing anything that meets > these requirements, because they're too busy trying to prove the > requirements don't exist or are somehow not real. Alright, let me try to propose something. How about defining a place in the python distribution, say, /usr/share/python2.x/eggs (but of course configurable when installing python), where this information should go? An installed package would look like a directory structure of .py files in /usr/lib/python2.x/site-packages, and a metadata file in /usr/share/python2.x/eggs. See line 2 of item 1 above; we need a "well-known location located relative to the installation location of their code". Python programs have limited access to knowledge about platform directory structure, except as expressed in sys.path. Also, they need to be able to support development environments, application plugin directories, and other sys.path manipulation. So, ideally the approach needs to be able to be applied uniformly to each sys.path entry. One of the reasons for this is that it allows the runtime to reason about the contents of sys.path. If the metadata is in a particular directory, it knows that by adding or removing that directory from sys.path, what packages will be added or removed from importability, and it also knows which packages may be shadowing other packages. In the case of namespace packages, it knows what subdirectories of sys.path directories need to be added to the package's in-memory __path__ in order to merge separately-distributed components into a common virtual package. These goals seem unachievable with an absolute location that's divorced from any direct connection to a sys.path entry, although there are a couple of workarounds possible. See below. In fact, this approach is quite similar to the .egg-info approach, but it would be better to put the files at the right place from the very beginning, and, more importantly, to be able to deal wit
Re: [Distutils] formencode as .egg in Debian ??
-->"Phillip" == Phillip J Eby <[EMAIL PROTECTED]> writes:
Phillip> I've only been getting the barest hint of what "crappy
Phillip> packaging" consists of, except for the loud-and-clear message
Phillip> that it's defined as Anything But Debian. Since I'm
Phillip> providing for users beyond Debian, that isn't useful
Phillip> information and doesn't help me investigate fixing any
Phillip> alleged crappiness.
>From here, the problem looks like:
Eggs provide a way of managing dependencies that is generic to Python,
across all platforms. From a Python developer or user's viewpoint, this
is great.
Debs (and RPMs, and ebuilds, and PKGs, and so on) provide a way of
managing dependencies that are generic to a Unix distribution. From a
Unix developer or user's viewpoint, this is great.
For each of these groups, the alternative approach is problematic.
>From a Python-centric viewpoint, Debian's (and RedHat, Gentoo, Solaris,
etc) packaging mechanism, however great, covers only one of the many
possible platforms that an application might need to support.
>From a Debian-centric viewpoint, dependency management is a solved
problem, with a decade of experience behind it, and proposing that
Python applications should have their dependencies managed separately is
blatantly foolhardy.
I kinda have a foot in both camps: I use Debian as my desktop, but I
produce Python apps and "libraries" for both Unix and Windows users.
So far as I know, Windows users don't have a standard way to manage
dependencies. And to be fair, Windows users are far more numerous than
those of any Unix/Linux distribution. So I can see the appeal of eggs,
both for a popular platform, and as a Python developer, as a
cross-platform do-it-once-only, mechanism.
But, as I've said (and I believe Phillip has acknowledged), from a Unix
point of view, this is problematic.
Tools to convert Python-centric, egg-managed dependency information into
Debian/Red Hat/Gentoo/Solaris/etc dependency information would appear to
help bridge this gap.
I'm not convinced that's the case (I'm not sure how reasonable it is to
automatically convert 'require("FooBar>=1.2")' into the equivalent
Debian package names and versions, but my expectation is that it's no
more difficult than it currently is to package a "product" for Debian.
My point here is: both sides of this discussion have a valid point. If
eggs are going to see the claimed rapid adoption (and I have no reason
to doubt the claims that this is happening), then systems that have an
existing dependency management system will need to cope. Assistance
from the Egg People to make this happen can only be useful.
Finally, this is ignoring other possible uses of egg metadata, which
might end up being far more problematic?
d
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: formencode as .egg in Debian ??
Phillip J. Eby wrote: > Note, by the way, that those two things are the only essentials here, as > best I can tell, and I've already stated my willingness to change *how* > those two things get accomplished. For clarity, I will repeat yet again, > in yet another way: > > 1. Egg-based projects need to install their published metadata, in a > well-known location relative to the installation location of their code, so > that it can be found by searching sys.path, so that it and other projects > can locate the metadata for currently-importable projects, *without* > needing to first import the project's code. > > 2. Egg-based projects need to be able to identify whether another Python > project package is installed and what version it is, without requiring > modification to that other project's code or needing to import it. (And > this is independent of whether the depended-on project was packaged as an > egg by its author.) > > As far as I'm aware, those are the irreducible technical minimum > requirements for making an egg-based project work. *How* these > requirements are met is quite flexible, as there are already three working > layouts that achieve this. As I said before, I'm quite willing to > implement a fourth. But nobody has been proposing anything that meets > these requirements, because they're too busy trying to prove the > requirements don't exist or are somehow not real. [Note: I am a happy Debian user, though not a DD. I am also one of the developers of a Debian-packaged Python package, and we're considering using pkg_resources to implement certain new features. I swear, this is like watching two parents fight. Anyways...] I think one of the sticking points with the Debian developers has been that the .egg-info metadata is being put into /usr/lib/... when according to Debian policy and general UNIX lore, such should be placed somewhere in /usr/share/ Would it be possible to treat /usr/share/pythonX.Y-egginfo/ as a proxy for /usr/lib/pythonX.Y/site-packages/ when searching for .egg-info directories? -- Robert Kern [EMAIL PROTECTED] "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
At 11:36 AM 11/24/2005 -0800, Robert Kern wrote: Phillip J. Eby wrote: > Note, by the way, that those two things are the only essentials here, as > best I can tell, and I've already stated my willingness to change *how* > those two things get accomplished. For clarity, I will repeat yet again, > in yet another way: > > 1. Egg-based projects need to install their published metadata, in a > well-known location relative to the installation location of their code, so > that it can be found by searching sys.path, so that it and other projects > can locate the metadata for currently-importable projects, *without* > needing to first import the project's code. > > 2. Egg-based projects need to be able to identify whether another Python > project package is installed and what version it is, without requiring > modification to that other project's code or needing to import it. (And > this is independent of whether the depended-on project was packaged as an > egg by its author.) > > As far as I'm aware, those are the irreducible technical minimum > requirements for making an egg-based project work. *How* these > requirements are met is quite flexible, as there are already three working > layouts that achieve this. As I said before, I'm quite willing to > implement a fourth. But nobody has been proposing anything that meets > these requirements, because they're too busy trying to prove the > requirements don't exist or are somehow not real. [Note: I am a happy Debian user, though not a DD. I am also one of the developers of a Debian-packaged Python package, and we're considering using pkg_resources to implement certain new features. I swear, this is like watching two parents fight. Anyways...] I think one of the sticking points with the Debian developers has been that the .egg-info metadata is being put into /usr/lib/... when according to Debian policy and general UNIX lore, such should be placed somewhere in /usr/share/ Would it be possible to treat /usr/share/pythonX.Y-egginfo/ as a proxy for /usr/lib/pythonX.Y/site-packages/ when searching for .egg-info directories? The sticking point is the relative relationship to a sys.path directory (line 2 of requirement 1, above). Since a program such as Trac can legitimately manipulate sys.path to e.g. add plugin directories, we need to be able to do some reasonable transformation to *every* sys.path entry, preferably without introducing some kind of distro-specific configuration to match special directories and redirect them. We really want to treat every sys.path entry as a valid place to find a project's metadata, since each is a valid place for the actual packages to live. There is of course also a performance issue; currently we need only scan each sys.path directory at most once in order to locate all packages and their metadata. (.egg directories don't need to be scanned unless looking for a dependency that's not already on sys.path.) The scan of non-.egg directories identifies any .egg-info directories there. So, for a cost of zero or one listdir() for each sys.path entry, we can find the locations of all the discoverable metadata. However, if the issue here is where the data files actually live, couldn't Debian just put the egg info wherever it wants, and symlink to it from site-packages? That would already work with the current runtime, although it seems to me like it's just making more work for the Debian packagers, for no benefit that I comprehend. (Especially since in the case of non-setuptools packages, it might be just an empty file anyway.) Note that Python packages have always been able to include data files inside their package directories, and in fact the *single most popular* distutils modification made by published Python packages is to *not* install data under /usr/share and instead put those files in the packages' directories, where the package can find it without needing to know the platform-specific or distro-local place for data. This is why setuptools pioneered an easier way to add package data almost 2 years ago, and that way was officially backported into the distutils in Python 2.4. See: http://python.org/doc/2.4.1/dist/node11.html Anyway, symlinks would work, but it seems to me rather inconsistent to push project-level data to /usr/share while leaving module-level data in /usr/lib. And pushing either one of the two to /usr/share seems like wasted effort on a non-problem to me. In neither case is the data going to be changed; it can be just as read-only as the code is. (But just because I don't think it's useful, I'm not going to act like it's not a real problem for the Debian team, or insist that putting files in /usr/share is undesirable. If it's desirable to them, it's desirable to them. The only relevant question is how can we both get our desires met.) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECT
Re: [Distutils] formencode as .egg in Debian ??
Le jeudi 24 novembre 2005 à 14:13 -0500, Phillip J. Eby a écrit : > On the contrary, quite useful technical discussion about how to make this > work has taken place, including proposed changes *which I have accepted* > and will implement. You just haven't been participating in any of that > discussion. Indeed, I arrived in the discussion a bit late, and didn't want to intervene before having read it all. However, the implementation that you proposed isn't clear from those messages. Putting things out of the zip archives and having a separate file for metadata is obviously a good thing. However, I'm still concerned about things not functioning without this metadata, and the requirement for adding metadata in packages that don't need it. As I still don't get the point of including this metadata in the package, I'd have a hard time proposing other implementations. > >My tone is aggressive because it's easy to feel irritated by so much > >foolishness, but on the grounds, this is still a request: please, change > >your distribution system so that it's possible to distribute your > >software in a sane way. > > Please help me do that by explaining "sane way" on grounds that don't > resort to religion. Please explain why including a package's *own > metadata* that the author *wished to include* is not sane. Please explain > why providing a consistent way (i.e. a cross-distribution, cross-platform > way) to identify what Python libraries are installed on a system is not > "sane". A sane way, first of all, means a consistent way. Having two sorts of Debian python packages is a no-go. Therefore, if we want to switch to a new way of distributing packages, there has to be some serious grounds for it. Currently, the picture I have of eggs doesn't show any advantages over the plain old distutils we are currently using. A sane way means, when there are meaningful files that are accessed by the interpreter in an easily understandable way, to ship them as is in a package, not to hide them in an archive. A sane way means not to alter functionality of other software. For example, impacting the overall python startup time by making it unzip all eggs is not sane, even if the impact is currently small. Modifying the default sys.path enters in the same category. A sane way means compliance with standards, especially the filesystem hierarchy standard. When some people are trying to separate .py and .pyc/.pyo files to respect it, you're asking to put them in a single archive. > 1. Egg-based projects need to install their published metadata, in a > well-known location relative to the installation location of their code, so > that it can be found by searching sys.path, so that it and other projects > can locate the metadata for currently-importable projects, *without* > needing to first import the project's code. > > 2. Egg-based projects need to be able to identify whether another Python > project package is installed and what version it is, without requiring > modification to that other project's code or needing to import it. (And > this is independent of whether the depended-on project was packaged as an > egg by its author.) > > As far as I'm aware, those are the irreducible technical minimum > requirements for making an egg-based project work. *How* these > requirements are met is quite flexible, as there are already three working > layouts that achieve this. As I said before, I'm quite willing to > implement a fourth. But nobody has been proposing anything that meets > these requirements, because they're too busy trying to prove the > requirements don't exist or are somehow not real. Alright, let me try to propose something. How about defining a place in the python distribution, say, /usr/share/python2.x/eggs (but of course configurable when installing python), where this information should go? An installed package would look like a directory structure of .py files in /usr/lib/python2.x/site-packages, and a metadata file in /usr/share/python2.x/eggs. In fact, this approach is quite similar to the .egg-info approach, but it would be better to put the files at the right place from the very beginning, and, more importantly, to be able to deal with other packages not having such .egg-info files. > Since I'm neither a Debian nor Mozilla developer, I have no way to know > what these problems you speak of are, nor any way to tell whether the > alleged flaws of the Mozilla packaging relate in any meaningful way to the > alleged flaws of eggs. It's you who's in the position of giving advocacy > without providing any real information about the issues. This is true. In short, my two main concerns about mozilla-related packaging are total absence of library versioning, and the shipping of extensions in .xpi files, which are tarballs containing information and meta-information about the package. You can surely understand that my concerns about eggs approach those of .xpi files. > It's plain t
Re: [Distutils] formencode as .egg in Debian ??
Am 24.11.2005 um 19:27 schrieb Josselin Mouette: Le jeudi 24 novembre 2005 à 12:49 -0500, Phillip J. Eby a écrit : I don't really care if you accept the proposals or not; you guys need to do whatever you think is best for Debian. I've only tried to educate you about your options regarding eggs, framed within the assumption that you *want* to distribute egg-based projects. The question of eggs' goodness or lack thereof is entirely irrelevant within that frame. Either you want to distribute those projects or you don't. If you don't, then this discussion is pointless. If you do, then arguing that people shouldn't use eggs is equally pointless. The only discussion that has any point is *how* we can get the needs of both Debian and the project developers met. Indeed. I'm sure no sane Debian developer would want to distribute egg-based projects if there was another way to distribute them, but they are here and people will want them. You are making things worse by advocating the egg way of doing things, and the only thing I can do if you don't stop is to urge developers to think twice before shipping their projects this way. Maybe it will make their development a bit more difficult, but it will improve the situation for their users. Eggs are not really a way to *ship* Python projects. It is a way to deploy Python projects so that they can provide discoverable meta- data at runtime. This meta-data can be used in many ways. An example: for the latest release of the Trac project management web-app we implemented a simple plugin system that is based on Python eggs. It uses the egg meta-data to discover other Python projects that provide plugins for itself. Thus, this meta-data is essential for using plugins in Trac, and if a package is installed without this meta-data, well, you could've just as well not installed it at all. Sure, every Python project that wanted to be extensible could just reinvent the wheel here and implement its own system for discovering and activating plugins. But Python eggs give us a unified solution to this problem, and a very nice one, too. The problems this approach may raise for package maintainers need to be discussed and addressed. But to simply say that eggs are a "crappy" technology is just ignorant. Cheers, Chris -- Christopher Lenz cmlenz at gmx.de http://www.cmlenz.net/
Re: [Distutils] formencode as .egg in Debian ??
At 07:27 PM 11/24/2005 +0100, Josselin Mouette wrote: Looks like I mistakenly hoped it was possible to change things on technical grounds, but if you're rebutting any technical discussion as "irrelevant", there's probably not much to do. On the contrary, quite useful technical discussion about how to make this work has taken place, including proposed changes *which I have accepted* and will implement. You just haven't been participating in any of that discussion. My tone is aggressive because it's easy to feel irritated by so much foolishness, but on the grounds, this is still a request: please, change your distribution system so that it's possible to distribute your software in a sane way. Please help me do that by explaining "sane way" on grounds that don't resort to religion. Please explain why including a package's *own metadata* that the author *wished to include* is not sane. Please explain why providing a consistent way (i.e. a cross-distribution, cross-platform way) to identify what Python libraries are installed on a system is not "sane". Note, by the way, that those two things are the only essentials here, as best I can tell, and I've already stated my willingness to change *how* those two things get accomplished. For clarity, I will repeat yet again, in yet another way: 1. Egg-based projects need to install their published metadata, in a well-known location relative to the installation location of their code, so that it can be found by searching sys.path, so that it and other projects can locate the metadata for currently-importable projects, *without* needing to first import the project's code. 2. Egg-based projects need to be able to identify whether another Python project package is installed and what version it is, without requiring modification to that other project's code or needing to import it. (And this is independent of whether the depended-on project was packaged as an egg by its author.) As far as I'm aware, those are the irreducible technical minimum requirements for making an egg-based project work. *How* these requirements are met is quite flexible, as there are already three working layouts that achieve this. As I said before, I'm quite willing to implement a fourth. But nobody has been proposing anything that meets these requirements, because they're too busy trying to prove the requirements don't exist or are somehow not real. Don't become like Mozilla. You can't imagine how much Debian time is wasted because Mozilla developers don't care about the Unix way of doing things. We are receiving weekly complaints of users about the quality of mozilla packages, and there's nothing we can do for them. Don't make python-related Debian maintainership look the same. Since I'm neither a Debian nor Mozilla developer, I have no way to know what these problems you speak of are, nor any way to tell whether the alleged flaws of the Mozilla packaging relate in any meaningful way to the alleged flaws of eggs. It's you who's in the position of giving advocacy without providing any real information about the issues. It's plain that you don't want crappy packaging. But I've only been getting the barest hint of what "crappy packaging" consists of, except for the loud-and-clear message that it's defined as Anything But Debian. Since I'm providing for users beyond Debian, that isn't useful information and doesn't help me investigate fixing any alleged crappiness. It would be especially helpful to define what *specifically* are the problems, rather than continuing all this vague handwaving about elegance and sanity and the unspecified crappiness of all things non-Debian. So far, every argument except startup performance and the length of sys.path has seemed to boil down to, "we don't like it so you should stop," and "we don't trust package developers to specify their own dependencies, so anything that allows it is evil". Those aren't useful arguments, because there is nothing I can *do* about them. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
Le jeudi 24 novembre 2005 à 13:05 -0500, Phillip J. Eby a écrit : > You seem to be confusing "users" with "Debian packagers" and "Debian > users", which are a subset of "users" where these projects are > concerned. TurboGears targets Mac and Windows as well as Linux, and I've > seen people on the TurboGears mailing list using at least three major > packaging *tools* (e.g. .rpm, .deb, etc.), to say nothing of the count of > Linux *distributions*. Obviously, Kevin *was* thinking of the users - eggs > are the only thing that would let his project reach so many of them. Of course, I'm talking about Unix users. You seem to have already taken care of Windows users. > >Again, the ability to distribute it as a single package is good, > > What "single package" are you talking > about? http://turbogears.org/download/ lists eggs and source packages for > *10* different Python projects that it depends on, written by five > different authors. Each is individually packaged, with eggs for Mac and > Windows, and source packages that can build eggs for everything else. You're reinventing the wheel. Really. This isn't a matter for the Windows world, as users are accustomed to various, broken tools to install stuff, and this one will probably be less broken than others. But MacOS and Linux distributions already have a unified packaging system, and I don't think it's useful to invent another package type, especially if it's only suitable for Python. > >Yes, having that information is good. But there should be a way to > >ignore it. Simply. Nothing more, nothing less. If egg-enabled packages > >can also work without all this extra stuff, there's no problem for the > >distributor. > > What you're saying is, you want TurboGears to be able to blindly trust that > its dependencies are installed. YES. Please. > This doesn't help users, though, because > it keeps the package author from being able to provide *end-user support* > without learning the ins and outs of every distro and packaging system, so > he can tell the user what to run to find out if the dependencies are > *really* met. No. Distributions all have their own bug-reporting tool, so that distribution-related issues can be filtered out before such issues are brought to you. > The funny thing here is that just the .egg names *alone* have been wildly > useful; when a user posts an error message to the TurboGears mailing list, > you can tell right away exactly what the project versions are for every > piece of code in the stack trace. It dramatically cuts down on the number > of, "Do you have version X of Y?" questions. Same answer: at least the Debian bug reporting tool does it. We are not trying to make your work harder, only to make our work easier. > Being able to support users is good for users. Being able to provide lots > of functionality using off-the-shelf libraries that have their own > documentation is good for users, too. This is *all* about the users. And making security bugs unfixable is good for users? We probably don't have the same users, indeed. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom signature.asc Description: Ceci est une partie de message numériquement signée
Re: [Distutils] formencode as .egg in Debian ??
Le jeudi 24 novembre 2005 à 12:49 -0500, Phillip J. Eby a écrit : > I don't really care if you accept the proposals or not; you guys need to do > whatever you think is best for Debian. I've only tried to educate you > about your options regarding eggs, framed within the assumption that you > *want* to distribute egg-based projects. The question of eggs' goodness or > lack thereof is entirely irrelevant within that frame. Either you want to > distribute those projects or you don't. If you don't, then this discussion > is pointless. If you do, then arguing that people shouldn't use eggs is > equally pointless. The only discussion that has any point is *how* we can > get the needs of both Debian and the project developers met. Indeed. I'm sure no sane Debian developer would want to distribute egg-based projects if there was another way to distribute them, but they are here and people will want them. You are making things worse by advocating the egg way of doing things, and the only thing I can do if you don't stop is to urge developers to think twice before shipping their projects this way. Maybe it will make their development a bit more difficult, but it will improve the situation for their users. > Bluntly put, it's Debian that's asking me to change my practices (and those > of egg-based project authors) to suit their needs, not the other way > around. You seem to be under some delusion that we need you or want you to > support those projects, when it's actually the existence of eggs that > allows us to not care whether *you* exist. We wouldn't be having this > discussion if somebody from your group hadn't asked for *our* help, so that > Debian can play in *our* sandbox. All this discussion about the relative > elegance of different packaging mechanisms is irrelevant. It seems that you can only think about things in a competitive way. "Use our stuff or die.", or "Change your way of doing things or you'll die.". Looks like I mistakenly hoped it was possible to change things on technical grounds, but if you're rebutting any technical discussion as "irrelevant", there's probably not much to do. I know what will happen now: people will package your crappy eggs, because users want the software that's inside. Packages will be crappy and users will complain, and you'll tell them their distribution sucks. And what do we get? Nothing interesting. If you don't want to do anything for users, it's pointless to try to discuss with you. My tone is aggressive because it's easy to feel irritated by so much foolishness, but on the grounds, this is still a request: please, change your distribution system so that it's possible to distribute your software in a sane way. Don't become like Mozilla. You can't imagine how much Debian time is wasted because Mozilla developers don't care about the Unix way of doing things. We are receiving weekly complaints of users about the quality of mozilla packages, and there's nothing we can do for them. Don't make python-related Debian maintainership look the same. Regards, -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom signature.asc Description: Ceci est une partie de message numériquement signée
Re: [Distutils] formencode as .egg in Debian ??
At 06:13 PM 11/24/2005 +0100, Josselin Mouette wrote: Le jeudi 24 novembre 2005 à 11:43 -0500, Phillip J. Eby a écrit : > That's an interesting perspective, but it's viewing the world through > vendor-colored glasses. Unless the project developer is wearing similar > glasses (i.e., has decided to commit to Debian as their sole platform), > though, it's not a practical one. From the point of view of people like > the author of TurboGears, it's the egg dependency system that allows them > not to have to worry about which packaging system the user has, or doesn't > have. > > I mention this not to be disagreeable, just to point out that the world in > which egg dependencies are of no benefit, needless complexity, etc., is not > the same world in which the project developers using eggs live. As I understand your explanation, developers who are using eggs are doing it so simplify *their* work, without a thought for users. Tools that simplify development processes are good and should be encouraged, but not if it means extra burden for users. One of the primary priorities of the Debian project is its users. Obviously this isn't yours. You seem to be confusing "users" with "Debian packagers" and "Debian users", which are a subset of "users" where these projects are concerned. TurboGears targets Mac and Windows as well as Linux, and I've seen people on the TurboGears mailing list using at least three major packaging *tools* (e.g. .rpm, .deb, etc.), to say nothing of the count of Linux *distributions*. Obviously, Kevin *was* thinking of the users - eggs are the only thing that would let his project reach so many of them. > Case in point: this thread began because somebody wanted to package > TurboGears and its dependencies for Debian. But that project wouldn't have > been viable without the egg system already existing, and there was > certainly no way for TurboGears to have started its life as anything but a > "non-system Python package". One reason TurboGears is popular is because > it's well-supported, and it's well-supported in part because it can > "complain properly" (as you describe it above) no matter what platform it's > running on. Again, the ability to distribute it as a single package is good, What "single package" are you talking about? http://turbogears.org/download/ lists eggs and source packages for *10* different Python projects that it depends on, written by five different authors. Each is individually packaged, with eggs for Mac and Windows, and source packages that can build eggs for everything else. > Again, I don't think you're "wrong" to think that egg dependencies are > redundant - from within your particular point of view. But you need to > understand that for *Python* developers, being able to practically depend > on other packages in a cross-platform way is a new and powerful feature > which is *not* provided by Debian or any other packaging system, unless > it's wrapping eggs. So, from your perspective, you already have this > feature, but for projects using eggs, you really *don't* have the feature, > because the data is not economically accessible to them. Yes, having that information is good. But there should be a way to ignore it. Simply. Nothing more, nothing less. If egg-enabled packages can also work without all this extra stuff, there's no problem for the distributor. What you're saying is, you want TurboGears to be able to blindly trust that its dependencies are installed. This doesn't help users, though, because it keeps the package author from being able to provide *end-user support* without learning the ins and outs of every distro and packaging system, so he can tell the user what to run to find out if the dependencies are *really* met. The funny thing here is that just the .egg names *alone* have been wildly useful; when a user posts an error message to the TurboGears mailing list, you can tell right away exactly what the project versions are for every piece of code in the stack trace. It dramatically cuts down on the number of, "Do you have version X of Y?" questions. Being able to support users is good for users. Being able to provide lots of functionality using off-the-shelf libraries that have their own documentation is good for users, too. This is *all* about the users. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
At 06:03 PM 11/24/2005 +0100, Josselin Mouette wrote: Le jeudi 24 novembre 2005 à 11:46 -0500, Phillip J. Eby a écrit : > And finally, and most importantly, you're ignoring the fact that this > discussion began because a Debian developer wanted to package a successful > egg-using project and its dependencies. Nobody came to you asking for your > blessing to package eggs or your approval of how useful eggs are; somebody > came and asked me for help in getting this to work with Debian. Help which > I've been giving. > For some reason, however, you seem to think that you are actually in a > position to pass judgment on the usefulness of eggs. If their usefulness > were in any meaningful question, we wouldn't be having this conversation in > the first place! You have to bear with it. In the open source world, anyone can come and discuss the usefulness of your project. This won't happen if your project isn't causing trouble to other projects, but when you're talking about breaking the python packaging practises in Debian, I think any Debian developer has the moral right to discuss your suggestions. My point is that I am not the one who just dropped by and asked somebody to change their practices. I was off doing other things and was pulled into this discussion by someone asking for assistance. Every proposal I've made in this thread has been directed at providing that assistance. Your dislike of or disagreement with any particular proposal doesn't excuse your attitude. I don't really care if you accept the proposals or not; you guys need to do whatever you think is best for Debian. I've only tried to educate you about your options regarding eggs, framed within the assumption that you *want* to distribute egg-based projects. The question of eggs' goodness or lack thereof is entirely irrelevant within that frame. Either you want to distribute those projects or you don't. If you don't, then this discussion is pointless. If you do, then arguing that people shouldn't use eggs is equally pointless. The only discussion that has any point is *how* we can get the needs of both Debian and the project developers met. Why the hell would a package having nothing to do with yours would have to add a file - especially an empty file - to cope with it? Geez, you make this sound like I just wandered in off the street and started telling people to redesign their packaging system. Somebody asked for help in Debianizing TurboGears. If you want to Debianize TurboGears, you'll have to add such a file to at least ElementTree's Debian package. The proposal I made is my attempt at offering a solution that doesn't require you to completely repackage ElementTree, while still allowing it to be used by TurboGears. Bluntly put, it's Debian that's asking me to change my practices (and those of egg-based project authors) to suit their needs, not the other way around. You seem to be under some delusion that we need you or want you to support those projects, when it's actually the existence of eggs that allows us to not care whether *you* exist. We wouldn't be having this discussion if somebody from your group hadn't asked for *our* help, so that Debian can play in *our* sandbox. All this discussion about the relative elegance of different packaging mechanisms is irrelevant. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
Le jeudi 24 novembre 2005 à 11:43 -0500, Phillip J. Eby a écrit : > That's an interesting perspective, but it's viewing the world through > vendor-colored glasses. Unless the project developer is wearing similar > glasses (i.e., has decided to commit to Debian as their sole platform), > though, it's not a practical one. From the point of view of people like > the author of TurboGears, it's the egg dependency system that allows them > not to have to worry about which packaging system the user has, or doesn't > have. > > I mention this not to be disagreeable, just to point out that the world in > which egg dependencies are of no benefit, needless complexity, etc., is not > the same world in which the project developers using eggs live. As I understand your explanation, developers who are using eggs are doing it so simplify *their* work, without a thought for users. Tools that simplify development processes are good and should be encouraged, but not if it means extra burden for users. One of the primary priorities of the Debian project is its users. Obviously this isn't yours. > Case in point: this thread began because somebody wanted to package > TurboGears and its dependencies for Debian. But that project wouldn't have > been viable without the egg system already existing, and there was > certainly no way for TurboGears to have started its life as anything but a > "non-system Python package". One reason TurboGears is popular is because > it's well-supported, and it's well-supported in part because it can > "complain properly" (as you describe it above) no matter what platform it's > running on. Again, the ability to distribute it as a single package is good, but it has been done by breaking existing practise for python packages distribution. I have nothing against eggs, I have something against being forced to use eggs to distribute a python project. > Again, I don't think you're "wrong" to think that egg dependencies are > redundant - from within your particular point of view. But you need to > understand that for *Python* developers, being able to practically depend > on other packages in a cross-platform way is a new and powerful feature > which is *not* provided by Debian or any other packaging system, unless > it's wrapping eggs. So, from your perspective, you already have this > feature, but for projects using eggs, you really *don't* have the feature, > because the data is not economically accessible to them. Yes, having that information is good. But there should be a way to ignore it. Simply. Nothing more, nothing less. If egg-enabled packages can also work without all this extra stuff, there's no problem for the distributor. Regards, -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: [Distutils] formencode as .egg in Debian ??
Le jeudi 24 novembre 2005 à 11:46 -0500, Phillip J. Eby a écrit : > No, what's happening is that you're not paying attention, because you > believe that Debian already solves those problems, even though it doesn't > solve them for Python developers who want their projects to be usable on > every platform. While solving a "problem" (which problem?) for other systems, you've just created a new problem for all Linux distributions. > You're also ignoring my repeated explanations about how > plugins work, and why Debian doesn't help with that problem either. You haven't given any useful explanation so far. > And finally, and most importantly, you're ignoring the fact that this > discussion began because a Debian developer wanted to package a successful > egg-using project and its dependencies. Nobody came to you asking for your > blessing to package eggs or your approval of how useful eggs are; somebody > came and asked me for help in getting this to work with Debian. Help which > I've been giving. My preoccupation is to have a clean integration of all python packages in Debian. Something you've not been helpful about, proposing to change the clean, correct way python modules are packaged so that they can deal with your "improvements". > For some reason, however, you seem to think that you are actually in a > position to pass judgment on the usefulness of eggs. If their usefulness > were in any meaningful question, we wouldn't be having this conversation in > the first place! You have to bear with it. In the open source world, anyone can come and discuss the usefulness of your project. This won't happen if your project isn't causing trouble to other projects, but when you're talking about breaking the python packaging practises in Debian, I think any Debian developer has the moral right to discuss your suggestions. I have seen the nightmare that mozilla extensions packaging can become, and I urge maintainers of python-related packages not to rush into egg packaging, as it is obviously going to reach a similar level of insanity. > I've been answering Martin's posts to kill FUD and to make progress where > progress could be made. But your posts aren't spreading any FUD I haven't > already answered, and you're clearly not interested in making any progress, > so I don't see any point in wasting either of our time any further on this > thread. Nonetheless, I remain committed to helping any system integrator - > Debian included - in helping their users by allowing them to satisfy egg > dependencies using their existing packages, as well as in packaging > egg-based projects in ways that work, not only for the project, but for the > system integrator. You should demonstrate that will, instead of repeatedly proposing suggestions that are simply not implementable. > In other words, setuptools 0.6a9 will have the features that at least > Martin and Matthias have suggested will be an acceptable solution for > packaging projects that explicitly contain egg metadata (such as > FormEncode), and will also allow older Python packages to add dependency > metadata with a single empty file. Why the hell would a package having nothing to do with yours would have to add a file - especially an empty file - to cope with it? Again, intra-package dependencies is *not* something useful for us, distributors. They have repeatedly proved to be unreliable, in many contexts. Regards, -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: [Distutils] formencode as .egg in Debian ??
At 04:20 PM 11/24/2005 +0100, Josselin Mouette wrote: Le jeudi 24 novembre 2005 à 10:14 -0500, Phillip J. Eby a écrit : > At 01:44 PM 11/24/2005 +0100, Josselin Mouette wrote: > >They only introduce more complexity, instead of bringing real features. > > Please read the hundreds of kilobytes of messages I've already posted on > this thread I've read them. All of them. You're only babbling about how eggs are great, instead of explaining what problem they are trying to solve. No, what's happening is that you're not paying attention, because you believe that Debian already solves those problems, even though it doesn't solve them for Python developers who want their projects to be usable on every platform. You're also ignoring my repeated explanations about how plugins work, and why Debian doesn't help with that problem either. And finally, and most importantly, you're ignoring the fact that this discussion began because a Debian developer wanted to package a successful egg-using project and its dependencies. Nobody came to you asking for your blessing to package eggs or your approval of how useful eggs are; somebody came and asked me for help in getting this to work with Debian. Help which I've been giving. For some reason, however, you seem to think that you are actually in a position to pass judgment on the usefulness of eggs. If their usefulness were in any meaningful question, we wouldn't be having this conversation in the first place! I've been answering Martin's posts to kill FUD and to make progress where progress could be made. But your posts aren't spreading any FUD I haven't already answered, and you're clearly not interested in making any progress, so I don't see any point in wasting either of our time any further on this thread. Nonetheless, I remain committed to helping any system integrator - Debian included - in helping their users by allowing them to satisfy egg dependencies using their existing packages, as well as in packaging egg-based projects in ways that work, not only for the project, but for the system integrator. In other words, setuptools 0.6a9 will have the features that at least Martin and Matthias have suggested will be an acceptable solution for packaging projects that explicitly contain egg metadata (such as FormEncode), and will also allow older Python packages to add dependency metadata with a single empty file. While I'm not a party to any Debian policy decisions regarding whether these features will be used for which packages and when, I will nonetheless be happy to continue offering technical input on these matters upon request. Luckily, the overall discussion has been productive despite the rancor. I've learned quite a lot that will be useful. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
At 03:49 PM 11/24/2005 +0100, Matthias Urlichs wrote:
Hi,
Christopher Lenz:
> (And no, I'm not going to repeat the numerous attempts by Phillip to
> politely and comprehensively explain it all.)
>
Sorry -- I don't buy that. I've read all these messages too, and I also
don't know what's in the metadata besides dependency information.
Whatever the project developer wants to put there, or wants people creating
projects that use his project to put there. It's essentially an
inter-project communication and configuration system:
http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-of-services-and-plugins
http://peak.telecommunity.com/DevCenter/setuptools#defining-additional-metadata
http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages
http://peak.telecommunity.com/DevCenter/PkgResources#entry-points
http://peak.telecommunity.com/DevCenter/PkgResources#metadata-api
These features are already being used by Trac, TurboGears, SQLObject,
Paste, and others. Other projects are contemplating moving to the use of
egg entry points to improve their extensibility, or increasing the use that
they currently make of these features.
Debian, rpm, and other Linux packaging systems don't need to install
Python-specific dependency information in their packages.
It's probably nice to have so that non-system Python packages can complain
properly, instead of throwing an incomprehensible exception later (and
leaving one with a partially-installed system or whatever), but that's a
different problem.
That's an interesting perspective, but it's viewing the world through
vendor-colored glasses. Unless the project developer is wearing similar
glasses (i.e., has decided to commit to Debian as their sole platform),
though, it's not a practical one. From the point of view of people like
the author of TurboGears, it's the egg dependency system that allows them
not to have to worry about which packaging system the user has, or doesn't
have.
I mention this not to be disagreeable, just to point out that the world in
which egg dependencies are of no benefit, needless complexity, etc., is not
the same world in which the project developers using eggs live.
Case in point: this thread began because somebody wanted to package
TurboGears and its dependencies for Debian. But that project wouldn't have
been viable without the egg system already existing, and there was
certainly no way for TurboGears to have started its life as anything but a
"non-system Python package". One reason TurboGears is popular is because
it's well-supported, and it's well-supported in part because it can
"complain properly" (as you describe it above) no matter what platform it's
running on.
Another reason TurboGears is popular because of its high degree of reuse of
existing code. Indeed, TurboGears has an unprecedented degree of reuse of
code for a Python project. How many Debian Python packages have even
*half* as many Python dependencies as TurboGears does?
Meanwhile, TurboGears is viral with respect to eggs. When you create new
web applications with it, those projects are also eggs, with dependencies
on TurboGears. And of course people are creating TurboGears add-ons, like
the identity package, and those have dependencies to TurboGears and other
eggs. Even if it were *only* TurboGears that was doing these things (and
it's not), you're going to have an "egg-splosion" on your hands here pretty
soon. ("Pretty soon" == ~1 year, assuming the current growth rate of eggs
is linear rather than exponential.)
Again, I don't think you're "wrong" to think that egg dependencies are
redundant - from within your particular point of view. But you need to
understand that for *Python* developers, being able to practically depend
on other packages in a cross-platform way is a new and powerful feature
which is *not* provided by Debian or any other packaging system, unless
it's wrapping eggs. So, from your perspective, you already have this
feature, but for projects using eggs, you really *don't* have the feature,
because the data is not economically accessible to them.
And that's only *one* feature of eggs, not even getting into each new
egg-specific feature that each project can create, building on the entry
point and metadata features. Of course, those features have a different
growth curve for adoption than the dependency features do, since plugin
features are more of a hub-and-spokes model with limited network
effects. And an individual plugin project is less likely to need to end up
as a Debian package anyway, so the plugin impact will be more slow-moving
unless somebody puts out a "killer app" that everybody wants plugins for.
Anyway, I appreciate your frank and open-minded participation in this
discussion, and I hope that together we can get the remaining issues resolved.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Con
Re: [Distutils] formencode as .egg in Debian ??
Alright, now to the lots of (unclear) documentation you pointed me to. Le jeudi 24 novembre 2005 à 10:14 -0500, Phillip J. Eby a écrit : > http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-of-services-and-plugins > > http://peak.telecommunity.com/DevCenter/setuptools#defining-additional-metadata I fail to see why this feature needs eggs. A generic plugin system is definitely a good idea, but integrating it with the way python modules are provided is a bad one. Other developers didn't wait for eggs to develop elegant plugin systems that don't require any kind of extra metadata. > http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages This is definitely not a feature. There's no way retrieving a module from several locations should be possible, at least in the distribution-provided packages. > http://peak.telecommunity.com/DevCenter/PkgResources#entry-points I don't even understand what this is trying to achieve. > http://peak.telecommunity.com/DevCenter/PkgResources#metadata-api Generic metadata? Why isn't it put in files lying in the /usr/share tree? I don't understand why it has to come with the python files themselves. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: [Distutils] formencode as .egg in Debian ??
At 01:44 PM 11/24/2005 +0100, Josselin Mouette wrote: They only introduce more complexity, instead of bringing real features. Please read the hundreds of kilobytes of messages I've already posted on this thread, and then when you're done, read these much shorter bits of documentation for some examples of the features they provide: http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-of-services-and-plugins http://peak.telecommunity.com/DevCenter/setuptools#defining-additional-metadata http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages http://peak.telecommunity.com/DevCenter/PkgResources#entry-points http://peak.telecommunity.com/DevCenter/PkgResources#metadata-api These are of course just the documentation for features that developers use, in order to create features of their own. I know that Ian's SQLObject project adds SQL-related metadata to projects using these APIs, and a variety of projects use entry points. The Chandler PIM will probably use project metadata for i18n stuff. > Obviously, every individual distribution would like to have Python packages > conform to their individual system. However, on the whole, it is clearly > better for the Python developer to have practical dependency management > that doesn't tie their efforts to a single platform, packaging system, or > distribution. And that's why there are things like dh_python to adapt python distutils installations to be compliant to the Debian way of things. However, eggs make it so complicated that the adaptation layer would be unmaintainable. As I said, you should probably read the hundreds of K already written in this thread before jumping to conclusions like that. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
Le jeudi 24 novembre 2005 à 10:14 -0500, Phillip J. Eby a écrit : > At 01:44 PM 11/24/2005 +0100, Josselin Mouette wrote: > >They only introduce more complexity, instead of bringing real features. > > Please read the hundreds of kilobytes of messages I've already posted on > this thread I've read them. All of them. You're only babbling about how eggs are great, instead of explaining what problem they are trying to solve. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: [Distutils] formencode as .egg in Debian ??
Le jeudi 24 novembre 2005 à 15:19 +0100, Christopher Lenz a écrit : > > You are advertising this metadata a lot, but what does it exactly > > contain? If it's for dependencies, we can really live better without > > them, or with a tool to convert them into .deb dependencies. > > You know, it is really unfair to make a statement like "The problem > is the added complexity for no real gain" if you're gonna ask the > above question in a follow-up message. You clearly don't understand > the purpose of Python eggs, yet you feel qualified to judge whether > or not it provides any benefit. Sigh. > > (And no, I'm not going to repeat the numerous attempts by Phillip to > politely and comprehensively explain it all.) Maybe you should ask yourself the same question the other way around. How can it be possible, after reading all messages from Phillip to the list - and I haven't omitted a single one - I have no idea of what benefits can eggs bring? Let me repeat the question: what useful information is there in the metadata? What *problem* are eggs exactly trying to solve? > In the end, what matters is that Python eggs are being quickly > adopted by Python application developers, because they allow us to > easily build extensible systems. Indeed, it is nice to developers, but it entirely forgets system integrators. > When Debian users want to install > and use those systems, you're gonna gave to tell them why eggs aren't > supported. There's no reason to forbid people to install eggs in /usr/local or in their home directories. After all, they are suitable for that. What I'm standing against is shipping eggs in Debian packages. Regards, -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: [Distutils] formencode as .egg in Debian ??
On 11/23/05, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > Actually, I think the better long-term approach is more likely to be tools > like easy_deb that wrap easy_install. "Better" here meaning that it can > save the system packager work, because it can handle finding and fetching > and building in an automated way even for non-setuptools packages it has > never seen before. While there are occasionally some issues with projects > that have unusual customizations to the distutils, those customizations > would potentially give a system packager similar troubles anyway. I have now made 3 attempts to write a message responding to this, and got muddled each time. So I'm just going to try giving my key points, unadorned. I can provide supporting discussion, and will do so if you like, but I want to separate the threads first: 1. Windows is a special case here, as traditionally, Windows system packages are provided by the project itself, whereas Linux distributions have separate packaging groups who handle that outside the project. 2. Eggs don't compete with system packagers. And yet projects such as TurboGears and Kid are providing *only* eggs, and no bdist_wininst packages. This is because it is *not possible* at present for projects to provide system packages which wrap eggs. In other words, the rapid take-up of eggs is killing bdist_wininst as a standard distribution format, before a replacement is ready. 3. Easy_install hides this problem, to an extent. Because it *is* a competitor to system packagers. You say that easy_deb wraps easy_install in a Debian package - I don't see how that would work, and regardless, I can't envision an equivalent for Windows. Also, the same issue exists - it doesn't exist *yet*. This is all transition. Assuming that ultimately eggs are the standard, and system packager integration gets solved, there's no problem. I can wait. (If I couldn't wait, I'd be obliged to get off my backside and produce some code :-)) But I do think that projects should be aware of the trade-offs, and I *don't* think they've been clear up to now - that's why I'm writing this summary. One example, then I'll stop. I was recently considering having a look at Kid. I hadn't got round to it, but I now discover that Kid is only distributed as an egg. So I'll forget Kid for now, until the egg/system installer issues settle. Trying out Kid isn't important enough to me to mix two packaging systems. Fully-aware-that-I'm-a-minority-view-ly y'rs Paul.
Re: [Distutils] formencode as .egg in Debian ??
Hi,
Christopher Lenz:
> (And no, I'm not going to repeat the numerous attempts by Phillip to
> politely and comprehensively explain it all.)
>
Sorry -- I don't buy that. I've read all these messages too, and I also
don't know what's in the metadata besides dependency information.
Debian, rpm, and other Linux packaging systems don't need to install
Python-specific dependency information in their packages.
It's probably nice to have so that non-system Python packages can complain
properly, instead of throwing an incomprehensible exception later (and
leaving one with a partially-installed system or whatever), but that's a
different problem.
> When Debian users want to install and use those systems, you're gonna
> gave to tell them why eggs aren't supported.
>
It seems that the next version of the .egg builder _will_ support a mode
that works for Debian packaging, which is *great*, so ...
--
Matthias Urlichs | {M:U} IT Design @ m-u-it.de | [EMAIL PROTECTED]
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
- -
Order is the sanity of the mind, the health of the body, the peace of the
city, the security of the state. As the beams to a house, as the bones to the
microcosm of man, so is order to all things.
-- Southey
signature.asc
Description: Digital signature
Re: [Distutils] formencode as .egg in Debian ??
Am 24.11.2005 um 14:26 schrieb Josselin Mouette: Le mardi 22 novembre 2005 à 11:46 -0600, Ian Bicking a écrit : Eggs give room for package metadata that doesn't exist otherwise. Putting dependencies aside, this is functionality that simply doesn't exist with the standard distutils installation. You are advertising this metadata a lot, but what does it exactly contain? If it's for dependencies, we can really live better without them, or with a tool to convert them into .deb dependencies. You know, it is really unfair to make a statement like "The problem is the added complexity for no real gain" if you're gonna ask the above question in a follow-up message. You clearly don't understand the purpose of Python eggs, yet you feel qualified to judge whether or not it provides any benefit. Sigh. (And no, I'm not going to repeat the numerous attempts by Phillip to politely and comprehensively explain it all.) In the end, what matters is that Python eggs are being quickly adopted by Python application developers, because they allow us to easily build extensible systems. When Debian users want to install and use those systems, you're gonna gave to tell them why eggs aren't supported. The requirements of packaging systems aren't being ignored here. We're (well, Phillip is) listening, and trying to provide constructive input. It'd be nice if the Debian folks would be similarly forthcoming. Cheers, Chris -- Christopher Lenz cmlenz at gmx.de http://www.cmlenz.net/
Re: [Distutils] formencode as .egg in Debian ??
On Thu, 2005-11-24 at 14:14 +0100, Josselin Mouette wrote: [...] > > or having multiple versions of a package > > installed at the same time. > > Is this a joke? Installing several versions of a package is fragile, > it's a security mistake, and takes place on the hard disk for no real > use. This is true for a production environment, but not for a development environment where you want to develop/test/support multiple versions of Python. Debian is used for both. It should support both. The broad diversity of Python applications, all with their own independent Python version dependencies and migration schedules, also makes this impractical for many production environments. Hence we should support multiple versions of Python. > I'm currently fighting so that we stop supporting so many python > versions, and I hope we won't have to support several versions for each [...] And there is still a lot of Debian Python people resisting :-) > package, each of them for several python versions! This would become a > complete nightmare. [...] I think we are pretty close to managing this already, although it could use some tweaks... mind you, I'm all talk and no action on this front, so I should probably be ignored. Disclaimer: This should not be interpreted as an argument for or against egg's. This thread is the first time I've even heard of them. -- Donovan Baarda <[EMAIL PROTECTED]> http://minkirri.apana.org.au/~abo/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: [Distutils] formencode as .egg in Debian ??
Le mardi 22 novembre 2005 à 11:46 -0600, Ian Bicking a écrit : > Eggs give room for package metadata that doesn't exist otherwise. > Putting dependencies aside, this is functionality that simply doesn't > exist with the standard distutils installation. You are advertising this metadata a lot, but what does it exactly contain? If it's for dependencies, we can really live better without them, or with a tool to convert them into .deb dependencies. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: [Distutils] formencode as .egg in Debian ??
Le mardi 22 novembre 2005 à 15:41 -0600, Bob Tanner a écrit : > When I read the above, my knee-jerk reaction is: Where is the data to backup > this statement? > > Follow up questions are: > > How much slower? We talking milliseconds, seconds, minutes? Yes, there are > variables, here, but narrow them to a set number and compare zip vs > directory? > > Previous post talked about memory consumption, again bytes, kilobytes, > megabytes? Performance is not the key point, although a package that affects performance of other packages is obviously broken in some way. The problem is the added complexity for no real gain. The only thing you gain is multiple versions of a package installed at the same time. Something that should be avoided. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: [Distutils] formencode as .egg in Debian ??
Le mardi 22 novembre 2005 à 18:47 -0500, Phillip J. Eby a écrit : > I don't understand you here. Are you saying that it's not possible for > dpkg to do a post-install or uninstall operation like adding or removing a > line from a file? It's possible, but it's fragile. > Of course, this creates additional work for package maintainers that > wouldn't be present with setuptools' normal .egg file/directory > distributions, and my assumption was that the maintainers would prefer to > be able to ignore such issues and get the benefit of dependencies defined > by the upstream developers. Debian is used to ignore dependencies defined by upstream developers, because in most cases they turn out to be unsuitable. That's why there's no tool to convert libtool versioning schemes to shlibs files, why there's no tool to convert configure.in requirements to build-dependencies, and why there's currently no tool to generate dependencies for python packages at all. The maintainer of each package is on his own to define dependencies, because he knows better. > Eggs keep each project in its own little > bubble, where it can't overwrite anything else and can be uninstalled > without removing any overlapping parts. I can't believe you are pushing a solution with such a "feature". Duplication is something we should avoid at all costs. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: [Distutils] formencode as .egg in Debian ??
Le mercredi 23 novembre 2005 à 18:06 -0500, Phillip J. Eby a écrit : > The degree of negativity from the Debian side at the outset of this > conversation (virtually all of it from you) has not been conducive to > making this happen. As a simple matter of practicality, I can't afford to > leave your comments unanswered, not because I feel any need to convince you > personally of anything, but because I don't want to leave anyone else with > the impression that your portrayal of these so-called "drawbacks" is a fair > one. Otherwise, I would have just ignored your comments and focused on > working with the people who seem more interested in finding solutions than > finding ways to declare a non-existence of the problem. As it is, I feel > forced to spend time replying to your comments point-by-point, that I could > otherwise spend on actually helping to resolve the issues. The reason why people from Debian are complaining is that all solutions you have proposed so far are not acceptable. Full stop. Currently, I can't see any way to package eggs in a clean and elegant way. > If I were to adopt your tone, I would be calling Debian a fragile and > broken system that is unable to deal well with simple matters like editing > a file upon installation, I suggest you go get yourself a cluebat. Editing a file upon installation is a completely stupid idea, and that's why we try to avoid it whenever possible. The reason should be obvious: while it's easy to add or remove files from the system, you can never guarantee a file will be successfully edited. Furthermore, editing a file for modifying the sys.path is an even more stupid idea, as python already has everything you need to deal with new modules: simply ship them in site-packages/ subdirectories. > or having multiple versions of a package > installed at the same time. Is this a joke? Installing several versions of a package is fragile, it's a security mistake, and takes place on the hard disk for no real use. I'm currently fighting so that we stop supporting so many python versions, and I hope we won't have to support several versions for each package, each of them for several python versions! This would become a complete nightmare. The whole purpose of eggs seems to be able to distribute whole projects, with their dependencies, in a simple format. Sorry, but it goes in the opposite direction of all Linux distributions. The whole point of inter-package dependencies is to avoid this, and believe me, there are lots of reasons to do that; otherwise, all binaries would be statically linked, or packed with their libraries. This is how things work in the Windows world, but dependencies were invented exactly to avoid it. > Sure, the limitation might exist, but is it > fair to call Debian fragile or broken because of it? Not a bit! I've > therefore been very careful to describe any such tradeoffs that Debian > makes in neutral terms rather than categorically pejorative ones. I would > prefer if you would extend me the same courtesy of not describing every > design tradeoff I make as being a "non-standard", "drawback", "for no good > reason". Refusing to edit files upon installation, for example, is not a tradeoff. It's a deliberate decision, based on the fact this is a bad idea. > If Debian adds this metadata marker for its non-setuptools Python packages, > then the Python packages will be "eggs" in the sense that other eggs will > be able to discover them via the pkg_resources API, and thus TurboGears > users will be able to use the Debian-supplied versions of ElementTree and > the like. Why would we need to modify hundreds of packages to contain empty files, just because another package cannot import its dependencies correctly? > >I don't see why multi-version support necessarily requires to > >increase sys.path. In the case of eggs, version dependencies are > >expressed explicitly in the code (through require() calls), > > Actually, they're expressed in the egg metadata, and the wrappers on a > project's scripts execute the require() calls, so that the code doesn't > have to contain explicit require() calls except for more-dynamic > situations, such as plugins and "optional extra features" that require > additional projects to be present. I'm used to remove such code checking for the version of dependencies in python packages, as it is broken most of the time. Introducing another broken way to check the dependencies can be worked around the same way, but it's sadly more burden on the package maintainers. Still, I can assure you it's less burden than adapting the system to handle eggs, as I understand them. > Note that setuptools is in release 0.6a8 at the moment - it is obviously > not a polished product, but it provides enough functionality to be quite > useful to many Python developers. To this point, directly working on > integration with external packaging tools has not been a focus, although I > always have gi
Re: [Distutils] formencode as .egg in Debian ??
Le mardi 22 novembre 2005 à 17:05 -0500, Phillip J. Eby a écrit : > And over the last few months, I believe we've also succeeded in stomping > most of the issues that people had with getting solid non-root > installations on their Linux distributions. So the reasons for developers > to prefer their dependencies to be managed as eggs will only improve over > time, as the egg system allows Python developers to control and introspect > their dependencies, rather than keeping that information hidden behind > diverse platform-specific packaging tools. This is useful for non-root installations, but it is only a hassle when you are making .deb's. Here, dependencies are already handled, and you rely on the system to provide correct versions for packages you depend upon. > It's only "competitive" if you feel that there must be only one way to do > it. (And if you do feel that way, then it also should be obvious that eggs > are the superior solution, since they don't take away any capabilities of > the old, only provide new ones.) They only introduce more complexity, instead of bringing real features. > Obviously, every individual distribution would like to have Python packages > conform to their individual system. However, on the whole, it is clearly > better for the Python developer to have practical dependency management > that doesn't tie their efforts to a single platform, packaging system, or > distribution. And that's why there are things like dh_python to adapt python distutils installations to be compliant to the Debian way of things. However, eggs make it so complicated that the adaptation layer would be unmaintainable. This is a good reason to think the whole egg system should be avoided. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
