Re: [Distutils] Python Package Management Sucks
On Wed, Oct 01, 2008 at 09:33:52PM -0700, Toshio Kuratomi wrote: Note that Debian has done a lot of neat things with python source recent(ish). Josselin, Matthias, and some of the other Debian devs could tell us if .py files get installed to /usr/share there. Currently the two helper tools install the .py (and .egg-info) files somewhere into /usr/share. One tool places the .pyc files in /usr/lib while the other places them in /var/lib (and uses a .pth file to get that directory on sys.path), both have symlinks to the real .py next too the .pyc files. This way the same .py file can be shared between more then one version of python. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] just use debian
Toshio Kuratomi wrote: I'm not 100% certain but I think that Josselin is speaking of glibc in particular here and you're speaking of c libraries in general. Maybe, but I don't see how this change the point: when you change the soname of a library, it has 0 impact on the source code of the software which links against it. That's not true in python. I've said before that ideally a Linux distribution only wants one version of a library. And ideally, developers do not want to care about versioning issues :) There is a middle ground to find; up to now, I think python distutils and co did not care at all about this, but you can not ask to move 180 degrees and do only as linux vendors want. I understand the reasons why OS vendors want to avoid distributing multiple versions as much as possible. But that's just not realistic in every case. Even in C-derived languages, this is sometimes a PITA. I could give you examples where distributions screwed up the packaging badly and hurt some projects I am involved with. But that would be unfair and besides the point (we all screw up); the point is that there are some practical reasons for sometimes including private copies. Because for example in Fortran's case, there is this huge mess of gfortran and g77 not being ABI compatible; there are examples in C++, and even in C. You also can't impose every software to follow distributions time-schedule. Reasons for single version for OS vendors are valid; but so are the ones to have multiple versions. I think compat modules would cover most needsl the problem is that python does not have a mechanism to request a particular version of a module. But wouldn't this help OS vendors to have such a mechanism (to decrease the burden of compat versions ?) In Fedora we're willing to have compat packages that hold old API versions if we must but by and large we would rather help upstream apps port their applications forward than to have compatibility packages. Yes, but here again the C comparison breaks. Some people use python as a tool, not so much as a programming language. Their applications are scripts, or softwares for experiment, that are not released, because they can't open source it, or simply because it has no use for anyone else. You can't port that. cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Phillip J. Eby wrote: I think install tools should handle it and keep it out of developers' hair. We should of course distinguish configuration and other writable data from static data, not to mention documentation. Any other file-related info is going to have to be optional, if that. I don't really think it's a good idea to ask developers to fill in information they don't understand. A developer who works entirely on Windows, for example, is not going to have a clue what to specify for FHS stuff, and they absolutely shouldn't have to if all they're doing is including some static data. I may be missing something, but why should the developer even care about FHS ? We should not standardize what goes where, but the kind of data needed to be installed (doc, etc...), and then have different (pluggable) implementations to put those where they should. Autotools does it this way, for example: you have pkg_data, etc... and every one of them can be changed. The proof that this is flexible is that fact that something like GoboLinux (which is a bit like what Mac OS X handles their files) is possible at all from the same sources. http://www.gobolinux.org/?page=doc/articles/compile I don't see the need for reinventing anything here: just start from the same concepts as autotools, modify it to handle non unix softwares (if it is even needed), and that's it. cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [pyconuk] just use debian
Le jeudi 02 octobre 2008 à 12:36 +1200, Greg Ewing a écrit : Josselin Mouette wrote: Le mardi 30 septembre 2008 à 17:20 +0200, Tarek Ziadé a écrit : I mean, if you change a public API of your package , you *have* to change its name ? Yes, this is the requirement for C libraries, and we try to enforce it as well for other languages. Things are somewhat different in C, because the filename of the .so isn't something you refer to in the source code. Applying the same thing to Python would require the version to be specified every time the module is mentioned in an import statement. There are no ABI issues with Python. Only when the API changes, software using it is broken. When the API changes incompatibly in C, the problem is the same. It is solved elegantly by pkg-config: the two versions can have a different (versioned) directory for the includes and a different library name, so you only change the pkg-config calls or the -I and -l build options. I wish we had a similar mechanism in Python, allowing to select the API version in some way. However, since there is no build step, you need to make it happen somewhere in the code. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `-our own. Resistance is futile. signature.asc Description: Ceci est une partie de message numériquement signée ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Le mercredi 01 octobre 2008 à 19:59 -0400, Phillip J. Eby a écrit :
locale/{message catalogs for various languages} -- These are binary
files that contain strings that the user may see when a message is
given. These, I think are data.
I lean the other way, since they're not editable.
Locale data should be shipped in standard form, in /usr/share/locale.
Again, Python is not the only thing you need to think of. Not only, if
you don’t use standard gettext catalogs, you are breaking translation
tools, but if you don’t ship them at the standard location, you are
breaking tools like localepurge (or anything that cleans up the
filesystem based on standard file locations).
Keep in mind that
some platforms have no locale directories as such, and thus trying
to support multiple locations thereof is a burden for cross-platform
app developers, vs. simply treating them as internal resources.
No, you are making the burden heavier for Linux platforms by trying to
be more portable. There is no reason why you can’t define a locale
directory on platforms where it does not exist, but when it exists a
standards document defining where files go, you must follow it. Any time
you don’t follow it, we consider it a serious breakage and we have to
patch the code anyway.
templates/*html -- These are templates that the application fills in
with variables intermixed with short bits of code.
[snip]
My leaning is that these are data.
If you follow this logic (create bytecode from it, can't run w/out
interp or compiler), then any .py file is *also* data; i.e., this
Does Not Follow.
Sorry, but things don’t work this way. Anything that is *not* a .py file
should not land in the python module directories. This is utter abuse of
a loophole of the implementation, and I can’t think of another language
that allows that. You will not find anything else than perl modules in
the perl modules directories. You will not find anything else than C#
modules in the Mono modules directory. And so on.
static/{javascript,css,images} -- These are things that are definitely
never executed. They are served by the webserver verbatim when their
URL is called. These are certainly data. (Note: I don't believe these
are referenced using the resources API, just via URL.)
Uh... that would depend entirely on the library or application. But
if they're static and the user's got no business tinkering with them,
then it's a resource.
No, again it is insane to ship them as resources. These are files that
need to be accessed directly by the webserver, and as such they need to
be shipped in a data directory, not in a python modules directory.
So... do you agree on which of these are data and which are resources?
Do you have an idea on how we can prevent application and framework
writers from misusing the resources API to load things that are data?
Apparently not. The alternative I would suggest is that under the
new standard, an install tool should be allowed to relocate any
non-Python files, and all access has to go through a resource
API. The install tool would then have to be responsible for putting
some kind of forwarding information in the package directory to tell
the resource API where it squirrelled the file(s) off to. Then we
can avoid all this angels-on-a-pin argument and the distros can Have
It Their Way[tm].
I don’t understand why you want to make it so complicated. All you need
is a way to specify directories where different kinds of files land and
a simple API to retrieve the file names/contents. Then, you can ship the
files at the place you like in eggs, and we can ship the files at the
standard places in our packages.
Cheers,
--
.''`.
: :' : We are debian.org. Lower your prices, surrender your code.
`. `' We will add your hardware and software distinctiveness to
`-our own. Resistance is futile.
signature.asc
Description: Ceci est une partie de message numériquement signée
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
-On [20081002 13:20], Josselin Mouette ([EMAIL PROTECTED]) wrote:
Locale data should be shipped in standard form, in /usr/share/locale.
Again, Python is not the only thing you need to think of.
Linux is not the only thing to think of by stating /usr/share/locale. :P
I guess ${PREFIX}/share/locale would be better. But still, for .mo files I
seriously do not see a problem if they're within an egg or something
similar. Imagine also the possibility of providing specific language
translations for a project as an egg (kind of like a plugin).
--
Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Whispering winds in moonlit wood, a totem oak once golden stood...
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Le jeudi 02 octobre 2008 à 15:33 +1200, Greg Ewing a écrit :
Toshio Kuratomi wrote:
nod this is what I was afraid of. This is definitely not a definition
of resource-only that has meaning for Linux distributions. None of the
data in /usr/share is user-modifiable
In that case it must be there because it's architecture-independent,
right?
But by that criterion, all .py files should be in /usr/share, too.
Also all shell scripts, Perl code, awk/sed scripts, etc, etc.
Does the FHS specify that?
The FHS is not that precise, but in Debian, we do:
* move architecture-independent perl modules to /usr/share/perl5
* move architecture-independent python modules
to /usr/share/{python-support,pyshared}
* when possible, move shell scripts (except those in /usr/bin)
from /usr/libexec or /usr/lib to /usr/share/$package
There’s no rocket science in that. But I wish we could make things as
simple for Python modules as they are for Perl ones. But Perl does not
allow messing with module namespaces with .pth files, nor shipping
resource files and whatnot in the perl module directory. Only perl
modules are allowed there.
Cheers,
--
.''`.
: :' : We are debian.org. Lower your prices, surrender your code.
`. `' We will add your hardware and software distinctiveness to
`-our own. Resistance is futile.
signature.asc
Description: Ceci est une partie de message numériquement signée
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Le jeudi 02 octobre 2008 à 13:25 +0200, Jeroen Ruigrok van der Werven a
écrit :
-On [20081002 13:20], Josselin Mouette ([EMAIL PROTECTED]) wrote:
Locale data should be shipped in standard form, in /usr/share/locale.
Again, Python is not the only thing you need to think of.
Linux is not the only thing to think of by stating /usr/share/locale. :P
I guess ${PREFIX}/share/locale would be better. But still, for .mo files I
seriously do not see a problem if they're within an egg or something
similar. Imagine also the possibility of providing specific language
translations for a project as an egg (kind of like a plugin).
There’s definitely no problem with shipping them in an egg, as long as
it is possible to ship them at standard locations.
Cheers,
--
.''`.
: :' : We are debian.org. Lower your prices, surrender your code.
`. `' We will add your hardware and software distinctiveness to
`-our own. Resistance is futile.
signature.asc
Description: Ceci est une partie de message numériquement signée
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Josselin Mouette wrote: I don’t understand why you want to make it so complicated. All you need is a way to specify directories where different kinds of files land and a simple API to retrieve the file names/contents. Then, you can ship the files at the place you like in eggs, and we can ship the files at the standard places in our packages. Yes, I don't understand all this complication and concepts either. I have not seen any reason not to do like autoconf. It is simple, obvious, has no burden for the developer, has been used by thousand of softwares for more than one decade, and is extremely flexible. The only thing needed in addition to autoconf is a python API to retrieve the location of each kind of files (python, data, doc, etc...) so that the package itself can find it independently of the way it was installed. cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
-On [20081002 13:29], Josselin Mouette ([EMAIL PROTECTED]) wrote: There’s definitely no problem with shipping them in an egg, as long as it is possible to ship them at standard locations. Standard according to whom though? Contrary to many Linux systems, for example, the BSDs tend to install Python files to /usr/local or /usr/pkg (if using pkgsrc) and not /usr. I'm sure /opt is also still widely used. Furthermore, if by standard you mean */share/locale/XX/LC_MESSAGES/, what good would an egg do at that place? -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Reality is an illusion, grimmer. The dreamlands are like masks within masks, and Time has no dominion beyond the Shroud... ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Le jeudi 02 octobre 2008 à 13:35 +0200, Jeroen Ruigrok van der Werven a écrit : -On [20081002 13:29], Josselin Mouette ([EMAIL PROTECTED]) wrote: There’s definitely no problem with shipping them in an egg, as long as it is possible to ship them at standard locations. Standard according to whom though? Contrary to many Linux systems, for example, the BSDs tend to install Python files to /usr/local or /usr/pkg (if using pkgsrc) and not /usr. I'm sure /opt is also still widely used. That’s not a problem. The only thing that is needed is to be able to select the path at installation time, just like we do with autoconf. BTW, I’m all for making /usr/local the default, since /usr being the default leads to manual installation overwriting packages instead of going to /usr/local. Furthermore, if by standard you mean */share/locale/XX/LC_MESSAGES/, what good would an egg do at that place? In the egg distribution, they can go at another place. Again, not a problem as long as we have a flag somewhere that can install them to the place we like. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `-our own. Resistance is futile. signature.asc Description: Ceci est une partie de message numériquement signée ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
At 07:40 PM 10/2/2008 +0900, David Cournapeau wrote: Phillip J. Eby wrote: I think install tools should handle it and keep it out of developers' hair. We should of course distinguish configuration and other writable data from static data, not to mention documentation. Any other file-related info is going to have to be optional, if that. I don't really think it's a good idea to ask developers to fill in information they don't understand. A developer who works entirely on Windows, for example, is not going to have a clue what to specify for FHS stuff, and they absolutely shouldn't have to if all they're doing is including some static data. I may be missing something, but why should the developer even care about FHS ? We should not standardize what goes where, but the kind of data needed to be installed (doc, etc...), and then have different (pluggable) implementations to put those where they should. Yep - that's precisely what I've proposed. We just need to define those categories in such a way as to minimize the burden on a Python developer. Where data files are concerned, however, a developer should only need to distinguish between read-only, read-write, and samples, because any finer-grained distinction that relies on platform-specific concepts (like locale directories) is probably going to be error-prone. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
On Wed, Oct 01, 2008 at 09:33:52PM -0700, Toshio Kuratomi wrote: Note that Debian has done a lot of neat things with python source recent(ish). Josselin, Matthias, and some of the other Debian devs could tell us if .py files get installed to /usr/share there. One of the current options in Debian (python-central) involves installing the .py files to /usr/share/pycentral/package_name and symlinking them in the site-package of each Python version the package is known to work with. You have thus separating of binary and source code. Gaël ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
On Thu, Oct 02, 2008 at 01:20:45PM +0200, Josselin Mouette wrote: Sorry, but things don’t work this way. Anything that is *not* a .py file should not land in the python module directories. This is utter abuse of a loophole of the implementation, and I can’t think of another language that allows that. You will not find anything else than perl modules in the perl modules directories. You will not find anything else than C# modules in the Mono modules directory. And so on. What about .so of compiled modules? Gaël ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Phillip J. Eby wrote: At 07:14 PM 10/1/2008 -0700, Toshio Kuratomi wrote: In terms of implementation I'd much rather see something less centered on the egg being the right way and the filesystem being a secondary concern. Eggs don't have anything to do with it; in Python, it's simply common sense to put static resources next to the code that uses them, if you want to write once, run anywhere. And given Python's strength as an interactive development language with no build step, having to *install* your data files somewhere else on the system to use them isn't a *feature* -- not for a developer, anyway. You're arguing about the developers point of view on something that's hidden behind an API. You've already made it so that the developer cannot just reference the file on the filesystem because the egg may be zipped. So for the developer there's no change here. I'm saying that there's no need to have a hardcoded path to lookup the information at and then make the install tool place forwarding information there to send the package somewhere else. We have metadata. We should use it. And our hypothetical de-jure standard won't replace the de-facto standard unless it's adopted by developers... and it won't be adopted if it makes their lives harder without a compensating benefit. For the developer, FHS support is a cost, not a benefit, and only relevant to a subset of platforms, so the spec should make it as transparent for them as possible, if they don't have an interest in explicit support for it. By the STASCTAP principle (Simple Things Are Simple, Complex Things Are Possible), it should be possible for distros to relocate, and simple for developers not to care about it. It's both a cost and a benefit. The cost is having to use an API which they have to use anyway due to eggs possibly being zip files. The benefit is getting their code packaged by Linux distributors quicker and getting more contributors as a result of the exposure. We should have metadata that tells us where the types of resources come from. When a package is installed on Linux the metadata could point locales at file:///usr/share/locale. When on Windows egg:locale (Perhaps the uninstalled case would use this too... that depends on how the egg structure and metadata evolves.) A question we'd have to decide is whether this particular metadata is something that should be defined globally or per package. Or globally with a chance for packages to override it. I think install tools should handle it and keep it out of developers' hair. We should of course distinguish configuration and other writable data from static data, not to mention documentation. Any other file-related info is going to have to be optional, if that. I don't really think it's a good idea to ask developers to fill in information they don't understand. A developer who works entirely on Windows, for example, is not going to have a clue what to specify for FHS stuff, and they absolutely shouldn't have to if all they're doing is including some static data. Needing to have some information about the files you ship is inevitable. Documentation is a good example. man pages, License.txt, gnome help files, windows help files, API docs, sphinx docs, etc each have to be installed in different places, some with requirements to register the files so the system knows they exist.All the knowledge about what to do with these files should be placed in the tool. But the knowledge of what type to mark a given file with will have to lay with the developer. Even today, there exist Python developers who don't use the distutils to distribute their packages, so anything that makes it even more difficult than it is today, isn't going to be a viable standard. The closer we can get in ease of use to just tarring up a directory, the more viable it'll be. (That's one reason, btw, why setuptools offers revision control support and find_packages() for automating discovery of what to include.) Actually, as a person who distributes upstream packages which don't use distutils and is exposed to others, I'd say that the shortcomings in terms of where to install files and how to reference the files after install is one of the reasons that distutils is not used. Are there other reasons? Sure. But this is definitely one of the reasons. I'd have preferred to avoid that complexity, but if the two of us can't agree then there's no way on earth to get a community consensus. Btw, pkg_resources' concept of metadata would also need to be relocatable, since e.g. the EggTranslations package uses that metadata to store localizations of image resources and message catalogs. (Other uses of the metadata files also inlcude scripts, dependencies, version info, etc.) Actually, we should decide whether we want to support that kind of thing within the egg metadata at all. The other things we've been talking about
Re: [Distutils] just use debian
On Thu, Oct 02, 2008 at 10:08:01AM -0700, Toshio Kuratomi wrote: 4) make sure your package works with vanilla upstream versions of the third party modules. It's tempting to fix things in your local copies of modules. If at all possible don't. If that's not possible, make sure upstream has incorporated the patch and make a note in the README -- using a patched version of Foo-x.y project. The patch is in their svn as of DATE. patch is located in myproject/foo-x.y/patches. Doing this means that the distribution packager of your package can take your patch to the packager of Foo and ask that the patch be incorporated there. Mercurial's patch queues can be of great help for this. http://www.selenic.com/mercurial/wiki/index.cgi/MqExtension development stuff. But I think that using eggs, workingenv, etc is a fine solution for the development case. Someone told me about http://0install.net/ but I have not tested it and do not know how good/bad it is. -- Nicolas Chauvat logilab.fr - services en informatique scientifique et gestion de connaissances ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
On Thu, Oct 02, 2008 at 10:05:49AM -0400, Phillip J. Eby wrote: I don't see how this is remotely relevant to how Python works or how people use (and have used) Python over the decades. Data files in package directories is something I've seen in Python since, oh, 1997 or so. If it were a mistake for Python, I think somebody would've noticed by now. Well, maybe someone just did notice! I don't see why you disagree so strongly with this as it is entirely in line with providing separate source tree locations for differents types of data which you are in favour of. Specifiying that this would be the recommended way and providing an API to deal with it doesn't mean you forbit the old way. You'd still be able to dig around with __file__. Even if any GNU/Linux packagers will try and convince upstreams not to do so. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
At 08:08 PM 10/2/2008 +0100, Floris Bruynooghe wrote: On Thu, Oct 02, 2008 at 10:05:49AM -0400, Phillip J. Eby wrote: I don't see how this is remotely relevant to how Python works or how people use (and have used) Python over the decades. Data files in package directories is something I've seen in Python since, oh, 1997 or so. If it were a mistake for Python, I think somebody would've noticed by now. Well, maybe someone just did notice! I don't see why you disagree so strongly with this as it is entirely in line with providing separate source tree locations for differents types of data which you are in favour of. Specifiying that this would be the recommended way and providing an API to deal with it doesn't mean you forbit the old way. You'd still be able to dig around with __file__. Even if any GNU/Linux packagers will try and convince upstreams not to do so. We're actually in violent agreement here regarding what is to be done at a practical level. Linux packagers want it one way, developers want it another, and both can have what they want... unless they want the other guys to agree their way is the right way of doing it. ;-) That is, the Linux packagers appear to be upset and offended by the mere idea that anyone should consider the Python way to be correct, but it's not reasonable to expect people to change their minds, any more than it's reasonable to exepct the Linux packagers to agree the Python way is the right way and that everything they've been doing is wrong. (And if you respond to this idea by thinking that it would be because the Linux packagers are *right*, then you're simply illustrating my point.) Now, on a practical level, if what you are trying to say is that everybody should access all data strictly through some new API created for that purpose, then that's a non-starter for practical reasons, rather than ideological ones. Even setuptools doesn't *require* that people use an API to access their static data, and does its darnedest to support that use case. And, if you want a new spec to actually *replace* setuptools, then you're going to have to be very careful about what setuptools features are dropped, or the spec is unlikely to go from de jure to de facto. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [zc.buildout] running in safe mode
On Oct 1, 2008, at 6:34 AM, Tarek Ziadé wrote: Hello I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them. Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that. I don't think backups are the right approach. It's a mistake to have recipes manage precious data. If you really really really think that's a good idea, then the recipe should at least manage uninstall and move precious data aside, rather than remove it. I don't think it is really the user's problem is a recipe misbehaves by allowing precious data to be removed. But I think we can improve zc.buildout a bit for that: what about introducing a safe-mode where the developer get prompted everytime zc.buildout.rmtree is about to call shutil.rmtree ? The option could be set in [buildout] like this: [buildout] ... safe-mode = true ... and challenge the user when a tree is about to be delete. (it might be overkill for single files, and they are most of the time unimportant configuration files) This is a small change, and it would avoid running a backup everytime the .cfg are changed. because it happens all day long when you are developing. I suspect this would be so annoying to use that no one would use it. I think it's probably easier to fix the broken recipes. I also think calling this safe mode is misleading. I could live with an option named something like prompt-before-removing-files-or-directories. A better option might be something like move-aside-on-uninstall, which would move files or directories aside rather than deleting them. I still think it would be better to just fix the broken recipes. Jim -- Jim Fulton Zope Corporation ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [zc.buildout] running in safe mode
Jim Fulton wrote: I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them. Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that. I don't think backups are the right approach. It's a mistake to have recipes manage precious data. If you really really really think that's a good idea, then the recipe should at least manage uninstall and move precious data aside, rather than remove it. I don't think it is really the user's problem is a recipe misbehaves by allowing precious data to be removed. I'll note fassembler uses a file abstraction layer so that its recipes are safe by default: https://svn.openplans.org/svn/fassembler/trunk/fassembler/filemaker.py I think buildout would be a lot more humane if it took the same approach. -- Ian Bicking : [EMAIL PROTECTED] : http://blog.ianbicking.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [zc.buildout] running in safe mode
On Oct 2, 2008, at 6:15 PM, Ian Bicking wrote: Jim Fulton wrote: I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them. Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that. I don't think backups are the right approach. It's a mistake to have recipes manage precious data. If you really really really think that's a good idea, then the recipe should at least manage uninstall and move precious data aside, rather than remove it. I don't think it is really the user's problem is a recipe misbehaves by allowing precious data to be removed. I'll note fassembler uses a file abstraction layer so that its recipes are safe by default: https://svn.openplans.org/svn/fassembler/trunk/fassembler/filemaker.py I think buildout would be a lot more humane if it took the same approach. I'd be interested to know what you mean by this, but I'm not willing to read that source to find out. Can you be a little more specific? Jim -- Jim Fulton Zope Corporation ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [zc.buildout] running in safe mode
On Fri, Oct 3, 2008 at 12:09 AM, Jim Fulton [EMAIL PROTECTED] wrote: A better option might be something like move-aside-on-uninstall, which would move files or directories aside rather than deleting them. ok why not, I still think it would be better to just fix the broken recipes. I agree it is the solution, but we tend not to be able to control all the recipes, and zc.buildout just calls out rmtree in some cases, wich is rather violent imho. ++ Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [zc.buildout] running in safe mode
Jim Fulton wrote:
On Oct 2, 2008, at 6:15 PM, Ian Bicking wrote:
Jim Fulton wrote:
I know it is a bad practice for a recipe to return some paths that
contains important data in the install() method,
because zc.buildout might remove them.
Nevertheless, it happens from time to time that a developer lose some
content because of a misconfiguration,
or a zealous recipe. That is his responsability, and backups are
done for that.
I don't think backups are the right approach. It's a mistake to have
recipes manage precious data. If you really really really think
that's a good idea, then the recipe should at least manage uninstall
and move precious data aside, rather than remove it.
I don't think it is really the user's problem is a recipe misbehaves
by allowing precious data to be removed.
I'll note fassembler uses a file abstraction layer so that its recipes
are safe by default:
https://svn.openplans.org/svn/fassembler/trunk/fassembler/filemaker.py
I think buildout would be a lot more humane if it took the same approach.
I'd be interested to know what you mean by this, but I'm not willing to
read that source to find out.
Can you be a little more specific?
Instead of using open(), etc, to write files, there's an instance of
Maker which holds some of the settings (--interactive, --simulate, a
base directory). Then you do all your file operations like:
maker.ensure_file('path/to/file.txt', content)
If that file exists with different content then the user gets asked
about what to do. It also logs all the writing, shows diffs, can make
backups, etc. You can force overwriting, but that's a keyword argument
that defaults to False, so only if you actually have good reason to
overwrite files (without asking) then that's fine, but you will start
developing the easy way, which is to be safe about this stuff.
--
Ian Bicking : [EMAIL PROTECTED] : http://blog.ianbicking.org
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [zc.buildout] running in safe mode
On Fri, Oct 3, 2008 at 12:37 AM, Ian Bicking [EMAIL PROTECTED] wrote:
Can you be a little more specific?
Instead of using open(), etc, to write files, there's an instance of Maker
which holds some of the settings (--interactive, --simulate, a base
directory). Then you do all your file operations like:
maker.ensure_file('path/to/file.txt', content)
If that file exists with different content then the user gets asked about
what to do.
That is basically what I wanted to introduce
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] AMP, anyone? (Re: just use debian)
David Cournapeau wrote: when you change the soname of a library, it has 0 impact on the source code of the software which links against it. That's not true in python. In the Eiffel world, there's a thing called an ACE, which stands for Assembly of Classes in Eiffel. It's a way of specifying a mapping between class names used internally in the code and where to get them from in the environment. I think Python could do with something similar for managing version issues. Perhaps we could study how ACEs work and see if we can use any ideas from there. We could call it an AMP -- Assembly of Modules in Python. -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Jeroen Ruigrok van der Werven wrote: -On [20081002 13:29], Josselin Mouette ([EMAIL PROTECTED]) wrote: There’s definitely no problem with shipping them in an egg, as long as it is possible to ship them at standard locations. Standard according to whom though? I think the idea is to *define* a standard location within the egg. An install tool is then free to extract them out of the egg and put them somewhere else if it wants. There should also be an API for finding them (rather than assuming they're still inside the egg) so that the code that uses them can still find them if they've been moved somewhere platform-specific. -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Phillip J. Eby wrote: Where data files are concerned, however, a developer should only need to distinguish between read-only, read-write, and samples, because any finer-grained distinction that relies on platform-specific concepts (like locale directories) is probably going to be error-prone. However, if we don't allow for such distinctions, then people who e.g. are passionate about their locale files ending up in the right place on Linux aren't going to be happy. -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
On Thu, Oct 02, 2008 at 01:20:45PM +0200, Josselin Mouette wrote: Sorry, but things don’t work this way. Anything that is *not* a .py file should not land in the python module directories. So where *should* they go, on platforms where there is no defined place for such things? And how can the code that uses them find them without platform-specific knowledge? -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
At 03:25 PM 10/3/2008 +1300, Greg Ewing wrote: Phillip J. Eby wrote: Where data files are concerned, however, a developer should only need to distinguish between read-only, read-write, and samples, because any finer-grained distinction that relies on platform-specific concepts (like locale directories) is probably going to be error-prone. However, if we don't allow for such distinctions, then people who e.g. are passionate about their locale files ending up in the right place on Linux aren't going to be happy. Yep. As I've said, it should be possible to define optional extensions that define these sorts of things. They're needed for e.g. icons, Windows registry entries, etc. anyway. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Ben Finney wrote: I'm not sure exactly what you mean by “do like autoconf”. Can you please describe exactly what behaviour you are envisaging, so that we don't all have a different interpretation of what “like autoconf” means ? Not autoconf, but the whole autotools suite (I think you need to use automake at least to do what I have in mind). The idea is deceptively simple: when you use ./configure, it gives you many options wrt paths: --bindir=DIR user executables [EPREFIX/bin] --sbindir=DIR system admin executables [EPREFIX/sbin] --libexecdir=DIR program executables [EPREFIX/libexec] --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIRmodifiable single-machine data [PREFIX/var] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIRC header files for non-gcc [/usr/include] --datarootdir=DIR read-only arch.-independent data root [PREFIX/share] --datadir=DIR read-only architecture-independent data [DATAROOTDIR] --infodir=DIR info documentation [DATAROOTDIR/info] --localedir=DIRlocale-dependent data [DATAROOTDIR/locale] --mandir=DIR man documentation [DATAROOTDIR/man] --docdir=DIR documentation root [DATAROOTDIR/doc/libsndfile] --htmldir=DIR html documentation [DOCDIR] --dvidir=DIR dvi documentation [DOCDIR] --pdfdir=DIR pdf documentation [DOCDIR] --psdir=DIRps documentation [DOCDIR] By default, they are laid out more or less like the FHS (and again, the detail do not matter: you can change them), BUT, it is extremely flexible. The only burden for the developer is to say which files go in which category, which he has to do anyway. You could think that because it is autoconf, it is not flexible for other OSes (windows), but with this, you can have extremely different ways of packaging, which are 100 % against the FHS spirit. I gave the gobolinux example, which does package every software in its own directory, a bit like a super-stow if you know stow. The thing is, I am sure the majority of softwares developers never think about this for their own software, still it is possible: it really come for free for the developer. And for the distributor, it is extremely easy (and self-documented). You could have different defaults on different platforms (windows comes to mind). cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] AMP, anyone? (Re: just use debian)
Greg Ewing wrote: I think Python could do with something similar for managing version issues. Perhaps we could study how ACEs work and see if we can use any ideas from there. Interesting. From your description, it does sound a bit like the GAC for MONO/.Net I was mentioning earlier: http://www.mono-project.com/Assemblies_and_the_GAC I know Bertrand Meyer say mostly good things about .net technologies, and Eiffel softwares went a lot toward so .net technologies, so I would not be surprised if it were somewhat similar. Do you have a link toward this. I could not find much info with google on this Assembly Classes for Eiffel. cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
