Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
ajaksu wrote: While Linux and OS X both view Python as essentially a first-class development platform-i.e., as something that shrink-wrap applications can be built on-Windows does not. Even on MacOSX and Linux, you can't rely on the system coming with the particular version of Python you want to use pre-installed. So unless you target a very old version of Python, you have to bundle anyway if you don't want users to have to download and install a bunch of dependencies. If the situation is any better on Linux, it's probably because Linux users tend to be more willing and/or able to track down and install dependencies along with an app. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin v. Löwis wrote: Sure, but what is precisely the semantics of uninstallation, in terms of changes to the system state? I think any model where uninstallation is merely the removal of files is too limited to be practical. The distutils only support the *addition* of files, so I'm not sure how only removing files is a limit here. Could you explain? For files, yes, it only supports addition. But it supports arbitrary other actions, such as: - addition of registry keys - addition of user accounts - creation of database tables in a relational database - updating the shared library loader path - creation and start of a system service - integration of documentation into info - registration of DTDs with the system catalog - ... It's turing-complete, and it has full interface to the operating system, so installation of a distutils package can do *much* more than merely installing files. Which is exactly what is wrong with distutils: turing completeness in an installer is an *anti* goal, from the perspective of manageability. I'd be willing to bet that if you asked system packagers to list the dozen or so packages which they *hate* to maintain, the large majority of each list would be packages which acutally use the full power of distutils. (Note: I'm aware that people believe it to be necessary to munge the Windows registry when installing Python packages; I just don't agree with the practice, and don't think we should distort Python's process to coddle it). Uninstallation needs to revert anything installation did, so it is often more than mere removal of files. Practically speacking, nobody but the author of the TC-abusing setup.py is *ever* going to be able to do this, and most of them won't get the edge cases right. Maybe we can just punt on such packages, and make a tool which actually works for the huge majority of distributions which don't need to do more than install files. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD4DBQFH6Cdr+gerLs4ltQ4RAlFzAJi3gs8fzb9o8/Dtct1G9P0EJxNSAKCc7V7m uT5MgTzltBDhdrgoNxt8nA== =zgqI -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin v. Löwis wrote: Oh, and application installation is (should be) completely different. On Windows, applications should probably be bundled with their own Python interpreter, a la py2exe. On Unix/Linux, I don't know what the standard is, so I'd have to defer to others. This I disagree with. I think it's an overall bad thing to have all kinds of applications ship their own copy of Python; see also Aza Raskin's PyCon keynote. On Linux, python typically comes with the system pre-installed; it is not even an option not to have it, except for minimalist installations. So if you write python scripts, you typically expect that #!/usr/bin/env python works; you might put python2.5 there if you don't trust that system one is good enough. For installing the application, you typically want the choice betaween a system installation (in /usr/bin, or perhaps /usr/local/bin), and a user installation. As distutils supports both cases, it works fairly well for applications as well. Sharing the system python is hugely problematic on a unix box which actually *uses* python for its own tools: the application is not safe from additions / updates / removeals of the packages in /usr/lib/python2.x/site-packages done to support those system tools. The problem gets worse as multiple non-system applications install files there: e.g., the 'twisted' package on Debian boxes depends on an ancient version of 'zope.interface', which can't be used with any currently supported version of Zope. virtualenv makes using the system python on such systems somewhat more tolerable, because each virtualenv can be isolated from the site-packages of the host environment (and therefore from the other applications). You still have to live with the choices the system packagers make (UCS4, for instance), but the pain is at least tolerable. For a long-running Python application (as opposed to desktop tools or scripts), installing a custom Python is the safest choice: it insulates the application not only from unexpected system-driven site-packages changes, but also allows greater control over other Python configuration choices (the UCS2 / UCS4 knob, etc.). Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH6Cqi+gerLs4ltQ4RAlqZAKCxr2lraLSycVsksYAevtf+urALOgCeLzs9 fE2g7IAb+22B+UbSUFPqj4w= =re0h -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On Mar 22, 5:13 pm, Martin v. Löwis [EMAIL PROTECTED] wrote: Can you give me a pointer to Aza Raskin's keynote? Is it online anywhere? I'd be interested in his point of view. Unfortunately no. I was looking for it, but couldn't find it. He mentioned a website with a call for action, but I couldn't find that, either :-( I guess the website could be http://www.toolness.com/wp/?p=23#more-23 - Python as a Platform. Via Ned Batchelder's notes at http://nedbatchelder.com/blog/200803/pycon_2008_notes.html From the post: Something that recently occurred to me is that the only operating system that doesn't come with Python pre-installed on it is Windows. While Linux and OS X both view Python as essentially a first-class development platform-i.e., as something that shrink-wrap applications can be built on-Windows does not. Instead, it's generally expected that a Python-based Windows application be frozen: bundled into a self-contained package that includes a copy of the Python interpreter and whatever libraries it uses, which are private to the particular application. While this ensures that the application will function as expected and not run into 'dependency hell', it also results in a relatively large download-distributing a simple 'Hello World' program is at least a megabyte in size, and makes extending the program's functionality more difficult. Regards, Daniel ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
In article [EMAIL PROTECTED], ajaksu [EMAIL PROTECTED] wrote: [...] While Linux and OS X both view Python as essentially a first-class development platform-i.e., as something that shrink-wrap applications can be built on-Windows does not. Instead, it's generally expected that a Python-based Windows application be frozen: bundled into a self-contained package that includes a copy of the Python interpreter and whatever libraries it uses, which are private to the particular application. While this ensures that the application will function as expected and not run into 'dependency hell', it also results in a relatively large download-distributing a simple 'Hello World' program is at least a megabyte in size, and makes extending the program's functionality more difficult. While it is true that OS X does provide a built-in Python that can be used as a shared component for shrink-wrap applications, it should be noted that py2app, the de facto standard tool for packaging Python apps on OS X, by default includes a private copy of the Python interpreter and library within the application bundle for similar reasons, i.e. avoiding dependency hell. py2app does, however, go to some trouble to analyze dependencies and include only a minimal set of required packages. http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html#id35 -- Ned Deily, [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On Sat, Mar 22, 2008 at 09:13:24PM +0100, Martin v. L?wis wrote: Unfortunately no. I was looking for it, but couldn't find it. He mentioned a website with a call for action, but I couldn't find that, either :-( We're working on the video, but I think it'll be a few weeks before things start to get posted. --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On Mar 22, 5:13 pm, Martin v. Löwis [EMAIL PROTECTED] wrote: Unfortunately no. I was looking for it, but couldn't find it. He mentioned a website with a call for action, but I couldn't find that, either :-( As I tried to reply (in a message that is waiting for moderation), the site seems to be http://www.toolness.com/wp/?p=23 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On 22 Mar, 2008, at 2:44, A.M. Kuchling wrote: On Fri, Mar 21, 2008 at 06:41:00PM -0400, Phillip J. Eby wrote: I'm making the assumption that the author(s) of PEP 262 had good reason for including what they did, rather than assuming that we should start the entire process over from scratch. The goal *was* originally to provide for RPM-like verification of file content, but I don't know that the verification feature really matters that much; OSes with packaging systems already support such a feature, probably, and it probably isn't particularly useful for systems without a packaging system. Why not? Checksums would allow you to build a small packaging system on top the python system. Ronald smime.p7s Description: S/MIME cryptographic signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Another use case, which I find in my world, is that there are always packages that interest me (found at pypi), that my vendor hasn't packaged as rpms yet. With finite resources, this will always be true. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
M.-A. Lemburg wrote: On 2008-03-21 22:21, Phillip J. Eby wrote: At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: I guess the only way to support all of these variants is to use a filesystem based approach, e.g. by placing a file with a special extension into some dir on sys.path. The database logic could then scan sys.path for these files, read the data and provide an interface to it. All bdist formats would then have to include these files. That's the idea behind the current version of PEP 262, yes, and I think it should be kept. A separate FILES section also doesn't seem to be necessary - we could just add one or more entries or the format: CreatesDir abc/ CreatesFile abc/xyz1.py CreatesDir abc/def/ CreatesFile abc/def/xyz2.py CreatesFile abc/def/xyz3.py CreatesFile abc/def/xyz4.ini I actually think the size and hash information is good, in order to be able to tell if you're looking at an original file. I'm not sure how useful the permissions and uid/gid info is. I'm hoping we'll hear from anybody who has a use case for that. You're heading off in the wrong direction: we should not be trying to rewrite RPM or InnoSetup in Python. Anything more complicated should be left to tools which are specifically written to manage complex software setups. We *do* need a way to play nice with all the various platform installers. This would surely be welcomed by the many third parties who now have to package Python for their platforms. I honestly believe that most people would be happy if we just provide these two things (and no more): * install a package from a local archive, a URL or PyPI * uninstall a package in way that doesn't break other installed packages and whatever the mechanism, avoid making any undercover changes to the Python installation such as adding .pth files, overriding site.py, etc. - these are not needed if the tool keeps to the simple task of installing and uninstalling Python packages. Examples: python pypi.py install mypkg-1.0.tgz python pypi.py install http://www.example.com/mypkg-1.0.tgz python pypi.py install mypkg-1.0 python pypi.py uninstall mypkg If there's a dependency problem, the tool should print the list of other packages it needs. It should not try to install things automagically. ... unless explicitly asked to do so? It seems silly to omit this capability when it's essentially just omitting a recursive call. If a package needs other modules as well, the package docs can point the user to use e.g. python pypi.py install mydep1-1.3 mydep2-2.3 mydep4-0.3 mypkg-1.0 instead. Why would this be better than using --load-dependencies? Anything more complicated should be left to specialized tools such as RPM, apt, MSI or the other such tools out there - after all the tool should be about Python *package* installation, not application installation. We *don't* need the tool to: * support multiple versions of a package (that's just bound to cause problems with pickle, isinstance() etc.) Another argument for installing apps as separate components with all dependencies. I really don't believe enough consideration has been given as to the essential difference between these two activities. * provide namespace hacking (is a completely separate issue and can be handled by the packages rather than the install tool) * support all kinds of funky version numbers (if a package wants to participate in the system, the author better make sure that the version string fits the standard format) Agreed. * provide some form of intra-package bus interface (ie. entry points as you call them) * provide support for keeping whole packages in ZIP files (doesn't play well with C extensions, clutters up the sys.path, is read-only, needs special importers, etc. etc. ) It shouldn't require special importers, though. And if the zip file contains compiled code the read-only nature doesn't matter if the zips are version-specific. * try automatic version matching for required packages * download things from SourceForge or other sites with special download mechanisms * scan websites for links * make coffee, clean the house, send the kids to school :-) But a package that *would* do this could be immensely popular. And of course, there are still some issues to be resolved regarding requirements, package name/version stuff, etc. But we can hash those out once we reach a quorum on the Distutils-SIG. Well, I've probably been killfiled into non-existence on this list by now, but it seems to me that we are in danger of answering the wrong problem yet again. But that's all I have to say on this topic, so you can all heave a sigh a relief and get on with messing it up ;-) regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Ok, well, I have a rationale for it: make it possible to get rid of eggs as a mechanism for supporting easy_install. Many people (yourself included) have criticized eggs as an installation mechanism, and this is an alternative that gets rid of .egg files and directories in that case, and most of the need for .pth file usage as well. How so? I cannot see the need for .egg files or .pth files in the first place. If they exist so that you can import stuff: just install to site-packages, and be done. The data isn't for them to use to meet their use cases, it's for them to *provide* so that Python tools don't stomp on, uninstall, or otherwise interfere with files installed by the system. In other words, for system packagers, it's a communication from the system to Python, rather than the other way around. Even though the distutils will build the file in the bdist, the system packaging tools would be free to generate their own file listing and signatures and such. Ok, that's a reasonable requirement. It will be difficult to implement, though, as it will require Linux distributors (in particular) to include the database snippets in their packages. Essentially, one would have to contribute patches to all the distributions (we care about, at least), and then nag the respective maintainers to include these patches. I probably should have brought this up, in fact, I think I mentioned it in a previous thread, but I would like to see PEP 262 add a way to say this is a system-installed package, *don't touch*. The idea again is not to do the job of the native packaging system, but rather to ensure that Python-specific tools (e.g. easy_install and friends) do not interfere or conflict with it. Something like a read-only flag? For those without the read-only flag, the specification should explicitly say what manipulation is allowed. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On 22/03/2008, Steve Holden [EMAIL PROTECTED] wrote: Well, I've probably been killfiled into non-existence on this list by now, but it seems to me that we are in danger of answering the wrong problem yet again. But that's all I have to say on this topic, so you can all heave a sigh a relief and get on with messing it up ;-) You probably have my company in the killfile, but I have a nagging feeling in the same direction. My biggest problem is that I can't express what I believe is the *right* problem, beyond the over-general statement that it seems crucial to me that there should be a single, unified way of managing *all* packages installed in a given Python installation. Whether that's a Python-only solution, or the system packager, doesn't matter. There should be only one way to do it, to reuse a well-known phrase :-) If you know how to state nature of the right problem, that would be useful. All this talk of playing nicely with the system packager seems to imply that people are designing a second solution, and trying to manage the interaction, rather than deciding on *one* solution (which by definition has no interaction to worry about). It's reasonable to have multiple solutions for multiple Python installations (system packager for the system python, python packager for a local install, for example) but that's a different matter. Oh, and application installation is (should be) completely different. On Windows, applications should probably be bundled with their own Python interpreter, a la py2exe. On Unix/Linux, I don't know what the standard is, so I'd have to defer to others. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
At 12:33 PM 3/22/2008 +0100, Martin v. Löwis wrote: I probably should have brought this up, in fact, I think I mentioned it in a previous thread, but I would like to see PEP 262 add a way to say this is a system-installed package, *don't touch*. The idea again is not to do the job of the native packaging system, but rather to ensure that Python-specific tools (e.g. easy_install and friends) do not interfere or conflict with it. Something like a read-only flag? Not exactly. More like, package management tool X claims exclusive rights to this package. Python tools would always defer this right to the system packager, i.e. a system packager is not obliged to respect a Python tool's claim to a file, but not the other way around. That way, system packaging tools don't need to do anything but mark the installed files as belonging to them. Since most vendors at least *begin* with a setup.py install, we could provide a way to indicate that the installation is being done on behalf of a system packaging tool, so that it can provide that indication. For those without the read-only flag, the specification should explicitly say what manipulation is allowed. Since a distribution isn't really mutable, I would think that uninstallation and reinstallation would be the only manipulation available. (As distinct from inspection, verification, and other read-only activities.) It's possible, though, that there might also be actions such as restoring or relocating scripts or data in shared locations outside of the sys.path directory. That will get clearer as the spec gets defined. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
For those without the read-only flag, the specification should explicitly say what manipulation is allowed. Since a distribution isn't really mutable, I would think that uninstallation and reinstallation would be the only manipulation available. (As distinct from inspection, verification, and other read-only activities.) Sure, but what is precisely the semantics of uninstallation, in terms of changes to the system state? I think any model where uninstallation is merely the removal of files is too limited to be practical. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On 22/03/2008, Phillip J. Eby [EMAIL PROTECTED] wrote: This probably needs to be refined a little. Exclusive right is too strong, and it goes against Paul Moore's desire for using a single tool. Huh? How's that? Don't forget that I'm on Windows, and on Windows there is no system tool - just bdist_wininst, bdist_msi and easy_install. The fact that bdist_wininst and bdist_msi link into the system UI for listing and uninstallation doesn't make packages using them system packages. If easy_install put add/remove program info in place, my single tool would be the add/remove list. If something else (for example, the proposed index of installed package, with a suitable UI) is deemed the single tool, then bdist_wininst and bdist_msi have to be changed to respect that. The only fly in this ointment is bdist_msi, which uses MSI format, which is a lot nearer to a Windows system packager than anything else. Whether that means bdist_msi can't be changed to work with a package index rather than (or as well as, I don't care) add/remove, I don't know. Unfortunately, a warning message might then need to be localized. So this idea still needs some work. The one way to do it uninstaller should be able to uninstall everything. If it needs to call the system uninstaller for a specific package, there's nothing wrong with doing that. But why tell me to run a different command? Just do it for me. I only want one UI, the rest is implementation detail. (Others may have different preferences, so a choice may need to be made. If so, and if it goes against me, fair enough, I have to decide what to do about that for myself. But I'd rather force people to tell me no, than leave people thinking they satisfied me and wondering why I'm still complaining...) Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
At 04:29 PM 3/22/2008 +0100, Martin v. Löwis wrote: For those without the read-only flag, the specification should explicitly say what manipulation is allowed. Since a distribution isn't really mutable, I would think that uninstallation and reinstallation would be the only manipulation available. (As distinct from inspection, verification, and other read-only activities.) Sure, but what is precisely the semantics of uninstallation, in terms of changes to the system state? I think any model where uninstallation is merely the removal of files is too limited to be practical. The distutils only support the *addition* of files, so I'm not sure how only removing files is a limit here. Could you explain? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
I more question the permissions and uid/gid stuff; I'm not really clear on what I'd use that stuff for in easy_install/uninstall/etc. I think this was all captured in amk's statement RPM-like verification. RPM not only verifies file content, but also whether the permissions are all correct. So if the administrator mistakenly does a chown -R on a subtree, he can get back to what it was with the package manager, without having to reinstall everything, or can atleast find out what packages he broke. IIUC, the Solaris package manager provides the same feature. MSI provides a repair installation, which doesn't really check anything, but reruns the entire installation (and then skipping those files who passed the checksum test, where checksums had been recorded in the MSI). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Neal Becker wrote: Another use case, which I find in my world, is that there are always packages that interest me (found at pypi), that my vendor hasn't packaged as rpms yet. With finite resources, this will always be true. Why do you need a package database for that? Can't you just run setup.py install, or perhaps setup.py bdist_rpm, and then install the RPM? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Huh? How's that? Don't forget that I'm on Windows, and on Windows there is no system tool - just bdist_wininst, bdist_msi and easy_install. The fact that bdist_wininst and bdist_msi link into the system UI for listing and uninstallation doesn't make packages using them system packages. In pje's terminology, they do. He uses system package as a shorthand for package in a format defined by the system vendor, not as package supplied by the system vendor (IIUC). So .msi files and self-extracting .exe files are all system packages, as opposed to .eggs (which are in a format that wasn't defined by the system vendor). The only fly in this ointment is bdist_msi, which uses MSI format, which is a lot nearer to a Windows system packager than anything else. Whether that means bdist_msi can't be changed to work with a package index rather than (or as well as, I don't care) add/remove, I don't know. If the package database is merely a directory with additional files in it, one file per package, then most likely both bdist_wininst and bdist_msi could support that. If the file needs to contain file names specific to the target system, then supporting it in bdist_msi is a bit tricky, as one would have to generate the file at installation time. That's a custom action; I'd probably generate a VB script at packaging time which is then run at installation time to edit the package database. The one way to do it uninstaller should be able to uninstall everything. If it needs to call the system uninstaller for a specific package, there's nothing wrong with doing that. But why tell me to run a different command? Just do it for me. I only want one UI, the rest is implementation detail. The uninstallation procedure of the system installer probably has a separate UI which can't really be suppressed. For example, uninstallation may be rejected as additional applications rely on the package, or uninstallation could cause automatic removal of prerequisite packages that are then no longer used, all requiring user confirmation. Also, on some systems, it's difficult to know what specific tool to run to interact with the system packaging. On some systems, you have a choice of multiple command-line, text mode, and GUI tools, some of which may not be installed, or may fail to work (e.g. when you don't have the windowing system running, and the tool is a windowed one), or may not be the user's preference. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Sure, but what is precisely the semantics of uninstallation, in terms of changes to the system state? I think any model where uninstallation is merely the removal of files is too limited to be practical. The distutils only support the *addition* of files, so I'm not sure how only removing files is a limit here. Could you explain? For files, yes, it only supports addition. But it supports arbitrary other actions, such as: - addition of registry keys - addition of user accounts - creation of database tables in a relational database - updating the shared library loader path - creation and start of a system service - integration of documentation into info - registration of DTDs with the system catalog - ... It's turing-complete, and it has full interface to the operating system, so installation of a distutils package can do *much* more than merely installing files. Uninstallation needs to revert anything installation did, so it is often more than mere removal of files. HTH, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Oh, and application installation is (should be) completely different. On Windows, applications should probably be bundled with their own Python interpreter, a la py2exe. On Unix/Linux, I don't know what the standard is, so I'd have to defer to others. This I disagree with. I think it's an overall bad thing to have all kinds of applications ship their own copy of Python; see also Aza Raskin's PyCon keynote. On Linux, python typically comes with the system pre-installed; it is not even an option not to have it, except for minimalist installations. So if you write python scripts, you typically expect that #!/usr/bin/env python works; you might put python2.5 there if you don't trust that system one is good enough. For installing the application, you typically want the choice between a system installation (in /usr/bin, or perhaps /usr/local/bin), and a user installation. As distutils supports both cases, it works fairly well for applications as well. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On 22/03/2008, Martin v. Löwis [EMAIL PROTECTED] wrote: Oh, and application installation is (should be) completely different. On Windows, applications should probably be bundled with their own Python interpreter, a la py2exe. On Unix/Linux, I don't know what the standard is, so I'd have to defer to others. This I disagree with. I think it's an overall bad thing to have all kinds of applications ship their own copy of Python; see also Aza Raskin's PyCon keynote. Is this on Windows? It's fairly common practice. Can you give me a pointer to Aza Raskin's keynote? Is it online anywhere? I'd be interested in his point of view. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Oh, and application installation is (should be) completely different. On Windows, applications should probably be bundled with their own Python interpreter, a la py2exe. On Unix/Linux, I don't know what the standard is, so I'd have to defer to others. This I disagree with. I think it's an overall bad thing to have all kinds of applications ship their own copy of Python; see also Aza Raskin's PyCon keynote. Is this on Windows? It's fairly common practice. Unfortunately so, yes. This can be viewed a burden to the adoption of Python: for a small application, you get this huge download to bundle. Can you give me a pointer to Aza Raskin's keynote? Is it online anywhere? I'd be interested in his point of view. Unfortunately no. I was looking for it, but couldn't find it. He mentioned a website with a call for action, but I couldn't find that, either :-( Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Phillip J. Eby wrote: Second, there were no uninstall tools for it, so I'd have had to write one myself. (Zed's easy_f'ing_uninstall to the contrary, it ain't easy, and I have an aversion to deleting stuff on people's systems without knowing what will break. There's a big difference between them typing 'rm -rf' themselves, and me doing it.) I think, the uninstall should _not_ 'rm -rf' but only 'rm' the files (and 'rmdir' directories, but not recursively) that it created, and that have not been modified in the meantime (after the installation). This can be easily achieved by recording a checksum (eg. md5 or sha) upon installation and only deleting a file if the checksum is correct and only deleting directories when they are empty (after the installed files in them have been deleted). Otherwise, the uninstall should complain and leave the modified files installed. Joachim ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On Fri, Mar 21, 2008 at 2:47 PM, Phillip J. Eby [EMAIL PROTECTED] wrote: Second, there were no uninstall tools for it, so I'd have had to write one myself. (Zed's easy_f'ing_uninstall to the contrary, it ain't easy, and I have an aversion to deleting stuff on people's systems without knowing what will break. There's a big difference between them typing 'rm -rf' themselves, and me doing it.) Agree. I tried a while ago to write a easy_uninstall but this is not possible from an application-point of view, to do clean things. zc.buildout resolves this by installing eggs locally, to an isolated environment, so my main Python installation doesn't hold any extensions at all. So if a database of installed package is created, I would be in favor of an application-oriented system where it is possible to create, update, install, a set of packages dedicated to an environment (default would be Python). Maybe by having a namespace tied to a list of versions. In other words; having the same feature virtualenv provides, in Python itself, and define somehow how to switch to it. $ easy_install the.package --environment MyApp ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Joachim I think, the uninstall should _not_ 'rm -rf' but only 'rm' the Joachim files (and 'rmdir' directories, but not recursively) that it Joachim created, and that have not been modified in the meantime (after Joachim the installation). That's not sufficient. Suppose file C (e.g. /usr/local/etc/mime.types) is in both packages A and B. Install A - this will create C Install B - this might overwrite C, saving a copy, or it might retain A's copy. Uninstall B - this has to know that C is used by A and not touch it Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Phillip J. Eby schrieb: Questions, comments... volunteers? :) I've yet to read the monster package utils thread so I can't comment on it. However I like to draw some attention to my PEP 370 http://python.org/dev/peps/pep-0370/. It's about a site packages directory in the users home directory. I think it quite related to the discussion. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
[EMAIL PROTECTED] writes: Joachim I think, the uninstall should _not_ 'rm -rf' but only 'rm' the Joachim files (and 'rmdir' directories, but not recursively) that it Joachim created, and that have not been modified in the meantime (after Joachim the installation). That's not sufficient. Suppose file C (e.g. /usr/local/etc/mime.types) is in both packages A and B. Install A - this will create C Install B - this might overwrite C, saving a copy, or it might retain A's copy. Uninstall B - this has to know that C is used by A and not touch it MacPorts has an expensive but interesting approach. When it finds a file used by another package, it backs it up to sth like $file.`date`. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
At 11:21 AM 3/21/2008 -0500, [EMAIL PROTECTED] wrote: Joachim I think, the uninstall should _not_ 'rm -rf' but only 'rm' the Joachim files (and 'rmdir' directories, but not recursively) that it Joachim created, and that have not been modified in the meantime (after Joachim the installation). That's not sufficient. Suppose file C (e.g. /usr/local/etc/mime.types) is in both packages A and B. Install A - this will create C Install B - this might overwrite C, saving a copy, or it might retain A's copy. Uninstall B - this has to know that C is used by A and not touch it Correct. However, in practice, B should not touch C, unless the file is shared between them. This is a key issue for support of namespace packages, at least if we want to avoid using .pth files. (Which is what setuptools-built system packages do for namespace packages currently.) Of course, one possible solution is for both A and B to depend on a virtual package that contains C, such that both A and B can install it if it's not there, and list it in their dependencies. But this is one of the handful of open issues that needs to be resolved with Real Life Package Management people, such as Debian, Fedora, etc. Neither overwriting, refusing to install, nor backups will properly address this issue. However, this is properly a topic for the Distutils-SIG or whatever SIG the actual spec goes to. On Python-Dev I'm only looking for a go/no-go on the overall approach. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On 2008-03-21 14:47, Phillip J. Eby wrote: So, to accomplish this, we (for some value of we) need to: 1. Hash out consensus around what changes or enhancements are needed to PEP 262, to resolve the previously-listed open issues, those that have come up since (namespace packages, dependency specifications, canonical name/version forms), and anything else that comes up. 2. Update or replace the implementation as appropriate, and modify the distutils to support it in Python 2.6 and beyond. And support it means, ensure that 'install' and *all* bdist commands update the database. The bdist_rpm, bdist_wininst, and bdist_msi commands, even bdist_dumb. (This should probably also include the add/remove programs stuff in the Windows case.) The bdist commands don't need to touch that database in any way, since they don't install anything, nor do they upload things anywhere. They simply package code and put the result into the dist/ subdir. That's all. What you probably mean is that the installers, pre/post-scripts, etc. run when installing one of those packages should update the database of installed packages. Note that there are several package formats which do not execute any code when installing them - the user simply unzips them in some directory. These packages won't be able to register themselves with a database. I guess the only way to support all of these variants is to use a filesystem based approach, e.g. by placing a file with a special extension into some dir on sys.path. The database logic could then scan sys.path for these files, read the data and provide an interface to it. All bdist formats would then have to include these files. distutils already writes .egg-info files when running python setup.py install, so perhaps that's a start (though I'd prefer a three letter extension such as .pkg). .egg-info files currently only include the package meta-data (the PKG-INFO section from PEP 262). We'd have to add a list of files making up the package (FILES section in PEP 262) and also some extra information about any extra files the package creates that can safely be removed in the uninstall process (e.g. .pyo and .pyc files, temporary files, database files, configuration data, registry entries, etc.) - this is currently not covered in PEP 262. I don't think the REQUIRES and PROVIDES sections from the PEP 262 are needed. That info can easily go into the PKG-INFO section. A separate FILES section also doesn't seem to be necessary - we could just add one or more entries or the format: CreatesDir abc/ CreatesFile abc/xyz1.py CreatesDir abc/def/ CreatesFile abc/def/xyz2.py CreatesFile abc/def/xyz3.py CreatesFile abc/def/xyz4.ini (BTW: wininst writes such a file for the uninstall process) So to keep things simple, the rfc822 approach defined in PEP 241 would easily cover everything needed and we could trim down the PEP 262 format to a simple rfc822 header list. In other words: the .egg-info files already provide the basis and only need to be extended with a list of created files, directories (and possibly other resources) as well as a list of resources which may be removed even if not installed explicitly such as byte-code files, etc. 3. Create a document for system packagers referencing the PEP and introducing them to what/why/how of the standard, in case they weren't one of the original participants in creating this. This should probably be a new PEP defining all the bits and pieces making up the installation database. It will probably take some non-trivial work to do all this for Python 2.6, but it's probably possible, if we start now. I don't think it's critical to have an uninstall tool distributed with 2.6, as long as there's a reasonable way to bootstrap its installation later. BTW: There's a simple uninstall command in mxSetup.py that we could contribute to distutils. It works much in the same way as the install command... except that it removes all the files it would have installed. Using pre-built packages, this works without having to rebuild the package just to be able to determine the list of things that need to be removed. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2008) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: I guess the only way to support all of these variants is to use a filesystem based approach, e.g. by placing a file with a special extension into some dir on sys.path. The database logic could then scan sys.path for these files, read the data and provide an interface to it. All bdist formats would then have to include these files. That's the idea behind the current version of PEP 262, yes, and I think it should be kept. A separate FILES section also doesn't seem to be necessary - we could just add one or more entries or the format: CreatesDir abc/ CreatesFile abc/xyz1.py CreatesDir abc/def/ CreatesFile abc/def/xyz2.py CreatesFile abc/def/xyz3.py CreatesFile abc/def/xyz4.ini I actually think the size and hash information is good, in order to be able to tell if you're looking at an original file. I'm not sure how useful the permissions and uid/gid info is. I'm hoping we'll hear from anybody who has a use case for that. And of course, there are still some issues to be resolved regarding requirements, package name/version stuff, etc. But we can hash those out once we reach a quorum on the Distutils-SIG. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On 2008-03-21 22:21, Phillip J. Eby wrote: At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: I guess the only way to support all of these variants is to use a filesystem based approach, e.g. by placing a file with a special extension into some dir on sys.path. The database logic could then scan sys.path for these files, read the data and provide an interface to it. All bdist formats would then have to include these files. That's the idea behind the current version of PEP 262, yes, and I think it should be kept. A separate FILES section also doesn't seem to be necessary - we could just add one or more entries or the format: CreatesDir abc/ CreatesFile abc/xyz1.py CreatesDir abc/def/ CreatesFile abc/def/xyz2.py CreatesFile abc/def/xyz3.py CreatesFile abc/def/xyz4.ini I actually think the size and hash information is good, in order to be able to tell if you're looking at an original file. I'm not sure how useful the permissions and uid/gid info is. I'm hoping we'll hear from anybody who has a use case for that. You're heading off in the wrong direction: we should not be trying to rewrite RPM or InnoSetup in Python. Anything more complicated should be left to tools which are specifically written to manage complex software setups. I honestly believe that most people would be happy if we just provide these two things (and no more): * install a package from a local archive, a URL or PyPI * uninstall a package in way that doesn't break other installed packages and whatever the mechanism, avoid making any undercover changes to the Python installation such as adding .pth files, overriding site.py, etc. - these are not needed if the tool keeps to the simple task of installing and uninstalling Python packages. Examples: python pypi.py install mypkg-1.0.tgz python pypi.py install http://www.example.com/mypkg-1.0.tgz python pypi.py install mypkg-1.0 python pypi.py uninstall mypkg If there's a dependency problem, the tool should print the list of other packages it needs. It should not try to install things automagically. If a package needs other modules as well, the package docs can point the user to use e.g. python pypi.py install mydep1-1.3 mydep2-2.3 mydep4-0.3 mypkg-1.0 instead. Anything more complicated should be left to specialized tools such as RPM, apt, MSI or the other such tools out there - after all the tool should be about Python *package* installation, not application installation. We *don't* need the tool to: * support multiple versions of a package (that's just bound to cause problems with pickle, isinstance() etc.) * provide namespace hacking (is a completely separate issue and can be handled by the packages rather than the install tool) * support all kinds of funky version numbers (if a package wants to participate in the system, the author better make sure that the version string fits the standard format) * provide some form of intra-package bus interface (ie. entry points as you call them) * provide support for keeping whole packages in ZIP files (doesn't play well with C extensions, clutters up the sys.path, is read-only, needs special importers, etc. etc. ) * try automatic version matching for required packages * download things from SourceForge or other sites with special download mechanisms * scan websites for links * make coffee, clean the house, send the kids to school :-) And of course, there are still some issues to be resolved regarding requirements, package name/version stuff, etc. But we can hash those out once we reach a quorum on the Distutils-SIG. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2008) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
At 11:13 PM 3/21/2008 +0100, M.-A. Lemburg wrote: On 2008-03-21 22:21, Phillip J. Eby wrote: At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: I guess the only way to support all of these variants is to use a filesystem based approach, e.g. by placing a file with a special extension into some dir on sys.path. The database logic could then scan sys.path for these files, read the data and provide an interface to it. All bdist formats would then have to include these files. That's the idea behind the current version of PEP 262, yes, and I think it should be kept. A separate FILES section also doesn't seem to be necessary - we could just add one or more entries or the format: CreatesDir abc/ CreatesFile abc/xyz1.py CreatesDir abc/def/ CreatesFile abc/def/xyz2.py CreatesFile abc/def/xyz3.py CreatesFile abc/def/xyz4.ini I actually think the size and hash information is good, in order to be able to tell if you're looking at an original file. I'm not sure how useful the permissions and uid/gid info is. I'm hoping we'll hear from anybody who has a use case for that. You're heading off in the wrong direction: we should not be trying to rewrite RPM or InnoSetup in Python. I'm making the assumption that the author(s) of PEP 262 had good reason for including what they did, rather than assuming that we should start the entire process over from scratch. Anything more complicated should be left to tools which are specifically written to manage complex software setups. Tools which will need this data, in order to do their work. Hence, the reason for standardizing the data, instead of the tool(s). [snip long list of features, both desired and undesired] Actually, *all* of these features are out of scope for stdlib development, because I'm not proposing including *any* tools for this in the stdlib, apart from distutils install and bdist_* support. I'm proposing, rather, that we finish the vision of PEP 262, of having a standard specification that *all* tools will abide by -- including rpm, dpkg, and what-have-you. Since *all* of these tools need to abide by that specification, their requirements will need to be considered in the formulation of the spec. And as I said, I'll be happy if all we do is get the distutils to abide by the spec for 2.6, even if it means we don't get an uninstall tool. That can always be installed later using Guido's bootstrap tool. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
I'm making the assumption that the author(s) of PEP 262 had good reason for including what they did, rather than assuming that we should start the entire process over from scratch. The objections to the PEP remain the same as they were then, though: In the requirements, it says we need, without saying why we need. It then goes on saying we want (rephrased) to duplicate APT and RPM, without saying why we want that, or why that brings us closer to what we need. IOW, the PEP is lacking a rationale. Anything more complicated should be left to tools which are specifically written to manage complex software setups. Tools which will need this data, in order to do their work. Hence, the reason for standardizing the data, instead of the tool(s). If there was a chance that the infrastructure being developed actually helps these tools, *that* would be a reasonable goal, IMO. However, I'm extremely skeptical that this can ever succeed to the degree that whoever provides RPMs, .debs, or MSI files will actually use such data, as they will find that the data are incomplete, and they have to redo all of it, anyway. I'm proposing, rather, that we finish the vision of PEP 262, of having a standard specification that *all* tools will abide by -- including rpm, dpkg, and what-have-you. Do you also envision the objective of PEP 262, then? I.e. to provide a database of installed packages, in .../install-db? And as I said, I'll be happy if all we do is get the distutils to abide by the spec for 2.6, even if it means we don't get an uninstall tool. That can always be installed later using Guido's bootstrap tool. :) I'm even more skeptical here. If the assumption is that the package database allows for uninstallation, I'm -1. IOW, RPM, deb, MSI should then *not* write to that package database, as they also write to a different database, out of the scope of the PEP, and this is what uninstallation should use. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
On Fri, Mar 21, 2008 at 06:41:00PM -0400, Phillip J. Eby wrote: I'm making the assumption that the author(s) of PEP 262 had good reason for including what they did, rather than assuming that we should start the entire process over from scratch. The goal *was* originally to provide for RPM-like verification of file content, but I don't know that the verification feature really matters that much; OSes with packaging systems already support such a feature, probably, and it probably isn't particularly useful for systems without a packaging system. --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
At 02:31 AM 3/22/2008 +0100, Martin v. Löwis wrote: I'm making the assumption that the author(s) of PEP 262 had good reason for including what they did, rather than assuming that we should start the entire process over from scratch. The objections to the PEP remain the same as they were then, though: In the requirements, it says we need, without saying why we need. It then goes on saying we want (rephrased) to duplicate APT and RPM, without saying why we want that, or why that brings us closer to what we need. IOW, the PEP is lacking a rationale. Ok, well, I have a rationale for it: make it possible to get rid of eggs as a mechanism for supporting easy_install. Many people (yourself included) have criticized eggs as an installation mechanism, and this is an alternative that gets rid of .egg files and directories in that case, and most of the need for .pth file usage as well. If there was a chance that the infrastructure being developed actually helps these tools, *that* would be a reasonable goal, IMO. Yes, I'm of course primarily interested in Python-specific tools such as virtualenv, easy_install, buildout, and the as-yet-unwritten uninstallers, package listers, etc., that can usefully read or write such data. However, I'm extremely skeptical that this can ever succeed to the degree that whoever provides RPMs, .debs, or MSI files will actually use such data, as they will find that the data are incomplete, and they have to redo all of it, anyway. The data isn't for them to use to meet their use cases, it's for them to *provide* so that Python tools don't stomp on, uninstall, or otherwise interfere with files installed by the system. In other words, for system packagers, it's a communication from the system to Python, rather than the other way around. Even though the distutils will build the file in the bdist, the system packaging tools would be free to generate their own file listing and signatures and such. Do you also envision the objective of PEP 262, then? I.e. to provide a database of installed packages, in .../install-db? In each directory relative to a given sys.path directory, yes. That is, installing a distutils distribution to any directory would result in a file being added to an install-db within that directory. (Assuming we use the proposed implementation model of PEP 262, which at the moment I don't see any substantial obstacle to.) And as I said, I'll be happy if all we do is get the distutils to abide by the spec for 2.6, even if it means we don't get an uninstall tool. That can always be installed later using Guido's bootstrap tool. :) I'm even more skeptical here. If the assumption is that the package database allows for uninstallation, I'm -1. IOW, RPM, deb, MSI should then *not* write to that package database, as they also write to a different database, out of the scope of the PEP, and this is what uninstallation should use. I probably should have brought this up, in fact, I think I mentioned it in a previous thread, but I would like to see PEP 262 add a way to say this is a system-installed package, *don't touch*. The idea again is not to do the job of the native packaging system, but rather to ensure that Python-specific tools (e.g. easy_install and friends) do not interfere or conflict with it. A big problem in the early development of easy_install, even using eggs, was that there was no way to tell whether it was safe to delete or overwrite an existing file or directory that was already installed on the system. A mechanism like this would allow tools like easy_install to say, oh, your system packager has a conflicting package here, you need to use that tool to sort this out if you really want to do something here. I'm not going to touch that. Without something like this, there is no way to tell the difference on many systems between a system package and something the user has put there with sudo python setup.py install. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
At 09:44 PM 3/21/2008 -0400, A.M. Kuchling wrote: On Fri, Mar 21, 2008 at 06:41:00PM -0400, Phillip J. Eby wrote: I'm making the assumption that the author(s) of PEP 262 had good reason for including what they did, rather than assuming that we should start the entire process over from scratch. The goal *was* originally to provide for RPM-like verification of file content, but I don't know that the verification feature really matters that much; OSes with packaging systems already support such a feature, probably, and it probably isn't particularly useful for systems without a packaging system. Actually, it's the places where there's no packaging system that it's most useful. For example, an application that installs plugins to itself. A development environment with multiple virtual pythons. Users installing stuff to their PYTHONPATH, etc. In these cases, having the Python-specific tools be able to verify content signatures is useful, to make sure that you know what you're updating or removing. This is particularly important if one installs anything just by unpacking it into the target directory; you could overwrite something and then have only size/signature info to sort out whose version of the file is actually there. I more question the permissions and uid/gid stuff; I'm not really clear on what I'd use that stuff for in easy_install/uninstall/etc. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Phillip J. Eby wrote: At 11:21 AM 3/21/2008 -0500, [EMAIL PROTECTED] wrote: Joachim I think, the uninstall should _not_ 'rm -rf' but only 'rm' the Joachim files (and 'rmdir' directories, but not recursively) that it Joachim created, and that have not been modified in the meantime (after Joachim the installation). That's not sufficient. Suppose file C (e.g. /usr/local/etc/mime.types) is in both packages A and B. Install A - this will create C Install B - this might overwrite C, saving a copy, or it might retain A's copy. Uninstall B - this has to know that C is used by A and not touch it Correct. However, in practice, B should not touch C, unless the file is shared between them. This is a key issue for support of namespace packages, at least if we want to avoid using .pth files. (Which is what setuptools-built system packages do for namespace packages currently.) Of course, one possible solution is for both A and B to depend on a virtual package that contains C, such that both A and B can install it if it's not there, and list it in their dependencies. But this is one of the handful of open issues that needs to be resolved with Real Life Package Management people, such as Debian, Fedora, etc. I've always thought that the right way to handle the dependency DAG is to treat it as a garbage collection problem. Assume that for each package there is a way to derive the following two pieces of information: (a) whether this package was installed explicitly by the user, or implicitly as the result of a dependency, and (b) the set of dependencies for this package. Then, starting with the list of 'explicit' packages as the root set, do a standard mark sweep; Any package not marked is a candidate for removal. -- Talin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
