Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
Ronald Oussoren wrote: Code should always specify their dependencies, not doing so will cause you problems down the line... Sure, and I do that for for any real code. But do you really expect that users are careful to specify dependencies for each and every script they write. Yes, because it is easy. That may be true for development shops, but I don't expect that casual users are so careful. Then they shouldn't be surprised when things go wrong for them. But this discussion is moot anyway, the proposed metadata is meant to help packaging tools to track which packages seem to be orphaned and for which they can ask the user if those packages should be removed from the system, instead of always removing seemingly orphaned packages. Indeed. Although I'd hope there's some --prune option or such which automatically removes all orphaned packages. cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On 20 Oct, 2009, at 14:11, Chris Withers wrote: Ronald Oussoren wrote: 2) Notice that simplejson gets installed and is a useful module, then use that in your own scripts At this point, you specify your scripts' dependencies on simplejson. Look at it another way: You follow you process above, but after step 2, your development machine suffers a disk failure so you start from scratch on a new machine. Are you surprised at this point that simplejson isn't installed when you set things up again, even though by this point you've decided not to use TurboGears? Code should always specify their dependencies, not doing so will cause you problems down the line... Sure, and I do that for for any real code. But do you really expect that users are careful to specify dependencies for each and every script they write. That may be true for development shops, but I don't expect that casual users are so careful. But this discussion is moot anyway, the proposed metadata is meant to help packaging tools to track which packages seem to be orphaned and for which they can ask the user if those packages should be removed from the system, instead of always removing seemingly orphaned packages. Ronald smime.p7s Description: S/MIME cryptographic signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
Ronald Oussoren wrote: 2) Notice that simplejson gets installed and is a useful module, then use that in your own scripts At this point, you specify your scripts' dependencies on simplejson. Look at it another way: You follow you process above, but after step 2, your development machine suffers a disk failure so you start from scratch on a new machine. Are you surprised at this point that simplejson isn't installed when you set things up again, even though by this point you've decided not to use TurboGears? Code should always specify their dependencies, not doing so will cause you problems down the line... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On 9 Oct, 2009, at 15:45, Carl Meyer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alex wrote: REQUESTED is fine, but I don't understand how the arguments apply, given that I'm not proposing to record information like _which_ package it was a dependency of. The same single bit (literally) of information is tracked either way, it's just a question of whether the presence or absence of a file signifies that bit. And what if the package is a dependency for multiple packages? Let's say we have packages A and B which are installed separately, in that order, and both depend on C. C gets installed with information that it was required by A. Now if A is uninstalled, won't C also get uninstalled? No. We are NOT talking about recording the full dependency graph in package metadata. It is, as has always been the case, up to an uninstall tool to calculate the depgraph based on actual installed packages at runtime. A package is orphaned iff it was not REQUESTED by the user AND it is no longer depended on by other installed packages. What about packages that are installed as a dependency of some other package and then used in user scripts without an explict depency on them? That is, I install SuperWebFramework==1.0 which happens to depend on peak-rules. I later start using peak-rules in my own simple scripts (without a setup.py or other explicit dependency information), and yet later decide to uninstall SuperWebFramework. If I understand the proposal correctly the uninstallation of SuperWebFrameWork would break my scripts. I also wonder how this interacts with system package managers. The .egg-info directory-structure and PEP377 where explicitly structured to allow (for example) RPM to create the egg-info directory without having to know about Python's package repository (which would be needed if that repository were a single-file database instead of the collection of egg-info directories). Ronald Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzz531j/fhc23WEARAlPfAJ93YHsbpzp5kF7hR0f98aSMOae6MwCgrP52 RZs5FU3h0U0eUBN2vgqm2HA= =jyi8 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig smime.p7s Description: S/MIME cryptographic signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ronald Oussoren wrote: That is, I install SuperWebFramework==1.0 which happens to depend on peak-rules. I later start using peak-rules in my own simple scripts (without a setup.py or other explicit dependency information), and yet later decide to uninstall SuperWebFramework. If I understand the proposal correctly the uninstallation of SuperWebFrameWork would break my scripts. Once again, let's distinguish this proposal to record a bit of metadata from the behavior of a hypothetical tool using that metadata. This proposal does not mandate any such behavior from an uninstall tool, or suggest that it be the default behavior. Many implementation choices are possible: to name just one possibility, the removal of orphan packages could be a separate command, which might prompt before removing anything. This proposal is merely to make available an extra bit of relevant metadata that may be useful to some tools, under some use cases. I also wonder how this interacts with system package managers. The .egg-info directory-structure and PEP377 where explicitly structured to allow (for example) RPM to create the egg-info directory without having to know about Python's package repository (which would be needed if that repository were a single-file database instead of the collection of egg-info directories). This concern is a general one about uninstallation, and is unrelated to the proposal to add REQUESTED metadata. The REQUESTED proposal does not create a centralized Python installation database, and changes nothing about how Python package managers and system package managers interact. FWIW, PEP 376 already addresses this concern via the installer metadata, so tools can choose to only uninstall packages they installed. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK0gYA1j/fhc23WEARAu2yAJ45DYsMiDy+a85VW8khFRoTOwttjACdEFXb mraA5/j/oZ3MjroQslnVwhM= =Cdzb -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On 11 Oct, 2009, at 16:27, Lennart Regebro wrote: 2009/10/11 Ronald Oussoren ronaldousso...@mac.com: What about packages that are installed as a dependency of some other package and then used in user scripts without an explict depency on them? That is, I install SuperWebFramework==1.0 which happens to depend on peak-rules. I later start using peak-rules in my own simple scripts (without a setup.py or other explicit dependency information), and yet later decide to uninstall SuperWebFramework. If I understand the proposal correctly the uninstallation of SuperWebFrameWork would break my scripts. Yes of course. IMHO that is a bad experience for the user, because it is very unintuitive that I have to explicitly install something that's already installed to ensure that it doesn't go away in the future. This might work in a serious development environment, but can end up to be very annoying and confusing for casual users. Ronald smime.p7s Description: S/MIME cryptographic signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
Ronald Oussoren kirjoitti: On 11 Oct, 2009, at 16:27, Lennart Regebro wrote: 2009/10/11 Ronald Oussoren ronaldousso...@mac.com: What about packages that are installed as a dependency of some other package and then used in user scripts without an explict depency on them? That is, I install SuperWebFramework==1.0 which happens to depend on peak-rules. I later start using peak-rules in my own simple scripts (without a setup.py or other explicit dependency information), and yet later decide to uninstall SuperWebFramework. If I understand the proposal correctly the uninstallation of SuperWebFrameWork would break my scripts. Yes of course. IMHO that is a bad experience for the user, because it is very unintuitive that I have to explicitly install something that's already installed to ensure that it doesn't go away in the future. This might work in a serious development environment, but can end up to be very annoying and confusing for casual users. The dpkg package manager (Debian) does not uninstall orphaned packages straight away, but notifies the user and allows them to clean up any orphans. Perhaps a similar approach would work here? Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
2009/10/11 Ronald Oussoren ronaldousso...@mac.com: That is, I install SuperWebFramework==1.0 which happens to depend on peak-rules. I later start using peak-rules in my own simple scripts (without a setup.py or other explicit dependency information), and yet later decide to uninstall SuperWebFramework. If I understand the proposal correctly the uninstallation of SuperWebFrameWork would break my scripts. Yes of course. IMHO that is a bad experience for the user, because it is very unintuitive that I have to explicitly install something that's already installed to ensure that it doesn't go away in the future. But this is how all installs/uninstalls work and MUST work. If you uninstall the software that you use, then you can't use it. It's a simple necessity of life. This might work in a serious development environment, but can end up to be very annoying and confusing for casual users. Can you come up with something better? How will the packaging registry know that you want a particular package installed if you don't tell it? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On 11 Oct, 2009, at 20:58, Lennart Regebro wrote: 2009/10/11 Ronald Oussoren ronaldousso...@mac.com: That is, I install SuperWebFramework==1.0 which happens to depend on peak-rules. I later start using peak-rules in my own simple scripts (without a setup.py or other explicit dependency information), and yet later decide to uninstall SuperWebFramework. If I understand the proposal correctly the uninstallation of SuperWebFrameWork would break my scripts. Yes of course. IMHO that is a bad experience for the user, because it is very unintuitive that I have to explicitly install something that's already installed to ensure that it doesn't go away in the future. But this is how all installs/uninstalls work and MUST work. If you uninstall the software that you use, then you can't use it. It's a simple necessity of life. Sigh. 1) Install TurboGears, this installs simplejson as one of its dependencies 2) Notice that simplejson gets installed and is a useful module, then use that in your own scripts 3) Uninstall TurboGears With proposal of tracking which packages are installed as a dependency of other packages and automaticly uninstalling them when the package that depends on them gets uninstalled step 3 would break my scripts unless I explictly tell the installer that I really want to use simplejson in step 2. That's what I meant by I have to explictly install something that is already installed to ensure that it doesn't go away in the future, and this is not a useful behaviour. This might work in a serious development environment, but can end up to be very annoying and confusing for casual users. Can you come up with something better? How will the packaging registry know that you want a particular package installed if you don't tell it? The current proposal without tracking if I explicitly installed a package works fine and matches how system package managers work. This does make it harder to uninstall something like TurboGears that installs lots of dependencies, but it should be possible to write a tool that makes that easier. One way to do this in a GUI-ish manner is to have a show dependencies checkbox when you uninstall a package, this would then show all dependencies and allow easy selection of them for deinstallation. This gets you simular functionality as tracking explictly requested installs without needing additional metadata. Ronald smime.p7s Description: S/MIME cryptographic signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
As usual, these debates are based on a ot of assumptions that seem obvious to one part but not the other. I assumed that when you mentioned uninstalling the framework, you meant to go through the procedure of actually uninstalling all parts of it. Instead you seem to have meant the case that any command that uninstalled the framework also by necessity HAD to result in the uninstalling of all dependencies, which is obviously not a good idea. Also, here are some more things I hold for obvious, to bar further misunderstandings: 1. Obviously any uninstaller would explicitly say which packages will be uninstalled. 2. Equally obvious, it should provide an option or maybe even ask, if you want the dependencies uninstalled or not. Plone, as an example, currently consists of some 170 packages. I think the most suprising result for a newbie here is that if you say easy_uninstall Plone and only one of these actually gets uninstalled. I agree the best option is asking. But barring that it's pretty obvious that the default should be either to uninstall all, or warn that there now are 169 orphaned packages and tell you how to uninstall them. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ronald Oussoren wrote: With proposal of tracking which packages are installed as a dependency of other packages and automaticly uninstalling them when the package that depends on them gets uninstalled Allow me to note (again) that the current proposal is ONLY part A (tracking). Part B (automatically uninstalling...) is left completely unspecified, and would depend on what uninstaller tools choose to do with this metadata. I am NOT proposing that the basic uninstall function built in to distutils would automatically remove orphans. Unless you would argue that there are NO reasonable use-cases for detection of orphan dependencies, I would hope you could at least support tracking the data. This does make it harder to uninstall something like TurboGears that installs lots of dependencies, but it should be possible to write a tool that makes that easier. One way to do this in a GUI-ish manner is to have a show dependencies checkbox when you uninstall a package, this would then show all dependencies and allow easy selection of them for deinstallation. This gets you simular functionality as tracking explictly requested installs without needing additional metadata. I don't agree that the functionality is comparable. I haven't yet seen an argument against simply tracking REQUESTED metadata; can you agree that at the very least the ability to _list_ orphaned packages could be useful, apart from any kind of automatic uninstall behavior? Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK0jbx1j/fhc23WEARAvUQAJ9xqAi/684TLDADsx6hYFuX3UBC+wCeJoVp VQw/F11l9nnBeFLCljTxQwg= =8PIr -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On Sun, Oct 11, 2009 at 9:50 PM, Carl Meyer c...@dirtcircle.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ronald Oussoren wrote: With proposal of tracking which packages are installed as a dependency of other packages and automaticly uninstalling them when the package that depends on them gets uninstalled Allow me to note (again) that the current proposal is ONLY part A (tracking). Part B (automatically uninstalling...) is left completely unspecified, and would depend on what uninstaller tools choose to do with this metadata. I am NOT proposing that the basic uninstall function built in to distutils would automatically remove orphans. PEP 376 doesn't want to go that far anyway. The uninstall API is just a reference API that can be used to remove *one* distribution. So if there's a tracking file, it should be the simplest thing possible. The goal of PEP 376 is to provide tools for installers/uninstallers to work with a site-package directory, not a full featured uninstaller. I am starting to think when reading back this whole thread, that there's a missing API: a way to read and write arbitrary files in the egg-info directory of an installed distribution. This would allow third-party tools to have their own extra metadata information regarding the distributions. That is, IIRC, what Jim proposed a while ago when he said that we should have a way to define arbitrary metadata for distributions. Tarek. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tarek Ziadé wrote: I am starting to think when reading back this whole thread, that there's a missing API: a way to read and write arbitrary files in the egg-info directory of an installed distribution. I thought of that as well when writing the demo REQUESTED functionality - - I was duplicating work that was already done for RECORD and INSTALLER. pkg_resources has this generic API and it seems to work well. Even if PEP 376 gains that API, IMO the REQUESTED metadata is useful enough to be standardized, rather than restricted to a particular tool. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK0jom1j/fhc23WEARAnZ2AKCoKM/rmG+CBPlzSa67UW3DYlAAnACdFZ/4 VfMIanS9/PLUUAMH3dZHx9w= =ITZH -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
2009/10/8 Carl Meyer c...@meyerloewen.net: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Gediminas Paulauskas wrote: Debian's Apt has this capability, see https://wiki.ubuntu.com/PackageDependencyManagement . It keeps a separate file to track the manually installed packages, and the flag is named Auto-Installed. REQUIRED is an ambiguous name -- is it required by other packages, or wanted by user for any reason (and that is the case)? I agree that the name is not ideal. Perhaps naming the file AUTO_INSTALLED, and flipping the semantics (so the presence of the file allows it to be considered orphaned) would be better. For backwards compatibility already installed packages have to be treated as manually installed, because it is not known why they were installed. So the presence of a new file or flag has to signify that it was AUTO_INSTALLED, i.e. the other way around than originally suggested. Gediminas ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
Ian Bicking wrote: I can imagine adding a little information, basically a log of when and why and who installed the package. For instance: agent: pip 0.5 install-date: 2009-10-08T13:44:00UTC installed-for-user: False installed-for-package: OtherPackage==0.3 I think this is a great idea. Recording more information rather than less and making sure it's kept up to date is a good idea here. Otherwise, when tools come along later down the line that want to make use of this information, they'll have to hack something themselves, and that's what got setuptools into the mess it currently is in the first place! Chris ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Gediminas Paulauskas wrote: For backwards compatibility already installed packages have to be treated as manually installed, because it is not known why they were installed. So the presence of a new file or flag has to signify that it was AUTO_INSTALLED, i.e. the other way around than originally suggested. I don't believe this is actually an issue. The PEP 376 installation format is markedly different from any existing installation format, so any tool dealing with installed packages will already have to know whether it's dealing with a PEP 376 install or a legacy install. Only PEP 376 installs could be considered orphaned. I don't have a problem with AUTO_INSTALLED, but more people seem to have favored REQUESTED thus far. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzzcYFRcxmeyPUXIRApWuAKCHDSfd0rKwNWeZkQPy71wyz/fJFwCdFUw7 XPWhAA4c0r+SFMzuFmz2rKo= =tAZN -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Toshio Kuratomi wrote: I would say REQUESTED due to my arguments for not recording installed-as-package-dependency. REQUESTED is fine, but I don't understand how the arguments apply, given that I'm not proposing to record information like _which_ package it was a dependency of. The same single bit (literally) of information is tracked either way, it's just a question of whether the presence or absence of a file signifies that bit. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzzeSFRcxmeyPUXIRAo0iAJ9QdQzxVUhTBO+rEGgCKmFg9zH7EwCcCfPc lWgoZVc+Z+H2OcIw1V/wNLI= =k12Y -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris Withers wrote: Ian Bicking wrote: I can imagine adding a little information, basically a log of when and why and who installed the package. For instance: agent: pip 0.5 install-date: 2009-10-08T13:44:00UTC installed-for-user: False installed-for-package: OtherPackage==0.3 I think this is a great idea. Recording more information rather than less and making sure it's kept up to date is a good idea here. Otherwise, when tools come along later down the line that want to make use of this information, they'll have to hack something themselves, and that's what got setuptools into the mess it currently is in the first place! The downside here is that it introduces one more wrinkle for installers to worry about handling correctly. There are strong use cases for the single bit requested vs auto-installed; nobody's yet presented use cases for the additional log info. The only thing that comes to my mind is UI niceties: being able to tell the user when, why, and by what agent a package was installed. I'm not aware of existing package managers that go that far; doesn't mean it's a bad idea. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzzjZ1j/fhc23WEARAvw+AKDInwCU+Zzr6NIxlSUeOX+KWB+O6wCg2BWU BVwtV1ndSSmJJcJmxKGu9kU= =XQ57 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
Carl Meyer kirjoitti: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Toshio Kuratomi wrote: I would say REQUESTED due to my arguments for not recording installed-as-package-dependency. REQUESTED is fine, but I don't understand how the arguments apply, given that I'm not proposing to record information like _which_ package it was a dependency of. The same single bit (literally) of information is tracked either way, it's just a question of whether the presence or absence of a file signifies that bit. And what if the package is a dependency for multiple packages? Let's say we have packages A and B which are installed separately, in that order, and both depend on C. C gets installed with information that it was required by A. Now if A is uninstalled, won't C also get uninstalled? Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzzeSFRcxmeyPUXIRAo0iAJ9QdQzxVUhTBO+rEGgCKmFg9zH7EwCcCfPc lWgoZVc+Z+H2OcIw1V/wNLI= =k12Y -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alex wrote: REQUESTED is fine, but I don't understand how the arguments apply, given that I'm not proposing to record information like _which_ package it was a dependency of. The same single bit (literally) of information is tracked either way, it's just a question of whether the presence or absence of a file signifies that bit. And what if the package is a dependency for multiple packages? Let's say we have packages A and B which are installed separately, in that order, and both depend on C. C gets installed with information that it was required by A. Now if A is uninstalled, won't C also get uninstalled? No. We are NOT talking about recording the full dependency graph in package metadata. It is, as has always been the case, up to an uninstall tool to calculate the depgraph based on actual installed packages at runtime. A package is orphaned iff it was not REQUESTED by the user AND it is no longer depended on by other installed packages. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzz531j/fhc23WEARAlPfAJ93YHsbpzp5kF7hR0f98aSMOae6MwCgrP52 RZs5FU3h0U0eUBN2vgqm2HA= =jyi8 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On 01:45 pm, c...@dirtcircle.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alex wrote: REQUESTED is fine, but I don't understand how the arguments apply, given that I'm not proposing to record information like _which_ package it was a dependency of. The same single bit (literally) of information is tracked either way, it's just a question of whether the presence or absence of a file signifies that bit. And what if the package is a dependency for multiple packages? Let's say we have packages A and B which are installed separately, in that order, and both depend on C. C gets installed with information that it was required by A. Now if A is uninstalled, won't C also get uninstalled? No. We are NOT talking about recording the full dependency graph in package metadata. It is, as has always been the case, up to an uninstall tool to calculate the depgraph based on actual installed packages at runtime. A package is orphaned iff it was not REQUESTED by the user AND it is no longer depended on by other installed packages. By doing this, I think you're dooming any Python package uninstaller to be unpleasantly slow. Jean-Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 exar...@twistedmatrix.com wrote: By doing this, I think you're dooming any Python package uninstaller to be unpleasantly slow. The process of searching for orphaned packages may be relatively slow on a system with many installed packages. I'm not sure this is a serious problem. I don't assume that an uninstaller implementation would (or should) automatically search for orphaned packages any time it takes any action. I also don't assume that tools could not maintain their own persistent caches of depgraph information to speed up certain operations. I do think there is some sense in leaving these choices to tool creators and keeping the metadata standard simple. If you'd like to propose that package installers should update the metadata of every dependency of an installed or removed package, so that each package always contains a full and reliable record of what other packages on the system depend on it, that would be interesting. But it is not this proposal. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKz0T51j/fhc23WEARAvyHAKCncp3D1BMBzDXKAmV8SIQqAdqkAgCgvIYT 3Du1wO9sftj5nAKHEgpnf3I= =gjq3 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
Carl Meyer wrote: The downside here is that it introduces one more wrinkle for installers to worry about handling correctly. How so? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris Withers wrote: Carl Meyer wrote: The downside here is that it introduces one more wrinkle for installers to worry about handling correctly. How so? Write some more metadata, figure out whether to write it to already-installed packages, etc. Not saying it's a hard problem, just one more thing to do. I already said I'm not opposed to it in theory, but (as I also said) haven't seen strong use cases for it. Mostly I don't want the (slightly) better to be the enemy of the good; the REQUESTED bit is the critical part we don't have now. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKz0pl1j/fhc23WEARAoEXAKDUsgEMOILvr3dvFMdc6PiN6MNqGgCbBvyb ttaA1Z3zbj7nIsC8FqOFf6c= =4ynG -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On Fri, Oct 09, 2009 at 09:21:29AM -0400, Carl Meyer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris Withers wrote: The downside here is that it introduces one more wrinkle for installers to worry about handling correctly. There are strong use cases for the single bit requested vs auto-installed; nobody's yet presented use cases for the additional log info. The only thing that comes to my mind is UI niceties: being able to tell the user when, why, and by what agent a package was installed. I'm not aware of existing package managers that go that far; doesn't mean it's a bad idea. rpm (one of the Linux package managers) tracks when a package was installed and when it was built. -Toshio pgppga6qfCHKg.pgp Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On Thu, Oct 8, 2009 at 11:39 AM, Carl Meyer c...@meyerloewen.net wrote: I propose adding a metadata file REQUIRED within the .egg-info directory. The presence of this file indicates that the user specifically required this distribution. The absence of the file indicates that the distribution was installed as a dependency. The contents of the file are not used. I can imagine adding a little information, basically a log of when and why and who installed the package. For instance: agent: pip 0.5 install-date: 2009-10-08T13:44:00UTC installed-for-user: False installed-for-package: OtherPackage==0.3 Potentially a package could have multiple records, because multiple installers may in some sense touch a package (e.g., if you install YetAnotherPackage that requires the same library as OtherPackage). You could use INI-style and maybe label each record with the date, like [2009-10-08T13:44:00UTC]. This information seems fairly easy to generate. Updating it after installation would be nice, but also means already-installed packages can be written to, which is not as nice IMHO. Being unable to write to this file should be a non-fatal error for an installer. Either way, a package could become REQUIRED (or user-requested) at any time after it is installed. E.g.: easy_install Markdown easy_install ElementTree # which is required by MarkDown Now ElementTree should not be considered orphaned if MarkDown is removed. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
2009/10/8 Carl Meyer c...@meyerloewen.net: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hey all, I propose adding a bit to the PEP 376 metadata that indicates whether a package was installed by user request or as a dependency of another package. This would allow (un)installer tools to intelligently remove orphaned dependencies, if they so choose. There might be questions about the details of such an uninstaller feature, but I'm not intending to discuss that here. The metadata itself is simple enough to track, and seems like it ought to be included for its future usefulness. I propose adding a metadata file REQUIRED within the .egg-info directory. The presence of this file indicates that the user specifically required this distribution. The absence of the file indicates that the distribution was installed as a dependency. The contents of the file are not used. Debian's Apt has this capability, see https://wiki.ubuntu.com/PackageDependencyManagement . It keeps a separate file to track the manually installed packages, and the flag is named Auto-Installed. REQUIRED is an ambiguous name -- is it required by other packages, or wanted by user for any reason (and that is the case)? For the API, I propose adding a required property to the Distribution class, which would be True or False based on the presence or absence of the REQUIRED file. I've added a demo implementation to a fork of Tarek's pep376 repo on bitbucket: http://bitbucket.org/carljm/pep376/changeset/0c8002e65cb7/ Thoughts? Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On Thu, Oct 08, 2009 at 12:39:33PM -0400, Carl Meyer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hey all, I propose adding a bit to the PEP 376 metadata that indicates whether a package was installed by user request or as a dependency of another package. This would allow (un)installer tools to intelligently remove orphaned dependencies, if they so choose. There might be questions about the details of such an uninstaller feature, but I'm not intending to discuss that here. The metadata itself is simple enough to track, and seems like it ought to be included for its future usefulness. I propose adding a metadata file REQUIRED within the .egg-info directory. The presence of this file indicates that the user specifically required this distribution. The absence of the file indicates that the distribution was installed as a dependency. The contents of the file are not used. For the API, I propose adding a required property to the Distribution class, which would be True or False based on the presence or absence of the REQUIRED file. I've added a demo implementation to a fork of Tarek's pep376 repo on bitbucket: http://bitbucket.org/carljm/pep376/changeset/0c8002e65cb7/ Thoughts? Note that Linux distributions have discussed this for ages and it's not always as useful as a naive first thought would imply. For instance, there are often many scripts written by a system administrator (or a user) that might need to have a module installed. This is not to say that it's a bad idea to record this information -- there could be some installers for specific use cases might find it useful or that it could be useful with confirmation by the user. Also note that a package manager should be able to tell required status from what is currently installed. So it might make more semantic sense to record what was requested by the user to be installed instead of what was required by a package. (When something is both required by a package and requested by a user, the user request is what takes precedence.) -Toshio pgp4FvE57IMvz.pgp Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ian Bicking wrote: I can imagine adding a little information, basically a log of when and why and who installed the package. For instance: agent: pip 0.5 install-date: 2009-10-08T13:44:00UTC installed-for-user: False installed-for-package: OtherPackage==0.3 Potentially a package could have multiple records, because multiple installers may in some sense touch a package (e.g., if you install YetAnotherPackage that requires the same library as OtherPackage). You could use INI-style and maybe label each record with the date, like [2009-10-08T13:44:00UTC]. Sure; I considered this possibility too, but wanted to keep the initial proposal to the simplest thing that could possibly work. I have no problem with a log-style approach that includes more information, and if it makes sense to others I can work on the implementation. This information seems fairly easy to generate. Updating it after installation would be nice, but also means already-installed packages can be written to, which is not as nice IMHO. Being unable to write to this file should be a non-fatal error for an installer. Agreed. In my simple proposal, the only time an already-installed package would need to be touched again is if you're explicitly-installing an already-installed dependency (as in your example below). In that case the REQUIRED file would be added, but in that case touching the installation should be both possible and uncontroversial, since you're specifically asking again to install it. Either way, a package could become REQUIRED (or user-requested) at any time after it is installed. E.g.: easy_install Markdown easy_install ElementTree # which is required by MarkDown Now ElementTree should not be considered orphaned if MarkDown is removed. Exactly. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD4DBQFKzj57FRcxmeyPUXIRAvtfAJ4wu1d+30FF1nmSu23qKGshLphPYACXfwqs 04ISfNfoulbSf0HGDZ3f6g== =vQ0O -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Gediminas Paulauskas wrote: Debian's Apt has this capability, see https://wiki.ubuntu.com/PackageDependencyManagement . It keeps a separate file to track the manually installed packages, and the flag is named Auto-Installed. REQUIRED is an ambiguous name -- is it required by other packages, or wanted by user for any reason (and that is the case)? I agree that the name is not ideal. Perhaps naming the file AUTO_INSTALLED, and flipping the semantics (so the presence of the file allows it to be considered orphaned) would be better. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzj7nFRcxmeyPUXIRAg+UAJkB51TXE/gKEOcsIhC7ID6/EhZbZACfcNS2 ZT7zcYCyy6VLy2qyxr6o+Ys= =q2mH -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Toshio Kuratomi wrote: Note that Linux distributions have discussed this for ages and it's not always as useful as a naive first thought would imply. For instance, there are often many scripts written by a system administrator (or a user) that might need to have a module installed. This is not to say that it's a bad idea to record this information -- there could be some installers for specific use cases might find it useful or that it could be useful with confirmation by the user. Sure, I'm aware of these issues, and didn't want to get too deep into implementation choices of (un)install tools. But it seems clear that under some circumstances it could be useful to have a record of which packages were installed intentionally. Also note that a package manager should be able to tell required status from what is currently installed. So it might make more semantic sense to record what was requested by the user to be installed instead of what was required by a package. (When something is both required by a package and requested by a user, the user request is what takes precedence.) Clearly my terminology choice was poor. REQUIRED in my proposal meant requested by name by a user (which has to be recorded at install time), not required as a dependency by other installed packages (which, as you say, can be calculated at runtime). Would REQUESTED, or AUTO_INSTALLED (with the sense flipped) be better options? Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzkB4FRcxmeyPUXIRArxxAJ9B2YcLng0H0fSbTtLztKc5ILmvWACfcF0y qztiKsrXS9usj14Iz37OE1g= =AlFA -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
On Thu, Oct 08, 2009 at 03:41:44PM -0400, Carl Meyer wrote: Also note that a package manager should be able to tell required status from what is currently installed. So it might make more semantic sense to record what was requested by the user to be installed instead of what was required by a package. (When something is both required by a package and requested by a user, the user request is what takes precedence.) Clearly my terminology choice was poor. REQUIRED in my proposal meant requested by name by a user (which has to be recorded at install time), not required as a dependency by other installed packages (which, as you say, can be calculated at runtime). Would REQUESTED, or AUTO_INSTALLED (with the sense flipped) be better options? I would say REQUESTED due to my arguments for not recording installed-as-package-dependency. -Toshio pgpcAsUr0vtmp.pgp Description: PGP signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata
Ian Bicking wrote: I can imagine adding a little information, basically a log of when and why and who installed the package. For instance: agent: pip 0.5 install-date: 2009-10-08T13:44:00UTC installed-for-user: False installed-for-package: OtherPackage==0.3 Potentially a package could have multiple records, because multiple installers may in some sense touch a package (e.g., if you install YetAnotherPackage that requires the same library as OtherPackage). You could use INI-style and maybe label each record with the date, like [2009-10-08T13:44:00UTC]. Carl: Sure; I considered this possibility too, but wanted to keep the initial proposal to the simplest thing that could possibly work. I have no problem with a log-style approach that includes more information, and if it makes sense to others I can work on the implementation. I'd go for the simplest approach at first. On Thu, Oct 8, 2009 at 9:52 PM, Toshio Kuratomi a.bad...@gmail.com wrote: I would say REQUESTED due to my arguments for not recording installed-as-package-dependency. +1 Tarek ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
