[Distutils] [patch] executable Scripts on windows
Hello, I would like to discuss a patch that adds executable .bat files for scripts installed with distutils. As a windows user with several Python versions installed I often create shortcuts for Python applications that were installed using distutils. But I also have to navigate to Scripts/ directory and mess with script files. Common problems: 1. python script doesn't have .py extension 2. script is actually a shell script 3. script is executed with wrong Python version The first problem is clear. Files without extension are not executable on windows. If distutils makes scripts executable on posix, why shouldn't it do the same on windows? The second problem should be addressed to package maintainer in the first place. The third one is also quite common - distutils fixes scripts and inserts correct Python path in shebang even on windows platform, but it makes little sense, because the script is executed with the only interpreter that was associated with .py extensions. To solve problems 1 and 3 I added patch that adds .bat file to execute the script with correct python version. As a positive side effect - the application can be executed through Run menu. The patch is posted into http://bugs.python.org/issue4015 and I would like to know your opinion about it. -- --anatoly t. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Symlinks vs API -- question for developers
So I have a question for all the developers on this list. Philip thinks that using symlinks will drive adoption better than an API to access package data. I think an API will have better adoption than a symlink hack. But the real question is what do people who maintain packages think? Since Philip's given his reasoning, here's mine: 1) Philip says that with symlinks distributions will likely have to submit patches to the build scripts to tag various files as belonging to certain categories. If you, as an upstream are going to accept a patch to your build scripts to place files in a different place wouldn't you also accept a patch to your source code to use a well defined API to pull files from a different source? This is a distribution's bread and butter and if there's a small, useful, well-liked, standard API for accessing data files you will start receiving patches from distributions that want to help you help them. 2) Symlinks cannot be used universally. Although it might not be common to want an FHS style install in such an environment, it isn't unheard of. At one time in the distant past I had to use cygwin so I know that while this may be a corner case, it does exist. 3) The primary argument for symlinks is that symlinks are compatible with __file__. But this compatibility comes at a cost -- symlinks can't do anything extra. In a different subthread Philip argues that setuptools provides more than distutils and that's why people switch and that the next generation tool needs to provide even more than setuptools. Symlinks cannot do that. 4) In contrast an API can do more: It can deal with writable files. On Unix, persistent, per user storage would go in the user's home directory, on other OS's it would go somewhere else. This is abstractable using an API at runtime but not using symlinks at install time. 5) cross package data. Using __file__ to detect file location is inherently not suitable for crossing package boundaries. Egg Translations would not be able to use a symlink based backend to do its work for this reason. 6) zipped eggs. These require an API. So moving to symlinks is actually a regression. 7) Philip says that the reason pkg_resources does not see widespread adoption is that the developer cost of using an API is too high compared to __file__. I don't believe that the difference between file and API is that great. An example of using an API could be something like this: Symlinks:: import os icondirectory = os.path.join(os.path.basename(__file__), 'icons') API:: import pkgdata icondirectory = pkgdata.resource(pkg='setuptools', \ category='icon', resource='setuptools.png') Instead I think the data handling portion of pkg_resources is not more widely adopted for these reasons: * pkg_resources's package handling is painful for the not-infrequent corner cases. So people who have encountered the problems with require() not overriding a default or not selecting the proper version when multiple packages specify overlapping version ranges already have a negative impression of the library before they even get to the data handling portion. * pkg_resources does too much: loading libraries by version really has nothing to do with loading data for use by a library. This is a drawback because people think of and promote pkg_resources as a way to enable easy_install rather than a way to enable abstraction of data location. * The only benefit (at least, being promoted in the documentation) is to allow zipped eggs to work. Distributions have no reason to create zipped eggs so they have no reason to submit patches to upstream to support the pkg_resources api. * Distributions, further, don't want to install all-in-one egg directories on the system. The pkg_resources API just gets in the way of doing things correctly in a distribution. I've had to patch code to not use pkg_resources if data is installed in the FHS mandated areas. Far from encouraging distributions to send patches upstream to make modules use pkg_resources this makes distributions actively discourage upstreams from using it. * The API isn't flexible enough. EggTranslations places its data within the metadata store of eggs instead of within the data store. This is because the metadata is able to be read outside of the package in which it is included while the package data can only be accessed from within the package. 8) To a distribution, symlinks are just a hack. We use them for things like php web apps when the web application is hardcoded to accept only one path for things (like the writable state files being intermixed with the program code). Managing a symlink farm is not something distributions are going to get excited over so adoption by distributions that this is the way to work with files won't happen until upstreams move on their own. Further, since the install tool is being proposed as a separate project from the metadata to mark files, the expectation is that the
Re: [Distutils] Symlinks vs API -- question for developers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Oct 17, 2008, at 1:32 PM, Toshio Kuratomi wrote: 7) Philip says that the reason pkg_resources does not see widespread adoption is that the developer cost of using an API is too high compared to __file__. I don't believe that the difference between file and API is that great. An example of using an API could be something like this: Symlinks:: import os icondirectory = os.path.join(os.path.basename(__file__), 'icons') s/basename/dirname/ I think. API:: import pkgdata icondirectory = pkgdata.resource(pkg='setuptools', \ category='icon', resource='setuptools.png') Having tried to be religious about using pkg_resources instead of __file__ in all my new code, I tend to agree that the API cost is not that high. I don't particularly like the verbosity of the names chosen, but I actually like not having to use the __file__ idiom. * Distributions, further, don't want to install all-in-one egg directories on the system. The pkg_resources API just gets in the way of doing things correctly in a distribution. I've had to patch code to not use pkg_resources if data is installed in the FHS mandated areas. Far from encouraging distributions to send patches upstream to make modules use pkg_resources this makes distributions actively discourage upstreams from using it. I hadn't thought of this, but yes, this is a serious negative. So once again, I think this boils down to these questions: if we have a small library whose sole purpose is to abstract a data store so you can find out where a particular non-code file lives on this system will you use it? I would. I apologize for not having followed the discussion that closely, but as an application developer, I would really like an API that hides all the location nonsense from me. As familiar as __file__ is, it's a fragile hack. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSPjP8nEjvBPtnXfVAQKOeQP/eJywpz1CUxJhUD9NhUj68rpoHbato8W4 fP2ZNRmKOSGUmtaj9hM1vduMoCszCN/vz8fX+gGZFu9ySkWyQfO5Q6Hh/kBrKSRN IzVYcd3lbV+e63+twk3Ht4gSX8j2iWnt375976kFgvmMc2iB7zn0r/TDblMIqvxV NUUi3d3zaPs= =5yrx -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Symlinks vs API -- question for developers
Toshio Kuratomi wrote: So I have a question for all the developers on this list. Philip thinks that using symlinks will drive adoption better than an API to access package data. I think an API will have better adoption than a symlink hack. But the real question is what do people who maintain packages think? Since Philip's given his reasoning, here's mine: 1) Philip says that with symlinks distributions will likely have to submit patches to the build scripts to tag various files as belonging to certain categories. If you, as an upstream are going to accept a patch to your build scripts to place files in a different place wouldn't you also accept a patch to your source code to use a well defined API to pull files from a different source? This is a distribution's bread and butter and if there's a small, useful, well-liked, standard API for accessing data files you will start receiving patches from distributions that want to help you help them. Annotating my files is extremely unlike to break code, so I am more likely to accept a patch that does that. 2) Symlinks cannot be used universally. Although it might not be common to want an FHS style install in such an environment, it isn't unheard of. At one time in the distant past I had to use cygwin so I know that while this may be a corner case, it does exist. 3) The primary argument for symlinks is that symlinks are compatible with __file__. But this compatibility comes at a cost -- symlinks can't do anything extra. In a different subthread Philip argues that setuptools provides more than distutils and that's why people switch and that the next generation tool needs to provide even more than setuptools. Symlinks cannot do that. As a library writer I have no motivation to do any of this. New features do drive adoption more quickly than simple cleanup, but only features that would help me as a developer in some way (including making it easier to support users). A new API wouldn't help me, and might hurt as it means more conventions to communicate to other developers. Also I'd have to debug problems with the resource loading, which be nothing but frustration. I hate platform issues, and moving files around just means there's more platform issues I'd be exposed to. Nothing platform-specific is of any interest to me as a developer -- unfortunately such problems come up often, but I don't want to go looking for new platform issues. 4) In contrast an API can do more: It can deal with writable files. On Unix, persistent, per user storage would go in the user's home directory, on other OS's it would go somewhere else. This is abstractable using an API at runtime but not using symlinks at install time. Writable stuff is quite different, IMHO. An API for writable files might be useful, but there's no current conventions around it, and I would expect that API to be entirely different from a resource API. 5) cross package data. Using __file__ to detect file location is inherently not suitable for crossing package boundaries. Egg Translations would not be able to use a symlink based backend to do its work for this reason. You'll need to explain further, as I am unclear of the problem with __file__ in this context. For instance, couldn't you symlink somepackage/translations/ to /usr/share/lang/somepackage ? 6) zipped eggs. These require an API. So moving to symlinks is actually a regression. True. 7) Philip says that the reason pkg_resources does not see widespread adoption is that the developer cost of using an API is too high compared to __file__. I don't believe that the difference between file and API is that great. An example of using an API could be something like this: Symlinks:: import os icondirectory = os.path.join(os.path.basename(__file__), 'icons') API:: import pkgdata icondirectory = pkgdata.resource(pkg='setuptools', \ category='icon', resource='setuptools.png') Instead I think the data handling portion of pkg_resources is not more widely adopted for these reasons: Just personally, it's entirely laziness on my part; I can't remember the signatures for the resource stuff, so I write what I most immediately remember. I think the Distro/package ambiguity also confuses me. * pkg_resources's package handling is painful for the not-infrequent corner cases. So people who have encountered the problems with require() not overriding a default or not selecting the proper version when multiple packages specify overlapping version ranges already have a negative impression of the library before they even get to the data handling portion. * pkg_resources does too much: loading libraries by version really has nothing to do with loading data for use by a library. This is a drawback because people think of and promote pkg_resources as a way to enable easy_install rather than a way to enable abstraction of data location. * The only benefit (at least, being promoted in the
Re: [Distutils] Symlinks vs API -- question for developers
2008/10/17 Barry Warsaw [EMAIL PROTECTED]: So once again, I think this boils down to these questions: if we have a small library whose sole purpose is to abstract a data store so you can find out where a particular non-code file lives on this system will you use it? I would. I apologize for not having followed the discussion that closely, but as an application developer, I would really like an API that hides all the location nonsense from me. As familiar as __file__ is, it's a fragile hack. I'd like an API, as well. It's probably the only truly cross-platform approach. Having said that, a key question is, what precisely is needed here? Python 2.6 has pkgutil.get_data, which abstracts the idea of grabbing the content of a file. There's not much else that can realistically be supported by fully general PEP 302 style loaders, so you have to start either (1) requiring additional functionality from loaders, or (2) restricting usage to filesystems, and losing the whole concept of a loader protocol which PEP 302 provided. (And that way lies incompatibilities with py2exe, which is very common on Windows, and zipped eggs, as well as possibly other more obscure cases). I'd fully support the development of an API for data access, but I'd suggest that it should take the form of a PEP extending PEP 302, plus an implementation in core Python (pkgutil is the obvious place), rather than being restricted purely to an external module like setuptools. Of course, having an external implementation for use in older versions of Python which don't have the API in core, would be fine, but the aim should be the core. (Otherwise, it's adding a dependency to projects that otherwise don't need it). Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Symlinks vs API -- question for developers
Toshio Kuratomi wrote: So once again, I think this boils down to these questions: if we have a small library whose sole purpose is to abstract a data store so you can find out where a particular non-code file lives on this system will you use it? As part of the stdlib? Yes, I'd use it. -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
