[Distutils] [issue142] easy_install ignores 'headers' passed to setup()
New submission from Tres Seaver tsea...@palladion.com: A bare distutils install of a package with 'headers' passed to 'setup()' dispatches to 'install_headers', which copies the exported header files into the system include path. 'easy_install' bypasses the distutils 'install()' when doing a normal egg install, and does not re-implement the 'install_headers' dance. Furthermore, the 'bdist_egg' it creates, even though it contains the header files, has no metadata which would allow the installer to discover that they should be copied. -- messages: 669 nosy: tseaver priority: bug status: unread title: easy_install ignores 'headers' passed to setup() ___ Setuptools tracker setupto...@bugs.python.org http://bugs.python.org/setuptools/issue142 ___ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [issue142] easy_install ignores 'headers' passed to setup()
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/09/2012 11:13 AM, Tres Seaver wrote: New submission from Tres Seaver tsea...@palladion.com: A bare distutils install of a package with 'headers' passed to 'setup()' dispatches to 'install_headers', which copies the exported header files into the system include path. 'easy_install' bypasses the distutils 'install()' when doing a normal egg install, and does not re-implement the 'install_headers' dance. Furthermore, the 'bdist_egg' it creates, even though it contains the header files, has no metadata which would allow the installer to discover that they should be copied. Note that distribute has the same bug, which I reported here: https://bitbucket.org/tarek/distribute/issue/295 - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/7AqQACgkQ+gerLs4ltQ4/kwCfWEBYSY+dlEfD0b/QNxXnPNxA lPcAoIj0XsQCu2AbPuVWYLEGmWpfKL4Y =VNY3 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding Provides-Extra as an edit to PEP 345
On Fri, Jul 6, 2012 at 11:20 AM, Daniel Holth dho...@gmail.com wrote: It is exactly the same as setuptools' extras feature, and I didn't quote the entire edit on-list. Also, there is a reference implementation at https://bitbucket.org/dholth/distribute/ in the DistInfoDistribution class. The 'test' and 'doc' reserved names are not an essential part of the Provides-Extra feature. Provides-Extra: is primarily for optional run-time (not build-time) dependencies. Though in the interest of not having too many different ways to do a similar thing, could this not also be used for tests/docs? Perhaps even Provides-Extra: tests, and Provides-Extra: docs should be reserved names as you suggested. I envision something like: Provides-Extra: tests Requires-Dist: nose; extra == 'tests' Provides-Extra: docs Requires-Dist: sphinx; extra == 'docs' $ pysetup test Tests require the 'nose' package. Please run `pysetup install mypackage[tests]` to install the test dependencies. Ditto for building the docs. In a sense, being able to run the tests, or build the docs could be considered 'extra' features. One could even have: Provides-Extra: setup Requires-Dist: mysetuphooks; extra == 'setup' for build-time dependencies. Although in that case, pysetup (or any other installer) would explicitly have to check for this in the metadata before doing anything else. So maybe this example stretches this syntax too far? Plus build-time dependencies are not exactly extras. It's very much required to do anything with the package. Erik ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding Provides-Extra as an edit to PEP 345
Though in the interest of not having too many different ways to do a similar thing, could this not also be used for tests/docs? Perhaps even Provides-Extra: tests, and Provides-Extra: docs should be reserved names as you suggested. I envision something like: ... Provides-Extra: setup Requires-Dist: mysetuphooks; extra == 'setup' for build-time dependencies. Although in that case, pysetup (or any other installer) would explicitly have to check for this in the metadata before doing anything else. So maybe this example stretches this syntax too far? Plus build-time dependencies are not exactly extras. It's very much required to do anything with the package. Pre-built or binary packages won't want to install the build-time dependencies. The check for a particular Provides-Extra is trivial, just a list membership test. I expect to see a lot of bugs in packages that require some of their build-time dependencies at run time. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding Provides-Extra as an edit to PEP 345
On Mon, Jul 9, 2012 at 1:27 PM, Daniel Holth dho...@gmail.com wrote: Though in the interest of not having too many different ways to do a similar thing, could this not also be used for tests/docs? Perhaps even Provides-Extra: tests, and Provides-Extra: docs should be reserved names as you suggested. I envision something like: ... Provides-Extra: setup Requires-Dist: mysetuphooks; extra == 'setup' for build-time dependencies. Although in that case, pysetup (or any other installer) would explicitly have to check for this in the metadata before doing anything else. So maybe this example stretches this syntax too far? Plus build-time dependencies are not exactly extras. It's very much required to do anything with the package. Pre-built or binary packages won't want to install the build-time dependencies. The check for a particular Provides-Extra is trivial, just a list membership test. I expect to see a lot of bugs in packages that require some of their build-time dependencies at run time. Maybe, but wouldn't that be the developer's problem? setup_requires has the same issue--it does not install the setup requirements into site-packages. Instead it just does an egg install into the source directory and activates it on the path. Outside of future setup.py runs, the setup_requires distributions are not available (you would have to add them to install_requires too if you need them at runtime). Under a scheme like this one would have to list that dependency under Requires-Dist twice: with and without the 'extra' marker. I might still prefer extending the metadata format to add a Setup-Requires-Dist or the like. Erik ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding Provides-Extra as an edit to PEP 345
I expect to see a lot of bugs in packages that require some of their build-time dependencies at run time. Maybe, but wouldn't that be the developer's problem? setup_requires has the same issue--it does not install the setup requirements into site-packages. Instead it just does an egg install into the source directory and activates it on the path. Outside of future setup.py runs, the setup_requires distributions are not available (you would have to add them to install_requires too if you need them at runtime). OK, it sounds like this bug has been mostly anticipated and taken care of already. Under a scheme like this one would have to list that dependency under Requires-Dist twice: with and without the 'extra' marker. I might still prefer extending the metadata format to add a Setup-Requires-Dist or the like. I don't follow. Does anyone build/install packages without also first installing the run-time requirements? Surely Setup-Requires-Dist would have exactly the same problem. It seems like two ways to say exactly the same thing. Let's just try to focus on getting Metadata 1.2 out the door and implemented before its 8th birthday, preferably during Julython... The only thing I need is to be able to represent the wildly popular extras feature in METADATA to benefit from PEP 345 and 376 without rewriting thousands of Python packages. That is all. Save the debate about reserved extra names for setup / build / test / doc dependencies for Metadata 1.3. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding Provides-Extra as an edit to PEP 345
On Mon, Jul 9, 2012 at 4:37 PM, Daniel Holth dho...@gmail.com wrote: I expect to see a lot of bugs in packages that require some of their build-time dependencies at run time. Maybe, but wouldn't that be the developer's problem? setup_requires has the same issue--it does not install the setup requirements into site-packages. Instead it just does an egg install into the source directory and activates it on the path. Outside of future setup.py runs, the setup_requires distributions are not available (you would have to add them to install_requires too if you need them at runtime). OK, it sounds like this bug has been mostly anticipated and taken care of already. Under a scheme like this one would have to list that dependency under Requires-Dist twice: with and without the 'extra' marker. I might still prefer extending the metadata format to add a Setup-Requires-Dist or the like. I don't follow. Does anyone build/install packages without also first installing the run-time requirements? Surely Setup-Requires-Dist would have exactly the same problem. It seems like two ways to say exactly the same thing. The difference is that setup/build dependencies generally should be downloaded, extracted, and added to the path before another commands are run, because the packaging framework allows for the behavior of built-in commands themselves to be modified, as well as certain metadata. The usefulness of this has been discussed in past threads. See http://mail.python.org/pipermail/distutils-sig/2012-May/018627.html and the thread Eric Araujo pointed out at: https://groups.google.com/forum/?fromgroups#!searchin/the-fellowship-of-the-packaging/setup-requires/the-fellowship-of-the-packaging/KVsDQawfEno/M-ODKx9Ne5YJ Let's just try to focus on getting Metadata 1.2 out the door and implemented before its 8th birthday, preferably during Julython... The only thing I need is to be able to represent the wildly popular extras feature in METADATA to benefit from PEP 345 and 376 without rewriting thousands of Python packages. That is all. Save the debate about reserved extra names for setup / build / test / doc dependencies for Metadata 1.3. I don't have data on the popularity of setup_requires--is there some easy way to find that out without downloading an entire PyPI mirror? That said, I'm anticipating that it would be very useful for packaging. As it is, setup.py files tend to contain a lot of repetitive boilerplate, especially for non-pure-Python packages. With the elimination of setup.py, more packages are going to require setup hook-script modules to be shipped with their code, and I could see some of these being highly reusable. I'm certain this is a useful feature--just not so certain whether it's desirable needed to modify the Metadata spec to account for it. In the worst case, I could imagine writing some kind of bootstrap module that can be dropped into a project and used via a setup-hook that is then possible of downloading other build-time dependencies. But that's exactly the sort of thing I'm trying to get away from--if I have to maintain a bootstrap script (or another module containing setup-hooks, custom commands, etc.) I have to update every project that uses it every time those scripts are updated. Erik ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding Provides-Extra as an edit to PEP 345
Under a scheme like this one would have to list that dependency under Requires-Dist twice: with and without the 'extra' marker. I might still prefer extending the metadata format to add a Setup-Requires-Dist or the like. The difference is that setup/build dependencies generally should be downloaded, extracted, and added to the path before another commands are run, because the packaging framework allows for the behavior of built-in commands themselves to be modified, as well as certain metadata. The usefulness of this has been discussed in past threads. Setup requirements are certainly necessary. I see what you mean now. 'Setup-Requires-Dist' is a different feature than 'Provides-Extra', since with 'Provides-Extra' there isn't an elegant way to figure out what just the setup requirements are. Provides-Extra is only useful if you want to install the mandatory runtime requirements at the same time (which is exactly what you want to do for 'extras'). You could make the argument that you only need setup_requires_dist in setup.cfg... since it will generate METADATA the first time (when downloading from revision control) anyway. But it can also go in METADATA, you will get no complaint from me. Of course I did download PyPI, that's the way to tell. Just under 80% of the newest sdists for each package on PyPI contain PKG-INFO. How many phases does a build + install have? Is this about right? 1. Install setup requirements Setup-Requires-Dist from .dist-info/METADATA 2. Run setup dist_info (pip always runs egg_info) to get a .dist-info directory with METADATA inside 3. Read up-to-date runtime requirements Requires-Dist 4. Install those 5. Install package setup install ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding Provides-Extra as an edit to PEP 345
New draft up at https://bitbucket.org/dholth/python-peps/changeset/537e83bd4068 Adds Provides-Extra: (with longer explanation), Setup-Requires-Dist: (full text below) Setup-Requires-Dist lives because it makes sense to list packages only needed for setup.py to run before install is possible. The reserved extra names test and doc live. No one should mind installing a distribution and its dependencies in order to run the unit tests or generate the documentation. Setup-Requires-Dist (multiple use) :: Like Requires-Dist, but names dependencies needed while the distributions's distutils / packaging `setup.py` / `setup.cfg` is run. Commonly used to generate a manifest from version control. Examples:: Setup-Requires-Dist: custom_setup_command Dependencies mentioned in `Setup-Requires-Dist` may be installed exclusively for setup and are not guaranteed to be available at run time. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
