Re: [Distutils] A setup-requires implementation
The subprocess is certainly more compatible. And pip does not really have an API besides the command line interface. But a lot of packages could probably work with "import pip". (Setup.py is already running in a pip subprocess when invoked by pip.) If you were pressed for time then you should be using wheel. 😺🎡 Might be interesting to add a bit more affordance for installers that want to implement setup requires themselves and skip the bootstrapper. On May 29, 2014 11:55 AM, "Erik Bray" wrote: > > On Mon, May 19, 2014 at 9:24 PM, Daniel Holth wrote: > > Here's a short setup.py replacement that makes setup-requires work: > > https://bitbucket.org/dholth/setup-requires/src/ . I'd appreciate a > > code review. > > > > Use by renaming your own package's setup.py to real-setup.py and > > copying this setup.py in its place. > > > > List only the requirements setup.py itself needs to run in the > > `setup-requires =` key of the `[metadata]` section of setup.cfg, > > one per line:: > > > > [metadata] > > setup-requires = cffi > > pip > > pycparser >= 2.10 > > > > (Only the name and required versions are allowed, not the full pip > > syntax of URLs to specific repositories. Instead, install internal > > setup-requires dependencies manually or set PIP_FIND_LINKS=... to point > > to the necessary repositories.) > > > > When run, setup-requires' setup.py checks that each distribution > > listed in setup-requires is installed; if not, it installs them into > > the ./setup-requires directory in a pip subprocess. Then real-setup.py > > continues to execute with the same arguments. > > > > Why a custom section in setup.cfg? Users are accustomed to editing > > setup.cfg to configure random things like unit tests, bdist_wheel > > etc.; this just adds a field instead of a new file. Unlike a .txt file > > it should be more intuitive that setup.cfg does not support the full > > pip requirements syntax. > > > > Please note that not every package installs correctly with pip -t. > > > > Now let's see some setup.py helper packages. > > Thanks, I like the approach this takes of using pip instead of > easy-install, and caching all the setup_requires packages in the > setup-requires directory without cluttering the source root with eggs. > I'm not crazy about having to launch a separate process to do > this--do you think it can be done without that, by just importing pip > and using its API? Or did you find that to be too problematic? > > I'm also not crazy about putting the real setup.py in a separate file, > but only because, in my case, I think it's likely to confuse some of > my co-developers. That said, I think that this approach could be > adapted to suit my needs. > > Erik > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] A setup-requires implementation
On Mon, May 19, 2014 at 9:24 PM, Daniel Holth wrote: > Here's a short setup.py replacement that makes setup-requires work: > https://bitbucket.org/dholth/setup-requires/src/ . I'd appreciate a > code review. > > Use by renaming your own package's setup.py to real-setup.py and > copying this setup.py in its place. > > List only the requirements setup.py itself needs to run in the > `setup-requires =` key of the `[metadata]` section of setup.cfg, > one per line:: > > [metadata] > setup-requires = cffi > pip > pycparser >= 2.10 > > (Only the name and required versions are allowed, not the full pip > syntax of URLs to specific repositories. Instead, install internal > setup-requires dependencies manually or set PIP_FIND_LINKS=... to point > to the necessary repositories.) > > When run, setup-requires' setup.py checks that each distribution > listed in setup-requires is installed; if not, it installs them into > the ./setup-requires directory in a pip subprocess. Then real-setup.py > continues to execute with the same arguments. > > Why a custom section in setup.cfg? Users are accustomed to editing > setup.cfg to configure random things like unit tests, bdist_wheel > etc.; this just adds a field instead of a new file. Unlike a .txt file > it should be more intuitive that setup.cfg does not support the full > pip requirements syntax. > > Please note that not every package installs correctly with pip -t. > > Now let's see some setup.py helper packages. Thanks, I like the approach this takes of using pip instead of easy-install, and caching all the setup_requires packages in the setup-requires directory without cluttering the source root with eggs. I'm not crazy about having to launch a separate process to do this--do you think it can be done without that, by just importing pip and using its API? Or did you find that to be too problematic? I'm also not crazy about putting the real setup.py in a separate file, but only because, in my case, I think it's likely to confuse some of my co-developers. That said, I think that this approach could be adapted to suit my needs. Erik ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] fix setup_requires by allowing it in setup.cfg or a setup_requires.txt
Did I forget to post this to the list?
https://bitbucket.org/dholth/setup-requires
It's 40 lines of boilerplate that makes setup-requires work. I'd very
much appreciate testing and feedback.
Daniel
#!/usr/bin/env python
# Install dependencies from a "[metadata] setup-requires = ..." section in
# setup.cfg, then run real-setup.py.
# From https://bitbucket.org/dholth/setup-requires
import sys, os, subprocess, codecs, pkg_resources
sys.path[0:0] = ['setup-requires']
pkg_resources.working_set.add_entry('setup-requires')
try:
import configparser
except:
import ConfigParser as configparser
def get_requirements():
if not os.path.exists('setup.cfg'): return
config = configparser.ConfigParser()
config.readfp(codecs.open('setup.cfg', encoding='utf-8'))
setup_requires = config.get('metadata', 'setup-requires')
specifiers = [line.strip() for line in setup_requires.splitlines()]
for specifier in specifiers:
try:
pkg_resources.require(specifier)
except pkg_resources.DistributionNotFound:
yield specifier
try:
to_install = list(get_requirements())
if to_install:
subprocess.call([sys.executable, "-m", "pip", "install",
"-t", "setup-requires"] + to_install)
except (configparser.NoSectionError, configparser.NoOptionError):
pass
# Run real-setup.py
exec(compile(open("real-setup.py").read().replace('\\r\\n', '\\n'),
__file__,
'exec'))
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] fix setup_requires by allowing it in setup.cfg or a setup_requires.txt
On Thu, May 15, 2014 at 6:31 PM, Daniel Holth wrote:
> On Thu, May 15, 2014 at 6:04 PM, Marcus Smith wrote:
>>> I'm of course hoping that someone uses the feature to do a
>>> setup.py-command-line-interface-compatible distutils replacement
>>> (rather than a distutils extension) like what Bento does with its
>>> pip-compatible setup.py replacement.
>>
>>
>> ah, I see. interesting.
>> I admit my initial reaction to "quack like a setup.py" alternative build
>> systems is that it sounds invasive and sneaky, and I want it to be explicit
>> and overt, like it will be in metadata 2.X.
>> but on the other hand, I hear you about wanting to see alternatives have
>> some way to get off the ground now.
>
> There's a bit of a chicken-egg problem obviously.
>
> For the moment I am more interested in just giving people a usable
> setup_requires.
Just now catching up on this, but I'm strongly in favor of fixing this
issue. It's a simple addition to setuptools.
I actually am about to release the first version of a package called
astropy_helpers, which basically is a bundle of all the build tools
used by the Astropy project so that other related projects can take
advantage of them as well. This has the same chicken-egg problem--you
want to be able to use astropy_helpers in setup.py, but you can't get
to it without first calling setup(setup_requires=['astropy_helpers']).
This can actually be gotten around by simply creating a dummy
Distribution object and calling it like:
from setuptools.dist import Distribution
Distribution({'setup_requires': ['astropy_helpers']})
But that's an ugly hack. To that end I added bootstrap script that's
imported at the top of setup.py that knows how to do this (it also has
code for finding astropy_helpers as a git submodule if the project has
one in its repo--useful for development).
d2to1 takes a slightly different approach in that one doesn't pass
anything to the normal setup() except for setup_requires=['d2to1'].
Once d2to1 is bootstrapped it takes over the entire setup process,
including reading additional setup_requires from setup.cfg as Daniel
suggested. I'm actually planning on relaunching d2to1 with a new name
(less tied to the defunct distutils2) because I still think it's a
useful alternative to setup.py, while still working within the
existing framework (and easily transferable to any new framework).
Erik
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
