Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-02 Thread Nathaniel Smith
[Adding distutils-sig to the CC as a heads-up. The context is that numpy is looking at deprecating the use of 'python setup.py install' and enforcing the use of 'pip install .' instead, and running into some issues that will probably need to be addressed if 'pip install .' is going to become the st

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-02 Thread Robert Collins
On 3 November 2015 at 14:57, Nathaniel Smith wrote: > [Adding distutils-sig to the CC as a heads-up. The context is that > numpy is looking at deprecating the use of 'python setup.py install' > and enforcing the use of 'pip install .' instead, and running into > some issues that will probably need

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-02 Thread Nathaniel Smith
On Nov 2, 2015 6:51 PM, "Robert Collins" wrote: > > On 3 November 2015 at 14:57, Nathaniel Smith wrote: > > [Adding distutils-sig to the CC as a heads-up. The context is that > > numpy is looking at deprecating the use of 'python setup.py install' > > and enforcing the use of 'pip install .' inst

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-02 Thread Robert Collins
On 3 November 2015 at 16:02, Nathaniel Smith wrote: > On Nov 2, 2015 6:51 PM, "Robert Collins" wrote: ... >> > Ugh. If 'pip (install/wheel) .' is supposed to become the standard way >> > to build things, then it should probably build in-place by default. >> > Working in a temp dir makes perfect s

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-02 Thread Robert Collins
BTW scipy list is rejecting all my emails (vs eg moderating), so I'm going to drop the cc in all future replies. -Rob On 3 November 2015 at 16:05, Robert Collins wrote: > On 3 November 2015 at 16:02, Nathaniel Smith wrote: >> On Nov 2, 2015 6:51 PM, "Robert Collins" wrote: > ... >>> > Ugh. If

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-03 Thread Chris Barker - NOAA Federal
>> I'm not talking about in place installs, I'm talking about e.g. building a >> wheel and then tweaking one file and rebuilding -- traditionally build >> systems go to some effort to keep track of intermediate artifacts and reuse >> them across builds when possible, but if you always copy the sour

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-05 Thread Ralf Gommers
On Tue, Nov 3, 2015 at 6:10 PM, Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: > >> I'm not talking about in place installs, I'm talking about e.g. > building a > >> wheel and then tweaking one file and rebuilding -- traditionally build > >> systems go to some effort to keep track of

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-05 Thread Donald Stufft
I’m not at my computer, but does ``pip install —no-clean —build `` make this work?  On November 5, 2015 at 5:25:16 PM, Ralf Gommers (ralf.gomm...@gmail.com) wrote: > On Tue, Nov 3, 2015 at 6:10 PM, Chris Barker - NOAA Federal < > chris.bar...@noaa.gov> wrote: > > > >> I'm not talking about in pl

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-05 Thread Ralf Gommers
On Thu, Nov 5, 2015 at 11:29 PM, Donald Stufft wrote: > I’m not at my computer, but does ``pip install —no-clean —build build dir>`` make this work? > No, that option seems to not work at all. I tried with both a relative and an absolute path to --build. In the specified dir there are subdirs c

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-05 Thread Ralf Gommers
On Thu, Nov 5, 2015 at 11:44 PM, Ralf Gommers wrote: > > > On Thu, Nov 5, 2015 at 11:29 PM, Donald Stufft wrote: > >> I’m not at my computer, but does ``pip install —no-clean —build > build dir>`` make this work? >> > > No, that option seems to not work at all. I tried with both a relative and >

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-05 Thread Donald Stufft
If ``pip install —build … —no-clean …`` worked to do incremental builds, would that satisfy this use case? (without the —upgrade and —no-deps, —no-deps is only needed because —upgrade and —upgrade is needed because of another ticket that I think will get fixed at some point). On November 5, 201

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-05 Thread Ralf Gommers
On Fri, Nov 6, 2015 at 12:37 AM, Donald Stufft wrote: > If ``pip install —build … —no-clean …`` worked to do incremental builds, > would that satisfy this use case? (without the —upgrade and —no-deps, > —no-deps is only needed because —upgrade and —upgrade is needed because of > another ticket th

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-06 Thread Paul Moore
On 6 November 2015 at 07:39, Ralf Gommers wrote: > On Fri, Nov 6, 2015 at 12:37 AM, Donald Stufft wrote: >> >> If ``pip install —build … —no-clean …`` worked to do incremental builds, >> would that satisfy this use case? (without the —upgrade and —no-deps, >> —no-deps is only needed because —upgr

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-06 Thread Nathaniel Smith
On Mon, Nov 2, 2015 at 5:57 PM, Nathaniel Smith wrote: > On Sun, Nov 1, 2015 at 3:16 PM, Ralf Gommers wrote: >> 2. ``pip install .`` silences build output, which may make sense for some >> usecases, but for numpy it just sits there for minutes with no output after >> printing "Running setup.py in

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-06 Thread Chris Barker - NOAA Federal
> While I understand what you're trying to achieve (and I'm in favour, > in general) it should be remembered that pip's core goal is installing > packages - not being a component of a development workflow. Yes -- clear separation of concerns here! So what IS supposed to be used in the development

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Paul Moore
On 7 November 2015 at 01:26, Chris Barker - NOAA Federal wrote: > So what IS supposed to be used in the development workflow? The new > mythical build system? Fair question. Unfortunately, the answer is honestly that there's no simple answer - pip is not a bad option, but it's not its core use ca

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Ralf Gommers
On Sat, Nov 7, 2015 at 2:02 PM, Paul Moore wrote: > On 7 November 2015 at 01:26, Chris Barker - NOAA Federal > wrote: > > So what IS supposed to be used in the development workflow? The new > > mythical build system? > I'd like to point out again that this is not just about development workflow

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Donald Stufft
On November 7, 2015 at 8:56:02 AM, Ralf Gommers (ralf.gomm...@gmail.com) wrote: > On Sat, Nov 7, 2015 at 2:02 PM, Paul Moore wrote: > > > On 7 November 2015 at 01:26, Chris Barker - NOAA Federal > > wrote: > > > So what IS supposed to be used in the development workflow? The new > > > mythical bu

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Paul Moore
On 7 November 2015 at 13:55, Ralf Gommers wrote: > On Sat, Nov 7, 2015 at 2:02 PM, Paul Moore wrote: >> >> On 7 November 2015 at 01:26, Chris Barker - NOAA Federal >> wrote: >> > So what IS supposed to be used in the development workflow? The new >> > mythical build system? > > I'd like to point

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Ralf Gommers
On Sat, Nov 7, 2015 at 3:57 PM, Paul Moore wrote: > On 7 November 2015 at 13:55, Ralf Gommers wrote: > > On Sat, Nov 7, 2015 at 2:02 PM, Paul Moore wrote: > >> > >> On 7 November 2015 at 01:26, Chris Barker - NOAA Federal > >> wrote: > >> > So what IS supposed to be used in the development wor

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Paul Moore
On 7 November 2015 at 16:33, Ralf Gommers wrote: > I had an example above of installing into different venvs. Full rebuilds for > that each time are very expensive. Why doesn't wheel caching solve this problem? That's what it's *for*, surely? Paul ___

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Donald Stufft
On November 7, 2015 at 5:45:24 PM, Paul Moore (p.f.mo...@gmail.com) wrote: > On 7 November 2015 at 16:33, Ralf Gommers wrote: > > I had an example above of installing into different venvs. Full rebuilds for > > that each time are very expensive. > > Why doesn't wheel caching solve this problem? T

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Donald Stufft
On November 7, 2015 at 5:46:53 PM, Paul Moore (p.f.mo...@gmail.com) wrote: > On 7 November 2015 at 22:46, Donald Stufft wrote: > > On November 7, 2015 at 5:45:24 PM, Paul Moore (p.f.mo...@gmail.com) wrote: > >> On 7 November 2015 at 16:33, Ralf Gommers wrote: > >> > I had an example above of instal

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Paul Moore
On 7 November 2015 at 22:46, Donald Stufft wrote: > On November 7, 2015 at 5:45:24 PM, Paul Moore (p.f.mo...@gmail.com) wrote: >> On 7 November 2015 at 16:33, Ralf Gommers wrote: >> > I had an example above of installing into different venvs. Full rebuilds >> > for >> > that each time are very ex

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Paul Moore
On 7 November 2015 at 22:47, Donald Stufft wrote: > Only if we think we can trust the version numbers to be unique from random > paths on the file system. Precisely. And that's the sort of trade-off that Ralf was asking to be clarified. Here, the trade off is that if we *are* allowed to rely on

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Nathaniel Smith
On Sat, Nov 7, 2015 at 2:44 PM, Paul Moore wrote: > On 7 November 2015 at 16:33, Ralf Gommers wrote: >> I had an example above of installing into different venvs. Full rebuilds for >> that each time are very expensive. > > Why doesn't wheel caching solve this problem? That's what it's *for*, sure

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Donald Stufft
On November 7, 2015 at 6:07:47 PM, Paul Moore (p.f.mo...@gmail.com) wrote: > On 7 November 2015 at 22:47, Donald Stufft wrote: > > Only if we think we can trust the version numbers to be unique from random > > paths on the > file system. > > Precisely. And that's the sort of trade-off that Ral

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Nathaniel Smith
On Sat, Nov 7, 2015 at 6:57 AM, Paul Moore wrote: > 2. (For here) Builds are not isolated from what's in the development > directory. So if you have your sdist definition wrong, what you build > locally may work, but when you release it it may fail. Obviously that > can be fixed by proper developm

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Donald Stufft
On November 7, 2015 at 6:16:34 PM, Donald Stufft (don...@stufft.io) wrote: > > I want to reduce the “paths” that an installation can go down.  I decided I’d make a little visual aid to help explain what I mean here (omitting development/editable installs because they are weird and will always be

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Nathaniel Smith
On Sat, Nov 7, 2015 at 3:16 PM, Donald Stufft wrote: [...] > The outcome of all of these types of installs are subtly different and we’ve > had a number of users regularly get confused when they act differently over > the years. I do not think it’s possible to make (5) act like anything else >

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Donald Stufft
On November 7, 2015 at 6:43:50 PM, Nathaniel Smith (n...@pobox.com) wrote: > On Sat, Nov 7, 2015 at 3:16 PM, Donald Stufft wrote: > [...] > > The outcome of all of these types of installs are subtly different and > > we’ve had a number > of users regularly get confused when they act differently

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Paul Moore
On 7 November 2015 at 16:33, Ralf Gommers wrote: > Your only concrete argument for it so far is aimed at developers I feel that there's some confusion over the classes of people involved here ("developers", "users", etc). For me, the core user base for pip is people who use "pip install" to inst

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Nathaniel Smith
On Sat, Nov 7, 2015 at 4:03 PM, Paul Moore wrote: > I see building a wheel as a release activity. As such, it should > produce a reproducible result, and so should not be affected by > arbitrary state in the development directory. I don't know whether you > consider "ensuring the wheels aren't wro

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-07 Thread Nathaniel Smith
On Sat, Nov 7, 2015 at 4:02 PM, Donald Stufft wrote: > On November 7, 2015 at 6:43:50 PM, Nathaniel Smith (n...@pobox.com) wrote: >> On Sat, Nov 7, 2015 at 3:16 PM, Donald Stufft wrote: >> [...] >> > The outcome of all of these types of installs are subtly different and >> > we’ve had a number >>

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Ralf Gommers
On Sun, Nov 8, 2015 at 1:03 AM, Paul Moore wrote: > On 7 November 2015 at 16:33, Ralf Gommers wrote: > > Your only concrete argument for it so far is aimed at developers > > I feel that there's some confusion over the classes of people involved > here ("developers", "users", etc). > Good point.

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Ralf Gommers
On Sun, Nov 8, 2015 at 12:38 AM, Donald Stufft wrote: > On November 7, 2015 at 6:16:34 PM, Donald Stufft (don...@stufft.io) wrote: > > > I want to reduce the “paths” that an installation can go down. > > I decided I’d make a little visual aid to help explain what I mean here > (omitting developme

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Paul Moore
On 8 November 2015 at 11:13, Ralf Gommers wrote: > You only have two categories? I'm missing at least one important category: > users who install things from a vcs or manually downloaded code (pre-release > that's not on pypi for example). This category is probably a lot larger that > than that of

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Ralf Gommers
On Sun, Nov 8, 2015 at 2:23 PM, Paul Moore wrote: > On 8 November 2015 at 11:13, Ralf Gommers wrote: > > > "wheels and sdists" != "release artifacts" > > Please explain. All you've done here is state that you don't agree > with me, but given no reasons. > Come on, I elaborated in the sentence r

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Paul Moore
On 8 November 2015 at 13:34, Ralf Gommers wrote: > On Sun, Nov 8, 2015 at 2:23 PM, Paul Moore wrote: >> >> On 8 November 2015 at 11:13, Ralf Gommers wrote: >> >> > "wheels and sdists" != "release artifacts" >> >> Please explain. All you've done here is state that you don't agree >> with me, but

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Ralf Gommers
On Sun, Nov 8, 2015 at 2:23 PM, Paul Moore wrote: > On 8 November 2015 at 11:13, Ralf Gommers wrote: > > You only have two categories? I'm missing at least one important > category: > > users who install things from a vcs or manually downloaded code > (pre-release > > that's not on pypi for exam

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Ralf Gommers
On Sun, Nov 8, 2015 at 2:45 PM, Paul Moore wrote: > On 8 November 2015 at 13:34, Ralf Gommers wrote: > > On Sun, Nov 8, 2015 at 2:23 PM, Paul Moore wrote: > >> > >> On 8 November 2015 at 11:13, Ralf Gommers > wrote: > >> > >> > "wheels and sdists" != "release artifacts" > >> > >> Please explai

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Paul Moore
On 8 November 2015 at 13:51, Ralf Gommers wrote: > To give an example for Numpy: > - there are 5-10 active developers with commit rights > - there are 50-100 contributors who submit PRs > - there are O(1000) people who read the mailing list > - there are O(1 million) downloads/installs per

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Nathaniel Smith
On Nov 8, 2015 5:23 AM, "Paul Moore" wrote: > [...] > I find it hard to imagine that there are a significant number of users > who install from development sources but who aren't developers (at > least to the extent that testers of pre-release code are also > developers). I'm not sure exactly wha

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-08 Thread Paul Moore
On 8 November 2015 at 17:42, Nathaniel Smith wrote: > I'm not sure exactly what's at stake in this terminological/ontological > debate, but it certainly is fairly common for developers to have > conversations like "thanks for reporting that issue, I think it's fixed in > master but can't reproduce

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

2015-11-12 Thread Nathaniel Smith
On Sun, Nov 8, 2015 at 12:52 PM, Paul Moore wrote: > On 8 November 2015 at 17:42, Nathaniel Smith wrote: >> I'm not sure exactly what's at stake in this terminological/ontological >> debate, but it certainly is fairly common for developers to have >> conversations like "thanks for reporting that