On 10 May 2016 at 07:08, Chris Barker wrote:
> But I started this whole line of conversation because it seemed that there
> was desire for:
>
> Ability to isolate the build environment.
> Ability to better handle/manage non-python dependencies
I don't care about the first
On Sun, May 8, 2016 at 5:31 AM, Nick Coghlan wrote:
> > any python developer is going to
> > run into these issues eventually...
>
> Aye, I know - conda was one of the systems I evaluated as a possible
> general purpose tool for user level package management in Fedora and
>
On 8 May 2016 at 04:15, Chris Barker wrote:
> On Sat, May 7, 2016 at 6:51 AM, Nick Coghlan wrote:
>>
>> On 7 May 2016 01:55, "Chris Barker" wrote:
>> > So my point is about scope-creep -- if you want the PyPa tools to solve
>> >
On Sat, May 7, 2016 at 6:17 AM, Greg Ewing
wrote:
> Do you expect that
>
>> projects ... should (somehow) contain simplified instructions on how to
>> build the various C/Fortran extensions supplied in the bundle as
>> source code?
>>
>
> Essentially, yes. I'm not
On Sat, May 7, 2016 at 6:51 AM, Nick Coghlan wrote:
> On 7 May 2016 01:55, "Chris Barker" wrote:
> > So my point is about scope-creep -- if you want the PyPa tools to solve
> all these problems, then you are re-inventing conda -- better to simply
>
On 7 May 2016 at 14:17, Greg Ewing wrote:
> I don't know if there are conventions for such things on
> Windows. I suspect not, in which case manual input is
> going to be needed one way or another.
There aren't. You typically need to specify the exact locations of
On 7 May 2016 01:55, "Chris Barker" wrote:
> So my point is about scope-creep -- if you want the PyPa tools to solve
all these problems, then you are re-inventing conda -- better to simply
adopt conda (or fork it and fix issues that I'm sure are there)
conda doesn't
Paul Moore wrote:
Do you expect that
projects ... should (somehow) contain simplified instructions on how to
build the various C/Fortran extensions supplied in the bundle as
source code?
Essentially, yes. I'm not sure how achievable it would
be, but ideally that's what I'd like.
would the
tl;dr version
I think you're right that terminology can be confusing. I think the
definitions people typically work to are:
1. The "packaging" or "release" process - the process (run on a
developer's machine) of creating files that get published for users to
download and install.
2. The
Chris Barker wrote:
But I think there is consensus here that build systems need to be
customisable -- which means arbitrary code may have to be run.
I think different people are using the word "build" in
different ways here.
To my mind, "building" is what the developer of a package
does, and
On Fri, May 6, 2016 at 9:39 AM, Donald Stufft wrote:
> On May 6, 2016, at 11:54 AM, Chris Barker wrote:
>
> So my point is about scope-creep -- if you want the PyPa tools to solve
> all these problems, then you are re-inventing conda -- better to simply
On 05/06/2016 09:48 AM, Leonardo Rochael Almeida wrote:
On 6 May 2016 at 13:15, Chris Barker wrote:
"python literals" is perfectly well defined -- both by the language
reference, and by "can be parsed by ast.literal_eval" and it addresses
>> the limitations of JSON and is fully declarative.
On 6 May 2016 at 13:15, Chris Barker wrote:
> On Fri, May 6, 2016 at 5:41 AM, Nick Coghlan wrote:
>
>> [...]
>
>
>
>> So rather than saying "the bootstrapping dependency declaration file
>> is Python-but-not-really", it's easier to say "it's an
> On May 6, 2016, at 11:54 AM, Chris Barker wrote:
>
> So my point is about scope-creep -- if you want the PyPa tools to solve all
> these problems, then you are re-inventing conda -- better to simply adopt
> conda (or fork it and fix issues that I'm sure are there….)
On Fri, May 6, 2016 at 5:41 AM, Nick Coghlan wrote:
> The "Python-with-imports" case is the status quo with setup.py, and we
> already know that's a pain because you need to set up an environment
> that already has the right dependencies installed to enable your
> module
On Fri, May 6, 2016 at 5:55 AM, Nick Coghlan wrote:
> > And maybe it's good to keep "new style" configuration clearly separate.
>
> Part of my motivation for suggesting re-using setup.cfg is the
> proliferation of packaging related config sprawl in project root
> directories
On Thu, May 5, 2016 at 6:37 PM, Robert Collins
wrote:
>
> Thats good. It occurs to me that scientific builds may be univerally
> broken because folk want to avoid easy-install and the high cost of
> double builds of things. E.g. adding bootstrap_requires will let folk
On Thu, May 5, 2016 at 4:32 PM, Nathaniel Smith wrote:
> > You do know that we're on our way to re-writing conda, now, don't you?
> :-)
> >
> > I think we need to be careful of scope-creep...
>
> conda did not invent the idea of creating separate python environments
> for
On Thu, May 5, 2016 at 10:45 PM, Greg Ewing
wrote:
> Even if python is available, you might not want to run
> arbitrary code just to install a package.
>
> If a config file can contain executable code, then it
> can contain bugs. Debugging is something the developer
On 6 May 2016 at 06:41, Chris Barker wrote:
> On Wed, May 4, 2016 at 8:09 PM, Nick Coghlan wrote:
>> 3. The ongoing popularity of setup.cfg shows that while ini-style may
>> not be perfect for this use case, it clearly makes it over the
>> threshold of
On 6 May 2016 at 06:30, Chris Barker wrote:
> On Wed, May 4, 2016 at 7:45 PM, Nick Coghlan wrote:
>> Usually that enforcement is
>> handled by making the configuration declarative - it's in some passive
>> format like an ini file or JSON, and if it gets
Chris Barker wrote:
This is python packaging, I'm still trying to see why you'd need to read
the config without python available.
Even if python is available, you might not want to run
arbitrary code just to install a package.
If a config file can contain executable code, then it
can contain
>From my point of view mandatory build isolation will make building thinks
slow and bad, besides being totally unrelated to the idea of a working
bootstrap requirements feature.
On Thu, May 5, 2016 at 9:37 PM Robert Collins
wrote:
> On 5 May 2016 at 21:47, Nathaniel
On 5 May 2016 at 21:47, Nathaniel Smith wrote:
> On Wed, May 4, 2016 at 11:57 PM, Robert Collins
...
>>> I don't think I've ever seen a package that had accurate
>>> setup_requires (outside the trivial case of packages where
>>> setup_requires=[] is accurate). Scientific packages
Clearly it should spin up a Gentoo VM from scratch each time. No half
measures.
On Thu, May 5, 2016, 19:32 Nathaniel Smith wrote:
> On Thu, May 5, 2016 at 3:18 PM, Chris Barker
> wrote:
> > On Thu, May 5, 2016 at 2:10 AM, Nathaniel Smith
On Thu, May 5, 2016 at 3:18 PM, Chris Barker wrote:
> On Thu, May 5, 2016 at 2:10 AM, Nathaniel Smith wrote:
>>
>> ...The main thing I want to point out though, is that all of these
>> problems you're raising are complications caused entirely by wanting
>>
On Thu, May 5, 2016 at 3:21 PM, Greg Ewing
wrote:
> Chris Barker wrote:
>
> OK -- that's more or less my thought -- if it's python that gets run,
>> then you've got your config generator built in -- why not?
>>
>
> The difference is that the config generator only
Last post! sorry to have not kept up last night
> to call the new feature setup_requires but some prefer to eliminate
> > uncertainty by calling it bootstrap_requires.
>
> The main advantage of a new feature name is that when someone searches
> the internet for "python bootstrap_requires",
On Thu, May 5, 2016 at 2:47 AM, Nathaniel Smith wrote:
> > When you introduce isolation, the build will only have the standard
> > library + whatever is declared as a dep: and pyqt5 has no source on
> > PyPI.
>
so build isolation isolates you from arbitrary system libs? now you
Chris Barker wrote:
OK -- that's more or less my thought -- if it's python that gets run,
then you've got your config generator built in -- why not?
The difference is that the config generator only gets run
once, when the config info needs to get produced. If the
config file itself is
On Thu, May 5, 2016 at 2:10 AM, Nathaniel Smith wrote:
> ...The main thing I want to point out though, is that all of these
> problems you're raising are complications caused entirely by wanting
> to avoid build isolation in the name of simplicity. If each package
> gets its own
On Wed, May 4, 2016 at 11:57 PM, Robert Collins
wrote:
>
> No. Old pip new package will break, new pip old package is entirely safe
> AFAICT.
That's the goal, yes?
So I think we need to get less obsessed with backward compatibility:
pip will retain (for along time)
OK, so which setup() arguments do we want to leave out of the static
metadata?
05.05.2016, 23:53, Daniel Holth kirjoitti:
This is a recurring point of confusion. setup.py is not ever going
away. In general it is necessary for you to be able to write software
to build your software, and there
C extensions, py-modules, ...
On Thu, May 5, 2016, 17:05 Alex Grönholm wrote:
> OK, so which setup() arguments do we want to leave out of the static
> metadata?
>
>
> 05.05.2016, 23:53, Daniel Holth kirjoitti:
>
> This is a recurring point of confusion. setup.py is not
We've spent a huge amount of effort on reaching the point where pretty
> much everything *can* be made pip installable. Heck, *PyQt5*, which is
> my personal benchmark for a probably-totally-unpackageable package,
> announced last week that they now have binary wheels on pypi for all
> of
This is a recurring point of confusion. setup.py is not ever going away. In
general it is necessary for you to be able to write software to build your
software, and there is no intention to take that feature away.
People repeatedly come to the conclusion that static metadata means the
entire
On Wed, May 4, 2016 at 8:09 PM, Nick Coghlan wrote:
> I know I'm one of the folks that has historically been dubious of the
> "just use setup.cfg" idea, due to the assorted problems with the
> ini-style format not extending particularly well to tree-structured
> data (beyond
I think it would be best to gather a few extreme examples of setup.py
files from real world projects and figure out if they can be implemented
in a declarative fashion. That at least would help us identify the pain
points.
For starters, gevent's setup.py looks like it needs a fair bit of
On Wed, May 4, 2016 at 7:45 PM, Nick Coghlan wrote:
> This configuration vs customisation distinction is probably worth
> spelling out for folks without a formal software engineering or
> computer science background, so:
>
fair enough -- good to be clear on the terms.
>
On Wed, May 4, 2016 at 3:28 PM, Paul Moore wrote:
> On 4 May 2016 at 23:11, Chris Barker wrote:
> > so it could be purely declarative, but users could also put code in
> there to
> > customize the configuration on the fly, too.
>
> That basically
On 5 May 2016 at 13:36, Daniel Holth wrote:
> Here's the kind of thing that you should expect. Someone will write
>
> setup.cfg:
>
> [bootstrap_requires]
> pbr
>
> pip installs pbr in a directory that is added to PYTHONPATH for that build.
Ah, so we don't install into the
On 5 May 2016 at 22:36, Daniel Holth wrote:
> Pedantic note
>
> setup_requires is a setuptools parameter used to install packages after
> setup() is called. Even though very many people expect or want those
> packages to be installed before setup.py executes. I think it is
Here's the kind of thing that you should expect. Someone will write
setup.cfg:
[bootstrap_requires]
pbr
pip installs pbr in a directory that is added to PYTHONPATH for that build.
The package builds.
And there was much rejoicing.
The big gain from this simple feature is that people will be
On 5 May 2016 at 10:10, Nathaniel Smith wrote:
> ...The main thing I want to point out though, is that all of these
> problems you're raising are complications caused entirely by wanting
> to avoid build isolation in the name of simplicity. If each package
> gets its own isolated
On 5 May 2016 at 19:47, Nathaniel Smith wrote:
> The reason I'm being so intense about this is that AFAICT these are all true:
>
> Premise 1: Without build isolation enabled by default, then in
> practice everyone will putter along putting up with broken builds all
> the time.
On 5 May 2016 at 14:22, Nathaniel Smith wrote:
> On Wed, May 4, 2016 at 6:28 AM, Nick Coghlan wrote:
>> On 4 May 2016 at 23:00, Daniel Holth wrote:
>>> +1 It would be great to start with a real setup_requires and probably would
>>> not
On Wed, May 4, 2016 at 11:57 PM, Robert Collins
wrote:
> On 5 May 2016 at 18:32, Nathaniel Smith wrote:
>> On Wed, May 4, 2016 at 10:42 PM, Robert Collins
>>...
>>> Yes, things will break: anyone using this will need a new pip, by
>>> definition. Not
On Thu, May 5, 2016 at 2:02 AM, Paul Moore wrote:
> On 5 May 2016 at 07:57, Robert Collins wrote:
>> We've a history in this group of biting off too much and things not
>> getting executed. We're *still* in the final phases of deploying
>> PEP-508,
On 5 May 2016 at 07:57, Robert Collins wrote:
> We've a history in this group of biting off too much and things not
> getting executed. We're *still* in the final phases of deploying
> PEP-508, and it was conceptually trivial. I'm not arguing that we
> shouldn't make
On 5 May 2016 at 00:58, Ethan Furman wrote:
> Somebody will have to distill that PEP, I have only an small inkling of what
> it's trying to say.
The relevant point for this discussion is "you can request that
particular packages are installed before the build step for your
On 5 May 2016 at 18:32, Nathaniel Smith wrote:
> On Wed, May 4, 2016 at 10:42 PM, Robert Collins
>...
>> Yes, things will break: anyone using this will need a new pip, by
>> definition. Not everyone will be willing to wait 10 years before using
>> it :).
>
> Just to clarify (since
On Wed, May 4, 2016 at 10:42 PM, Robert Collins
wrote:
> On 5 May 2016 at 16:22, Nathaniel Smith wrote:
> ...
>> I'm sympathetic to the general approach, but on net I think I prefer a
>> slightly different proposal.
>>
>> Downsides to just standardizing
On 5 May 2016 at 16:22, Nathaniel Smith wrote:
...
> I'm sympathetic to the general approach, but on net I think I prefer a
> slightly different proposal.
>
> Downsides to just standardizing [setup_requires]:
If I write a PEP, it won't be standardising setup_requires, it will be
On Wed, May 4, 2016 at 6:28 AM, Nick Coghlan wrote:
> On 4 May 2016 at 23:00, Daniel Holth wrote:
>> +1 It would be great to start with a real setup_requires and probably would
>> not interfere with later build system abstractions at all.
>
> If we're going
Ok so, if i draft a pep for said proposal, will it die under the weight of
a thousand bike sheds?
On 5 May 2016 3:09 PM, "Nick Coghlan" wrote:
> On 5 May 2016 at 06:28, Robert Collins wrote:
> > the only reason I got involved in build system
On 5 May 2016 at 06:28, Robert Collins wrote:
> the only reason I got involved in build system discussions was
> pushback 18months or so back when I implemented a proof of concept for
> pip that just used setup.cfg. I'd be very happy to ignore all the
> build system
On 5 May 2016 at 08:28, Paul Moore wrote:
> On 4 May 2016 at 23:11, Chris Barker wrote:
>> so it could be purely declarative, but users could also put code in there to
>> customize the configuration on the fly, too.
>
> That basically repeats the
The only part that needs to be static is the metadata. The actual build can
have code without hindering our dependency resolution aspirations.
On Wed, May 4, 2016, 19:59 Ethan Furman wrote:
> On 05/04/2016 04:29 PM, Alex Grönholm wrote:
>
> > Different files for what?
On 05/04/2016 04:29 PM, Alex Grönholm wrote:
Different files for what? Something not covered by PEP 508?
Somebody will have to distill that PEP, I have only an small inkling of
what it's trying to say.
As for my specific use case: I have Python3-only files in my
distribution, so they
Different files for what? Something not covered by PEP 508?
05.05.2016, 02:23, Ethan Furman kirjoitti:
On 05/04/2016 03:28 PM, Paul Moore wrote:
On 4 May 2016 at 23:11, Chris Barker wrote:
That basically repeats the mistake that was made with setup.py. We
explicitly don't want an
On 05/04/2016 03:28 PM, Paul Moore wrote:
On 4 May 2016 at 23:11, Chris Barker wrote:
That basically repeats the mistake that was made with setup.py. We
explicitly don't want an executable format for specifying build
configuration.
Executable code or not, we need to be able to specify
On 4 May 2016 at 23:11, Chris Barker wrote:
> so it could be purely declarative, but users could also put code in there to
> customize the configuration on the fly, too.
That basically repeats the mistake that was made with setup.py. We
explicitly don't want an executable
On Wed, May 4, 2016 at 3:33 AM, Donald Stufft wrote:
> I'd actually prefer not using JSON for something that is human
> editable/writable because I think it's a pretty poor format for that case.
> It
> is overly restrictive in what it allows (for instance, no trailing comma
>
On 4 May 2016 at 22:33, Donald Stufft wrote:
>
..> I also believe that we can't provide a replacement for setup.py
without either
> purposely declaring we no longer support something that people used from it or
> providing a way to support that in the new, setup.py-less format.
Just call it Steve
On Wed, May 4, 2016, 16:25 Robert Collins wrote:
> On 4 May 2016 at 19:39, Nick Coghlan wrote:
> > On 4 May 2016 at 16:03, Robert Collins
> wrote:
> >> The edits I'd expect to make if the conclusions
On 4 May 2016 at 19:39, Nick Coghlan wrote:
> On 4 May 2016 at 16:03, Robert Collins wrote:
>> The edits I'd expect to make if the conclusions I suggested in
>> https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html
>> are adopted
Agree.
On Wed, May 4, 2016, 09:28 Nick Coghlan wrote:
> On 4 May 2016 at 23:00, Daniel Holth wrote:
> > +1 It would be great to start with a real setup_requires and probably
> would
> > not interfere with later build system abstractions at all.
>
> If
On 4 May 2016 at 23:00, Daniel Holth wrote:
> +1 It would be great to start with a real setup_requires and probably would
> not interfere with later build system abstractions at all.
If we're going to go down that path, perhaps it might make sense to
just define a standard
+1 It would be great to start with a real setup_requires and probably would
not interfere with later build system abstractions at all.
On Wed, May 4, 2016 at 6:33 AM Donald Stufft wrote:
>
> > On May 4, 2016, at 3:39 AM, Nick Coghlan wrote:
> >
> > On 4
> On May 4, 2016, at 3:39 AM, Nick Coghlan wrote:
>
> On 4 May 2016 at 16:03, Robert Collins wrote:
>> The edits I'd expect to make if the conclusions I suggested in
>> https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html
>> are
On 4 May 2016 at 16:03, Robert Collins wrote:
> The edits I'd expect to make if the conclusions I suggested in
> https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html
> are adopted are:
>
> - change to a Python API
> - BFDL call on the file format and
On 4 May 2016 at 05:10, Paul Moore wrote:
> Nick - do you have the time to pick this up? Or does it need someone
> to step up as BDFL-delegate? Robert, Nathaniel, do you have time to
> spend on a final round of discussion on this, on the assumption that
> the goal will be a
On 4 May 2016 at 04:28, Nathaniel Smith wrote:
> On Tue, May 3, 2016 at 10:10 AM, Paul Moore wrote:
>> On 3 May 2016 at 17:47, Donald Stufft wrote:
>>> It will likely get decided as part of the build system PEP, whenever that
>>> gets
I certainly have no problem with Daniel's suggestion (and it would be
much better than my solution) but would involve yet more standards work.
Who's going to do that and when?
03.05.2016, 22:39, Donald Stufft kirjoitti:
On May 3, 2016, at 3:35 PM, Daniel Holth wrote:
Who
> On May 3, 2016, at 3:35 PM, Daniel Holth wrote:
>
> Who cares exactly why it doesn't work? We know how to fix it by doing
> something different (put build dependencies in a static file and have them
> installed by pip before running setup.py).
>
Presumably Alex would
Who cares exactly why it doesn't work? We know how to fix it by doing
something different (put build dependencies in a static file and have them
installed by pip before running setup.py).
On Tue, May 3, 2016 at 3:33 PM Donald Stufft wrote:
>
> On May 3, 2016, at 3:28 PM, Alex
> On May 3, 2016, at 3:28 PM, Alex Grönholm wrote:
>
> As I pointed out in my previous post, "extended commands" are not the only
> use case for setup_requires -- upgrading setuptools itself would enable
> support for PEP 508 style conditional requirements. This
As I pointed out in my previous post, "extended commands" are not the
only use case for setup_requires -- upgrading setuptools itself would
enable support for PEP 508 style conditional requirements. This
currently does not work because if you have such requirements in your
setup(), setuptools
On 3 May 2016 at 19:28, Nathaniel Smith wrote:
> No, no, Nick's not the blocker. I'm the blocker! (Sorry)
>
> Donald + Robert + I had a longish conversation about this on IRC a
> month ago [1]. I volunteered to summarize back to the mailing list,
> and then I flaked -- so I guess
Thanks for the update! Glad this is still moving forward. I'll continue to
prod the list if things stall again as I want to respond to "Python
packaging is broken" with "actually your knowledge is just outdated, go
read packaging.python.org". :)
On Tue, 3 May 2016 at 11:28 Nathaniel Smith
What happened is that only a half-dozen setuptools experts (I am not one of
those six people) know how to write an extended command or whatever that
would actually be able to take advantage of setup requirements as
implemented by setuptools. Everyone else wants to "import x" at the top of
setup.py
No, setuptools parses the install requirements before acting on setup
requirements. That is the source of the problem. If setuptools only
parsed and acted on setup requirements before even parsing install
requirements, this wouldn't be an issue.
03.05.2016, 21:26, Leonardo Rochael Almeida
On Tue, May 3, 2016 at 10:10 AM, Paul Moore wrote:
> On 3 May 2016 at 17:47, Donald Stufft wrote:
>> It will likely get decided as part of the build system PEP, whenever that
>> gets picked up again.
>
> Yes, but on 15th March
>
We did separate build from install. Now we just want to be able to build
without [having to emulate] distutils; just having some dependencies
installed before setup.py runs would also be a great boon.
I'm reading part of this conversation as "a simple bdist_wheel bug is a
reason to do a lot of
Having setuptools process the setup requirements before parsing install
requirements would be a good step forward. Had that been done before, we
could've just added a setup requirement for a newer setuptools to enable
PEP 508 conditional requirements.
03.05.2016, 21:04, Daniel Holth
On Tue, May 3, 2016 at 12:10 PM, Paul Moore wrote:
> On 3 May 2016 at 17:47, Donald Stufft wrote:
>> It will likely get decided as part of the build system PEP, whenever that
>> gets picked up again.
>
> Yes, but on 15th March
>
On 3 May 2016 at 17:47, Donald Stufft wrote:
> It will likely get decided as part of the build system PEP, whenever that
> gets picked up again.
Yes, but on 15th March
(https://mail.python.org/pipermail/distutils-sig/2016-March/028457.html)
Robert posted
> Just to set
On Wed, 27 Apr 2016 at 10:53 Donald Stufft wrote:
> This isn't really a problem with what you're doing. Rather it's an issue
> with the toolchain and and open question whether or not wheels should
> conceptually be able to be produced from a checkout, or if they should only
>
88 matches
Mail list logo