Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

> The setuptools and pip master branches are for things which *are*
expected to go into the next version, not for experimental changes

I agree with that statement (the PRs as they currently are need work
because they have become stale) in the sense that code in the master branch
should work at all times. But that's different than the PEP being fully
fleshed out for all build systems rather than just pip and setuptools. And
also, a big reason for this is that to be frank, the first implementation
will probably not work. That's not just some special rule that I have
applied to myself, but it's also the reality with PEP 518. The reality is
that PEP 518 does not currently work and it is in the master branch, but
people did not discover that fact until it was in the master branch. You
can't know what you can't know.

If an implementation that does not comply with the PEP yet but works with
appropriate fallbacks to verify setuptools support and compliance, then
it's not the end of the world if the packages are still built and installed
correctly.

2017-08-19 16:48 GMT-04:00 Paul Moore :

> On 19 August 2017 at 21:30, xoviat  wrote:
> > Also, I disagree with this point. I think it's still possible (and in
> fact
> > preferable) to have setuptools and pip 'test this out' with appropriate
> > fallbacks before opening this up to the wider community. Most people
> > wouldn't even notice because they wouldn't be using HEAD.
>
> The same people who would notice it are probably just as able to apply
> a patch/PR or grab a fork. So your PRs are available for people to
> test if they want to, they don't need to be merged for that to happen.
>
> The setuptools and pip master branches are for things which *are*
> expected to go into the next version, not for experimental changes
> (for pip, we should be able to release from master whenever we want
> to, I suspect setuptools has a similar policy).
>
> Paul
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Paul Moore]

On 19 August 2017 at 21:30, xoviat  wrote:
> Also, I disagree with this point. I think it's still possible (and in fact
> preferable) to have setuptools and pip 'test this out' with appropriate
> fallbacks before opening this up to the wider community. Most people
> wouldn't even notice because they wouldn't be using HEAD.

The same people who would notice it are probably just as able to apply
a patch/PR or grab a fork. So your PRs are available for people to
test if they want to, they don't need to be merged for that to happen.

The setuptools and pip master branches are for things which *are*
expected to go into the next version, not for experimental changes
(for pip, we should be able to release from master whenever we want
to, I suspect setuptools has a similar policy).

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

Also, I disagree with this point. I think it's still possible (and in fact
preferable) to have setuptools and pip 'test this out' with appropriate
fallbacks before opening this up to the wider community. Most people
wouldn't even notice because they wouldn't be using HEAD.

2017-08-18 15:28 GMT-04:00 Thomas Kluyver :

> I wouldn't expect established tools like pip & setuptools to accept pull
> requests implementing a PEP which is still under discussion and subject to
> change. They will naturally want a stable, accepted spec before they
> consider merging code for it.
>
> We've probably all wished that the discussion could be brought to a swift
> conclusion. But there are real questions to work out, and people have many
> other things to pay attention to. I'm frustrated by how long it's taking as
> well, but there's no magic button anyone can press to make it go quickly.
>
> Thomas
>
>
>
>
> On Fri, Aug 18, 2017, at 08:17 PM, xoviat wrote:
>
> At this point, I can only offer implementations as I have been attempting
> to do (some of which are outdated because the PEP has changed), but I
> cannot make anyone pay attention to or accept my pull requests and it seems
> I cannot accelerate the discussion here.
>
> 2017-08-18 14:08 GMT-05:00 xoviat :
>
> I have attempted to implement prototype machinery here
>  and here
> , but it's been widely
> ignored IHMO because as Thomas said, the spec keeps changing.
>
> 2017-08-18 12:37 GMT-05:00 Thomas Kluyver :
>
>
> There's prototype machinery to call the hooks here, though it's outdated
> because the spec keeps changing:
> https://github.com/takluyver/pep517
>
> There's a prototype backend in a branch of flit, but it's even more
> outdated:
> https://github.com/takluyver/flit/tree/toml-config
>
>
> On Fri, Aug 18, 2017, at 06:27 PM, Daniel Holth wrote:
>
> Apart from the issues, can we get some prototype implementations?
>
> On Fri, Aug 18, 2017, 13:24 xoviat  wrote:
>
> Thomas:
>
>
> What are the specific issues that need to be worked out?
>
>
> Regards,
>
>
> xoviat
>
> 2017-08-18 3:09 GMT-05:00 Thomas Kluyver :
>
> On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
> > The other aspect I'm not clear on is whether or not both PEP authors
> > are otherwise happy for the current version to be *considered* for
> > acceptance
>
> Nathaniel has mentioned to me a list of issues he sees that we still
> need to work through. So I think we're going to be working on this for a
> while yet.
>
> Thomas
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

Excuse me, but what I meant to say is that sys.path would be adjusted after
the subprocess was loaded (in my implementation I adjust sys.path and
os.environ['PYTHONPATH']).

2017-08-19 16:22 GMT-04:00 xoviat :

> Yes, it probably is. But then again, I assumed that it was obvious that
> PYTHONPATH would not be modified before invoking the hook (at least obvious
> to me) until the 'PEP 517 Status' discussion implied that it was not so
> obvious and needed to be specified in the PEP.
>
> 2017-08-19 16:10 GMT-04:00 Daniel Holth :
>
>> It's probably a tiny wrapper running the hook in its own subprocess.
>> Probably few modules loaded.
>>
>> On Sat, Aug 19, 2017, 14:31 xoviat  wrote:
>>
>>> Ah the joy of Python 2.7; it seems I've forgotten its perils: unloading
>>> is not possible which means that we would need to come up with another
>>> solution to this problem. Perhaps setting aside a namespace for the build
>>> frontend in the subprocess?
>>>
>>> 2017-08-19 14:23 GMT-04:00 xoviat :
>>>
 I assume that the outstanding issues mentioned are related to sys.path
 in the build tree. My view on that is the following: the build frontend
 should be responsible using any mechanism that it chooses for invoking the
 build backend, which must be imported with '' at the front of sys.path.
 This obviously means that if the build frontend experiences faults before
 it manages to import and invoke the backend, then it's the build frontend's
 fault.

 The only potential issue remaining that I can think of then is: what
 about modules that are already imported when the build backend is called?
 WRT standard library modules, we could simply say that build backends must
 be prepared to function in any environment where standard library modules
 are already imported, which I think is a reasonable requirement. But what
 about imports that aren't standard library modules but are fairly common,
 like pip imports? The simplest way to simplify the interface is probably to
 say that all non-standard library modules must be removed from sys.modules
 (unloaded) before the build backend is called.

 What do others think?

 2017-08-18 23:41 GMT-04:00 Nick Coghlan :

> On 19 August 2017 at 05:28, Thomas Kluyver 
> wrote:
> > We've probably all wished that the discussion could be brought to a
> swift
> > conclusion. But there are real questions to work out, and people
> have many
> > other things to pay attention to. I'm frustrated by how long it's
> taking as
> > well, but there's no magic button anyone can press to make it go
> quickly.
>
> Technically, commercial redistributors do have a magic button we could
> press called "ongoing funding for sustaining engineering in the
> upstream Python ecosystem" (since that kind of funding can also cover
> needs-driven UX improvements), but alas, whether or not we ever
> actually press that is conditional on potential customers pressing the
> "customer demand" button hard enough and often enough to light up the
> "viable business opportunity" sign :P
>
> I'm actually genuine curious to see how those commercial dynamics
> start changing as the end date for community support of the Python 2.x
> series gets closer :)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>


>>> ___
>>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

Yes, it probably is. But then again, I assumed that it was obvious that
PYTHONPATH would not be modified before invoking the hook (at least obvious
to me) until the 'PEP 517 Status' discussion implied that it was not so
obvious and needed to be specified in the PEP.

2017-08-19 16:10 GMT-04:00 Daniel Holth :

> It's probably a tiny wrapper running the hook in its own subprocess.
> Probably few modules loaded.
>
> On Sat, Aug 19, 2017, 14:31 xoviat  wrote:
>
>> Ah the joy of Python 2.7; it seems I've forgotten its perils: unloading
>> is not possible which means that we would need to come up with another
>> solution to this problem. Perhaps setting aside a namespace for the build
>> frontend in the subprocess?
>>
>> 2017-08-19 14:23 GMT-04:00 xoviat :
>>
>>> I assume that the outstanding issues mentioned are related to sys.path
>>> in the build tree. My view on that is the following: the build frontend
>>> should be responsible using any mechanism that it chooses for invoking the
>>> build backend, which must be imported with '' at the front of sys.path.
>>> This obviously means that if the build frontend experiences faults before
>>> it manages to import and invoke the backend, then it's the build frontend's
>>> fault.
>>>
>>> The only potential issue remaining that I can think of then is: what
>>> about modules that are already imported when the build backend is called?
>>> WRT standard library modules, we could simply say that build backends must
>>> be prepared to function in any environment where standard library modules
>>> are already imported, which I think is a reasonable requirement. But what
>>> about imports that aren't standard library modules but are fairly common,
>>> like pip imports? The simplest way to simplify the interface is probably to
>>> say that all non-standard library modules must be removed from sys.modules
>>> (unloaded) before the build backend is called.
>>>
>>> What do others think?
>>>
>>> 2017-08-18 23:41 GMT-04:00 Nick Coghlan :
>>>
 On 19 August 2017 at 05:28, Thomas Kluyver 
 wrote:
 > We've probably all wished that the discussion could be brought to a
 swift
 > conclusion. But there are real questions to work out, and people have
 many
 > other things to pay attention to. I'm frustrated by how long it's
 taking as
 > well, but there's no magic button anyone can press to make it go
 quickly.

 Technically, commercial redistributors do have a magic button we could
 press called "ongoing funding for sustaining engineering in the
 upstream Python ecosystem" (since that kind of funding can also cover
 needs-driven UX improvements), but alas, whether or not we ever
 actually press that is conditional on potential customers pressing the
 "customer demand" button hard enough and often enough to light up the
 "viable business opportunity" sign :P

 I'm actually genuine curious to see how those commercial dynamics
 start changing as the end date for community support of the Python 2.x
 series gets closer :)

 Cheers,
 Nick.

 --
 Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia

>>>
>>>
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Daniel Holth]

It's probably a tiny wrapper running the hook in its own subprocess.
Probably few modules loaded.

On Sat, Aug 19, 2017, 14:31 xoviat  wrote:

> Ah the joy of Python 2.7; it seems I've forgotten its perils: unloading is
> not possible which means that we would need to come up with another
> solution to this problem. Perhaps setting aside a namespace for the build
> frontend in the subprocess?
>
> 2017-08-19 14:23 GMT-04:00 xoviat :
>
>> I assume that the outstanding issues mentioned are related to sys.path in
>> the build tree. My view on that is the following: the build frontend should
>> be responsible using any mechanism that it chooses for invoking the build
>> backend, which must be imported with '' at the front of sys.path. This
>> obviously means that if the build frontend experiences faults before it
>> manages to import and invoke the backend, then it's the build frontend's
>> fault.
>>
>> The only potential issue remaining that I can think of then is: what
>> about modules that are already imported when the build backend is called?
>> WRT standard library modules, we could simply say that build backends must
>> be prepared to function in any environment where standard library modules
>> are already imported, which I think is a reasonable requirement. But what
>> about imports that aren't standard library modules but are fairly common,
>> like pip imports? The simplest way to simplify the interface is probably to
>> say that all non-standard library modules must be removed from sys.modules
>> (unloaded) before the build backend is called.
>>
>> What do others think?
>>
>> 2017-08-18 23:41 GMT-04:00 Nick Coghlan :
>>
>>> On 19 August 2017 at 05:28, Thomas Kluyver  wrote:
>>> > We've probably all wished that the discussion could be brought to a
>>> swift
>>> > conclusion. But there are real questions to work out, and people have
>>> many
>>> > other things to pay attention to. I'm frustrated by how long it's
>>> taking as
>>> > well, but there's no magic button anyone can press to make it go
>>> quickly.
>>>
>>> Technically, commercial redistributors do have a magic button we could
>>> press called "ongoing funding for sustaining engineering in the
>>> upstream Python ecosystem" (since that kind of funding can also cover
>>> needs-driven UX improvements), but alas, whether or not we ever
>>> actually press that is conditional on potential customers pressing the
>>> "customer demand" button hard enough and often enough to light up the
>>> "viable business opportunity" sign :P
>>>
>>> I'm actually genuine curious to see how those commercial dynamics
>>> start changing as the end date for community support of the Python 2.x
>>> series gets closer :)
>>>
>>> Cheers,
>>> Nick.
>>>
>>> --
>>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>>>
>>
>>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

Ah the joy of Python 2.7; it seems I've forgotten its perils: unloading is
not possible which means that we would need to come up with another
solution to this problem. Perhaps setting aside a namespace for the build
frontend in the subprocess?

2017-08-19 14:23 GMT-04:00 xoviat :

> I assume that the outstanding issues mentioned are related to sys.path in
> the build tree. My view on that is the following: the build frontend should
> be responsible using any mechanism that it chooses for invoking the build
> backend, which must be imported with '' at the front of sys.path. This
> obviously means that if the build frontend experiences faults before it
> manages to import and invoke the backend, then it's the build frontend's
> fault.
>
> The only potential issue remaining that I can think of then is: what about
> modules that are already imported when the build backend is called? WRT
> standard library modules, we could simply say that build backends must be
> prepared to function in any environment where standard library modules are
> already imported, which I think is a reasonable requirement. But what about
> imports that aren't standard library modules but are fairly common, like
> pip imports? The simplest way to simplify the interface is probably to say
> that all non-standard library modules must be removed from sys.modules
> (unloaded) before the build backend is called.
>
> What do others think?
>
> 2017-08-18 23:41 GMT-04:00 Nick Coghlan :
>
>> On 19 August 2017 at 05:28, Thomas Kluyver  wrote:
>> > We've probably all wished that the discussion could be brought to a
>> swift
>> > conclusion. But there are real questions to work out, and people have
>> many
>> > other things to pay attention to. I'm frustrated by how long it's
>> taking as
>> > well, but there's no magic button anyone can press to make it go
>> quickly.
>>
>> Technically, commercial redistributors do have a magic button we could
>> press called "ongoing funding for sustaining engineering in the
>> upstream Python ecosystem" (since that kind of funding can also cover
>> needs-driven UX improvements), but alas, whether or not we ever
>> actually press that is conditional on potential customers pressing the
>> "customer demand" button hard enough and often enough to light up the
>> "viable business opportunity" sign :P
>>
>> I'm actually genuine curious to see how those commercial dynamics
>> start changing as the end date for community support of the Python 2.x
>> series gets closer :)
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>>
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

I assume that the outstanding issues mentioned are related to sys.path in
the build tree. My view on that is the following: the build frontend should
be responsible using any mechanism that it chooses for invoking the build
backend, which must be imported with '' at the front of sys.path. This
obviously means that if the build frontend experiences faults before it
manages to import and invoke the backend, then it's the build frontend's
fault.

The only potential issue remaining that I can think of then is: what about
modules that are already imported when the build backend is called? WRT
standard library modules, we could simply say that build backends must be
prepared to function in any environment where standard library modules are
already imported, which I think is a reasonable requirement. But what about
imports that aren't standard library modules but are fairly common, like
pip imports? The simplest way to simplify the interface is probably to say
that all non-standard library modules must be removed from sys.modules
(unloaded) before the build backend is called.

What do others think?

2017-08-18 23:41 GMT-04:00 Nick Coghlan :

> On 19 August 2017 at 05:28, Thomas Kluyver  wrote:
> > We've probably all wished that the discussion could be brought to a swift
> > conclusion. But there are real questions to work out, and people have
> many
> > other things to pay attention to. I'm frustrated by how long it's taking
> as
> > well, but there's no magic button anyone can press to make it go quickly.
>
> Technically, commercial redistributors do have a magic button we could
> press called "ongoing funding for sustaining engineering in the
> upstream Python ecosystem" (since that kind of funding can also cover
> needs-driven UX improvements), but alas, whether or not we ever
> actually press that is conditional on potential customers pressing the
> "customer demand" button hard enough and often enough to light up the
> "viable business opportunity" sign :P
>
> I'm actually genuine curious to see how those commercial dynamics
> start changing as the end date for community support of the Python 2.x
> series gets closer :)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Nick Coghlan]

On 19 August 2017 at 05:28, Thomas Kluyver  wrote:
> We've probably all wished that the discussion could be brought to a swift
> conclusion. But there are real questions to work out, and people have many
> other things to pay attention to. I'm frustrated by how long it's taking as
> well, but there's no magic button anyone can press to make it go quickly.

Technically, commercial redistributors do have a magic button we could
press called "ongoing funding for sustaining engineering in the
upstream Python ecosystem" (since that kind of funding can also cover
needs-driven UX improvements), but alas, whether or not we ever
actually press that is conditional on potential customers pressing the
"customer demand" button hard enough and often enough to light up the
"viable business opportunity" sign :P

I'm actually genuine curious to see how those commercial dynamics
start changing as the end date for community support of the Python 2.x
series gets closer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Thomas Kluyver]

I wouldn't expect established tools like pip & setuptools to accept pull
requests implementing a PEP which is still under discussion and subject
to change. They will naturally want a stable, accepted spec before they
consider merging code for it.
We've probably all wished that the discussion could be brought to a
swift conclusion. But there are real questions to work out, and people
have many other things to pay attention to. I'm frustrated by how long
it's taking as well, but there's no magic button anyone can press to
make it go quickly.
Thomas




On Fri, Aug 18, 2017, at 08:17 PM, xoviat wrote:
> At this point, I can only offer implementations as I have been
> attempting to do (some of which are outdated because the PEP has
> changed), but I cannot make anyone pay attention to or accept my pull
> requests and it seems I cannot accelerate the discussion here.> 
> 2017-08-18 14:08 GMT-05:00 xoviat :
>> I have attempted to implement prototype machinery here[1] and
>> here[2], but it's been widely ignored IHMO because as Thomas said,
>> the spec keeps changing.>> 
>> 2017-08-18 12:37 GMT-05:00 Thomas Kluyver :
>>> __
>>> There's prototype machinery to call the hooks here, though it's
>>> outdated because the spec keeps changing:>>> 
>>> https://github.com/takluyver/pep517
>>> 
>>> There's a prototype backend in a branch of flit, but it's even more
>>> outdated:>>> https://github.com/takluyver/flit/tree/toml-config
>>> 
>>> 
>>> On Fri, Aug 18, 2017, at 06:27 PM, Daniel Holth wrote:
 Apart from the issues, can we get some prototype implementations? 
 On Fri, Aug 18, 2017, 13:24 xoviat  wrote:
> Thomas:
> 
> 
> What are the specific issues that need to be worked out? 
> 
> 
> Regards,
> 
> 
> xoviat
> 
> 2017-08-18 3:09 GMT-05:00 Thomas Kluyver :
>> On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
>>  > The other aspect I'm not clear on is whether or not both PEP
>>  > authors are otherwise happy for the current version to be
>>  > *considered* for acceptance
>>
>> Nathaniel has mentioned to me a list of issues he sees that we
>> still>> need to work through. So I think we're going to be working on
>> this for a>> while yet.
>> 
>> Thomas
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
> 
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>>> 


Links:

  1. https://github.com/pypa/pip/pull/4589
  2. https://github.com/pypa/setuptools/pull/1039
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

At this point, I can only offer implementations as I have been attempting
to do (some of which are outdated because the PEP has changed), but I
cannot make anyone pay attention to or accept my pull requests and it seems
I cannot accelerate the discussion here.

2017-08-18 14:08 GMT-05:00 xoviat :

> I have attempted to implement prototype machinery here
>  and here
> , but it's been widely
> ignored IHMO because as Thomas said, the spec keeps changing.
>
> 2017-08-18 12:37 GMT-05:00 Thomas Kluyver :
>
>> There's prototype machinery to call the hooks here, though it's outdated
>> because the spec keeps changing:
>> https://github.com/takluyver/pep517
>>
>> There's a prototype backend in a branch of flit, but it's even more
>> outdated:
>> https://github.com/takluyver/flit/tree/toml-config
>>
>>
>> On Fri, Aug 18, 2017, at 06:27 PM, Daniel Holth wrote:
>>
>> Apart from the issues, can we get some prototype implementations?
>>
>> On Fri, Aug 18, 2017, 13:24 xoviat  wrote:
>>
>> Thomas:
>>
>>
>> What are the specific issues that need to be worked out?
>>
>>
>> Regards,
>>
>>
>> xoviat
>>
>> 2017-08-18 3:09 GMT-05:00 Thomas Kluyver :
>>
>> On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
>> > The other aspect I'm not clear on is whether or not both PEP authors
>> > are otherwise happy for the current version to be *considered* for
>> > acceptance
>>
>> Nathaniel has mentioned to me a list of issues he sees that we still
>> need to work through. So I think we're going to be working on this for a
>> while yet.
>>
>> Thomas
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

I have attempted to implement prototype machinery here
 and here
, but it's been widely
ignored IHMO because as Thomas said, the spec keeps changing.

2017-08-18 12:37 GMT-05:00 Thomas Kluyver :

> There's prototype machinery to call the hooks here, though it's outdated
> because the spec keeps changing:
> https://github.com/takluyver/pep517
>
> There's a prototype backend in a branch of flit, but it's even more
> outdated:
> https://github.com/takluyver/flit/tree/toml-config
>
>
> On Fri, Aug 18, 2017, at 06:27 PM, Daniel Holth wrote:
>
> Apart from the issues, can we get some prototype implementations?
>
> On Fri, Aug 18, 2017, 13:24 xoviat  wrote:
>
> Thomas:
>
>
> What are the specific issues that need to be worked out?
>
>
> Regards,
>
>
> xoviat
>
> 2017-08-18 3:09 GMT-05:00 Thomas Kluyver :
>
> On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
> > The other aspect I'm not clear on is whether or not both PEP authors
> > are otherwise happy for the current version to be *considered* for
> > acceptance
>
> Nathaniel has mentioned to me a list of issues he sees that we still
> need to work through. So I think we're going to be working on this for a
> while yet.
>
> Thomas
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Thomas Kluyver]

There's prototype machinery to call the hooks here, though it's outdated
because the spec keeps changing:https://github.com/takluyver/pep517

There's a prototype backend in a branch of flit, but it's even
more outdated:https://github.com/takluyver/flit/tree/toml-config


On Fri, Aug 18, 2017, at 06:27 PM, Daniel Holth wrote:
> Apart from the issues, can we get some prototype implementations?


> 
> On Fri, Aug 18, 2017, 13:24 xoviat  wrote:
>> Thomas:
>> 
>> 
>> What are the specific issues that need to be worked out? 
>> 
>> 
>> Regards,
>> 
>> 
>> xoviat
>> 
>> 2017-08-18 3:09 GMT-05:00 Thomas Kluyver :
>>> On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
>>>  > The other aspect I'm not clear on is whether or not both PEP
>>>  > authors are otherwise happy for the current version to be
>>>  > *considered* for acceptance
>>>
>>> Nathaniel has mentioned to me a list of issues he sees that we still>>>  
>>> need to work through. So I think we're going to be working on this
>>>  for a>>>  while yet.
>>> 
>>>  Thomas
>>> ___
>>>  Distutils-SIG maillist  -  Distutils-SIG@python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>> 
>> ___
>>  Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Daniel Holth]

Apart from the issues, can we get some prototype implementations?

On Fri, Aug 18, 2017, 13:24 xoviat  wrote:

> Thomas:
>
>
> What are the specific issues that need to be worked out?
>
>
> Regards,
>
>
> xoviat
>
> 2017-08-18 3:09 GMT-05:00 Thomas Kluyver :
>
>> On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
>> > The other aspect I'm not clear on is whether or not both PEP authors
>> > are otherwise happy for the current version to be *considered* for
>> > acceptance
>>
>> Nathaniel has mentioned to me a list of issues he sees that we still
>> need to work through. So I think we're going to be working on this for a
>> while yet.
>>
>> Thomas
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Thomas Kluyver]

I'll let Nathaniel relay them.


On Fri, Aug 18, 2017, at 06:23 PM, xoviat wrote:
> Thomas:
> 
> 
> What are the specific issues that need to be worked out? 
> 
> 
> Regards,
> 
> 
> xoviat
> 
> 2017-08-18 3:09 GMT-05:00 Thomas Kluyver :
>> On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
>>  > The other aspect I'm not clear on is whether or not both PEP
>>  > authors are otherwise happy for the current version to be
>>  > *considered* for acceptance
>>
>> Nathaniel has mentioned to me a list of issues he sees that we still>>  need 
>> to work through. So I think we're going to be working on
>>  this for a>>  while yet.
>> 
>>  Thomas
>> ___
>>  Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

Thomas:


What are the specific issues that need to be worked out?


Regards,


xoviat

2017-08-18 3:09 GMT-05:00 Thomas Kluyver :

> On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
> > The other aspect I'm not clear on is whether or not both PEP authors
> > are otherwise happy for the current version to be *considered* for
> > acceptance
>
> Nathaniel has mentioned to me a list of issues he sees that we still
> need to work through. So I think we're going to be working on this for a
> while yet.
>
> Thomas
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Thomas Kluyver]

On Fri, Aug 18, 2017, at 07:09 AM, Nick Coghlan wrote:
> The other aspect I'm not clear on is whether or not both PEP authors
> are otherwise happy for the current version to be *considered* for
> acceptance

Nathaniel has mentioned to me a list of issues he sees that we still
need to work through. So I think we're going to be working on this for a
while yet.

Thomas
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Nick Coghlan]

On 18 August 2017 at 14:16, xoviat  wrote:
> I'm still not convinced, but this PEP has been WIP for too long. I think it
> should be marked Accepted.

As previously noted, I'm mostly ready to mark it as Accepted (we're
just missing some specific wording around the use of
NotImplementedError to signal anticipated backend failures from
build_sdist and build_wheel that may trigger a fallback option in the
frontend).

The other aspect I'm not clear on is whether or not both PEP authors
are otherwise happy for the current version to be *considered* for
acceptance - that requires an explicit request for pronouncement, and
we haven't had a fresh one of those since the last round of updates :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[xoviat]

I'm still not convinced, but this PEP has been WIP for too long. I think it
should be marked Accepted.

On Aug 17, 2017 9:51 PM, "Nick Coghlan"  wrote:

> On 18 August 2017 at 02:05, 12345 67890  wrote:
> > After reviewing the PEP, I personally feel that it relies too much on the
> > backend Doing The Right Thing. As it currently stands, it is my
> > understanding that the build backend is called in the source directory
> and
> > is then responsible for handling the entire process to build a compliant
> > wheel. In a PEP 517 world, the build backends may be poorly written
> > initially and may not fully comply with the PEP’s goal to produce the
> same
> > wheel that would be produced from an sdist.
> >
> > Current process:
> >
> > Frontend calls backend in source directory build-wheel
>
> The PEP *allows* this, but it doesn't *require* it.
>
> > Proposed process:
> >
> > Frontend copies source tree to temporary directory
> > Frontend invokes build-sdist to build an sdist
> > Frontend extracts sdist to new temporary directory
> > Frontend reloads backend from sdist directory and invokes build-wheel
>
> A particularly paranoid front-end remains free to work this way if it
> wants to.
>
> From the PEP's point of view, it's just four distinct operations:
>
> Step 0: make a copy of the original source tree
> Step 1: build an sdist from the copied source tree
> Step 2: unpack the sdist
> Step 3: build a wheel from the unpacked sdist
>
> > The proposed process is more computationally intensive, but moves
> compliance
> > logic out of the build backend. In addition, we can make some
> modifications
> > based on this proposed process:
> >
> > * Remove parameter for build_dir in build_wheel since it is always the
> current
> > directory
>
> We've had that discussion - build_dir models something different from
> what you propose (specifically, it's designed to invoke the backend's
> native out-of-tree build support, when that's what the frontend
> actually wants to invoke. If a frontend doesn't want that, don't use
> the setting)
>
> > Optimization: add new parameter to build_sdist called copy_extra, that
> when
> > set to false, only includes files in sdist needed to build a  wheel.
>
> No, this is the operation we decided makes no sense (and hence removed
> from the latest iteration of the PEP), because no build system we
> could find actually provides it natively.
>
> That means it would necessarily be a Python-specific capability
> anyway, and that further means that any backend which could be trusted
> to implement this feature correctly can instead be trusted to
> implement building an sdist or wheel directly from a source tree
> correctly.
>
> It's also not a major concern if early backend iterations *don't* do
> things correctly - those errors will be reported as bugs, fixed
> versions of the backends will be released, and future builds will
> start doing the right thing.
>
> > Specify that build_wheel should fail if PKG-INFO is not present so that
> it
> > is always invoked in an sdist
>
> No, we don't want to make that a mandatory requirement, since
> frontends are already free to enforce that rule if they want to (by
> checking for PKG-INFO before they call build_wheel), and particular
> backends are also free to enforce it for the packages they manage (by
> raising NotImplemented when they don't find PKG-INFO).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517: Build frontend responsibilities

[Nick Coghlan]

On 18 August 2017 at 02:05, 12345 67890  wrote:
> After reviewing the PEP, I personally feel that it relies too much on the
> backend Doing The Right Thing. As it currently stands, it is my
> understanding that the build backend is called in the source directory and
> is then responsible for handling the entire process to build a compliant
> wheel. In a PEP 517 world, the build backends may be poorly written
> initially and may not fully comply with the PEP’s goal to produce the same
> wheel that would be produced from an sdist.
>
> Current process:
>
> Frontend calls backend in source directory build-wheel

The PEP *allows* this, but it doesn't *require* it.

> Proposed process:
>
> Frontend copies source tree to temporary directory
> Frontend invokes build-sdist to build an sdist
> Frontend extracts sdist to new temporary directory
> Frontend reloads backend from sdist directory and invokes build-wheel

A particularly paranoid front-end remains free to work this way if it wants to.

From the PEP's point of view, it's just four distinct operations:

Step 0: make a copy of the original source tree
Step 1: build an sdist from the copied source tree
Step 2: unpack the sdist
Step 3: build a wheel from the unpacked sdist

> The proposed process is more computationally intensive, but moves compliance
> logic out of the build backend. In addition, we can make some modifications
> based on this proposed process:
>
> * Remove parameter for build_dir in build_wheel since it is always the current
> directory

We've had that discussion - build_dir models something different from
what you propose (specifically, it's designed to invoke the backend's
native out-of-tree build support, when that's what the frontend
actually wants to invoke. If a frontend doesn't want that, don't use
the setting)

> Optimization: add new parameter to build_sdist called copy_extra, that when
> set to false, only includes files in sdist needed to build a  wheel.

No, this is the operation we decided makes no sense (and hence removed
from the latest iteration of the PEP), because no build system we
could find actually provides it natively.

That means it would necessarily be a Python-specific capability
anyway, and that further means that any backend which could be trusted
to implement this feature correctly can instead be trusted to
implement building an sdist or wheel directly from a source tree
correctly.

It's also not a major concern if early backend iterations *don't* do
things correctly - those errors will be reported as bugs, fixed
versions of the backends will be released, and future builds will
start doing the right thing.

> Specify that build_wheel should fail if PKG-INFO is not present so that it
> is always invoked in an sdist

No, we don't want to make that a mandatory requirement, since
frontends are already free to enforce that rule if they want to (by
checking for PKG-INFO before they call build_wheel), and particular
backends are also free to enforce it for the packages they manage (by
raising NotImplemented when they don't find PKG-INFO).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] PEP 517: Build frontend responsibilities

[12345 67890]

All:

After reviewing the PEP, I personally feel that it relies too much on the 
backend Doing The Right Thing. As it currently stands, it is my understanding 
that the build backend is called in the source directory and is then 
responsible for handling the entire process to build a compliant wheel. In a 
PEP 517 world, the build backends may be poorly written initially and may not 
fully comply with the PEP’s goal to produce the same wheel that would be 
produced from an sdist. 

Current process:

- Frontend calls backend in source directory build-wheel

Proposed process:
- Frontend copies source tree to temporary directory
- Frontend invokes build-sdist to build an sdist
- Frontend extracts sdist to new temporary directory
- Frontend reloads backend from sdist directory and invokes build-wheel

The proposed process is more computationally intensive, but moves compliance 
logic out of the build backend. In addition, we can make some modifications 
based on this proposed process:

- Remove parameter for build_dir in build_wheel since it is always the current 
directory
- Optimization: add new parameter to build_sdist called copy_extra, that when 
set to false, only includes files in sdist needed to build a  wheel. 
- Specify that build_wheel should fail if PKG-INFO is not present so that it is 
always invoked in an sdist

I realized that I may have pushed to have this completed earlier, but I agree 
with Donald that we need to enforce a process rather than hoping for the best. 


Regards,


xoviat
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig