Re: [Distutils] PEP 517 again

[Nathaniel Smith]

On Sun, Aug 27, 2017 at 4:27 PM, Greg Ewing  wrote:
> Nathaniel Smith wrote:
>>
>> - creating an sdist failed for unexpected reasons, that need a human
>> to sort out (due to a broken system, or bugs – hey, they happen – or
>> ...)
>
>
> I think that should still be reported via an exception. Returning
> None should only be for the specific case that the backend doesn't
> support the requested operation.

Well, you can't exactly say "if your code is buggy, then you should
signal that by doing this well defined thing" :-). One of my
objections to None is that it's very easy to return accidentally,
i.e., buggy code *will* sometimes return None no matter what the spec
says.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 again

[Greg Ewing]

Nathaniel Smith wrote:

- creating an sdist failed for unexpected reasons, that need a human
to sort out (due to a broken system, or bugs – hey, they happen – or
...)


I think that should still be reported via an exception. Returning
None should only be for the specific case that the backend doesn't
support the requested operation.

--
Greg
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Documentation link on PyPI.org

[Brett Cannon]

If you search the archive of this mailing list you will notice I asked this
exact question about a month or 2 ago (I think). The answer I got was it is
used on PyPI.org, but I don't know how to set it with setuptools (flit will
probably add support after PEP 517).

On Sun, Aug 27, 2017, 13:01 Ronald Oussoren  wrote:

> On 27 Aug 2017, at 17:33, Wes Turner  wrote:
>
> You can add a URL in the long_description (e.g. to
> https://read-the-docs.readthedocs.org/ )
>
>
> I know, but that adds the URL to the project description and not the
> sidebar. PEP 345 appears to specify metadata that could be used for this,
> and warehouse seems to contain to make use of this (or at least a database
> model that allows for specifying more links than just the default one).
>
> Adding links to the sidebar would be nicer with the new design on PyPI.org,
> but there doesn’t seem to be a clean way to add “Project-URL” metadata as
> described in PEP 345 to a wheel file. I could tweak my setup.py files to
> add Project-URL metadata, but before I do that I’d like to confirm that (a)
> that metadata is actually used by warehouse and (b) there is no cleaner way
> to add this metadata using setuptools.
>
> Ronald
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Documentation link on PyPI.org

[Ronald Oussoren]

> On 27 Aug 2017, at 17:33, Wes Turner  wrote:
> 
> You can add a URL in the long_description (e.g. to 
> https://read-the-docs.readthedocs.org/ 
>  )

I know, but that adds the URL to the project description and not the sidebar. 
PEP 345 appears to specify metadata that could be used for this, and warehouse 
seems to contain to make use of this (or at least a database model that allows 
for specifying more links than just the default one). 

Adding links to the sidebar would be nicer with the new design on PyPI.org 
, but there doesn’t seem to be a clean way to add 
“Project-URL” metadata as described in PEP 345 to a wheel file. I could tweak 
my setup.py files to add Project-URL metadata, but before I do that I’d like to 
confirm that (a) that metadata is actually used by warehouse and (b) there is 
no cleaner way to add this metadata using setuptools.

Ronald___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Documentation link on PyPI.org

[Wes Turner]

You can add a URL in the long_description (e.g. to
https://read-the-docs.readthedocs.org/ )

It'd be useful to mention that here:
https://packaging.python.org/tutorials/distributing-packages/

It would sure be great to be able to express with structured data that a
project has [Sphinx] docs at a particular URL. It'd then be possible to
discover and search docs for each project automatically.

http://markmail.org/search/?q=project-url+documentation+list%3Aorg.python.distutils-sig

"[Distutils] Provide separate development and documentation URLs in PyPI
metadata?"

http://markmail.org/search/?q=project-url+documentation+list%3Aorg.python.distutils-sig#query:project-url%20documentation%20list%3Aorg.python.distutils-sig+page:1+mid:e2ilbaevv2s3fl2j+state:results

On Sunday, August 27, 2017, Ronald Oussoren  wrote:

> Hi,
>
> Is it possible to create a documentation in the section with “Project
> Links” on the page describing a release on PyPI.org? That is, can I add
> some metadata to an sdist or wheel that would add an entry to that list,
> and if so how? I’m using setuptools to create the distribution artefacts.
>
> The reason I ask is that I currently use an index.html on pythonhosted.org
> that redirects to the real documentation because that adds a link to the
> documentation when looking at the release on pypi.python.org. That
> doesn’t work pypi.org.
>
> Ronald
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Documentation link on PyPI.org

[Ronald Oussoren]

Hi,

Is it possible to create a documentation in the section with “Project Links” on 
the page describing a release on PyPI.org ? That is, can I 
add some metadata to an sdist or wheel that would add an entry to that list, 
and if so how? I’m using setuptools to create the distribution artefacts.

The reason I ask is that I currently use an index.html on pythonhosted.org that 
redirects to the real documentation because that adds a link to the 
documentation when looking at the release on pypi.python.org. That doesn’t work 
pypi.org.   

Ronald___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 again

[Daniel Holth]

Return none makes sense, but I don't think the outcome of that or the
"configurable src path" decisions will affect the success of the proposal.

On Sun, Aug 27, 2017, 05:59 Paul Moore  wrote:

> On 26 August 2017 at 23:13, Nathaniel Smith  wrote:
> > This whole discussion is about handling a specific case: suppose you
> > have a frontend like pip that when given a source directory and asked
> > to build a wheel, wants to implement that as:
> >   - build sdist
> >   - unpack sdist
> >   - build wheel from unpacked sdist
>
> While I can't speak for Donald or the other pip devs, I'd like to
> point out that in
> https://mail.python.org/pipermail/distutils-sig/2017-August/031288.html
> I confirmed that I'm basically persuaded now that pip should just
> "trust the backend" and use build_wheel directly. So this whole debate
> isn't relevant for pip. We'll need to wait for other
> currently-hypothetical frontends like tox or twine which might use the
> build_wheel hook, before we'll have any practical experience to
> confirm if there's an issue with "return None" as the way for backends
> to signal that they weren't able to build a wheel.
>
> Paul
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 - a plan of action

[Jeremy Kloth]

On Sun, Aug 27, 2017 at 7:04 AM, Paul Moore  wrote:
> On 27 August 2017 at 13:41, Jeremy Kloth  wrote:
>> Again, I see what is being developed here (PEP 517) as not that
>> different from what has already been done with DB-API (PEP 249).  If
>> databases can achieve consensus, I'm sure that build tools can too. :)
>
> One difference with the DB-API is that DB exceptions are intended to
> be seen by the user, to be caught and reported in application code.
> The case we're discussing here is where a hook wants to report an
> unusual, but normal, condition (that the backend discovers *at
> runtime* that it can't do what has been requested of it) and the error
> is to be handled by "the other end of the protocol" (the front end)
> and is not for reporting to the user (except in the limited sense that
> we're considering whether the return value/exception needs to carry
> "why this failed" information that the frontend can report to the user
> if it wishes to).

I guess to enhance my point, the correlation I see with DB-API is this:

pip (frontend) --> sqlalchemy
flit|setuptools|scons|make|my-awesome-build-tool (backend) --> mysql/
psycopg/sqlite

In my usage of DB-API backends, I've certainly never thought of their
exceptions as "user facing".  Meaning that if a DB-API defined
exception occurred, it was my responsibility as a developer to handle
it; wrap it in an application-specific exception, format it for
logging, retry the operation, what-have-you.  Just  never let it
escape to the user of my application (in this case, the frontend, aka
pip).

Just attempting to bring a different perspective to the conversation.

-- 
Jeremy Kloth
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 - a plan of action

[Paul Moore]

On 27 August 2017 at 13:41, Jeremy Kloth  wrote:
> Again, I see what is being developed here (PEP 517) as not that
> different from what has already been done with DB-API (PEP 249).  If
> databases can achieve consensus, I'm sure that build tools can too. :)

One difference with the DB-API is that DB exceptions are intended to
be seen by the user, to be caught and reported in application code.
The case we're discussing here is where a hook wants to report an
unusual, but normal, condition (that the backend discovers *at
runtime* that it can't do what has been requested of it) and the error
is to be handled by "the other end of the protocol" (the front end)
and is not for reporting to the user (except in the limited sense that
we're considering whether the return value/exception needs to carry
"why this failed" information that the frontend can report to the user
if it wishes to).

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 - a plan of action

[Paul Moore]

On 27 August 2017 at 12:03, Thomas Kluyver  wrote:
> I believe that the remaining questions are:
> - srcdir-on-sys.path
> - Notimplemented/Error (/None)
>
> Unfortunately, it's not clear to me what's the 'simplest' solution on
> either, and we can't easily change whichever decision we make. Nathaniel
> & Nick still disagree on srcdir-on-sys.path, I believe; I'm not sure how
> we resolve that.
>
> I still think the NotImplemented/Error issue is a prime example of
> bikeshedding. There clearly isn't one 'right' way to do it, because
> intelligent people continue to disagree about it. I'd like somebody to
> make the decision and end the argument, but I'm not sure who is 'above
> the fray' enough to be able to do that.

My feeling on this one (and I know, I'm biased here) is that Guido's
suggestion of returning None to mean "couldn't build a sdist" is the
simplest solution, at least for now. Initial frontend code will be

sdist_path = build_sdist(...)
if sdist_path is None:
deal with no sdist

and that's easy enough to modify to either

sdist_path = build_sdist(...)
if sdist_path is None or sdist_path is NotImplemented:
deal with no sdist

or

try:
sdist_path = build_sdist(...)
# Backward compatibility
if sdist_path is None:
raise NotImplementedError
except NotImplementedError:
deal with no sdist


Add to that the fact that I'm suggesting that there mightn't even be
any call to build_sdist in pip, at least in the first instance, and it
doesn't seem worth doing anything more complex for now.

On the srcdir-on-sys.path question, I don't really have a good answer.
It's part of the discussion I've been skimming/skipping - and the fact
that I have been is one of the things that prompted me to make this
suggestion. If you want, I'll go back and re-read the posts and see
what my view is - but you may be happier if I don't add yet another
viewpoint to confuse things further ;-)

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 - a plan of action

[Jeremy Kloth]

On Sun, Aug 27, 2017 at 5:03 AM, Thomas Kluyver  wrote:
> I believe that the remaining questions are:
> - srcdir-on-sys.path
> - Notimplemented/Error (/None)
>
> Unfortunately, it's not clear to me what's the 'simplest' solution on
> either, and we can't easily change whichever decision we make. Nathaniel
> & Nick still disagree on srcdir-on-sys.path, I believe; I'm not sure how
> we resolve that.
>
> I still think the NotImplemented/Error issue is a prime example of
> bikeshedding. There clearly isn't one 'right' way to do it, because
> intelligent people continue to disagree about it. I'd like somebody to
> make the decision and end the argument, but I'm not sure who is 'above
> the fray' enough to be able to do that.

I just want to bring an additional point to the
NotImplemented(Error)/None debate.  It seems to me that this PEP is
akin to the DB-API (https://www.python.org/dev/peps/pep-0249/).  In
DB-API, structured exceptions *are* used, just defined by the drivers
themselves.  It doesn't seem to be any problem for the driver
implementations to provide them or for clients to handle them.

That said, I would also lean towards its approach towards optional
features.  To quote:

As with all DB API optional features, the database module authors are
free to not implement these additional attributes and methods (using
them will then result in an AttributeError) or to raise a
NotSupportedError in case the availability can only be checked at
run-time.

Again, I see what is being developed here (PEP 517) as not that
different from what has already been done with DB-API (PEP 249).  If
databases can achieve consensus, I'm sure that build tools can too. :)

-- 
Jeremy Kloth
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 - a plan of action

[Thomas Kluyver]

Thanks Paul for providing us with a way to move forwards. I'm in favour
of what you propose.

On Sun, Aug 27, 2017, at 11:37 AM, Paul Moore wrote:
> 2. Thomas to complete and publish his library of helper functions for
> PEP 517 consumers (optional - but I think he said he had such a thing
> in progress).

I do. It's not entirely up-to-date with the spec, but it can be found
here:
https://github.com/takluyver/pep517

My plan is to provide two levels of API:
1. Lower-level machinery to easily call PEP 517 hooks in a subprocess
(this is what I've been writing so far)
2. A higher-level 'give me a wheel' interface which identifies the build
dependencies, installs them into a build environment, and calls the
build_wheel hook. Alternatively, this could be a separate package.

I'd like to transfer this repo to PyPA once it's a bit more ready -
assuming PyPA wants it.

> 3. Build PEP 517 interfaces for flit and setuptools.

The work for flit is well out of date, but it can be found here:
https://github.com/takluyver/flit/tree/toml-config

> 1. On any remaining questions for the PEP, let's just go for the
> simplest solution for now. Let the PEP authors decide what that is,
> with Nick as BDFL arbitrating if there's disagreement.

I believe that the remaining questions are:
- srcdir-on-sys.path
- Notimplemented/Error (/None)

Unfortunately, it's not clear to me what's the 'simplest' solution on
either, and we can't easily change whichever decision we make. Nathaniel
& Nick still disagree on srcdir-on-sys.path, I believe; I'm not sure how
we resolve that.

I still think the NotImplemented/Error issue is a prime example of
bikeshedding. There clearly isn't one 'right' way to do it, because
intelligent people continue to disagree about it. I'd like somebody to
make the decision and end the argument, but I'm not sure who is 'above
the fray' enough to be able to do that.

> Please understand - I'm not trying to steamroller the process here, or
> force particular decisions. It's just that I know I'm burned out on
> the discussions, to the extent that I'm skimming mails and largely
> ignoring many of the hypothetical arguments, and I'm sure others are
> too.

+1
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] PEP 517 - a plan of action

[Paul Moore]

I feel that the discussion on PEP 517 is nearing an end. The
outstanding questions are pretty detail-level, and there's little
progress being made on them because the situations involved are mostly
theoretical in the absence of actual implementations. So I propose the
following plan of action:

1. Accept PEP 517 (at least provisionally).
2. Thomas to complete and publish his library of helper functions for
PEP 517 consumers (optional - but I think he said he had such a thing
in progress).
3. Build PEP 517 interfaces for flit and setuptools.
4. Implement PEP 517 support in pip.
5. Review what we learned from the above.
6. If needed, revise the PEP in view of actual implementation
experience, and finalise it.

Further frontends and backends can wait for the results of this
process if they are concerned about tracking a provisional PEP.
Hopefully, it shouldn't be too long a process (I'd like to see PEP 517
in the next release of pip, if possible).

Notes
-

1. On any remaining questions for the PEP, let's just go for the
simplest solution for now. Let the PEP authors decide what that is,
with Nick as BDFL arbitrating if there's disagreement.
2. In my view, the PEP 517 support in pip should fully embrace the
principle in the PEP that it's the backend's job to ensure that
sdist->wheel and build_wheel produce equivalent results (when both are
possible), and should therefore *only* use build_wheel, at least until
experience demonstrates that we have to protect users against badly
written backends.
3. My suspicion is that the setuptools backend will pretty clearly
demonstrate whether it's hard or easy to conform to the requirements
in the PEP, which can inform pip's stance on a "pure build_wheel"
approach.
4. If pip goes for direct wheel builds, this does mean that the
build_sdist and get_requires_for_build_sdist hooks won't be exercised
in any actual code during this process. IMO, that's OK, as those hooks
cover important (if currently theoretical) cases for validating the
"publish my project" workflow, and tools that want to address those
cases (such as tox) are likely to need the sdist hooks later.

If this seems like a reasonable plan, I'd suggest that people add
links to issues or PRs for PEP 517 implementations in flit, setuptools
and pip to this thread - as I'm sure all the participants here will
want to review the concrete implementations.

Please understand - I'm not trying to steamroller the process here, or
force particular decisions. It's just that I know I'm burned out on
the discussions, to the extent that I'm skimming mails and largely
ignoring many of the hypothetical arguments, and I'm sure others are
too. So we need to move to a point where we can discuss actual,
in-development code rather than possibilities, and I believe the above
plan gets us to that point as fast as possible.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 again

[Paul Moore]

On 26 August 2017 at 23:13, Nathaniel Smith  wrote:
> This whole discussion is about handling a specific case: suppose you
> have a frontend like pip that when given a source directory and asked
> to build a wheel, wants to implement that as:
>   - build sdist
>   - unpack sdist
>   - build wheel from unpacked sdist

While I can't speak for Donald or the other pip devs, I'd like to
point out that in
https://mail.python.org/pipermail/distutils-sig/2017-August/031288.html
I confirmed that I'm basically persuaded now that pip should just
"trust the backend" and use build_wheel directly. So this whole debate
isn't relevant for pip. We'll need to wait for other
currently-hypothetical frontends like tox or twine which might use the
build_wheel hook, before we'll have any practical experience to
confirm if there's an issue with "return None" as the way for backends
to signal that they weren't able to build a wheel.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig