[Distutils] Fwd: What is the official position on distutils?
Hi Nick (re-sending to the list) On Sun, Sep 4, 2016 at 5:16 PM, Nick Coghlan wrote: > > It's important to note that one of our key goals in PyPA is to get to > a point where the API of a developer's publishing toolchain of choice > only depends on the *version of the publishing tools* they're using, > and not on the specific version of Python (as long as they're using a > supported version). Encouraging the expansion and increased use of the > distutils API runs counter to that goal in most cases (since it > encourages writing publication and distribution code that will only > work with newer versions of Python), but may sometimes make sense if > it's info that really is CPython centric, or if it's a feature > primarily aimed at tightly controlled environments that can > aggressively drop support for older versions. Today's state of things is that any project that is not a trivial pure-python module often has use of a combination of all three libraries: distutils, setuptools and pip in their setup.py and in a convoluted way. Newcomers to the toolchain look in similar projects for the right incantations that will fulfill their needs. We are pretty far from being in a state where good practices can even be defined. As a package author, I generally make sure that my need is not fulfilled by a proper usage of the standard library before adding a dependency to a third-party library, but packaging is an area where this does not work. Requiring pip only to know where distutils puts assets is pretty bad. Interestingly, on the subject of building extensions (where the has_flag method discussion applies), distutils.ccompiler is the only available tool and happens to not be monkey-patched by setuptools at the moment. Although setuptools re-defines buildext (and tries to get a version from cython first!) So it seems that there no real "right place" within setuptools to monkey-patch distutils' ccompiler. Sylvain ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What is the official position on distutils?
+1 On Sun, Sep 4, 2016 at 7:02 AM, Steve Dower wrote: > "add it to setuptools first, and then add things to distutils where we > feel they're sufficiently stable to not need the benefit of the faster > setuptools update cycle" > > I nominate this as the official policy on API changes for distutils, with > regular maintenance mode policies applying to anything else. > > Top-posted from my Windows Phone > -- > From: Nick Coghlan > Sent: 9/4/2016 2:19 > To: Sylvain Corlay > Cc: distutils sig > Subject: Re: [Distutils] What is the official position on distutils? > > On 4 September 2016 at 06:44, Sylvain Corlay > wrote: > > Hi Brett, > > > > On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon wrote: > >> > >> > >> If Jason is up for the responsibility that seems like a reasonable > >> approach to take. It also helps test out features in setuptools first > before > >> upstreaming it. > >> > > > > How do you see `has_flag` get into setuptools? By monkey-patching > distutils' > > ccompiler to have it aside of `has_function` when setuptools is imported? > > > > I find really weird the idea of adding this in a convoluted fashion > instead > > of allowing incremental improvement of distutils. > > The change to distutils would still be a plain patch to distutils, it > would just be accepted at the API design level in setuptools first. > > The problem you're running into right now isn't a technical one - it's > that there isn't anyone that currently feels like they have sufficient > design authority over the distutils API to accept your proposal, hence > Brett starting this thread to address that underlying recurring > question, rather than the specifics of your change. > > Jason *definitely* has that design authority over setuptools though, > and will be tasked with making any API additions available on older > versions of Python via setuptools regardless of what policy we adopt > for distutils maintenance, so if he's amenable to the idea, it makes > sense to me to invert the order we ask those questions: add it to > setuptools first, and then add things to distutils where we feel > they're sufficiently stable to not need the benefit of the faster > setuptools update cycle. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -- cordially, Anna ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What is the official position on distutils?
On 5 September 2016 at 00:27, Sylvain Corlay wrote: > Hi Steve, > > - I posted an on setuptools github > https://github.com/pypa/setuptools/issues/771 to ask the setuptools folks > (and especially Jason) whether they would want such a feature. > > I still find it baffling that the preferred way to add distutils features > would be to monkey-patch it via setuptools first and then to add it into > distutils later, while my proposed approach was to add it to distutils and > monkey patch for earlier versions of python. Imagine a world where distutils wasn't rife with global state, and hence setuptools didn't need to monkeypatch it, and the approach seems more logical (e.g. this is roughly the way I've worked for larger contextlib changes - the ContextStack precursor to ExitStack only exists in contextlib2, since I didn't push that API to the stdlib until I was happy with both the design and the name). > - Regarding the policy that you just nominated, does this apply to things > that have been in pip for a long time? In the initial email that triggered > this discussion, it was also question of adding the > `pip.location.distutils_scheme` convenience function to distutils itself. Cf > https://bugs.python.org/issue26955 and pip's implementation at > https://github.com/pypa/pip/blob/281eb61b09d87765d7c2b92f6982b3fe76ccb0af/pip/locations.py#L124 > > Indeed, the problem with certain distutils commands like `install_headers` > is that the only way to retrieve the directory where things were installed > is to instantiate a `distutils.dist.Distribution` > object, which is not trivial to do. pip's distutils_scheme convenience > function does it. This function is *about* distutils, and IMO should really > be *in* distutils. Should it be in distutils, or should it be in the version independent "packaging" helper library that pip uses? It's important to note that one of our key goals in PyPA is to get to a point where the API of a developer's publishing toolchain of choice only depends on the *version of the publishing tools* they're using, and not on the specific version of Python (as long as they're using a supported version). Encouraging the expansion and increased use of the distutils API runs counter to that goal in most cases (since it encourages writing publication and distribution code that will only work with newer versions of Python), but may sometimes make sense if it's info that really is CPython centric, or if it's a feature primarily aimed at tightly controlled environments that can aggressively drop support for older versions. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What is the official position on distutils?
Hi Steve, - I posted an on setuptools github https://github.com/pypa/ setuptools/issues/771 to ask the setuptools folks (and especially Jason) whether they would want such a feature. I still find it baffling that the preferred way to add distutils features would be to monkey-patch it via setuptools first and then to add it into distutils later, while my proposed approach was to add it to distutils and monkey patch for earlier versions of python. - Regarding the policy that you just nominated, does this apply to things that have been in pip for a long time? In the initial email that triggered this discussion, it was also question of adding the `pip.location.distutils_scheme` convenience function to distutils itself. Cf https://bugs.python.org/issue26955 and pip's implementation at https://github.com/pypa/pip/blob/281eb61b09d87765d7c2b92f6982b3fe76ccb0af/pip/locations.py#L124 Indeed, the problem with certain distutils commands like `install_headers` is that the only way to retrieve the directory where things were installed is to instantiate a `distutils.dist.Distribution` object, which is not trivial to do. pip's distutils_scheme convenience function does it. This function is *about* distutils, and IMO should really be *in* distutils. Thanks, Sylvain On Sun, Sep 4, 2016 at 4:02 PM, Steve Dower wrote: > "add it to setuptools first, and then add things to distutils where we > feel they're sufficiently stable to not need the benefit of the faster > setuptools update cycle" > > I nominate this as the official policy on API changes for distutils, with > regular maintenance mode policies applying to anything else. > > Top-posted from my Windows Phone > -- > From: Nick Coghlan > Sent: 9/4/2016 2:19 > To: Sylvain Corlay > Cc: distutils sig > Subject: Re: [Distutils] What is the official position on distutils? > > On 4 September 2016 at 06:44, Sylvain Corlay > wrote: > > Hi Brett, > > > > On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon wrote: > >> > >> > >> If Jason is up for the responsibility that seems like a reasonable > >> approach to take. It also helps test out features in setuptools first > before > >> upstreaming it. > >> > > > > How do you see `has_flag` get into setuptools? By monkey-patching > distutils' > > ccompiler to have it aside of `has_function` when setuptools is imported? > > > > I find really weird the idea of adding this in a convoluted fashion > instead > > of allowing incremental improvement of distutils. > > The change to distutils would still be a plain patch to distutils, it > would just be accepted at the API design level in setuptools first. > > The problem you're running into right now isn't a technical one - it's > that there isn't anyone that currently feels like they have sufficient > design authority over the distutils API to accept your proposal, hence > Brett starting this thread to address that underlying recurring > question, rather than the specifics of your change. > > Jason *definitely* has that design authority over setuptools though, > and will be tasked with making any API additions available on older > versions of Python via setuptools regardless of what policy we adopt > for distutils maintenance, so if he's amenable to the idea, it makes > sense to me to invert the order we ask those questions: add it to > setuptools first, and then add things to distutils where we feel > they're sufficiently stable to not need the benefit of the faster > setuptools update cycle. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What is the official position on distutils?
"add it to setuptools first, and then add things to distutils where we feel they're sufficiently stable to not need the benefit of the faster setuptools update cycle" I nominate this as the official policy on API changes for distutils, with regular maintenance mode policies applying to anything else. Top-posted from my Windows Phone -Original Message- From: "Nick Coghlan" Sent: 9/4/2016 2:19 To: "Sylvain Corlay" Cc: "distutils sig" Subject: Re: [Distutils] What is the official position on distutils? On 4 September 2016 at 06:44, Sylvain Corlay wrote: > Hi Brett, > > On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon wrote: >> >> >> If Jason is up for the responsibility that seems like a reasonable >> approach to take. It also helps test out features in setuptools first before >> upstreaming it. >> > > How do you see `has_flag` get into setuptools? By monkey-patching distutils' > ccompiler to have it aside of `has_function` when setuptools is imported? > > I find really weird the idea of adding this in a convoluted fashion instead > of allowing incremental improvement of distutils. The change to distutils would still be a plain patch to distutils, it would just be accepted at the API design level in setuptools first. The problem you're running into right now isn't a technical one - it's that there isn't anyone that currently feels like they have sufficient design authority over the distutils API to accept your proposal, hence Brett starting this thread to address that underlying recurring question, rather than the specifics of your change. Jason *definitely* has that design authority over setuptools though, and will be tasked with making any API additions available on older versions of Python via setuptools regardless of what policy we adopt for distutils maintenance, so if he's amenable to the idea, it makes sense to me to invert the order we ask those questions: add it to setuptools first, and then add things to distutils where we feel they're sufficiently stable to not need the benefit of the faster setuptools update cycle. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What is the official position on distutils?
On 4 September 2016 at 06:44, Sylvain Corlay wrote: > Hi Brett, > > On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon wrote: >> >> >> If Jason is up for the responsibility that seems like a reasonable >> approach to take. It also helps test out features in setuptools first before >> upstreaming it. >> > > How do you see `has_flag` get into setuptools? By monkey-patching distutils' > ccompiler to have it aside of `has_function` when setuptools is imported? > > I find really weird the idea of adding this in a convoluted fashion instead > of allowing incremental improvement of distutils. The change to distutils would still be a plain patch to distutils, it would just be accepted at the API design level in setuptools first. The problem you're running into right now isn't a technical one - it's that there isn't anyone that currently feels like they have sufficient design authority over the distutils API to accept your proposal, hence Brett starting this thread to address that underlying recurring question, rather than the specifics of your change. Jason *definitely* has that design authority over setuptools though, and will be tasked with making any API additions available on older versions of Python via setuptools regardless of what policy we adopt for distutils maintenance, so if he's amenable to the idea, it makes sense to me to invert the order we ask those questions: add it to setuptools first, and then add things to distutils where we feel they're sufficiently stable to not need the benefit of the faster setuptools update cycle. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
