[Distutils] Re: Critical problem in PyCharm caused by the removal of "--build-dir" in 2020.3

[Pradyun Gedam]

For anyone following along, there is an issue on pip's issue tracker now: 
https://github.com/pypa/pip/issues/9193

Mikhail, if you could hop over there to help coordinate on this, that'd be 
great!
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/7ADV4V255QTWHBZS75HK5UQDH25LSBTP/

[Distutils] Re: pip 20.3 release (new resolver as default)

[Pradyun Gedam]

> ## What to expect in 20.1
>
> We aim to release pip 20.1 in January 2021, per our [usual release
> cadence](
> https://pip.pypa.io/en/latest/development/release-process/#release-cadence).
>
> You can expect:
>
> * Removal of [Python
> 2.7](
> https://pip.pypa.io/en/latest/development/release-process/#python-2-support)
>
> and 3.5 support
> * Further improvements in the new resolver
> * Removal of legacy resolver support
>

A small correction: the next pip release is 21.0, not 20.1.

Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/CXX5MU6GMKGAK2VHFKLBKETK6LNMTN5F/

[Distutils] Archive this list & redirect conversation elsewhere?

[Pradyun Gedam]

TL;DR: OK to archive this mailing list? Reply by Aug 30th.

Below: Context, Proposal, Reasoning, and timeline.

*Context*:

For multiple years now, we've been advising users to use setuptools and
to move away from using distutils. There has been a long standing
plan [^1] to vendor distutils into setuptools to allow it to evolve
independently of the Python standard library and to allow for removal of
distutils from the Python standard library.

Recently, setuptools adopted the distutils [^2] which, as far as I can
tell, starts the long process of removing distutils from the Python
standard library. PEP 517 [^3] also removed the special status of
distutils/setuptools as the only pipeline that can be used for
generating distributions for Python projects.

For some time now, "distutils" has not been a "primary" tool in the
broader Python Packaging ecosystem, with setuptools being an overall
superior tool that also has a good interoperability story with the other
Python packaging tooling. This is acknowledged in the description of
this mailing list as:

> Now, it's better described as the "packaging interoperability SIG",
> where issues that cut across different parts of the Python packaging
> ecosystem get discussed and resolved.

However, this mailing list is no longer serving this stated role either,
with the Packaging category on discuss.python.org becoming the primary
location for packaging tool interoperability discussions.

Over the last year, the Packaging category on discuss.python.org had 841
active topics, with only 40 topics with 3 or fewer responses. [^5]
In the last 100 days, the Packaging category on discuss.python.org has
had 91 active topics. More than 10 PEPs have been discussed in the
Packaging category on discuss.python.org in the last 100 days.

Over the last year, distutils-sig had ~109 active threads, with
(based on a quick skim) most having 3 or fewer responses/posters. [^4]
In the last 100 days, distutils-sig has had 32 active threads (at least
7 of these have the same subject as another thread with Re:/Fwd: added).
There has been only 1 PEP-related feedback discussion on distutils-sig
in the last year. Most of the other threads are user support requests or
announcements.

*Proposal*:

I suggest that, one month from now, we stop posting to this list
(distutils-sig@python.org) and archive it.

*Reasoning*:

I think we do not use this mailing list for its dedicated purpose, and
it does not serve any secondary function that isn't better served by a
different communication channel already.

(1) this mailing list is no longer the primary location for
interoperability discussions

(2) we have better channels for user support requests (such as
issue trackers of various projects, packaging-problems etc.)

(3) we have other channels for making announcements (such as the
   pypi-announce mailing list, the PyPA twitter account,
   discuss.python.org etc.)

*Timeline*:

Here's what I suggest, and what I will carry out if there is no
objection.

In one month, on August 30th, I would verify that no one has argued here
for why this mailing list should not be closed/archived.

Or, even if a few people have objected to closing the list, I would
check for rough consensus, especially of people who are doing SOMETHING
productive having to do with Python Packaging (such as maintainers of
Python packages, maintainers of Python Packaging tooling, folks running
key infrastructure, etc.).

Then, I would request the list administrators to post a final message
to this list (marking its close and suggesting that people use
discuss.python.org instead) and, to archive this mailing list. This
would leave archives available at their current URLs, so links, browsing
and search would work.

And finally, I would look through relevant documentation within PyPA
repositories to see what needs updating (READMEs and so on pointing to
the old list), and submit pull requests.

I appreciate the work folks here have done to carry forward Python
packaging over the past 21 years of this mailing list. I don't mean to
diminish that or to insult anyone here. I want to help us out, and I
think closing this list will help focus our energy better. But I am
open to hearing that I am wrong.

Realizing-that-distutils-sig-is-older-than-me-ly,
Pradyun Gedam

[^1]:
https://www.pypa.io/en/latest/roadmap/#vendor-distutils-into-setuptools
[^2]: https://github.com/pypa/setuptools/pull/2143
[^3]: https://www.python.org/dev/peps/pep-0517/
[^4]:
https://mail.python.org/archives/list/distutils-sig@python.org/latest?count=110=1
[^5]: https://discuss.python.org/c/packaging/14/l/top/yearly?order=posts
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/VZL5AMZ2WXWVOSAFCBSPH2Q3FLZJDOCS/

[Distutils] Announcement: pip 20.2 release!

[Pradyun Gedam]

On behalf of the PyPA, I am pleased to announce that we have just released
pip 20.2, a new version of pip. You can install it by running python -m pip
install --upgrade pip.

The highlights for this release are:

   - The beta of the next-generation dependency resolver is available
   - Faster installations from wheel files
   - Improved handling of wheels containing non-ASCII file contents
   - Faster pip list using parallelized network operations
   - Installed packages now contain metadata about whether they were
   directly
   requested by the user (PEP 376’s REQUESTED file)

The new dependency resolver is *off by default* because it is *not yet
ready for everyday use*. The new dependency resolver is significantly
stricter and more consistent when it receives incompatible instructions,
and reduces support for certain kinds of constraints files, so some
workarounds and workflows may break. Please test it with the
--use-feature=2020-resolver flag. Please see our guide on how to test and
migrate, and how to report issues
.
We are preparing to change the default dependency resolution behavior and
make the new resolver the default in pip 20.3 (in October 2020).

This release also partially optimizes pip’s network usage during
installation (as part of a Google Summer of Code project by McSinyx
). Please
test it with pip install --use-feature=fast-deps ... and report bugs to the
issue tracker
. This
functionality is *still experimental* and *not ready for everyday use*.

You can find more details (including deprecations and removals) in the
changelog .

As with all pip releases, a significant amount of the work was contributed
by pip’s user community. Huge thanks to all who have contributed, whether
through code, documentation, issue reports and/or discussion. Your help
keeps pip improving, and is hugely appreciated.

Specific thanks go to Mozilla (through its Mozilla Open Source Support
 Awards) and to the Chan Zuckerberg
Initiative  DAF, an advised fund of
Silicon Valley Community Foundation, for their funding that enabled
substantial work on the new resolver.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/S7TAGEX34PKXX45KZUXSPNFZ6QAP2IXI/

[Distutils] Re: Announcement: pip 20.1b1 beta release

[Pradyun Gedam]

pip 20.1 has been released! You can install it by running `python -m pip
install --upgrade pip`.

Huge thanks to all who have helped by testing the beta release! During this
beta period, we were able to identify and fix regressions as well as
improve newly added functionality, based on reports and feedback from our
testers. You can find more details in the changelog
<https://pip.pypa.io/en/stable/news/>.

Cheers,
Pradyun

On Tue, Apr 21, 2020 at 7:42 AM Pradyun Gedam  wrote:

> On behalf of the PyPA, I am pleased to announce a beta release of pip,
> pip 20.1b1 has been released.
>
> The highlights for this release are:
>
> * Significant speedups when building local directories, by changing
>   behavior to perform in-place builds, instead of copying to temporary
>   directories.
> * Significant speedups in `pip list --outdated`, by parallelizing
>   network access. This is the first instance of parallel code within
>   pip's codebase.
> * A new `pip cache` command, which makes it possible to introspect and
>   manage pip's cache directory.
> * Better `pip freeze` for packages installed from direct URLs, enabled
>   by the implementation of PEP 610.
>
> We would be grateful for all the testing that users could do, to ensure
> that when pip 20.1 is released, it's as solid as we can make it.
>
> This release also contains an alpha version of pip's next generation
> resolver. It is *off by default* because it
> *unstable and notready for everyday use*.
>
> As with all pip releases, a significant amount of the work was
> contributed by pip's user community. Huge thanks to all who have
> contributed, whether through code, documentation, issue reports and/or
> discussion. Your help keeps pip improving, and is hugely appreciated.
>
> Specific thanks go to Mozilla (through its Mozilla Open Source Support
> <https://www.mozilla.org/en-US/moss/>
> Awards) <https://www.mozilla.org/en-US/moss/> and to the Chan Zuckerberg
> Initiative <https://chanzuckerberg.com/eoss/> DAF, an advised fund of
> Silicon Valley Community Foundation, for their support that enabled the
> work on the new resolver.
>
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/WTIYCFYXXSCOBIOXNIJIINPSGRFAIVMN/

[Distutils] Announcement: pip 20.1b1 beta release

[Pradyun Gedam]

On behalf of the PyPA, I am pleased to announce a beta release of pip,
pip 20.1b1 has been released.

The highlights for this release are:

* Significant speedups when building local directories, by changing
  behavior to perform in-place builds, instead of copying to temporary
  directories.
* Significant speedups in `pip list --outdated`, by parallelizing
  network access. This is the first instance of parallel code within
  pip's codebase.
* A new `pip cache` command, which makes it possible to introspect and
  manage pip's cache directory.
* Better `pip freeze` for packages installed from direct URLs, enabled
  by the implementation of PEP 610.

We would be grateful for all the testing that users could do, to ensure
that when pip 20.1 is released, it's as solid as we can make it.

This release also contains an alpha version of pip's next generation
resolver. It is *off by default* because it
*unstable and notready for everyday use*.

As with all pip releases, a significant amount of the work was
contributed by pip's user community. Huge thanks to all who have
contributed, whether through code, documentation, issue reports and/or
discussion. Your help keeps pip improving, and is hugely appreciated.

Specific thanks go to Mozilla (through its Mozilla Open Source Support

Awards)  and to the Chan Zuckerberg
Initiative  DAF, an advised fund of
Silicon Valley Community Foundation, for their support that enabled the
work on the new resolver.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/5OEXP4XQQHFLA67Z6TXFZRKABV5TRNPL/

[Distutils] pip 20.0 release!

[Pradyun Gedam]

Hello everyone!

On behalf of the PyPA, I am pleased to announce that pip 20.0 has been
released.

This release contains quite a lot of changes (you can see the full list here
).
The highlights for this release are:

- Shorter filenames in output, when installing from PyPI.
- Automatic fall back to user installs, when main site-packages directory
is not
  writable and user installs are allowed.
- Significantly improved wheel building and wheel caching.
- Significantly reworked PEP 425 (binary compatibility tag) computation.
- Better handling of system packages in virtual environments created with
  `venv` (PEP 405).
- Older pip entry points have been reinstated temporarily, to aid users with
  stale wrapper scripts.

We expect that upgrading to this version of pip, would be painless for most
users. We recommend that users test the release before deploying in
production,
as with any other software release.

The pip development team is extremely grateful to everyone in the community
for
their contributions. Thanks to everyone who put so much effort into the new
release. Many of the contributions came from community members, whether in
the
form of code, participation in design discussions and/or bug reports.

Cheers,
Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/6335A5OXLP257KQVRCRAVE7DISNSRJZX/

[Distutils] Re: Packaging Summit 2020: Save the Date!

[Pradyun Gedam]

That opening definitely made me laugh. :)

Thanks for the date confirmation, and for picking up the responsibility for
putting this together Paul and Bernat!

Best,
Pradyun

On Wed, Jan 15, 2020 at 7:06 AM Paul Ganssle  wrote:

> Greetings one and all!
>
> Each year millions of eyes around the world eagerly wait at their computer
> with baited breath to learn the timing of the most fabulous, extravagant,
> decadent and hyperbolically oversold event of the year: The Python
> Packaging Summit. While most of the details aren't finalized (and in fact
> basically no details are finalized), in the interest of giving those who
> would like to attend a chance at some additional notice on their travel
> plans, we have established that the date of the summit: *Thursday, 16
> April 2020* (the day before the first PyCon talk day).
>
> For the past two years we've unofficially held our mini-summit on the
> first day of PyCon Sprints, but given that this summit is of particular
> interest to maintainers - many of whom would like to spend the first day of
> sprints actually sprinting - we have decided to run the summit on the
> tutorials day parallel to the Education Summit. We have applied for a spot
> in the Hatchery program, but even if we are not accepted as an official
> PyCon evenat, we will make arrangements to run the summit off-site on that
> day.
>
> For those unaware of the nature of the summit, you may want to pay
> attention to the "hyperbolically oversold" portion of my description less
> than the "decadent" and "extravagant" portions. The purpose of the summit
> is to get a bunch of people interested in Python Packaging together to
> discuss and coordinate on important topics in packaging and how we can move
> them forward. If you are on the fence about joining and want to know more,
> here are some detailed notes taken at the summit last year:
> https://docs.google.com/document/d/1Wz2-ECkicJgAmQDxMFivWmU2ZunKvPZ2UfQ59zDGj7g/edit#heading=h.7rw2gk16okic
>
> As I mentioned earlier, we have finalized essentially *no* details of
> this summit other than the date. I do not know how many people will be able
> to attend or by what mechanism we will prioritize attendance at the summit
> if there is limited space.
>
> In the coming months please keep an eye out for:
>
>- Registration details (if such a thing is deemed necessary)
>- A call for topic suggestions
>- Voting on how to prioritize topic suggestions
>
> Looking forward to seeing you all there!
> Paul
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/ZEZNY2MCUN3S3JGUFF6U6OWVEOKW2UAF/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/WNBWSX7AYRPGA4RUBEULAXYKF4JAXYUG/

[Distutils] Re: Keywords field in metadata: space separated or comma separated?

[Pradyun Gedam]

Resending, since the previous attempt was blocked by the mailing list.

On Tue, 19 Nov 2019 at 11:53 PM, Pradyun Gedam  wrote:

>
>
> On Tue, Nov 19, 2019 at 1:31 PM Thomas Kluyver 
> wrote:
>
>> There seems to be a loose agreement in favour of changing the spec to
>> match what distutils & setuptools do, so I've opened a PR against the spec:
>>
>> https://github.com/pypa/packaging.python.org/pull/674
>>
>
> The PR has been merged.
>
> Overall, this looks like a good idea to me, as might be clear from the
> fact that I approved the PR.
>
> --
> Pradyun
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/ID3EKBTQWF5NS35WRLH4NL5VETXA4EJ7/

[Distutils] Re: distutils.util.get_host_platform() and AIX

[Pradyun Gedam]

Resending since I messed up the reply.

On Sun, 18 Aug 2019 at 8:29 PM, Michael  wrote:

> Would be 'nice' if old meant 3.6 and earlier as I would like to target 3.7
> and later.
>

I understand we're talking about generating the PEP 425 tags here. If not,
please ignore this next paragraph.

The "algorithm" essentially needs to be a check for "is this an AIX python"
and also generate/provide a signal of what the system/binary compatibility
looks like. This algorithm would executed on a target Python interpreter
(which can be anything from Python 2.7, 3.3, 3.7 or a hypothetical 3.10),
to check if that interpreter compatible with whatever tag you plan to
generate.

Reiterating - continue only here, or expand and include python-dev?
>

Continue here.

python-dev has, practically, delegated Packaging discussions to this list.

(opinion) If you're okay with changing communication channel, I have a
strong preference for using discuss.python.org/c/packaging instead. Anyway,
most of the packaging-related discussions have been happening there lately.

Cheers,
Pradyun

--
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/KKNRPX2RH4WVK5ZEY4TOQPM24Y4CQOPP/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/W4B7W4QYI3HXS37K4CRZLKQ7WXZQGJIQ/

[Distutils] pip 19.2 is now available!

[Pradyun Gedam]

Hello everyone!

On behalf of the PyPA, I am pleased to announce that pip 19.2 has just
been released.

The highlights of this release are:

- Python 3.4 support has been dropped
- Support for "yanked" files on package indexes (PEP 592)
- keyring can now be used for managing credentials
- A new `pip debug` command, for helping debug installations
- Many bug fixes and lots of minor improvements

We've also documented how Python 2 support in pip, would be maintained
going forward. [1]

To install pip 19.2, you can use get-pip (as described in [2]) or run:

python -m pip install --upgrade pip

Note that if you are using a version of pip supplied by your distribution
vendor, vendor-supplied upgrades will be available in due course.

The pip development team is extremely grateful to everyone in the community
for their contributions. Thanks to everyone who put so much effort into the
new release. Many of the contributions came from community members, whether
in the form of code, participation in design discussions and/or bug reports.

[1]: https://pip.pypa.io/en/latest/development/release-process/#python-2-support
[2]: https://pip.pypa.io/en/latest/installing
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/CHZ45EAXGIOD3D4OJR62656FQOTQCCXJ/

[Distutils] Re: Seeking a new maintainer for packaging.python.org and Twine.

[Pradyun Gedam]

Thanks Thea for all of the work you've done!

Best,
Pradyun

On Sat, 20 Jul 2019 at 4:03 AM, Thea Flowers via Distutils-SIG <
distutils-sig@python.org> wrote:

> Hi friends!
> I'm stepping away from several things in the Python community. I've served
> as the maintainer of packaging.python.org and Twine for a while, but it's
> time for me to move on.
>
> I'm looking for someone to take on primary responsibility for
> packaging.python.org and another person to help share the load for Twine.
>
> If you or someone you know is interested, please reply to this thread and
> let me know.
>
> -Thea Flowers
> magicalg...@google.com
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/M7VRNT5KP4YQ6UPVI4MN4IIWM2Z3IXCH/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/AXGRBVO2GKBOCZJBSYSHGJNAOJDWNYBV/

[Distutils] Re: Seeking a new maintainer for packaging.python.org and Twine.

[Pradyun Gedam]

On Sat, 20 Jul 2019 at 4:19 AM, Thea Flowers via Distutils-SIG <
distutils-sig@python.org> wrote:

> Dustin,
> I've blessed you with the keys for packaging.python.org and Twine. You
> can pick others to help you out as needed.
>

Dustin,

If you want a co-maintainer for packaging.python.org, I'm happy to be that.

Best,
Pradyun

I'm gonna step away now and let Dustin make decisions from here on out. If
> there are any questions about how stuff works or a key I forgot to grant,
> feel free to let me know.
>
> On Fri, Jul 19, 2019 at 3:42 PM Bernat Gabor 
> wrote:
>
>> I'll be okay in chiming in on twine All the best Thea!
>>
>> On Fri, Jul 19, 2019, 23:38 Dustin Ingram  wrote:
>>
>>> Sorry to see you go, Thea! You've done a great job.
>>>
>>> I'd be happy to help maintain both these projects.
>>>
>>> On Fri, Jul 19, 2019 at 5:33 PM Thea Flowers via Distutils-SIG <
>>> distutils-sig@python.org> wrote:
>>>
 Hi friends!
 I'm stepping away from several things in the Python community. I've
 served as the maintainer of packaging.python.org and Twine for a
 while, but it's time for me to move on.

 I'm looking for someone to take on primary responsibility for
 packaging.python.org and another person to help share the load for
 Twine.

 If you or someone you know is interested, please reply to this thread
 and let me know.

 -Thea Flowers
 magicalg...@google.com
 --
 Distutils-SIG mailing list -- distutils-sig@python.org
 To unsubscribe send an email to distutils-sig-le...@python.org
 https://mail.python.org/mailman3/lists/distutils-sig.python.org/
 Message archived at
 https://mail.python.org/archives/list/distutils-sig@python.org/message/M7VRNT5KP4YQ6UPVI4MN4IIWM2Z3IXCH/

>>> --
>>> Distutils-SIG mailing list -- distutils-sig@python.org
>>> To unsubscribe send an email to distutils-sig-le...@python.org
>>> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/distutils-sig@python.org/message/GNEQW6JB44ED575NWD47SAFILOEFJY7A/
>>>
>> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/VURBPJMSB6WOEXJXSHILGL7JGKKV4ITQ/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/UTKYGCQ3ZCN7IUMKVKVN7WAQ7FXCVGPS/

[Distutils] Re: Why lockbot?

[Pradyun Gedam]

On Tue, 4 Jun 2019 at 1:00 AM, Brett Cannon  wrote:

> In the lock.yml you can specify to not leave a comment upon locking. See
> https://github.com/microsoft/vscode-python/blob/master/.github/lock.yml
> as an  example.
>

We did this yesterday.

Sorry for the email spam folks! I completely misjudged how long the bot
would take to catch up to the ridiculous amount of closed old issues on the
tracker, due to rate limits. If it's any comfort, I had to face the same
spam. 

If there are any concerns other than email spam, please comment on
https://github.com/pypa/pip/issues/6563 and not here.

Thanks,
Pradyun

>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/YN3DHLPXVH5RYLV2P4QJFSWHC6KKRGNH/

[Distutils] Re: question about claiming potentially abandoned namespaces on PyPI

[Pradyun Gedam]

On Sat, 4 May 2019 at 7:50 PM, Emma Tosch  wrote:

> Hi all,
>
> Apologies if this is not the proper channel for the following question.
>
> I have a project that I would like to post on PyPI that clashes with an
> existing project that I believe may be abandoned (
> https://pypi.org/project/toybox/#description). According to this PEP (
> https://www.python.org/dev/peps/pep-0541/) it appears that the
> appropriate course of action is to attempt to contact the owner of this
> namespace and find out if they will transfer ownership to me. In the
> section of the PEP that discusses prior art, there is a reference to cc'ing
> folks from npm when analogous issues arise over in JS land. It isn't clear
> to me from this PEP or its resolution what the proper procedure for
> pursuing ownership of a claimed, but possibly abandoned namespace is for
> PyPI. Is there an email address I should cc when contacting the namespace
> owner? How does one initiate an official request for an abandoned namespace?
>
> Thanks!
> Emma
>
> --
>
> Emma Tosch
> PhD Candidate, College of Information & Computer Sciences
> University of Massachusetts amhersteto...@cs.umass.edu 
> http://cs.umass.edu/~etosch
>
>
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/HSUNHFMJO34DM25FE6AEOKJKKVI6CFGI/


Hi Emma!

Thanks for reaching out. I suggest you file an issue at
https://github.com/pypa/warehouse for a PEP 541 request. That'll probably
be the best place.

Best,
Pradyun (on mobile, can't remove quote above, sorry!)
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/KVUQ23RU7JSFE4MGRY2PSB6SXULC3RN3/

[Distutils] Re: Quiet IRC channel?

[Pradyun Gedam]

On Fri, 26 Apr 2019 at 7:00 PM, Derek Kozel  wrote:

> Hello,
>
> I've been on the Distutils Freenode IRC channel listed on the Python.org
> website for a bit over two weeks now and several people have come by
> asking questions but it seems that the channel is disused as there's
> been no response from anyone. Is that true? If so, could someone change
> the channel message and maybe note that on the Python website?
> https://www.python.org/community/irc/
>
> I had joined to ask a question about using distutils to install a module
> on Ubuntu but having it go to site-packages rather than the OS specific
> dist-packages. I don't think this list is a good place to ask a question
> like that (but code and log below if anyone can help). Can you point me
> at a better place to ask questions?
>
> Code:
>
> https://github.com/dkozel/linux-gpib_debian/tree/upstream/linux-gpib-user/language/python
> Log: https://gist.github.com/dkozel/1c1ff3357ea71bf21c69e8beeaacfa78
>
> Thanks,
> Derek


Thanks for flagging this Derek!

Folks are mostly on #pypa and #pypa-dev now. #pypa would likely be the
correct place for asking questions about tooling.

Best,
Pradyun

>
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/distutils-sig@python.org/message/GYIJIYGSFSSHDM4TKQLB6NRIA5M2MJSU/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/4WXYD4ILPZKA4WGLFA677VUCMKWCKXHT/

[Distutils] PyCon US Packaging Mini Summit

[Pradyun Gedam]

A lot of people interested in Python Packaging will be in one place, at
PyCon US this year, and we're planning to utilize this opportunity to
discuss various topics and make progress on them in a Packaging Mini-Summit.

If you'll be at PyCon US 2019 and want to take part in this, please provide
some details about your PyCon attendance at [1].

The exact structure and flow of the Mini-Summit are still under discussion
at [2]. Though not finalized, it will likely include (at least) one open
space and one meeting, likely during the sprints.

Suggestions for topics to be discussed at the Mini-Summit are being
collected at [3], one per post. If you have any suggestions for topics that
should be discussed, please post them there.

[1]:
https://docs.google.com/spreadsheets/d/1DOcLSW2BlH-rebmPNnGIaOpfcEzKVFeu_BhP6ZvmUfE/edit?usp=sharing
[2]: https://discuss.python.org/t/pycon-us-packaging-mini-summit-2019/833
[3]:
https://discuss.python.org/t/packaging-mini-summit-pycon-us-2019-topic-suggestions/1534

Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/DCBTUFP6MLIGATRAFSZZDGQ6ZBS7MTDY/

[Distutils] Re: PEP 517 - source dir and sys.path

[Pradyun Gedam]

On Mon, 28 Jan 2019 at 2:35 AM, Thomas Kluyver  wrote:

> On Sun, Jan 27, 2019, at 7:47 PM, Paul Moore wrote:
> > What is more difficult is the question of whether the PEP should
> > change. As Chris pointed out, the previous discussion ended up saying
> > that the build directory should not be on sys.path, but acknowledged
> > that mandating that might cause issues. So the question now is, are
> > the issues we've seen big enough that we want to change PEP 517 to say
> > that the build directory *should* be on sys.path?
>
> I don't see that this issue is an argument for changing the PEP. It seems
> like a problem with the backend that should be supporting legacy use cases.
> The rule about the frontend not using the CWD to load the backend doesn't
> constrain what the backend does once loaded.
>
> From my perspective, the original arguments about the CWD still stand, but
> in addition PEP 517 is out now and there are tools implementing it, so it
> would be more disruptive to make changes. So at least for now, I'm -1 on
> changing the rule about the CWD.
>
> Thomas


Ditto.

--

Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/5ITEWSMK2ZEOAVEU2DIKAX5ZFFVAK7S6/

[Distutils] pip 19.0 is now available!

[Pradyun Gedam]

On behalf of the PyPA, I am pleased to announce that pip 19.0 has just
been released. This release brings support for multiple new packaging
standards.

The highlights of this release are:

- Python 3.4 support is deprecated.
- PEP 517 support is now implemented.
- manylinux2010 wheels are now supported.
- Dependency links support has been removed.
- Many bug fixes and lots of minor improvements.

In addition, when run on Python 2.7, pip 19.0 prints a warning regarding
the upcoming 2020 EOL of Python 2.7. A future version of pip will drop
support for Python 2.7.

To install pip 19.0, you can use get-pip (as described in [1]) or run::

python -m pip install --upgrade pip

Note that if you are using a version of pip supplied by your
distribution vendor, vendor-supplied upgrades will be available in due
course.

The pip development team is extremely grateful to everyone in the
community for their contributions. Thanks to everyone who put so much
effort into the new release. Many of the contributions came from
community members, whether in the form of code, participation in design
discussions and/or bug reports.

[1]: https://pip.pypa.io/en/latest/installing

Best,
Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/SJHO4TPFIB6NV4BRH22ABLWR472IKW55/

[Distutils] "Packaging and Python 2" on discuss.python.org

[Pradyun Gedam]

Hi all!

I've opened a topic on discuss.python.org for discussing how to handle the
Python 2.7 EOL in the various packaging tools, with the intent of figuring
out how we plan to handle this transition.

PyPA project maintainers: if you have already documented/announced your
plans, it would be great if you could link/mention it in that thread.

https://discuss.python.org/t/packaging-and-python-2/662

Best,
Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/RGYNRPUEDKG2MHXWLGVB7GY3HZ7NE4LT/

[Distutils] Re: Idea: perennial manylinux tag

[Pradyun Gedam]

On Fri, 30 Nov 2018 at 1:42 PM, Nathaniel Smith  wrote:

> April 2018: PEP 517 accepted
>

I think you got the wrong PEP number.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/ZPH3LLZ3BNDGZ4VIMAO4JNT2VO5E7FOX/

[Distutils] Re: Announcing Wheelodex

[Pradyun Gedam]

This is some nice work!
On Thu, Oct 18, 2018 at 9:39 PM John Thorvald Wodder II
 wrote:
>
> I've created a website that is likely to be of interest to people here: 
> Wheelodex , a site for browsing the metadata of 
> wheels on PyPI.
>
> It allows you to find out what projects a wheel depends on, what other 
> projects depend on a given project, what commands & other entry points a 
> wheel defines, what files are in a wheel, etc.  You can even search for 
> wheels containing a given module or file, or browse a list of all commands & 
> other entry points defined by wheels.  There's also a basic API for getting 
> wheel data as JSON: .
>
> I'm open to suggestions on what else to do with the data.  I'm also open to 
> suggestions on how to make the interface look less sucky.
>
> -- John Wodder
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/IHJU6RAUBFIEEDVR46BDT76F7XOHQ7OM/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/V76JHHMQWEIJSSFBSYRIUZ73AASKX2YA/

[Distutils] pip 18.1 has been released!

[Pradyun Gedam]

On behalf of the PyPA, I am pleased to announce that pip 18.1 has just
been released.

To install pip 18.1, you can run::

  python -m pip install --upgrade pip

or use get-pip as described in
https://pip.pypa.io/en/latest/installing. Note that
if you are using a version of pip supplied by your distribution
vendor, vendor-supplied
upgrades will be available in due course.

The highlights of this release are:

- Python 3.7 is now supported
- Dependency Links support is now scheduled for removal in pip 19.0
(the next pip
  release, scheduled in January 2019).
- Plaform specific options can now be used with the --target option,
to enable certain
  workflows.
- Much more helpful error messages on invalid configuration files
- Many bug fixes and minor improvements

Thanks to everyone who put so much effort into the new release. Many of the
contributions came from community members, whether in the form of code,
participation in design discussions and/or bug reports. The pip development
team is extremely grateful to everyone in the community for their contributions.

Thanks,
Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/YBYAYIXJ2WUUYCJLM7EWMQETJOW5W6ZZ/

[Distutils] Re: setuptools configuration in pyproject.toml

[Pradyun Gedam]

On Mon, 24 Sep 2018, 21:46 Daniel Holth,  wrote:

> You could probably implement this outside of setuptools as an extension.
> Clients would add a load-toml line to setup.py. Do build requirements work
> yet?
>

Yep. PEP 518 is supported by the latest version of pip (18.0).


> One obstacle might be reconciling the all-strings nature of .cfg with
> typed toml.
>
> On Mon, Sep 24, 2018, 12:06 RonnyPfannschmidt <
> opensou...@ronnypfannschmidt.de> wrote:
>
>> I believe contributions in the directions would be welcome,
>> for now having something like
>> tool.setuptools.{metadata,options,command.*} might be interesting to grow
>> and experiment with
>>
>> but whats really missing is a setuptools conributor with more than just
>> thinly stretched time.
>>
>> -- Ronny
>> Am 24.09.18 um 17:30 schrieb Bernat Gabor:
>>
>> I'm aware this might be a controversial subject, so let's have the
>> initial discussion about it here first for full disclosure and see what
>> people think about it. Should setuptools support pyproject.toml as
>> configuration source or not (alternative to setup.cfg which it already does
>> -
>> https://setuptools.readthedocs.io/en/latest/setuptools.html#configuring-setup-using-setup-cfg-files)?
>>
>>
>> The main benefit of having this would be to decrease configuration files
>> and have build dependencies and other types of dependencies in one
>> location. Furthermore many other packaging projects (flit, poetry) already
>> do define their dependencies inside pyproject.toml; so would create one
>> unified location where to look for such in the future.
>>
>> The counter-argument is that "a big part of pyproject.toml was keeping
>> that file clean" and would furthermore increase the size of that file down
>> the line.
>>
>> So what do people think? Should we encourage or discourage to have a
>> single python project file?
>>
>> I'm personally supporting build/code quality tools supporting
>> pyproject.toml as their main configuration file.
>>
>> Thanks
>>
>> --
>> Distutils-SIG mailing list -- distutils-sig@python.org
>> To unsubscribe send an email to 
>> distutils-sig-leave@python.orghttps://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
>> Message archived at 
>> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/C3JEBOCQEILLPXK4FDQPADCFO6WWW6JT/
>>
>> --
>> Distutils-SIG mailing list -- distutils-sig@python.org
>> To unsubscribe send an email to distutils-sig-le...@python.org
>> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
>> Message archived at
>> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/VBC35YTBWZYJCFPTRQRLSFBTV5FZV4OR/
>>
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/KV72IYDNA2M3WXHKWEV5RPX4UJ4OACA7/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/MSM2O4NJOZG6G2T3OWO6AYAGKYS2OFIE/

[Distutils] Re: New virtualenv maintainer

[Pradyun Gedam]

On Thu, 20 Sep 2018, 18:54 Donald Stufft,  wrote:

> We don’t typically bother to make big announcements on distutils-sig for
> maintainers of projects, however I’m making an exception in this case
> because of the recent threads on virtualenv maintenance. The current core
> team of virtualenv agrees that it is an important package, and deserves
> additional maintenance, but have somewhat stepped back from it in recent
> times for a variety of reasons (many ephemeral in nature).
>
> We’ve been happy to have a number of people step forward to offer help in
> maintaining the project, and after some thought and discussion, we’ve
> decided to extend an invitation of a commit bit to Bernat Gabor, and I’m
> happy to say that he has graciously accepted, and is now part of the team.
> For those unaware, Bernat is currently a developer for the tox project,
> which is one of the major consumers of virtualenv which made him a great
> fit for maintainership since it gives another real world perspective
> (consuming virtualenv as an API, instead of as a tool) that the current
> team currently lacked.
>
> So Welcome Bernat!
>

Welcome Bernat! Thanks for taking this up! :)

--
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/N2NNAUKBMIYPPNFAHZ4FS3KHR3ZIT3MV/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/4AVP7UVE7DSIUST7GPJJDIK2NS73P3Y6/

[Distutils] Re: pipenv and pip

[Pradyun Gedam]

Hey!

On Tue, Aug 21, 2018 at 7:51 PM Tzu-ping Chung  wrote:
>
> Speaking of Zazo[1], I actually found its abstraction extremely similar to 
> our current
>
> abstraction, ResolveLib[2]. I’m not sure it’s coincidence, we took 
> inspiration from
>
> similar sources, or this is simply the right way to do it, but how the logic 
> matches
>
> is quite surprising to me.

A quick skim at the abstractions of ResolveLib, it does abstract in
lot of the same
ways that I intended for zazo. It is a pleasant surprise to me too.

I have more thoughts on this but I'll avoid going into any more detail
here. I'll
be happy to have a chat with you elsewhere (off list, GH tracker etc)
on this and
collaborate here.

> I’m quite sure we would be able to find some way to consolidate efforts once 
> we
>
> find the chance, but for the moment, progress in ResolveLib (and by extension
>
> Pipenv) would likely benefit pip developing a proper resolver (be it Zazo or
>
> otherwise).
>
>
> TP
>
>
> [1]: https://github.com/pradyunsg/zazo
> [2]: https://github.com/sarugaku/resolvelib
>
>
>
> Sent from Mail for Windows 10
>
>
>
> From: Dan Ryan
> Sent: 21 August 2018 22:03
> To: Tzu-ping Chung
> Cc: Chris Jerdonek; distutils sig
> Subject: Re: [Distutils] Re: pipenv and pip
>
>
>
> There was a specific bug related to pipenv-only functionality which is why 
> the vcs ref is obtained. Pip now by default prefers ephemeral wheels, while 
> pipenv maintains a copy of editable repos currently. The work Tzu Ping has 
> been discussing and the work in pipenv currently are separate. You can also 
> not simply read some lines of pipenv and assume there should be a 1-1 
> functional equivalence. Sometimes we will have overlap and sometimes we will 
> have bugs. In the specific case you mentioned, we simply make sure to check 
> out whatever version is requested in the Pipfile before doing resolution and 
> handing off the results to pip for installation.  So while it may seem like 
> we are simply doing things over again that pip already handles, we have 
> different motivations and while we very likely have plenty of bugs, there is 
> more context than that we did something that pip also does.
>
>
>
> In any event, I’m not sure this mailing list is the right place to do code 
> reviews; we would happily accept any feedback if you make it on the project 
> itself.  Pipenv has a lot of cleanup work to do. We are in the process of 
> tackling that now. If you have ideas for how to tackle that, we would love to 
> hear them :)
>
>
> With regard to zazo — it’s been mentioned to us many times but we’ve been in 
> touch with Pradyun as I mentioned and he said he was very busy and couldn’t 
> coordinate efforts at all on the resolver front.

While I didn't really say that, I understand you might have been
interpreted what I
said as that. None the less, I'll be happy to coordinate on this front
moving forward. :)

> Zazo isn’t actually an implementation, it’s an abstraction layer. We built a 
> directed graph and layered a resolver on top of it.

zazo is intended to be something along those lines, an abstraction that allows
multiple libraries to interface with it, while providing a single well
tested core provided
the resolution capabilities. This separated the resolution logic from
the rest of the
codebase pretty well, which was a concern since pip didn't really do
that (at all)
earlier. Cleaning up the technical debt that has piled up over the
years in pip is
something I've been doing in the time I'm have had to work on pip.

> Since this is a primary functionality pipenv has always wanted to offer (as 
> far as my understanding went) and has always basically failed at because of 
> various tooling issues, we have been working for the last 4-8 weeks pretty 
> much in seclusion trying to tackle this.

That's the sort of time I've not really had to work on zazo since
after GSoC, for
various non-technical reasons. I'm glad someone has made the time to
look into this.
I'll be happy to help any way I can.

Pradyun

>
> Dan Ryan // pipenv maintainer
>
> gh: @techalchemy
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/UH6KMBA4FOVQEHHAWGHWKRORICY722FH/

[Distutils] Re: Make an ordered list of sdists to be installed?

[Pradyun Gedam]

On Mon, Jul 23, 2018 at 3:19 PM Thomas Kluyver  wrote:
>
> Hi all,
>
> Do we know of any tool that can, given the name of one or more packages, 
> follow dependency chains and produce a list of packages in the order they 
> need to be installed, assuming every package needed will be built from source?
>
> Running "pip download --no-binary :all: ipython" gets me a set of sdists to 
> be installed, but I lose any information about the order. I assume some 
> packages will fail to build if their dependencies are not installed first, so 
> the order is significant.
>
> Pip appears to keep track of the ordering internally: if I run "pip install 
> --no-binary :all: ipython", all the dependencies are downloaded, and then the 
> collected packages are installed starting from those with no dependencies and 
> finishing with the package I requested. But I don't know of any way to get 
> this information out of pip. Is there an option that I'm overlooking? Or some 
> other tool that can do this?
>
> The use case I'm thinking about is to automatically generate instructions for 
> a build system which separates the downloading and installing steps, so for 
> each step it expects one or more URLs to download, along with instructions 
> for how to install that piece. The installation steps shouldn't download 
> further data. I could work around the issue by telling it to download all the 
> sdists in a single step and then install in one shot with --no-index and 
> --find-links. But it's more elegant - and better for caching - if we can 
> install each package as a single step.
>
> Thanks,
> Thomas
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/LGTH3IYBMVKBS4PYGFJ6A7N5GW5ZKFUY/

Hey!

This is actually related to a really old feature request on pip. [1]
Essentially, the request is to allow the user to get the order and
versions of packages that pip would choose to install. Some of the
preparatory refactoring that's been done for bringing in a newer
implementation of the dependency resolver to pip [2] means that this
should be much easier to implement today than it would have been in
the past.

I should point out that these don't include build-dependencies for PEP 518.

Cheers,
Pradyun

[1]: https://github.com/pypa/pip/issues/53
[2]: https://github.com/pradyunsg/zazo
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/MJL5LSSHNFV76UJHS372EHZ72IC7CPFS/

[Distutils] pip 18.0 has been released!

[Pradyun Gedam]

On behalf of the PyPA, I am pleased to announce that pip 18.0 has just
been released.

To install pip 18.0, you can run

python -m pip install --upgrade pip

or use get-pip as described in https://pip.pypa.io/en/latest/installing.
Note that if you are using a version of pip supplied by your
distribution vendor, vendor-supplied upgrades will be available in due
course (or you can use pip 18 in a virtual environment).

This is the first pip release since adopting 3 month release cadence and
a Calendar based versioning scheme (also known as CalVer). In simpler
words, there will be a new pip release every 3 months unless there are
no changes since the previous release. More details such as release
months can be found in pip's development documentation.

The highlights of this release are:

- Python 3.3 is no longer supported - if you need pip on Python 3.3,
  you should stay on pip 10, which is the last version to support Python
  3.3.
- Complete PEP 518 support - includes support for installation of build
  dependencies from source and Unicode support on Python 2 and Windows
- New --prefer-binary flag, to prefer older wheels over newer sdists
- Many bug fixes and minor improvements

Thanks to everyone who put so much effort into the new release. Many of
the contributions came from community members, whether in the form of
code, participation in design discussions and/or bug reports. The pip
development team is extremely grateful to everyone in the community for
their contributions.

Thanks,
Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/AQBMJKBON6WJ5S53WTHLZN7LXLP43L2Y/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Tue, 17 Jul 2018, 20:44 Donald Stufft,  wrote:

>
> On Jul 17, 2018, at 5:27 AM, Paul Moore  wrote:
>
> There's also a PR cost, in that projects who have enthusiastically
> adopted pyproject,toml as being a nice "common configuration"
> location, will be left feeling that maybe that wasn't such a good
> decision if it's causing them problems like this. Plus, projects like
> towncrier will need to update their docs (yes, that's a bit ideal
> world - the reality is that people simply keep doing what they do now,
> and we get the message out via a gradual process of addressing bug
> reports and providing the explanation there).
>
>
> I think that this is the most important reason not to go too crazy here. We
> should think of breaking changes as having a limited budget for introducing
> breaking changes, and the question then becomes where do we want to spend
> our
> budget? While something being relatively new means that the cost to our
> budget
> is smaller, it still has a cost and I just don’t think this is a great
> place to
> spend our budget at.
>
> On top of that, I don’t think the end result is significantly or
> meaningfully
> better for the end users, particularly if we ever get to a world where we
> isolate everything by default (which would mean that we would have to have
> an
> implicit requires of setuptools/wheel anyways).
>

Fair enough. ​I just wanted to
​make the build-system table to be more visible to end
users​, since I feel the cost here is justified by the benefit
​s​
. If others don't agree,
that's cool. I don't feel super strongly about this tbh
​. I am not opposed to keeping
the​ ​status quo on this matter​ and am on board with Paul's suggestion of
making the
build-system table optional. :)

​Pradyun​
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/TGCPWPHBP34R7CKDNVAHRQTZUCZNXT5Z/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Tue, 17 Jul 2018, 09:11 Brett Cannon,  wrote:

> My point is I don't think enough pieces are in place to force everyone
> over to --no-isolation to get by if they don't set up pyproject.toml as
> you're suggesting they be required to. IOW I think you're advocating
> something that's going to alienate people by pushing too hard.
>

Ah. I now understand what you meant.

I don't think the transition costs are very high here (more on this below). I
also think most projects who've adopted pyproject.toml won't mind adding 2
lines to it, given enough time.

(pip-specific deprecation details follow; I'm not sure how to explain the
transition related costs in any other way)

Since this would be a behavior change in pip, this would have to go through
the standard deprecation cycle which would probably be about as long as the
feature has been available currently. This means projects would have time
to make releases with the key, that's similar to the amount of time the
feature has been available as a part of pip.

A functional difference here is that older releases with such
pyproject.toml files would need an escape hatch after the change is made,
to be installable. I don't think this is a major issue since when we'd flip
the switch to start refusing, we'd start suggesting users to use the escape
hatch or upgrade to a newer version, since the deprecation would make
projects add the key and make newer releases. And, --no-isolation can serve
as the escape hatch too.

Pradyun


> On Mon, Jul 16, 2018, 19:30 Pradyun Gedam,  wrote:
>
>>
>>
>> On Tue, 17 Jul 2018, 00:45 Brett Cannon,  wrote:
>>
>>>
>>>
>>> On Mon, 16 Jul 2018 at 10:47 Pradyun Gedam  wrote:
>>>
>>>>
>>>>
>>>> On Mon, 16 Jul 2018, 21:11 Paul Moore,  wrote:
>>>>
>>>>> On 16 July 2018 at 15:56, Pradyun Gedam  wrote:
>>>>> >
>>>>> > On Mon, 16 Jul 2018, 20:16 Brett Cannon,  wrote:
>>>>> >>
>>>>> >> I will update the PEP and add you as a reviewer, Paul (might not
>>>>> get to it
>>>>> >> until Friday, though).
>>>>> >>
>>>>> >> On Mon, Jul 16, 2018, 02:23 Paul Moore, 
>>>>> wrote:
>>>>> >>>
>>>>> >>> The discussion on this appears to have died down.
>>>>> >>>
>>>>> >>> As far as I can tell, the consensus is essentially:
>>>>> >>>
>>>>> >>> 1. It should be legal for pyproject.toml to *not* contain a
>>>>> >>> [build-system] section.
>>>>> >>> 2. If a [build-system] section is present, the requires key must be
>>>>> >>> present.
>>>>> >>>
>>>>> >>> Tools should behave as follows:
>>>>> >>>
>>>>> >>> 1. If [build-system] is present but requires is missing, raise an
>>>>> error.
>>>>> >>> 2. If [build-system] is missing, they can take one of the following
>>>>> >>> two approaches:
>>>>> >>>a) Act as if pyproject.toml is missing altogether
>>>>> >>>b) Act as if [build-system] is present, with a requires value of
>>>>> >>> ["setuptools", "wheel"]
>>>>> >>>
>>>>> >>> Whether tools act differently in cases 2a and 2b is tool-dependent
>>>>> >>> (for pip, we would isolate in case 2b but not in case 2a) which is
>>>>> why
>>>>> >>> the choice is left to individual tools. That makes the
>>>>> >>> "Thomas/Nathaniel" debate into a tool implementation choice, and
>>>>> both
>>>>> >>> of the options are allowable from the perspective of the PEP.
>>>>> >>>
>>>>> >>> Is everyone OK with this resolution? If so, will someone raise a PR
>>>>> >>> for PEP 518? I can do that if no-one else can.
>>>>> >
>>>>> > While I don't mind if we'd go ahead with this (and 2b, to not change
>>>>> > existing behavior), I still think making the key mandatory would be a
>>>>> > good/better idea.
>>>>>
>>>>> I get the feeling that no-one is sufficiently motivated to argue
>>>>> strongly for any particular resolution - which is why my summary is
>>>>> basically "accept the current behaviour as correct" :-)
>>>>&g

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Tue, 17 Jul 2018, 00:45 Brett Cannon,  wrote:

>
>
> On Mon, 16 Jul 2018 at 10:47 Pradyun Gedam  wrote:
>
>>
>>
>> On Mon, 16 Jul 2018, 21:11 Paul Moore,  wrote:
>>
>>> On 16 July 2018 at 15:56, Pradyun Gedam  wrote:
>>> >
>>> > On Mon, 16 Jul 2018, 20:16 Brett Cannon,  wrote:
>>> >>
>>> >> I will update the PEP and add you as a reviewer, Paul (might not get
>>> to it
>>> >> until Friday, though).
>>> >>
>>> >> On Mon, Jul 16, 2018, 02:23 Paul Moore,  wrote:
>>> >>>
>>> >>> The discussion on this appears to have died down.
>>> >>>
>>> >>> As far as I can tell, the consensus is essentially:
>>> >>>
>>> >>> 1. It should be legal for pyproject.toml to *not* contain a
>>> >>> [build-system] section.
>>> >>> 2. If a [build-system] section is present, the requires key must be
>>> >>> present.
>>> >>>
>>> >>> Tools should behave as follows:
>>> >>>
>>> >>> 1. If [build-system] is present but requires is missing, raise an
>>> error.
>>> >>> 2. If [build-system] is missing, they can take one of the following
>>> >>> two approaches:
>>> >>>a) Act as if pyproject.toml is missing altogether
>>> >>>b) Act as if [build-system] is present, with a requires value of
>>> >>> ["setuptools", "wheel"]
>>> >>>
>>> >>> Whether tools act differently in cases 2a and 2b is tool-dependent
>>> >>> (for pip, we would isolate in case 2b but not in case 2a) which is
>>> why
>>> >>> the choice is left to individual tools. That makes the
>>> >>> "Thomas/Nathaniel" debate into a tool implementation choice, and both
>>> >>> of the options are allowable from the perspective of the PEP.
>>> >>>
>>> >>> Is everyone OK with this resolution? If so, will someone raise a PR
>>> >>> for PEP 518? I can do that if no-one else can.
>>> >
>>> > While I don't mind if we'd go ahead with this (and 2b, to not change
>>> > existing behavior), I still think making the key mandatory would be a
>>> > good/better idea.
>>>
>>> I get the feeling that no-one is sufficiently motivated to argue
>>> strongly for any particular resolution - which is why my summary is
>>> basically "accept the current behaviour as correct" :-)
>>>
>>
>> I just perceive this as a case of - there's a nicer and arguably, cleaner
>> option and there's not much cost to getting there. Let's do that. :)
>>
>> I don't mind the status quo though.
>>
>>
>>> > The visibility (and hence familiarity) that this would gain from the
>>> > specification of build-system.requires being mandatory would be nice to
>>> > have. I feel that this would otherwise be a missed opportunity to push
>>> for
>>> > the key and standard to be more visible to users and packagers.
>>>
>>> But adoption of pyproject.toml as a standard configuration file by
>>> projects like towncrier and black is doing that already.
>>>
>>
>> I'm referring to familiarity with the build-system table (through
>> build-system.requires being mandatory). If you have to add that table/key
>> when using those tools, people who work on those projects get to know about
>> the table.
>>
>>
>>> > The transitory cost for existing packages that break due to this being
>>> > mandatory would probably be fairly minor (I realize that this point is
>>> where
>>> > I dwelled into implementation details earlier, so I'll not digress
>>> there
>>> > now).
>>>
>>> I believe the transition costs for projects (and their users) using
>>> pyproject.toml for config only are:
>>>
>>> Optional, only isolate if [build-system] is present: None
>>> Optional, isolate if pyproject.toml is present: The few *end users*
>>> for whom isolation breaks the build have to pass --no-build-isolation
>>> Mandatory: The *project* has to add the [build-system] section, and
>>> make a new release. End users have to wait for that release.
>>>
>>
>> Not really. With mandatory, you can use --no-isolation - it would still
>> work fine for basically every 

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Mon, 16 Jul 2018, 21:11 Paul Moore,  wrote:

> On 16 July 2018 at 15:56, Pradyun Gedam  wrote:
> >
> > On Mon, 16 Jul 2018, 20:16 Brett Cannon,  wrote:
> >>
> >> I will update the PEP and add you as a reviewer, Paul (might not get to
> it
> >> until Friday, though).
> >>
> >> On Mon, Jul 16, 2018, 02:23 Paul Moore,  wrote:
> >>>
> >>> The discussion on this appears to have died down.
> >>>
> >>> As far as I can tell, the consensus is essentially:
> >>>
> >>> 1. It should be legal for pyproject.toml to *not* contain a
> >>> [build-system] section.
> >>> 2. If a [build-system] section is present, the requires key must be
> >>> present.
> >>>
> >>> Tools should behave as follows:
> >>>
> >>> 1. If [build-system] is present but requires is missing, raise an
> error.
> >>> 2. If [build-system] is missing, they can take one of the following
> >>> two approaches:
> >>>a) Act as if pyproject.toml is missing altogether
> >>>b) Act as if [build-system] is present, with a requires value of
> >>> ["setuptools", "wheel"]
> >>>
> >>> Whether tools act differently in cases 2a and 2b is tool-dependent
> >>> (for pip, we would isolate in case 2b but not in case 2a) which is why
> >>> the choice is left to individual tools. That makes the
> >>> "Thomas/Nathaniel" debate into a tool implementation choice, and both
> >>> of the options are allowable from the perspective of the PEP.
> >>>
> >>> Is everyone OK with this resolution? If so, will someone raise a PR
> >>> for PEP 518? I can do that if no-one else can.
> >
> > While I don't mind if we'd go ahead with this (and 2b, to not change
> > existing behavior), I still think making the key mandatory would be a
> > good/better idea.
>
> I get the feeling that no-one is sufficiently motivated to argue
> strongly for any particular resolution - which is why my summary is
> basically "accept the current behaviour as correct" :-)
>

I just perceive this as a case of - there's a nicer and arguably, cleaner
option and there's not much cost to getting there. Let's do that. :)

I don't mind the status quo though.


> > The visibility (and hence familiarity) that this would gain from the
> > specification of build-system.requires being mandatory would be nice to
> > have. I feel that this would otherwise be a missed opportunity to push
> for
> > the key and standard to be more visible to users and packagers.
>
> But adoption of pyproject.toml as a standard configuration file by
> projects like towncrier and black is doing that already.
>

I'm referring to familiarity with the build-system table (through
build-system.requires being mandatory). If you have to add that table/key
when using those tools, people who work on those projects get to know about
the table.


> > The transitory cost for existing packages that break due to this being
> > mandatory would probably be fairly minor (I realize that this point is
> where
> > I dwelled into implementation details earlier, so I'll not digress there
> > now).
>
> I believe the transition costs for projects (and their users) using
> pyproject.toml for config only are:
>
> Optional, only isolate if [build-system] is present: None
> Optional, isolate if pyproject.toml is present: The few *end users*
> for whom isolation breaks the build have to pass --no-build-isolation
> Mandatory: The *project* has to add the [build-system] section, and
> make a new release. End users have to wait for that release.
>

Not really. With mandatory, you can use --no-isolation - it would still
work fine for basically every user that has setuptools and wheel installed
in their current environment (that's almost every environment I'd say).


> If I've got that right, the costs for making the key mandatory are a
> lot higher.


My intuition/feeling here is that this cost is worth the network effects
that would make more users familiar with them.

Pradyun

The costs for 2b are there, but a lot smaller (and the end
> user has a workaround). Option 2a has no costs, but also misses the
> opportunity to move towards build isolation by default for a few extra
> cases.
>
> Paul
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/YJ22D4JIGXLNFCZXCCT22NR67OBRW6F5/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Mon, 16 Jul 2018, 20:16 Brett Cannon,  wrote:

> I will update the PEP and add you as a reviewer, Paul (might not get to it
> until Friday, though).
>
> On Mon, Jul 16, 2018, 02:23 Paul Moore,  wrote:
>
>> The discussion on this appears to have died down.
>>
>> As far as I can tell, the consensus is essentially:
>>
>> 1. It should be legal for pyproject.toml to *not* contain a
>> [build-system] section.
>> 2. If a [build-system] section is present, the requires key must be
>> present.
>>
>> Tools should behave as follows:
>>
>> 1. If [build-system] is present but requires is missing, raise an error.
>> 2. If [build-system] is missing, they can take one of the following
>> two approaches:
>>a) Act as if pyproject.toml is missing altogether
>>b) Act as if [build-system] is present, with a requires value of
>> ["setuptools", "wheel"]
>>
>> Whether tools act differently in cases 2a and 2b is tool-dependent
>> (for pip, we would isolate in case 2b but not in case 2a) which is why
>> the choice is left to individual tools. That makes the
>> "Thomas/Nathaniel" debate into a tool implementation choice, and both
>> of the options are allowable from the perspective of the PEP.
>>
>> Is everyone OK with this resolution? If so, will someone raise a PR
>> for PEP 518? I can do that if no-one else can.
>>
>
While I don't mind if we'd go ahead with this (and 2b, to not change
existing behavior), I still think making the key mandatory would be a
good/better idea.

The visibility (and hence familiarity) that this would gain from the
specification of build-system.requires being mandatory would be nice to
have. I feel that this would otherwise be a missed opportunity to push for
the key and standard to be more visible to users and packagers.

The transitory cost for existing packages that break due to this being
mandatory would probably be fairly minor (I realize that this point is
where I dwelled into implementation details earlier, so I'll not digress
there now).

Pradyun


>> Paul
>>
>> PS Following on from
>>
>> >> I think it would be helpful for this discussion if we could look at
>> these
>> >> bug reports – do does anyone have links?
>> >
>> > Good point. I will hunt them out and post here.
>>
>> I mentioned this on the pip issue, but the only pip problem which has
>> been raised with the current behaviour is around cases where the user
>> disabled PyPI access and doesn't have a local copy of
>> setuptools/wheel, which means we can't build the isolated environment.
>> But that's a corner case that is easily resolvable, and I don't think
>> it needs to affect pip's choice of behaviour (much less what the PEP
>> says).
>>
>>
>> On 10 July 2018 at 08:03, Paul Moore  wrote:
>> > On 10 July 2018 at 05:09, Pradyun Gedam  wrote:
>> >> On Tue, Jul 10, 2018 at 2:30 AM Paul Moore 
>> wrote:
>> >>> For now I'll point out that PEP 518 doesn't say *anything* about how
>> >>> tools use the information in `pyproject.toml` - there's no mention of
>> >>> build isolation. Unless I missed something - please point it out if I
>> >>> did, The only thing I can find is in PEP 517. So discussions of pip's
>> >>> isolation behaviour are mostly pip-specific implementation details at
>> >>> the moment, and not really relevant to this thread.
>> >>
>> >> Ah, okay. So, isolation is purely an implementation issue, so it
>> doesn't
>> >> need to come around in this discussion which is about how we should
>> >> change the PEP. I guess I'm still figuring out where to draw the line
>> >> between implementation details and the PEP details here since they
>> >> should/would influence each other both ways. I'll try to be more
>> careful
>> >> about stuff like this in the future. :)
>> >
>> > Not a problem. Isolation is discussed in PEP 517, which is why I was
>> > getting confused about what was related to the standard and what to
>> > the implementation. We will need to be careful to review all this once
>> > we start implementing PEP 517, but for now at least that's a level of
>> > complexity we don't need in this discussion.
>> >
>> > Paul
>>
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/BERTMHYYQVYW36PKSER5H3TBUDSNMESW/

[Distutils]Re: pypi/twine complains about license

[Pradyun Gedam]

On Thu, 12 Jul 2018, 13:06 Thomas Kluyver,  wrote:

> On Thu, Jul 12, 2018, at 8:19 AM, Robin Becker wrote:
> > The previous
> > answer says there is a list so I can use that.
>
> I don't think anyone linked to it yet, so:
> https://pypi.org/pypi?%3Aaction=list_classifiers


https://pypi.org/classifiers is a newer nicer home for them now.


> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/SYCAFU7SQSNGDLCLDGDPEBA4MYSKIALV/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/ZOXIVAYO5NXN67MGMY2XEJVWFWVKB3PC/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Tue, Jul 10, 2018 at 2:30 AM Paul Moore  wrote:
>
> On 9 July 2018 at 20:27, Pradyun Gedam  wrote:
> > On Sun, Jul 8, 2018 at 9:14 AM Nathaniel Smith  wrote:
> >> This suggests that our decision should be based on: if we want to be
> >> relatively more aggressive about rolling out build isolation, then we
> >> should key on the existence of pyproject.toml. If we want to be
> >> relatively more conservative, then we should key on the existence of
> >> build-system.requires.
> >
> > Indeed. I too feel that's what this comes down to. I had a wordier way to
> > come to this conclusion which I've removed from this mail now. :)
> >
> > An important thing here is that being aggressive here let's us piggy-back on
> > the adoption of tools that use pyproject.toml and I'd say it's a good thing
> > to have more people using the standard explicitly.
>
> OK. That's basically pip's current behaviour.
>
> This discussion is getting fragmented, unfortunately. I've just
> commented on the pip issue at
> https://github.com/pypa/pip/issues/5416#issuecomment-403616780 as I'm
> still trying to find the motivating issues behind this discussion.
>
> For now I'll point out that PEP 518 doesn't say *anything* about how
> tools use the information in `pyproject.toml` - there's no mention of
> build isolation. Unless I missed something - please point it out if I
> did, The only thing I can find is in PEP 517. So discussions of pip's
> isolation behaviour are mostly pip-specific implementation details at
> the moment, and not really relevant to this thread.

Ah, okay. So, isolation is purely an implementation issue, so it doesn't
need to come around in this discussion which is about how we should
change the PEP. I guess I'm still figuring out where to draw the line
between implementation details and the PEP details here since they
should/would influence each other both ways. I'll try to be more careful
about stuff like this in the future. :)

Pradyun

> Once I've collected a bit more information, I'll summarise here. But I
> think there's only minor changes to PEP 518 needed, and nothing for
> pip.
>
> Paul
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/CJG4MCEGS5G2ZBZ6525FKC7LN67VGB3B/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

(I thought I'd sent this but it was in my drafts instead)

On Sun, Jul 8, 2018 at 9:14 AM Nathaniel Smith  wrote:
>
> On Sat, Jul 7, 2018 at 3:41 AM, Paul Moore  wrote:
> > On 30 June 2018 at 06:33, Nick Coghlan  wrote:
> >> On 28 June 2018 at 11:37, Nathaniel Smith  wrote:
> >>> So my inclination is to plan on ending up with build-system.requires
> >>> defaulting to ["setuptools", "wheel"], and build-system.backend
> >>> defaulting to "setuptools". Hopefully we'll eventually get to a place
> >>> where ~no-one uses these defaults, but carrying around the code to
> >>> handle the defaults isn't really a burden.
> >>
> >> While I was going to post to say I liked this approach, after a bit of
> >> reflection, I realised I prefer Thomas Kluyver's suggestion: instead
> >> of taking "pyproject.toml" as indicating a build-isolation compatible
> >> sdist file, instead make "pyproject.toml with a build-system table"
> >> the marker for that case.
> >
> > As far as I can see, the only difference this makes is that it means
> > pip retains the legacy (non-isolated) behaviour in a few more places
> > (specifically places where it's quite likley the project hasn't
> > thought about build isolation). So it's basically a slightly more
> > forgiving version of Nathaniel's proposal.
> >
> > The part of Nathaniel's approach that I think would be most confusing
> > is a project that currently uses setup_requires which adds a
> > pyproject.toml for (say) towncrier. The build would become isolated,
> > but setup_requires (which is implemented by setuptools, not pip) would
> > ignore the isolated environment and install in the wrong place (maybe?
> > I honestly don't know). I'm quite happy to call this deprecated
> > behaviour and point out that the project should switch to explicitly
> > using PEP 518, but given that this whole discussion is because people
> > haven't done that, I suspect Nathaniel's proposal doesn't actually
> > solve the root issue here...
>
> Re: the interaction of build isolation and setup_requires: it looks
> like this is totally fine, actually. Based on some experiments +
> checking the docs, it appears that setup_requires has always done some
> magic where it doesn't actually try to install the requested packages
> into the current environment, but instead drops them inside the build
> directory and then uses some import system tricks to "overlay" them
> onto the current process's python path. So build isolation +
> setup_requires actually work very well together.
>
> I think in the long run, we want to enable build isolation everywhere.
> Packages that break when installed with build isolation are already
> broken when running 'pip install' in a fresh virtualenv. There
> probably are a few of these out there still that say things like
> "before installing this package, please install these other packages,
> as a separate call to pip", but it's been a long time now since I've
> seen one of those. And since they're already asking users to follow
> some finicky manual install procedure, requiring --no-build-isolation
> isn't a big deal.
>
> So, I don't care that much about what we use to trigger build
> isolation mode, because it's only a temporary thing anyway. The value
> of keying off something involving pyproject.toml is that it
> automatically gives us a kind of soft rollout: people adopting
> pyproject.toml are probably more willing to put up with issues with
> new packaging features, so we can hopefully shake out any problems
> before it becomes the standard.
>
> This suggests that our decision should be based on: if we want to be
> relatively more aggressive about rolling out build isolation, then we
> should key on the existence of pyproject.toml. If we want to be
> relatively more conservative, then we should key on the existence of
> build-system.requires.

Indeed. I too feel that's what this comes down to. I had a wordier way to
come to this conclusion which I've removed from this mail now. :)

An important thing here is that being aggressive here let's us piggy-back
on the adoption of tools that use pyproject.toml and I'd say it's a good
thing to have more people using the standard explicitly.

Having to specify the build-system.requires in pyproject.toml isn't going
to be a problem for most (almost all?) packages. The status quo is that
*they are using isolation* if there's a pyproject.toml. It won't
break/change anything if they do specify the key in a future release within
the pyproject.toml that's already there. In fact, requiring packages
specify this information would serve to make people more familiar with it.

FWIW, if build isolation is causing issues for a package, they'll have the
same UX as today (passing --no-build-isolation) if we go ahead with
compulsory requires.

As for the existing released packages with missing requires, we'll have an
escape hatch (the same --no-build-isolation or something more targeted if
we want that'd be an implementation thing) that'll work just fine 

[Distutils]Re: Provisional PEPs can now actually be marked as Provisional

[Pradyun Gedam]

On Sun, 8 Jul 2018, 13:03 Nick Coghlan,  wrote:

> Hi folks,
>
> Some time ago, we adjusted the distutils-sig PEP approval process to
> include a "Provisional" status, where we approved changes for
> inclusion in the reference implementations (typically pip and
> setuptools), but still reserved the right to make adjustments if
> practical experience showed that we'd missed something critical.
>
> Earlier today, I merged the amendment to PEP 1 that makes that part of
> the Python Enhancement Proposal process in general, and explicitly
> marked the affected PEPs as Provisional.
>
> For distutils-sig, that update affects both of the pyproject.toml PEPs
> (although I expect PEP 518 isn't too far off being marked as Final -
> the question of the precise meaning of a missing build-system table
> when pyproject.toml is present seems to be the only underspecified
> point that anyone has found during the Provisional period).
>

Should we be marking PEP 517 as provisional?


> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/SZDYU3VKNS4RYH3JPEYVXYSDT4YCQSQD/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/3AAWQVM7BBKVWGHOH23N5FTE43P6GC6H/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Sat, 7 Jul 2018, 16:12 Paul Moore,  wrote:

> On 30 June 2018 at 06:33, Nick Coghlan  wrote:
> > On 28 June 2018 at 11:37, Nathaniel Smith  wrote:
> >> So my inclination is to plan on ending up with build-system.requires
> >> defaulting to ["setuptools", "wheel"], and build-system.backend
> >> defaulting to "setuptools". Hopefully we'll eventually get to a place
> >> where ~no-one uses these defaults, but carrying around the code to
> >> handle the defaults isn't really a burden.
> >
> > While I was going to post to say I liked this approach, after a bit of
> > reflection, I realised I prefer Thomas Kluyver's suggestion: instead
> > of taking "pyproject.toml" as indicating a build-isolation compatible
> > sdist file, instead make "pyproject.toml with a build-system table"
> > the marker for that case.
>
> As far as I can see, the only difference this makes is that it means
> pip retains the legacy (non-isolated) behaviour in a few more places
> (specifically places where it's quite likley the project hasn't
> thought about build isolation). So it's basically a slightly more
> forgiving version of Nathaniel's proposal.
>
> The part of Nathaniel's approach that I think would be most confusing
> is a project that currently uses setup_requires which adds a
> pyproject.toml for (say) towncrier. The build would become isolated,
> but setup_requires (which is implemented by setuptools, not pip) would
> ignore the isolated environment and install in the wrong place (maybe?
> I honestly don't know). I'm quite happy to call this deprecated
> behaviour and point out that the project should switch to explicitly
> using PEP 518, but given that this whole discussion is because people
> haven't done that, I suspect Nathaniel's proposal doesn't actually
> solve the root issue here...
>
> > If you don't have a build-system table at all, then you'll continue to
> > get the legacy sdist handling, allowing the addition of other tool
> > config without impacting the way your sdist gets built.
> >
> > If you do add a build-system table, then you have to populates the
> > "requires" field properly, even if you're using setuptools as your
> > build backend.
> >
> > That way, the "build-system.backend defaults to setuptools" behaviour
> > is only there to support pyproject.toml files that have already opted
> > in to build isolation by writing:
> >
> > [build-system]
> > requires = ["setuptools", "wheel"]
> >
>
> That sounds fair to me. In terms of PEP wording:
>
> 1. build-system.requires becomes *optional* in pyproject.toml
> 2. Tools should process projects without pyproject.toml in the same
> way as they always have (backward compatibility). For pip, that means
> no build isolation, and the old-style processing path.
> 3. Tools should treat projects with pyproject.toml, but with *no*
> build-system.requires key the same way as (2).
> 4. Tools can assume that no legacy behaviour is needed for projects
> that specify pyproject.toml and build-system.requires.
>
> Moving forward to PEP 517, we'd allow a default for
> build-system.backend purely as a convenience because PEP 518 was
> implemented before PEP 517 - but there's no intention or commitment to
> retain *current* PEP 518 code paths once PEP 517 is implemented (i.e,
> nobody's suggesting that `build-system.backend missing` should *ever*
> be different from `build-system.backend = "setuptools"`).
>
> Any objections? Specifically Brett made the point that this means that

as a community we're OK with pyproject.toml being the standard
> location for tool configuration, and not just for specifying build
> tools. I guess I'm personally OK with this (although I do feel that
> it's something we didn't fully talk through when writing the PEP, and
> we're now getting pushed down this route by circumstance). It might
> warrant a change to the PEP title, just to clarify the modified
> intent.
>
> Nathaniel - are you happy with this variant rather than the one you
> proposed?
>
> Is someone going to prepare PRs for the PEP and for pip to match this
> behaviour? It sounds important enough that it should probably go into
> pip 18.0 (although that's a discussion we can have on the pip tracker
> - I'm not sure there's much we need to do, and there may be a PR
> already).
>

I'll be happy to update pip for this after the PEP update. It's probably
gonna be updating a conditional or so. :)


> Paul
>
> PS For completeness, here's the range of options. Shout if I
> misrepresented anything.
>
> pyproject.tomlbuild-system.requiresBehaviour
> ----
> Does not existN/A  Legacy
> ExistsDoes not exist   Current: Invalid file (error)
>Thomas: Legacy
>Nathaniel: Isolated
> (setuptools, wheel)
> ExistsExists   Isolated (full PEP 518)
>

The pip 10 behavior is 

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Wed, Jun 27, 2018 at 9:15 PM Paul Moore  wrote:

> On 27 June 2018 at 15:59, Pradyun Gedam  wrote:
> >
> > On Sun, Jun 24, 2018 at 8:20 AM Nick Coghlan  wrote:
> >>
> >> On 23 June 2018 at 07:47, Pradyun Gedam  wrote:
> >> > On Sat, 23 Jun 2018, 03:08 Brett Cannon,  wrote:
> >> >> The JSON schema is for "illustrative purposes only", so it should not
> >> >> be
> >> >> viewed as part of the spec.
> >> >
> >> > Yeah, that's exactly what I pointed out to the user in the pip issue
> >> > tracker
> >> > as well.
> >>
> >> Something you may want to consider is that while PEP 518 explicitly
> >> specifies that a missing pyproject.toml should be interpreted as
> >> "build-system.build-requires=['setuptools', 'wheel']", there's nothing
> >> that specifically prohibits an installer from offering a
> >> "default-for-missing-build-requires" setting.
> >>
> >> Right now, `pip` effectively defaults that hypothetical setting to
> >> "['setuptools', 'wheel']", with the proposed change being to switch it
> >> to "[]" instead (which better aligns with the intent of PEP 518).
> >>
> >> However, the new semantics mean that some sdist releases of some
> >> projects will become uninstallable. Making the installer level setting
> >> explicit rather than hypothetical would allow anyone affected to deal
> >> with that problem without either being stuck on the older version of
> >> pip, or being forced to concurrently upgrade to newer versions of the
> >> affected projects.
> >>
> >> Cheers,
> >> Nick.
> >>
> >> --
> >> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> >
> >
> > Assuming we are going to disallow missing build-requires,
> > I think a better way for this would be to allow a user to override
> > build-requires on a per-package basis. It'd be a more verbose
> > and also clearer about which packages are needing some sort
> > of work-around to install, pushing packages to just directly
> > specify build-requires in future releases.
>
> I'm not sure what you're suggesting here (but I suspect it's a
> pip-specific implementation detail rather than a standards issue).


Now that I think about it, yeah. Sorry for the noise!
​

> But
> ​ ​
> there are three separate cases to consider:
>
> 1. Project has pyproject.toml with build-system.requires specified. No
> problem, full PEP 518 behaviour (and pip uses build isolation).
> 2. Project has pyproject.toml but no build-system.requires, Illegal,
> confirmed by this discussion, so terminate with an error (pip currenly
> warns, but will move to an error after a deprecation period).
> 3. Project has no pyproject.toml. Old-style project, PEP 518 says that
> the default is [setuptools, wheel]. Pip will actually use the legacy
> (no build isolation) code path, which is a backward compatibility
> choice. I'm not actually sure PEP 518 needs to even comment on this
> case, as such projects clearly don't conform to that PEP and tools
> like pip will of necessity have to handle them as "legacy".
>
> The only case where I can see your "per-package overrides" fitting in
> would be (3), which is outside the scope of PEP 518 and so really a
> pip-only issue.
> ​
>
 ​
​I am suggesting it for (2).
 I'll elaborate on the tracker.

But if you're saying that the end user, when
> installing (from source) a package that has no pyproject.toml, can say
> something like "instead of just installing setuptools and wheel,
> install A, B, anc C", then
>
> a) A, B and C had better include setuptools and wheel, or we're hosed,
> b) We're on the legacy path, so we're not actually using build
> isolation (and so we're not able to install anything),
> c) I'm not sure I see why the end user would want to override what the
> project expected anyway.
>
> I'm pretty sure I've missed something here. I suggest we move this
> discussion to a feature request on the pip tracker, where you can
> explain your proposal in a bit more detail.
>

​Indeed.
​

> > This is already a feature request for pip for a different use case
> > and I think it's a reasonable request.
>
> What's the issue number? I don't recall seeing anything like this.
>

https://github.com/pypa/pip/issues/4582​
​​


> Sure, it is extra hoops to jump through but in the long term it
> > makes things fairly frictionless since everyone will start
> > specifying build-system.requires -- if there's a missing
> > build-system.requires, use the override mechanism to use
> > setuptools and wheel (and anything else possibly) to build it.
>
> I don't see how this (an end user feature) will encourage projects to
> move to using pyproject.toml - it seems like the opposite is more
> likely.


> Paul
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/YJL65VRPGTG4EURCSGXC4LKX6OT3VJFZ/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Sun, Jun 24, 2018 at 10:50 AM Nathaniel Smith  wrote:

> To go a bit against the grain here, I think at this point I'd suggest
> that if "build-system.requires" is missing, it should be silently
> treated as if it had been set to ["setuptools", "wheel"]. Reasoning:
>
> - Implementing this should require only a trivial amount of code, now
> and in the long run. In particular, I'm *not* suggesting that if the
> "build-system.requires" key is missing then we should act like
> pyproject.toml is missing altogether -- that's a much more complex
> legacy code path that we'd like to eventually remove. I'm suggesting
> we literally do something like:
>
> try:
> requires = config["build-system"]["requires"]
> except KeyError:
> requires = ["setuptools", "wheel"]
>
> and then treat them exactly the same from then on.
>

Defaulting to this behavior means that the way source distribution is
built changes (build isolation is enabled by pip) because configuration
for a tool was added. This is surprising for users since one of things
this means is they need to have provide wheels (so that pip that can
find/use them) for `setuptools` and `wheel` to install packages. We've
had multiple users report this on pip's tracker.

Having users specify their build-requirements explicitly is a stronger
opt-in that can used to explain the behavior in this case as that project
using PEP 518 and build-isolation vs that project has configuration for
towncrier. :)

The only other option is falling back to legacy behavior in this case,
which obviously isn't what we want here.

- Not doing this breaks a number of real projects. Sometimes this is
> justifiable because we have to break things to make progress, but it
> always creates busywork and pisses people off, so we should only do it
> when we have a good reason. In this case providing a default value is
> pretty trivial, and will prevent a lot of frustrated queries about why
> it's mandatory.
>

> - Providing a default doesn't really compromise the final vision for
> the feature: we envision that eventually, pretty much every project
> will be specifying this explicitly, and won't *want* to leave it
> blank. There isn't any other meaning we want to assign to this being
> left blank.
>
> - We're soon going to have to jump through all these hoops *anyway*
> for the PEP 517 "build-system.build-backend" key. If it's missing,
> then we're going to want to default it to "setuptools" (once
> setuptools exports a PEP 517 build backend), which means we're going
> to be hardcoding some defaults and knowledge of setuptools into the
> pyproject.toml defaults. So we might as well do this for both keys in
> the same way.
>
> -n​


Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/WNGISFXZ4JDSD32KZS35FUD5NPTK4DCQ/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Sun, Jun 24, 2018 at 8:20 AM Nick Coghlan  wrote:

> On 23 June 2018 at 07:47, Pradyun Gedam  wrote:
> > On Sat, 23 Jun 2018, 03:08 Brett Cannon,  wrote:
> >> The JSON schema is for "illustrative purposes only", so it should not be
> >> viewed as part of the spec.
> >
> > Yeah, that's exactly what I pointed out to the user in the pip issue
> tracker
> > as well.
>
> Something you may want to consider is that while PEP 518 explicitly
> specifies that a missing pyproject.toml should be interpreted as
> "build-system.build-requires=['setuptools', 'wheel']", there's nothing
> that specifically prohibits an installer from offering a
> "default-for-missing-build-requires" setting.
>
> Right now, `pip` effectively defaults that hypothetical setting to
> "['setuptools', 'wheel']", with the proposed change being to switch it
> to "[]" instead (which better aligns with the intent of PEP 518).
>
> However, the new semantics mean that some sdist releases of some
> projects will become uninstallable. Making the installer level setting
> explicit rather than hypothetical would allow anyone affected to deal
> with that problem without either being stuck on the older version of
> pip, or being forced to concurrently upgrade to newer versions of the
> affected projects.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>

​Assuming we are going to disallow missing build-requires,
​I think a better way for this would be to allow a user to override
build-requires on a per-package basis. It'd be a more verbose
and also clearer about which packages are needing some sort
of work-around to install, pushing packages to just directly
specify build-requires in future releases.

This is already a feature request for pip for a different use case
and I think it's a reasonable request.

Sure, it is extra hoops to jump through but in the long term it
makes things fairly frictionless since everyone will start
specifying build-system.requires -- if there's a missing
build-system.requires, use the override mechanism to use
setuptools and wheel (and anything else possibly) to build it.

Pradyun
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/VDVIQYVC2HMBOROJD4EF73RHXLSP3G2D/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Sat, 23 Jun 2018, 03:08 Brett Cannon,  wrote:

>
>
> On Fri, 22 Jun 2018 at 10:19 Pradyun Gedam  wrote:
>
>>
>>
>> On Fri, 22 Jun 2018, 22:31 Brett Cannon,  wrote:
>>
>>>
>>>
>>> On Fri, 22 Jun 2018 at 09:35 Pradyun Gedam  wrote:
>>>
>>>> Hey everyone!
>>>>
>>>> In PEP 518, it is not clearly specified how a project that has a
>>>> pyproject.toml
>>>> file but has no build-system.requires should be treated (i.e.
>>>> build-system
>>>> table).
>>>>
>>>> In pip 10, such a pyproject.toml file was ​allowed and built with
>>>> setuptools
>>>> and wheel, which has resulted in a lot of projects making releases that
>>>> assumed
>>>> that such a pyproject.toml file is valid and they use setuptools and
>>>> wheel.
>>>> I understand that at least pytest, towncrier and Twisted might have
>>>> done so.
>>>> This happen​ed ​since these projects have included configuration for
>>>> some tool in
>>>> pyproject.toml (some of which use ​only pyproject.toml for
>>>> configuration --
>>>> black, towncrier).
>>>>
>>>> There's a little bit of subtlety here, in pip 10's implementation:
>>>> adding a
>>>> pyproject.toml file enables a new code path that does the build in
>>>> isolation
>>>> (in preparation for PEP 517; it's a good idea on it's own too) with
>>>> only the
>>>> build-system.requires packages ​available. When the
>>>> build-system.requires key
>>>> is missing, pip falls back to assuming it should be ["setuptools",
>>>> "wheel"].
>>>> The in-development version of pip currently prints warnings when the
>>>> key is
>>>> not specified -- along the lines​ ​of "build-system.requires is
>>>> missing" +
>>>> "A future version of pip will reject pyproject.toml files that do not
>>>> comply
>>>> with PEP 518." and falls back to legacy behavior.
>>>>
>>>> Basically, pip 10 has a distinction between a missing pyproject.toml and
>>>> build-system.requires = ["setuptools", "wheel"] and the PEP doesn't.
>>>> However,
>>>> the PEP's precise wording here would help inform the debate about how
>>>> pip
>>>> should behave in this edge case.
>>>>
>>>
>>> I think the precise wording is there by not having any wording. ;) If
>>> 'requires' was meant to be optional then the PEP would have said that.
>>>
>>
>> The PEP does say that the requires key is mandatory. Someone pointed out
>> on the pip's tracker that it doesn't say that the build-system table is
>> mandatory (or otherwise).
>>
>
> It says "There will be a [build-system] table in the configuration file
> to store build-related data". The word "will" does not suggest optional to
> me. ;) Once again, no one said "optional", so assume it's required.
>

Sounds good to me.

>
>
>> And it seems that the JSON schema in the spec allows the table to be
>> skipped entirely. That's part of the reason for this thread.
>>
>
> The JSON schema is for "illustrative purposes only", so it should not be
> viewed as part of the spec.
>

Yeah, that's exactly what I pointed out to the user in the pip issue
tracker as well.


> IOW if you need some authoritative judgement on this you have it from one
> of the co-authors of the PEP. :)
>

Yeah. I think it's clear now what we have planned for pip is the right
thing to do here. Thanks!

Pradyun


> -Brett
>
>
>>
>>
>>>
>>>>
>>>> I can think of at least 2 options for behavior when
>>>> build-system.requires is
>>>> missing:
>>>>
>>>> 1. Consider a missing build-system.requires equivalent to either a
>>>> missing
>>>>pyproject.toml or build-system.requires = ["setuptools", "wheel"].
>>>>
>>>> 2. Making the build-system table mandatory in pyproject.toml.
>>>>
>>>> I personally think (2) would be fine -- "Explicit is better than
>>>> implicit."
>>>>
>>>
>>> And I think that's what pip's warning is saying the future will be, but
>>> they aren't quite yet ready to be that strict yet with their users (which I
>>> can understand).
>>>
>>>
>>>>
>>>> It'll be easy to detect and error out in this case, in a way that it's
>>>> possible
>>>> to provide meaningful information to the user about what to do here.
>>>> However,
>>>> this does mean that some existing releases of projects become
>>>> not-installable,
>>>> which is concerning; I do think the benefits outweigh the costs though.
>>>>
>>>> Thoughts on this?
>>>>
>>>
>>> I think what pip has planned with the warning makes sense.
>>>
>>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/ZN6MWEJPYAYV37XVDYGWHRWP4MGYGKBT/

[Distutils]Re: PEP 518 - pyproject.toml with no build-system.requires

[Pradyun Gedam]

On Fri, 22 Jun 2018, 22:31 Brett Cannon,  wrote:

>
>
> On Fri, 22 Jun 2018 at 09:35 Pradyun Gedam  wrote:
>
>> Hey everyone!
>>
>> In PEP 518, it is not clearly specified how a project that has a
>> pyproject.toml
>> file but has no build-system.requires should be treated (i.e. build-system
>> table).
>>
>> In pip 10, such a pyproject.toml file was ​allowed and built with
>> setuptools
>> and wheel, which has resulted in a lot of projects making releases that
>> assumed
>> that such a pyproject.toml file is valid and they use setuptools and
>> wheel.
>> I understand that at least pytest, towncrier and Twisted might have done
>> so.
>> This happen​ed ​since these projects have included configuration for some
>> tool in
>> pyproject.toml (some of which use ​only pyproject.toml for configuration
>> --
>> black, towncrier).
>>
>> There's a little bit of subtlety here, in pip 10's implementation: adding
>> a
>> pyproject.toml file enables a new code path that does the build in
>> isolation
>> (in preparation for PEP 517; it's a good idea on it's own too) with only
>> the
>> build-system.requires packages ​available. When the build-system.requires
>> key
>> is missing, pip falls back to assuming it should be ["setuptools",
>> "wheel"].
>> The in-development version of pip currently prints warnings when the key
>> is
>> not specified -- along the lines​ ​of "build-system.requires is missing" +
>> "A future version of pip will reject pyproject.toml files that do not
>> comply
>> with PEP 518." and falls back to legacy behavior.
>>
>> Basically, pip 10 has a distinction between a missing pyproject.toml and
>> build-system.requires = ["setuptools", "wheel"] and the PEP doesn't.
>> However,
>> the PEP's precise wording here would help inform the debate about how pip
>> should behave in this edge case.
>>
>
> I think the precise wording is there by not having any wording. ;) If
> 'requires' was meant to be optional then the PEP would have said that.
>

The PEP does say that the requires key is mandatory. Someone pointed out on
the pip's tracker that it doesn't say that the build-system table is
mandatory (or otherwise). And it seems that the JSON schema in the spec
allows the table to be skipped entirely. That's part of the reason for this
thread.


>
>>
>> I can think of at least 2 options for behavior when build-system.requires
>> is
>> missing:
>>
>> 1. Consider a missing build-system.requires equivalent to either a missing
>>pyproject.toml or build-system.requires = ["setuptools", "wheel"].
>>
>> 2. Making the build-system table mandatory in pyproject.toml.
>>
>> I personally think (2) would be fine -- "Explicit is better than
>> implicit."
>>
>
> And I think that's what pip's warning is saying the future will be, but
> they aren't quite yet ready to be that strict yet with their users (which I
> can understand).
>
>
>>
>> It'll be easy to detect and error out in this case, in a way that it's
>> possible
>> to provide meaningful information to the user about what to do here.
>> However,
>> this does mean that some existing releases of projects become
>> not-installable,
>> which is concerning; I do think the benefits outweigh the costs though.
>>
>> Thoughts on this?
>>
>
> I think what pip has planned with the warning makes sense.
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/PFVHC5WSFGZYFOLT2UL7THWDAFZEIDKW/

[Distutils] Re: Improving Communication

[Pradyun Gedam]

Sorry for poking an old thread. I actually wanted to respond earlier but
life got in the way.

Just a small point though.

On Sat, Apr 21, 2018 at 7:37 AM Donald Stufft  wrote:

> Currently in the packaging space, we have a number of avenues for
> communication, which are:
>
> - distutils-sig
> - pypa-dev
> - virtualenv-users
> - Other project specific mailing lists
> - IRC
> - gitter
> - Various issue trackers spread across multiple platforms.
> - Probably more places I’m not remembering.
>
> The result of this is that all discussion ends up being super fractured
> amongst the various places. Sometimes that is exactly what you want (for
> instance, someone who is working on the wheel specs probably doesn’t care
> about deprecation policy and internal module renaming in pip) and sometimes
> that ends up being the opposite of what you want (for instance, when you’re
> describing something that touches PyPI, setuptools, flit, pip, etc all at
> once).
>
> Theoretically the idea is that distutils-sig is where cross project
> reaching stuff goes, IRC/gitter is where real time discussion goes, and the
> various project mailing lists and issue trackers are where the project
> specific bits go. The problem is that often times doesn’t actually happen
> in practice except for the largest and most obvious of changes.
>

​For real time discussions, something like pypa/discuss and
pypa/dev-discuss (or some other spelling) on gitter, bridged to #pypa and
#pypa-dev would allow us to merge those two channels. That would be nice.

It seems that Julia does something like this:
https://groups.google.com/forum/#!topic/julia-users/ImKYzqHXA90

I think our current “communications stack” kind of sucks, and I’d love to
> figure out a better way for us to handle this that solves the sort of weird
> “independent but related” set of projects we have here.
>
> From my POV, a list of our major problems are:
>
> * Discussion gets fractured across a variety of platforms and locations,
> which can make it difficult to actually keep up with what’s going on but
> also to know how to loop in someone relevant if their input would be
> valuable. You have to more or less simultaneously know someone’s email,
> Github username, IRC nick, bitbucket username, etc to be able to bring
> threads of discussion to people’s attention.
>
> * It’s not always clear to users where a discussion should go, often times
> they’ll come to one location and need to get redirected to another
> location. If any discussion did happen in the incorrect location, it tends
> to need to get restarted in the new location (and depending on the specific
> platform, it may be impossible to actually redirect everyone over to the
> proper location, so you again, end up fractured with the discussion
> happening in two places).
>
> * A lot of the technology in this stack is particularly old, and lacks a
> lot of the modern day affordances that newer things have. An example is
> being able to edit a discussion post to fix typos that can hinder the
> ability of others to actually understand whats being talked about. In your
> typical mailing list or IRC there’s no mechanism by which you can edit an
> already sent message, so your only option is to either let the problem ride
> and hope it doesn’t trip up too many people, or send an additional message
> to correct the error. However these show up as additional, later messages
> which someone might not even see until they’ve already been thoroughly
> confused by the first message (since people tend to read email/IRC in a
> linear fashion).
>   - There is a lot of things in this one, other things are things like
> being able to control in a more fine grained manner what email you’re going
> to get.
>   - Holy crap, formatting and typography to make things actually readable
> and not a big block of plaintext.
>
> * We don’t have a clear way for users to get help, leaving users to treat
> random issues, discussion areas, etc as a support forum, rather than some
> place that’s actually optimized for that. Some of this ties back into some
> of the earlier things too, where it’s hard to actually redirect discussions
>
> These aren’t *new* problems, and often times the existing members of a
> community are the least effected becasue they’ve already spent effort
> learning the ins and outs and also curating a (typically custom) workflow
> that they’ve grown accustomed too. The problem with that is that often
> times that means that new users are left out, and the community gets
> smaller and smaller as time goes on as people leave and aren’t replaced
> with new blood, because they’re driven off but the issues with the stack.
>
> A big part of the place this is coming from, is me sitting back and
> realizing that I tend to be drawn towards pulling discussions into Github
> issues rather than onto the varying mailing lists, not because that’s
> always the most appropriate place for it, but because it’s the least
> painful 

Re: [Distutils] providing a way for pip to communicate extra info to users

[Pradyun Gedam]

On Tue, 10 Apr 2018, 05:17 Chris Jerdonek,  wrote:

> On the pypa-dev Google group, a suggestion was raised about giving pip
> a way to communicate extra info to users.
>
> This was during a thread started by Matthew Brett about pip breaking
> for certain macOS users due to certain TLS changes ("Impending silent
> breakage of pip / macOS likely to cause severe confusion"). Donald
> said this behavior is governed by PEP 503 and that the topic was best
> discussed on distutils-sig:
> https://groups.google.com/d/msg/pypa-dev/Oz6SGA7gefo/RRXQBQSBBAAJ
> so I'm raising the suggestion here to continue the discussion.
>
> One of Donald's comments in response to the idea (and that occurred to
> me too and that I agree with) is that providing a way to communicate
> messages to users introduces another possible avenue for attack.
>
> A possible middle-ground could be to hard-code a message in pip. Pip
> could display the message in certain circumstances, e.g. in response
> to certain types of failures. For example, the message could tell
> users to check a certain URL maintained by PyPA for further
> information / possible announcements.
>
> What do people think?
>

I like the idea.

I think linking to a location where we can make informative comments would
be a good idea — ideally where we can show announcements reverse
chronologically.

I don’t know how relevant they are but scenarios where this would help,
that come to my mind are:

- Status Page: "pypi.org is undergoing an incident and installations may
fail. You can find more information at status.python.org."
- Major Features: for things like PEP 517 when it's released. "There's
news. Have a look at pypi.org/news" or something like that.


> --


> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Renaming this list to python-packaging

[Pradyun Gedam]

On Mon, 19 Mar 2018 at 23:09 Paul Moore <p.f.mo...@gmail.com> wrote:

> On 19 March 2018 at 17:30, Pradyun Gedam <pradyu...@gmail.com> wrote:
> > Hey all!
> >
> > Given the direction we're moving with PEP 517/518, I propose that we
> rename
> > this list to something like python-packaging or something similar, to
> make
> > it a more relavantly named place for having discussions related to Python
> > Packaging in general.
> >
> > I'm curious what others think about this. I'm definitely open to thoughts
> > and other ideas. :)


> Maybe packaging-sig, to fit in with the other SIGs? See
> https://www.python.org/community/sigs/
>
> Paul
>

Yes. That sounds like a good idea!

Pradyun
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Renaming this list to python-packaging

[Pradyun Gedam]

Hey all!

Given the direction we're moving with PEP 517/518, I propose that we rename
this list to something like python-packaging or something similar, to make
it a more relavantly named place for having discussions related to Python
Packaging in general.

I'm curious what others think about this. I'm definitely open to thoughts
and other ideas. :)

Cheers,
Pradyun
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Installed Extras Metadata

[Pradyun Gedam]

On Sat, 27 Jan 2018 at 09:16 Nathaniel Smith <n...@pobox.com> wrote:

> On Fri, Jan 26, 2018 at 7:11 AM, Pradyun Gedam <pradyu...@gmail.com>
> wrote:
> > Hello! I hope everyone's had a great start to 2018! :)
> >
> > A few months back, while working on pip, I had noticed an oddity about
> > extras.
> >
> > Installing a package with extras would not store information about the
> fact
> > that the extras were requested. This means, later, it is not possible to
> > know which extra-based optional dependencies of a package have to be
> > considered when verifying that the packages are compatible with each
> other.
> > This information is relavant for resolution/validation since without it,
> it
> > is not possible to know which the extra-requirements to care about.
> >
> > As an example, installing ``requests[security]`` and then uninstalling
> > ``PyOpenSSL`` leaves you in a state where you don't really satisfy what
> was
> > asked for but there's no way to detect that either.
>
> Another important use case is upgrades: if requests[security] v1 just
> depends on pyopenssl, and then requests[security] v2 adds a dependency
> on certifi, and I do
>
> pip install requests[security] == 1
> pip upgrade
>
> then upgrade should give me requests[security] == 2, and thus install
> certifi. But this doesn't work if you don't have any record that
> 'requests[security]' is even installed :-).
>

Yes! Essentially, if there's a situation where a package may be modified,
we should care about having this information, to ensure it still does
satisfy the extra's requirements which may change themselves when the
base package changes.


> > Thus, obviously, I'm interested in making pip to be able to store this
> > information. As I understand, this is done needs to be specified in a PEP
> > and/or on PyPUG's specification page.
> >
> > To that end, here's seeding proposal for the discussion: a new
> > `extras-requested.txt` file in the .dist-info directory, storing the
> extra
> > names in a one-per-line format.
>
> I'm going to put in another plug here for my "reified extras" idea:
> https://mail.python.org/pipermail/distutils-sig/2015-October/027364.html
>
> Essentially, the idea is to promote extras to full packages --
> normally ones that contain no files, just metadata like dependencies,
> though that's not a necessary requirement, it's just how we'd
> interpret existing extras specifications.
>
> Then installing 'requests[security]' would install the
> 'requests[security]' package, which depends on both 'requests' and
> 'pyopenssl', and we have a 'requests[security]-$VERSION.dist-info'
> directory recording that we installed it.
>

I like this. This is how I'm modelling extras within the resolver currently,
by just considering extras as just-another-requirement and having them
depend on the base package and the extra dependencies. Prof. Justin
Cappos had suggested this to me. I imagine this'll result in simplification
somewhere due to this consistency between what the resolver consumes
and what's on the disk.

I think if we go this way, we should probably aim to just something
equivalent of Debian's metapackages for now. The rest of the advanced
features can be brought in at a latter stage.

The advantages are:
>
> - it's a simpler way to record information the information you want
> here, without adding more special cases to dist-info: most code
> doesn't even have to know what 'extras' are, just what packages are
>
> - it opens the door to lots of more advanced features, like
> 'foo[test]' being a package that actually contains foo's tests, or
> build variants like 'numpy[mkl]' being numpy built against the MKL
> library, or maybe making it possible to track which version of numpy's
> ABI different packages use. (The latter two cases need some kind of
> provides: support, which is impossible right now because we don't want
> to allow random-other-package to say 'provides-dist: cryptography';
> but, it would be okay if 'numpy[mkl]' said 'provides-dist: numpy',
> because we know 'numpy[mkl]' and 'numpy' are maintained by the same
> people.)
>
> I know there's a lot of precedent for this kind of clever use of
> metadata-only packages in Debian (e.g. search for "metapackages"), and
> I guess the RPM world probably has similar tricks.
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 508, environment markers & the possibility of Python 3.10

[Pradyun Gedam]

On Sun, 28 Jan 2018 at 11:15 Nick Coghlan  wrote:

> Hi folks,
>
> In https://github.com/python/peps/issues/560, a user pointed out that
> the current definition of python_version in PEP 508 assumes
> single-digit major and minor version numbers:
>
>platform.python_version()[:3]
>
> There's a reasonable chance we'll see 3.10 rather than 4.0 in a few
> years time, at which point that definition would break.
>
> The suggested fix is to amend that definition to be:
>
> ".".join(platform.python_version_tuple()[:2])
>
> This seems like a good suggestion to me, so my inclination is to
> handle this in a similar way to
> https://www.python.org/dev/peps/pep-0440/#summary-of-changes-to-pep-440:
> fix it in place, and add a section at the end of the PEP listing the
> post-publication changes.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


+1 from me too! :)
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Installed Extras Metadata

[Pradyun Gedam]

On Fri, 26 Jan 2018 at 23:10 Pradyun Gedam <pradyu...@gmail.com> wrote:

> Resending coz I clicked the wrong button.
>
> On Fri, 26 Jan 2018 at 22:16 Paul Moore <p.f.mo...@gmail.com> wrote:
>
>> On 26 January 2018 at 16:07, Pradyun Gedam <pradyu...@gmail.com> wrote:
>> >> Is there any other place where current functionality needs this
>> >> information, *apart* from pip check? Are there any proposed additional
>> >> features that might need it?
>> >
>> > Yes. I currently have a branch where during pip install, after the
>> packages
>> > to be installed are selected, it does a pip check on what would be the
>> > installed packages if the installation is successful. This would mean
>> that
>> > if we do select an incompatible version for installation, pip install
>> would
>> > be able to print a warning.
>>
>> I'd rather see us get a proper solver that would mean this should
>> never happen :-)
>
>
>>
> Yep, I am working on that. This is more of a short-term thing that I'm
> thinking of, just in case the proper solver isn't ready in time for pip 10.
> :)
>

And the resolver will need this extras-installed metadata as well.


> Aside, I've just posted an update over at #988 --
> https://github.com/pypa/pip/issues/988#issuecomment-360846457 -- about
> the state of the resolver work and the aforementioned branch.
>
>
>> Paul
>>
>
> --
>
> Pradyun
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Installed Extras Metadata

[Pradyun Gedam]

Resending coz I clicked the wrong button.

On Fri, 26 Jan 2018 at 22:16 Paul Moore <p.f.mo...@gmail.com> wrote:

> On 26 January 2018 at 16:07, Pradyun Gedam <pradyu...@gmail.com> wrote:
> >> Is there any other place where current functionality needs this
> >> information, *apart* from pip check? Are there any proposed additional
> >> features that might need it?
> >
> > Yes. I currently have a branch where during pip install, after the
> packages
> > to be installed are selected, it does a pip check on what would be the
> > installed packages if the installation is successful. This would mean
> that
> > if we do select an incompatible version for installation, pip install
> would
> > be able to print a warning.
>
> I'd rather see us get a proper solver that would mean this should
> never happen :-)
>
>
Yep, I am working on that. This is more of a short-term thing that I'm
thinking of, just in case the proper solver isn't ready in time for pip 10.
:)

Aside, I've just posted an update over at #988 --
https://github.com/pypa/pip/issues/988#issuecomment-360846457 -- about the
state of the resolver work and the aforementioned branch.


> Paul
>

--

Pradyun
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Installed Extras Metadata

[Pradyun Gedam]

On Fri, 26 Jan 2018, 21:21 Paul Moore, <p.f.mo...@gmail.com> wrote:

> On 26 January 2018 at 15:11, Pradyun Gedam <pradyu...@gmail.com> wrote:
> > Installing a package with extras would not store information about the
> fact
> > that the extras were requested. This means, later, it is not possible to
> > know which extra-based optional dependencies of a package have to be
> > considered when verifying that the packages are compatible with each
> other.
> > This information is relavant for resolution/validation since without it,
> it
> > is not possible to know which the extra-requirements to care about.
> >
> > As an example, installing ``requests[security]`` and then uninstalling
> > ``PyOpenSSL`` leaves you in a state where you don't really satisfy what
> was
> > asked for but there's no way to detect that either.
>
> 1. pip uninstall doesn't check validity, so there's no issue for the
> uninstall
> 2. pip check needs this information if it's to complain, and I believe
> that's the key point in your question
>

Indeed. Thanks for adding this clarification. I guess this answers Alex's
question as well.


> I think that if we want pip check to validate this situation, we need
> to store the data when we install. Where we store it needs to be
> decided - I'd assume it would go in the dist-info directory for
> requests somewhere, and that's the bit of metadata that needs
> agreeing.
>
> Is there any other place where current functionality needs this
> information, *apart* from pip check? Are there any proposed additional
> features that might need it?
>

Yes. I currently have a branch where during pip install, after the packages
to be installed are selected, it does a pip check on what would be the
installed packages if the installation is successful. This would mean that
if we do select an incompatible version for installation, pip install would
be able to print a warning.

In other words, a pip check would be done during every pip install run.

> Thus, obviously, I'm interested in making pip to be able to store this
> > information. As I understand, this is done needs to be specified in a PEP
> > and/or on PyPUG's specification page.
> >
> > To that end, here's seeding proposal for the discussion: a new
> > `extras-requested.txt` file in the .dist-info directory, storing the
> extra
> > names in a one-per-line format.
>
> Looks OK to me.
>
> But I don't know how important it is to satisfy this use case. I've
> never needed this feature (I don't think I've ever used pip check, and
> I've very rarely used extras) so I won't comment on that.
>
> Paul
>
> PS I know we talked a bit off-list about this and I said I didn't have
> any opinion. I'd misunderstood what you were suggesting a bit, mostly
> because the conversation veered off into uninstalling
> requests[security] and what that means... So now that you've restated
> the issue, I have a bit of an opinion (although still not much :-))
>

Hehe. No problems. :)

>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Where to get help changing ownership of Pypi package?

[Pradyun Gedam]

On Sun, 5 Nov 2017, 21:16 Wes Turner,  wrote:

> There's a "package claim" label in the pypa/pypi-legacy Github issue
> tracker:
>
> https://github.com/pypa/pypi-legacy/issues
>
> What is the documented procedure for handling transfer of package
> ownership?
>
>
There's PEP 541; it's still a draft though. I think there was a thread for
this in early this year and also more recently around September.

Until that PEP is accepted (likely even after), the URL that Wes posted is
probably the right place...
Best,
Pradyun


>
> On Sunday, November 5, 2017, Stuart Axon via Distutils-SIG <
> distutils-sig@python.org> wrote:
>
>> Hi,
>>I realise this is the wrong place, but does anyone know who could help
>> resolve ownership of the Pypi package PyCairo
>>
>> https://sourceforge.net/p/pypi/support-requests/728/
>>
>>
>> There are also a bunch of similar tickets, I guess nobody checks these
>> any more ? :(
>>
>> S++
>>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Conditionless setup.py

[Pradyun Gedam]

Hi Thomas!

Have you seen Pipfile? It's something along those lines - a declarative
specification of project data. :)

github.com/pypa/pipfile

Regards,
Pradyun Gedam

On Fri, Aug 25, 2017, 14:12 Thomas Güttler <guettl...@thomas-guettler.de>
wrote:

> The setup.py of numpy looks big and complicated:
> https://github.com/numpy/numpy/blob/master/setup.py
>
> I guess there are even more complicated setup.py files.
>
> Do you think it will be possible in the future to make setup.py
> conditionless (no "if", no "else", no function calls)?
>
> I mean that the setuptools/distutils/whatever libraries will handle all
> needed conditions?
>
> This would mean setup.py would be a pure data structure which could be
> encoded in json or yaml.
>
> What do you think?
>
> Regards,
>Thomas Güttler
>
>
> --
> Thomas Guettler http://www.thomas-guettler.de/
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] GSoC 2017 - Request for Comments on Proposal

[Pradyun Gedam]

On Sun, Mar 26, 2017, 08:01 Wayne Werner <waynejwer...@gmail.com> wrote:

> Tiny change: I believe its Wesley, not Weasley :)
>
Indeed. My bad. I'll fix it. Thank you!


> On Sat, Mar 25, 2017, 12:33 PM Paul Moore <p.f.mo...@gmail.com> wrote:
>
> On 25 March 2017 at 15:44, Pradyun Gedam <pradyu...@gmail.com> wrote:
> > Hello Everyone!
> >
> > I had previously sent a mail on this list, stating that I would like to
> work
> > on pip's dependency resolution for my GSoC 2017. I now have drafted a
> > proposal for the same; with help from my mentors - Donald Stufft and
> Justin
> > Cappos. I'd also take this opportunity to thank them for agreeing to be
> my
> > mentors for this GSoC.
> >
> > I would like to request for comments on my proposal for GSoC - it is
> hosted
> > at https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb.
> >
> > Please find trailing a MarkDown version of the proposal.
>
> Hi Pradyun,
> Your proposal looks pretty impressive - well structured and thought
> out. I've looked through it and the plans seem reasonable - I've
> looked more at what you're proposing than at the timescales, but your
> staged approach seems sensible - if you do hit issues with time, it
> looks like you'll be able to deliver real improvements even if you
> don't complete everything, which is fantastic.
>
> Best of luck - assuming you complete all the work you've planned, it
> will be a significant benefit to pip. If I can be of any help, with PR
> reviews or anything similar, feel free to ping me.
>
> Paul
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] GSoC 2017 - RFC on Proposal

[Pradyun Gedam]

I apologize for the duplicate mail. Please respond on this thread.

On Sat, 25 Mar 2017 at 21:16 Pradyun Gedam <pradyu...@gmail.com> wrote:

> Hello Everyone!
>
> I had previously sent a mail on this list, stating that I would like to
> work on pip's dependency resolution for my GSoC 2017. I now have drafted a
> proposal for the same; with help from my mentors - Donald Stufft and Justin
> Cappos. I'd also take this opportunity to thank them for agreeing to be my
> mentors for this GSoC.
>
> I would like to request for comments on my proposal for GSoC - it is
> hosted at
> https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb.
>
> Please find trailing a MarkDown version of the proposal.
>
> Thanks,
> Pradyun Gedam
>
> -
>
> # Adding Proper Dependency Resolution to pip
>
> - **Name:** Pradyun S. Gedam
> - **Email:** [pradyu...@gmail.com][mailto-email]
> - **Github:** [pradyunsg][github-profile]
> - **University:** [VIT University, Vellore, India][vit-homepage]
> - **Course:** Bachelor of Technology in Computer Science and Engineering
> - **Course Term:** 2016/17 - 2019/20 (4 Year)
> - **Timezone:** IST (GMT +5:30)
> - **GSoC Blog RSS Feed URL:** <
> https://pradyunsg.github.io/gsoc-2017/feed.xml>
>
> [github-profile]: http://github.com/pradyunsg/
> [vit-homepage]: http://vit.ac.in/
> [mailto-email]: mailto:pradyu...@gmail.com
>
> ## About Me
>
> I was introduced to Python about five years ago, through "Core Python
> Programming" by Weasley J Chun. After the initial struggle with Python
> 2/3, the
> ball was set rolling and hasn't stopped since. I have fiddled around with
> Game Programming (PyGame), Computer Vision (OpenCV), Data Analytics
> (Pandas,
> SciPy, NumPy), transcompilers (Py2C) and more.
>
> As a high school student, I did internship at Enthought in 2013 and 2014.
> The two summers that I spent at Enthought were a great learning experience
> and
> I thoroughly enjoyed the environment there.
>
> Other than Python, I have also used C, C++, Web Technologies (JavaScript,
> HTML,
> CSS) and preprocessors (Pug, TypeScript, LESS/SASS/SCSS), Java and
> Bash/Zsh for
> some other projects.
>
> Curious to understand how pip works, I began looking into pip's source
> code.
> I started contributing to pip in May 2016. I am now fairly familiar with
> the
> design of pip and have a fair understanding of how it works, due to the
> various
> contributions I have made to pip in the past year.
>
> ## Mentors
>
> - Donald Stufft (@dstufft on GitHub)
> - Justin Cappos (@JustinCappos on GitHub)
>
> Communication with the mentors will be done mostly on issues and pull
> requests
> on pip's GitHub repository. If at any point in time, a real time
> discussion is
> to be done with the mentors, it would be done over IRC or Skype. Email can
> also
> be used if needed.
>
> ## Proposal
>
> This project is regarding improving dependency resolution performed within
> pip
> by implementing a dependency resolver within it.
>
> ### Abstract
>
> Currently, pip does not resolve dependencies correctly when there are
> conflicting requirements. The lack of dependency resolution has caused
> hard-to-debug bugs/failures due to the installation of incompatible
> packages.
> The lack of a dependency resolver is also a blocker for various other
> features -
> adding an upgrade-all functionality to pip and properly determining
> build-time
> dependencies for packages are two such features.
>
> ### Deliverables
>
> At the end of this project, pip will have the ability to:
>
> - detect requirement conflicts
> - resolve conflicting dependency requirements where possible
>
> ### Implementation
>
> The implementation language will be Python. The code will maintain the
> compatibility requirements of pip - the same source code will support the
> multiple Python implementations and version, including but not limited to,
> CPython 2.7, CPython 3.3+, PyPy 2, PyPy3.
>
> New Tests for the functionality introduced will be added to pip's current
> test
> suite.
>
> User documentation would not be a major part of this project. The only
> changes
> would be to mention pip can now resolve dependencies properly. There are
> certain
> sections that might need updating.
>
>  Overview
>
> The project will be composed of the following stages:
>
> 1. Refactor the dependency resolution logic of pip into a separate module.
> 1. Implement dependency information caching in pip.
> 1. Implement a backtracking dependency resolver to resolve the
> dependencies.
>
> Every stage depends on the previous ones being completed. This step-wise
> appr

[Distutils] GSoC 2017 - RFC on Proposal

[Pradyun Gedam]

Hello Everyone!

I had previously sent a mail on this list, stating that I would like to
work on pip's dependency resolution for my GSoC 2017. I now have drafted a
proposal for the same; with help from my mentors - Donald Stufft and Justin
Cappos. I'd also take this opportunity to thank them for agreeing to be my
mentors for this GSoC.

I would like to request for comments on my proposal for GSoC - it is hosted
at https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb.

Please find trailing a MarkDown version of the proposal.

Thanks,
Pradyun Gedam

-

# Adding Proper Dependency Resolution to pip

- **Name:** Pradyun S. Gedam
- **Email:** [pradyu...@gmail.com][mailto-email]
- **Github:** [pradyunsg][github-profile]
- **University:** [VIT University, Vellore, India][vit-homepage]
- **Course:** Bachelor of Technology in Computer Science and Engineering
- **Course Term:** 2016/17 - 2019/20 (4 Year)
- **Timezone:** IST (GMT +5:30)
- **GSoC Blog RSS Feed URL:** <
https://pradyunsg.github.io/gsoc-2017/feed.xml>

[github-profile]: http://github.com/pradyunsg/
[vit-homepage]: http://vit.ac.in/
[mailto-email]: mailto:pradyu...@gmail.com

## About Me

I was introduced to Python about five years ago, through "Core Python
Programming" by Weasley J Chun. After the initial struggle with Python 2/3,
the
ball was set rolling and hasn't stopped since. I have fiddled around with
Game Programming (PyGame), Computer Vision (OpenCV), Data Analytics (Pandas,
SciPy, NumPy), transcompilers (Py2C) and more.

As a high school student, I did internship at Enthought in 2013 and 2014.
The two summers that I spent at Enthought were a great learning experience
and
I thoroughly enjoyed the environment there.

Other than Python, I have also used C, C++, Web Technologies (JavaScript,
HTML,
CSS) and preprocessors (Pug, TypeScript, LESS/SASS/SCSS), Java and Bash/Zsh
for
some other projects.

Curious to understand how pip works, I began looking into pip's source code.
I started contributing to pip in May 2016. I am now fairly familiar with the
design of pip and have a fair understanding of how it works, due to the
various
contributions I have made to pip in the past year.

## Mentors

- Donald Stufft (@dstufft on GitHub)
- Justin Cappos (@JustinCappos on GitHub)

Communication with the mentors will be done mostly on issues and pull
requests
on pip's GitHub repository. If at any point in time, a real time discussion
is
to be done with the mentors, it would be done over IRC or Skype. Email can
also
be used if needed.

## Proposal

This project is regarding improving dependency resolution performed within
pip
by implementing a dependency resolver within it.

### Abstract

Currently, pip does not resolve dependencies correctly when there are
conflicting requirements. The lack of dependency resolution has caused
hard-to-debug bugs/failures due to the installation of incompatible
packages.
The lack of a dependency resolver is also a blocker for various other
features -
adding an upgrade-all functionality to pip and properly determining
build-time
dependencies for packages are two such features.

### Deliverables

At the end of this project, pip will have the ability to:

- detect requirement conflicts
- resolve conflicting dependency requirements where possible

### Implementation

The implementation language will be Python. The code will maintain the
compatibility requirements of pip - the same source code will support the
multiple Python implementations and version, including but not limited to,
CPython 2.7, CPython 3.3+, PyPy 2, PyPy3.

New Tests for the functionality introduced will be added to pip's current
test
suite.

User documentation would not be a major part of this project. The only
changes
would be to mention pip can now resolve dependencies properly. There are
certain
sections that might need updating.

 Overview

The project will be composed of the following stages:

1. Refactor the dependency resolution logic of pip into a separate module.
1. Implement dependency information caching in pip.
1. Implement a backtracking dependency resolver to resolve the dependencies.

Every stage depends on the previous ones being completed. This step-wise
approach would make incremental improvements so that there is a constant
feedback on the work being done as well as making it easy to change course
without losing existing work; if needed for some unforeseen reason.

 Discussion

There is a class in pip - `RequirementSet`, which is currently a God class.
It
is responsible for the following:

  1. Holding a list of things to be installed
  1. Downloading Files
  1. Dependency Resolution
  1. Unpacking downloaded files (preparation for installation)
  1. Uninstalling packages
  1. Installing packages

This is clearly a bad situation since this is most of the heavy lifting
involved in pip. These responsibilities need to be separated and moved out
into
their independent modules/classes, to allow fo

[Distutils] GSoC 2017 - Request for Comments on Proposal

[Pradyun Gedam]

Hello Everyone!

I had previously sent a mail on this list, stating that I would like to
work on pip's dependency resolution for my GSoC 2017. I now have drafted a
proposal for the same; with help from my mentors - Donald Stufft and Justin
Cappos. I'd also take this opportunity to thank them for agreeing to be my
mentors for this GSoC.

I would like to request for comments on my proposal for GSoC - it is hosted
at https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb.

Please find trailing a MarkDown version of the proposal.

Thanks,
Pradyun Gedam

-

# Adding Proper Dependency Resolution to pip

- **Name:** Pradyun S. Gedam
- **Email:** [pradyu...@gmail.com][mailto-email]
- **Github:** [pradyunsg][github-profile]
- **University:** [VIT University, Vellore, India][vit-homepage]
- **Course:** Bachelor of Technology in Computer Science and Engineering
- **Course Term:** 2016/17 - 2019/20 (4 Year)
- **Timezone:** IST (GMT +5:30)
- **GSoC Blog RSS Feed URL:** <
https://pradyunsg.github.io/gsoc-2017/feed.xml>

[github-profile]: http://github.com/pradyunsg/
[vit-homepage]: http://vit.ac.in/
[mailto-email]: mailto:pradyu...@gmail.com

## About Me

I was introduced to Python about five years ago, through "Core Python
Programming" by Weasley J Chun. After the initial struggle with Python 2/3,
the
ball was set rolling and hasn't stopped since. I have fiddled around with
Game Programming (PyGame), Computer Vision (OpenCV), Data Analytics (Pandas,
SciPy, NumPy), transcompilers (Py2C) and more.

As a high school student, I did internship at Enthought in 2013 and 2014.
The two summers that I spent at Enthought were a great learning experience
and
I thoroughly enjoyed the environment there.

Other than Python, I have also used C, C++, Web Technologies (JavaScript,
HTML,
CSS) and preprocessors (Pug, TypeScript, LESS/SASS/SCSS), Java and Bash/Zsh
for
some other projects.

Curious to understand how pip works, I began looking into pip's source code.
I started contributing to pip in May 2016. I am now fairly familiar with the
design of pip and have a fair understanding of how it works, due to the
various
contributions I have made to pip in the past year.

## Mentors

- Donald Stufft (@dstufft on GitHub)
- Justin Cappos (@JustinCappos on GitHub)

Communication with the mentors will be done mostly on issues and pull
requests
on pip's GitHub repository. If at any point in time, a real time discussion
is
to be done with the mentors, it would be done over IRC or Skype. Email can
also
be used if needed.

## Proposal

This project is regarding improving dependency resolution performed within
pip
by implementing a dependency resolver within it.

### Abstract

Currently, pip does not resolve dependencies correctly when there are
conflicting requirements. The lack of dependency resolution has caused
hard-to-debug bugs/failures due to the installation of incompatible
packages.
The lack of a dependency resolver is also a blocker for various other
features -
adding an upgrade-all functionality to pip and properly determining
build-time
dependencies for packages are two such features.

### Deliverables

At the end of this project, pip will have the ability to:

- detect requirement conflicts
- resolve conflicting dependency requirements where possible

### Implementation

The implementation language will be Python. The code will maintain the
compatibility requirements of pip - the same source code will support the
multiple Python implementations and version, including but not limited to,
CPython 2.7, CPython 3.3+, PyPy 2, PyPy3.

New Tests for the functionality introduced will be added to pip's current
test
suite.

User documentation would not be a major part of this project. The only
changes
would be to mention pip can now resolve dependencies properly. There are
certain
sections that might need updating.

 Overview

The project will be composed of the following stages:

1. Refactor the dependency resolution logic of pip into a separate module.
1. Implement dependency information caching in pip.
1. Implement a backtracking dependency resolver to resolve the dependencies.

Every stage depends on the previous ones being completed. This step-wise
approach would make incremental improvements so that there is a constant
feedback on the work being done as well as making it easy to change course
without losing existing work; if needed for some unforeseen reason.

 Discussion

There is a class in pip - `RequirementSet`, which is currently a God class.
It
is responsible for the following:

  1. Holding a list of things to be installed
  1. Downloading Files
  1. Dependency Resolution
  1. Unpacking downloaded files (preparation for installation)
  1. Uninstalling packages
  1. Installing packages

This is clearly a bad situation since this is most of the heavy lifting
involved in pip. These responsibilities need to be separated and moved out
into
their independent modules/classes, to allow fo

Re: [Distutils] GSoC 2017 - Plan of Action for dependency resolver

[Pradyun Gedam]

On Sat, 4 Mar 2017 at 22:58 Donald Stufft <don...@stufft.io> wrote:

>
> On Mar 4, 2017, at 12:25 PM, Pradyun Gedam <pradyu...@gmail.com> wrote:
>
> Since PyPI does not have such information in a static declarative format,
> that approach is not feasible. pip will have to download packages and
> execute setup.py to know what the dependencies are.
>
>
>
> I will note, that we can expose that information in PyPI for *wheels*, but
> not for sdists currently. It would be a lot more work though because it’d
> essentially require a whole new repository API and I doubt Pradyun wants to
> tackle that right now :)
>

Yeah... For now, it's just dependency resolution in pip.


> Keeping a future in mind where we can get a least some of that information
> without downloading would be good though, at least to keep in mind when
> structuring code.
>

Duly noted.


> —
>
> Donald Stufft
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] GSoC 2017 - Plan of Action for dependency resolver

[Pradyun Gedam]

On Wed, 1 Mar 2017 at 15:58 Pradyun Gedam <pradyu...@gmail.com> wrote:

>
>
> On Tue, Feb 28, 2017, 21:18 Jim Fulton <j...@jimfulton.info> wrote:
>
> On Tue, Feb 28, 2017 at 10:14 AM, Pradyun Gedam <pradyu...@gmail.com>
> wrote:
> ...
>
> 4. (if time permits) Move any dependency resolution code out into a
> separate library.
>
>This would make it possible for other projects (like buildout or a
> future pip replacement) to reuse the dependency resolver.
>
>
> Thank you!
>
>
> Welcome!
>
>
> ...
>
> I do intend to reuse some of the work done by Robert Collins in PR #2716
> on pip's GitHub repository.
>
>
> Are you aware of the proof of concept in distlib?
>
>
> I am. I had looked at it a few weeks back. IIRC it makes a dependency
> graph using distlib and operates with that.
>
> I haven't really understood how it gets the information about dependencies
> without downloading the packages... I'll give it another pass this weekend.
>

I went through it.

As Paul Moore said, it is hitting http://www.red-dove.com/pypi/ which has
metdata on what the requirements are of a package. (saying this on the
basis of [1])

Since PyPI does not have such information in a static declarative format,
that approach is not feasible. pip will have to download packages and
execute setup.py to know what the dependencies are.

[1]: https://www.red-dove.com/pypi/projects/S/Sphinx/package-1.3.json


>
>
> https://distil.readthedocs.io/en/0.1.0/overview.html#actual-improvements
>
> Jim
>
> --
> Jim Fulton
> http://jimfulton.info
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] GSoC 2017 - Plan of Action for dependency resolver

[Pradyun Gedam]

Thanks for the pointer Ralf! :)

I was actually drafting a mail to send to Donald directly for thanking him
for being willing to mentor me as well as pointing this out to him.

I guess I can discard that draft now...

On Thu, Mar 2, 2017, 01:37 Donald Stufft <don...@stufft.io> wrote:

>
> On Mar 1, 2017, at 3:02 PM, Ralf Gommers <ralf.gomm...@gmail.com> wrote:
>
>
>
> On Wed, Mar 1, 2017 at 4:14 AM, Pradyun Gedam <pradyu...@gmail.com> wrote:
>
> Hello Everyone!
>
> Google released the list of accepted organizations for GSoC 2017 and PSF
> is one of them.
>
>
> I see pip is not yet listed as a PSF sub-org on http://python-gsoc.org/.
> This is pretty urgent to arrange:
>
> *"March 3* - Last day for Python sub-orgs to apply to participate
> with the PSF.
> (Assuming we get accepted by Google and can support sub-orgs, of
> course!)
> This deadline is for orgs who applies on their own and didn't make it,
> but still
>  wish to participate under the umbrella. "
>
> The original deadline was Feb 7. There's a good chance that Pip will still
> be accepted after March 3, but I wouldn't gamble on it.
>
> There are instructions under "Project Ideas" on http://python-gsoc.org/
> on how to get accepted as a sub-org.
>
>
>
> Oh. I’ve never done this before and Pradyun reached out so I had no idea I
> had to do this. I’ll go ahead and do this.
>
>
> —
>
> Donald Stufft
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] GSoC 2017 - Plan of Action for dependency resolver

[Pradyun Gedam]

On Tue, Feb 28, 2017, 21:18 Jim Fulton <j...@jimfulton.info> wrote:

On Tue, Feb 28, 2017 at 10:14 AM, Pradyun Gedam <pradyu...@gmail.com> wrote:
...

4. (if time permits) Move any dependency resolution code out into a
separate library.

   This would make it possible for other projects (like buildout or a
future pip replacement) to reuse the dependency resolver.


Thank you!


Welcome!


...

I do intend to reuse some of the work done by Robert Collins in PR #2716 on
pip's GitHub repository.


Are you aware of the proof of concept in distlib?


I am. I had looked at it a few weeks back. IIRC it makes a dependency graph
using distlib and operates with that.

I haven't really understood how it gets the information about dependencies
without downloading the packages... I'll give it another pass this weekend.


https://distil.readthedocs.io/en/0.1.0/overview.html#actual-improvements

Jim

-- 
Jim Fulton
http://jimfulton.info
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] GSoC 2017 - Plan of Action for dependency resolver

[Pradyun Gedam]

Hello Everyone!

Google released the list of accepted organizations for GSoC 2017 and PSF is
one of them. I guess this would a good time for me to seek feedback on the
approach I'm planning to take for my potential GSoC project. I hope this
mailing list is the right place to do so.

---

Here's my current plan of action along with reasoning for the choices made:

A separate PR will be made for each of these stages. Every stage does
depend on the previous ones being completed.

1. Refactor all dependency resolution responsibility in pip into a new,
separate module.

   This would allow any future changes/improvements in the dependency
resolution to be added without major changes in the rest of the code-base.

   As of today, the RequirementSet class within pip seems to be doing a lot
of work and dependency resolution is a responsibility that doesn't need to
given to it, especially when it's avoidable.

2. Implement dependency information caching.

   This would allow the resolver to not cause the re-computation of the
dependencies of a package, if they have already been computed, speeding up
the resolution.

3. Implement a backtracking resolver.

   A backtracking solver would be appropriate given that we don't have a
way to pre-compute the dependencies for *all* the packages or statically
determine the dependencies - a SAT solver would not be feasible.

4. (if time permits) Move any dependency resolution code out into a
separate library.

   This would make it possible for other projects (like buildout or a
future pip replacement) to reuse the dependency resolver.

By making each of the stages separate PRs, incremental improvements would
be made so that even if I leave this project midway, there will be some
work merged already if someone comes back to this problem later. That said,
I don't intend to leave this project midway.

I do intend to reuse some of the work done by Robert Collins in PR #2716 on
pip's GitHub repository.

Stages 2 and 3 are separate because I see them as distinctly different
tasks which touch very different portions of the code-base. There's is
strong coupling between them though.

I'm looking forward to the feedback. :)

Regards,
Pradyun
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] GSoC 2017 - Working on pip

[Pradyun Gedam]

Yay! Thank you so much for a prompt and positive response! I'm pretty
excited and looking forward to this.

On Thu, Feb 9, 2017, 20:23 Donald Stufft <don...@stufft.io> wrote:

I’ve never done it before, but I’m happy to provide mentoring on this.

On Feb 8, 2017, at 9:15 PM, Pradyun Gedam <pradyu...@gmail.com> wrote:

Hello Everyone!

Ralf Gommers suggested that I put this proposal here on this list, for
feedback and for seeing if anyone would be willing to mentor me. So, here
it is.

-

My name is Pradyun Gedam. I'm currently a first year student VIT University
in India.

I would like to apply for GSoC 2017 under PSF.

I currently have a project in mind - the "pip needs a dependency resolver"
issue [1]. I would like to take on this specific project but am willing to
do some other project as well.

For some background, around mid 2016, I started contributing to pip. The
first issue I tackled was #59 [2] - a request for upgrade command and an
upgrade-all command that has been open for over 5.5 years. Over the months
following that, I've have had the opportunity to work with and understand
multiple parts of pip's codebase while working on this issue and a few
others. This search on GitHub issues [3] also provides a good summary of
what work I've done on pip.

[2]: https://github.com/pypa/pip/issues/988
[2]: https://github.com/pypa/pip/issues/59
[3]: https://github.com/pypa/pip/issues?q=author%3Apradyunsg

Eagerly-waiting-for-a-response-ly,
Pradyun Gedam

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig



—

Donald Stufft
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] GSoC 2017 - Working on pip

[Pradyun Gedam]

Hello Everyone!

Ralf Gommers suggested that I put this proposal here on this list, for
feedback and for seeing if anyone would be willing to mentor me. So, here
it is.

-

My name is Pradyun Gedam. I'm currently a first year student VIT University
in India.

I would like to apply for GSoC 2017 under PSF.

I currently have a project in mind - the "pip needs a dependency resolver"
issue [1]. I would like to take on this specific project but am willing to
do some other project as well.

For some background, around mid 2016, I started contributing to pip. The
first issue I tackled was #59 [2] - a request for upgrade command and an
upgrade-all command that has been open for over 5.5 years. Over the months
following that, I've have had the opportunity to work with and understand
multiple parts of pip's codebase while working on this issue and a few
others. This search on GitHub issues [3] also provides a good summary of
what work I've done on pip.

[2]: https://github.com/pypa/pip/issues/988
[2]: https://github.com/pypa/pip/issues/59
[3]: https://github.com/pypa/pip/issues?q=author%3Apradyunsg

Eagerly-waiting-for-a-response-ly,
Pradyun Gedam
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

Indeed. Let's move the further discussion over to
https://github.com/pypa/pip/issues/3786.

On Mon, 27 Jun 2016 at 12:57 Robert Collins 
wrote:

> I've replied in the github issue; I don't really want to carry out
> *two* conversations, so perhaps we can pick one forum and stick there?
>
> -Rob
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

On Mon, 27 Jun 2016 at 12:21 Ralf Gommers <ralf.gomm...@gmail.com> wrote:

> On Mon, Jun 27, 2016 at 8:36 AM, Pradyun Gedam <pradyu...@gmail.com>
> wrote:
>
>> Hello List!
>>
>> I feel it’s fine to hold back the other changes for later but the
>> upgrade-strategy change should get shipped out to the world as quickly as
>> possible. Even how the change is exposed the user can also be discussed
>> later.
>>
> What do you mean by "ship" if you say the behavior can still be changed
> later?
>

Sorry for the confusion and a useless-mail! Allow me to re-phrase that
statement...

I think it's fine if we hold back the changes to install-as-upstall and
--target for later. The
upgrade-strategy change should get released to the world as quickly as
possible. If no one has any outstanding issues with switching over to
non-eager upgrades by default, may we start a discussing on how the change
is to be exposed the user?


>
>> I request the list members to focus on *only* the change of the default
>> upgrade strategy to be non-eager.
>>
>> Does anyone have any concerns regarding the change of the default upgrade
>> strategy to be non-eager? If not, let’s get *just* that shipped out as
>> soon as possible.
>>
> The concerns were always with how to change it
>

There were security-related concerns raised by Robert which were addressed
[1] by Donald.


> one of:
> (1) add "pip upgrade"
> (2) change behavior of "pip install --upgrade"
> (3) change behavior of "pip install"
>
> Your sentence above suggests you're asking for agreement on (2), but I
> think you want agreement on (3) right? At least that was the conclusion of
> your PEP-style writeup.
>
>
As it stands, we currently have (3) implemented (as proposed in that
write-up) since that was what was extensively discussed and decided over at
Github. If no one has issues with it, let's go ahead with it. If there is
anyone has issues, I'm fine with going (2) as well. I do not like (1) due
to the weird model it creates providing 2 ways to do what most people would
see as one thing.

Basically, If we were voting, -1 for (1), +0.5 for (2), +1 for (3).

Personally I don't have a preference anymore, as long as a choice is made
> so we don't remain stuck where we are now.
>
> Ralf
>

[1] There's probably a better word for this than "addressed".

 --

Pradyun Gedam
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

On Sun, 26 Jun 2016 at 23:02 Donald Stufft <don...@stufft.io> wrote:

>
> On Jun 25, 2016, at 6:25 AM, Pradyun Gedam <pradyu...@gmail.com> wrote:
>
> There is currently a proposal to change the behaviour to pip install to
> upgrade a package that is passed even if it is already installed.
>
> This behaviour change is accompanied with a change in the upgrade strategy
> - pip would stop “eagerly” upgrading dependencies and would become more
> conservative, upgrading a dependency only when it doesn’t meet lower
> constraints of the newer version of a parent. Moreover, the behaviour of pip
> install --target would also be changed so that --upgrade no longer
> affects it.
>
> I think bundling these two changes (and I think I might have been the one
> that originally suggested it) is making this discussion harder than it
> needs to be as folks are having to fight on multiple different fronts at
> once. I think the change to the default behavior of pip install is
> dependent on the change to —upgrade, so I suggest we focus on the change to
> —upgrade first, changing from a “recursive” to a “conservative” strategy.
> Once we get that change figured out and landed then we can worry about what
> to do with pip install.
>

You were. In fact, the majority swayed in favour of changing the behaviour
of pip install post one of your comments on Github.

I'll be happier *only* seeing in change the behaviour of --upgrade and not
--target or pip install. It reduces the number of things that changes from
3 to 1. Much easier to discuss about.

I’m not going to repeat the entire post, but I just made a fairly lengthy
> comment at https://github.com/pypa/pip/issues/3786#issuecomment-228611906 but
> to try and boil it down to a few points:
>

Thanks for this.


> * ``pip install —upgrade`` is not a good security mechanism, relying on it
> is inconsistent at best. If we want to support trying to keep people on
> secure versions of software we need a better mechanism than this anyways,
> so we shouldn’t let it influence our choice here.
>

AFAIK, this was the only outstanding concern raised against having a
non-eager (conservative) upgrade strategy.

* For the general case, it’s not going to matter a lot which way we go, but
> not upgrading has the greatest chance of not breaking *already installed
> software*.
>

I strongly agree with this. Another thing worth a mention is that it's
easier to get the lower bounds of your requirements correct, rather than
upper bounds.


> * For the hard-to-upgrade case, the current behavior is so bad that people
> are outright attempting to subvert the way pip typically behaviors, *AND*
> advocating for other’s to do the same, in an attempt to escape that
> behavior. I think that this is not a good place to be in.
>

Ditto.

—
>
> Donald Stufft
>

Happy-to-see-Donald's-response-ly,
Pradyun Gedam
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

I’ll be happier *only* seeing in change

s/*only* seeing/seeing *only*/​
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

Hello List!

I feel it’s fine to hold back the other changes for later but the
upgrade-strategy change should get shipped out to the world as quickly as
possible. Even how the change is exposed the user can also be discussed
later.

I request the list members to focus on *only* the change of the default
upgrade strategy to be non-eager.

Does anyone have any concerns regarding the change of the default upgrade
strategy to be non-eager? If not, let’s get *just* that shipped out as soon
as possible.

Cheers,
Pradyun Gedam

On Mon, 27 Jun 2016 at 12:02 Pradyun Gedam pradyu...@gmail.com
<http://mailto:pradyu...@gmail.com> wrote:

On Sun, 26 Jun 2016 at 23:02 Donald Stufft <don...@stufft.io> wrote:
>
>>
>> On Jun 25, 2016, at 6:25 AM, Pradyun Gedam <pradyu...@gmail.com> wrote:
>>
>> There is currently a proposal to change the behaviour to pip install to
>> upgrade a package that is passed even if it is already installed.
>>
>> This behaviour change is accompanied with a change in the upgrade
>> strategy - pip would stop “eagerly” upgrading dependencies and would become
>> more conservative, upgrading a dependency only when it doesn’t meet lower
>> constraints of the newer version of a parent. Moreover, the behaviour of pip
>> install --target would also be changed so that --upgrade no longer
>> affects it.
>>
>> I think bundling these two changes (and I think I might have been the one
>> that originally suggested it) is making this discussion harder than it
>> needs to be as folks are having to fight on multiple different fronts at
>> once. I think the change to the default behavior of pip install is
>> dependent on the change to —upgrade, so I suggest we focus on the change to
>> —upgrade first, changing from a “recursive” to a “conservative” strategy.
>> Once we get that change figured out and landed then we can worry about what
>> to do with pip install.
>>
>
> You were. In fact, the majority swayed in favour of changing the behaviour
> of pip install post one of your comments on Github.
>
> I'll be happier *only* seeing in change the behaviour of --upgrade and not
> --target or pip install. It reduces the number of things that changes from
> 3 to 1. Much easier to discuss about.
>
> I’m not going to repeat the entire post, but I just made a fairly lengthy
>> comment at https://github.com/pypa/pip/issues/3786#issuecomment-228611906 but
>> to try and boil it down to a few points:
>>
>
> Thanks for this.
>
>
>> * ``pip install —upgrade`` is not a good security mechanism, relying on
>> it is inconsistent at best. If we want to support trying to keep people on
>> secure versions of software we need a better mechanism than this anyways,
>> so we shouldn’t let it influence our choice here.
>>
>
> AFAIK, this was the only outstanding concern raised against having a
> non-eager (conservative) upgrade strategy.
>
> * For the general case, it’s not going to matter a lot which way we go,
>> but not upgrading has the greatest chance of not breaking *already
>> installed software*.
>>
>
> I strongly agree with this. Another thing worth a mention is that it's
> easier to get the lower bounds of your requirements correct, rather than
> upper bounds.
>
>
>> * For the hard-to-upgrade case, the current behavior is so bad that
>> people are outright attempting to subvert the way pip typically behaviors,
>> *AND* advocating for other’s to do the same, in an attempt to escape that
>> behavior. I think that this is not a good place to be in.
>>
>
> Ditto.
>
> —
>>
>> Donald Stufft
>>
>
> Happy-to-see-Donald's-response-ly,
> Pradyun Gedam
>
​
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

I think it's useful to see what other tools and package managers do. Doing
something like them because they do it is not a good reason. Doing it
because it's better UX is a good reason.

I like what git does, printing a (possibly long) message in a version cycle
to warn users that the behavior would be changing in a future version and
how they can/should act on it. It has a clean transition period that allows
users to make educated decisions.

I, personally, am more in the favor of providing a `--upgrade-strategy
(eager/non-eager)` flag (currently defaulting to eager) with `--upgrade`
with the the warning message printing as above followed by a switch to
non-eager.

These two combined is IMO the best way to do a transition to non-eager
upgrades by default.

That said, a change to non-eager upgrades be could potentially cause a
security vulnerability because a security package is kept at an older
version. This asks if we should default to non-eager upgrades. This
definitely something that should be addressed first, before we even talk
about the transition. I'm by no means in a position to make an proper
response and decision on this but someone else on this thread probably is.
On Sun, 26 Jun 2016 at 10:59 Nick Coghlan  wrote:

> On 25 June 2016 at 21:59, Robert Collins 
> wrote:
> > Lastly, by defaulting to non-recursive upgrades we're putting the
> > burden on our users to identify sensitive components and manage them
> > much more carefully.
>
> Huh, true. I was looking at this proposal from the point of view of
> container build processes where the system packages are visible from
> the venv, and there the "only install what I tell you, and only
> upgrade what you absolutely have to" behaviour is useful (especially
> since you're mainly doing it in the context of generating a new
> requirements.txt that is used to to the *actual* build).
>
> However, I now think Robert's right that that's the wrong way to look
> at it - I am *not* a suitable audience for the defaults, since we can
> adapt our automation pipeline to whatever combination of settings pip
> tells us we need to get the behaviour we want (as long as we're given
> suitable deprecation periods to adjust to any behavioural changes, and
> we can largely control that ourselves by controlling when we upgrade
> pip, including patching it if absolutely necessary).
>
> By contrast, for folks that *aren't* using something like VersionEye
> or requires.io to stay on top of security updates, "always run the
> latest version of everything, and try to keep up with that upgrade
> treadmill" really is the safest way to go, and that's what the current
> eager upgrade behaviour provides.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

Hello List!

There is currently a proposal to change the behaviour to pip install to
upgrade a package that is passed even if it is already installed.

This behaviour change is accompanied with a change in the upgrade strategy
- pip would stop “eagerly” upgrading dependencies and would become more
conservative, upgrading a dependency only when it doesn’t meet lower
constraints of the newer version of a parent. Moreover, the behaviour of pip
install --target would also be changed so that --upgrade no longer affects
it.

A PEP-style write-up (
https://gist.github.com/pradyunsg/4c9db6a212239fee69b429c96cdc3d73)
documents the reasoning behind this proposed change, what the change is and
some alternate proposals that were rejected in the discussions.

This is a request for comments on the pull-request implementing this
proposal (https://github.com/pypa/pip/pull/3806) for your views, thoughts
and possible concerns related to it.

—

Cheers,
Pradyun Gedam
​
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] If you want wheel to be successful, provide a build server.

[Pradyun Gedam]

(this is my first mail to the list, hopefully, this goes through)

Hey Matthew,

FYI, the --prefer-binary flag that you mention has come up in earlier
discussions and already has an issue over at pip’s github repo (
https://github.com/pypa/pip/issues/3785).

Regards,
Pradyun
​

On Tue, 14 Jun 2016 at 00:21 Matthew Brett  wrote:

> Hi,
>
> On Thu, May 26, 2016 at 11:47 AM, Donald Stufft  wrote:
> >
> >> On May 26, 2016, at 2:41 PM, Matthew Brett 
> wrote:
> >>
> >> On Thu, May 26, 2016 at 2:28 PM, Daniel Holth  wrote:
> >>> Maybe there could be a way to say "the most recent release that has a
> wheel
> >>> for my platform". That would help with the problem of binaries not
> being
> >>> available concurrently with a new source distribution.
> >>
> >> Yes, that would certainly help get over some of the immediate problems.
> >>
> >> Sorry for my ignorance - but does ``--only-binary`` search for an
> >> earlier release with a binary or just bomb out if the latest release
> >> does not have a binary?   It would also be good to have a flag to say
> >> "if this is pure Python go ahead and use the source, otherwise error".
> >>   Otherwise I guess we'd have to rely on everyone with a pure Python
> >> package generating wheels.
> >
> > I believe it would find the latest version that has a wheel available,
> > I could be misremembering though.
>
> Reflecting a bit more on this - how easy would be be to add a flag
> ``--prefer-binary`` that would have the effect of:
>
> * Installing a binary wheel for current release if available, otherwise;
> * Checking previous release for binary wheel, installing if present,
> otherwise;
> * Install from sdist
>
> I think that would help a great deal in reducing surprise in wheel
> installs while we get better at putting up wheels at the same time as
> sdists.
>
> Cheers,
>
> Matthew
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig