Re: [Distutils] Implementing large changes in small increments
Has PyPA considered contacting GitHub support? I'm happy to do the same since I've wanted this for a while myself on other projects. On Fri, Mar 6, 2015 at 5:15 PM, Paul Moore wrote: > On 6 March 2015 at 23:01, Donald Stufft wrote: >> Tooling wise, Github PRs work well for us. I don’t (and I don’t believe that >> any of the other core devs) have any major issues with them. >> >> Github issues on the other hand, they function “OK” but it would be nice to >> have something that we can allow anyone to modify the state of tickets to >> help with triage. However even this isn’t a super pressing concern because >> our ticket count is small enough that I don’t think there’s likely to be too >> many to be handled by people commenting on issues and a core team coming in >> to change things. However if someone has a proposal for a different issue >> tracker (and plans for how to migrate to it), personally I’d be willing to >> listen. > > I'm also fine with github. I don't have an issue with the issue > tracker, although as Donald says it would be helpful if it had a > concept of "tracker privileges" separate from "core committer". But > that's *not* a big enough concern to me that I'd want to go to a > different tool. > > Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments
On 6 March 2015 at 23:01, Donald Stufft wrote: > Tooling wise, Github PRs work well for us. I don’t (and I don’t believe that > any of the other core devs) have any major issues with them. > > Github issues on the other hand, they function “OK” but it would be nice to > have something that we can allow anyone to modify the state of tickets to > help with triage. However even this isn’t a super pressing concern because > our ticket count is small enough that I don’t think there’s likely to be too > many to be handled by people commenting on issues and a core team coming in > to change things. However if someone has a proposal for a different issue > tracker (and plans for how to migrate to it), personally I’d be willing to > listen. I'm also fine with github. I don't have an issue with the issue tracker, although as Donald says it would be helpful if it had a concept of "tracker privileges" separate from "core committer". But that's *not* a big enough concern to me that I'd want to go to a different tool. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments
> On Mar 6, 2015, at 5:43 PM, Ian Cordasco wrote: > > On Fri, Mar 6, 2015 at 4:04 PM, Nick Coghlan wrote: >> >> On 6 Mar 2015 22:10, "Ben Finney" wrote: >>> >>> Nick Coghlan writes: >>> CPython uses the Reitveld instance integrated with bugs.python.org, and has the same problem as pip: incremental changes are a pain to publish, review, and merge, so we review and accept monolithic patches instead (cf the problem statement in https://www.python.org/dev/peps/pep-0462/) >>> >>> Fair enough. I don't know of a good code review tool for Mercurial. >> >> I'd like to ensure Kallithea fits that bill, but the actual work on that >> seems to mostly be driven by the folks at Unity3D at the moment. >> >> In the meantime, Phabricator is a decent choice if you just want to use an >> existing GitHub independent tool that works with either git or Mercurial. >> pip adopting that workflow would also be a good proof of concept for >> Donald's proposal to also adopt that workflow for CPython (or at least its >> support repos). >> While the main UI is very busy, I've actually quite liked my own experience with Gerrit for http://gerrit.beaker-project.org/ >>> >>> My understanding is that Gerrit makes it tedious to review a sequence of >>> revisions, in proportion to the number of revisions in the sequence. >> >> When the goal is to break a change up into small, independently reviewable >> changes that's generally a feature rather than a defect :) >> >>> If >>> I understand correctly, such a sequence must have separate reviews for >>> every revision, and an aggregate of all the changes is not available to >>> the reviewer. >> >> Correct, but my understanding is that when using it in tandem with GitHub, >> there's nothing stopping you from also submitting a PR if a reviewer wants >> an all-inclusive view. >> >>> I'm impressed by GitLab's code review tool UI; see an example at >>> https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/344/diffs>. >>> The merge request page has tabs for the discussion, the commit log, and >>> the overall diff – and you choose from inline diff or side-by-side diff. >>> >>> GitLab is free software, including all its tools; anyone can set up a >>> GitLab instance and the project data can move from one instance to >>> another without loss. For the purposes of the past thread where some >>> proposed migrating to the proprietary lock-in site GitHub, those >>> objections don't exist with GitLab: a project can migrate to a different >>> host and keep all the valuable data it accumulated. >>> >>> A move to GitLab would be unobjectionable, in my view. That it has good >>> code review features would help the issues in this thread too. >> >> It doesn't have the integration with other services and the low barriers to >> contribution that are the main reasons a lot of projects prefer GitHub. >> >> Of course, when your problem is already "we're receiving more contributions >> than we can process effectively", deciding to require a slightly higher >> level of engagement in order to submit a change for consideration isn't >> necessarily a bad thing :) >> >>> If anyone knows of equivalent hosting for Mercurial with equivalent code >>> review tools under free-software terms with no lock-in, that would be >>> even better I think. >> >> That's what I'd like forge.python.org to eventually be for the core Python >> ecosystem, but we don't know yet whether that's going to be an entirely >> self-hosted Kallithea instance (my preference) or a Phabricator instance >> backed by GitHub (Donald's preference). >> >> Hence my suggestion that a "forge.pypa.io" Phabricator instance might be an >> interesting thing to set up and start using for pip. Donald's already done >> the research on that in the context of >> https://www.python.org/dev/peps/pep-0481/ and for pip that's a matter of >> "just add Phabricator" without having to migrate anything (except perhaps >> the issues if folks wanted to do that). >> >> Cheers, >> Nick. >> >>> >>> -- >>> \ “Don't be misled by the enormous flow of money into bad defacto | >>> `\standards for unsophisticated buyers using poor adaptations of | >>> _o__) incomplete ideas.” —Alan Kay | >>> Ben Finney >>> >>> ___ >>> Distutils-SIG maillist - Distutils-SIG@python.org >>> https://mail.python.org/mailman/listinfo/distutils-sig >> >> >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> > > I'm fairly concerned that what has turned into a "how can we increase > the feedback received for people submitting pull requests" has turned > into a bike shed moment for using F/LOSS tooling instead of GitHub > when the cores who actually work on the project have already expressed > a disinterest in moving and a satisfaction with GitHub. > > GitLa
Re: [Distutils] Implementing large changes in small increments
On Fri, Mar 6, 2015 at 4:04 PM, Nick Coghlan wrote: > > On 6 Mar 2015 22:10, "Ben Finney" wrote: >> >> Nick Coghlan writes: >> >> > CPython uses the Reitveld instance integrated with bugs.python.org, >> > and has the same problem as pip: incremental changes are a pain to >> > publish, review, and merge, so we review and accept monolithic patches >> > instead (cf the problem statement in >> > https://www.python.org/dev/peps/pep-0462/) >> >> Fair enough. I don't know of a good code review tool for Mercurial. > > I'd like to ensure Kallithea fits that bill, but the actual work on that > seems to mostly be driven by the folks at Unity3D at the moment. > > In the meantime, Phabricator is a decent choice if you just want to use an > existing GitHub independent tool that works with either git or Mercurial. > pip adopting that workflow would also be a good proof of concept for > Donald's proposal to also adopt that workflow for CPython (or at least its > support repos). > >> > While the main UI is very busy, I've actually quite liked my own >> > experience with Gerrit for http://gerrit.beaker-project.org/ >> >> My understanding is that Gerrit makes it tedious to review a sequence of >> revisions, in proportion to the number of revisions in the sequence. > > When the goal is to break a change up into small, independently reviewable > changes that's generally a feature rather than a defect :) > >> If >> I understand correctly, such a sequence must have separate reviews for >> every revision, and an aggregate of all the changes is not available to >> the reviewer. > > Correct, but my understanding is that when using it in tandem with GitHub, > there's nothing stopping you from also submitting a PR if a reviewer wants > an all-inclusive view. > >> I'm impressed by GitLab's code review tool UI; see an example at >> https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/344/diffs>. >> The merge request page has tabs for the discussion, the commit log, and >> the overall diff – and you choose from inline diff or side-by-side diff. >> >> GitLab is free software, including all its tools; anyone can set up a >> GitLab instance and the project data can move from one instance to >> another without loss. For the purposes of the past thread where some >> proposed migrating to the proprietary lock-in site GitHub, those >> objections don't exist with GitLab: a project can migrate to a different >> host and keep all the valuable data it accumulated. >> >> A move to GitLab would be unobjectionable, in my view. That it has good >> code review features would help the issues in this thread too. > > It doesn't have the integration with other services and the low barriers to > contribution that are the main reasons a lot of projects prefer GitHub. > > Of course, when your problem is already "we're receiving more contributions > than we can process effectively", deciding to require a slightly higher > level of engagement in order to submit a change for consideration isn't > necessarily a bad thing :) > >> If anyone knows of equivalent hosting for Mercurial with equivalent code >> review tools under free-software terms with no lock-in, that would be >> even better I think. > > That's what I'd like forge.python.org to eventually be for the core Python > ecosystem, but we don't know yet whether that's going to be an entirely > self-hosted Kallithea instance (my preference) or a Phabricator instance > backed by GitHub (Donald's preference). > > Hence my suggestion that a "forge.pypa.io" Phabricator instance might be an > interesting thing to set up and start using for pip. Donald's already done > the research on that in the context of > https://www.python.org/dev/peps/pep-0481/ and for pip that's a matter of > "just add Phabricator" without having to migrate anything (except perhaps > the issues if folks wanted to do that). > > Cheers, > Nick. > >> >> -- >> \ “Don't be misled by the enormous flow of money into bad defacto | >> `\standards for unsophisticated buyers using poor adaptations of | >> _o__) incomplete ideas.” —Alan Kay | >> Ben Finney >> >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig > > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > I'm fairly concerned that what has turned into a "how can we increase the feedback received for people submitting pull requests" has turned into a bike shed moment for using F/LOSS tooling instead of GitHub when the cores who actually work on the project have already expressed a disinterest in moving and a satisfaction with GitHub. GitLab's UI would do nothing to improve review management. Phabricator, while nice, again adds yet another layer to the piece for new contributors to involve themselves in. GitHub is one monol
Re: [Distutils] Implementing large changes in small increments
On Fri, Mar 6, 2015 at 2:17 AM, anatoly techtonik wrote: > Stealing some packaging code from Go and tweaking may not be that bad idea. =) > At least they use code review system to let new people learn and old > people share. Yep and most of the people working on Go are paid to do so. Sharing information through code review is necessary inside a corporation and worth being fired over if you don't. > On Fri, Mar 6, 2015 at 5:51 AM, Ian Cordasco > wrote: >> Wait, I have an idea. Let's rewrite pip in Rust! ;) >> >> On Thu, Mar 5, 2015 at 7:16 PM, Greg Ewing >> wrote: >>> On 03/06/2015 11:06 AM, Ben Finney wrote: >>> That's “small changes only, otherwise your change gets rejected”, of course. >>> >>> >>> Yes, otherwise submitting a patch that replaces the entire source >>> code of Python with Ruby would be a sure-fire way to get it >>> accepted. :-) >>> >>> -- >>> Greg >>> >>> >>> ___ >>> Distutils-SIG maillist - Distutils-SIG@python.org >>> https://mail.python.org/mailman/listinfo/distutils-sig >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig > > > > -- > anatoly t. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Granting permissions to Xavier Fernandez and Marc Abramowitz?
On Fri, Mar 6, 2015 at 5:56 AM, Piotr Dobrogost wrote: > Hi > > As an external observer of pip project at github I see two men, namely > Xavier Fernandez (https://github.com/xavfernandez) and Marc Abramowitz > (https://github.com/msabramo) with many valuable contributions. I > think it would be beneficial if they had been granted some more > permissions to the project/repo. > > Regards, > Piotr Dobrogost > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig It seems incredibly inappropriate for anyone except an existing core to be proposing new cores. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Getting more momentum for pip
On 7 Mar 2015 06:44, "Ian Cordasco" wrote: > On Fri, Mar 6, 2015 at 5:55 AM, Paul Moore wrote: > > > > The problem with discussing this sort of thing is that it's *very* > > wide-ranging, and tends to produce huge rambling mega-threads[1] when > > discussed in a public list. I'm not advocating any sort of private > > cabal deciding the fate of pip, but maybe somewhere where the core > > devs could agree their *own* opinions before having to face the public > > wouldn't be such a bad thing. That's more or less what I'd expected > > the pypa-dev list to be (as a parallel to the python-dev list) but it > > doesn't feel like it's turned out that way, maybe because it doesn't > > have a clear enough charter, or maybe because there's no obvious > > *other* place to direct people to for off-topic posts (like > > python-list is for python-dev). > > So sometimes private cabals need to be made in order to get a basis of > what is reasonable. The WSGI working group tried to do that but that > failed after about a week as more people tried to join the cabal and > were allowed to do so. It's worth noting that CPython didn't get public source control until it was already around 9 years old (see the What's New for Python 2.0). My understanding is also that the architecture & philosophy for CPython were very much set by the original Python Labs crew (Guido, Tim Peters, Barry Warsaw, Fred Drake) when they worked for Zope Corporation, just as the direction of beaker-project.org is very much governed by what the core team that works full time on it for Red Hat wants to do. Confusing "open source" and even "open governance" with "no hierarchy" is a common mistake, when the only essential requirement is that anyone is welcome to observe and even suggest changes, whether to artefacts (open source) or decision making processes (open governance). The one thing that potential (and current!) contributors have to accept is that the existing contributors are the ones that decide between "yes", "no", and "maybe, let's discuss it some more", regardless of whether the proposed change is to code, processes, or who has the authority to accept changes. All of which can be summarised in the phrase: "Those that do the work, make the rules" :) Cheers, Nick. > > > Or maybe grand designs are a distraction in themselves, and none of > > the core devs being interested in a PR means just that - not that they > > don't have the time, or that the use case isn't valid, or anything > > else. Just that they aren't interested, sorry. > > > > [1] Please, don't start a rambling mega-thread from *this* post :-) > > > > Paul > > > > PS I just spend way too long composing this email, and now I'm burned > > out. Maybe my time would have been better spent commenting on a couple > > of PRs... > > Go rest. These discussions can exhaust even the best rested of us. > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Granting permissions to Xavier Fernandez and Marc Abramowitz?
> On Mar 6, 2015, at 6:56 AM, Piotr Dobrogost > wrote: > > Hi > > As an external observer of pip project at github I see two men, namely > Xavier Fernandez (https://github.com/xavfernandez) and Marc Abramowitz > (https://github.com/msabramo) with many valuable contributions. I > think it would be beneficial if they had been granted some more > permissions to the project/repo. I’m not going to respond to the actual content of this, but I’d like to make a plea. If you think someone should be a core on one of the PyPA projects, I recommend talking to one of the existing core developer privately instead of posting publicly. Historically these projects decide on core members privately so that people’s concerns or comments on potential core members are not public. This is done for three main reasons: 1) By keeping things private core members can fully express their opinions on whether or not that person is a good candidate for core member, or whether they might be in the future if they work on something that is potentially concerning. 2) It doesn’t put any potential core members in a situation where they are feeling judged or on trial or whatever colloquium you want to insert here. 3) It allows us to privately communicate with the potential team member about what (if anything) they need to improve on if they’d like to be a core member. We don’t have defined guidelines as to what we look for in a core team member, perhaps we should, generally though it’s been looking out for people who seem to be trying to help out with the efforts that are lacking manpower. Just to be clear, this has nothing to do with either Xavier or Marc, I’m just asking that in the future if someone thinks that they notice someone who might be a good candidate for being on the core team. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Granting permissions to Xavier Fernandez and Marc Abramowitz?
On 7 Mar 2015 05:41, "Piotr Dobrogost" wrote: > > Hi > > As an external observer of pip project at github I see two men, namely > Xavier Fernandez (https://github.com/xavfernandez) and Marc Abramowitz > (https://github.com/msabramo) with many valuable contributions. I > think it would be beneficial if they had been granted some more > permissions to the project/repo. How many *other* people have they helped have contributions accepted? That's one of the most important responsibilities of a core reviewer, rather than writing new code themselves. I also can't recall seeing any feedback from them on this list regarding the metadata 2.0 design PEPs or the TUF metadata PEPs, which suggests there may be a risk that they're currently focused primarily on attending to their own needs, rather than wanting to help guide the evolution of the overall Python packaging ecosystem. While that's not an issue for *submitting* change requests to pip, it's a potential problem for *accepting* them, as managing that evolution has essentially been the responsibility of the pip development team (and PyPA in general) since PEP 453 made the decision to bundle pip with the reference interpreter by default. Regards, Nick. > > Regards, > Piotr Dobrogost > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments
On 6 Mar 2015 22:10, "Ben Finney" wrote: > > Nick Coghlan writes: > > > CPython uses the Reitveld instance integrated with bugs.python.org, > > and has the same problem as pip: incremental changes are a pain to > > publish, review, and merge, so we review and accept monolithic patches > > instead (cf the problem statement in > > https://www.python.org/dev/peps/pep-0462/) > > Fair enough. I don't know of a good code review tool for Mercurial. I'd like to ensure Kallithea fits that bill, but the actual work on that seems to mostly be driven by the folks at Unity3D at the moment. In the meantime, Phabricator is a decent choice if you just want to use an existing GitHub independent tool that works with either git or Mercurial. pip adopting that workflow would also be a good proof of concept for Donald's proposal to also adopt that workflow for CPython (or at least its support repos). > > While the main UI is very busy, I've actually quite liked my own > > experience with Gerrit for http://gerrit.beaker-project.org/ > > My understanding is that Gerrit makes it tedious to review a sequence of > revisions, in proportion to the number of revisions in the sequence. When the goal is to break a change up into small, independently reviewable changes that's generally a feature rather than a defect :) > If > I understand correctly, such a sequence must have separate reviews for > every revision, and an aggregate of all the changes is not available to > the reviewer. Correct, but my understanding is that when using it in tandem with GitHub, there's nothing stopping you from also submitting a PR if a reviewer wants an all-inclusive view. > I'm impressed by GitLab's code review tool UI; see an example at > https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/344/diffs>. > The merge request page has tabs for the discussion, the commit log, and > the overall diff – and you choose from inline diff or side-by-side diff. > > GitLab is free software, including all its tools; anyone can set up a > GitLab instance and the project data can move from one instance to > another without loss. For the purposes of the past thread where some > proposed migrating to the proprietary lock-in site GitHub, those > objections don't exist with GitLab: a project can migrate to a different > host and keep all the valuable data it accumulated. > > A move to GitLab would be unobjectionable, in my view. That it has good > code review features would help the issues in this thread too. It doesn't have the integration with other services and the low barriers to contribution that are the main reasons a lot of projects prefer GitHub. Of course, when your problem is already "we're receiving more contributions than we can process effectively", deciding to require a slightly higher level of engagement in order to submit a change for consideration isn't necessarily a bad thing :) > If anyone knows of equivalent hosting for Mercurial with equivalent code > review tools under free-software terms with no lock-in, that would be > even better I think. That's what I'd like forge.python.org to eventually be for the core Python ecosystem, but we don't know yet whether that's going to be an entirely self-hosted Kallithea instance (my preference) or a Phabricator instance backed by GitHub (Donald's preference). Hence my suggestion that a "forge.pypa.io" Phabricator instance might be an interesting thing to set up and start using for pip. Donald's already done the research on that in the context of https://www.python.org/dev/peps/pep-0481/ and for pip that's a matter of "just add Phabricator" without having to migrate anything (except perhaps the issues if folks wanted to do that). Cheers, Nick. > > -- > \ “Don't be misled by the enormous flow of money into bad defacto | > `\standards for unsophisticated buyers using poor adaptations of | > _o__) incomplete ideas.” —Alan Kay | > Ben Finney > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Getting more momentum for pip
On Fri, Mar 6, 2015 at 5:55 AM, Paul Moore wrote: > On 5 March 2015 at 16:38, Marc Abramowitz wrote: >> This makes me think that the folks who review the PRs are overburdened >> and/or the process needs a bit more structure (e.g.: each person thinks >> someone else is going to review the PR and so no one does it). > > One thing I, personally, find difficult when reviewing PRs > (specifically feature requests) is the fact that I usually don't > actually have a *need* for the functionality being proposed. It's very > easy for me to say "this doesn't help me personally, so I'll ignore > it", but that is ducking a big part of the responsibility of being a > core committer. But forming a view on something I've no experience of > or direct interest in is *hard*, and takes a lot of time. Discussions > tend to involve a lot of people with strong opinions (e.g. the PR > author) who can't move the change forward, and a few people with > weaker opinions (e.g. me :-)) who can. It's very easy to think "just > accept it because it helps someone". But that's a cop-out and > long-term isn't a sustainable approach. +1. This is a problem I have with Flake8. People keep asking for more command-line arguments because "It's just one more option. It won't hurt anyone." But Flake8 is another project without a great set of tests. It would be easy to say "Yeah sure, just this one other option that only one person has ever asked for" but there's only ever one person reviewing pull requests - me. It's also not sustainable to keep adding poorly named command-line flags. > It's not "thinking someone else will review the PR", it's more making > a conscious decision on how much energy and effort I'm willing to put > into a PR that doesn't have any benefit for me. (And even just > *discussing* a PR can be a lot of energy, it's not easy to politely > explain to someone that you don't think their use case, that they went > to a lot of trouble writing a PR for, isn't worth it). > > What would help a *lot* is some sort of agreement on what pip is, and > what its core goals are. Something similar to what it means to be > "pythonic" for the Python language itself. At the moment, I don't > think this is very clearly understood even within the core dev group > (so external contributors have no hope...) And for me, it'd help avoid > the endless debates that often start with the phrase "pip should..." +10 > For example, is the lack of a programmable API an issue for pip? I > think it is, and having people able to write their own tools that use > pip's finder, or its wheel installer, is a (long term) goal for pip, > rather than, say, continually adding more pip subcommands. But I don't > know if that's the consensus. And to my knowledge, no 3rd party PRs > have *ever* been of the form "Encapsulate pip's functionality X in a > clean API so I can use it in my script"... If pip is ever refactored appropriately (which I acknowledge is not a trivial condition to meet), maybe then pip could consider presenting a public API, but I think there are currently too many people who already reach into pip to ignore the need for such an interface. Perhaps the answer is, as pip is refactored, to create libraries that are then vendored into pip and that people can install independently to do that one thing they need to do. > Or is the "pip search" command a wart that should be removed because > it isn't pip's job to do PyPI searches? There's some low-hanging fruit > if a more focused tool is the goal... There's also how many different replacements for "pip search" on PyPI? > Or should pip give you tools to replicate your current environment > (pip freeze, requirements files)? What about "remove anything *not* in > this requirement file"? Personally, I only use requirements files to > bundle up "install this lot of stuff". I don't write the sort of thing > where a "pin every dependency" philosophy is appropriate, so freeze > isn't something I use. But lots of people do, so what's the workflow > that pip freeze supports? > > The problem with discussing this sort of thing is that it's *very* > wide-ranging, and tends to produce huge rambling mega-threads[1] when > discussed in a public list. I'm not advocating any sort of private > cabal deciding the fate of pip, but maybe somewhere where the core > devs could agree their *own* opinions before having to face the public > wouldn't be such a bad thing. That's more or less what I'd expected > the pypa-dev list to be (as a parallel to the python-dev list) but it > doesn't feel like it's turned out that way, maybe because it doesn't > have a clear enough charter, or maybe because there's no obvious > *other* place to direct people to for off-topic posts (like > python-list is for python-dev). So sometimes private cabals need to be made in order to get a basis of what is reasonable. The WSGI working group tried to do that but that failed after about a week as more people tried to join the cabal and were allowed t
[Distutils] Granting permissions to Xavier Fernandez and Marc Abramowitz?
Hi As an external observer of pip project at github I see two men, namely Xavier Fernandez (https://github.com/xavfernandez) and Marc Abramowitz (https://github.com/msabramo) with many valuable contributions. I think it would be beneficial if they had been granted some more permissions to the project/repo. Regards, Piotr Dobrogost ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments
Stealing some packaging code from Go and tweaking may not be that bad idea. =) At least they use code review system to let new people learn and old people share. On Fri, Mar 6, 2015 at 5:51 AM, Ian Cordasco wrote: > Wait, I have an idea. Let's rewrite pip in Rust! ;) > > On Thu, Mar 5, 2015 at 7:16 PM, Greg Ewing > wrote: >> On 03/06/2015 11:06 AM, Ben Finney wrote: >> >>> That's “small changes only, otherwise your change gets rejected”, of >>> course. >> >> >> Yes, otherwise submitting a patch that replaces the entire source >> code of Python with Ruby would be a sure-fire way to get it >> accepted. :-) >> >> -- >> Greg >> >> >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig -- anatoly t. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Getting more momentum for pip
On Thu, Mar 5, 2015 at 11:11 PM, Ian Cordasco wrote: > And for CI, we need people who will help with the windows CI solution > on more than one front clearly. https://ci.appveyor.com/ works for open source projects. -- anatoly t. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Getting more momentum for pip
On Thu, Mar 5, 2015 at 7:38 PM, Marc Abramowitz wrote: > - Add more computer automation > > #3 seems most appealing to me, but of course it requires humans to develop > it in the first place, but at least it's an investment that could pay > dividends. On page https://bitbucket.org/techtonik/python-stdlib/src there is a working proof-of-concept that fetches patch files from bugs.python.org, detects filenames and sees to which module the patch belongs (module layout is described in .json file). It is possible to tweak the script to also lookup who are the recent authors for the module files. This will allow to them to review automatically. -- anatoly t. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Granting permissions to Xavier Fernandez and Marc Abramowitz?
Hi As an external observer of pip project at github I see two men, namely Xavier Fernandez (https://github.com/xavfernandez) and Marc Abramowitz (https://github.com/msabramo) with many valuable contributions. I think it would be beneficial if they had been granted some more permissions to the project/repo. Regards, Piotr Dobrogost ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Installing a file into sitepackages
On Fri, Feb 20, 2015 at 1:49 PM, Stuart Axon
wrote:
> Hi,
>In my project, I install a .pth file into site-packages, I use the
> data_files... in Ubuntu this seems to work OK, but in a Windows VM the file
> doesn't seem to be being installed:
>
> setup(
>
># Install the import hook
> data_files=[
> (site_packages_path, ["vext_importer.pth"] if
> environ.get('VIRTUAL_ENV') else []),
> ],
> )
>
>
> - Is there a better way to do this ?
>
>
>
> I realise it's a bit odd installing a .pth - my project is to allow certain
> packages to use the system site packages from a virtualenv -
> https://github.com/stuaxo/vext
Hi Stuart,
I know this is old so sorry to anyone else. But since no one
replied--I haven't looked too closely at what it is you're trying to
accomplish. But whatever the reason, that seems like a
reasonable-enough way to me if you need to get a .pth file installed
into site-packages. I'm not sure why it isn't working for you in
WIndows but it wasn't clear what the problem was. I just tried this
in a Windows VM and it worked fine?
Best,
Erik
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments (was: Getting more momentum for pip)
> On Mar 6, 2015, at 8:55 AM, Jeremy Stanley wrote: > > On 2015-03-06 21:37:31 +1000 (+1000), Nick Coghlan wrote: > [...] >> I've never used Gerrit in the OpenStack context though, so I don't >> know if Donald dislikes Gerrit in its own right, or just the way >> OpenStack uses it. > [...] > > Having talked with him about it regularly, I gather that he (and > others) dislike the Gerrit/LKML "rebase, revise and refine your > patch" workflow, instead preferring a Github-like "incrementally > build on your pull request with new commits" workflow... though > presumably he can explain it in better detail. > > In my experience it comes down to a trade-off where the Github model > is easier on patch submitters because they can just keep piling > fixes for their pull request on top if it until the corresponding > topic branch is suitable to merge, while the Gerrit model is easier > on reviewers because they're reviewing a patch in context rather > than a topic branch. > >> The Beaker workflow is an example of vanilla Gerrit usage, rather >> than using OpenStack's custom fork: > [...] > > OpenStack hasn't been running a fork of Gerrit since upgrading to > 2.8 back in April 2014 (modulo a few simple backports from 2.9), and > has plans to upgrade to 2.9 next month or the month after. That's > not to say that there isn't a bunch of additional tooling and > automation built up around it (the Zuul CI system in particular) but > aside from some minimal theming and including a little Javascript to > tie outside data sources into the interface it's just plain Gerrit. > -- > Jeremy Stanley > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig In general I’m fine with Gerrit (or Gerrit like systems). I think Gerrit has a crappy looking interface, and I think that the interface is harder to use than GitHub but the process itself doesn’t bother me much. I do think that it would be better if it would review whole PRs instead of individual commits (and if you want to squash them, the tool should do a squash merge). I don’t think that pip’s problems are ones that would be solved by switching to a different code review tool. GitHub functions well for that task, we don’t require multiple core reviewers to agree, only one, so the merge button is functionally equivalent to a +1 button and then having a machine later do the merge. Our velocity isn’t near high enough to where we need separate check and merge gating or anything like that. I would be against moving away from GitHub for PRs without a really compelling reason, GitHub PRs are easy to use, and it’s popular. We reduce the barrier to entry to contributing by making our process the same as every other project on GitHub’s. A better test suite and a more comprehensive CI system is where most of our tooling problems are. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments (was: Getting more momentum for pip)
On 2015-03-06 21:37:31 +1000 (+1000), Nick Coghlan wrote: [...] > I've never used Gerrit in the OpenStack context though, so I don't > know if Donald dislikes Gerrit in its own right, or just the way > OpenStack uses it. [...] Having talked with him about it regularly, I gather that he (and others) dislike the Gerrit/LKML "rebase, revise and refine your patch" workflow, instead preferring a Github-like "incrementally build on your pull request with new commits" workflow... though presumably he can explain it in better detail. In my experience it comes down to a trade-off where the Github model is easier on patch submitters because they can just keep piling fixes for their pull request on top if it until the corresponding topic branch is suitable to merge, while the Gerrit model is easier on reviewers because they're reviewing a patch in context rather than a topic branch. > The Beaker workflow is an example of vanilla Gerrit usage, rather > than using OpenStack's custom fork: [...] OpenStack hasn't been running a fork of Gerrit since upgrading to 2.8 back in April 2014 (modulo a few simple backports from 2.9), and has plans to upgrade to 2.9 next month or the month after. That's not to say that there isn't a bunch of additional tooling and automation built up around it (the Zuul CI system in particular) but aside from some minimal theming and including a little Javascript to tie outside data sources into the interface it's just plain Gerrit. -- Jeremy Stanley ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Getting more momentum for pip
Anatoly, We already ruled out AppVeyor On Mar 6, 2015 2:11 AM, "anatoly techtonik" wrote: > On Thu, Mar 5, 2015 at 11:11 PM, Ian Cordasco > wrote: > > And for CI, we need people who will help with the windows CI solution > > on more than one front clearly. > > https://ci.appveyor.com/ works for open source projects. > > > -- > anatoly t. > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments
Nick Coghlan writes: > CPython uses the Reitveld instance integrated with bugs.python.org, > and has the same problem as pip: incremental changes are a pain to > publish, review, and merge, so we review and accept monolithic patches > instead (cf the problem statement in > https://www.python.org/dev/peps/pep-0462/) Fair enough. I don't know of a good code review tool for Mercurial. > While the main UI is very busy, I've actually quite liked my own > experience with Gerrit for http://gerrit.beaker-project.org/ My understanding is that Gerrit makes it tedious to review a sequence of revisions, in proportion to the number of revisions in the sequence. If I understand correctly, such a sequence must have separate reviews for every revision, and an aggregate of all the changes is not available to the reviewer. I'm impressed by GitLab's code review tool UI; see an example at https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/344/diffs>. The merge request page has tabs for the discussion, the commit log, and the overall diff – and you choose from inline diff or side-by-side diff. GitLab is free software, including all its tools; anyone can set up a GitLab instance and the project data can move from one instance to another without loss. For the purposes of the past thread where some proposed migrating to the proprietary lock-in site GitHub, those objections don't exist with GitLab: a project can migrate to a different host and keep all the valuable data it accumulated. A move to GitLab would be unobjectionable, in my view. That it has good code review features would help the issues in this thread too. If anyone knows of equivalent hosting for Mercurial with equivalent code review tools under free-software terms with no lock-in, that would be even better I think. -- \ “Don't be misled by the enormous flow of money into bad defacto | `\standards for unsophisticated buyers using poor adaptations of | _o__) incomplete ideas.” —Alan Kay | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Getting more momentum for pip
On 5 March 2015 at 16:38, Marc Abramowitz wrote: > This makes me think that the folks who review the PRs are overburdened > and/or the process needs a bit more structure (e.g.: each person thinks > someone else is going to review the PR and so no one does it). One thing I, personally, find difficult when reviewing PRs (specifically feature requests) is the fact that I usually don't actually have a *need* for the functionality being proposed. It's very easy for me to say "this doesn't help me personally, so I'll ignore it", but that is ducking a big part of the responsibility of being a core committer. But forming a view on something I've no experience of or direct interest in is *hard*, and takes a lot of time. Discussions tend to involve a lot of people with strong opinions (e.g. the PR author) who can't move the change forward, and a few people with weaker opinions (e.g. me :-)) who can. It's very easy to think "just accept it because it helps someone". But that's a cop-out and long-term isn't a sustainable approach. It's not "thinking someone else will review the PR", it's more making a conscious decision on how much energy and effort I'm willing to put into a PR that doesn't have any benefit for me. (And even just *discussing* a PR can be a lot of energy, it's not easy to politely explain to someone that you don't think their use case, that they went to a lot of trouble writing a PR for, isn't worth it). What would help a *lot* is some sort of agreement on what pip is, and what its core goals are. Something similar to what it means to be "pythonic" for the Python language itself. At the moment, I don't think this is very clearly understood even within the core dev group (so external contributors have no hope...) And for me, it'd help avoid the endless debates that often start with the phrase "pip should..." For example, is the lack of a programmable API an issue for pip? I think it is, and having people able to write their own tools that use pip's finder, or its wheel installer, is a (long term) goal for pip, rather than, say, continually adding more pip subcommands. But I don't know if that's the consensus. And to my knowledge, no 3rd party PRs have *ever* been of the form "Encapsulate pip's functionality X in a clean API so I can use it in my script"... Or is the "pip search" command a wart that should be removed because it isn't pip's job to do PyPI searches? There's some low-hanging fruit if a more focused tool is the goal... Or should pip give you tools to replicate your current environment (pip freeze, requirements files)? What about "remove anything *not* in this requirement file"? Personally, I only use requirements files to bundle up "install this lot of stuff". I don't write the sort of thing where a "pin every dependency" philosophy is appropriate, so freeze isn't something I use. But lots of people do, so what's the workflow that pip freeze supports? The problem with discussing this sort of thing is that it's *very* wide-ranging, and tends to produce huge rambling mega-threads[1] when discussed in a public list. I'm not advocating any sort of private cabal deciding the fate of pip, but maybe somewhere where the core devs could agree their *own* opinions before having to face the public wouldn't be such a bad thing. That's more or less what I'd expected the pypa-dev list to be (as a parallel to the python-dev list) but it doesn't feel like it's turned out that way, maybe because it doesn't have a clear enough charter, or maybe because there's no obvious *other* place to direct people to for off-topic posts (like python-list is for python-dev). Or maybe grand designs are a distraction in themselves, and none of the core devs being interested in a PR means just that - not that they don't have the time, or that the use case isn't valid, or anything else. Just that they aren't interested, sorry. [1] Please, don't start a rambling mega-thread from *this* post :-) Paul PS I just spend way too long composing this email, and now I'm burned out. Maybe my time would have been better spent commenting on a couple of PRs... ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Implementing large changes in small increments (was: Getting more momentum for pip)
On 6 March 2015 at 08:00, Ben Finney wrote: > Donald Stufft writes: > >> Sadly with how the code in pip is written, sometimes it’s just not >> reasonable to make small PRs because things are not well factored and >> changing things requires touching a lot of different areas. > > I've seen a number of other projects enforce “small revisions only, > otherwise your change gets accepted”. If actually enforced, it is a > highly successful way to get meaningful review of changes, and does not > appear to limit the scope of the eventual change. > > What does end up happening in such projects (e.g., Linux) is the > community learns how to – and teaches newcomers how to – implement large > changes as smaller refactorings, each of which results in a working > system. This is: a) a really good idea; and b) really painful without good tooling support Linux does it via emailed patchbombs, as do a lot of other open source projects which don't have a separate code review tool. That works if your contributors are used to *consuming* patches that way, but inapplicable to projects used to web based reviews. CPython uses the Reitveld instance integrated with bugs.python.org, and has the same problem as pip: incremental changes are a pain to publish, review, and merge, so we review and accept monolithic patches instead (cf the problem statement in https://www.python.org/dev/peps/pep-0462/) While the main UI is very busy, I've actually quite liked my own experience with Gerrit for http://gerrit.beaker-project.org/ (I was the dev lead for Red Hat's Beaker hardware integration testing system from Oct 2012 until mid 2014, and the product owner until a couple of weeks ago). I've never used Gerrit in the OpenStack context though, so I don't know if Donald dislikes Gerrit in its own right, or just the way OpenStack uses it. That means one option potentially worth exploring might be http://gerrithub.io/. I haven't used GerritHub yet myself, but I'm pretty sure it lets you mix & match between GitHub PRs for simple changes and GerritHub reviews for more complex ones. The Beaker workflow is an example of vanilla Gerrit usage, rather than using OpenStack's custom fork: https://beaker-project.org/dev/guide/writing-a-patch.html#submitting-your-patch http://gerrit.beaker-project.org/#/c/4025 is an example of a fairly deep patch stack, where each patch can be reviewed independently, but later patches won't be merged until after earlier ones have been submitted. (Rebasing support is also baked directly into the tool) Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Getting more momentum for pip
On 6 March 2015 at 03:16, Marcus Smith wrote: > So I guess my suggestions boil down to: >> >> >> - Add more humans >> - Add more money to make humans more efficient >> - Add more computer automation > > > > maybe agree to always maintain < X open issues and < Y open PRs, before > adding features. > where x can vary as needed, but for starters, x=250, and y=25 sounds > reasonable. This is one of the key differences between open source projects and corporate projects. In a corporate context, it's important to keep your backlog under control by saying "we're never going to invest time in fixing this" and declare such issues as "won't fix". In a community driven open source project, the situation is different. Here, because everyone is free to spend their time however they like (or however they can persuade someone to pay them), the only real reason to "won't fix" an issue is because it's a genuinely bad idea (or based on a flawed understanding of the situation) and there's no way to redeem the suggestion. For anything else, it's often better to leave it open as an opportunity for someone to persuade the core developers that it's worth accepting the liability of maintaining that code into the future - those that do the work, make the rules. It does require a lot of education work, though. Most folks (especially when just starting to learn to code) are inclined to think that contributing more code is purely beneficial. It's not, as code is a liability that costs you long term effort to maintain. As Jack Diederich puts it: "code is the enemy and you want as little of it in your product as possible". What you actually want is to *solve people's problems in the general case*, such that the net gain in time saved across the user community vastly outweighs the maintenance cost of allowing that code to exist in the project. For mature projects with a fairly well defined scope, this means the default answer is going to be "no", and the most positive likely initial answer is "maybe". Hence the split of the core CPython mailing lists into python-dev (where the default answer is just a straight up "no") and python-ideas (where the default answer is more along the lines of "that's a potentially interesting idea, let's discuss it further"). This is arguably a flaw in more recent approaches to getting folks engaged in open source projects, with a focus on quick wins and immediate contributions. While those do exist in most projects, they're generally about implementing *existing* ideas, or improving docs, or fixing bugs. For folks that want to get a *specific* issue fixed, then what they generally need to do is to learn what the core contributors care about, and how to get them interested in addressing that problem. There isn't the simple "I'm a paying customer, this is a problem I'm reporting, we have an SLA, please resolve my issue in accordance with that" dynamic that exists in a traditional vendor relationship. While it's only been intermittently effective, one useful tactic CPython core developers have occasionally applied is a "5-for-1" review trade: if someone else reviews 5 open patches, then they'll review one that the new reviewer cares about. That's a pretty straightforward quid pro quo, and gets people into the habit of contributing time to things others are interested in so as to get time contributed back to their own issues in return. It relies on core contributors having the spare capacity to offer that deal though. pythonmentors.com describes another approach, which allows folks to self-identify as wanting to invest time in becoming a core committer/reviewer themselves, rather than just wanting to get a drive-by patch merged. Those folks then gain the benefit of getting the attention of the more active mentors in the core developer group, as well as a community of like-minded peers all attempting to learn the tricks of navigating the vagaries of CPython core development. Another way is when an existing contributors deliberately recruits someone they trust to take over a task from them. In those cases, the mentorship relationship already formed outside the particular community of contributors, and is transferred into the new context. Finally, there's straight up trading of favours: if you contribute something an existing core contributor wants themselves, a) you'll likely get their attention on that original review pretty easily; and b) if you have something *else* you want reviewed, they're far more likely to try to find the time if you're already helped them cross a lingering item off their generally voluminous todo lists. Regards, Nick. P.S. The other useful thing to do is to better educate folks on how to make the case for spending work time on upstream projects and key dependencies, *without* a specific near term business need. I'm pretty happy with this piece I recently wrote for Red Hat as an example of that kind of thing: http://community.redhat.com/blog/2015/02/the-quid-pro-quo-of-open-inf
