Re: [python-committers] number of active core devs [was: Comments on moving issues to GitHub]
On 3 June 2018 at 11:07, Guido van Rossum wrote: > Sounds to me like these are probably just past committers who are no > longer active for whatever personal reasons, and took no action when we > moved to GitHub. We basically never remove the commit bit from anyone > except by request, and I only recall seeing one such request, ever. Some of > them probably expect to come back in the future (like Neil Schemenauer > did). I recall only one person who said they refused to move to GitHub (but > AFAIK we didn't remove their commit bit from b.p.o), so I don't think that > we can blame these numbers on the move to GitHub. > OpenHub [1] shows the average rate of commits declining fairly steadily since the exceptional ~40-commits-per-day spike in September 2016 down to our current steady state of ~4 commits per day (we still get spikes up to 10+ commits per day for PyCon US and the core dev sprints, but not of the magnitude of previous sprints). Those metrics only record the actual commit rate (not the code churn rate), so some of that may be due to the switch to a PR based workflow with pre-merge CI reducing the volume of fix-up commits, and I also don't know how the switch from our patch-and-merge-forward workflow in Mercurial to the squash-merge-and-cherry-pick workflow in git affects the accounting. While the switch to GitHub does show up clearly in the "contributor" stats on OpenHub, the move to git is also when the VCS metadata started recording the committer and author information separately in a way that OpenHub can read (rather than only providing the patch author information in the commit message and NEWS entry), so someone would need to go back and extract the real pre-git contributor metrics to make that a valid comparison. On the issue management & patch review side of things, while https://bugs.python.org/issue?@template=stats does show the number of open issues with patches declining slightly post-migration, it's since leveled off and then started climbing again. So based on the numbers we're seeing, my own assessment would be that the move to GitHub didn't hurt, but it also didn't really help address the review bottleneck problem either (which surprises me as much as it does anyone else - perhaps now that patch reviews are more pleasant to engage in, we're also making them more thorough?). Cheers, Nick. [1] https://www.openhub.net/p/python -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [python-committers] Comments on moving issues to GitHub
On Sat, 2 Jun 2018 at 14:58 Ezio Melotti wrote: > Hi, > > On Sat, Jun 2, 2018 at 10:26 PM, Brett Cannon wrote: > > > > > > On Sat, 2 Jun 2018 at 12:47 Mariatta Wijaya > > wrote: > >> > >> [SNIP] > >> > >>> 2. Better support for core developers in the tracker. > >> > >> > >> Not sure what you mean by "support"? There are only two maintainers of > the > >> bug tracker, they both are also Python core developers: Brett and Ezio. > My > >> personal opinion is: they're more valuable elsewhere instead of > supporting > >> the bug tracker. At its current state, the bug tracker is not ready to > take > >> up new contributors, and it will not be easy effort to onboard new bpo > >> maintainers. > > > > > > I actually wouldn't list me as a maintainer of b.p.o. I only have passing > > knowledge of the code due to reviewing Ezio's changes to support the CLA > > bot. It used to be RDM, Ezio, and Maciej, then DRM got busy, and then > Maciej > > got busy with helping move our hosting over to OpenShift and off of our > > previously free hosting provider who sold their business (I also think > > Maciej is also busy with other things). I don't know what Ezio's > > availability currently sits at, but I have not heard from him recently. > > > > I haven't actively worked on the tracker, but I kept an eye on it and > acting when needed (e.g. yesterday I deployed a fix committed by > Berker :). > If needed I can pick it up again. > Great! So now we're tied for GitHub and b.p.o maintenance. ;) > It should also be mentioned that before us, MvL also used to work on > the tracker, and he added the Rietveld and openid integration (and > these parts are not very well maintained now). > Oh, the list of former contributors was not meant to be exhaustive, more just who I could remember during the GitHub transition days. > > > If you look at > > https://hg.python.org/tracker/roundup/shortlog/bugs.python.org there > has not > > been an update to the repo's code in 8 months but there have been > reported > > issues as recently as yesterday: > > http://psf.upfronthosting.co.za/roundup/meta/ . > > > > This is a bit misleading: > * that repo is updated only when Roundup is updated otherwise it sits > there waiting for a new release. Roundup 1.6 is going to be released > soon; > Sorry about that, I just grabbed the URL the contribution docs say to work from for Roundup and didn't notice the extra URL for configuration. > * the repo for our instance is > https://hg.python.org/tracker/python-dev/shortlog/default . This also > didn't get many commits lately, however > * the meta tracker also didn't get many new issues. Only 7 issues in > the metatracker have had any activity in the last 3 months, 2 are > about SSL failure/certificates, 3 are about email ending up in the > spam, one is about Google auth (however I'm not familiar with that > part of the code), and the last one is a minor issue with a simple fix > that needs to be tested/deployed. > > IOW there aren't many commits because there aren't that many issues > with the code and because Roundup 1.6 hasn't been released yet. > > > As mentioned above, the Roundup team is about to release Roundup 1.6. > This release drops support for Python <2.7. > AFAIU the infra team wanted to move/upgrade the machine running b.p.o > (that is currently still running Python 2.6). When that happens (and > once Roundup 1.6 is released) we will upgrade it. > > > > IOW I consider b.p.o unmaintained ATM. Mark Mangoba and the PSF > > infrastructure team can re-start the instance if it falls over, but no > one > > is working on e.g. fixing log-in issues or any of the various UX issues > we > > are all painfully aware that b.p.o has. > > > > As I said at the language summit, if people want to keep b.p.o around > then > > we need to get some volunteers to staff it. I personally would like to > see > > three people step forward and say they will work to improve b.p.o to make > > sure it functions as expected now and plan to improve it long-term. But I > > personally would settle for just two people actively working towards > making > > b.p.o a tenable solution (the three person goal is just from experience > of > > always wanting at least one backup even if someone goes on vacation or > does > > an OSS detox). > > > > It depends on what kind of maintenance we need. In its current state > b.p.o is quite stable and requires little maintenance IMHO. > I would be more inclined to agree if we didn't have so many login problems (e.g. I have needed to manually go in and set people's passwords to reset it due to issues getting password reset emails). > If instead we want to start adding (and fixing/maintaining) new > features, then the situation is different. > > For the latter to happen, I would like to first see a PEP discussing > what desired features GitHub has that Roundup doesn't and vice versa > (Nick's list and Mariatta's slides and notes are a good starting > point, but they should be consolidated
Re: [python-committers] number of active core devs [was: Comments on moving issues to GitHub]
Sounds to me like these are probably just past committers who are no longer active for whatever personal reasons, and took no action when we moved to GitHub. We basically never remove the commit bit from anyone except by request, and I only recall seeing one such request, ever. Some of them probably expect to come back in the future (like Neil Schemenauer did). I recall only one person who said they refused to move to GitHub (but AFAIK we didn't remove their commit bit from b.p.o), so I don't think that we can blame these numbers on the move to GitHub. It's definitely disturbing that we have so few active committers though -- it means that a small number of people take on a lot of the load (my intuition tells me it's even more skewed than Mariatta's numbers reveal). The best course of action seems to be to take measures to acquire new committers (and contributors), not to try and reactivate old inactive committers. On Sat, Jun 2, 2018 at 5:42 PM, Donald Stufft wrote: > Is that a 50% reduction or is that just 50% of the people who could be > active are? > > Sent from my iPhone > > > On Jun 2, 2018, at 8:33 PM, Ethan Furman wrote: > > > >> On 06/02/2018 12:46 PM, Mariatta Wijaya wrote: > >> > >> And perhaps this is to be discussed in a separate thread: even though > in the b.p.o we appear to have 170 committers, > >> really there are 90 core devs (people who has commit right to CPython > on GitHub). and out of those 90, I think only > >> about half are currently active (since the migration to GitHub). > > > > 50% reduction in activity? Ouch. > > > > -- > > ~Ethan~ > > ___ > > python-committers mailing list > > python-committers@python.org > > https://mail.python.org/mailman/listinfo/python-committers > > Code of Conduct: https://www.python.org/psf/codeofconduct/ > > ___ > python-committers mailing list > python-committers@python.org > https://mail.python.org/mailman/listinfo/python-committers > Code of Conduct: https://www.python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) ___ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [python-committers] number of active core devs [was: Comments on moving issues to GitHub]
Is that a 50% reduction or is that just 50% of the people who could be active are? Sent from my iPhone > On Jun 2, 2018, at 8:33 PM, Ethan Furman wrote: > >> On 06/02/2018 12:46 PM, Mariatta Wijaya wrote: >> >> And perhaps this is to be discussed in a separate thread: even though in the >> b.p.o we appear to have 170 committers, >> really there are 90 core devs (people who has commit right to CPython on >> GitHub). and out of those 90, I think only >> about half are currently active (since the migration to GitHub). > > 50% reduction in activity? Ouch. > > -- > ~Ethan~ > ___ > python-committers mailing list > python-committers@python.org > https://mail.python.org/mailman/listinfo/python-committers > Code of Conduct: https://www.python.org/psf/codeofconduct/ ___ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers Code of Conduct: https://www.python.org/psf/codeofconduct/
[python-committers] number of active core devs [was: Comments on moving issues to GitHub]
On 06/02/2018 12:46 PM, Mariatta Wijaya wrote: And perhaps this is to be discussed in a separate thread: even though in the b.p.o we appear to have 170 committers, really there are 90 core devs (people who has commit right to CPython on GitHub). and out of those 90, I think only about half are currently active (since the migration to GitHub). 50% reduction in activity? Ouch. -- ~Ethan~ ___ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers Code of Conduct: https://www.python.org/psf/codeofconduct/
[python-committers] Comments on moving issues to GitHub
Reading the comments in the thread and having used Github issues myself for a few years now, I find the idea of moving from a dedicated issue tracker we can easily customize to our needs (or hire someone to do so via the PSF) to a simplistic tracker add-on, which Github issues is, not a very promising approach. Github issues are fine for simple projects, but I wouldn't even want to use it for more than a hundred issues on Github. As with many such proposals, if an existing system is seen to be lacking in certain ways, the first thing people suggest is to ditch it and move to this other new shiny thing or even worse, suggest to build a new one. I've rarely seen this work. Most of the time you end up having a system with just different issues which leaves you with a situation that's not better than before. The time invested in migration and making sure that at least part of the legacy will forward to the new solution is often better invested in addressing the issues with the older system. Yes, that's not as interesting and exciting as building something new, but in the light of productivity and getting a working solution, it's very often the better approach. So if there is a real need to fix issues with bpo, then I'd suggest to write up a proposal to get them fixed and find a group of developers to work on these. Have the PSF provide a grant to make it worthwhile and manage this project instead of spending time with migrations. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jun 02 2018) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [python-committers] Comments on moving issues to GitHub
Hi, On Sat, Jun 2, 2018 at 10:26 PM, Brett Cannon wrote: > > > On Sat, 2 Jun 2018 at 12:47 Mariatta Wijaya > wrote: >> >> [SNIP] >> >>> 2. Better support for core developers in the tracker. >> >> >> Not sure what you mean by "support"? There are only two maintainers of the >> bug tracker, they both are also Python core developers: Brett and Ezio. My >> personal opinion is: they're more valuable elsewhere instead of supporting >> the bug tracker. At its current state, the bug tracker is not ready to take >> up new contributors, and it will not be easy effort to onboard new bpo >> maintainers. > > > I actually wouldn't list me as a maintainer of b.p.o. I only have passing > knowledge of the code due to reviewing Ezio's changes to support the CLA > bot. It used to be RDM, Ezio, and Maciej, then DRM got busy, and then Maciej > got busy with helping move our hosting over to OpenShift and off of our > previously free hosting provider who sold their business (I also think > Maciej is also busy with other things). I don't know what Ezio's > availability currently sits at, but I have not heard from him recently. > I haven't actively worked on the tracker, but I kept an eye on it and acting when needed (e.g. yesterday I deployed a fix committed by Berker :). If needed I can pick it up again. It should also be mentioned that before us, MvL also used to work on the tracker, and he added the Rietveld and openid integration (and these parts are not very well maintained now). > If you look at > https://hg.python.org/tracker/roundup/shortlog/bugs.python.org there has not > been an update to the repo's code in 8 months but there have been reported > issues as recently as yesterday: > http://psf.upfronthosting.co.za/roundup/meta/ . > This is a bit misleading: * that repo is updated only when Roundup is updated otherwise it sits there waiting for a new release. Roundup 1.6 is going to be released soon; * the repo for our instance is https://hg.python.org/tracker/python-dev/shortlog/default . This also didn't get many commits lately, however * the meta tracker also didn't get many new issues. Only 7 issues in the metatracker have had any activity in the last 3 months, 2 are about SSL failure/certificates, 3 are about email ending up in the spam, one is about Google auth (however I'm not familiar with that part of the code), and the last one is a minor issue with a simple fix that needs to be tested/deployed. IOW there aren't many commits because there aren't that many issues with the code and because Roundup 1.6 hasn't been released yet. As mentioned above, the Roundup team is about to release Roundup 1.6. This release drops support for Python <2.7. AFAIU the infra team wanted to move/upgrade the machine running b.p.o (that is currently still running Python 2.6). When that happens (and once Roundup 1.6 is released) we will upgrade it. > IOW I consider b.p.o unmaintained ATM. Mark Mangoba and the PSF > infrastructure team can re-start the instance if it falls over, but no one > is working on e.g. fixing log-in issues or any of the various UX issues we > are all painfully aware that b.p.o has. > > As I said at the language summit, if people want to keep b.p.o around then > we need to get some volunteers to staff it. I personally would like to see > three people step forward and say they will work to improve b.p.o to make > sure it functions as expected now and plan to improve it long-term. But I > personally would settle for just two people actively working towards making > b.p.o a tenable solution (the three person goal is just from experience of > always wanting at least one backup even if someone goes on vacation or does > an OSS detox). > It depends on what kind of maintenance we need. In its current state b.p.o is quite stable and requires little maintenance IMHO. If instead we want to start adding (and fixing/maintaining) new features, then the situation is different. For the latter to happen, I would like to first see a PEP discussing what desired features GitHub has that Roundup doesn't and vice versa (Nick's list and Mariatta's slides and notes are a good starting point, but they should be consolidated and include the feedback expressed by other core devs on this thread). >From there we can evaluate how difficult it would be to implement those in Roundup and how this will compare with the difficulty of switching to GitHub or some other system. I already discussed with the Roundup devs about some of the features listed by Nick, so I can comment on them: > > Some examples of problems that would benefit from attention: > > - fixing the SSL/TLS connectivity issues This is also being discussed at https://github.com/python/psf-infra-meta/issues/4 (since it's an infra issue). One of the Roundup devs recently suggested a solution that still needs to be tested. > - making the issue tracker usable on mobile devices The Roundup team has already some working prototype. > - ability to edit the description
Re: [python-committers] Comments on moving issues to GitHub
On Sat, 2 Jun 2018 at 12:47 Mariatta Wijaya wrote: > [SNIP] > > 2. Better support for core developers in the tracker. > > > Not sure what you mean by "support"? There are only two maintainers of the > bug tracker, they both are also Python core developers: Brett and Ezio. My > personal opinion is: they're more valuable elsewhere instead of supporting > the bug tracker. At its current state, the bug tracker is not ready to take > up new contributors, and it will not be easy effort to onboard new bpo > maintainers. > I actually wouldn't list me as a maintainer of b.p.o. I only have passing knowledge of the code due to reviewing Ezio's changes to support the CLA bot. It used to be RDM, Ezio, and Maciej, then DRM got busy, and then Maciej got busy with helping move our hosting over to OpenShift and off of our previously free hosting provider who sold their business (I also think Maciej is also busy with other things). I don't know what Ezio's availability currently sits at, but I have not heard from him recently. If you look at https://hg.python.org/tracker/roundup/shortlog/bugs.python.org there has not been an update to the repo's code in 8 months but there have been reported issues as recently as yesterday: http://psf.upfronthosting.co.za/roundup/meta/ . IOW I consider b.p.o unmaintained ATM. Mark Mangoba and the PSF infrastructure team can re-start the instance if it falls over, but no one is working on e.g. fixing log-in issues or any of the various UX issues we are all painfully aware that b.p.o has. As I said at the language summit, if people want to keep b.p.o around then we need to get some volunteers to staff it. I personally would like to see three people step forward and say they will work to improve b.p.o to make sure it functions as expected now and plan to improve it long-term. But I personally would settle for just two people actively working towards making b.p.o a tenable solution (the three person goal is just from experience of always wanting at least one backup even if someone goes on vacation or does an OSS detox). But as of right now we have zero people supporting b.p.o (and GitHub has one with Mariatta which puts GH ahead in my book). Because of this, in my opinion this discussion shouldn't include b.p.o as an option until those volunteers materialize. We can argue GitHub versus GitLab versus some other issue tracker (open or closed source, self-hosted or service-hosted I personally don't care; heck write it from scratch like Warehouse if that's what it takes), but unless we get some people to step forward to help with b.p.o then I personally consider it our temporary solution until we figure out where we're going next with b.p.o and not a viable option in this discussion. ___ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [python-committers] Comments on moving issues to GitHub
> > "Old and languishing issues should just be closed / ignored" > I disagree with doing this blindly, and I would be mightily annoyed if > someone did so with IDLE issues and hide valuable ideas and code. Since you are IDLE's maintainer, I'll also be annoyed if other people except yourself (or other IDLE maintainers) blindly close IDLE issues without consulting you. Many modules don't have maintainers anymore. If such issues have been ignored for all these years, we'll probably never get to it. Might as well close it. The proposed idea is to provide a button that can copy over conversations from a b.p.o issue to GitHub, and to continue discussions in GitHub. If core devs have a list of issues they still care about, then they use this not yet existing magic button. The closed issue will still be in bpo, and anyone motivated enough can migrate it to GitHub. To deal with issues better, we need > 1. More core developers, so more modules can have maintainers. We need more core developers anyway, regardless of what tracker we're using. That is a separate problem. And perhaps this is to be discussed in a separate thread: even though in the b.p.o we appear to have 170 committers, really there are 90 core devs (people who has commit right to CPython on GitHub). and out of those 90, I think only about half are currently active (since the migration to GitHub). So yes, we need more (active) core devs. 2. Better support for core developers in the tracker. Not sure what you mean by "support"? There are only two maintainers of the bug tracker, they both are also Python core developers: Brett and Ezio. My personal opinion is: they're more valuable elsewhere instead of supporting the bug tracker. At its current state, the bug tracker is not ready to take up new contributors, and it will not be easy effort to onboard new bpo maintainers. 2b. Associated (linked) manager or dashboard for issues pertaining to a > module or group of modules. We can try the project boards as Ivan mentioned? https://help.github.com/ articles/about-project-boards/ * Labels can be grouped using name prefix and color, for example (we have > similar structure in mypy): > - priority-low > - priority-normal > - priority-etc... > - kind-bug > - kind-docs > - kind-feature > - topic-asincio > - topic-etc.. I kinda like those! I wonder if other hosted services, such as Gitlab, offer a more > sophisticated issue tracker. One thing I'm trying to avoid is having separate issue tracker and repo, and needing different accounts. Possibly useful for planning, if we had someone who was responsible for that > Perhaps for a project the size of Python we should have a dedicated Project manager. Mariatta ᐧ ___ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [python-committers] Comments on moving issues to GitHub
I think boards have improved since I last used them, but when I tried they added nothing but overhead. Possibly useful for planning, if we had someone who was responsible for that (maybe individual planning? But then you can’t really expect contributors to keep it up to date for you). Milestones are one-per-issue, and get rolled up in a way that is most useful for planning rather than search or review. I use these all the time on work projects. A good set of tags (which unfortunately are shared with the set of tags you can put on a PR) and some automation to auto-subscribe the core devs associated with that tag is a bare minimum, as far as I’m concerned. It would be nice if issue search supported the OR operator as well, but it can only do AND. I’m far from convinced that GitHub issues will work well for an active team as large as ours with as little coordination as we use. It doesn’t work well for the “bucket of bugs” I keep open on one of my work projects, even though the team is smaller, and our tracker is almost entirely a bucket of bugs. Top-posted from my Windows 10 phone From: Ivan Levkivskyi Sent: Friday, June 1, 2018 18:05 To: Barry Warsaw Cc: python-committers Subject: Re: [python-committers] Comments on moving issues to GitHub On 1 June 2018 at 20:29, Barry Warsaw wrote: On Jun 1, 2018, at 16:54, Antoine Pitrou wrote: > > I wonder if other hosted services, such as Gitlab, offer a more > sophisticated issue tracker. Note that GitHub (and I think GitLab too) provides two additional ways to categorise issues: project boards, and milestones. I think together with labels this may simulate current b.p.o. structure to certain extent. For example (approximately): * We can have milestones for releases (including past releases) * We can have "project boards" (slightly abusing this feature): new, triaged, PR review * Labels can be grouped using name prefix and color, for example (we have similar structure in mypy): - priority-low - priority-normal - priority-etc... - kind-bug - kind-docs - kind-feature - topic-asincio - topic-etc.. This is still quite limited, but together with bots this can practically replace current categorisation. -- Ivan ___ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers Code of Conduct: https://www.python.org/psf/codeofconduct/
