Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Brett Cannon]

On Wed, 6 Jan 2016 at 23:56 Ezio Melotti  wrote:

> On Wed, Jan 6, 2016 at 8:18 PM, Brett Cannon  wrote:
> >
> >
> > On Tue, 5 Jan 2016 at 23:46 Ezio Melotti  wrote:
> >> ...
> >> If you agree, this is what needs to be done:
> >> 1) automatically add PRs to b.p.o issues;
> >
> >
> > This is a blocker.
> >
> >>
> >> 2) automatically add a message on b.p.o when a review is posted on
> github;
> >
> >
> > With the way GitHub does reviews, this won't make sense. Each comment on
> a
> > PR is essentially its own review (as Eric complained about because that
> > means each comment is treated as a separate thing and thus generates its
> own
> > email). It isn't like with Rietveld where there is a "Review + Email"
> step
> > to send draft reviews and hence a review is a group of comments.
> >
>
> Good to know.
> I can think of two alternative solutions:
> 1) have a "number of comments" and "date/time of the last comment"
> next to the PR, and update it whenever a new comment is posted;
> 2) check periodically for new comments, aggregate them somehow, and
> post a "X new comments have been added to PR Y." message;
>

Either of those ideas work for me. Would option #2 be done as a comment and
thus send out what amounts to a summary email of PR review comments? As
long as we do that **only** when there have been changes and not blindly
doing it every day (e.g., no "0 comments since yesterday" email), that
would be acceptable to me in terms of email volume. Do the other people who
wanted to follow PRs on b.p.o like that level of frequency?


>
> >>
> >> 3) add a "github username" field to b.p.o users to link accounts;
> >
> >
> > That's a blocker for CLA enforcement.
> >
> >>
> >> 4) automatically add the PR author (during step 1) and reviewers
> >> (during step 2) to the issue nosy list on b.p.o;
> >
> >
> > I view this as a great nicety but not a blocker.
> >
>
> OK, I considered this a blocker because otherwise PR authors and
> reviewers might miss b.p.o comments, but we can let them know manually
> until it's implemented.
>

Yep, exactly. I want to block on key stuff that would make GitHub worse
than what we have now, but not on stuff that makes it better. This is
obviously not to say that I don't want to see all of these great
improvements happen, but I do want to get over to the new workflow sooner
rather than later as I suspect contributors will find it much easier to
work with.

-Brett


>
> Best Regards,
> Ezio Melotti
>
> >>
> >> 5) add an option to disable review emails (optional);
> >
> >
> > This will have to be discussed in connection with the emails per PR
> comment
> > in case there is a misunderstanding of what a review on GitHub is.
> >
> >>
> >>
> >> I would consider all these points except the 5th as blockers.
> >
> >
> > Obviously I don't think they all are, but definitely some.
> >
> > -Brett
> >
> >>
> >>
> >> Best Regards,
> >> Ezio Melotti
>
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Zachary Ware]

On Thu, Jan 7, 2016 at 12:40 PM, Ezio Melotti  wrote:
> We can also try to do something smarter by checking e.g. every 15
> minutes and posting the message only if no new messages have been
> added in the last 15 minutes (so the reviewer has likely finished
> commenting).

I like this plan, though 15 minutes might be a bit short for the
cooldown period; half an hour might be a bit better.  On the other
hand, it is possible to write out all your review comments without
hitting the 'comment' button on any of them, then scroll through and
click comment on each of them, in which case the cooldown could be
even shorter than 15 minutes.

Alternately, what about checking for a marker in a top-level (not a
line note) comment, which signals "I finished a review, aggregate it
and post it to b.p.o"?

-- 
Zach
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Berker Peksağ]

On Thu, Jan 7, 2016 at 8:40 PM, Ezio Melotti  wrote:
> The goal is to generate at least 1 message/email if a review (possibly
> comprising several comments) is posted.
> If there are no new comments, nothing is posted, but if there are new
> comments, a new message will be posted at some point.
> We just need to find a compromise between delay and number of messages
> (and that's something we can figure out later with a bit of trial and
> error).
> Checking daily might result in hours of delay, but no more than one
> daily message.
> Checking hourly has less delay, but it might result in more messages.
> We can also try to do something smarter by checking e.g. every 15
> minutes and posting the message only if no new messages have been
> added in the last 15 minutes (so the reviewer has likely finished
> commenting).

I think this still will create too much noise. I'd prefer not to see
comments like "this needs to be tested", "needs versionadded", "please
don't change function signature" etc. in issues. I like following
Windows and IDLE issues, but I'm not really interested seeing review
comments about them, for example. Wouldn't a new pull request field in
the issue detail page be enough to link pull requests? Django uses a
similar solution to this: https://code.djangoproject.com/ticket/25995
(see "Pull Requests: 5928 build:success") We could show total number
of comments too.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Brett Cannon]

On Thu, 7 Jan 2016 at 13:06 francismb  wrote:

> Hi,
>
> On 01/05/2016 07:13 PM, Brett Cannon wrote:
> > Day 1 summary
> > 
> >
> > Decisions made
> > ---
> >
> > - Seems like our current commit ID -> URL service can be updated to
> handle
> > our transition
> >
> >
> > Open issues
> > ---
> > - What tools and commands will we use to convert the repos?
> > - How do we want to handle Misc/NEWS?
> > - What are the exact commands we are going to expect core devs to run
> > (which will be documented in the devguide)?
> > - Who to use for CI (although Donald is the only one to speak up and
> likes
> > Travis as do I, so this might be decided)?
> > - Who to use for code coverage (Donald has suggested codecov)?
> > - Do we want to add GitHub login support for b.p.o to ease in adding
> > people's GitHub username and to minimize the need for two accounts to
> > propose a patch that has a corresponding issue?
> >
>
> a small question: could it be possible to access (some of) the services
> in a REST-ish form through a wrapper on https://www.python.org/dev
>
> I mean something like:
>
> Python-
> community-
> Service External-ServicePython-Website-Address-Wrapper
> 
> cpython GitHub   https://www.python.org/dev/cpython


We will probably have git.python.org handle this.


>
> issues  ...  https://www.python.org/dev/cpython-issues


bugs.python.org already exists.


>
> website GitHub   https://www.python.org/dev/website
> buildbots   ...  https://www.python.org/dev/cpython-buildbots


buildbot.python.org already exists.


>
> devinabox   GitHub   https://www.python.org/dev/cpython-devinabox


It's just a repo and not special enough to deserve a unique URL.


>
> coveragecodecov  https://www.python.org/dev/cpython-coverage


This might be worth a URL if we get it set up.


>
> docsGitHub   https://www.python.org/dev/cpython-docs


That's docs.python.org.


>
> ...
>
>
> Couldn't be then some bots external-provider independent/decoupled?
>

The plan is that anything we write for this will be sufficiently decoupled
from GitHub that changing hosts will just require updating abstraction code
to work with different REST APIs and not a major rewrite of the bot itself.

-Brett


>
>
> Thanks in advance!
>
> francis
>
> PS: Sorry if I got some external-service wrong
>
> ___
> core-workflow mailing list
> core-workflow@python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
>
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[francismb]

Hi,

On 01/05/2016 07:13 PM, Brett Cannon wrote:
> Day 1 summary
> 
> 
> Decisions made
> ---
> 
> - Seems like our current commit ID -> URL service can be updated to handle
> our transition
> 
> 
> Open issues
> ---
> - What tools and commands will we use to convert the repos?
> - How do we want to handle Misc/NEWS?
> - What are the exact commands we are going to expect core devs to run
> (which will be documented in the devguide)?
> - Who to use for CI (although Donald is the only one to speak up and likes
> Travis as do I, so this might be decided)?
> - Who to use for code coverage (Donald has suggested codecov)?
> - Do we want to add GitHub login support for b.p.o to ease in adding
> people's GitHub username and to minimize the need for two accounts to
> propose a patch that has a corresponding issue?
> 

a small question: could it be possible to access (some of) the services
in a REST-ish form through a wrapper on https://www.python.org/dev

I mean something like:

Python-
community-
Service External-ServicePython-Website-Address-Wrapper

cpython GitHub   https://www.python.org/dev/cpython
issues  ...  https://www.python.org/dev/cpython-issues
website GitHub   https://www.python.org/dev/website
buildbots   ...  https://www.python.org/dev/cpython-buildbots
devinabox   GitHub   https://www.python.org/dev/cpython-devinabox
coveragecodecov  https://www.python.org/dev/cpython-coverage
docsGitHub   https://www.python.org/dev/cpython-docs
...


Couldn't be then some bots external-provider independent/decoupled?


Thanks in advance!

francis

PS: Sorry if I got some external-service wrong

___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Brett Cannon]

On Tue, 5 Jan 2016 at 23:46 Ezio Melotti  wrote:

> On Tue, Jan 5, 2016 at 9:18 PM, Brett Cannon  wrote:
> >
> >
> > On Tue, 5 Jan 2016 at 10:48 R. David Murray 
> wrote:
> >>
> >> On Tue, 05 Jan 2016 17:50:53 +, Brett Cannon 
> wrote:
> >> > If people are that worried, we could do a daily dump of the data. But
> >> > unless we can have all GitHub-related comments to an issue not trigger
> >> > an
> >> > email update I don't think that's feasible as it would mean two emails
> >> > for
> >> > every PR comment; one from GitHub and one from b.p.o.
> >>
> >> We can fairly easily have all github-originated comments not trigger an
> >> email to the nosy list.  We'd just need to add a check for the
> >> originating email address to the nosy reaction detector.  Even nicer
> >> would be to make it a per-user setting (I'd rather get the emails from
> >> b.p.o. and disable them on github, myself), but that would be a bit more
> >> work.
> >>
> >> It would be nice to have *all* of the discussion in one place.
> >
> >
> > If someone wants to put the work in to make that happen then that's fine
> by
> > me, but I do consider this a nicety and not a blocker. We haven't had
> code
> > review comments go into b.p.o on Rietveld either so this isn't that far
> off
> > from what we have now, especially if we block the transition on adding a
> > link back to the GitHub PR  through a web hook.
>
> I agree it's not a blocker, but a better integration between Rietveld
> and b.p.o was often requested (I tried -- and failed -- to tackle the
> issue a couple of times).
> What I want to avoid is having people on github missing b.p.o
> messages, and people on b.p.o missing PRs/reviews on github.  (This
> was already happening when people were discussing the patch on
> Rietveld, and others were only looking at b.p.o and/or missing the
> reviews, both because the email from rietveld ended up in the spam
> folder, and because there was no way to know from b.p.o if someone
> posted a review.)
>
> I think it's ok to have discussions on b.p.o and reviews on github
> (similarly to what we currently do with Rietveld), as long as the
> everyone gets notified.  Discussions that are not strictly related to
> the code/review shouldn't happen on github.
>
> As for the notifications, I think the best option is:
> * when a PR is posted, it's also automatically added to the issue (to
> the list of PRs, similar to the list of patches; generates an email
> too);
> * when a review is posted, a new message with a link is added to the
> issue (this is like a regular message, and generates an email too);
> * when someone posts a PR/review on github, they will also be added
> automatically on the nosy list of the issue on b.p.o (so they will
> receive b.p.o messages);
> * when a PR/review is posted, people in the issue nosy on b.p.o are
> NOT added to the PR nosy list (they already get new PRs/reviews
> notifications from b.p.o).
>
> This means that:
> * when a PR is posted:
>   * the PR author will get a mail from b.p.o confirming that the PR
> has been added to the issue if they have a linked account (possibly on
> top of any email github might send for creating a PR);
>   * people in the issue nosy on b.p.o will get an email that lets them
> know a PR has been added;
> * when a review is posted:
>   * the PR author and the reviewers will get two emails (one from
> github and one from b.p.o if they have a linked account) or one (from
> github, if they don't);
>   * people in the issue nosy will only get one email (from b.p.o, with
> a link to the review);
> * when a message is posted:
>   * the PR author and reviewers will get one email (from b.p.o if they
> have a linked account) or zero (if they don't);
>   * people in the issue nosy will get one email (from b.p.o);
>
> The "problematic" cases are:
> * the PR author and the reviewers will receive two emails for reviews
> if they have a linked account;
> * the PR author and the reviewers will receive zero emails for b.p.o
> messages if they don't have a linked account;
>
> The former can be solved if we add a way to disable review emails from
> b.p.o or if github provides one already (or using email filters).  The
> latter can't be solved unless they link accounts (I don't think b.p.o
> messages should go to github).
>
> FWIW I would personally prefer to get all emails from b.p.o, either
> ignoring/filtering duplicated review emails from github, or disabling
> them from github if possible.
>
> If you agree, this is what needs to be done:
> 1) automatically add PRs to b.p.o issues;
>

This is a blocker.


> 2) automatically add a message on b.p.o when a review is posted on github;
>

With the way GitHub does reviews, this won't make sense. Each comment on a
PR is essentially its own review (as Eric complained about because that
means each comment is treated as a separate thing and thus generates its
own email). It 

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Laura Creighton]

In a message of Tue, 05 Jan 2016 12:22:03 +0100, "M.-A. Lemburg" writes:
>Given that we want to make it possible to move away from Github
>without too much fuzz, wouldn't it be better to have the
>PR discussions on b.p.o and Rietvield ?

+1

>If we start using Github for this, we'd lose that part of the
>review history when moving away.
>
>Moving from the SF issue tracker to b.p.o was a major piece of work
>(mostly done by Martin von Löwis IIRC) and it's not clear how we
>could retrofit those discussions into the b.p.o issue discussions.

You recall very correctly.

Laura
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Ezio Melotti]

On Wed, Jan 6, 2016 at 8:18 PM, Brett Cannon  wrote:
>
>
> On Tue, 5 Jan 2016 at 23:46 Ezio Melotti  wrote:
>> ...
>> If you agree, this is what needs to be done:
>> 1) automatically add PRs to b.p.o issues;
>
>
> This is a blocker.
>
>>
>> 2) automatically add a message on b.p.o when a review is posted on github;
>
>
> With the way GitHub does reviews, this won't make sense. Each comment on a
> PR is essentially its own review (as Eric complained about because that
> means each comment is treated as a separate thing and thus generates its own
> email). It isn't like with Rietveld where there is a "Review + Email" step
> to send draft reviews and hence a review is a group of comments.
>

Good to know.
I can think of two alternative solutions:
1) have a "number of comments" and "date/time of the last comment"
next to the PR, and update it whenever a new comment is posted;
2) check periodically for new comments, aggregate them somehow, and
post a "X new comments have been added to PR Y." message;

>>
>> 3) add a "github username" field to b.p.o users to link accounts;
>
>
> That's a blocker for CLA enforcement.
>
>>
>> 4) automatically add the PR author (during step 1) and reviewers
>> (during step 2) to the issue nosy list on b.p.o;
>
>
> I view this as a great nicety but not a blocker.
>

OK, I considered this a blocker because otherwise PR authors and
reviewers might miss b.p.o comments, but we can let them know manually
until it's implemented.

Best Regards,
Ezio Melotti

>>
>> 5) add an option to disable review emails (optional);
>
>
> This will have to be discussed in connection with the emails per PR comment
> in case there is a misunderstanding of what a review on GitHub is.
>
>>
>>
>> I would consider all these points except the 5th as blockers.
>
>
> Obviously I don't think they all are, but definitely some.
>
> -Brett
>
>>
>>
>> Best Regards,
>> Ezio Melotti
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Brett Cannon]

Day 1 summary


Decisions made
---
- We will create a python-dev team in the python org and add that team to
all of the relevant repos that get migrated
- We will add a GitHub field to Roundup and then write/get a CLA bot that
will query Roundup to check if someone has (not) signed the CLA
- For ancillary repos, the merge-of-least-resistance workflow is fine, so
no need to worry about maintaining a linear history
- We are going to keep all of the cpython history in a single repo
- We will have PRs go against master and then we will cherry pick into
bugfix branches
- Misc/ACKS will be kept and we will write code to update it on occasion --
probably at release time -- based on git commit data
- Seems like our current commit ID -> URL service can be updated to handle
our transition


Open issues
---
- What tools and commands will we use to convert the repos?
- How do we want to handle Misc/NEWS?
- What are the exact commands we are going to expect core devs to run
(which will be documented in the devguide)?
- Who to use for CI (although Donald is the only one to speak up and likes
Travis as do I, so this might be decided)?
- Who to use for code coverage (Donald has suggested codecov)?
- Do we want to add GitHub login support for b.p.o to ease in adding
people's GitHub username and to minimize the need for two accounts to
propose a patch that has a corresponding issue?

On Mon, 4 Jan 2016 at 16:42 Brett Cannon  wrote:

> So consider this the starting discussion of the PEP that will be the
> hg.python.org -> GitHub transition PEP that I will be in charge of. Once
> we have general agreement on the steps necessary I will start the actual
> PEP and check it in, but I figure there's no point in have a skeleton PEP
> if we can't agree on the skeleton. :) While I list steps influencing all
> the repos, I want to focus on the ones stopping any repo from moving over
> for now, expanding what we worry about to the cpython repo as we knock
> blockers down until we move everything over and start adding GitHub perks.
>
> The way I see it, we have 4 repos to move: devinabox, benchmarks, peps,
> devguide, and cpython. I also think that's essentially the order we should
> migrate them over. Some things will need to be universally handled before
> we transition a single repo, while other things are only a blocker for some
> of the repos.
>
> Universal blockers
> ==
> There are four blockers that must be resolved before we even consider
> moving a repo over. They can be solved in parallel, but they all need to
> have a selected solution before we can move even the devinabox repo.
>
> First, we need to decide how we are going to handle adding all the core
> devs to GitHub. Are we simply going to add all of them to the python
> organization, or do we want something like a specific python-dev gteamroup
> that gets added to all of the relevant repos? Basically I'm not sure how
> picky we want to be about the people who already have organization access
> on GitHub about them implicitly getting access to the cpython repo at the
> end of the day (and the same goes for any of the other repos in the python
> organization). For tracking names, I figure we will create a file in the
> devguide where people can check in their GitHub usernames and I can
> manually add people as people add themselves to the file.
>
> Second, CLA enforcement. As of right now people go to
> https://www.python.org/psf/contrib/contrib-form/, fill in the form, and
> then Ewa gets an email where she manually flips a flag in Roundup. If we
> want to use a web hook to verify someone has signed a CLA then we need to
> decide where the ground truth for CLAs are. Do we want to keep using
> Roundup to manage CLA agreements and thus add a GitHub field in
> bugs.python.org for people's profile and a web hook or bot that will
> signal if someone has the flag flipped on bugs.python.org? Or is there
> some prepackaged service that we can use that will keep track of this which
> would cause us to not use Roundup (which might be easier, but depending on
> the service require everyone to re-sign)? There's also the issue of
> supporting people who want to submit code by uploading a patch to
> bugs.python.org but not use GitHub. Either way I don't want to have to
> ask everyone who submits a PR what their bugs.python.org username is and
> then go check that manually.
>
> Third, how do we want to do the repo conversions? We need to choose the
> tool(s) and command(s) that we want to use. There was mention of wanting a
> mapping from hg commit ID to git commit ID. If we have that we could have a
> static bugs.python.org/commit/ page that had the mapping embedded in
> some JavaScript and if  matched then we just forward them to the
> corresponding GitHub commit page, otherwise just blindly forward to GitHub
> and assume the ID is git-only, giving us a stable URL for commit web views.
>
> Fourth, for the 

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Brett Cannon]

On Tue, 5 Jan 2016 at 10:11 Donald Stufft  wrote:

> On Jan 5, 2016, at 1:03 PM, Brett Cannon  wrote:
>
>
>
> On Mon, 4 Jan 2016 at 21:54 Ezio Melotti  wrote:
>
>> On Tue, Jan 5, 2016 at 2:42 AM, Brett Cannon  wrote:
>> > So consider this the starting discussion of the PEP that will be the
>> > hg.python.org -> GitHub transition PEP that I will be in charge of.
>> Once we
>> > have general agreement on the steps necessary I will start the actual
>> PEP
>> > and check it in, but I figure there's no point in have a skeleton PEP
>> if we
>> > can't agree on the skeleton. :) While I list steps influencing all the
>> > repos, I want to focus on the ones stopping any repo from moving over
>> for
>> > now, expanding what we worry about to the cpython repo as we knock
>> blockers
>> > down until we move everything over and start adding GitHub perks.
>> >
>> > The way I see it, we have 4 repos to move: devinabox, benchmarks, peps,
>> > devguide, and cpython.
>>
>> On top of this, there is also the test repo
>> (https://hg.python.org/test) and all the tracker repos
>> (https://hg.python.org/tracker/).
>> I think it would be useful to port the former since it will provide a
>> place for devs to try things out and experiment (a new test repo could
>> also be created though).
>> It would be nice to port the tracker repos too and be consistent with
>> the others, but it's not a priority.  When we switched to HG they kept
>> being on SVN until I ported them, so I guess the same thing can be
>> done (unless R. David or Martin prefer to stick to HG).
>>
>> > I also think that's essentially the order we should
>> > migrate them over. Some things will need to be universally handled
>> before we
>> > transition a single repo, while other things are only a blocker for
>> some of
>> > the repos.
>> >
>> > Universal blockers
>> > ==
>> > There are four blockers that must be resolved before we even consider
>> moving
>> > a repo over. They can be solved in parallel, but they all need to have a
>> > selected solution before we can move even the devinabox repo.
>> >
>> > First, we need to decide how we are going to handle adding all the core
>> devs
>> > to GitHub. Are we simply going to add all of them to the python
>> > organization, or do we want something like a specific python-dev
>> gteamroup
>> > that gets added to all of the relevant repos? Basically I'm not sure how
>> > picky we want to be about the people who already have organization
>> access on
>> > GitHub about them implicitly getting access to the cpython repo at the
>> end
>> > of the day (and the same goes for any of the other repos in the python
>> > organization). For tracking names, I figure we will create a file in the
>> > devguide where people can check in their GitHub usernames and I can
>> manually
>> > add people as people add themselves to the file.
>> >
>>
>> I think the current list of core-devs should be converted to a group
>> and given access to the same repos they have access to now (i.e.
>> cpython/devguide/peps and possibly others).  Then additional
>> repo-specific groups can be created in case we want to let specific
>> contributors work on peps or the devguide.
>>
>
> This seems to be the general consensus, so we will create a python-dev
> team under the python org and add the core devs there.
>
>
> Just to expand on this more, GitHub has recently updated their permission
> model more to also allow “outside contributors” to be added to a single
> repository without adding them to a group (Not sure if this would be useful
> for us or not).
>

It will be on the ancillary repos like peps and devguide where we may want
to be more liberal in letting people commit (e.g., maybe we want to add all
PEP authors as outside contributors so that they can maintain their own
PEP).


>
> I imagine the python-dev team would have “Write Access” (See
> https://help.github.com/articles/repository-permission-levels-for-an-organization/)
> unless we think it needs admin access.
>

I think write access is perfect for the vast majority of people. The
question becomes if we want to add anyone specifically with admin access to
help manage repo stuff like web hooks, or if we just bug someone on the
infrastructure team to set it for us.

I do think that some of us -- probably those of us on hgaccounts -- should
be team owners.


>
>
>
>>
>> > Second, CLA enforcement. As of right now people go to
>> > https://www.python.org/psf/contrib/contrib-form/, fill in the form,
>> and then
>> > Ewa gets an email where she manually flips a flag in Roundup. If we
>> want to
>> > use a web hook to verify someone has signed a CLA then we need to decide
>> > where the ground truth for CLAs are. Do we want to keep using Roundup to
>> > manage CLA agreements and thus add a GitHub field in bugs.python.org
>> for
>> > people's profile and a web hook or bot that will signal if someone has
>> the
>> > 

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[R. David Murray]

On Tue, 05 Jan 2016 17:50:53 +, Brett Cannon  wrote:
> If people are that worried, we could do a daily dump of the data. But
> unless we can have all GitHub-related comments to an issue not trigger an
> email update I don't think that's feasible as it would mean two emails for
> every PR comment; one from GitHub and one from b.p.o.

We can fairly easily have all github-originated comments not trigger an
email to the nosy list.  We'd just need to add a check for the
originating email address to the nosy reaction detector.  Even nicer
would be to make it a per-user setting (I'd rather get the emails from 
b.p.o. and disable them on github, myself), but that would be a bit more
work.

It would be nice to have *all* of the discussion in one place.

--David
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Senthil Kumaran]

Hi Brett,

On Tue, Jan 5, 2016 at 10:13 AM, Brett Cannon  wrote:

> - We are going to keep all of the cpython history in a single repo


Are we not going to have 2.7 as a separate repo?
I believe, this is possible even if we keep cpython with full history (and
branches).

Thanks,
Senthil
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Guido van Rossum]

On Tue, Jan 5, 2016 at 12:35 PM, Senthil Kumaran 
wrote:

> Are we not going to have 2.7 as a separate repo?
>

Why would you want this?


> I believe, this is possible even if we keep cpython with full history (and
> branches).
>

-- 
--Guido van Rossum (python.org/~guido)
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Ezio Melotti]

On Tue, Jan 5, 2016 at 9:18 PM, Brett Cannon  wrote:
>
>
> On Tue, 5 Jan 2016 at 10:48 R. David Murray  wrote:
>>
>> On Tue, 05 Jan 2016 17:50:53 +, Brett Cannon  wrote:
>> > If people are that worried, we could do a daily dump of the data. But
>> > unless we can have all GitHub-related comments to an issue not trigger
>> > an
>> > email update I don't think that's feasible as it would mean two emails
>> > for
>> > every PR comment; one from GitHub and one from b.p.o.
>>
>> We can fairly easily have all github-originated comments not trigger an
>> email to the nosy list.  We'd just need to add a check for the
>> originating email address to the nosy reaction detector.  Even nicer
>> would be to make it a per-user setting (I'd rather get the emails from
>> b.p.o. and disable them on github, myself), but that would be a bit more
>> work.
>>
>> It would be nice to have *all* of the discussion in one place.
>
>
> If someone wants to put the work in to make that happen then that's fine by
> me, but I do consider this a nicety and not a blocker. We haven't had code
> review comments go into b.p.o on Rietveld either so this isn't that far off
> from what we have now, especially if we block the transition on adding a
> link back to the GitHub PR  through a web hook.

I agree it's not a blocker, but a better integration between Rietveld
and b.p.o was often requested (I tried -- and failed -- to tackle the
issue a couple of times).
What I want to avoid is having people on github missing b.p.o
messages, and people on b.p.o missing PRs/reviews on github.  (This
was already happening when people were discussing the patch on
Rietveld, and others were only looking at b.p.o and/or missing the
reviews, both because the email from rietveld ended up in the spam
folder, and because there was no way to know from b.p.o if someone
posted a review.)

I think it's ok to have discussions on b.p.o and reviews on github
(similarly to what we currently do with Rietveld), as long as the
everyone gets notified.  Discussions that are not strictly related to
the code/review shouldn't happen on github.

As for the notifications, I think the best option is:
* when a PR is posted, it's also automatically added to the issue (to
the list of PRs, similar to the list of patches; generates an email
too);
* when a review is posted, a new message with a link is added to the
issue (this is like a regular message, and generates an email too);
* when someone posts a PR/review on github, they will also be added
automatically on the nosy list of the issue on b.p.o (so they will
receive b.p.o messages);
* when a PR/review is posted, people in the issue nosy on b.p.o are
NOT added to the PR nosy list (they already get new PRs/reviews
notifications from b.p.o).

This means that:
* when a PR is posted:
  * the PR author will get a mail from b.p.o confirming that the PR
has been added to the issue if they have a linked account (possibly on
top of any email github might send for creating a PR);
  * people in the issue nosy on b.p.o will get an email that lets them
know a PR has been added;
* when a review is posted:
  * the PR author and the reviewers will get two emails (one from
github and one from b.p.o if they have a linked account) or one (from
github, if they don't);
  * people in the issue nosy will only get one email (from b.p.o, with
a link to the review);
* when a message is posted:
  * the PR author and reviewers will get one email (from b.p.o if they
have a linked account) or zero (if they don't);
  * people in the issue nosy will get one email (from b.p.o);

The "problematic" cases are:
* the PR author and the reviewers will receive two emails for reviews
if they have a linked account;
* the PR author and the reviewers will receive zero emails for b.p.o
messages if they don't have a linked account;

The former can be solved if we add a way to disable review emails from
b.p.o or if github provides one already (or using email filters).  The
latter can't be solved unless they link accounts (I don't think b.p.o
messages should go to github).

FWIW I would personally prefer to get all emails from b.p.o, either
ignoring/filtering duplicated review emails from github, or disabling
them from github if possible.

If you agree, this is what needs to be done:
1) automatically add PRs to b.p.o issues;
2) automatically add a message on b.p.o when a review is posted on github;
3) add a "github username" field to b.p.o users to link accounts;
4) automatically add the PR author (during step 1) and reviewers
(during step 2) to the issue nosy list on b.p.o;
5) add an option to disable review emails (optional);

I would consider all these points except the 5th as blockers.

Best Regards,
Ezio Melotti
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Nicolás Alvarez]

2016-01-05 15:13 GMT-03:00 Brett Cannon :
> Open issues
> ---
> - What tools and commands will we use to convert the repos?

I will investigate this soon. I don't claim ESR-level experience in
repository conversion, but I did migrate most of KDE from SVN to Git
(which was one hell of a task) :)

However, I will be on vacation and trying to stay away from my email
inbox (except urgent KDE sysadmin matters) from Jan 9th to Jan ~15th.

What's the general timeframe you're thinking for all this? I know
Python is not a corporate project with "deadlines", but I bet nobody
would like "CPython will be in GitHub by early 2017"...

-- 
Nicolás
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Nicolás Alvarez]

Donald Stufft  writes:
> 
> There is CLAHub (https://www.clahub.com/) but I don’t have any idea how good 
it is, I just know of it’s existence.

CLAHub is apparently dying, see https://github.com/clahub/clahub/issues/111.

Another option is https://cla-assistant.io/ which already has features that 
CLAHub doesn't, such as posting "You haven't signed the CLA" comments on review 
requests from new contributors.

-- 
Nicolás
Apologies for crappy quoting etc. I'm using Gmane to post this message.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Nicholas Chammas]

We can set a commit status that will show red if the user hasn’t signed the
CLA (just like if Travis tests failed or so). No need to use a banner or
anything.

This is a great idea. Almost any automated check we want to run against PRs
can be captured as a Travis/CI test that shows up on the PR with its own
status .

Nick
​

On Tue, Jan 5, 2016 at 5:40 PM Brett Cannon  wrote:

> On Tue, 5 Jan 2016 at 14:19 Eric Snow  wrote:
>
>> On Tue, Jan 5, 2016 at 11:13 AM, Brett Cannon  wrote:
>> > Day 1 summary
>> > 
>> >
>> > Decisions made
>> > ---
>> >
>> > Open issues
>> > ---
>>
>> And a couple things that we are punting on:
>>
>> * code review tool (if GH proves undesirable)
>>
>
> Well, that's implicit if we find GitHub doesn't work for us for code
> review. I don't think it requires explicitly calling it out.
>
>
>> * separate (sub)repos for docs/tutorials--they could have a less
>> restricted workflow than the rest of the cpython repo, a la the
>> devguide
>>
>
> Sure, it can be mentioned.
>
>
>>
>> Both of these can wait until later, though they still deserve mention
>> in the PEP.
>
>
> You really don't like GitHub's review tool, huh? ;)
> ___
> core-workflow mailing list
> core-workflow@python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Eric Snow]

On Tue, Jan 5, 2016 at 11:13 AM, Brett Cannon  wrote:
> Day 1 summary
> 
>
> Decisions made
> ---
>
> Open issues
> ---

And a couple things that we are punting on:

* code review tool (if GH proves undesirable)
* separate (sub)repos for docs/tutorials--they could have a less
restricted workflow than the rest of the cpython repo, a la the
devguide

Both of these can wait until later, though they still deserve mention
in the PEP.

-eric
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[M.-A. Lemburg]

On 05.01.2016 06:53, Ezio Melotti wrote:
>> Or is there some prepackaged service that
>> we can use that will keep track of this which would cause us to not use
>> Roundup (which might be easier, but depending on the service require
>> everyone to re-sign)? There's also the issue of supporting people who want
>> to submit code by uploading a patch to bugs.python.org but not use GitHub.
>> Either way I don't want to have to ask everyone who submits a PR what their
>> bugs.python.org username is and then go check that manually.
>>
> 
> This also brings up another problem.
> Since the discussion about an issue happens on b.p.o and the PRs are
> submitted on GitHub, this means that:
> 1) users with only a GitHub account have to create a b.p.o account if
> they want to comment on the issue (exclusing review comments);
> 2) users with only a b.p.o account have to create a GitHub account if
> they want to review a PR;
> 3) users with both can comment on b.p.o and review on GitHub, but they
> might need to login twice.
> 
> It would be better if users didn't need to create and use two separate 
> accounts.

Given that we want to make it possible to move away from Github
without too much fuzz, wouldn't it be better to have the
PR discussions on b.p.o and Rietvield ?

If we start using Github for this, we'd lose that part of the
review history when moving away.

Moving from the SF issue tracker to b.p.o was a major piece of work
(mostly done by Martin von Löwis IIRC) and it's not clear how we
could retrofit those discussions into the b.p.o issue discussions.

Perhaps we could gateway the emails that Github sends for PR
discussions back into b.p.o in some way (the emails contain the
PR number, repo name and Github account name of the sender
in the headers).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 05 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Donald Stufft]

> On Jan 5, 2016, at 12:53 AM, Ezio Melotti  wrote:
> 
>> 
>> Or is there some prepackaged service that
>> we can use that will keep track of this which would cause us to not use
>> Roundup (which might be easier, but depending on the service require
>> everyone to re-sign)? There's also the issue of supporting people who want
>> to submit code by uploading a patch to bugs.python.org but not use GitHub.
>> Either way I don't want to have to ask everyone who submits a PR what their
>> bugs.python.org username is and then go check that manually.
>> 
> 
> This also brings up another problem.
> Since the discussion about an issue happens on b.p.o and the PRs are
> submitted on GitHub, this means that:
> 1) users with only a GitHub account have to create a b.p.o account if
> they want to comment on the issue (exclusing review comments);
> 2) users with only a b.p.o account have to create a GitHub account if
> they want to review a PR;
> 3) users with both can comment on b.p.o and review on GitHub, but they
> might need to login twice.
> 
> It would be better if users didn't need to create and use two separate 
> accounts.

Can we allow (not mandate) login to b.p.o with a GitHub account? That should 
solve #1 and should sort of solve #3 as well. Ideally it’d allow you to 
associate a Github account with an existing b.p.o account (and doing so should 
also automatically create an association that X on b.p.o is Y on Github).

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Brett Cannon]

On Tue, 5 Jan 2016 at 12:35 Senthil Kumaran  wrote:

> Hi Brett,
>
> On Tue, Jan 5, 2016 at 10:13 AM, Brett Cannon  wrote:
>
>> - We are going to keep all of the cpython history in a single repo
>
>
> Are we not going to have 2.7 as a separate repo?
> I believe, this is possible even if we keep cpython with full history (and
> branches).
>

No, we are not going to break 2.7 into a separate repo; everyone else wants
to keep the cpython repo as-is with all branches in a single repo.

>
>
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Berker Peksağ]

On Tue, Jan 5, 2016 at 8:13 PM, Brett Cannon  wrote:
> Day 1 summary
> 
>
> Decisions made
> ---
> - We will create a python-dev team in the python org and add that team to
> all of the relevant repos that get migrated
> - We will add a GitHub field to Roundup and then write/get a CLA bot that
> will query Roundup to check if someone has (not) signed the CLA
> - For ancillary repos, the merge-of-least-resistance workflow is fine, so no
> need to worry about maintaining a linear history
> - We are going to keep all of the cpython history in a single repo
> - We will have PRs go against master and then we will cherry pick into
> bugfix branches
> - Misc/ACKS will be kept and we will write code to update it on occasion --
> probably at release time -- based on git commit data
> - Seems like our current commit ID -> URL service can be updated to handle
> our transition
>
>
> Open issues
> ---
> - What tools and commands will we use to convert the repos?
> - How do we want to handle Misc/NEWS?
> - What are the exact commands we are going to expect core devs to run (which
> will be documented in the devguide)?

I use the following commands for python.org:

$ git pr NNN

"git pr" is an alias and an equivalent of

$ git fetch upstream pull/NNN/head:pr/NNN
$ git checkout pr/NNN

For example, if I want to review
https://github.com/python/pythondotorg/pull/859, I use "git pr 859".

Run tests, tweak commit message, code style, and commit

$ git commit -m "Update foo bar"

Squash commits

$ git rebase -i HEAD~2

And merge

   $ git checkout master
   $ git merge --ff-only pr/NNN

(--ff-only can be added to .gitconfig)

Backport to 3.5 (the branch name is "release" for python.org)

   $ git checkout 3.5
   $ git cherry-pick `git rev-parse master`

And push to upstream

   $ git push --all upstream

We probably should mention the -u option (or create a new section for
Git configuration and best practices) in the devguide.

> - Who to use for CI (although Donald is the only one to speak up and likes
> Travis as do I, so this might be decided)?

+1 to use Travis as a pre-merge CI. I'd vote for keeping Travis
configuration simple (e.g. use only Linux and GCC, run rstlint in
Doc/)

--Berker
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Barry Warsaw]

On Jan 05, 2016, at 06:03 PM, Brett Cannon wrote:

>If people are that worried about others being so adverse to using GitHub that
>they won't even do an anonymous clone from their servers then we can get a
>Bitbucket or GitLab clone set up

Once the migration settles down, I do plan on hooking up this GitLab mirror,
which was original created as a PEP 507 experiment:

https://gitlab.com/python-devs/python

Cheers,
-Barry


pgp48Yjdq6o9H.pgp
Description: OpenPGP digital signature
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Senthil Kumaran]

On Tue, Jan 5, 2016 at 12:36 PM, Guido van Rossum  wrote:
>
>
> Why would you want this?
>

In general, our changes to 2.7 are separate from 3.x lines. So the pull
requests and reviews could be handled separately.

We could encourage feature contributions only to cpython (3.x) repo and any
maintainance improvements from the folks who have 2.7 in production could
be handled in 2.7 repo separately.

Also, I was mapping our hg branches to git repos.

Currently we have 3.4->3.5->default (active)
And 2.7(active)

-- 
Senthil
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Donald Stufft]

> On Jan 4, 2016, at 9:42 PM, Barry Warsaw  wrote:
> 
> Something else to consider.  We've long talked about splitting out the stdlib
> to make it easier for the alternative implementations to import.  If some or
> all of them also switch to git, we could do that pretty easily with git
> submodules.

They could also use a subtree merge - 
https://jrsmith3.github.io/merging-a-subdirectory-from-another-repo-via-git-subtree.html

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Donald Stufft]

> On Jan 4, 2016, at 10:45 PM, Nick Coghlan  wrote:
> 
> On 5 January 2016 at 11:08, Donald Stufft  wrote:
>> 
>> On Jan 4, 2016, at 7:42 PM, Brett Cannon  wrote:
>>> We should try to get test coverage wired up as well per CI. I don't know if
>>> coveralls.io or some other provider is best, but we should see what is
>>> available and find out if we can use them to either get basic coverage or
>>> thorough coverage (read https://hg.python.org/devinabox/file/tip/README#l124
>>> to see what thorough coverage entails, but it does require a checkout of
>>> coverage.py).
>> 
>> I prefer codecov, but it shouldn’t be too hard to do. I tried to get Python
>> + C coverage checking in the demo with that, but I failed at making the C
>> coverage work.
> 
> Another posslble tool worth considering is diff_cover:
> https://pypi.python.org/pypi/diff_cover/
> 
> That uses git diff to find the lines affected by a patch and
> specifically looks up *those lines* in a coverage report, so it can
> ensure that any lines changed by a patch are covered by the regression
> test suite. It appears to be a neat way of guiding a code base towards
> more comprehensive test coverage.
> 


FWIW codecov has that built in.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Donald Stufft]

> On Jan 4, 2016, at 7:42 PM, Brett Cannon  wrote:
> 
> So consider this the starting discussion of the PEP that will be the 
> hg.python.org  -> GitHub transition PEP that I will be 
> in charge of. Once we have general agreement on the steps necessary I will 
> start the actual PEP and check it in, but I figure there's no point in have a 
> skeleton PEP if we can't agree on the skeleton. :) While I list steps 
> influencing all the repos, I want to focus on the ones stopping any repo from 
> moving over for now, expanding what we worry about to the cpython repo as we 
> knock blockers down until we move everything over and start adding GitHub 
> perks.
> 
> The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, 
> devguide, and cpython. I also think that's essentially the order we should 
> migrate them over. Some things will need to be universally handled before we 
> transition a single repo, while other things are only a blocker for some of 
> the repos.
> 
> Universal blockers
> ==
> There are four blockers that must be resolved before we even consider moving 
> a repo over. They can be solved in parallel, but they all need to have a 
> selected solution before we can move even the devinabox repo.
> 
> First, we need to decide how we are going to handle adding all the core devs 
> to GitHub. Are we simply going to add all of them to the python organization, 
> or do we want something like a specific python-dev gteamroup that gets added 
> to all of the relevant repos? Basically I'm not sure how picky we want to be 
> about the people who already have organization access on GitHub about them 
> implicitly getting access to the cpython repo at the end of the day (and the 
> same goes for any of the other repos in the python organization). For 
> tracking names, I figure we will create a file in the devguide where people 
> can check in their GitHub usernames and I can manually add people as people 
> add themselves to the file.

We should have a team for each logical group of access I think. We can add 
people to multiple teams, or to just one team. The list of people who can 
access *all* repos on the Python org is https://caremad.io/s/7Q0wcgOcoE/, 
everyone else (currently) has restricted access to one or more repositories.

> 
> Second, CLA enforcement. As of right now people go to 
> https://www.python.org/psf/contrib/contrib-form/ 
> , fill in the form, and 
> then Ewa gets an email where she manually flips a flag in Roundup. If we want 
> to use a web hook to verify someone has signed a CLA then we need to decide 
> where the ground truth for CLAs are. Do we want to keep using Roundup to 
> manage CLA agreements and thus add a GitHub field in bugs.python.org 
>  for people's profile and a web hook or bot that 
> will signal if someone has the flag flipped on bugs.python.org 
> ? Or is there some prepackaged service that we can 
> use that will keep track of this which would cause us to not use Roundup 
> (which might be easier, but depending on the service require everyone to 
> re-sign)? There's also the issue of supporting people who want to submit code 
> by uploading a patch to bugs.python.org  but not use 
> GitHub. Either way I don't want to have to ask everyone who submits a PR what 
> their bugs.python.org  username is and then go check 
> that manually.

There is CLAHub (https://www.clahub.com/ ) but I don’t 
have any idea how good it is, I just know of it’s existence.

> 
> Third, how do we want to do the repo conversions? We need to choose the 
> tool(s) and command(s) that we want to use. There was mention of wanting a 
> mapping from hg commit ID to git commit ID. If we have that we could have a 
> static bugs.python.org/commit/  page that 
> had the mapping embedded in some JavaScript and if  matched then we just 
> forward them to the corresponding GitHub commit page, otherwise just blindly 
> forward to GitHub and assume the ID is git-only, giving us a stable URL for 
> commit web views.

This is the tool I used for the demo repo, it seemed to work ok as long as I 
ran it on Linux: https://github.com/frej/fast-export 


It lets you annotate the git commits with the HG hash as a “git note”, though 
tool support for git notes doesn’t seem to be very good. Github doesn’t display 
them but they are available in the CLI if you run the right command to ask for 
the hg hash that a particular git commit hash came from.

> 
> Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, 
> do we care if we use the GitHub merge button for PRs or do we want to enforce 
> a linear history with all repos? We just need to decide if care about linear 
> histories and 

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Nick Coghlan]

On 5 January 2016 at 10:42, Brett Cannon  wrote:
> So consider this the starting discussion of the PEP that will be the
> hg.python.org -> GitHub transition PEP that I will be in charge of. Once we
> have general agreement on the steps necessary I will start the actual PEP
> and check it in, but I figure there's no point in have a skeleton PEP if we
> can't agree on the skeleton. :) While I list steps influencing all the
> repos, I want to focus on the ones stopping any repo from moving over for
> now, expanding what we worry about to the cpython repo as we knock blockers
> down until we move everything over and start adding GitHub perks.
>
> The way I see it, we have 4 repos to move: devinabox, benchmarks, peps,
> devguide, and cpython. I also think that's essentially the order we should
> migrate them over. Some things will need to be universally handled before we
> transition a single repo, while other things are only a blocker for some of
> the repos.
>
> Universal blockers
> ==
> There are four blockers that must be resolved before we even consider moving
> a repo over. They can be solved in parallel, but they all need to have a
> selected solution before we can move even the devinabox repo.
>
> First, we need to decide how we are going to handle adding all the core devs
> to GitHub. Are we simply going to add all of them to the python
> organization, or do we want something like a specific python-dev gteamroup
> that gets added to all of the relevant repos? Basically I'm not sure how
> picky we want to be about the people who already have organization access on
> GitHub about them implicitly getting access to the cpython repo at the end
> of the day (and the same goes for any of the other repos in the python
> organization). For tracking names, I figure we will create a file in the
> devguide where people can check in their GitHub usernames and I can manually
> add people as people add themselves to the file.

I think we want at least one group to track CPython commit privileges
and potentially a second to track CLA signatories.

For collecting GitHub username info, I think it makes more sense to
add a new User profile field in Roundup than it does to use a text
file in the devguide:
http://roundup.sourceforge.net/docs/customizing.html#tracker-schema

That way whether or not someone has signed the CLA and whether or not
they have commit privileges is directly associated with both their
Roundup user ID and their GitHub one, but the latter is only linked if
they choose to provide it.

It also means that if we eventually have a Roundup hook submitting
patches on behalf of people (perhaps triggered by a "Create PR" link
on the patch display rather than implicitly), it can set "Author" on
the PR correctly if the author has provided a GitHub username.

We should also consider the question of who needs to be added to the
admin group for the GitHub python organisation.

> Second, CLA enforcement. As of right now people go to
> https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then
> Ewa gets an email where she manually flips a flag in Roundup. If we want to
> use a web hook to verify someone has signed a CLA then we need to decide
> where the ground truth for CLAs are. Do we want to keep using Roundup to
> manage CLA agreements and thus add a GitHub field in bugs.python.org for
> people's profile and a web hook or bot that will signal if someone has the
> flag flipped on bugs.python.org? Or is there some prepackaged service that
> we can use that will keep track of this which would cause us to not use
> Roundup (which might be easier, but depending on the service require
> everyone to re-sign)? There's also the issue of supporting people who want
> to submit code by uploading a patch to bugs.python.org but not use GitHub.
> Either way I don't want to have to ask everyone who submits a PR what their
> bugs.python.org username is and then go check that manually.

The way kubernetes does this is that googlebot checks if the submitter
has signed the CLA, and if they have it sets a green "cla: yes" flag
on the PR: https://github.com/kubernetes/kubernetes/labels/cla%3A%20yes

If they haven't, then it posts a message asking them to sign it and
applies a red "cla: no" label:
https://github.com/kubernetes/kubernetes/pull/19271#issuecomment-168836357

For us, I think the approach that makes the most sense depends on
whether or not it's easy to query the Roundup user DB based on a
custom field. If it's easy, then I think we should just have the bot
query Roundup to ask "Is this GitHub user id linked to a user account
that has signed the CLA?".

If querying by custom field is a pain, then I think we should instead
have a GitHub group to track CLA signatories and tweak Roundup to add
someone to that group with their signatory status is updated. The bot
would then check the derived GitHub group rather than querying Roundup
directly.

> Fourth, for the ancillary repos of 

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Nick Coghlan]

On 5 January 2016 at 11:08, Donald Stufft  wrote:
>
> On Jan 4, 2016, at 7:42 PM, Brett Cannon  wrote:
>> We should try to get test coverage wired up as well per CI. I don't know if
>> coveralls.io or some other provider is best, but we should see what is
>> available and find out if we can use them to either get basic coverage or
>> thorough coverage (read https://hg.python.org/devinabox/file/tip/README#l124
>> to see what thorough coverage entails, but it does require a checkout of
>> coverage.py).
>
> I prefer codecov, but it shouldn’t be too hard to do. I tried to get Python
> + C coverage checking in the demo with that, but I failed at making the C
> coverage work.

Another posslble tool worth considering is diff_cover:
https://pypi.python.org/pypi/diff_cover/

That uses git diff to find the lines affected by a patch and
specifically looks up *those lines* in a coverage report, so it can
ensure that any lines changed by a patch are covered by the regression
test suite. It appears to be a neat way of guiding a code base towards
more comprehensive test coverage.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Barry Warsaw]

On Jan 05, 2016, at 12:42 AM, Brett Cannon wrote:

>The way I see it, we have 4 repos to move: devinabox, benchmarks, peps,
>devguide, and cpython.

Arthur: Each core dev converts four repos...
Knight: Five repos
Arthur: He who converts the repos four...
Knight: Five repos
Arthur: Five repos may hack in safety

>There are six things to work out before we move over cpython. First, do we
>want to split out Python 2 branches into their own repo? There might be a
>clone size benefit which obviously is nice for people on slow Internet
>connections. It also clearly separates out Python 2 from 3 and lets those
>who prefer to focus on one compared to the other do that more easily. It
>does potentially make any single fix that spans 2 and 3 require a bit more
>work since it won't be an intra-clone change. We could also contemplate
>sub-repos for things like the Doc/ or Tools/ directories (although I doubt
>it's worth it).

I suspect cherry-picking between Python 2 and 3 branches won't be very smooth
no matter what approach is chosen.  Still, it's probably just as easy to keep
everything in one repo.

>Second, do we want all fixes to go into master and then cherry-pick into
>other branches

IME, yes, that usually works the best for the person doing the cherry picking.
It also seems that all contributed PRs are always just made against master
anyway.

>We should build a bot. It must use a Monty Python reference to trigger
>(black-knight, none-shall-pass, man-from-scene-24, five-questions,
>what-is-your-quest, what-is-your-favourite-colour, etc.; obviously I'm
>leaning towards the Black Knight or Bridge of Death

You've clearly done your homework!  Which probably involved watching MPatHG at
least 10 times. :)

Something else to consider.  We've long talked about splitting out the stdlib
to make it easier for the alternative implementations to import.  If some or
all of them also switch to git, we could do that pretty easily with git
submodules.

Cheers,
-Barry


pgp_bEVrvO0ne.pgp
Description: OpenPGP digital signature
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Ezio Melotti]

On Tue, Jan 5, 2016 at 5:18 AM, Nick Coghlan  wrote:
> On 5 January 2016 at 10:42, Brett Cannon  wrote:
>> ...
>>
>> First, we need to decide how we are going to handle adding all the core devs
>> to GitHub. Are we simply going to add all of them to the python
>> organization, or do we want something like a specific python-dev gteamroup
>> that gets added to all of the relevant repos? Basically I'm not sure how
>> picky we want to be about the people who already have organization access on
>> GitHub about them implicitly getting access to the cpython repo at the end
>> of the day (and the same goes for any of the other repos in the python
>> organization). For tracking names, I figure we will create a file in the
>> devguide where people can check in their GitHub usernames and I can manually
>> add people as people add themselves to the file.
>
> I think we want at least one group to track CPython commit privileges
> and potentially a second to track CLA signatories.
>
> For collecting GitHub username info, I think it makes more sense to
> add a new User profile field in Roundup than it does to use a text
> file in the devguide:
> http://roundup.sourceforge.net/docs/customizing.html#tracker-schema
>

Agreed.

> That way whether or not someone has signed the CLA and whether or not
> they have commit privileges is directly associated with both their
> Roundup user ID and their GitHub one, but the latter is only linked if
> they choose to provide it.
>
> It also means that if we eventually have a Roundup hook submitting
> patches on behalf of people (perhaps triggered by a "Create PR" link
> on the patch display rather than implicitly), it can set "Author" on
> the PR correctly if the author has provided a GitHub username.
>
> We should also consider the question of who needs to be added to the
> admin group for the GitHub python organisation.
>
>> Second, CLA enforcement. As of right now people go to
>> https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then
>> Ewa gets an email where she manually flips a flag in Roundup. If we want to
>> use a web hook to verify someone has signed a CLA then we need to decide
>> where the ground truth for CLAs are. Do we want to keep using Roundup to
>> manage CLA agreements and thus add a GitHub field in bugs.python.org for
>> people's profile and a web hook or bot that will signal if someone has the
>> flag flipped on bugs.python.org? Or is there some prepackaged service that
>> we can use that will keep track of this which would cause us to not use
>> Roundup (which might be easier, but depending on the service require
>> everyone to re-sign)? There's also the issue of supporting people who want
>> to submit code by uploading a patch to bugs.python.org but not use GitHub.
>> Either way I don't want to have to ask everyone who submits a PR what their
>> bugs.python.org username is and then go check that manually.
>
> The way kubernetes does this is that googlebot checks if the submitter
> has signed the CLA, and if they have it sets a green "cla: yes" flag
> on the PR: https://github.com/kubernetes/kubernetes/labels/cla%3A%20yes
>
> If they haven't, then it posts a message asking them to sign it and
> applies a red "cla: no" label:
> https://github.com/kubernetes/kubernetes/pull/19271#issuecomment-168836357
>
> For us, I think the approach that makes the most sense depends on
> whether or not it's easy to query the Roundup user DB based on a
> custom field. If it's easy, then I think we should just have the bot
> query Roundup to ask "Is this GitHub user id linked to a user account
> that has signed the CLA?".
>

It should be easy.

Best Regards,
Ezio Melotti

> If querying by custom field is a pain, then I think we should instead
> have a GitHub group to track CLA signatories and tweak Roundup to add
> someone to that group with their signatory status is updated. The bot
> would then check the derived GitHub group rather than querying Roundup
> directly.
>
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] My initial thoughts on the steps/blockers of the transition

[Ezio Melotti]

On Tue, Jan 5, 2016 at 2:42 AM, Brett Cannon  wrote:
> So consider this the starting discussion of the PEP that will be the
> hg.python.org -> GitHub transition PEP that I will be in charge of. Once we
> have general agreement on the steps necessary I will start the actual PEP
> and check it in, but I figure there's no point in have a skeleton PEP if we
> can't agree on the skeleton. :) While I list steps influencing all the
> repos, I want to focus on the ones stopping any repo from moving over for
> now, expanding what we worry about to the cpython repo as we knock blockers
> down until we move everything over and start adding GitHub perks.
>
> The way I see it, we have 4 repos to move: devinabox, benchmarks, peps,
> devguide, and cpython.

On top of this, there is also the test repo
(https://hg.python.org/test) and all the tracker repos
(https://hg.python.org/tracker/).
I think it would be useful to port the former since it will provide a
place for devs to try things out and experiment (a new test repo could
also be created though).
It would be nice to port the tracker repos too and be consistent with
the others, but it's not a priority.  When we switched to HG they kept
being on SVN until I ported them, so I guess the same thing can be
done (unless R. David or Martin prefer to stick to HG).

> I also think that's essentially the order we should
> migrate them over. Some things will need to be universally handled before we
> transition a single repo, while other things are only a blocker for some of
> the repos.
>
> Universal blockers
> ==
> There are four blockers that must be resolved before we even consider moving
> a repo over. They can be solved in parallel, but they all need to have a
> selected solution before we can move even the devinabox repo.
>
> First, we need to decide how we are going to handle adding all the core devs
> to GitHub. Are we simply going to add all of them to the python
> organization, or do we want something like a specific python-dev gteamroup
> that gets added to all of the relevant repos? Basically I'm not sure how
> picky we want to be about the people who already have organization access on
> GitHub about them implicitly getting access to the cpython repo at the end
> of the day (and the same goes for any of the other repos in the python
> organization). For tracking names, I figure we will create a file in the
> devguide where people can check in their GitHub usernames and I can manually
> add people as people add themselves to the file.
>

I think the current list of core-devs should be converted to a group
and given access to the same repos they have access to now (i.e.
cpython/devguide/peps and possibly others).  Then additional
repo-specific groups can be created in case we want to let specific
contributors work on peps or the devguide.

> Second, CLA enforcement. As of right now people go to
> https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then
> Ewa gets an email where she manually flips a flag in Roundup. If we want to
> use a web hook to verify someone has signed a CLA then we need to decide
> where the ground truth for CLAs are. Do we want to keep using Roundup to
> manage CLA agreements and thus add a GitHub field in bugs.python.org for
> people's profile and a web hook or bot that will signal if someone has the
> flag flipped on bugs.python.org?

This can be done.  We can add a "GitHub" username field to Roundup
users so that we can link the two.


> Or is there some prepackaged service that
> we can use that will keep track of this which would cause us to not use
> Roundup (which might be easier, but depending on the service require
> everyone to re-sign)? There's also the issue of supporting people who want
> to submit code by uploading a patch to bugs.python.org but not use GitHub.
> Either way I don't want to have to ask everyone who submits a PR what their
> bugs.python.org username is and then go check that manually.
>

This also brings up another problem.
Since the discussion about an issue happens on b.p.o and the PRs are
submitted on GitHub, this means that:
1) users with only a GitHub account have to create a b.p.o account if
they want to comment on the issue (exclusing review comments);
2) users with only a b.p.o account have to create a GitHub account if
they want to review a PR;
3) users with both can comment on b.p.o and review on GitHub, but they
might need to login twice.

It would be better if users didn't need to create and use two separate accounts.

> Third, how do we want to do the repo conversions? We need to choose the
> tool(s) and command(s) that we want to use. There was mention of wanting a
> mapping from hg commit ID to git commit ID. If we have that we could have a
> static bugs.python.org/commit/ page that had the mapping embedded in
> some JavaScript and if  matched then we just forward them to the
> corresponding GitHub commit page, otherwise just blindly forward to