[DISCUSS] JIRA vs GitHub Issues

2021-11-28 Thread Vladimir Sitnikov
Hi,

What do you think of moving to GitHub Issues for Calcite?

Currently, my (Calcite) development workflow focuses on pull requests.
That is all contributions I see come via pull requests.

At the same time, issues are hosted in JIRA, which creates friction: PR
merging requires changes to both GitHub and JIRA.
I guess it would be easier if issue management was in GitHub as well.

Then, I believe, co-locating issues and PRs would make external
contributions easier.
The links between Issues and PRs would be easier to navigate.

There's a possibility to enable GitHub discussions as well (see
https://github.com/apache/airflow/discussions ).
Co-locating Issues, and Discussions look promising.

I think it is not that painful, however, GitHub has a limitation of 20
collaborators (non-committers who want to assign, edit, close issues, and
pull requests):
https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-AssigningexternalcollaboratorswiththetriageroleonGitHub

I have no strong opinion, however, I just realized having issues in GitHub
would ease friction.

Any thoughts?

Just to gather opinion:
-1..+1 keep using ASF JIRA for issue management
-1..+1 migrate to GitHub Issues

Here's my vote:
+0.5 keep using ASF JIRA for issue management. It more-or-less works,
however, merging PRs and navigating from PR to issue is far from perfect.
+1 migrate to GitHub Issues. It would simplify navigation between issues
and PRs, and I believe it would reduce friction for external contributors.
In theory, GitHub Issues can be automated with Actions (e.g. labels). I'm
not sure if we need that, however, it might be useful.

Vladimir


Re:[DISCUSS] JIRA vs GitHub Issues

2021-11-28 Thread Zhe Hu
+0.5
As you mentioned, "co-locating issues and PRs would make external
contributions easier”, I do agree with it. What’s more, when people browse this 
Project, they could have a basic picture about the latest progress more 
intuitive.
But I also have some doubts about this migration. How do we process the 
existing issues in JIRA? Moving all of them to Github or just the open issues?


Best,
Zhe Hu







On 11/29/2021 00:43,Vladimir Sitnikov wrote:
Hi,

What do you think of moving to GitHub Issues for Calcite?

Currently, my (Calcite) development workflow focuses on pull requests.
That is all contributions I see come via pull requests.

At the same time, issues are hosted in JIRA, which creates friction: PR
merging requires changes to both GitHub and JIRA.
I guess it would be easier if issue management was in GitHub as well.

Then, I believe, co-locating issues and PRs would make external
contributions easier.
The links between Issues and PRs would be easier to navigate.

There's a possibility to enable GitHub discussions as well (see
https://github.com/apache/airflow/discussions ).
Co-locating Issues, and Discussions look promising.

I think it is not that painful, however, GitHub has a limitation of 20
collaborators (non-committers who want to assign, edit, close issues, and
pull requests):
https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-AssigningexternalcollaboratorswiththetriageroleonGitHub

I have no strong opinion, however, I just realized having issues in GitHub
would ease friction.

Any thoughts?

Just to gather opinion:
-1..+1 keep using ASF JIRA for issue management
-1..+1 migrate to GitHub Issues

Here's my vote:
+0.5 keep using ASF JIRA for issue management. It more-or-less works,
however, merging PRs and navigating from PR to issue is far from perfect.
+1 migrate to GitHub Issues. It would simplify navigation between issues
and PRs, and I believe it would reduce friction for external contributors.
In theory, GitHub Issues can be automated with Actions (e.g. labels). I'm
not sure if we need that, however, it might be useful.

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-28 Thread Vladimir Sitnikov
We can copy all the issues, including resolved ones.

We can import issues. GitHub has api for importing without notifications.
Infra team can execute such a script (we need to find it)

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-28 Thread Jing Zhang
Hi Vladimir,
Thanks a lot for driving this discussion.
+ 0.5
1. First, I have same concern with @Zhe, how to deal with the existing
issues.
If there already exists tools to import these issue, I'm OK with that.
2. There are some useful functions in JIRA, I would list my favorite
functions, I'm not sure whether github issue tracking could support these
features.
   1. Mark the fix versions of each JIRA issue. It is visible for users to
find which version contain this functionality or fix this bug.
   2. Watch the interested issue so I could receive email once this issue
has been changed.
   3. Relate the current JIRA issue with it's related JIRA. It is useful
for other developers or users to understand this story.
3. JIRA is intuitive and friendly to non-technical people and new technical
users.
From my experience in Apache Flink community, this point is very useful
because most of bug issues are reported by users instead of developer.

Best,
Jing Zhang

Vladimir Sitnikov  于2021年11月29日周一 下午2:39写道:

> We can copy all the issues, including resolved ones.
>
> We can import issues. GitHub has api for importing without notifications.
> Infra team can execute such a script (we need to find it)
>
> Vladimir
>


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-29 Thread Vladimir Sitnikov
>Mark the fix versions of each JIRA issue. It is visible for users to
>find which version contain this functionality or fix this bug.

GitHub has "milestones".
See how it is used in Airflow:
https://github.com/apache/airflow/milestone/45?closed=1

>2. Watch the interested issue so I could receive email once this issue
>has been changed.

With GitHub you can subscribe to the issues or PRs you like, and you get
the notifications.

>3. Relate the current JIRA issue with it's related JIRA.

Here's an example: https://github.com/apache/airflow/issues/19788

Note how the following appears in the #19788 (so cross-links like
Issue-Issue, Issue-PR, PR-PR, and so on work)  issues:
>uranusjr mentioned this issue 6 days ago
>Adjust built-in base_aws methods to avoid Deprecation warnings #19725

Here's how the mention look like:
https://github.com/apache/airflow/pull/19725#issuecomment-977078128

Note: GitHub has a beta program for Projects at the moment which would
improve
issue management way further:
https://docs.github.com/en/issues/trying-out-the-new-projects-experience/about-projects

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-29 Thread Vladimir Sitnikov
Here's Airflow vote thread (~2020):
https://lists.apache.org/thread/s5nk0mgblkzq9jrqgrvp12yn4kmy0o7b
here's the discussion:
https://lists.apache.org/thread/6lfth3o4gzk3s891bcqy045z5gl4fs07

I believe draft PRs could avoid "DISCUS" threads that suck time from
everybody and those DISCUSS threads add nothing to the code :-/
For instance, Julian has recently launched "Refactoring tests".
I think it would be just fine if Julian pushed a draft of the intended
changes, we just added +1 and went forward.

Just to clarify: the whole idea of moving to GitHub issues for Calcite
started when
Apache JMeter team started a discussion regarding moving from Bugzilla to
GitHub Issues.

Currently, I think that JIRA for Calcite is like learned helplessness.
Everybody knows "JIRA is required", however, nobody questions that.

I do not say JIRA is bad. I just say 99% of code changes happen in GitHub,
so unifying issues and PRs would be a big plus.



Airflow team mentions that having a JIRA issue for each PR adds unnecessary
overhead.
I believe in many cases we could be just fine if we have everything right
in the PR itself.

For example, do we really need to create JIRA issues like "[CALCITE-4836]
Upgrade protobuf-java 3.6.1 -> 3.17.1"?
I believe filing a PR is just enough

In many cases, having a proper comment in PR helps *A LOT* for the reviewer.
https://github.com/apache/calcite/pull/2615 is a perfect example.
PR 2615 does have a description that clarifies the reason for the change.
It is just fine without JIRA.

In many cases, I tend to copy descriptions between JIRA and PR because it
does help for the review:
it is much faster to switch between "conversation" and "files changed" tabs
than to load the corresponding JIRA.
Here's an example: https://github.com/apache/calcite/pull/2410

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-29 Thread Michael Mior
+0 to migrate to GH issues

I haven't found navigating JIRA + GitHub PRs to be a significant
problem, but if others think it would help, I'm for it. One question
when it comes to migration: are comments and everything transferred
using the ASF-linked GH account? That is, my JIRA account is mmior,
but my GitHub is michaelmior. Since I have this GH account linked to
my ASF account, will my GH comments end up as coming from
@michaelmior?

--
Michael Mior
mm...@apache.org

Le dim. 28 nov. 2021 à 11:43, Vladimir Sitnikov
 a écrit :
>
> Hi,
>
> What do you think of moving to GitHub Issues for Calcite?
>
> Currently, my (Calcite) development workflow focuses on pull requests.
> That is all contributions I see come via pull requests.
>
> At the same time, issues are hosted in JIRA, which creates friction: PR
> merging requires changes to both GitHub and JIRA.
> I guess it would be easier if issue management was in GitHub as well.
>
> Then, I believe, co-locating issues and PRs would make external
> contributions easier.
> The links between Issues and PRs would be easier to navigate.
>
> There's a possibility to enable GitHub discussions as well (see
> https://github.com/apache/airflow/discussions ).
> Co-locating Issues, and Discussions look promising.
>
> I think it is not that painful, however, GitHub has a limitation of 20
> collaborators (non-committers who want to assign, edit, close issues, and
> pull requests):
> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-AssigningexternalcollaboratorswiththetriageroleonGitHub
>
> I have no strong opinion, however, I just realized having issues in GitHub
> would ease friction.
>
> Any thoughts?
>
> Just to gather opinion:
> -1..+1 keep using ASF JIRA for issue management
> -1..+1 migrate to GitHub Issues
>
> Here's my vote:
> +0.5 keep using ASF JIRA for issue management. It more-or-less works,
> however, merging PRs and navigating from PR to issue is far from perfect.
> +1 migrate to GitHub Issues. It would simplify navigation between issues
> and PRs, and I believe it would reduce friction for external contributors.
> In theory, GitHub Issues can be automated with Actions (e.g. labels). I'm
> not sure if we need that, however, it might be useful.
>
> Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-29 Thread Vladimir Sitnikov
>I haven't found navigating JIRA + GitHub PRs to be a significant
>problem, but if others think it would help

GitHub has a single search field, so you can search for both PRs and issues
at the same time which is nice as well.

>will my GH comments end up as coming from @michaelmior

If we migrate issues, then they will be created by a "bot" account.
Of course, we can add comment author in the comment itself.
Here's how Coq migrated their issues: https://github.com/coq/coq/issues/2268

We can figure out which accounts do we want to use (e.g. create email ->
account mapping file).

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-29 Thread Josh Elser
-0 on migrating to GH issues. I'd strongly warn against making a change 
without a clear and documented plan for issue tagging into a release and 
how the release process would change. I do think that we need to have a 
single place to manage issues and that Github could have less friction 
associated with it; however, I'd want to see that demonstrated before 
changing all of Calcite over.


Anecdote: I was involved with another project a long time ago which 
moved from Jira issues to Github issues. Despite concerns of others, the 
migration moved ahead without stepping through the common use cases 
(e.g. developer finds a bug they want to fix, contributor wants to fix a 
bug, developer/PMC wants to make a release) and how issues should be 
created/tagged/modified in each scenario.


IMO, GH issues created a black hole where anyone could create issues and 
they could just get "lost" if not assigned to a milestone and properly 
moved between them. Should PRs always have issues or is a PR sufficient 
for inclusion in a release?


All that said, this was a number of years ago and perhaps the 
project/release management on the Github side is more structured than 
before. A little bit of documentation will go a very long way in making 
sure that such a fundamental change in process does not cause a 
significant burden.


I'm a little confused in your original message, you have two questions. 
Are you suggesting that Calcite would use both Jira and Github issues?


On 11/28/21 11:43 AM, Vladimir Sitnikov wrote:

Hi,

What do you think of moving to GitHub Issues for Calcite?

Currently, my (Calcite) development workflow focuses on pull requests.
That is all contributions I see come via pull requests.

At the same time, issues are hosted in JIRA, which creates friction: PR
merging requires changes to both GitHub and JIRA.
I guess it would be easier if issue management was in GitHub as well.

Then, I believe, co-locating issues and PRs would make external
contributions easier.
The links between Issues and PRs would be easier to navigate.

There's a possibility to enable GitHub discussions as well (see
https://github.com/apache/airflow/discussions ).
Co-locating Issues, and Discussions look promising.

I think it is not that painful, however, GitHub has a limitation of 20
collaborators (non-committers who want to assign, edit, close issues, and
pull requests):
https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-AssigningexternalcollaboratorswiththetriageroleonGitHub

I have no strong opinion, however, I just realized having issues in GitHub
would ease friction.

Any thoughts?

Just to gather opinion:
-1..+1 keep using ASF JIRA for issue management
-1..+1 migrate to GitHub Issues

Here's my vote:
+0.5 keep using ASF JIRA for issue management. It more-or-less works,
however, merging PRs and navigating from PR to issue is far from perfect.
+1 migrate to GitHub Issues. It would simplify navigation between issues
and PRs, and I believe it would reduce friction for external contributors.
In theory, GitHub Issues can be automated with Actions (e.g. labels). I'm
not sure if we need that, however, it might be useful.

Vladimir



Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-29 Thread Vladimir Sitnikov
>All that said, this was a number of years ago

I agree. GitHub makes noticeable progress nowadays.

>I'd want to see that demonstrated before
>changing all of Calcite over.

How do you want that demonstrated?
Apache Airflow does use GitHub Issues, they do ship releases.

Many top-starred Apache repositories use GitHub issues:
https://github.com/orgs/apache/repositories?q=&type=&language=&sort=stargazers

Do you think Calcite has something unique in the release process that makes
GitHub infeasible?

Apparently, issues and PRs can be assigned to a milestone.
Apparently, committers can close the milestone.

>I'm a little confused in your original message, you have two questions.
>Are you suggesting that Calcite would use both Jira and Github issues?

No way. I think we should move all issue management to GitHub.
I just thought that questions like "should we keep JIRA" or "should we move
to GitHub" might
provoke slightly different discussions with helpful comments even though it
is basically the same question.

>Should PRs always have issues or is a PR sufficient
>for inclusion in a release?

I believe, the current development workflow allows:
* PR without JIRA
* commits without PR
* commits without PR and JIRA (just commits)

It is up to the committer if the change is significant enough to have JIRA
or PR or whatever.

>GH issues created a black hole where anyone could create issues and
>they could just get "lost"

Nothing prevents the issue from being "lost" in Calcite JIRA :-/

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-30 Thread Josh Elser

Thanks for your reply, Vladimir.

On 11/29/21 4:44 PM, Vladimir Sitnikov wrote:

Do you think Calcite has something unique in the release process that makes
GitHub infeasible?


Nope, just calling attention that we want releases to be "easy" to make 
and anyone should be able to make them (not just someone who has done 
one before)



Apparently, issues and PRs can be assigned to a milestone.
Apparently, committers can close the milestone.


I'll acknowledge that I'm asking a vague question that you probably 
don't have an answer to :). I think the easiest thing is to focus on the 
"workflow" for each "persona" we have:


* contributor
* developer
* release manager

For each of these people, define a contrived scenario and just make a 
list of what steps they would do. For example, what would a contributor 
do on Github to provide a patch for a bug they found? What about a 
contributor reporting an issue? How should a developer structure a "big" 
change to Calcite which might contain multiple commits? Where does a 
release manager look to determine if a release is ready (all necessary 
issues are fixed)?


This doesn't need to be a perfect or exhaustive list, but having 
something to reference (which others can then use/modify/update) would 
be a huge help for the whole community.



I'm a little confused in your original message, you have two questions.
Are you suggesting that Calcite would use both Jira and Github issues?


No way. I think we should move all issue management to GitHub.
I just thought that questions like "should we keep JIRA" or "should we move
to GitHub" might
provoke slightly different discussions with helpful comments even though it
is basically the same question.


Gotcha. I definitely think we need to have exactly one system.


Should PRs always have issues or is a PR sufficient
for inclusion in a release?


I believe, the current development workflow allows:
* PR without JIRA
* commits without PR
* commits without PR and JIRA (just commits)

It is up to the committer if the change is significant enough to have JIRA
or PR or whatever.


Writing down what you believe to be "allowed" and making sure everyone 
else is on the same page is perfect.


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-30 Thread Vladimir Sitnikov
>I think the easiest thing is to focus on the
>"workflow" for each "persona" we have

While I agree the questions you raise are valid, I would say, that moving
to GitHub Issues
does not really change the workflow except in certain small cases.

I doubt there's an exhaustive list of answers for JIRA.
No way I want to bomb you with answers, however, I truly believe replacing
JIRA with GitHub is not really disrupting for Calcite.

On the other hand, moving to GitHub issues would likely bring new
contributors and reviewers.

>Writing down what you believe to be "allowed" and making sure everyone
>else is on the same page is perfect.

https://calcite.apache.org/develop/#contributing is more or less relevant
if you replace JIRA with GitHub Issue.

>what would a contributor
>do on Github to provide a patch for a bug they found?

Like with the current workflow: they file a PR.
They don't have to file a JIRA first. They just file a PR that explains the
problem and suggests the fix.

>What about a contributor reporting an issue?

a) If they are not sure, they can file an issue at GitHub
b) They can ask on a dev@calcite mailing list
c) If they can't understand how to use Calcite APIs, they could file GitHub
Discussion (if we enable it).
It might be useful to distinguish "bugs in Calcite" vs "user
questions regarding the way to use Calcite".
d) If they have a reproducer, they could just file a PR that reproduces the
issue. It would immediately
show up as a build failure, and everybody would be able to see how it
breaks.

>How should a developer structure a "big"
>change to Calcite which might contain multiple commits?

a) They can file PR that contains several commits (like we currently do
sometimes)
b) They can create an issue and reference the relevant PRs/issues in the
description

"tasks lists" is useful here:
https://docs.github.com/en/issues/tracking-your-work-with-issues/about-task-lists

^^ the above answers are pretty much the same as our current workflow
except JIRA is replaced with GitHub issues.

>Where does a
>release manager look to determine if a release is ready (all necessary
>issues are fixed)?

Typically, when planning a release, we ask everybody to figure out which
issues are "blockers" for the release.
With GitHub, we can associate the relevant issues/PRs with the milestone.

Here's how "open items" look in Airflow:
https://github.com/apache/airflow/milestones/Airflow%202.2.3
The release manager opens the milestone, and they see if it is all ready.

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-30 Thread Stamatis Zampetakis
Thanks for starting this discussion Vladimir and all those sharing your
experience. Quite interesting information so far.

As usual, there are advantages and disadvantages between GitHub issues and
JIRA issues. In order to avoid writing an overly long email I will try to
focus on those aspects that I consider more important.

As a user I want to be able to easily find a problem/feature, see if it is
fixed, and if yes on which version(s). When Google fails me, I turn to JIRA
or GitHub and it turns out that I am able to find the information easier
with JIRA.

I find JIRA search superior to GitHub both when it comes to approximate
text search and exact filters. Moreover, many Apache projects are using
JIRA already which makes it easy to perform cross project search. I admit,
I have more experience with JIRA compared to GitHub issues so maybe I am
just lacking knowledge.

Clearly, the interaction between JIRA and GitHub is not perfect but doesn't
bother me much.

One unpleasant situation which comes up from time to time is when
discussions develop in both places. However, migrating to GitHub issues
will not completely solve this since we will still need to navigate back
and forth between issue and PR.

Last but not least, I know people/companies who have built tools/plugins
around JIRA for getting useful information such as which internal forks
have this particular fix, etc. It could be trivial to adapt but still it is
work that needs to be done. This is to say that the migration may have a
bigger impact than expected.

Small reminder:
Since someone brought up how to determine if a release is ready; in JIRA we
currently have the following dashboards:
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333951
I am using the first for monitoring the state of the release mostly when I
am RM and the second for staying up to date in discussions, important
issues, todos, etc.

Best,
Stamatis

On Tue, Nov 30, 2021 at 7:49 PM Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> wrote:

> >I think the easiest thing is to focus on the
> >"workflow" for each "persona" we have
>
> While I agree the questions you raise are valid, I would say, that moving
> to GitHub Issues
> does not really change the workflow except in certain small cases.
>
> I doubt there's an exhaustive list of answers for JIRA.
> No way I want to bomb you with answers, however, I truly believe replacing
> JIRA with GitHub is not really disrupting for Calcite.
>
> On the other hand, moving to GitHub issues would likely bring new
> contributors and reviewers.
>
> >Writing down what you believe to be "allowed" and making sure everyone
> >else is on the same page is perfect.
>
> https://calcite.apache.org/develop/#contributing is more or less relevant
> if you replace JIRA with GitHub Issue.
>
> >what would a contributor
> >do on Github to provide a patch for a bug they found?
>
> Like with the current workflow: they file a PR.
> They don't have to file a JIRA first. They just file a PR that explains the
> problem and suggests the fix.
>
> >What about a contributor reporting an issue?
>
> a) If they are not sure, they can file an issue at GitHub
> b) They can ask on a dev@calcite mailing list
> c) If they can't understand how to use Calcite APIs, they could file GitHub
> Discussion (if we enable it).
> It might be useful to distinguish "bugs in Calcite" vs "user
> questions regarding the way to use Calcite".
> d) If they have a reproducer, they could just file a PR that reproduces the
> issue. It would immediately
> show up as a build failure, and everybody would be able to see how it
> breaks.
>
> >How should a developer structure a "big"
> >change to Calcite which might contain multiple commits?
>
> a) They can file PR that contains several commits (like we currently do
> sometimes)
> b) They can create an issue and reference the relevant PRs/issues in the
> description
>
> "tasks lists" is useful here:
>
> https://docs.github.com/en/issues/tracking-your-work-with-issues/about-task-lists
>
> ^^ the above answers are pretty much the same as our current workflow
> except JIRA is replaced with GitHub issues.
>
> >Where does a
> >release manager look to determine if a release is ready (all necessary
> >issues are fixed)?
>
> Typically, when planning a release, we ask everybody to figure out which
> issues are "blockers" for the release.
> With GitHub, we can associate the relevant issues/PRs with the milestone.
>
> Here's how "open items" look in Airflow:
> https://github.com/apache/airflow/milestones/Airflow%202.2.3
> The release manager opens the milestone, and they see if it is all ready.
>
> Vladimir
>


Re: [DISCUSS] JIRA vs GitHub Issues

2021-11-30 Thread Vladimir Sitnikov
>I know people/companies who have built tools/plugins
>around JIRA for getting useful information such as which internal forks
>have this particular fix, etc

That is true. However, I do not know what should we do about it.
Of course, any change brings hidden costs, so if the benefits are unclear,
there's not much point in making changes.

On the other hand, the more I discover, the more I am sure GitHub Issues
would be great for Calcite.
I started the discussion like "it looks we are fine, but we can try
GitHub", and now I am quite
enthusiastic that moving to GitHub would bring multiple improvements in
various aspects.

>I turn to JIRA
>or GitHub and it turns out that I am able to find the information easier
>with JIRA.

GitHub issues are searchable, including comments.
We currently "enforce" JIRA tickets, so PRs lack descriptions and
explanations, so it is expected you can't find them.

Try using JIRA to figure out when we bumped Guava to 31. There's no JIRA
for that.

On the other hand, GitHub search returns PRs, code files, and commits that
match the search.
https://github.com/apache/calcite/search?q=guava+31

Then, if you know Git SHA (e.g. from "git blame"), you can find PRs related
to that SHA.
For example
https://github.com/apache/calcite/search?q=890eb61ef486e2192110cefe4cac5aa6f150&type=issues
Frankly speaking, I never thought of trying to search via commit SHA, but
it is a definitely cool trick I learned
when reading
https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests

When it comes to versions, GitHub PRs and issues would have "milestones" (~
"fix version"),
and when you browse commits, GitHub shows the list of tags that include
this commit.
For instance, if you open
https://github.com/apache/calcite/commit/2326cf8dc97f473d0efa2b0e4883fef87a40746b
,
you know Guava was updated to 31 since Calcite 1.28.0

>One unpleasant situation which comes up from time to time is when
>discussions develop in both places. However, migrating to GitHub issues
>will not completely solve this since we will still need to navigate back
>and forth between issue and PR

If we allow PR-first (no issue, no JIRA, just PR with the proper
description) contributions, then:
1) It will be in a single place, no need to switch
2) Navigation between issues and comments would still be easier in GitHub
than commenting like "please continue in JIRA"

GitHub allows responding to issues/PRs via mail, and I find it a nice
feature.
I don't use it often, however, I use it from time to time, and I see others
using it.

>Moreover, many Apache projects are using
>JIRA already which makes it easy to perform cross project search

GitHub has an out-of-the-box way to scope search for organizations.

Here's a search for Calcite in the Apache organization:
https://github.com/search?q=org%3Aapache+calcite

At any point in time, you can remove org:apache, and you get GitHub-wide
search.
GitHub hosts more projects than ASF JIRA, so being able to search and link
across them (all, not just ASF projects) would be useful.

>I find JIRA search superior to GitHub both when it comes to approximate
>text search and exact filters

I might be wrong here, however, I think we do not really use JIRA metadata.
In most cases, we use only the ticket title, description, and the fix
version.

There's "affects version" in JIRA, and I do not know if GitHub has a
one-to-one replacement for that.
However, I would be fine if the affected version was in the issue/PR
description.

"component" (~core, linq4j, cassandra, ...) could become a label in GitHub.
AFAIK the limitation is that labels can only be changed by committers or
collaborators.
We can mitigate that by:
a) Adding collaborators (via asf.yaml)
b) Adding committers
c) Adding GitHub Actions that label Issues based on descriptions or even
PRs based on the set of touched files.

GitHub's search features are more than enough for us. We have no fancy
metadata for complicated searches :)

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-01 Thread Vladimir Sitnikov
Stamatis>Quite interesting information so far

One more data point: LLVM is migrating issues from their Bugzilla to GitHub
Issues right now (52K issues!)
See https://llvm.discourse.group/t/bugzilla-migration/4848

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-01 Thread Jacques Nadeau
I'm generally -0.47 on this.

This seems to solve a problem that doesn't exist (or at least isn't
pressing). Calcite contribution speed is very helpful. Don't get me wrong,
I hate jira as much as anyone. Nonetheless, process change takes time in
both the direct costs (migration/workflow disruption/etc) and the indirect
costs (learning/refining a new system).

A bigger problem around our process that I see is not enough reviews/triage
for the contributions we do receive. We have 224 open prs and I'd estimate
a median open time of 1 year+. Many of those may now be abandoned but it'd
be great to actually mark them as such (if we think that). I'm much more
excited about creative ways to solve that problem and if someone feels like
putting time against process improvement, I think that would be a good
place for it.

To be clear, I prefer GitHub issues for new things and do agree that it
reduces the bar towards contribution.



On Tue, Nov 30, 2021 at 11:39 PM Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> wrote:

> >I know people/companies who have built tools/plugins
> >around JIRA for getting useful information such as which internal forks
> >have this particular fix, etc
>
> That is true. However, I do not know what should we do about it.
> Of course, any change brings hidden costs, so if the benefits are unclear,
> there's not much point in making changes.
>
> On the other hand, the more I discover, the more I am sure GitHub Issues
> would be great for Calcite.
> I started the discussion like "it looks we are fine, but we can try
> GitHub", and now I am quite
> enthusiastic that moving to GitHub would bring multiple improvements in
> various aspects.
>
> >I turn to JIRA
> >or GitHub and it turns out that I am able to find the information easier
> >with JIRA.
>
> GitHub issues are searchable, including comments.
> We currently "enforce" JIRA tickets, so PRs lack descriptions and
> explanations, so it is expected you can't find them.
>
> Try using JIRA to figure out when we bumped Guava to 31. There's no JIRA
> for that.
>
> On the other hand, GitHub search returns PRs, code files, and commits that
> match the search.
> https://github.com/apache/calcite/search?q=guava+31
>
> Then, if you know Git SHA (e.g. from "git blame"), you can find PRs related
> to that SHA.
> For example
>
> https://github.com/apache/calcite/search?q=890eb61ef486e2192110cefe4cac5aa6f150&type=issues
> Frankly speaking, I never thought of trying to search via commit SHA, but
> it is a definitely cool trick I learned
> when reading
>
> https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests
>
> When it comes to versions, GitHub PRs and issues would have "milestones" (~
> "fix version"),
> and when you browse commits, GitHub shows the list of tags that include
> this commit.
> For instance, if you open
>
> https://github.com/apache/calcite/commit/2326cf8dc97f473d0efa2b0e4883fef87a40746b
> ,
> you know Guava was updated to 31 since Calcite 1.28.0
>
> >One unpleasant situation which comes up from time to time is when
> >discussions develop in both places. However, migrating to GitHub issues
> >will not completely solve this since we will still need to navigate back
> >and forth between issue and PR
>
> If we allow PR-first (no issue, no JIRA, just PR with the proper
> description) contributions, then:
> 1) It will be in a single place, no need to switch
> 2) Navigation between issues and comments would still be easier in GitHub
> than commenting like "please continue in JIRA"
>
> GitHub allows responding to issues/PRs via mail, and I find it a nice
> feature.
> I don't use it often, however, I use it from time to time, and I see others
> using it.
>
> >Moreover, many Apache projects are using
> >JIRA already which makes it easy to perform cross project search
>
> GitHub has an out-of-the-box way to scope search for organizations.
>
> Here's a search for Calcite in the Apache organization:
> https://github.com/search?q=org%3Aapache+calcite
>
> At any point in time, you can remove org:apache, and you get GitHub-wide
> search.
> GitHub hosts more projects than ASF JIRA, so being able to search and link
> across them (all, not just ASF projects) would be useful.
>
> >I find JIRA search superior to GitHub both when it comes to approximate
> >text search and exact filters
>
> I might be wrong here, however, I think we do not really use JIRA metadata.
> In most cases, we use only the ticket title, description, and the fix
> version.
>
> There's "affects version" in JIRA, and I do not know if GitHub has a
> one-to-one replacement for that.
> However, I would be fine if the affected version was in the issue/PR
> description.
>
> "component" (~core, linq4j, cassandra, ...) could become a label in GitHub.
> AFAIK the limitation is that labels can only be changed by committers or
> collaborators.
> We can mitigate that by:
> a) Adding collaborators (via asf.yaml)
> b) Adding committers
> c) Adding GitHub Act

Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-01 Thread Vladimir Sitnikov
>A bigger problem around our process that I see is not enough reviews/triage
for the contributions we do receive. We have 224 open prs and I'd estimate
a median open time of 1 year+

I am sure JIRA is slowing us down here.

If we accept we do not need JIRA, PRs could become the way to go for the
reviews.

Jacques, I see you've just created
https://github.com/apache/calcite/pull/2625 (generic visitor), and you
followed the exact pattern I suggest: you filed a PR without JIRA, and you
put all the clarifications into the description.
Congratulations. It confirms JIRA adds no value to our workflow.

Your https://github.com/apache/calcite/pull/2624 (introduce
handlerbased...) is less fun. Why did you create JIRA? ;)
It could have been a JIRA-less PR as well.

As you put description to the PR itself, it becomes easier to review.

That is what I mean.



We rarely reject contribution or ideas, and, I believe, we could reduce
time-to-merge by lowering the barrier and allowing PR-first contributions.

Let me put it as follows: in many cases committers and experienced
contributors know they are doing good.
For example, I see you are moving towards AOT. That is fine, and I do not
really need all those JIRA's.
It would be so much easier to review and follow if you created a single
Issue that references all your AOT PRs.

It is not really fun to create tickets for moving to JUnit5. We could just
do it instead of spending time on JIRA comments.



>I'd estimate a median open time of 1 year+

I suggested moving "core test framework classes" from core/src/test/ into
its own test module long ago. My mistake was I sent an email instead of
just sending a PR.

The response to my mail was strongly negative, however, recently I did the
split, and it works great.

>This seems to solve a problem that doesn't exist

It does exist. My current mail highlights it right away: you could submit
JIRA-less PRs, everybody would have a single page for reviews, and so on.
It is really sad to hear "does not exist".

JIRA causes excessive mails: "jira created", "pr created".

In case we skip JIRA completely, it would be less notifications for
everybody.

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-02 Thread Julian Hyde
For what it’s worth, I actually prefer JIRA’s workflow model. I strongly 
believe that people should log a JIRA case before starting work on a bug or 
feature. Then discussion can happen before coding starts.

And I like making holistic reviews - reviewing the whole change, its goals, its 
impact, and what’s missing - rather than reviewing line by line.

GitHub makes it easy to review line by line, but you can then miss the 
important stuff.

GitHub also makes it easy to write code, submit it as a PR, and that PR becomes 
the issue.

Lastly, another quibble about GitHub: that it allows people to squash, rebase 
and amend commits. The line-by-line comments quickly go out of date.

As to the backlog of PRs. There are multiple problems and solutions. The main 
one is that reviewing PRs quite simply takes work. We need more people to spend 
more time reviewing PRs. We have solved the problem for RMs - people volunteer, 
and put in the effort, and there is a rota so that no one person is doing all 
the work.

Julian



> On Dec 1, 2021, at 10:14 PM, Vladimir Sitnikov  
> wrote:
> 
>> A bigger problem around our process that I see is not enough reviews/triage
> for the contributions we do receive. We have 224 open prs and I'd estimate
> a median open time of 1 year+
> 
> I am sure JIRA is slowing us down here.
> 
> If we accept we do not need JIRA, PRs could become the way to go for the
> reviews.
> 
> Jacques, I see you've just created
> https://github.com/apache/calcite/pull/2625 (generic visitor), and you
> followed the exact pattern I suggest: you filed a PR without JIRA, and you
> put all the clarifications into the description.
> Congratulations. It confirms JIRA adds no value to our workflow.
> 
> Your https://github.com/apache/calcite/pull/2624 (introduce
> handlerbased...) is less fun. Why did you create JIRA? ;)
> It could have been a JIRA-less PR as well.
> 
> As you put description to the PR itself, it becomes easier to review.
> 
> That is what I mean.
> 
> 
> 
> We rarely reject contribution or ideas, and, I believe, we could reduce
> time-to-merge by lowering the barrier and allowing PR-first contributions.
> 
> Let me put it as follows: in many cases committers and experienced
> contributors know they are doing good.
> For example, I see you are moving towards AOT. That is fine, and I do not
> really need all those JIRA's.
> It would be so much easier to review and follow if you created a single
> Issue that references all your AOT PRs.
> 
> It is not really fun to create tickets for moving to JUnit5. We could just
> do it instead of spending time on JIRA comments.
> 
> 
> 
>> I'd estimate a median open time of 1 year+
> 
> I suggested moving "core test framework classes" from core/src/test/ into
> its own test module long ago. My mistake was I sent an email instead of
> just sending a PR.
> 
> The response to my mail was strongly negative, however, recently I did the
> split, and it works great.
> 
>> This seems to solve a problem that doesn't exist
> 
> It does exist. My current mail highlights it right away: you could submit
> JIRA-less PRs, everybody would have a single page for reviews, and so on.
> It is really sad to hear "does not exist".
> 
> JIRA causes excessive mails: "jira created", "pr created".
> 
> In case we skip JIRA completely, it would be less notifications for
> everybody.
> 
> Vladimir



Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-02 Thread Vladimir Sitnikov
Julian, it looks like you have not been using GitHub for the past 5 years,
and you miss a lot of improvements.
GitHub allows a superset of what we do in JIRA, so I really can't
understand why do you suggest enforcing JIRA.

None of what you say sounds like a "good point to keep using JIRA", and
none of what you say sounds like "a pitfall of using GitHub".

>As to the backlog of PRs

I could have just merged 2628, however:
1) We spend time on discussions in JIRA
2) After PR merge, someone would have to "resolve the JIRA"

That consumes the reviewer's time.

Julian>GitHub also makes it easy to write code, submit it as a PR, and that
PR becomes the issue.

That is a great GitHub feature. I do not see how do you treat it as
something bad.
It really makes no sense to create JIRA for adding tests, updating
documentation.

Julian>Lastly, another quibble about GitHub: that it allows people to
squash, rebase and amend commits. The line-by-line comments quickly go out
of date.

squash, rebase, and amendments make JIRA comments go out of date at the
same rate.
However, in GitHub there's an explicit "resolve discussion", "reopen
discussion" features,
so there's a clear way to tell when the discussion is closed.

JIRA has no conversation threads. GitHub has threads (code-related).

Julian>GitHub makes it easy to review line by line, but you can then miss
the important stuff.

Do you mean we should stop code reviews?
If we review code, then using both "JIRA and GitHub" makes it way easier to
miss stuff.

However, I agree we should review high-level ideas first, and put
line-by-line comments
only when high-level direction is OK.

GitHub UI does have regular comments that are not attached to the diff
lines,
so you can comment there just like you add JIRA comments.

Julian>And I like making holistic reviews

What stops you from adding an overall review in GitHub?

Julian>Then discussion can happen before coding starts.

https://github.com/apache/calcite/pull/2628 is a perfect example here.
The PR is trivial, and it adds a couple of tests.

The PR looks good to me, except for method names.
However, I truly do not know how to make test names better in Java, so I
just ignore it.
In Kotlin, one could use method name with spaces, so it would be better,
however, I just ignored it.

Then Julian comments "The test case names are not very useful".
This should really belong to line-by-line diff rather than JIRA ticket.
Now everyone should navigate between JIRA and PR to figure out the meaning.

I see Julian suggests adding all the tests into a single method,
however, I think we should just merge the thing and get over it.

I am sure the discussion in CALCITE-4917 is not worth the time we spend
there.
We are discussing 2 lines of code there. Exactly two lines of test code.
It would be OK to discuss method vs class when it comes to 100 lines.

Even adding tests to JdbcTest does not have those levels of discussions.

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-06 Thread Vladimir Sitnikov
Moving the reply to DISCUSS thread.

Ruben>My vote might be biased because I have used Jira for many years (and
I have
Ruben>never used Github for issue tracking).

It is really sad to hear.
You authored 87 PRs for Calcite!
You did use GitHub for tracking already.
https://github.com/apache/calcite/pulls?q=is%3Apr+author%3Arubenada+is%3Aclosed

Here's a good example: https://github.com/apache/calcite/pull/2329
PR description says the intention of the feature.
There are overall and line-targeted comments.
Creating GitHub Issue is not that different from creating a PR.

Ruben>I have the impression that some of the problems described discussion
are
Ruben>not per se Jira-related, but they appear because we misuse the current
Ruben>tools (and it would remain more or less the same if we switched from
Jira
Ruben>to GitHub Issues).

Ruben, I claim that we split information for no reason.
What is the real meaning behind having 50% of the review comments in JIRA,
and 50% of the comments in PR?
What is the real meaning behind putting "common description in JIRA, and
putting NO description in the PR"?
Does it help to have different markup languages for JIRA and GitHub?

My main motivation is not "replace JIRA with Issues". That replacement
alone has few benefits.
The key improvement to the current process is creating PRs **with
description**, and keeping the relevant discussion in the PR itself.
I really see no point in PR comments like "let's continue in JIRA".
If a comment targets a specific PR, then why don't we just use PR's comment
fields for that?

Ruben>high level discussion (e.g. feature design) should
Ruben>happen in Jira

The vast majority of the changes are small. Adding Jira barrier for every
change is a productivity killer (for both contributors and for the
reviewers).
I don't mean just "switch between JIRA and PR".
I mean that comments are duplicated at different sites.
The description of the issue and the code changes are separated.
The notifications are duplicated (gitbox duplicates GitHub notifications).
The markup format is different.
The search is not coherent (there's no way to find issues and PRs at the
same time).
GitHub blame does not show the full picture (it shows PRs, however, opening
the JIRA is a separate step).
PR (and GitHub Issue) can be associated to a version, so it would be much
easier to relate the PR with its release version.
I think the list can go and go.

Ruben>IMO every change must have a dedicated Jira ticket

What are the benefits of "a dedicated Jira ticket"?
How "a dedicated Jira ticket" is different from "a dedicated PR"?

Ruben>high level discussion (e.g. feature design) should
Ruben>happen in Jira, low-level details (e.g. line by line code review) in
the
Ruben>PR;

Both can be in PR, so what are the benefits from splitting "low-level" vs
"high-level"?
There is a lot of cases when people discuss low-level details in JIRA.

It would be so much better, if we aligned the review approach.
Jira and GitHub are just tools, and if we don't agree to review high-level
stuff first, then no tool helps.
That implies, GitHub is just fine for high-level reviews.

Ruben>like customized dashboards, or linked issues ("issue A is blocked by /
Ruben>caused by / duplicated by / ... issue B").

GitHub allows links, and links have no properties like "caused by",
"duplicated by".
However, basic links are just enough for Calcite.

If you have a "customized dashboard" that is useful for many contributors,
please show it.
The ASF is a set of volunteers, so it is not right to enforce usage of JIRA
when only a single person needs "a customized dashboard".

Vladimir


Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-06 Thread Ruben Q L
Thanks for moving the discussion here, Vladimir.

First of all, I honestly appreciate the fact that you launched this
discussion, I think you raised some valid points; what you propose sounds
reasonable, and it seems a valid approach too. My main argument is that I
have no big issues with the current approach (which, of course, is not
perfect).

I guess my bottom-line is that I believe that issue tracking and version
control (+ code review) are two different things, which serve two different
purposes, and it is arguably a good thing to keep them separated: using
Jira for the former and Github PRs for the latter is adequate IMO.

> You did use GitHub for tracking already.
As I see it, I have used Jira for issue tracking, and GitHub PRs for code
review + merge. If you ask me, perhaps I am old-fashioned, but I think the
development workflow should focus on the Jira ticket, and not on the GitHub
PR.

> Here's a good example: https://github.com/apache/calcite/pull/2329
I agree it is a good example. The associated Jira has several comments with
a high-level discussion, the PR contains code-review comments for technical
details. I am happy with this decoupling, although sometimes this is not
always the case in all tickets; perhaps we should clarify our current
workflow and try to enforce some rules, instead of changing it into
something completely new.
It is also a good example because the Jira reporter and the PR creator were
not the same person, plus there was a span of almost two years between the
ticket creation and the PR submission. I know this is usually uncommon, but
it shows that there are two different roles / actions: one thing is
describing / analyzing an issue; and a second, different thing is carrying
out its implementation (with its corresponding code review).

> GitHub allows links, and links have no properties like "caused by",
"duplicated by". However, basic links are just enough for Calcite.
I think it would be sad to lose this feature, but I admit it is a minor
detail.

> If you have a "customized dashboard" that is useful for many
contributors, please show it.
I was thinking for example at the dashboard mentioned by Stamatis, which is
very useful (not only for the release manager, but for all contributors)
when a release date is coming.

Best regards,
Ruben


On Mon, Dec 6, 2021 at 12:44 PM Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> wrote:

> Moving the reply to DISCUSS thread.
>
> Ruben>My vote might be biased because I have used Jira for many years (and
> I have
> Ruben>never used Github for issue tracking).
>
> It is really sad to hear.
> You authored 87 PRs for Calcite!
> You did use GitHub for tracking already.
>
> https://github.com/apache/calcite/pulls?q=is%3Apr+author%3Arubenada+is%3Aclosed
>
> Here's a good example: https://github.com/apache/calcite/pull/2329
> PR description says the intention of the feature.
> There are overall and line-targeted comments.
> Creating GitHub Issue is not that different from creating a PR.
>
> Ruben>I have the impression that some of the problems described discussion
> are
> Ruben>not per se Jira-related, but they appear because we misuse the
> current
> Ruben>tools (and it would remain more or less the same if we switched from
> Jira
> Ruben>to GitHub Issues).
>
> Ruben, I claim that we split information for no reason.
> What is the real meaning behind having 50% of the review comments in JIRA,
> and 50% of the comments in PR?
> What is the real meaning behind putting "common description in JIRA, and
> putting NO description in the PR"?
> Does it help to have different markup languages for JIRA and GitHub?
>
> My main motivation is not "replace JIRA with Issues". That replacement
> alone has few benefits.
> The key improvement to the current process is creating PRs **with
> description**, and keeping the relevant discussion in the PR itself.
> I really see no point in PR comments like "let's continue in JIRA".
> If a comment targets a specific PR, then why don't we just use PR's
> comment fields for that?
>
> Ruben>high level discussion (e.g. feature design) should
> Ruben>happen in Jira
>
> The vast majority of the changes are small. Adding Jira barrier for every
> change is a productivity killer (for both contributors and for the
> reviewers).
> I don't mean just "switch between JIRA and PR".
> I mean that comments are duplicated at different sites.
> The description of the issue and the code changes are separated.
> The notifications are duplicated (gitbox duplicates GitHub notifications).
> The markup format is different.
> The search is not coherent (there's no way to find issues and PRs at the
> same time).
> GitHub blame does not show the full picture (it shows PRs, however,
> opening the JIRA is a separate step).
> PR (and GitHub Issue) can be associated to a version, so it would be much
> easier to relate the PR with its release version.
> I think the list can go and go.
>
> Ruben>IMO every change must have a dedicated Jira ticket
>
> 

Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-18 Thread Stamatis Zampetakis
There are many good ideas in this thread and some of them we could probably
directly use without changing the current workflows a lot.

Inspired from the discussion so far my main suggestion would be to enable
and start using the GitHub milestones keeping a one-to-one mapping with
Calcite versions.

As other people have mentioned already, currently we allow JIRAless pull
requests and it does make sense in many cases (I will not repeat here what
others have said already). The drawback with the current process is that we
cannot associate a "fix version" if the PR does not have a JIRA and it
might be difficult to track down what changes went on which version. If we
have milestones then we can solve this inconvenience with some other added
benefits.

Note that I am not proposing to replace JIRA. People are free to create
JIRAs and/or pull requests. The difference is that every pull request
before it gets merged should have an assigned milestone. This small
addition in our workflow will make all patches (with or without JIRA)
easier to track.

The main goal of this change is to allow people to use JIRA, GitHub, or
both, as they prefer. This essentially means that we relax the requirement
to have a JIRA case for every patch. At a second time, we can introduce PR
templates to ensure all the necessary information is there when somebody
raises a PR.

If people think it's worth enabling GitHub milestones I can prepare a PR
with changes in the documentation to see concretely how this will
affect our current procedures. On the other hand, if nobody sees any
advantage in using milestones then I'm fine leaving things as they are
right now.

Best,
Stamatis

On Mon, Dec 6, 2021 at 7:33 PM Ruben Q L  wrote:

> Thanks for moving the discussion here, Vladimir.
>
> First of all, I honestly appreciate the fact that you launched this
> discussion, I think you raised some valid points; what you propose sounds
> reasonable, and it seems a valid approach too. My main argument is that I
> have no big issues with the current approach (which, of course, is not
> perfect).
>
> I guess my bottom-line is that I believe that issue tracking and version
> control (+ code review) are two different things, which serve two different
> purposes, and it is arguably a good thing to keep them separated: using
> Jira for the former and Github PRs for the latter is adequate IMO.
>
> > You did use GitHub for tracking already.
> As I see it, I have used Jira for issue tracking, and GitHub PRs for code
> review + merge. If you ask me, perhaps I am old-fashioned, but I think the
> development workflow should focus on the Jira ticket, and not on the GitHub
> PR.
>
> > Here's a good example: https://github.com/apache/calcite/pull/2329
> I agree it is a good example. The associated Jira has several comments with
> a high-level discussion, the PR contains code-review comments for technical
> details. I am happy with this decoupling, although sometimes this is not
> always the case in all tickets; perhaps we should clarify our current
> workflow and try to enforce some rules, instead of changing it into
> something completely new.
> It is also a good example because the Jira reporter and the PR creator were
> not the same person, plus there was a span of almost two years between the
> ticket creation and the PR submission. I know this is usually uncommon, but
> it shows that there are two different roles / actions: one thing is
> describing / analyzing an issue; and a second, different thing is carrying
> out its implementation (with its corresponding code review).
>
> > GitHub allows links, and links have no properties like "caused by",
> "duplicated by". However, basic links are just enough for Calcite.
> I think it would be sad to lose this feature, but I admit it is a minor
> detail.
>
> > If you have a "customized dashboard" that is useful for many
> contributors, please show it.
> I was thinking for example at the dashboard mentioned by Stamatis, which is
> very useful (not only for the release manager, but for all contributors)
> when a release date is coming.
>
> Best regards,
> Ruben
>
>
> On Mon, Dec 6, 2021 at 12:44 PM Vladimir Sitnikov <
> sitnikov.vladi...@gmail.com> wrote:
>
> > Moving the reply to DISCUSS thread.
> >
> > Ruben>My vote might be biased because I have used Jira for many years
> (and
> > I have
> > Ruben>never used Github for issue tracking).
> >
> > It is really sad to hear.
> > You authored 87 PRs for Calcite!
> > You did use GitHub for tracking already.
> >
> >
> https://github.com/apache/calcite/pulls?q=is%3Apr+author%3Arubenada+is%3Aclosed
> >
> > Here's a good example: https://github.com/apache/calcite/pull/2329
> > PR description says the intention of the feature.
> > There are overall and line-targeted comments.
> > Creating GitHub Issue is not that different from creating a PR.
> >
> > Ruben>I have the impression that some of the problems described
> discussion
> > are
> > Ruben>not per se Jira-related, but they appear b

Re: [DISCUSS] JIRA vs GitHub Issues

2021-12-20 Thread Julian Hyde
Stamatis,

You use the word ‘patch’ to describe all code changes, but code changes are 
different in nature, and to be successful need to be accompanied by more than 
code.

A bug fix needs a good description of the bug. Not ‘I changed these lines and 
the problem I was having went away and so I think you should accept this 
patch’. The description needs to be good enough that anyone reading the release 
notes would understand what the problem was. And there needs to be a test case.

A feature needs a description of the feature, why it is beneficial, an 
illustration how to use the feature, a description of the limits of the feature 
(edge cases) and some test cases.

For both bugs and features, all that description needs to come first. For 
those, I just don’t agree that people can use GitHub first. GitHub tends to 
focus on the code, and the important stuff gets forgotten.

There are other kinds of changes - improving documentation, changing the web 
site, adding test cases, fixing the occasional ’no brainer’ bug - but most are 
bug fixes or features, and I believe that “Jira first” has served us well.

As a reviewer, I often won’t even look at a PR until the Jira description is in 
good shape. I would encourage other reviewers to do the same.

Calcite is about ten years old, and is set to go on for another ten or twenty. 
It is in good shape because we have required analysis and documentation of code 
changes, not just accepted every patch that was thrown at us. Jira is a record 
of the design decisions that we have made. We can cross reference new bugs with 
the design decisions that caused them.

In short, Jira is how we do software engineering. We would lose a lot if it 
ceases to be a central part of our process.

I don’t understand what ‘milestones in GitHub’ would entail. I have no 
objections to that per se.

Julian



> On Dec 18, 2021, at 10:44 AM, Stamatis Zampetakis  wrote:
> 
> There are many good ideas in this thread and some of them we could probably
> directly use without changing the current workflows a lot.
> 
> Inspired from the discussion so far my main suggestion would be to enable
> and start using the GitHub milestones keeping a one-to-one mapping with
> Calcite versions.
> 
> As other people have mentioned already, currently we allow JIRAless pull
> requests and it does make sense in many cases (I will not repeat here what
> others have said already). The drawback with the current process is that we
> cannot associate a "fix version" if the PR does not have a JIRA and it
> might be difficult to track down what changes went on which version. If we
> have milestones then we can solve this inconvenience with some other added
> benefits.
> 
> Note that I am not proposing to replace JIRA. People are free to create
> JIRAs and/or pull requests. The difference is that every pull request
> before it gets merged should have an assigned milestone. This small
> addition in our workflow will make all patches (with or without JIRA)
> easier to track.
> 
> The main goal of this change is to allow people to use JIRA, GitHub, or
> both, as they prefer. This essentially means that we relax the requirement
> to have a JIRA case for every patch. At a second time, we can introduce PR
> templates to ensure all the necessary information is there when somebody
> raises a PR.
> 
> If people think it's worth enabling GitHub milestones I can prepare a PR
> with changes in the documentation to see concretely how this will
> affect our current procedures. On the other hand, if nobody sees any
> advantage in using milestones then I'm fine leaving things as they are
> right now.
> 
> Best,
> Stamatis
> 
> On Mon, Dec 6, 2021 at 7:33 PM Ruben Q L  wrote:
> 
>> Thanks for moving the discussion here, Vladimir.
>> 
>> First of all, I honestly appreciate the fact that you launched this
>> discussion, I think you raised some valid points; what you propose sounds
>> reasonable, and it seems a valid approach too. My main argument is that I
>> have no big issues with the current approach (which, of course, is not
>> perfect).
>> 
>> I guess my bottom-line is that I believe that issue tracking and version
>> control (+ code review) are two different things, which serve two different
>> purposes, and it is arguably a good thing to keep them separated: using
>> Jira for the former and Github PRs for the latter is adequate IMO.
>> 
>>> You did use GitHub for tracking already.
>> As I see it, I have used Jira for issue tracking, and GitHub PRs for code
>> review + merge. If you ask me, perhaps I am old-fashioned, but I think the
>> development workflow should focus on the Jira ticket, and not on the GitHub
>> PR.
>> 
>>> Here's a good example: https://github.com/apache/calcite/pull/2329
>> I agree it is a good example. The associated Jira has several comments with
>> a high-level discussion, the PR contains code-review comments for technical
>> details. I am happy with this decoupling, although sometimes this is not
>> alwa

回复:[DISCUSS] JIRA vs GitHub Issues

2021-11-29 Thread Chunwei Lei
+1 keep using ASF JIRA for issue management.

I agree with all the advantages in Jing's apply. I think JIRA issues work well 
and navigating from PR to issue is not a big deal.


Best,
Chunwei


--
发件人:Jing Zhang 
发送时间:2021年11月29日(星期一) 15:52
收件人:dev 
主 题:Re: [DISCUSS] JIRA vs GitHub Issues

Hi Vladimir,
Thanks a lot for driving this discussion.
+ 0.5
1. First, I have same concern with @Zhe, how to deal with the existing
issues.
If there already exists tools to import these issue, I'm OK with that.
2. There are some useful functions in JIRA, I would list my favorite
functions, I'm not sure whether github issue tracking could support these
features.
   1. Mark the fix versions of each JIRA issue. It is visible for users to
find which version contain this functionality or fix this bug.
   2. Watch the interested issue so I could receive email once this issue
has been changed.
   3. Relate the current JIRA issue with it's related JIRA. It is useful
for other developers or users to understand this story.
3. JIRA is intuitive and friendly to non-technical people and new technical
users.
From my experience in Apache Flink community, this point is very useful
because most of bug issues are reported by users instead of developer.

Best,
Jing Zhang

Vladimir Sitnikov  于2021年11月29日周一 下午2:39写道:

> We can copy all the issues, including resolved ones.
>
> We can import issues. GitHub has api for importing without notifications.
> Infra team can execute such a script (we need to find it)
>
> Vladimir
>