I also think that we are at the point where a document describing them side-by-side is needed. I would very much like to help. I strongly support moving to GitHub Issues.
I'm less concerned about pros/cons (I think the one big pro of "everyone knows it and already has an account" outweighs almost any con) but I want to build a very clear plan of how we will map Jira features to GitHub features. I use quite a lot of Jira's features. In particular, a lot of things seem like they'll become conventions around labels, which I expect to often be low enough data quality that we would just not bother, unless we can control it a bit. I eagerly await the link! Feel free to share very early :-) Kenn On Thu, Jan 13, 2022 at 1:48 PM Aizhamal Nurmamat kyzy <aizha...@apache.org> wrote: > I think I am enthusiastic enough to help with the doc :) will share the > link soon. > > On Thu, Jan 13, 2022 at 10:12 AM Robert Bradshaw <rober...@google.com> > wrote: > >> I don't know if we have consensus, but it seems that some people are >> quite supportive (myself included), and some are ambivalent. The only >> major con I can see is that github doesn't support tagging an issue to >> multiple milestones (but it's unclear how important that is). >> >> I would suggest that someone enthusiastic about this proposal put >> together a doc where we can enumerate the pros and cons and once the >> list seems complete we can bring it back to the list for further >> discussion and/or a vote (if needed, likely not). >> >> On Thu, Jan 13, 2022 at 9:27 AM Alexey Romanenko >> <aromanenko....@gmail.com> wrote: >> > >> > I’m not sure that we have a consensus on this. Since this thread >> initially was started to discuss and gather some feedback then I think it >> would be great to have a summary with pros and cons of this migration. >> > >> > — >> > Alexey >> > >> > On 13 Jan 2022, at 00:11, Aizhamal Nurmamat kyzy <aizha...@apache.org> >> wrote: >> > >> > Hi all, >> > >> > Is there a consensus to migrate to GitHub? >> > >> > On Wed, Dec 15, 2021 at 9:17 AM Brian Hulette <bhule...@google.com> >> wrote: >> >> >> >> >> >> >> >> On Tue, Dec 14, 2021 at 1:14 PM Kenneth Knowles <k...@google.com> >> wrote: >> >>> >> >>> >> >>> >> >>> On Thu, Dec 9, 2021 at 11:50 PM Jean-Baptiste Onofre <j...@nanthrax.net> >> wrote: >> >>>> >> >>>> Hi, >> >>>> >> >>>> No problem for me. The only thing I don’t like with GitHub issues is >> that fact that it’s not possible to “assign” several milestones to an issue. >> >>>> When we maintain several active branch/version, it sucks (one issue >> == one milestone), as we have to create several issue. >> >>> >> >>> >> >>> This is a good point to consider. In Beam we often create multiple >> issues anyhow when we intend to backport/cherrypick a fix. One issue for >> the original fix and one each targeted cherrypick. This way their >> resolution status can be tracked separately. But it is nice for users to be >> able to go back and edit the original bug report to say which versions are >> affected and which are not. >> >> >> >> >> >> I looked into this a little bit. It looks like milestones don't have >> to represent a release (e.g. they could represent some abstract goal), but >> they are often associated with releases. This seems like a reasonable field >> to map to "Fix Version/s" in jira, but jira does support specifying >> multiple releases. So one issue == one milestone would be a regression. >> >> As Kenn pointed out though we often create a separate jira to track >> backports anyway (even though we could just specify multiple fix versions), >> so I'm not sure this is a significant blocker. >> >> >> >> If we want to use milestones to track abstract goals, I think we'd be >> out of luck. We could just use labels, but the GitHub UI doesn't present a >> nice burndown chart for those. See >> https://github.com/pandas-dev/pandas/milestones vs. >> https://github.com/pandas-dev/pandas/labels. FWIW jira doesn't have >> great functionality here either. >> >> >> >>> >> >>> >> >>> Kenn >> >>> >> >>>> >> >>>> >> >>>> Regards >> >>>> JB >> >>>> >> >>>> > Le 10 déc. 2021 à 01:28, Kyle Weaver <kcwea...@google.com> a >> écrit : >> >>>> > >> >>>> > I’m in favor of switching to Github issues. I can’t think of a >> single thing jira does better. >> >>>> > >> >>>> > Thanks Jarek, this is a really great resource [1]. For another >> reference, the Calcite project is engaged in the same discussion right now >> [2]. I came up with many of the same points independently before I saw >> their thread. >> >>>> > >> >>>> > When evaluating feature parity, we should make a distinction >> between non-structured (text) and structured data. And we don’t need a >> strict mechanical mapping for everything unless we’re planning on >> automatically migrating all existing issues. I don’t see the point in >> automatic migration, though; as Jarek pointed out, we’d end up perpetuating >> a ton of obsolete issues. >> >>>> > >> >>>> > • We use nested issues and issue relations in jira, but as >> far as I know robots don’t use them and we don’t query them much, so we’re >> not losing anything by moving from an API to plain English descriptions: >> “This issue is blocked by issue #n.” Mentions show up automatically on >> other issues. >> >>>> > • For component, type, priority, etc., we can use Github >> labels. >> >>>> > • Version(s) affected is used inconsistently, and as far as >> I know only by humans, so a simple English description is fine. We can >> follow the example of other projects and make the version affected a part >> of the issue template. >> >>>> > • For fix version, which we use to track which issues we >> want to fix in upcoming releases, as well as automatically generate release >> notes: Github has “milestones,” which can be marked on PRs or issues, or >> both. >> >>>> > • IMO the automatically generated JIRA release notes >> are not especially useful anyway. They are too detailed for a quick >> summary, and not precise enough to show everything. For a readable summary, >> we use CHANGES.md to highlight changes we especially want users to know >> about. For a complete list of changes, there’s the git commit log, which is >> the ultimate source of truth. >> >>>> > • We’d only want to preserve reporter and assignee if we’re >> planning on migrating everything automatically, and even then I think it’d >> be fine to compile a map of active contributors and drop the rest. >> >>>> > >> >>>> > As for the advantages of switching (just the ones off the top of >> my head): >> >>>> > • As others have mentioned, it’s less burden for new >> contributors to create new issues and comment on existing ones. >> >>>> > • Effortless linking between issues and PRs. >> >>>> > • Github -> jira links were working for a short >> while, but they seem to be broken at the moment. >> >>>> > • Jira -> github links only show: “links to GitHub >> Pull Request #xxxxx”. They don’t say the status of the PR, so you have to >> follow the link to find out. Especially inconvenient when one jira maps to >> several PRs, and you have to open all the links to get a summary of what >> work was done. >> >>>> > • When you mention a GH issue in a pull request, a >> link to the PR will automatically appear on the issue, including not just >> the ID but also the PR’s description and status >> (open/closed/draft/merged/etc.), and if you hover it will show a preview as >> well. >> >>>> > • We frequently merge a PR and then forget to mark >> the jira as closed. Whereas if a PR is linked to a GH issue using the >> “closes” keyword, the GH issue will automatically be closed [3]. >> >>>> > • I don’t have to look up or guess whether a github account >> and jira account belong to the same person. >> >>>> > • There’s a single unified search bar to find issues, PRs, >> and code. >> >>>> > • Github enables markdown formatting everywhere, which is >> more or less the industry standard, whereas Jira has its own bespoke system >> [4]. >> >>>> > • In GH issues, links to Github code snippets will >> automatically display the code snippet inline. >> >>>> > • GH labels are scoped to each project, whereas ASF Jira >> labels are an unmanageable, infinitely growing namespace (see “flake,” >> “flaky,” “flakey,” “Flaky,” “flaky-test”...). >> >>>> > >> >>>> > [1] >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191332632 >> >>>> > [2] >> https://mail-archives.apache.org/mod_mbox/calcite-dev/202112.mbox/%3CCAB%3DJe-EuaijDjwb6umU_N2TaqFZawE%2BUbgZAgZYvrgPFypfAYQ%40mail.gmail.com%3E >> >>>> > [3] >> https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword >> >>>> > [4] >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191332632 >> >>>> > >> https://jira.atlassian.com/secure/WikiRendererHelpAction.jspa?section=all >> >>>> > >> >>>> > >> >>>> > On Wed, Dec 8, 2021 at 9:13 AM Alexey Romanenko < >> aromanenko....@gmail.com> wrote: >> >>>> > Many thanks for details, Jarek! >> >>>> > >> >>>> > Actually, your experience proves that the full data transfer is >> very expensive (if even possible) and not necessary, especially taking the >> fact that the number of Beam Jira issues is a couple of orders more than >> Airflow one. So, very likely that we will end up by living with two issue >> trackers, at least for some time, to avoid issue duplications and have an >> access to old ones. This can be very confusing. >> >>>> > >> >>>> > In the same time, except the argument of “one tool for >> everything”, which is quite strong for sure, I don’t see any other >> advantages of GH issues over Jira issues. Also, the more important is not >> to lose what we have for now, as Jan mentioned below. >> >>>> > >> >>>> > So, my vote for now is -0 since it has significant pros and cons >> and the final impact is not evident. >> >>>> > >> >>>> > — >> >>>> > Alexey >> >>>> > >> >>>> > > On 8 Dec 2021, at 01:38, Jarek Potiuk <ja...@potiuk.com> wrote: >> >>>> > > >> >>>> > >> Do I understand correctly that this transition (if it will >> happen) includes the transfer of all Beam Jira archive to GitHub issues >> with a proper statuses/comments/refs/etc? If not, what are the options? >> >>>> > > >> >>>> > > Suggestion from the experience of Airflow again - you can look >> it up >> >>>> > > in our notes. >> >>>> > > >> >>>> > > We've tried it initially to copy the issues manually or in bulk, >> but >> >>>> > > eventually we decided to tap into the wisdom and cooperation of >> our >> >>>> > > community. >> >>>> > > >> >>>> > > We migrated some (not many) important things only and asked our >> users >> >>>> > > to move the important issues if they think they are still >> >>>> > > relevant/important to them. We closed the JIRA for entry and >> left the >> >>>> > > issues in JIRA in read-only state so that we could always refer >> to >> >>>> > > them if needed. >> >>>> > > >> >>>> > > So rather than proactively copy the issues, we asked the users >> to make >> >>>> > > the decision which issues are important to them and proactively >> move >> >>>> > > it and we left an option of reactive moving if someone came back >> to >> >>>> > > the issue later. >> >>>> > > >> >>>> > > That turned out to be a smart decision considering the effort it >> would >> >>>> > > require to smartly move the issues vs. the results achieved. And >> >>>> > > helped us to clean some "stale/useless/not important" issues. >> >>>> > > >> >>>> > > We've had 1719 open JIRA issues when we migrated. Over the >> course of >> >>>> > > ~1.5 years (since about April 2020) we've had ~140 issues that >> refer >> >>>> > > to any of the JIRA issues >> >>>> > > >> https://github.com/apache/airflow/issues?q=is%3Aissue+is%3Aclosed+%22https%3A%2F%2Fissues.apache.org%2Fjira%22+ >> . >> >>>> > > Currently we have > 4500 GH issues (3700 closed, 800 opened). >> >>>> > > >> >>>> > > This means that roughly speaking only < 10% of original open JIRA >> >>>> > > issues were actually somewhat valuable (roughly speaking of >> course) >> >>>> > > and they were < 5% of today's numbers. Of course some of the new >> GH >> >>>> > > issues duplicated those JIRA ones. But not many I think, >> especially >> >>>> > > that those issues in JIRA referred mostly to older Airflow >> versions. >> >>>> > > >> >>>> > > One more comment for the migration - I STRONGLY recommend using >> well >> >>>> > > designed templates for GH issues from day one. That significantly >> >>>> > > improves the quality of issues - and using Discussions as the >> place >> >>>> > > where you move unclear/not reproducible issues (and for example >> >>>> > > guiding users to use discussions if they have no clearly >> reproducible >> >>>> > > case). This significantly reduces the "bad issue overload" (see >> also >> >>>> > > more detailed comments in >> >>>> > > >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191332632 >> ). >> >>>> > > >> >>>> > > I personally think a well designed issue entry process for new >> issues >> >>>> > > is more important than migrating old issues in bulk. Especially >> if you >> >>>> > > will ask users to help - as they will have to make a structured >> entry >> >>>> > > with potentially more detailed information/reproducibility) or >> they >> >>>> > > will decide themselves that opening a github discussion is >> better than >> >>>> > > opening an issue if they do not have a reproducible case. Or >> they will >> >>>> > > give up if too much information is needed (but this means that >> their >> >>>> > > issue is essentially not that important IMHO). >> >>>> > > >> >>>> > > But this is just friendly advice from the experience of those >> who did >> >>>> > > it quite some time ago :) >> >>>> > > >> >>>> > > J. >> >>>> > > >> >>>> > > On Wed, Dec 8, 2021 at 1:08 AM Brian Hulette < >> bhule...@google.com> wrote: >> >>>> > >> >> >>>> > >> At this point I just wanted to see if the community is >> interested in such a change or if there are any hard blockers. If we do go >> down this path I think we should port jiras over to GH Issues. You're right >> this isn't trivial, there's no ready-made solution we can use, we'd need to >> decide on a mapping for everything and write a tool to do the migration. It >> sounds like there may be other work in this area we can build on (e.g. >> Airflow may have made a tool we can work from?). >> >>>> > >> >> >>>> > >> I honestly don't have much experience with GH Issues so I can't >> provide concrete examples of better usability (maybe Jarek can?). From my >> perspective: >> >>>> > >> - I hear a lot of grumbling about jira, and a lot of praise for >> GitHub Issues. >> >>>> > >> - Most new users/contributors already have a GitHub account, >> and very few already have an ASF account. It sounds silly, but I'm sure >> this is a barrier for engaging with the community. Filing an issue, or >> commenting on one to provide additional context, or asking a clarifying >> question about a starter task should be very quick and easy - I bet a lot >> of these interactions are blocked at the jira registration page. >> >>>> > >> >> >>>> > >> Brian >> >>>> > >> >> >>>> > >> On Tue, Dec 7, 2021 at 9:04 AM Alexey Romanenko < >> aromanenko....@gmail.com> wrote: >> >>>> > >>> >> >>>> > >>> Do I understand correctly that this transition (if it will >> happen) includes the transfer of all Beam Jira archive to GitHub issues >> with a proper statuses/comments/refs/etc? If not, what are the options? >> >>>> > >>> >> >>>> > >>> Since this transfer looks quite complicated at the first >> glance, what are the real key advantages (some concrete examples are very >> appreciated) to initiate this process and what are the show-stoppers for us >> with a current Jira workflow? >> >>>> > >>> >> >>>> > >>> — >> >>>> > >>> Alexey >> >>>> > >>> >> >>>> > >>> On 6 Dec 2021, at 19:48, Udi Meiri <eh...@google.com> wrote: >> >>>> > >>> >> >>>> > >>> +1 on migrating to GH issues. >> >>>> > >>> We will need to update our release process. Hopefully it'll >> make it simpler. >> >>>> > >>> >> >>>> > >>> >> >>>> > >>> On Sat, Dec 4, 2021 at 2:35 AM Jarek Potiuk <ja...@potiuk.com> >> wrote: >> >>>> > >>>> >> >>>> > >>>> Just to add a comment on those requirements Kenneth, looking >> into the >> >>>> > >>>> near future. >> >>>> > >>>> >> >>>> > >>>> Soon GitHub issues will open for GA a whole new way of >> interacting >> >>>> > >>>> with the issues (without removing the current way) which will >> greatly >> >>>> > >>>> improve iI think all aspects of what You mentioned). The >> issues (and >> >>>> > >>>> associated projects) will gain new capabilities: >> >>>> > >>>> >> >>>> > >>>> * structured metadata that you will be able to define (much >> better >> >>>> > >>>> than unstructured labels) >> >>>> > >>>> * table-like visualisations which will allow for fast, bulk, >> >>>> > >>>> keyboard-driven management >> >>>> > >>>> * better automation of workflows >> >>>> > >>>> * complete APIs to manage the issues (good for GitHub Actions >> >>>> > >>>> integration for example) >> >>>> > >>>> >> >>>> > >>>> Re: assigning by non-committers is one of the things that >> won't work >> >>>> > >>>> currently. Only comitters can assign the issues, and only if >> a user >> >>>> > >>>> commented on the issue. But it nicely works - when a user >> comments "I >> >>>> > >>>> want to work on that issue", a committer assigns the user. >> And It >> >>>> > >>>> could be easily automated as well. >> >>>> > >>>> >> >>>> > >>>> You can see what it will is about here: >> https://github.com/features/issues >> >>>> > >>>> >> >>>> > >>>> They are currently at the "Public Beta" and heading towards >> General >> >>>> > >>>> Availability, but it is not available to "open" projects yet. >> However >> >>>> > >>>> I have a promise from the GitHub Product manager (my friend >> heads the >> >>>> > >>>> team implementing it) that ASF will be the first on the list >> when the >> >>>> > >>>> public projects will be enabled, because it looks like it >> will make >> >>>> > >>>> our triaging and organisation much better. >> >>>> > >>>> >> >>>> > >>>> J. >> >>>> > >>>> >> >>>> > >>>> On Sat, Dec 4, 2021 at 1:46 AM Kenneth Knowles < >> k...@apache.org> wrote: >> >>>> > >>>>> >> >>>> > >>>>> This sounds really good to me. Much more familiar to >> newcomers. I think we end up doing a lot more ad hoc stuff with labels, >> yes? Probably worth having a specific plan. Things I care about: >> >>>> > >>>>> >> >>>> > >>>>> - priorities with documented meaning >> >>>> > >>>>> - targeting issues to future releases >> >>>> > >>>>> - basic visualizations (mainly total vs open issues over >> time) >> >>>> > >>>>> - tags / components >> >>>> > >>>>> - editing/assigning by non-committers >> >>>> > >>>>> - workflow supporting "needs triage" (default) -> open -> >> resolved >> >>>> > >>>>> >> >>>> > >>>>> I think a lot of the above is done via ad hoc labels but I'm >> not sure if there are other fancy ways to do it. >> >>>> > >>>>> >> >>>> > >>>>> Anyhow we should switch even if there is a feature gap for >> the sake of community. >> >>>> > >>>>> >> >>>> > >>>>> Kenn >> >>>> > >>>>> >> >>>> > >>>>> On Fri, Dec 3, 2021 at 3:06 PM David Huntsperger < >> dhuntsper...@google.com> wrote: >> >>>> > >>>>>> >> >>>> > >>>>>> Yes, please. I can help clean up the website issues as part >> of a migration. >> >>>> > >>>>>> >> >>>> > >>>>>> On Fri, Dec 3, 2021 at 1:46 PM Robert Burke < >> rob...@frantil.com> wrote: >> >>>> > >>>>>>> >> >>>> > >>>>>>> Similar thing happened for Go migrating to use GH issues >> for everything from Language Feature proposals to bugs. Much easier than >> the very gerrit driven process it was before, and User Discussions are far >> more discoverable by users: they usually already have a GH account, and >> don't need to create a new separate one. >> >>>> > >>>>>>> >> >>>> > >>>>>>> GitHub does seem to permit user directed templates for >> issues so we can simplify issue triage by users: Eg for Go there are a >> number of requests one can make: >> https://github.com/golang/go/issues/new/choose >> >>>> > >>>>>>> >> >>>> > >>>>>>> On Fri, Dec 3, 2021, 12:17 PM Andy Ye <yea...@google.com> >> wrote: >> >>>> > >>>>>>>> >> >>>> > >>>>>>>> Chiming in from the perspective of a new Beam >> contributor. +1 on Github issues. I feel like it would be easier to learn >> about and contribute to existing issues/bugs if it were tracked in the same >> place as that of the source code, rather than bouncing back and forth >> between the two different sites. >> >>>> > >>>>>>>> >> >>>> > >>>>>>>> On Fri, Dec 3, 2021 at 1:18 PM Jarek Potiuk < >> ja...@potiuk.com> wrote: >> >>>> > >>>>>>>>> >> >>>> > >>>>>>>>> Comment from a friendly outsider. >> >>>> > >>>>>>>>> >> >>>> > >>>>>>>>> TL; DR; Yes. Do migrate. Highly recommended. >> >>>> > >>>>>>>>> >> >>>> > >>>>>>>>> There were already similar discussions happening >> recently (community >> >>>> > >>>>>>>>> and infra mailing lists) and as a result I captured >> Airflow's >> >>>> > >>>>>>>>> experiences and recommendations in the BUILD wiki. You >> might find some >> >>>> > >>>>>>>>> hints and suggestions to follow as well as our >> experiences at Airflow: >> >>>> > >>>>>>>>> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191332632 >> >>>> > >>>>>>>>> >> >>>> > >>>>>>>>> J, >> >>>> > >>>>>>>>> >> >>>> > >>>>>>>>> >> >>>> > >>>>>>>>> On Fri, Dec 3, 2021 at 7:46 PM Brian Hulette < >> bhule...@google.com> wrote: >> >>>> > >>>>>>>>>> >> >>>> > >>>>>>>>>> Hi all, >> >>>> > >>>>>>>>>> I wanted to start a discussion to gauge interest on >> moving our issue tracking from the ASF Jira to GitHub Issues. >> >>>> > >>>>>>>>>> >> >>>> > >>>>>>>>>> Pros: >> >>>> > >>>>>>>>>> + GH Issues is more discoverable and approachable for >> new users and contributors. >> >>>> > >>>>>>>>>> + For contributors at Google: we have tooling to >> integrate GH Issues with internal issue tracking, which would help us be >> more accountable (Full disclosure: this is the reason I started thinking >> about this). >> >>>> > >>>>>>>>>> >> >>>> > >>>>>>>>>> Cons: >> >>>> > >>>>>>>>>> - GH Issues can't be linked to jiras for other ASF >> projects (I don't think we do this often in jira anyway). >> >>>> > >>>>>>>>>> - We would likely need to do a one-time migration of >> jiras to GH Issues, and update any processes or automation built on jira >> (e.g. release notes). >> >>>> > >>>>>>>>>> - Anything else? >> >>>> > >>>>>>>>>> >> >>>> > >>>>>>>>>> I've always thought that using ASF Jira was a hard >> requirement for Apache projects, but that is not the case. Other Apache >> projects are using GitHub Issues today, for example the Arrow DataFusion >> sub-project uses GitHub issues now [1,2] and Airflow migrated from jira [3] >> to GitHub issues [4]. >> >>>> > >>>>>>>>>> >> >>>> > >>>>>>>>>> [1] >> https://lists.apache.org/thread/w3dr1vlt9115r3x9m7bprmo4zpnog483 >> >>>> > >>>>>>>>>> [2] https://github.com/apache/arrow-datafusion/issues >> >>>> > >>>>>>>>>> [3] >> https://issues.apache.org/jira/projects/AIRFLOW/issues >> >>>> > >>>>>>>>>> [4] https://github.com/apache/airflow/issues >> >>>> > >>> >> >>>> > >>> >> >>>> > >> >>>> >> > >> >
