Jarek Nice summary. I can learn many from it. And setting up a robot to resolve this kind of problems seems a very productive way. Look forward to it. SkyWalking community faces very similar scenarios as Airflow. GitHub discussion does help.
Jarek Potiuk <ja...@potiuk.com>于2021年8月29日 周日上午12:47写道： > Some update from the most recent attempt of Apache Airflow to improve our > issue handling using some "new-ish" features available in GitHub: Issue > Forms and GitHub Discussions. > > TL;DR; We don not have yet results but we think in Airflow making the full > use of GitHub Form Templates and endorsing the use of GitHub Discussions > where appropriate (and at the issue entry time), should help us a lot by > "gating" issues better, improving their quality and getting our users into > "pre-clasifying" the issues (or indeed using Discussions where it is a > better choice). > > *Problems we observed:* > > * with the standard MARKDOWN templates we have many issues where people did > not provide useful information (version of airflow, operating system etc.) > * we had a number of issues where users would simply delete the markdown > template content straight away and replaced with their own issue > description - without "reproducible steps", or really to ask a question > about their deployment problems without even trying to attempt to > investigate it. > We've ended up with many "discussion" kind of question posted as issues. > Then we would "convert" such issues into discussion but it required > maintainers comment and explanation. Mostly it was because the users did > not even know they can (and should) open a discussion instead. > * the markdown templates were difficult to read/fill in - it was not clear > what you should do with the parts which were relevant - we left > instructions in the comments, which were sometimes left/sometimes deleted, > generally the issues were "structur-ish" rather than "structured". > * often people opened Airflow 1.10 issues even if it reached End-Of-Life in > June (they should open discussions instead - which is still great because > even if there are no way we will handle the issues but either us or other > users can help them still for workaround or even directing t) > > *Discovery* > > Then we discovered that we already can use "Forms for GitHub Issues" (beta > feature but it is available for everyone now - > > https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms > ). > This looked like not only a great way to structure and pre-classify the > issues (by the users) but also seems to be a great communication tool with > the users. We believe it will allow us to address most of the issues above. > Basically instead of committers having to explain the rules over and over > and instruct the users how they should report the issues - it's all there > explained really clearly and nicely while the user enters the issue. > > *Solution* > > We defined forms with required fields that cannot be "skipped". also when > you do not fill a "textarea" entry it is marked as "Not Provided" rather > than deleted. We added helpful comments and hints as well as explanation in > which cases you should use GitHub Discussions (including lack of ability to > select 1.10 version and link to open GitHub Discussions) instead of issues > (with direct link). We've added logo and welcoming message to "soften" the > "more formalized form entry need. We made it clear that if there is no > reproduction steps, the users should open GitHub Discussion instead. We've > added more issue types - we've separated "Airflow Core", "Airflow > Providers" (we have more than 70 providers that extend Airflow's > capabilities) and "Airflow Docs" issues - automatically applying correct > labels, so we do not have to do triage ourselves. We also created a > "maintainer only" issue type with allows to enter pretty much free-form > information (for tasks/todos etc.) and we've added a required "checkbox" to > confirm you are maintainer, to discourage people from using it to raise > their "free form" questions there. We wanted it to be "easy" for committers > to enter such "free form" issue but "not easy" to skip structured > information by the users - at the same time guiding them to use > "Discussions" which are much "easier" to enter any content and ask > questions. > > What is even more - this structured form will allow us to automate some > stuff if we find it is needed. For example if someone submits an entry > without providing "reproduction steps" we can write a bot to automatically > convert such issue into discussion. Or automatically close an issue if > someone opens a "free-form" one while not being maintainer. > > *Demo/Take a look yourselves* > > You can try it yourself here: > https://github.com/apache/airflow/issues/new/choose > > And see some screenshots how our issue form entries looks like: > > - https://ibb.co/Lxgx7xn > - https://ibb.co/0rYqm1Q > - https://ibb.co/fX45nBB > - https://ibb.co/dbTR9GV > > PR introducing it is here: https://github.com/apache/airflow/pull/17855 > > We really hope it will drive the issue quality up and issue number down > eventually. > > J. > > > On Thu, Jul 15, 2021 at 5:17 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > Good luck :). > > > > On Thu, Jul 15, 2021 at 3:53 PM Erik Ritter <erik...@apache.org> wrote: > > > >> Thank you all for the great advice and thoughts! I see a lot of > alignment > >> between items suggested here and things we've tried or thought about. > >> > >> Being proactive about closing issues against old versions of Superset is > >> definitely something we want to do, but haven't gone through and done a > >> mass issue closing yet. That could certainly help cull the issue count. > >> > >> We have set up some bots on github to close stale issues, but their > >> timeline is probably too long (I think we wait 3 or even 6 months before > >> closing a stale issue). Setting a more aggressive timeline for closing > >> feels like a reasonable approach. > >> > >> A lot of the issues we see seem to be legitimate bugs/issues with the > >> product, but Superset has such a large surface area (supporting dozens > of > >> database query engines) that many committers don't have the time or > ability > >> to replicate someone's environment to truly repro an issue. I'd imagine > >> making it easier for new contributors to set up a dev environment and > test > >> with their own data could help solve this issue. Our new developer > >> onboarding docs/workflow is somewhat lacking, but enabling more > community > >> members to resolve issues could provide the incentives to improve it. > >> > >> Finally, I do think we have an issue with committers/PMC members paying > >> attention to issues filed, probably because Github email spam is all too > >> common, and we pretty much all have jobs that touch more than only > >> Superset. As mentioned, assigning work isn't really possible for an open > >> source Apache project, but I would hope that we could improve the > triaging > >> rate of issues at least. Perhaps gamifying in some way, or creating a > >> volunteer group to do a bi-weekly or monthly triage session could help > us > >> start to get some control over the issues filed. > >> > >> Thanks again for all the great advice, and I'm glad to know it's not > only > >> us having these issues. We've got some plans in the works to improve > this > >> state, and hopefully in a few months I can follow up on this thread > with a > >> success story! > >> > >> Erik > >> > >> On 2021/07/09 12:01:06, Paul Angus <p...@angus.uk.com.INVALID> wrote: > >> > Hi Erik, > >> > > >> > My 2c - you (the project) may have to 'take a view' on some of the > >> issues if they date back to older versions. > >> > > >> > In my experience fixing bugs obviously requires replicating the > >> problem, fixing it and confirming that its fixed, but if the issue was > >> reported against an old version, then someone needs to replicate it in > the > >> old version (to be sure its not PICNIC - Problem In Chair Not In > Computer) > >> then replicate it in the current version to check that its still a bug, > >> and _then_ start fixing it... > >> > > >> > Volunteers are unlikely to want to take on all that effort, so it does > >> mean an ever increasing number of old bugs (as Jarek says, only > >> stakeholders tend to allocate resources to that kind of thing). > >> > > >> > So I would start by seeing if you have a lot of really old bug reports > >> and consider closing them as a matter of course. Huge bug lists also > tend > >> to put people of as they don't know where to start and don't feel that > they > >> can even make a dent in the pile. > >> > > >> > Kind Regards > >> > > >> > > >> > Paul Angus > >> > > >> > -----Original Message----- > >> > From: Jarek Potiuk <ja...@potiuk.com> > >> > Sent: Friday, July 9, 2021 12:51 PM > >> > To: dev <dev@community.apache.org> > >> > Subject: Re: Issue Management in Apache Projects > >> > > >> > We are struggling with it as well in Apache Airflow. > >> > > >> > I can write about some of the things we actively do to try to bring it > >> down (and we can see how it will work after some time). > >> > We have not succeeded yet (we also have ~800 issues opened) but we for > >> example have ~130 opened PR and we used to have > 200 of them so we see > the > >> sign of improvement. > >> > > >> > * we triage and respond to the issues pretty quickly and > "aggressively". > >> > I.e when there is not enough information or the issue is very likely > to > >> be caused by external factor, we close the issue explaining what's > missing, > >> what the author should do, what information should be provided and add > info > >> that it will re-open as soon as more information is provided. I found > >> closing issues in this case works much better for motivation of the > user to > >> add more information (or save the hassle of maintaining status and > closing > >> the issue later). > >> > > >> > * we have automated stale-bot that closes inactive issues and PRs > after > >> (30 day inactivity = notice, + 7 day = closing) > >> > > >> > * when the user raises the issue which is a question, we actively > >> redirect the user to "Discussions" rather than issue and .... close the > >> issue :). We found "GitHub Discussions" pretty useful and active, and > more > >> and more users are opening discussions rather than issues. This keeps > the > >> "issues" > >> > down to some "real" issues. > >> > > >> > * we have a triage team that virtually meets from time to time and > >> actively reviews, classifies the issues (adds labels) but also runs some > >> stats on which areas are "under-staffed". They meet semi-regularly and > >> discuss and send some summaries. > >> > > >> > * we continuously encourage new users to contribute and add more > >> committers especially in the areas that are "under-staffed" (recently UI > >> committers "team" and "Kubernetes" team has greatly increased in > capacity) > >> and it immediately improved the situation there) > >> > > >> > * what helps there is that some of those committers are full-time > >> employed or part-time paid as freelancers by important stakeholders in > the > >> project (Astronomer, Google). Also those stakeholders are fully aware of > >> the value it brings, so they gladly pay the committers for their > community > >> effort, even if it is not directly responding to their needs > (disclaimer - > >> I am one of those freelancers that is part-time paid by the > stakeholders) > >> > > >> > * the rule we have is that we do not need issues at all. People are > >> encouraged (in the docs and workshops) to open directly PRs rather than > >> issues > >> > > >> > * we added "Are you willing to submit PR?" question in the issue > >> template. > >> > When the issue is relatively simple and the user says "yes" we assign > >> the user to it. When the answer is missing - we actively ask the user if > >> there is a will to submit the PR. More often than not, the users are > >> willing to when encouraged (at least initially). > >> > > >> > * we mark the issues that are simple as "good-first-issue" which then > >> lands in http://github.com/apache/airflow/contribute . More often than > >> not we have people commenting "Hey I want to implement this, can you > assign > >> me?" > >> > which we do pretty immediately when they ask. That often works and we > >> have new contributors :). > >> > > >> > * we have a "really quick to start" development environment for > Airflow > >> (Called Breeze) that we continuously improve and try to make easier to > >> start contributing. > >> > > >> > Last but not least. We put a lot of effort into training, guiding and > >> encouraging new contributors to contribute to Airflow: > >> > > >> > * we run semi-regular workshops for new contributors - we **just** > >> started Airflow Summit 2021 yesterday and for example today we have the > >> "first time contributor's workshop" > >> > > >> > https://airflowsummit.org/sessions/2021/workshop-contributing-apache-airflow/ > >> > - 3 hours hands-on when we teach the new contributors how to > contribute. > >> > This is I think 5th or 6th time we do it (we have a few physical > events > >> and over last 1.5 year we had I think 4 online ones). This time we have > 20 > >> people who signed up - from literally all over the world (and BTW. all > >> proceedings from that cheap 50 USD workshop go to Apache Software > >> Foundation as donation). > >> > > >> > * yesterday was a "community" day at the Summit where we had three > >> talks encouraging people to contribute: > >> > > >> > > >> > https://airflowsummit.org/sessions/2021/contributing-journey-becoming-leading-contributor/ > >> > - the road of Kaxil, the PMC of Airflow through committership > >> https://airflowsummit.org/sessions/2021/contributing-first-steps/ - the > >> first steps by a fresh contributor to Airlfow who shared his experiences > >> https://airflowsummit.org/sessions/2021/dont-have-to-wait/ - "You > don't > >> have to wait for someone to fix it for you" - the talk from one of the > >> committers to Airflow, Leah and her co-worker Rachel > >> > > >> > And we have quite few more talks for those who want to start > >> contributing to Airflow: > >> > > >> > https://airflowsummit.org/sessions/2021/guide-airflow-architecture/ > - > >> The newcomer's guide to Airflow Architecture > >> > > >> > And finally, there are things we plan based on some upcoming features > in > >> > GitHub: > >> > > >> > * we are eyeing very closely the new GitHub Issues introduced > recently: > >> > https://github.blog/2021-06-23-introducing-new-github-issues/ . They > >> seem to be much more developer-friendly and automation-friendly and they > >> might help with better organizing/handling the issues. I am working with > >> Github Issues Product Manager (we are going to have a meeting about it > next > >> week) to enable the new GitHub Issues for the whole Apache Software > >> Foundation (I agreed that with Infra) and I hope very soon we will get > it > >> for all ASF projects (as an option to use) > >> > > >> > * we are waiting for Codespaces General Availability and our > >> development environment is prepared to be used there out-of-the-box. > This > >> will make even easier path for new contributors to start contributing > their > >> code straight from the GitHub UI. > https://github.com/features/codespaces. > >> > > >> > Sorry for such a long mail - this is basically a summary of ~ year of > >> discussing and acting in this area. > >> > > >> > I hope some of those might be helpful :) > >> > > >> > J. > >> > > >> > > >> > On Thu, Jul 8, 2021 at 11:33 PM Christopher <ctubb...@apache.org> > >> wrote: > >> > > >> > > Hi Erik, > >> > > > >> > > Do you have a good understanding of *why* there are more issues > being > >> > > opened than being closed? If so, that might hint at some possible > >> > > solutions. > >> > > > >> > > For example, if you just don't have enough people to write code, > then > >> > > the PMC could focus on inviting new committers to try to grow the > >> > > community, or mentoring new developers. > >> > > > >> > > If, on the other hand, the quality of the issues is poor, such that > >> > > they aren't very actionable, you could ask for more information from > >> > > the reporter, and add a label that shows its status, such as > "waiting > >> > > on reporter". If no response is given in a reasonable time, you can > >> > > close old issues. You can also try to address issue quality using > >> > > GitHub issue templates: > >> > > > >> > > > >> https://docs.github.com/en/communities/using-templates-to-encourage-us > >> > > > eful-issues-and-pull-requests/configuring-issue-templates-for-your-rep > >> > > ository > >> > > > >> > > You could also set up something to auto-close very old issues that > >> > > haven't been updated in a long time, under the premise that they are > >> > > probably not relevant anymore. If they are, they can always be > >> > > re-opened. > >> > > > >> > > You can also use GitHub "projects" (which I see you're already > using: > >> > > https://github.com/apache/superset/projects) to help organize > >> related > >> > > tasks, so they can be closed when the overall project is done. > >> > > > >> > > If the problem is that your committers aren't paying attention to > >> open > >> > > issues, you can try to ping your community's dev@ list to remind > >> > > people of how many issues are outstanding, as a way of encouraging > >> > > people to help triage, close, and bring down the number. You could > >> try > >> > > to find other ways to "gamify" the count, too. But, ultimately, it > >> > > comes down to volunteer effort. > >> > > > >> > > If the problem is that your committers are having trouble tracking > >> the > >> > > activity on GitHub, you can double check your mailing list > >> > > configuration to ensure activity gets copied to a notifications@ or > >> > > issues@ list that your committers can track (you can also configure > >> > > them to go to dev@, but that tends to get spammy and redundant, > >> > > especially for your committers who are happy seeing the notification > >> > > dots and/or emails directly from GitHub). > >> > > > >> > > Ultimately, you'll need to figure out why the situation is the way > it > >> > > is, and address it accordingly. You won't be able to force volunteer > >> > > community members to participate to bring the number down, but > >> perhaps > >> > > there's ways to encourage them, depending on why it's happening in > >> the > >> > > first place. > >> > > > >> > > On Thu, Jul 8, 2021 at 5:04 PM Erik Ritter <erik.t.rit...@gmail.com > > > >> > > wrote: > >> > > > > >> > > > Hi all, > >> > > > > >> > > > I'm a PMC member for Apache Superset, and we've recently been > >> > > > struggling with the number of issues reported in our Github repo. > >> > > > We're currently > >> > > at > > >> > > > 800 open issues, and are having trouble keeping up with responding > >> > > > and addressing all the user issues and feedback. We were curious > if > >> > > > any other Apache projects had a way of managing Github issues that > >> works for them. > >> > > We > >> > > > were considering setting up a bot that assigns new issues to a > >> > > > random committer/PMC member, but are open to other ideas too. > >> Thanks > >> > > > for your > >> > > help > >> > > > and advice! > >> > > > > >> > > > Best, > >> > > > Erik Ritter > >> > > > >> > > > --------------------------------------------------------------------- > >> > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > >> > > For additional commands, e-mail: dev-h...@community.apache.org > >> > > > >> > > > >> > > >> > -- > >> > +48 660 796 129 > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > >> For additional commands, e-mail: dev-h...@community.apache.org > >> > >> > > > > -- > > +48 660 796 129 > > > > > -- > +48 660 796 129 > -- Sheng Wu 吴晟 Apache SkyWalking Apache Incubator Apache ShardingSphere, ECharts, DolphinScheduler podlings Zipkin Twitter, wusheng1108
