Re: [Proposal] Component changes for Apache Airflow

2020-03-10 Thread Aizhamal Nurmamat kyzy
Agree, we need to adopt more careful and thoughtful processes among PMC around creating new components. When the new component is created, it is also worth sharing its description with the rest of the community on the dev list, so more of us are aware of it and we can label issues more accurately.

Re: [Proposal] Component changes for Apache Airflow

2020-03-10 Thread Jarek Potiuk
Agree. I think I created some of them without realising they are created automatically. Being more careful should be enough. We can always do such cleanups periodically. I think being able to add components as we need them is important - as we might have new components added (for example soon we wi

Re: [Proposal] Component changes for Apache Airflow

2020-03-10 Thread Ash Berlin-Taylor
Not really a fan of removing admin as that limits other operations we can do elsewhere -- it's not the end of the world if we get new components created. We should just ask the PMC members to be careful/thoughtful when they create components. And `executor` vs `executor-kubernetes` makes sense

Re: [Proposal] Component changes for Apache Airflow

2020-03-09 Thread Aizhamal Nurmamat kyzy
Seems the new components were created by the folks who had Project Admin permissions: in our case all of the PMC have that permission [1]. One solution is to remove PMC from project admins and establish a process of "requesting creation of new components" in the dev list. I think it is necessary,

Re: [Proposal] Component changes for Apache Airflow

2020-03-03 Thread Jiajie Zhong
I'm not quick familiar with JIRA, but after we remove the ability to create them from the bug-creating screen, Do we could avoid someone create components by accident? Best Wish — Jiajie

Re: [Proposal] Component changes for Apache Airflow

2020-03-03 Thread Aizhamal Nurmamat kyzy
Please, disregard the link in my previous email that was added accidentally :/ On Tue, Mar 3, 2020, 10:27 AM Aizhamal Nurmamat kyzy wrote: > Thank you for spotting it, Jiajie! We should definitely combine them, and > remove duplicates. > > My guess is that the new components were added accident

Re: [Proposal] Component changes for Apache Airflow

2020-03-03 Thread Aizhamal Nurmamat kyzy
Thank you for spotting it, Jiajie! We should definitely combine them, and remove duplicates. My guess is that the new components were added accidentally by committers / pmc members in the bug-creating screen. It seems one can create a component by pressing enter twice on the new component. If po

Re: [Proposal] Component changes for Apache Airflow

2020-03-03 Thread Jiajie Zhong
Hey guys, sad to said that we have some similar component in JIRA now, such as `db` vs `database`, or `test` vs `flaky_test`, or `executor` vs `executor-kubernetes`. Should we combine them, and limit fix the component to avoid regression? Best Wish — Jiajie

[DISCUSS] Prefixing component names in JIRA (was: [Proposal] Component changes for Apache Airflow)

2019-05-30 Thread Aizhamal Nurmamat kyzy
Hello all, One of the motivations from the component refactor is trying to separate issues related to Airflow as a system, vs. issues related to implementations of Airflow’s API for Operators/Hooks/(Sensors?). Daniel correctly pointed at Kubernetes. Airflow has a KubernetesPodOperator, and also a

Re: [Proposal] Component changes for Apache Airflow

2019-05-30 Thread Jarek Potiuk
Aizhamal, sure, but give me some time for that, I am travelling to Bay Area next week and I have pretty busy schedule :( On Thu, May 30, 2019 at 8:38 AM Aizhamal Nurmamat kyzy wrote: > We really need to keep this discussion going around adding prefixes to > components. > > @Jarek, would you min

Re: [Proposal] Component changes for Apache Airflow

2019-05-29 Thread Aizhamal Nurmamat kyzy
We really need to keep this discussion going around adding prefixes to components. @Jarek, would you mind kicking off a new thread? I will close this thread as the points I wanted to discuss and things to do regarding the first component changes are mostly done. Separating threads will help us to

Re: [Proposal] Component changes for Apache Airflow

2019-05-29 Thread Jarek Potiuk
I think there is a discussion (and some work already done) to move more mature operators/hooks from the 'contrib' directory to the main directory for hooks/operators so I think "contrib-" prefix might be too easily confused with the *"*contrib*"* folder until this is finalised. I think we need a b

Re: [Proposal] Component changes for Apache Airflow

2019-05-29 Thread Aizhamal Nurmamat kyzy
Hi Daniel, The reason I had removed `kubernetes` component is that the component attracted two different kinds of issues. I would like to be able to differentiate issues related to kubernetes executor, and kubernetes-related operators and hooks. I have been thinking that prefixes may be helpful in

Re: [Proposal] Component changes for Apache Airflow

2019-05-28 Thread Daniel Imberman
@Aizhamal could we also please add a component for kubernetes-based features or would that fall under other segments? Thank you again for setting this up! On Tue, May 28, 2019 at 8:08 PM Aizhamal Nurmamat kyzy wrote: > Thanks everyone for your support and help! > > I am done with most of the ta

Re: [Proposal] Component changes for Apache Airflow

2019-05-28 Thread Aizhamal Nurmamat kyzy
Thanks everyone for your support and help! I am done with most of the tasks outlined in the document. Over the next few weeks I will keep going through issues that have no components and try to add one to them. Also, I have added descriptions for the components that I know about, but there are st

Re: [Proposal] Component changes for Apache Airflow

2019-05-28 Thread Daniel Imberman
Hi Aizhamal, Thank you for taking this on :). I've been looking to add epics recently but that's just because I didn't know that components were a thing (they make much more sense here). +1! Daniel On Tue, May 28, 2019 at 1:06 PM Aizhamal Nurmamat kyzy wrote: > Yes, that should work. Thanks J

Re: [Proposal] Component changes for Apache Airflow

2019-05-28 Thread Aizhamal Nurmamat kyzy
Yes, that should work. Thanks Jarek :) On Tue, May 28, 2019 at 12:54 PM Jarek Potiuk wrote: > We already have "unreleased versions" there and 2.0.0 is there. We use > 2.0.0 when we mark features as resolved in JIRA when we merge in master - > so that should be perfectly OK to use it, > > On Tue,

Re: [Proposal] Component changes for Apache Airflow

2019-05-28 Thread Jarek Potiuk
We already have "unreleased versions" there and 2.0.0 is there. We use 2.0.0 when we mark features as resolved in JIRA when we merge in master - so that should be perfectly OK to use it, On Tue, May 28, 2019 at 9:01 PM Aizhamal Nurmamat kyzy wrote: > One more thing that we missed in discussions:

Re: [Proposal] Component changes for Apache Airflow

2019-05-28 Thread Aizhamal Nurmamat kyzy
One more thing that we missed in discussions: now that the ‘Affects version’ is a required field, how should we file issues that are currently in master, but are not in a release? is it possible to add a future version to Jira? Or ‘master’ as a version (though it may defeat the purpose)? On Tue, M

Re: [Proposal] Component changes for Apache Airflow

2019-05-28 Thread Jarek Potiuk
All good ! These are required fields now. Thanks for setting this up Aizhamal. On Tue, May 28, 2019 at 8:49 PM Aizhamal Nurmamat kyzy wrote: > Hello everyone, > > `Component` & `Affects version` are required fields when filing Airflow > Jira issues now. Could you please help me test it and confi

Re: [Proposal] Component changes for Apache Airflow

2019-05-28 Thread Aizhamal Nurmamat kyzy
Hello everyone, `Component` & `Affects version` are required fields when filing Airflow Jira issues now. Could you please help me test it and confirm that it is working as desired? Thanks Aizhamal On Wed, May 22, 2019 at 11:01 PM Jarek Potiuk wrote: > I think we indeed need to do some minimal

Re: [Proposal] Component changes for Apache Airflow

2019-05-22 Thread Jarek Potiuk
I think we indeed need to do some minimal and as automated as possible "triage" process in place and having some JIRA automation around it would be great. I think it's something that Ash after he returns from his holidays mentioned that he would like to take care about - especially with the upcomi

Re: [Proposal] Component changes for Apache Airflow

2019-05-21 Thread Aizhamal Nurmamat kyzy
Hi Fokko, Sid gave me the admin rights yesterday, forgot to update here. Thank you. I have edited the components themselves, but to make components the required filed I need to ask Infra. I created this ticket [1], still waiting for it to be resolved. Wouldn't it be smarter that users won't tag

Re: [Proposal] Component changes for Apache Airflow

2019-05-21 Thread airflowuser
Wouldn't it be smarter that users won't tag labels/component themselves? Any new ticket will get "triage" tag and once the community review the request they will set the proper components/priority ? Users can easily tag the wrong components so it's likely that the ordered Jira we have now won't

Re: [Proposal] Component changes for Apache Airflow

2019-05-21 Thread Driesprong, Fokko
Hi Aizhamal, Sorry for the late reply. You're still an administrator on the Jira. Please let me know if you're missing any privileges. Cheers, Fokko Op za 18 mei 2019 om 05:24 schreef Aizhamal Nurmamat kyzy : > Hello everyone, > it took me longer than expected, but I finally got around to makin

Re: [Proposal] Component changes for Apache Airflow

2019-05-17 Thread Aizhamal Nurmamat kyzy
Hello everyone, it took me longer than expected, but I finally got around to making these changes. I've moved all the issues to rightful components (except for those lacking any component - I'll work on those afterwards). I think I am unable to make changes to the components themselves (renames / d

Re: [Proposal] Component changes for Apache Airflow

2019-05-10 Thread Aizhamal Nurmamat kyzy
Thank you for Jira powers, Fokko :) I am aiming to get it done by Monday. Regards, Aizhamal *From: *Driesprong, Fokko *Date: *Fri, May 10, 2019 at 4:05 AM *To: * *Cc: *Siddharth Anand, airflowuser Hi Aizhamal, > > Great suggestions. The Jira needs some love. I've added you to the > administrat

Re: [Proposal] Component changes for Apache Airflow

2019-05-10 Thread Driesprong, Fokko
Hi Aizhamal, Great suggestions. The Jira needs some love. I've added you to the administrator group. Let me know if there are any issues. Cheers, Fokko Op ma 6 mei 2019 om 20:28 schreef Aizhamal Nurmamat kyzy : > Hello all, > > If it's okay by everyone, I would like to start performing the chan

Re: [Proposal] Component changes for Apache Airflow

2019-05-06 Thread Aizhamal Nurmamat kyzy
Hello all, If it's okay by everyone, I would like to start performing the changes outlined in the doc. +Siddharth Anand could you grant me the pertinent privileges to Airflow JIRA? Once you grant me those, I will also have Bulk Change permission[1] to make those changes faster. I will perform th

Re: [Proposal] Component changes for Apache Airflow

2019-05-03 Thread Jarek Potiuk
+1 on mandatory component. +1 on mandatory version. I like the proposed cleanup suggestions. According to this discussion: https://community.atlassian.com/t5/Jira-Core-questions/Project-based-labels-in-Jira/qaq-p/101203 we can disable labels field and create our own per-project custom field with c

Re: [Proposal] Component changes for Apache Airflow

2019-05-02 Thread Aizhamal Nurmamat kyzy
+1 on Ash's points. is it possible to disable labels to begin with? > I see no great benefit in having components with labels. > We can do just fine with only components. > I think we need to keep the labels for searchability and findability: eg. user creates an issue with 'redshift' label within

Re: [Proposal] Component changes for Apache Airflow

2019-05-01 Thread airflowuser
is it possible to disable labels to begin with? I see no great benefit in having components with labels. We can do just fine with only components. I also think that the "version" field should be mandatory. It's important to know against which airflow version the ticket is reported. Sent with Pro

Re: [Proposal] Component changes for Apache Airflow

2019-05-01 Thread Ash Berlin-Taylor
This sounds like a fantastic idea. I would add to the list: - Remove labels that should be components (we have a gcp label and a gcp component for instance) - having duplication in them is confusing - Possibly use a label to indicate when an issue has already been triaged (to avoid duplicated e

[Proposal] Component changes for Apache Airflow

2019-04-30 Thread Aizhamal Nurmamat kyzy
Hello everyone, I would like to propose a few changes for the Apache Airflow JIRA. The reason behind this proposal is that the set of components is disorganized, and it could use some improvements to track the status of the project and improve the Jira triage. I outlined all the proposed changes