+1 Kevin on the call :). On Wed, Aug 26, 2020 at 12:59 PM Kaxil Naik <kaxiln...@gmail.com> wrote:
> Thanks Kevin, Looking forward to see you on the next call. > > On Wed, Aug 26, 2020, 08:54 Kevin Yang <yrql...@gmail.com> wrote: > > > Thank you Kaxil, this is extremely helpful. We'll try to join at least > the > > next meeting trying to see if we can provide more perspectives on > > SmartSensor and anything else we can help. > > > > > > Cheers, > > Kevin Y > > > > On Tue, Aug 25, 2020 at 4:28 PM Kaxil Naik <kaxiln...@gmail.com> wrote: > > > > > Hi all, > > > > > > I have created a document to summarize the discussion from our second > dev > > > call for Airflow 2.0. > > > > > > Thank you all who joined the call. > > > > > > *Doc Link*: > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-#2:24Aug2020 > <https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-%232:24Aug2020> > > < > https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-%232:24Aug2020 > > > > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-%232:24Aug2020 > > > > > > > > > To all those who attended, can you please double-check and add if I > have > > > missed anything? > > > > > > To all those who didn't join, if you disagree to anything in the > Summary > > > please voice your opinion. > > > > > > Including the Summary here too (might potentially break formatting): > > > > > > *Key Decisions* > > > > > > - *Smart Sensors – *in 2.0 or 2.1 > > > - AIP-17 > > > < > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-17%3A+Consolidate+and+de-duplicate+sensor+tasks+in+airflow+Smart+Sensor > > > > > > > | > > > PR: https://github.com/apache/airflow/pull/5499 > > > - We have not come to a conclusion yet on whether this should be > > > included in 2.0 or not. The majority is towards adding it in 2.0 > > (as > > > it > > > supports Airflow 2.0's Scalability story) and marking it as > > > *experimental*. > > > - There were some questions raised around supporting this new > > > feature. So we decided that *everyone would take a look at the PR > > > itself and we will spend a few minutes in the next meeting to > > decide > > > whether it is 2.0 or not*. > > > - *Simplification of KubernetesExecutor / KubernetesPodOperator* > > > - PR: https://github.com/apache/airflow/pull/10393 > > > - This will be part of *Airflow 2.0* > > > - *Airflow Upgrade Check* (airflow upgrade-check)* command * > > > - WIP PR: PR: https://github.com/apache/airflow/pull/9467 | > Design > > > Doc: > > > > > > > > > https://docs.google.com/document/d/17tB9KZrH871q3AEafqR_i2I7Nrn-OT7le_P49G65VzM/edit#heading=h.vv80w6y621gv > > > - *Scope*: > > > - Users bash script won’t be included but anything in the core > > > Airflow would be covered > > > - > > > > > > *DAG Definitions*: > > > - Changes in Path for contrib to Providers packages > > > - DAG Interfaces: changes in arguments of a DAG / > > BaseOperator > > > - *Configurations*: > > > - Option to auto-replace deprecated configs with new > options > > > - *Run-time Core items*: > > > - Changes like "Connection type can't be null". The > > > upgrade-check should at least shown warning if it can't > > > provide option to > > > detect the type. > > > - *CLI refactor is out-of-scope* > > > - Automatic refactor is *out-of-scope* as it is too > difficult > > > to cover all the cases in the Users bash scripts. > > > - This will be covered by docs or by showing warnings via > the > > > upgrade-check command > > > - *Experimental API to New API refactor is out-of-scope* (will > > be > > > covered by Migration docs) > > > - We agreed that the airflow upgrade-check command *needs to be > > > available in the last release before Airflow 2.0* (1.10.x or > > 1.11.x) > > > - Potential problems with time-consuming DB Migration were also > > > discussed. If we identify such a DB Migration (example the one > > involving > > > TaskInstance table) should be noted separately in Updating.md to > > > provide a > > > warning to the users. > > > - *DEV Calls Feedback* > > > - We agreed on having *Weekly calls from 7 September onwards* > > > - Calls will start with a 5-min reviewing the progress from the > > last > > > call towards 2.0 > > > - *Process* > > > - A *2.0.0-test* branch will be created on 10 Sep 2020 > > > - Changelog: > > > - The current way of Changelog is OK. We don't need further > > > categorization like Webserver, Scheduler etc. > > > - Separate Changelog would be created for Providers Packages > > > - We need to figure a way to tag/label PRs & Issues with > correct > > > categories. Some options that were discussed were: > > > - Adding labels on the PRs & Issues via Bot > > > - A field in PR template for PR authors to add, the bot > would > > > then read the field which would be used to label the PR > > > - Add rules, for example Committers needs to add > appropriate > > > labels to the PR before merging it. We could have a > > > scheduled Github > > > Actions workflow that would fail if it finds PRs without > > > labels. > > > > > > *Things to Discuss Next* > > > > > > - *7 September* > > > - Progress, Current Work & Discussions > > > - API > > > - Providers Packages > > > - Discuss open questions > > > - Improvements to SubDags / Concept of TaskGroup > > > - AIP-34 <https://github.com/apache/airflow/pull/10153> > > > - *14 September* > > > - Process: > > > - When should we defer the in-scope items to post-2.0 > > > - Completion by a date? > > > - Progress by a date? > > > - Progress, Current Work & Discussions > > > - Scheduler HA > > > - Docs Improvements > > > - Helm Chart > > > - Discuss the issue with sources > > > > > > > > > Regards, > > > Kaxil > > > > > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
