Hi all, I have updated our meeting notes document to summarize the discussion from our dev call for Airflow 2.0 yesterday.
Thank you all who joined the call. *Doc Link*: https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-#7:5Oct2020 To all those who attended, can you please double-check and add if I have missed anything? To all those who didn't join, if you disagree to anything in the Summary please voice your opinion. Also please let me know if someone wants to include an item in Next call's Agenda. Including the Summary here too (might potentially break formatting): *Key Decisions* - *Scheduler HA* - *Progress* - MySQL 8 tests have been added - Adoption of orphaned tasks have been added to Kubernetes and to the new CeleryKubernetesExecutor (PR <https://github.com/apache/airflow/pull/10901>) - All (but one) Tests are passing - Instructions to run the PR with Multiple Schedulers using docker-compose have been added (link to the comment <https://github.com/apache/airflow/pull/10956#issuecomment-702997850> ) - Received some reviews on Monday -- Should be addressed by today/tomorrow - All Big/Critical Concerns have been addressed (escape hatch for locks, MySQL 8 tests, Moving callbacks to DagFileProcessor) - Room to improve code readability which isn't a blocker and would be addressed in follow-up PR - Ash is in talks with Kevin to test his Giant DAG to verify Scheduler Performance to address one of Kevin Yang's concern - There would be a separate PR for retrying transactions to avoid deadlocks as suggested by MySQL guide on handling deadlocks <https://dev.mysql.com/doc/refman/5.7/en/innodb-deadlocks-handling.html> - *API* - Kamil and Omair are hoping to complete work on the Task Instance Endpoint PR <https://github.com/apache/airflow/pull/9597> by the end of this week - James is working to map existing permissions to the UI, hoping to complete by end of the week too. Link to PRs <https://github.com/apache/airflow/pulls/jhtimmins> - *Backport Providers* - Backport packages 2020.10.5 will be released without cncf.kubernetes. We will create a separate release for cncf.kubernetes that will address the issues Kamil found in RCs. Link to VOTE thread <https://lists.apache.org/thread.html/r4dfc18d2376601667efe7dfaf0bf1c07b840f9bcd9f6311d48cb3c64%40%3Cdev.airflow.apache.org%3E> . - *Airflow 1.10.13 Release* - Will be released as per the planned timeline (Airflow 2.0 - Planning <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning>) i.e. Last Week of October. - *Airflow 2.0 Alpha* - Revisit on Thursday based on the progress with Scheduler HA & API whether we would be able to release first alpha this Friday. If not we will update timelines accordingly. - *Other small items*: - Kaxil to create Github Issues for the feedback received on the Documentation Improvement Thread and create threads asking for VOTES for previously discussed items like removing pickles. - Vikram will write up, share and all can review on what we mean by ALPHA and BETAs. Regards, Kaxil
