Just an update from my side regarding planning: I had some planning meeting for the work for the next week and the only thing I will be able to do (I plan it for tomorrow) is to release backport packages.
* Release a 2nd wave of Backport packages #10014 https://github.com/apache/airflow/issues/10014 I will not have time for this - but I believe Kaxil/Ash are working on it: * Enable MySQL 8 CI jobs #11164 https://github.com/apache/airflow/issues/11164 I also plan to finish next week the review of HA scheduler to the point that I will be comfortable with merging it. I saw the results posted by Kaxiil, they look super promising and I think once we finish the review and the CI works for MySQL 8, I am cool with merging it for people to start testing. The other things are up for grabs to anyone who would like to help - those are not strong prerequisites for 2.0, Alpha release - they are mostly optimizations or splitting the providers which can be released as part of further beta releases, so they are non-blocking. * [OPTIMISATION] Selective builds for different types of tests #10507 https://github.com/apache/airflow/issues/10507 * Fully separate provider packages from the Airflow core (AIP-8) https://github.com/apache/airflow/issues/11163 J. On Tue, Sep 29, 2020 at 7:12 PM Vikram Koka <vik...@astronomer.io> wrote: > Hi all, > > I am sending this out since Kaxil is juggling other obligations. I have > updated our meeting notes document to summarize the discussion from > our dev call for Airflow 2.0 yesterday. > > Thank you all who joined the call. > > *Doc Link*: > https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-#6:28Sep2020 > <https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-%236:28Sep2020> > > To all those who attended, can you please double-check and add if I have > missed anything? > To all those who didn't join, if you disagree to anything in > the Summary please voice your opinion. > > Also please let me know if someone wants to include an item in Next call's > Agenda. > Including the Summary here too (might potentially break formatting): > *Key Decisions* > > - *Scheduler HA* > - *Locking discussion*: Document how Scheduler HA would work with a > HA database configuration, such as an Active / Passive database > configuration with multiple read replicas, but a single writer. > - Add MySQL 8 to the our CI pipeline > https://github.com/apache/airflow/issues/11164 > - *Progress update*: Ash said that the unit tests should be green > by tomorrow. One thing left is the change_state_for_to_without_dagrun - > currently called for every time (expensive), will change to call every > 30 > seconds. PR is feature complete after that is done and will no longer be > "in draft". Will need to move first functionally complete build to next > week instead of current week, to give time for all unit tests to pass > and > for reviews. > - *Benchmarks*: Will be run based on this branch, since the current > benchmarks are based on an initial draft branch before rebase with > master. > > > - *API* > - *Task Instance Endpoint:* Only thing open. Kaxil to speak with > Kamil to get status. > - *Existing permissions map to UI*: WIP PRs. Feedback requested by > James on this PR https://github.com/apache/airflow/pull/11158/ > - *Migrations: *Is there a different migration process needed? Can > be run as a standard alembic migration. > - *Clients:* QP said that the GO client based on the REST API is > already completed and he is using it as part of his airflow-terraform > module. > - *UI Improvements* > - *New HomePage:* Ryan has updated the UX (look) and shared on > slack. Split Actions and Buttons, using Google Material Fonts. Feedback > from meeting was very positive. PR to be final tomorrow. > - *Splitting Providers Package* > - *Separate provider packages from core: *Need some help to get > this wrapped up. https://github.com/apache/airflow/issues/11163 > - *Build optimization*: This would really help speed-up builds by > only running tests for changed providers > https://github.com/apache/airflow/issues/10507 > - *SemVer*: Kaxil to send email to dev list confirming the decision > in the meeting about using SemVer (lazy-consensus) > - *Backport Providers in next 1.10.x release* > - *EmailOperator: *Location has changed (moved to core) > - *CNCF.Kubernetes providers*: Need to be checked if these can be > released. *Insert github issue link created by Kamil for core > operators* > - *Timeline update:* > - *Functionally complete build: *Will be on the week of Oct 5th, > instead of this week as described above in the Scheduler HA notes. > > > The timeline on the main 2.0 planning page has also been updated to > reflect the above. > *Doc Link*: > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning > > > Best regards, > > Vikram > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
