Hey everyone, Thank you for attending the dev call yesterday. I updated our meeting notes document in the Airflow 3.x wiki <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3.x>to capture the notes. The link for those notes is here <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=373886699#Airflow3.xDevCall:Meetingnotes-Summary.9>
The meeting was focussed entirely on the upcoming Airflow 3.1 release, for which we are planning to cut the release candidate next week. I therefore really appreciate all the async updates as well. To everyone who attended the meeting, please check the summary and add anything that I may have missed. For those who could not join, please let us know if you disagree with anything discussed and agreed upon in the meeting. Also, please do ask questions if something is unclear. Our next meeting is scheduled for the 25th of September at the same time. The agenda is quite open at this time. Please let me know if you would like to add anything to the agenda <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=373886699#Airflow3.xDevCall:Meetingnotes-ProposedAgenda.10> . Best regards, Vikram -- Below is the summary from the call: - Development Updates and Presentations: - Airflow 3.1 test plan update (Rahul) - Rahul shared that the overall testing status was positive across most components. - He shared that the Task SDK regression testing had passed without import issues, the UI filters implementation were tested for audit log and XComm filters, and that the "Human in the loop" feature testing was on track. - Rahul also said that AIP-81 testing ran into authentication issues with token/username-password. Bugra shared that the issue is being investigated and that the current suspicion is that it is environment related. - Rahul was concerned about "UI Plugins" requiring retesting after recent changes by Brent and Pierre. - Rahul also asked Dennis to update the testing status for "Deadline Alerts". - Airflow 3.1 beta release update (Kaxil) - Kaxil shared that the Airflow 3.1 beta version released earlier in the week did include the two major PRs highlighted in the last dev call i.e. TP's serialized DAG implementation changes and Kaxil's changes for defaults. - As a result of the above, the server-side code now only uses serialized DAGs and Task Groups are decoupled between server and SDK. - Also, migration testing from 2.11 to 3.1 beta has successfully been tested. - Deadline Alerts (Dennis) - Dennis shared that async support was being added to Notifiers. - He also said that UI aspects were deferred to post 3.1 and that synchronous callbacks were in development, targeted for Airflow 3.2 - Language translations update (Shahar) - Shahar shared that the language translations are at 97% coverage overall and we have been able to add new translations including Catalan and Simplified Chinese. Portuguese and Italian are still in PR, but may still be able to make it before release - And that the the translation freeze has been extended until the 3.1 release to get to 100% coverage - Amazing work here, I am astonished that we are now up to 14 language translations! - Discussion topics: - UI plugins recommendation (Pierre) - Pierre raised a discussion topic around marking React app plugins as experimental. - Pierre said that the edge provider plugin work by Jens revealed needs for optimization and would probably require changes in build tools and dependency sharing. This would probably only affect very complex UI plugins, but it would be safer at this time to mark this part of the UI plugins work as "experimental". - Based on this recommendation by Pierre and Brent, the team agreed with this recommendation. - Airflow CTL update (Bugra) - Bugra said that the first beta was released for "airflow ctl" and that testing was ongoing with limited community interaction to date. - Bugra said that a follow-on release candidate is planned on this, rather than a second beta. - Following on, Bugra created a new AIP to decouple remote commands from the Airflow core CLI, which included a transition approach. - Vikram, Jarek, and Kaxil raised concerns about existing automation dependencies based on the preexisting airflow CLI commands. - Vikram recommended that Bugra delay asking for a vote on this AIP until after the 3.1 release, so that the team has the cognitive capacity to process the changes. - Release preparation and related action items: - Kaxil asked for the AIP (feature) owners to review the Airflow 3.1 draft release notes. Specifically including: - Human in the loop (Wei, et al) - Deadline alerts documentation (Dennis / Rajesh): - Mention of the async vs. sync differences and what's coming in 3.1 vs. later - Migration guide additions to explain the transition from SLA callbacks (in 2.x) to Deadline Alerts in 3.1 - UI React plugins documentation (PIerre / Brent): - Including what is experimental vs. not - Rahul asked Bugra to get back on the "airflow ctl" bugfix and testing - Rahul asked Dennis to update the Deadline Alerts testing - Vikram asked the team to review and cleanup the priority blocker issues (issues with label priority:upgrade_to_airflow3 <https://github.com/apache/airflow/issues?q=state%3Aopen%20label%3A%22priority%3Aupgrade_to_airflow3%22>) and the 3.1 blocking issues - Elad asked Dennis to confirm what was needed for Notifier provider release in conjunction with the 3.1 release -- Vikram Koka Chief Strategy Officer Email: [email protected] <https://www.astronomer.io/>
