Thank you for your feedback, Bolke and Andrey! Bolke, I have replied to some of your comments in the doc. I will provide a detailed write up on the "Interactive DAG run" (or synchronous DAG run) capability, which has generated some early questions. I had intended to get an AIP published for that as a follow-up, but I believe that a simpler write up would be useful ahead of the AIP.
Andrey, You raise an interesting point. As part of the Airflow 2.0 release, we as a community had decided to strictly adhere to Semver as detailed in the document you referenced. We also consciously split out the "Core Airflow" releases from the "Provider" releases at that time. We had a clear expectation then for the cadence of both minor and patch releases, which we have generally adhered to since then. Personally, I am more concerned about our Provider releases right now, as compared to the cadence of our major releases. I believe that one of the proposed changes in the Airflow 3 document i.e. the clear separation for Task Execution will help here, but more may be needed. Definitely interested in more feedback on this as well. Vikram On Sat, May 4, 2024 at 10:57 AM Andrey Anshin <andrey.ans...@taragol.is> wrote: > I would like to propose to change (at least discuss) release policy around > the Major version of Airflow. > > Right now it is described as "These releases do not happen with any regular > interval or on any predictable schedule." : > > https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#term-Major-release > > So maybe it is time to make it schedulable, e.g. one per two years or so. > This one could help us to avoid such a discussion in the future, like "We > don't know when Airflow 4 is coming.". At the moment when the new major > version will be released new features wouldn't be added in the old major > version, however we would support bug / security for a while, e.g. 1 year > for bug fixes, 3 years for security fixes with a total 5 year lifecycle per > a major version. These just are approximate time periods for a definition > of current period, bugfix period and security fix period. > > In contributors' perspective it helps with dropping the deprecated stuff > which resolves some old problem: we have to support everything including > deprecated stuff and without schedulable lifecycle for the deprecated stuff > it could be showstopper for the new feature, because sometimes it hard to > support two different approaches for long period of time with no hope that > it will happen soon. For some fundamental stuff which do not require a lot > things time to support we could postponed removal for next after the next > release, e.g. deprecate in Airflow 3, but remove it in Airflow 5 > > In the user perspective, they have at least bug fix support for a while, if > someone want to use legacy version it their choice, however no new > features, no new version of providers (after one year) > > > ---- > Best Wishes > *Andrey Anshin* > > > > On Sat, 4 May 2024 at 19:17, Bolke de Bruin <bdbr...@gmail.com> wrote: > > > I have left several comments :-). And on interactive dag runs even after > > the explanation of Vikram I still don't have a clue what we want to > > accomplish there :-P. > > > > I would like to see a mantra or team for Airflow 3. That helps nudging > > people in the same direction. Suggestions in the comments. > > > > Bolke > > Sent from my iPhone > > > > > On 4 May 2024, at 01:14, Vikram Koka <vik...@astronomer.io.invalid> > > wrote: > > > > > > Good point Jed. > > > I responded back to your comment in the doc as well and very open to > > > changing the term in the doc. > > > > > > Used the term "interactive DAG run" as the ability to invoke or > trigger a > > > DAG run through the API, with the expectation of getting back a result > > > immediately. An alternate term could be a "synchronous DAG run". > > > > > > Regardless, this is a significant change so a good term to indicate the > > > expansion from "batch runs only" is warranted. Very open to different > > terms > > > here. > > > > > >> On Fri, May 3, 2024 at 4:05 PM Jed Cunningham < > jedcunning...@apache.org > > > > > >> wrote: > > >> > > >> Very exciting! Looks like we will have a busy period of time ahead of > > us. > > >> Overall I like the plan so far, especially using this year's Airflow > > Summit > > >> as an opportunity to announce and gather feedback, and the 2025 > version > > to > > >> pitch upgrading. > > >> > > >> I left a comment in the doc, but we might want to iterate on the > > >> terminology we use for high priority or "synchronous" DAG runs to > serve > > LLM > > >> responses - I find "interactive DAG runs" a bit confusing. > > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > >
