Hi all, I have modified the AIP document to reflect the conclusions we had during the previous Dev call. Most significantly, the beginning of the Migration section has been rewritten to declare that Airflow 3 will continue to support the pre-AIP-80 templating syntax.
Please take another look and tell me what you think. If nothing comes up, I will formally declare the AIP as accepted after the next Dev call (22 Aug). Fortunately, we do have all the technically boxes ticked, so there’s no additional work needed if you feel the current version is good enough. TP > On 8 Aug 2024, at 17:50, Michał Modras <michalmod...@google.com.INVALID> > wrote: > > Yes, there are two options. One - forward compatibility layer, and two - > backwards compatibility layer. > I strongly believe that if we care for Airflow 3 adoption, providing > forward compatibility layers only is not enough, and lack of backwards > compatibility layer in case of changes that bring mostly syntactic value is > in my opinion against the principles we've discussed in the Airflow 3 dev > calls (e.g. breaking backwards compatibility when there's value brought to > the users, assuring smooth migration, etc.) - here's where our views > differ. I think the discussion should be continued with more stakeholders > in the Airflow 3 dev calls. > > On Thu, Aug 8, 2024 at 11:12 AM Tzu-ping Chung <t...@astronomer.io.invalid> > wrote: > >> The topic here are TWO compatibility layers in this message: >> https://lists.apache.org/thread/4s58ho5cw1537sl9ql20n3xslxkjrhyy >> >> The first one is the path described in the AIP, which I consider the main >> way most people would migrate. >> >> The second one is what I consider would encourage users to not change >> things, and force maintainers to indefinitely maintain. I do not think this >> is worthwhile, and did not include it in the AIP. >> >> The community will provide a compatibility layer. The point of contest >> here is if we should support ANOTHER layer, either instead of or together >> with the one I proposed. >> >> >> >>> On 7 Aug 2024, at 21:11, Jarek Potiuk <ja...@potiuk.com> wrote: >>> >>>> I expect the compatibility layer to be delivered when 3.0 is generally >>> available for testing, and to continue to work during the entire duration >>> of Airflow 3.x—this should not be a big ask since the 2.x line is not >> going >>> to receive new features, and the new syntax should not break >> compatibility >>> for until the theoretical 4.0. >>> >>> I read the above statement as "yes we are adding the "Airflow 2 operators >>> and DAGs working with Airflow 3" compatibility layer as part of the AIP. >>> >>> On Wed, Aug 7, 2024 at 10:32 AM Tzu-ping Chung <t...@astronomer.io.invalid >>> >>> wrote: >>> >>>> I think I’m fine with having this as a provider that if someone wants to >>>> maintain it. Not every provider needs to be maintained by every Airflow >>>> maintainer anyway. I’m not making it a goal for the AIP, but there’s >> also >>>> nothing in there that would prevent it from happening. While I don’t see >>>> myself maintaining the provider, I’ll be happy to tweak things if it >> makes >>>> the implementation easier too. >>>> >>> >>> Now - this one seems to contradict it: "I’m not making it a goal for the >>> AIP" - and "3rd party package" is especially concerning. >>> >>> I understood it otherwise - also after reading the updated AIP - and the >>> "compatibility included" is what gets my +1). >>> Also as far as I can see all the (+1s) above as I read them were also >>> including the compatibility layer to be part of the AIP. And the updated >>> AIP text explains it as well as part of the AIP. >>> >>> If we (as the community that is voting on it) - won't commit to >> supporting >>> compatibility layer, then this is a huge risk for the adoption of >> Airflow 3 >>> - huge risk, for very little benefits if you ask me. >>> >>> If we don't support the compatibility layer as a community and won't >> commit >>> to supporting it, this is really the only change that expects the users >> to >>> make bulk changes to most of their DAGs **before** the migration if they >>> followed the "intentional" and correct way of authoring DAGs (and not >>> misusing them). >>> >>> IMHO - supporting compatibility is a condition of the AIP and goal, >> rather >>> than an option. The compatibility layer there should be tested and >>> supported for us for as long we tell our users we support it. And we >> should >>> be explicit about it. >>> >>> J. >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
