ust 26, 2024 9:35:51 AM
To: dev@airflow.apache.org
Subject: [RESULT][VOTE] AIP-80: Explicit Template Fields in Operator Arguments
Hi,
The vote for AIP-80: Explicit Template Fields in Operator Arguments has
concluded successfully. The proposal has been accepted.
Since there’s an update midway t
Hi,
The vote for AIP-80: Explicit Template Fields in Operator Arguments has
concluded successfully. The proposal has been accepted.
Since there’s an update midway through the vote, I’ll calculate votes in two
parts. If you voted in more than once, only the last one counts.
(Sorry I really mean
Thanks TP & everyone for the discussion here: +1 binding
On Mon, 19 Aug 2024 at 13:07, Jarek Potiuk wrote:
> +1 (binding). Thanks for responding to the concerns of compatibility, I
> think personally this is crucial to have good Airflow 3 adoption.
>
> On Mon, Aug 19, 2024 at 1:34 PM Tzu-ping Ch
+1 (binding). Thanks for responding to the concerns of compatibility, I
think personally this is crucial to have good Airflow 3 adoption.
On Mon, Aug 19, 2024 at 1:34 PM Tzu-ping Chung
wrote:
> Hi all,
>
> I have modified the AIP document to reflect the conclusions we had during
> the previous D
Hi all,
I have modified the AIP document to reflect the conclusions we had during the
previous Dev call. Most significantly, the beginning of the Migration section
has been rewritten to declare that Airflow 3 will continue to support the
pre-AIP-80 templating syntax.
Please take another look a
@Emanuel You can send an email to dev-unsubscr...@airflow.apache.org
On Thu, 8 Aug 2024 at 10:59, Emanuel Oliveira wrote:
> How can i remove myself from these emails? i want to follow airflow project
> technically but not interested on ongoing people-project management
> thingies.
> Thanks 🙏😊
>
How can i remove myself from these emails? i want to follow airflow project
technically but not interested on ongoing people-project management
thingies.
Thanks 🙏😊
Best Regards,
Emanuel Oliveira
On Thu 8 Aug 2024, 10:50 Michał Modras,
wrote:
> Yes, there are two options. One - forward compatibil
Yes, there are two options. One - forward compatibility layer, and two -
backwards compatibility layer.
I strongly believe that if we care for Airflow 3 adoption, providing
forward compatibility layers only is not enough, and lack of backwards
compatibility layer in case of changes that bring mostl
The topic here are TWO compatibility layers in this message:
https://lists.apache.org/thread/4s58ho5cw1537sl9ql20n3xslxkjrhyy
The first one is the path described in the AIP, which I consider the main way
most people would migrate.
The second one is what I consider would encourage users to not ch
> I expect the compatibility layer to be delivered when 3.0 is generally
available for testing, and to continue to work during the entire duration
of Airflow 3.x—this should not be a big ask since the 2.x line is not going
to receive new features, and the new syntax should not break compatibility
f
until we drop 2.x support from
> >>> all
> >>>>>> providers
> >>>>>> - Apply compatibility layer to all providers
> >>>>>> - Drop compatibility layer from providers when they drop 2.x support
> >>>>>>
y layer from providers when they drop 2.x support
>>>>>> - Not a hard requirement
>>>>>>
>>>>>> Third-party operator authors
>>>>>> - Apply compatibility layer to all operators
>>>>>> - Drop compatibility layer from operators whe
>> >>> upgrade
>> >>> - Convert DAGs to use the new syntax
>> >>>
>> >>>
>> >>>> On 30 Jul 2024, at 04:55, Scheffler Jens (XC-AS/EAE-ADA-T) <
>> >>> jens.scheff...@de.bosch.com.INVALID> wrote:
>> >&
; >>>
> >>>> On 30 Jul 2024, at 04:55, Scheffler Jens (XC-AS/EAE-ADA-T) <
> >>> jens.scheff...@de.bosch.com.INVALID> wrote:
> >>>>
> >>>> Thanks TP for the rework.
> >>>>
> >>>> I added some comments (iteration 2) on the migr
to get to version 3. I very much favor
>> the
>>> new templating but am not sure how many DAG authors we leave with
>> migration
>>> problems behind.
>>>> Do we have a guess or estimation how much burden we as airflow
>>> developers need to keep as compata
but am not sure how many DAG authors we leave with
> migration
> > problems behind.
> > > Do we have a guess or estimation how much burden we as airflow
> > developers need to keep as compatability compared to the amount of DAG
> > templates that people neet to
evelopers need to keep as compatability compared to the amount of DAG
> templates that people neet to adjust?
> >
> > Sent from Outlook for iOS<https://aka.ms/o0ukef>
> >
> > From: Tzu-ping Chung
> > Sent: Monday, July 29, 202
nt from Outlook for iOS<https://aka.ms/o0ukef>
>
> From: Tzu-ping Chung
> Sent: Monday, July 29, 2024 8:54:58 PM
> To: dev@airflow.apache.org
> Cc: michalmod...@google.com
> Subject: Re: [VOTE] AIP-80: Explicit Template Fields in Ope
kef>
From: Tzu-ping Chung
Sent: Monday, July 29, 2024 8:54:58 PM
To: dev@airflow.apache.org
Cc: michalmod...@google.com
Subject: Re: [VOTE] AIP-80: Explicit Template Fields in Operator Arguments
I have updated the AIP to include the additional compatibility discussions in
this thread. Please
I have updated the AIP to include the additional compatibility discussions in
this thread. Please take a look again.
Specifically (although by no means exclusively) it would be awesome if Michał
you could have a look and see if it addresses more of the concerns and could be
viable for you. Alth
strongly
> than
> >> that, but based on the direction the conversation is going, I like the
> >> issues that have been addressed and adjustments that are being made.
> >>
> >> +1 (binding)
> >>
> >>
> >> - ferruzzi
> >>
> >>
> >&
>>
>> ________
>> From: Jarek Potiuk
>> Sent: Friday, July 26, 2024 9:48 AM
>> To: dev@airflow.apache.org
>> Subject: RE: [EXT] [VOTE] AIP-80: Explicit Template Fields in Operator
>> Arguments
>>
>> CAUTI
k Potiuk
> Sent: Friday, July 26, 2024 9:48 AM
> To: dev@airflow.apache.org
> Subject: RE: [EXT] [VOTE] AIP-80: Explicit Template Fields in Operator
> Arguments
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments
.
+1 (binding)
- ferruzzi
From: Jarek Potiuk
Sent: Friday, July 26, 2024 9:48 AM
To: dev@airflow.apache.org
Subject: RE: [EXT] [VOTE] AIP-80: Explicit Template Fields in Operator Arguments
CAUTION: This email originated from outside of the organization. Do not
Yeah. I think if we have the operator compatibility and a way how we could
just develop providers in "Airflow 3" mode that will keep automatically
compatibility for Airlfow 2 (for a long-ish time) - I'd change my vote from
+0.5 to +1. That would alleviate all my concerns.
On Fri, Jul 26, 2024 at 5
+1 (binding) - it's an important feature IMO, and after reading the AIP and
the comments here - I think that TP's suggestion for compatibility and
migration mitigates the related concerns.
On Thu, Jul 25, 2024 at 10:44 AM Tzu-ping Chung
wrote:
> Hi all,
>
> I’m calling for a vote on AIP-80: Exp
> For the first case, the operator does not actually interact with the
> templates—only Airflow is (logic in BaseOperator)—and the custom operator
> class only cares about what fields are templated, and what the rendered
> values are. I’ve decided therefore to make BaseOperator in Airflow 3 still
>
I considered breakages that are likely to happen for a third-party operator.
Without actually looking into them, from experience, they generally need to do
one of the following two things (or both) in the task:
1. Access the templated values in execute()
2. Access the arguments containing a temp
I understand the concern, but would also wonder that a code base, be it user
DAGs or third-party operators, lacks support in such a way, they would be
correct to not encourage users to upgrade anyway. Airflow 3 is already going to
have various breakages, some of them significantly less obvious t
-1 (non-binding)
While the cleaner approach to templates is appealing, the blast radius of
this change in its current shape is enormous. I am worried that it would
strongly impede migration of users from Airflow 2 to Airflow 3, especially
that not all Airflow users are proficient in Airflow, and f
I am personally perfectly fine with that approach. Looks like a good design
for the approach when we decide that "breaking most DAGs is acceptable"
in general.
J.
My idea toward managing migration is mostly drawn from how Python (eventually)
managed to pull most people over, so let me use that as a parallel to describe
what I want to do.
Initially Python tried using an automated script (2to3) to magically convert
code into Python 3 compatible, which peop
Valid concerns about migrations -- I share the same concern
On Thu, 25 Jul 2024 at 12:17, Jarek Potiuk wrote:
> +0.5 (binding). (See https://www.apache.org/foundation/voting - for
> fractional votes).
>
> A bit more comment here:
>
> See my last comment in the AIP. I am not sure if this is the r
+0.5 (binding). (See https://www.apache.org/foundation/voting - for
fractional votes).
A bit more comment here:
See my last comment in the AIP. I am not sure if this is the right way but
it's conditional +1. I love the idea, and proposal. Mostly because it will
make DAG authoring more "modern" lo
+1 (non-binding)
--
Regards,
Aritra Basu
On Thu, 25 Jul 2024, 1:14 pm Tzu-ping Chung,
wrote:
> Hi all,
>
> I’m calling for a vote on AIP-80: Explicit Template Fields in Operator
> Arguments.
> https://cwiki.apache.org/confluence/x/2grOEg
>
> This proposal aims to improve how Airflow defines temp
Hi all,
I’m calling for a vote on AIP-80: Explicit Template Fields in Operator
Arguments.
https://cwiki.apache.org/confluence/x/2grOEg
This proposal aims to improve how Airflow defines template fields, and help
users avoid annoying pitfalls currently exist.
Discussion thread:
https://lists.apa
36 matches
Mail list logo
