Re: Operator Templating in Airflow 3

2024-07-22 Thread Guangyang Li
> > From: Jarek Potiuk > > Sent: Tuesday, July 16, 2024 12:52:05 PM > > To: dev@airflow.apache.org > > Subject: Re: Operator Templating in Airflow 3 > > > > I am all for the change - it's been a source of confusion. > > > > And yes,

Re: Operator Templating in Airflow 3

2024-07-18 Thread Vincent Beck
___ > From: Jarek Potiuk > Sent: Tuesday, July 16, 2024 12:52:05 PM > To: dev@airflow.apache.org > Subject: Re: Operator Templating in Airflow 3 > > I am all for the change - it's been a source of confusion. > > And yes, automatically rewriting the DAGs a

Re: Operator Templating in Airflow 3

2024-07-18 Thread Scheffler Jens (XC-AS/EAE-ADA-T)
tps://aka.ms/o0ukef> From: Jarek Potiuk Sent: Tuesday, July 16, 2024 12:52:05 PM To: dev@airflow.apache.org Subject: Re: Operator Templating in Airflow 3 I am all for the change - it's been a source of confusion. And yes, automatically rewriting the DAGs and possibl

Re: Operator Templating in Airflow 3

2024-07-16 Thread Jarek Potiuk
I am all for the change - it's been a source of confusion. And yes, automatically rewriting the DAGs and possibly even detection with error explaining what to do (on Airflow 3) if template string is passed to "old" template_field - are really a prerequisite for that (and should be quite possible

Re: Operator Templating in Airflow 3

2024-07-16 Thread Ephraim Anierobi
Good proposal. Big +1. I once had trouble with the trailing space issue and spent hours debugging it. Provided the migration tool can rewrite the DAGs, I support it. On Tue, 16 Jul 2024 at 08:42, Tzu-ping Chung wrote: > Good idea. Personally I think changing the import might be easier to read,

Re: Operator Templating in Airflow 3

2024-07-16 Thread Tzu-ping Chung
Good idea. Personally I think changing the import might be easier to read, but having a DAG policy is better if the goal is to rewrite DAG files as little as possible. Not sure if that’s a goal I’d agree with, but that’s not my decision to make  I’ll find a way to add it to the document. >

Re: Operator Templating in Airflow 3

2024-07-16 Thread Tzu-ping Chung
> On Jul 16, 2024, at 14:38, Bas Harenslak wrote: > > +1 for this change! I especially had lots of trouble with template files with > templated filenames and this would solve it. > > Few comments/questions: > "improved debuggability when things go wrong": Could you elaborate on how > this

Re: Operator Templating in Airflow 3

2024-07-16 Thread Abhishek Bhakat
Seems nice. For the 2nd proposed change, I'm thinking another method for compat may be a dag_policy that drop-in replaces the params of any Operator. Avi

Re: Operator Templating in Airflow 3

2024-07-16 Thread Bas Harenslak
+1 for this change! I especially had lots of trouble with template files with templated filenames and this would solve it. Few comments/questions: "improved debuggability when things go wrong": Could you elaborate on how this proposal improves debuggability? "better discoverability of whether a