Completing the sentence: If we think that we need a feature, rather than implementing it in Jinja, we should improve TaskFlow to add it if we cannot do it now.
On Mon, Aug 29, 2022 at 9:18 AM Jarek Potiuk <[email protected]> wrote: > Agree with Ash, > > And I think if we add it, it will detract people from doing the same thing > in a better way. > > I think what I really like about TaskFlow is that it actually does not > need to use Jinja2 at all. > > Jinja2 as I see it, is a super-convoluted way of running (subset of) > Python via strings. Not only dangerous from a security point of view but > also has a number of deficiencies (no type validation really possible, no > good support in IDES, limited set of filters to use and plenty of other > limitations and problems). > > You could not avoid using Jinja2 in "classic" operators which are treated > as black-box. And this is the only reason IMHO we use JINJA2 at all > in Airflow. > > If you look at what features TaskFlow provides - one of its features is to > make our code pure-Python, and Jinja2-less. Simply, if you want to > dynamically derive a value - do it in Python code, not in JINJA2. If we > think that we need a feature, rather than implementing it in Jinja. For me > this is quite simple equation: > > * Classic Operators = Use Jinja2 > * Task Flow = Do not use Jinja2 > > The example of Ash is a pure manifestation of that. > > J. > > > On Mon, Aug 29, 2022 at 9:01 AM Ash Berlin-Taylor <[email protected]> wrote: > >> In this example I'd say it shouldn't be templated, as you can do it via >> >> @task >> def get_templated_echo(ds): >> return f'echo "The date is {ds}"' >> >> (This is nothing particular to XcomArg either, just the way Xcom had >> always worked.) >> >> I don't think we can change the current/default behaviour without it >> likely being a breaking change to someone's workflow. >> >> If there's enough desire and a good enough use case we can add a way to >> make it something you can turn on via one way or another. >> >> -ash >> >> >> >> On 29 August 2022 07:46:01 BST, Tzu-ping Chung <[email protected]> >> wrote: >>> >>> Context: https://github.com/apache/airflow/issues/26016 >>> >>> Currently if a value is passed to a downstream via XComArg, e.g. >>> >>> @taskdef get_templated_echo(): return 'echo "The date is {{ ds }}"' >>> BashOperator(task_id="echo_task", bash_command=get_templated_echo()) >>> >>> >>> Then this value does not go through Jinja2 templating, i.e. the >>> BashOperator above would print The date is {{ ds }} instead of The date >>> is 2022-08-29. The question here is, is this a conscious design >>> decision, and is there reasoning behind it? Personally I can see arguments >>> from both sides, but some context would be very appreciated. >>> >>> And, if there wasn’t an explicit decision, should template-rendering an >>> XComArg be supported? >>> >>> TP >>> >>
