matthewblock commented on issue #44819:
URL: https://github.com/apache/airflow/issues/44819#issuecomment-2536407180
> Task flow operators do not have templateable field at all
Maybe my terminology is off, but if I decorate `PythonOperator`, the
following are still templateable fields, because the user can specify a Jinja
template string and it will get rendered by the time the task executes.
- `op_kwargs`
- `op_args`
- `templates_dict`
The first two technically can't be specified in a decorated task, instead
Airflow is rolling up any "extra" args or kwargs the user specifies, but they
still get rendered.
Here is a test DAG that demonstrates what I'm talking about - See the last
task, which prints the un-rendered function.
```
from datetime import datetime, timedelta
from airflow.decorators import dag, task
default_args = {
"owner": "airflow",
"depends_on_past": False,
"retry_delay": timedelta(minutes=1),
"retries": 0,
}
@dag(
default_args=default_args,
start_date=datetime(2000, 1, 1),
catchup=False,
schedule=None,
)
def test_jinja_functions_dag():
def get_foo(context, jinja_env):
return context["ti"]
@task
def args_kwargs_task(*args, **kwargs):
print(args) # Prints tuple containing current date (ds)
print(kwargs["foo"]) # Prints Task Instance object (ti)
@task(templates_dict={"some_thing": "{{ ds }}"})
def templates_dict_class(**kwargs):
print(kwargs["templates_dict"]["some_thing"]) # Prints current date
(ds)
@task
def direct_kwarg_task(foo):
print(foo) # Prints Task Instance object (ti)
@task
def direct_kwarg_no_rendering_task(foo):
print(foo) # Prints function object get_foo
(
args_kwargs_task("{{ ds }}", foo="{{ ti }}"),
templates_dict_class(),
direct_kwarg_task(foo="{{ ti }}"),
direct_kwarg_no_rendering_task(foo=get_foo),
)
test_jinja_functions_dag()
```
> And in task flow task you can simply call a Python function from inside
the decorated function - there is no need to pass callable to the taskflow
parameter if you can call them directly.
You can also do this in your `python_callable` in a non-decorated
`PythonOperator` task 🤷♂️ but that's not the point. As a user, it doesn't make
sense why I'm encouraged to use TaskFlow API going forward, but I can only use
a new syntax introduced in 2.10.0 using the old style PythonOperator.
When I was developing my DAG, I was surprised that my template function
didn't actually get called; my task didn't fail because my function outputted a
string and instead the name of the function ```<function
test_jinja_functions_dag.<***s>.get_foo at 0xffffacac0a40>``` was getting
passed everywhere.
At the least I think the documentation should be updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
