elephantmetropolis opened a new issue, #31801:
URL: https://github.com/apache/airflow/issues/31801

   ### Description
   
   Currently, it is not possible to use Jinja templates inside the `params` 
field of a task.
   
   It is not possible to do the following : 
   ```python
   bo = BashOperator(
      task_id="bash_task",
      bash_command = "do-something.sh {{params.path}}",
      params = {"path": "/path/to/thing/{{ ds }}/thing.csv"}
   )
   ```
   
   ### Use case/motivation
   
   We have a case using the `PostgresOperator` : 
   
   ```python
   PostgresOperator(
         task_id="do_postgres_operation",
         sql="my_sql_script.sql",
         params={
               "schema": "my_schema",
               "bucket": "my_bucket"
         }
   )
   ```
   
   Say `my_sql_script.sql` is a script that let you copy data from S3 bucket to 
Postgres. It's pretty common and we would like to reuse it here and there. But 
in our case, we get the `s3_file_name` from the xcom of the previous task, so 
to get that filename we need to use `{{ 
ti.xcom_pull(task_ids='my_previous_task') }}` and we would like to use it in 
`params` that way :
   ```python
         ...
         params={
               "schema": "my_schema",
               "bucket": "my_bucket",
               "s3_file_name": "{{ ti.xcom_pull(task_ids='my_previous_task') }}"
         }
         ...
   ```
   But it's not possible, we thus need to hard-code `{{ 
ti.xcom_pull(task_ids='my_previous_task') }}` inside `my_sql_script.sql` script.
   
   But later in the code, we don't want `s3_file_name` to come from xcom but 
rather from a python variable, and it's impossible because it's hard coded in 
the sql template. We have to create an other sql file with the exact same code 
except for `s3_file_name`.
   
   ### Related issues
   
   This issue is related to the problem : 
https://github.com/apache/airflow/issues/7910
   It has been marked as resolved but the 2 PRs associated are closed
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to