josh-fell commented on code in PR #30176: URL: https://github.com/apache/airflow/pull/30176#discussion_r1428855485
########## docs/apache-airflow/howto/operator/bash.rst: ########## @@ -23,126 +23,324 @@ BashOperator ============ Use the :class:`~airflow.operators.bash.BashOperator` to execute -commands in a `Bash <https://www.gnu.org/software/bash/>`__ shell. +commands in a `Bash <https://www.gnu.org/software/bash/>`__ shell. The Bash command or script to execute is +determined by: + +1. The ``bash_command`` argument when using ``BashOperator``, or + +2. If using the TaskFlow decorator, ``@task.bash``, a non-empty string value returned from the decorated callable. + + +.. tip:: + + The ``@task.bash`` decorator is recommended over the classic ``BashOperator`` to execute Bash commands. + + +.. tab-set:: + + .. tab-item:: @task.bash + :sync: taskflow + + .. exampleinclude:: /../../airflow/example_dags/example_bash_decorator.py + :language: python + :dedent: 4 + :start-after: [START howto_decorator_bash] + :end-before: [END howto_decorator_bash] + + .. tab-item:: BashOperator + :sync: operator + + .. exampleinclude:: /../../airflow/example_dags/example_bash_operator.py + :language: python + :dedent: 4 + :start-after: [START howto_operator_bash] + :end-before: [END howto_operator_bash] -.. exampleinclude:: /../../airflow/example_dags/example_bash_operator.py - :language: python - :dedent: 4 - :start-after: [START howto_operator_bash] - :end-before: [END howto_operator_bash] Templating ---------- -You can use :ref:`Jinja templates <concepts:jinja-templating>` to parameterize the -``bash_command`` argument. +You can use :ref:`Jinja templates <concepts:jinja-templating>` to parameterize the Bash command. + +.. tab-set:: + + .. tab-item:: @task.bash + :sync: taskflow + + .. exampleinclude:: /../../airflow/example_dags/example_bash_decorator.py + :language: python + :dedent: 4 + :start-after: [START howto_decorator_bash_template] + :end-before: [END howto_decorator_bash_template] + + .. tab-item:: BashOperator + :sync: operator + + .. exampleinclude:: /../../airflow/example_dags/example_bash_operator.py + :language: python + :dedent: 4 + :start-after: [START howto_operator_bash_template] + :end-before: [END howto_operator_bash_template] + +Using the ``@task.bash`` TaskFlow decorator allows you to return a formatted string and take advantage of +having all :ref:`execution context variables directly accessible to decorated tasks <taskflow/accessing_context_variables>`. -.. exampleinclude:: /../../airflow/example_dags/example_bash_operator.py +.. exampleinclude:: /../../airflow/example_dags/example_bash_decorator.py :language: python :dedent: 4 - :start-after: [START howto_operator_bash_template] - :end-before: [END howto_operator_bash_template] + :start-after: [START howto_decorator_bash_context_vars] + :end-before: [END howto_decorator_bash_context_vars] +You are encouraged to take advantage of this approach as it fits nicely into the overall TaskFlow paradigm. -.. warning:: +.. caution:: - Care should be taken with "user" input or when using Jinja templates in the - ``bash_command``, as this bash operator does not perform any escaping or - sanitization of the command. + Care should be taken with "user" input when using Jinja templates in the Bash command as escaping and + sanitization of the Bash command is not performed. - This applies mostly to using "dag_run" conf, as that can be submitted via - users in the Web UI. Most of the default template variables are not at - risk. + This applies mostly to using ``dag_run.conf``, as that can be submitted via users in the Web UI. Most of + the default template variables are not at risk. -For example, do **not** do this: + For example, do **not** do: -.. code-block:: python + .. tab-set:: - bash_task = BashOperator( - task_id="bash_task", - bash_command='echo "Here is the message: \'{{ dag_run.conf["message"] if dag_run else "" }}\'"', - ) + .. tab-item:: @task.bash + :sync: taskflow -Instead, you should pass this via the ``env`` kwarg and use double-quotes -inside the bash_command, as below: + .. code-block:: python -.. code-block:: python + @task.bash + def bash_task() -> str: + return 'echo "Here is the message: \'{{ dag_run.conf["message"] if dag_run.conf else "" }}\'"' + + + # Or directly accessing `dag_run.conf` + @task.bash + def bash_task(dag_run) -> str: + message = dag_run.conf["message"] if dag_run.conf else "" + return f'echo "here is the message: {message}"' Review Comment: Not _entirely_ sure if this point is valid when accessing the `dag_run` context object directly and not through Jinja. Of course, yes, there should be caution with user/web input generally, but I wasn't sure if this method should be taken with the same level of care. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
