sunank200 commented on code in PR #37101: URL: https://github.com/apache/airflow/pull/37101#discussion_r1500222490
########## docs/apache-airflow/authoring-and-scheduling/datasets.rst: ########## @@ -237,6 +237,85 @@ Example: Note that this example is using `(.values() | first | first) <https://jinja.palletsprojects.com/en/3.1.x/templates/#jinja-filters.first>`_ to fetch the first of one Dataset given to the DAG, and the first of one DatasetEvent for that Dataset. An implementation may be quite complex if you have multiple Datasets, potentially with multiple DatasetEvents. +Advanced Dataset Scheduling with Conditional Expressions +-------------------------------------------------------- + +Apache Airflow introduces advanced scheduling capabilities that leverage conditional expressions with datasets. This feature allows Airflow users to define complex dependencies for DAG executions based on dataset updates, using logical operators for more granular control over workflow triggers. + +Logical Operators for Datasets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Airflow supports two logical operators for combining dataset conditions: + +- **AND (``&``)**: Specifies that the DAG should be triggered only after all of the specified datasets have been updated. +- **OR (``|``)**: Specifies that the DAG should be triggered when any one of the specified datasets is updated. + +These operators enable the expression of complex dataset update conditions, enhancing the dynamism and flexibility of Airflow workflows. + +Example Usage +------------- + +**Scheduling Based on Multiple Dataset Updates** + +To schedule a DAG to run only when two specific datasets have both been updated, use the AND operator (``&``): + +.. code-block:: python + + from airflow.models import DAG + from airflow.operators.bash import BashOperator + from airflow.datasets import Dataset + import pendulum + Review Comment: Sure. Changed it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
