This is an automated email from the ASF dual-hosted git repository.
shahar pushed a commit to branch v2-10-stable
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v2-10-stable by this push:
new ac40b27bd7 Typo fix dataset guide (#41353)
ac40b27bd7 is described below
commit ac40b27bd7d694277b00a370538268ce653dc9f9
Author: Tamara Janina Fingerlin <[email protected]>
AuthorDate: Fri Aug 9 17:54:57 2024 +0200
Typo fix dataset guide (#41353)
---
docs/apache-airflow/authoring-and-scheduling/datasets.rst | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/docs/apache-airflow/authoring-and-scheduling/datasets.rst
b/docs/apache-airflow/authoring-and-scheduling/datasets.rst
index 07eb571153..a69c09bc13 100644
--- a/docs/apache-airflow/authoring-and-scheduling/datasets.rst
+++ b/docs/apache-airflow/authoring-and-scheduling/datasets.rst
@@ -338,7 +338,7 @@ In this example, the DAG ``waiting_for_dataset_1_and_2``
will be triggered when
...
-``quededEvent`` API endpoints are introduced to manipulate such records.
+``queuedEvent`` API endpoints are introduced to manipulate such records.
* Get a queued Dataset event for a DAG: ``/datasets/queuedEvent/{uri}``
* Get queued Dataset events for a DAG: ``/dags/{dag_id}/datasets/queuedEvent``
@@ -347,7 +347,7 @@ In this example, the DAG ``waiting_for_dataset_1_and_2``
will be triggered when
* Get queued Dataset events for a Dataset:
``/dags/{dag_id}/datasets/queuedEvent/{uri}``
* Delete queued Dataset events for a Dataset: ``DELETE
/dags/{dag_id}/datasets/queuedEvent/{uri}``
- For how to use REST API and the parameters needed for these endpoints, please
refer to :doc:`Airflow API </stable-rest-api-ref>`
+ For how to use REST API and the parameters needed for these endpoints, please
refer to :doc:`Airflow API </stable-rest-api-ref>`.
Advanced dataset scheduling with conditional expressions
--------------------------------------------------------
@@ -444,7 +444,7 @@ The following example creates a dataset event against the
S3 URI ``f"s3://bucket
@task(outlets=[DatasetAlias("my-task-outputs")])
def my_task_with_metadata():
- s3_dataset = Dataset("s3://bucket/my-task}")
+ s3_dataset = Dataset("s3://bucket/my-task")
yield Metadata(s3_dataset, extra={"k": "v"}, alias="my-task-outputs")
Only one dataset event is emitted for an added dataset, even if it is added to
the alias multiple times, or added to multiple aliases. However, if different
``extra`` values are passed, it can emit multiple dataset events. In the
following example, two dataset events will be emitted.
@@ -470,7 +470,7 @@ Only one dataset event is emitted for an added dataset,
even if it is added to t
Scheduling based on dataset aliases
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Since dataset events added to an alias are just simple dataset events, a
downstream depending on the actual dataset can read dataset events of it
normally, without considering the associated aliases. A downstream can also
depend on a dataset alias. The authoring syntax is referencing the
``DatasetAlias`` by name, and the associated dataset events are picked up for
scheduling. Note that a DAG can be triggered by a task with
``outlets=DatasetAlias("xxx")`` if and only if the alias is resol [...]
+Since dataset events added to an alias are just simple dataset events, a
downstream DAG depending on the actual dataset can read dataset events of it
normally, without considering the associated aliases. A downstream DAG can also
depend on a dataset alias. The authoring syntax is referencing the
``DatasetAlias`` by name, and the associated dataset events are picked up for
scheduling. Note that a DAG can be triggered by a task with
``outlets=DatasetAlias("xxx")`` if and only if the alias [...]
The dataset alias is resolved to the datasets during DAG parsing. Thus, if the
"min_file_process_interval" configuration is set to a high value, there is a
possibility that the dataset alias may not be resolved. To resolve this issue,
you can trigger DAG parsing.