eladkal commented on code in PR #61274:
URL: https://github.com/apache/airflow/pull/61274#discussion_r2877487167
##########
airflow-core/docs/administration-and-deployment/logging-monitoring/callbacks.rst:
##########
@@ -53,23 +53,47 @@ Callback Types
There are six types of events that can trigger a callback:
-===========================================
================================================================
-Name Description
-===========================================
================================================================
-``on_success_callback`` Invoked when the :ref:`Dag
succeeds <dag-run:dag-run-status>` or :ref:`task succeeds
<concepts:task-instances>`.
- Available at the Dag or task level.
-``on_failure_callback`` Invoked when the task :ref:`fails
<concepts:task-instances>`.
- Available at the Dag or task level.
-``on_retry_callback`` Invoked when the task is :ref:`up
for retry <concepts:task-instances>`.
- Available only at the task level.
-``on_execute_callback`` Invoked right before the task
begins executing.
- Available only at the task level.
-``on_skipped_callback`` Invoked when the task is
:ref:`running <concepts:task-instances>` and AirflowSkipException raised.
- Explicitly it is NOT called if a
task is not started to be executed because of a preceding branching
- decision in the Dag or a trigger
rule which causes execution to skip so that the task execution
- is never scheduled.
- Available only at the task level.
-===========================================
================================================================
+===========================================
=======================================================================
=================
+Name Description
Availability
+===========================================
=======================================================================
=================
+``on_success_callback`` Invoked when the :ref:`Dag
succeeds <dag-run:dag-run-status>` Dag or Task
+ or :ref:`task succeeds
<concepts:task-instances>`.
+``on_failure_callback`` Invoked when the :ref:`Dag fails
<dag-run:dag-run-status>` Dag or Task
+ or task :ref:`fails
<concepts:task-instances>`.
+``on_retry_callback`` Invoked when the task is :ref:`up
for retry <concepts:task-instances>`. Task
+``on_execute_callback`` Invoked right before the task
begins executing. Task
+``on_skipped_callback`` Invoked when the task is
:ref:`running <concepts:task-instances>` Task
+ and AirflowSkipException raised.
Explicitly it is NOT called if a task
+ is not started to be executed
because of a preceding branching
+ decision in the Dag or a trigger
rule which causes execution
+ to skip so that the task execution
is never scheduled.
+===========================================
=======================================================================
=================
+
+
+Context Mapping
+---------------
+
+A context mapping that contains runtime information about a task instance is
passed to every callback.
+Full list of variables available in ``context`` are in :doc:`docs
<../../templates-ref>` and `code
<https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/definitions/context.py>`_.
+
+
+Dag Callbacks
+^^^^^^^^^^^^^
+
+As the context mapping describes execution of a task instance, contexts passed
to Dag callbacks will also contain task instance variables,
+and the task selected depends on the state of a Dag:
+
+#. On regular failure, the latest failed task is selected.
+#. On Dag run timeout, the latest started but not finished task is passed.
+#. If tasks are deadlocked, a task that should have run next but couldn't is
passed.
+#. On success, the latest succeeded task is passed.
+
+It's not recommended to rely on task instance variables in Dag callbacks
except for human analysis, as they reflect only partial information about the
Dag's state.
+For example, a timeout may be caused by a number of stalling tasks, but only
one will eventually be selected for context.
+
+.. note::
+ Before Airflow 3.1.9, the rules above did not apply and the task instance
passed to Dag callback was not related to Dag state, rather being selected as
the latest task in the Dag
Review Comment:
cc @vatsrahul1001 if this PR is merged before cut of 3.1.8 we'll need to
change this entry just before mrege
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
