This is an automated email from the ASF dual-hosted git repository.
o-nikolas pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 6c64331e41c Let the Athena Spark system test follow default_deferrable
(#77173)
6c64331e41c is described below
commit 6c64331e41cc6abc38e034ef6b291378044a2569
Author: Sean Ghaeli <[email protected]>
AuthorDate: Tue Nov 17 14:33:12 2026 -0700
Let the Athena Spark system test follow default_deferrable (#77173)
The deferrable example task forced deferrable=True, so the Dag hangs
when it runs with an executor and no triggerer. Like the other Amazon
system tests, keep one calculation task and leave deferrable unset so it
follows operators.default_deferrable.
---
providers/amazon/docs/operators/athena/athena_spark.rst | 6 ------
.../amazon/tests/system/amazon/aws/example_athena_spark.py | 13 -------------
2 files changed, 19 deletions(-)
diff --git a/providers/amazon/docs/operators/athena/athena_spark.rst
b/providers/amazon/docs/operators/athena/athena_spark.rst
index 2bbf43f1d7d..de4431e52e8 100644
--- a/providers/amazon/docs/operators/athena/athena_spark.rst
+++ b/providers/amazon/docs/operators/athena/athena_spark.rst
@@ -65,12 +65,6 @@ in both synchronous and deferrable modes. Both modes return
the same execution
metadata when the calculation completes successfully and raise an exception
if it fails or is canceled.
-.. exampleinclude::
/../../amazon/tests/system/amazon/aws/example_athena_spark.py
- :language: python
- :dedent: 4
- :start-after: [START howto_operator_athena_spark_deferrable]
- :end-before: [END howto_operator_athena_spark_deferrable]
-
.. note::
When the trigger reports that polling attempts have been exhausted, the
diff --git a/providers/amazon/tests/system/amazon/aws/example_athena_spark.py
b/providers/amazon/tests/system/amazon/aws/example_athena_spark.py
index 32f44502770..f479b45347a 100644
--- a/providers/amazon/tests/system/amazon/aws/example_athena_spark.py
+++ b/providers/amazon/tests/system/amazon/aws/example_athena_spark.py
@@ -132,21 +132,9 @@ with DAG(
code_block="print('hello from athena spark')",
waiter_delay=30,
waiter_max_attempts=120,
- deferrable=False,
)
# [END howto_operator_athena_spark]
- # [START howto_operator_athena_spark_deferrable]
- run_spark_calculation_deferrable = AthenaSparkOperator(
- task_id="run_spark_calculation_deferrable",
- session_id=idle_session_id,
- code_block="print('hello from athena spark in deferrable mode')",
- waiter_delay=30,
- waiter_max_attempts=120,
- deferrable=True,
- )
- # [END howto_operator_athena_spark_deferrable]
-
calculation_execution_id = start_athena_spark_calculation(idle_session_id)
# [START howto_sensor_athena_spark]
@@ -176,7 +164,6 @@ with DAG(
idle_session_id,
# TEST BODY
run_spark_calculation,
- run_spark_calculation_deferrable,
calculation_execution_id,
await_spark_calculation,
# TEST TEARDOWN