SameerMesiah97 opened a new pull request, #72229: URL: https://github.com/apache/airflow/pull/72229
**Description** This change introduces a new sensor, `AthenaSparkSensor`, to wait for an existing Amazon Athena Spark calculation execution to reach a terminal state. The sensor delegates to `AthenaHook` Spark calculation helpers to poll the calculation status. It succeeds when the calculation reaches `COMPLETED`, continues poking while the calculation is in an in-progress state, and raises when the calculation reaches a failed, canceled, or unexpected state. **Rationale** Amazon Athena Spark calculations are identified by a `calculation_execution_id` after they are submitted. While `AthenaSparkOperator` can submit Spark code and wait for completion, users may also need to monitor calculations started outside the operator, for example by another task or by an external system. Adding `AthenaSparkSensor` gives DAG authors a provider-native way to wait on an existing Athena Spark calculation without writing custom polling logic around boto3. **Tests** Added unit tests verifying that: * the sensor inherits the expected AWS hook configuration behavior. * successful calculation states are handled correctly. * in-progress states such as `CREATING`, `CREATED`, `QUEUED`, and `RUNNING` continue waiting. * failed and canceled calculation states raise an error. * the state change reason is included when Athena provides one. **Example DAG** Updated the Athena Spark example DAG to demonstrate waiting for an existing Athena Spark calculation execution using `AthenaSparkSensor`. **Documentation** Added documentation for `AthenaSparkSensor`, including its expected `calculation_execution_id` input and an example usage snippet. **Backwards Compatibility** This is a new sensor and does not modify existing Athena sensor behavior. No breaking changes are introduced. ###### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: [GPT 5.5] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
