Abdulrehman-PIAIC80387 opened a new pull request, #68327: URL: https://github.com/apache/airflow/pull/68327
A soft-fail (or never-fail) sensor that **defers** (uses a trigger) and then requests a **reschedule** on resume is wrongly marked `SKIPPED` instead of going `UP_FOR_RESCHEDULE`. `BaseSensorOperator.resume_execution` wraps the resumed call in a broad `except (AirflowException, TaskDeferralError)` that converts the exception into `AirflowSkipException` when `soft_fail` / `never_fail` is set. Because `AirflowRescheduleException` subclasses `AirflowException`, a reschedule request raised on resume is caught by that handler and turned into a skip — so the task silently finishes instead of rescheduling. The non-deferred poke loop in `execute()` does not have this problem: it raises `AirflowRescheduleException` outside the `soft_fail`/`never_fail` handling, so a reschedule always propagates. `resume_execution` was simply inconsistent with `execute()`. ## Fix Re-raise `AirflowRescheduleException` from `resume_execution` before the `soft_fail` / `never_fail` conversion. The runtime already handles a propagated `AirflowRescheduleException` by marking the task `UP_FOR_RESCHEDULE` (`task_runner.py`), so the reschedule now takes effect on the deferred path too. ## Design notes - The new `except AirflowRescheduleException: raise` must come **before** the broad `except (AirflowException, TaskDeferralError)` because `AirflowRescheduleException` is a subclass of `AirflowException`; Python matches the first compatible handler. - This mirrors `execute()`, which never converts a reschedule into a skip — so behaviour is now consistent between the poke and deferred paths. - Genuine failures and trigger timeouts are unchanged: they still skip under `soft_fail` / `never_fail` (covered by the existing tests, which continue to pass). ## Tests Added `test_reschedule_after_resuming_deferred_sensor_propagates`, parametrized over default / `soft_fail=True` / `never_fail=True`, asserting that a reschedule raised on resume propagates as `AirflowRescheduleException` in every mode. Before the fix the `soft_fail` / `never_fail` cases raised `AirflowSkipException`. Full `test_sensor.py` suite passes (52 passed). closes: #49329 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Mirza Generated-by: Mirza following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
