shahar1 opened a new pull request, #73376: URL: https://github.com/apache/airflow/pull/73376
closes: #31544 ## Human Summary A long-time requested feature of allowing to run a task instance even if it's upstream dependencies aren't met. It adds an `ignore_upstream_deps` flag to the clear-task instances API, as well as a "Force run" checkbox to the UI. Mapped tasks can be force-run only if the values they expand over have already been evaluated. Caveats are explained in the docs. ## AI Summary <details><summary>Click here</summary> Clearing a task only re-runs it once its dependencies are met again, so a task blocked by a failed upstream, by a branch or `ShortCircuitOperator` skip, or by `depends_on_past` cannot be re-run from the UI at all. The usual workaround is to mark the upstream tasks as success — rewriting history just to get one task to run. This adds an additive `ignore_upstream_deps` boolean to the clear-task-instances API, surfaced in the UI as a **Force run (ignore upstream dependencies)** checkbox in the clear dialog. **What it bypasses:** exactly the dependencies on *other task instances* — `TriggerRuleDep`, `NotPreviouslySkippedDep`, `PrevDagrunDep`, `MappedTaskUpstreamDep`. **What it deliberately does not bypass:** the timing dependencies. Skipping `NotInRetryPeriodDep` would let a forced retry spin ignoring `retry_delay`, and skipping `ReadyToRescheduleDep` would hot-loop a reschedule-mode sensor, for as long as the flag is set. Pools, `max_active_tasks`, `max_active_tis_per_dag(_per_dagrun)`, `max_active_runs`, paused Dags and Dag-run state are enforced by SQL filters in the scheduler and are untouched — a force run is not a queue jump. The flag is persisted on the `task_instance` row rather than being a property of the request, so the scheduler can read it at dependency-evaluation time. The check lives in `TaskInstance.get_failed_dep_statuses`, a single chokepoint, which keeps the scheduler, the REST `/dependencies` endpoint and `airflow tasks failed-deps` in agreement. Each try is snapshotted into `task_instance_history`, and any later clear by any means resets the flag. Guard rails: * The API rejects `ignore_upstream_deps` combined with `include_upstream` / `include_downstream` (422) — forcing a subtree would ignore the edges inside the subtree and run it in arbitrary order. The UI disables those two toggles while the box is ticked. * The UI sends `only_failed=false` while the box is ticked, since a task blocked on its dependencies is in `None`/`scheduled`/`skipped` state and would otherwise be silently excluded. * Mapped expansion copies the flag onto newly created map-index rows, so a forced mapped task is not forced at `map_index` 0 only. Conversely, indexes added later by a re-expansion the user never selected are evaluated normally rather than inheriting the bypass. * No new permission — same `requires_access_dag(method="PUT", access_entity=TASK_INSTANCE)` as clear and mark-state, and the existing audit log already captures the request body. The caveats that come with it (XCom holes from upstreams that never ran, `depends_on_past` ordering guarantees, downstreams staying skipped) are documented under "Re-run Tasks" in `dag-run.rst`. </details> --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5) Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
