kalluripradeep commented on issue #66905:
URL: https://github.com/apache/airflow/issues/66905#issuecomment-4766236919
I've traced the full call stack for this issue and opened a fix PR: #68828
**Root cause confirmed:**
The tenacity `@retry` in `Client.request()` retries `httpx.RequestError`
(transport errors). When `POST dag-runs/{dag_id}/{run_id}` creates the run
server-side but the client sees a transport error, the retry gets `409
Conflict`. `DagRunOperations.trigger()` converts that to
`ErrorResponse(DAGRUN_ALREADY_EXISTS)`. In `_handle_trigger_dag_run`, the
`else` branch (when `skip_when_already_exists=False`, the default) marks the
task **FAILED** without pushing `trigger_run_id` XCom.
**Fix:**
When `DAGRUN_ALREADY_EXISTS` arrives and `skip_when_already_exists=False`,
the requested `run_id` already exists — the desired end state. The fix falls
through to the normal success path (push XCom, honour `wait_for_completion`)
instead of failing.
The `skip_when_already_exists=True` branch is unchanged.
Call stack for reference:
1. `TriggerDagRunOperator._trigger_dag_af_3()` → raises
`DagRunTriggerException`
2. `task_runner.run()` → `_handle_trigger_dag_run(drte, context, ti, log)`
3. `SUPERVISOR_COMMS.send(TriggerDagRun(...))` → supervisor calls
`client.dag_runs.trigger(...)`
4. `DagRunOperations.trigger()`: POST → transport error → retry → `409
Conflict` → `ErrorResponse(DAGRUN_ALREADY_EXISTS)`
5. Old: FAIL. New: fall through to XCom push + success.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
