sidshas03 commented on PR #63355: URL: https://github.com/apache/airflow/pull/63355#issuecomment-4043391958
> @sidshas03 Could you post logs please Hi @kaxil, as requested, sharing the relevant logs and exact lines. <img width="1571" height="433" alt="Screenshot 2026-03-11 at 22 05 05" src="https://github.com/user-attachments/assets/c0cd943c-375e-4cf9-ace8-0f8a41847b6c"; /> 1) Symptom from #63183 (task completed, then 409, then process exits) - `"event": "Done. Returned value was: None"` - `"exc_value": "API_SERVER_ERROR: {'status_code': 409, ... 'reason': 'invalid_state', ... 'previous_state': 'success'}"` - `"event": "Process exited abnormally"` 2) Scheduler race timeline from #57618 (same TI getting two tries) - `2025-12-15T04:42:29.146414Z [info] Add task ... try_number=1` - `2025-12-15T04:43:58.258654Z [info] Reset the following 12 orphaned TaskInstances ...` - `2025-12-15T04:44:01.790234Z [info] Add task ... try_number=2` - `2025-12-15T04:45:19.337468Z [info] Marking run <DagRun sadp_dag_128 ...> successful` - `2025-12-15T04:45:20.038291Z [info] Creating kubernetes pod ... try_number=1` - `2025-12-15T04:46:04.784337Z [info] Received executor event with state failed ... try_number=1` 3) API/server side conflict log from #57618 - `{"event":"API server error","status_code":409,"detail":{"detail":{"reason":"invalid_state","previous_state":"success"}}}` - `{"event":"Top level error", ... "API_SERVER_ERROR: {'status_code': 409, ... 'previous_state': 'success'}"}` - `airflow.sdk.api.client.ServerResponseError: Server returned error` Links: - https://github.com/apache/airflow/issues/63183 - https://github.com/apache/airflow/issues/57618#issuecomment-3654781912 - https://github.com/apache/airflow/issues/57618#issuecomment-3665287828 From these logs, root cause is scheduler-side HA race/duplicate scheduling path; #63355 is a focused mitigation so duplicate-success 409 does not fail task process, while scheduler race handling is in #63367. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
