jason810496 commented on code in PR #66911:
URL: https://github.com/apache/airflow/pull/66911#discussion_r3258080345
##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -870,9 +870,18 @@ def trigger(
)
try:
- self.client.post(
- f"dag-runs/{dag_id}/{run_id}",
content=body.model_dump_json(exclude_defaults=True)
+ self.client._request_without_retry(
+ "POST", f"dag-runs/{dag_id}/{run_id}",
content=body.model_dump_json(exclude_defaults=True)
)
+ except httpx.RequestError:
+ if not reset_dag_run and self._dag_run_exists(dag_id=dag_id,
run_id=run_id):
Review Comment:
More specific:
1. Pre-check GET dag-runs/{dag_id}/{run_id} before the POST.
2. If it already exists:
- reset_dag_run=True -> call clear()
- otherwise -> return
ErrorResponse(error=ErrorType.DAGRUN_ALREADY_EXISTS)
3. Only if it did not exist, issue the POST.
4. If the POST later gets a 409 after an ambiguous retry, return
OKResponse(ok=True) because the first POST likely created it.
5. If a final RequestError happens, only convert to OK when the pre-check
said it did not exist and a follow-up GET now finds it.
Best version: track whether the failed POST was ambiguous, such as
ReadTimeout/ReadError, and do not convert ConnectError or
PoolTimeout into success just because the Dag run exists afterward.
Tests should cover:
- pre-existing Dag run + transport error does not return OK
- pre-existing Dag run returns DAGRUN_ALREADY_EXISTS
- first POST creates the Dag run, response is lost, retry sees 409 -> OK
- POST fails before reaching server and Dag run still does not exist ->
propagate the error
- reset_dag_run=True still clears pre-existing runs
##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -870,9 +870,18 @@ def trigger(
)
try:
- self.client.post(
- f"dag-runs/{dag_id}/{run_id}",
content=body.model_dump_json(exclude_defaults=True)
+ self.client._request_without_retry(
+ "POST", f"dag-runs/{dag_id}/{run_id}",
content=body.model_dump_json(exclude_defaults=True)
Review Comment:
We should pre-check whether the DagRun exist before the POST API call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
