Andrushika commented on code in PR #66854:
URL: https://github.com/apache/airflow/pull/66854#discussion_r3608976524
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py:
##########
@@ -499,6 +506,9 @@ def ti_update_state(
extra=json.dumps({"host_name": hostname}) if hostname else
None,
)
)
+ # Commit now to release the task_instance row lock before asset
registration runs, so
+ # concurrent task completions don't pile up waiting on it.
+ session.commit()
Review Comment:
I think the transactional outbox (durable marker) approach is the right
direction, but IIUC it is not yet implemented atomically here.
Should the `asset_event_queue` row be inserted before this
`session.commit()`, so the TI state and the durable marker are committed in the
same transaction? Otherwise, the process could die after `SUCCESS` is committed
but before asset registration or enqueueing, leaving no event and no queue row
for the scheduler to recover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
