pierrejeambrun commented on code in PR #67900:
URL: https://github.com/apache/airflow/pull/67900#discussion_r3461155552
##########
airflow-core/src/airflow/models/backfill.py:
##########
@@ -628,7 +628,15 @@ def _create_backfill(
if not serdag:
raise DagNotFound(f"Could not find dag {dag_id}")
- dag_model = session.scalar(select(DagModel).where(DagModel.dag_id ==
dag_id).limit(1))
+ # Lock the DagModel row (FOR UPDATE on PG/MySQL) to serialize
concurrent
+ # backfill creation for the same dag_id. This prevents two requests
from
+ # both passing the num_active check before either commits.
+ dag_model = session.scalar(
+ with_row_locks(
+ query=select(DagModel).where(DagModel.dag_id ==
dag_id).limit(1),
+ session=session,
+ )
Review Comment:
I wouldn't add this lock just to prevent a sqlite potential issue. Deadlock
for production db are a concern and I think the handling added above is enough
since sqlite isn't production ready anyway
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
