boushphong commented on issue #35870: URL: https://github.com/apache/airflow/issues/35870#issuecomment-1837489928
@nathadfield Just submitted a pull request. It's a DRAFT PR for now. https://github.com/apache/airflow/pull/36032/files#diff-4253adbb36bfb93cb75ab00c7d509518134e5bf1ad16473b64a2a6d8fa456c92L208-L214 I went with the idea to remove primary key for the `dataset_dag_run_queue` table so that when we insert a new record in the table as in ([code](https://github.com/apache/airflow/blob/9439111e739e24f0e3751350186b0e2130d2c821/airflow/datasets/manager.py#L128)): ```python stmt = insert(DatasetDagRunQueue).values(dataset_id=dataset.id).on_conflict_do_nothing() ``` so that we won't face any conflict issue because if a `Dag` has multiple tasks updating the same `Dataset`, we would get a conflict because we insert 2 records but they'd conflict with each other due to the primary key constraint. Just briefing my idea before committing more time to this solution. WDYT? By the way, if I make changes to the model, Do I have to modify the migrations package and if so where would I have to look into. Cheers! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
