HsiuChuanHsu commented on issue #53905: URL: https://github.com/apache/airflow/issues/53905#issuecomment-3177163581
Hi @enayati239, @vikramkoka Look into this issue, I think this issue is likely caused by: 1. **Race Condition**: Multiple processes trying to insert the same rendered task instance fields simultaneously 2. **Database Transaction Issues**: Incomplete transactions leaving partial data 3. **Retry Logic**: Task retries attempting to re-insert already existing records **Two fixs i think might help us with this issus:** 1. **Code-Level Fix** Handle duplicate insertions error gracefully here. https://github.com/apache/airflow/blob/e1f4595f20ecdcca0a37a389577b495c32864afb/airflow-core/src/airflow/models/renderedtifields.py#L204-L212 2. **Database-Level Fix** We can consider add better indexing here. https://github.com/apache/airflow/blob/e1f4595f20ecdcca0a37a389577b495c32864afb/airflow-core/src/airflow/models/renderedtifields.py#L79-L98 Let me know what you think! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
