[GitHub] [airflow] Sangameshmsopen commented on issue #25060: Airflow scheduler crashes due to 'duplicate key value violates unique constraint "task_instance_pkey"'
Sangameshmsopen commented on issue #25060: URL: https://github.com/apache/airflow/issues/25060#issuecomment-1436735346 @potiuk Sure i'm providing the details. Same issue has occurred even in one of our testing environment. **Apache airflow version:** 2.4.3 **Deployed in Google cloud composer:** 2.1.4 **What happened:** We have one scheduling DAG which will trigger at 12:00 AM. In that DAG we have main task and sub tasks as well. Sub tasks will be created dynamically based on few arguments. When Sub task (which has created dynamically) starts in DAG, I can see instance details as null (means no instance has created for that task. Please refer screenshot 1). So i don't get any logs for that task. Screenshot 1: https://user-images.githubusercontent.com/107921145/220083694-b090e7b8-1836-46b2-be18-443a3bf763a9.png;> But when i checked the logs in composer service. I can see the error log which has occurred under scheduler and time is almost near to stopping of scheduler heart beat. (Please refer screenshot 2) Screenshot 2:   Please let me know if any other details are required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] Sangameshmsopen commented on issue #25060: Airflow scheduler crashes due to 'duplicate key value violates unique constraint "task_instance_pkey"'
Sangameshmsopen commented on issue #25060: URL: https://github.com/apache/airflow/issues/25060#issuecomment-1424199989 Hello all, We have observed that in Airflow scheduler heartbeat has stopped and all the DAGs were in queued or running state. We have checked scheduler logs and got the below error. May i know what might be the cause for this. Details: We are running Airflow in Google cloud Composer service with below versions composer-2.1.4 [airflow-2](https://issues.apache.org/jira/browse/AIRFLOW-2).4.3 Error: sqlalchemy.exc.PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "task_instance_pkey" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
