ephraimbuddy commented on code in PR #66788:
URL: https://github.com/apache/airflow/pull/66788#discussion_r3228149538
##########
airflow-core/src/airflow/dag_processing/collection.py:
##########
@@ -295,6 +310,29 @@ def _serialize_dag_capturing_errors(
return []
except OperationalError:
raise
+ except IntegrityError as exc:
+ # Multiple Dag processors writing the same brand-new Dag can race on
the INSERT.
+ # The loser's transaction is already invalid, so we must roll the
session back to
+ # avoid PendingRollbackError on subsequent per-Dag work in this
parsing cycle.
+ # The winning peer already produced the correct row, so this is not an
import error
+ # and we don't retry. Non-unique IntegrityErrors (e.g. NOT-NULL
violations from a
+ # genuinely malformed Dag) fall through to the generic Exception arm.
Review Comment:
Can you share a log of where you saw this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
