ephraimbuddy commented on code in PR #44976:
URL: https://github.com/apache/airflow/pull/44976#discussion_r1916965307
##########
airflow/dag_processing/manager.py:
##########
@@ -653,6 +653,10 @@ def _refresh_dag_bundles(self):
self.log.info("Refreshing DAG bundles")
for bundle in self._dag_bundles:
+ # TODO: AIP-66 handle errors in the case of incomplete cloning?
And test this.
+ # What if the cloning/refreshing took too long(longer than the
dag processor timeout)
+ if not bundle.is_initialized:
+ bundle.initialize()
Review Comment:
@jedcunningham , I noticed that if ssh connection is not configured well,
then it will get stuck cloning the repo until the dag processor times out. On
the restart, the dag processor will stay on crash loop.
Dag processor default timeout is 50 secs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
