jedcunningham commented on a change in pull request #17649: URL: https://github.com/apache/airflow/pull/17649#discussion_r690623986
########## File path: airflow/providers/cncf/kubernetes/utils/pod_launcher.py ########## @@ -217,7 +223,7 @@ def base_container_is_running(self, pod: V1Pod): return False return status.state.running is not None - @tenacity.retry(stop=tenacity.stop_after_attempt(3), wait=tenacity.wait_exponential(), reraise=True) + @tenacity.retry(stop=tenacity.stop_after_attempt(4), wait=tenacity.wait_exponential(), reraise=True) Review comment: How'd you land on 4 attempts? Do we even need to bump this with the other change? ########## File path: airflow/providers/cncf/kubernetes/utils/pod_launcher.py ########## @@ -143,12 +143,21 @@ def monitor_pod(self, pod: V1Pod, get_logs: bool) -> Tuple[State, V1Pod, Optiona read_logs_since_sec = None last_log_time = None while True: - logs = self.read_pod_logs(pod, timestamps=True, since_seconds=read_logs_since_sec) - for line in logs: - timestamp, message = self.parse_log_line(line.decode('utf-8')) - self.log.info(message) - if timestamp: - last_log_time = timestamp + try: + logs = self.read_pod_logs(pod, timestamps=True, since_seconds=read_logs_since_sec) + for line in logs: + timestamp, message = self.parse_log_line(line.decode('utf-8')) + self.log.info(message) + if timestamp: + last_log_time = timestamp + except Exception as e: Review comment: This is pretty broad, maybe we only look for `TimeoutError`'s? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org