Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/24744 )
Change subject: IMPALA-15290: Retry for exists check in HadoopFsCommandLineClient ...................................................................... Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/24744/2/tests/util/hdfs_util.py File tests/util/hdfs_util.py: http://gerrit.cloudera.org:8080/#/c/24744/2/tests/util/hdfs_util.py@299 PS2, Line 299: (status, stdout, stderr) = self._hadoop_fs_shell(['-test', '-e', path]) : # Retry for transient errors like S3 503 Slow Down. Path absence has empty stderr. : if status != 0 and not stderr.strip(): : LOG.warning('{0} exists failed: {1}; {2}. Retry after 1s.'.format( : self.filesystem_type, stderr, stdout)) : time.sleep(1) : (status, stdout, stderr) = self._hadoop_fs_shell(['-test', '-e', path]) : if status != 0 and not stderr.strip(): : LOG.warning('{0} exists failed: {1}; {2}. Retry after 1s.'.format( : self.filesystem_type, stderr, stdout)) Should we make this a loop and retry more than once? If it fails with multiple retries, it would be nice to have the error say how many times it tried. -- To view, visit http://gerrit.cloudera.org:8080/24744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I071472529ec1b923e74e960a24e19b9e70c80011 Gerrit-Change-Number: 24744 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]> Gerrit-Comment-Date: Mon, 31 Aug 2026 22:06:34 +0000 Gerrit-HasComments: Yes
