David Knupp has posted comments on this change. ( http://gerrit.cloudera.org:8080/13020 )
Change subject: IMPALA-8344: Add support for running the minicluster with S3Guard ...................................................................... Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/13020/2/tests/util/hdfs_util.py File tests/util/hdfs_util.py: http://gerrit.cloudera.org:8080/#/c/13020/2/tests/util/hdfs_util.py@170 PS2, Line 170: f = tempfile.NamedTemporaryFile(delete=False) Caveat: I commented and then realized this class is just being moved, so take this as non-blocking. Use a context manager? with tempfile.NamedTemporaryFile(delete=False) as tmp_file: tmp_file.write(file_data) That calls close() for you automatically. You can later still get tmp_file.name on a closed file handle, so no need for tmp_path. http://gerrit.cloudera.org:8080/#/c/13020/2/tests/util/hdfs_util.py@181 PS2, Line 181: return True Caveat: I commented and then realized this class is just being moved, so take this as non-blocking. Maybe do the same thing as the previous method? (status, stdout, stderr) = self._hadoop_fs_shell(['-mkdir', '-p', fixed_path]) return status == 0 http://gerrit.cloudera.org:8080/#/c/13020/2/tests/util/hdfs_util.py@213 PS2, Line 213: fname = f['name'].split("/")[-1] Caveat: I commented and then realized this class is just being moved, so take this as non-blocking. Don't if it helps in this case, but the python os lib has a lot of path manipulation helpers, e.g.: >>> import os >>> f = 'foo/bar/baz' >>> os.path.dirname(f) 'foo/bar' >>> os.path.basename(f) 'baz' >>> os.path.abspath(f) '/<pwd>/foo/bar/baz' -- To view, visit http://gerrit.cloudera.org:8080/13020 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c748529a494bb6e70fec96dc031523ff79bf61d Gerrit-Change-Number: 13020 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: David Knupp <dkn...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Laszlo Gaal <laszlo.g...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com> Gerrit-Comment-Date: Mon, 29 Apr 2019 20:10:34 +0000 Gerrit-HasComments: Yes
