Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14311 )
Change subject: IMPALA-8950: Add -d, -f options to hdfs copyFromLocal, put, cp ...................................................................... Patch Set 1: Code-Review+1 (2 comments) http://gerrit.cloudera.org:8080/#/c/14311/1/tests/common/impala_test_suite.py File tests/common/impala_test_suite.py: http://gerrit.cloudera.org:8080/#/c/14311/1/tests/common/impala_test_suite.py@198 PS1, Line 198: cli_client naming: ImpalaTestSuite.cli_client doesn't tell you much about what to expect from this client. Maybe hdfs_cli. hdfs_helper, or something similar would be more descriptive. http://gerrit.cloudera.org:8080/#/c/14311/1/tests/util/hdfs_util.py File tests/util/hdfs_util.py: http://gerrit.cloudera.org:8080/#/c/14311/1/tests/util/hdfs_util.py@217 PS1, Line 217: def copy_from_local(self, src, dst): : """Wrapper around 'hdfs dfs -copyFromLocal [-f] [-p] [-l] [-d] <localsrc> ... <dst>'. : Overwrites files by default to avoid S3 consistency issues. Specifes the '-d' option : by default, which 'Skip[s] creation of temporary file with the suffix ._COPYING_.' to : avoid extraneous copies on S3. 'src' must be either a string or a list of strings.""" : assert isinstance(src, list) or isinstance(src, basestring) : src_list = src if isinstance(src, list) else [src] : (status, stdout, stderr) = self._hadoop_fs_shell(['-copyFromLocal', '-d', '-f'] + : src_list + [dst]) : assert status == 0, '{0} copy from {1} to {2} failed: '.format(self.filesystem_type, : src, dst) + stderr + '; ' + stdout Is it useful to keep both copy_from_local() and put()? I think that these do exactly the same, so I think it is a good time go get rid of one of them. -- To view, visit http://gerrit.cloudera.org:8080/14311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0d45db1c00554e6fb6bcc0b552596d86d4e30144 Gerrit-Change-Number: 14311 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar <stak...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Comment-Date: Thu, 26 Sep 2019 16:53:14 +0000 Gerrit-HasComments: Yes
