Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14311 )
Change subject: IMPALA-8950: Add -d, -f options to hdfs copyFromLocal, put, cp ...................................................................... Patch Set 2: (1 comment) Removed the Python-based implementations of 'copy' and replaced it with CLI calls because (1) the Python implementation isn't great, it reads the whole file into memory and then writes the file out, and (2) there were not many users of self.filesystem_client.copy, so changing it should be low impact. http://gerrit.cloudera.org:8080/#/c/14311/2/tests/common/impala_test_suite.py File tests/common/impala_test_suite.py: http://gerrit.cloudera.org:8080/#/c/14311/2/tests/common/impala_test_suite.py@184 PS2, Line 184: # 'filesystem_client' is set depending on the value of the 'TARGET_FILESYSTEM'. For : # HDFS, it is the same as the 'hdfs_client'. For S3 and and ABFS, the client is a : # HadoopFsCommandLineClient which is a simple wrapper around 'hdfs dfs' commands. : # For ADLS, the 'filesystem_client' is an ADLSClient. > When would you use filesystem_client vs fs_cli_client? Merged the two. Created a DelegatingHdfsFilesystem extends the BaseFilesystem and either delegates to HadoopFsCommandLineClient or PyWebHdfsClientWithChmod. Did the same with the ADLSClient. -- To view, visit http://gerrit.cloudera.org:8080/14311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0d45db1c00554e6fb6bcc0b552596d86d4e30144 Gerrit-Change-Number: 14311 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar <stak...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com> Gerrit-Comment-Date: Fri, 27 Sep 2019 01:11:34 +0000 Gerrit-HasComments: Yes
