[ https://issues.apache.org/jira/browse/HADOOP-17139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17355593#comment-17355593 ]
Bogdan Stolojan commented on HADOOP-17139: ------------------------------------------ Would like to have a look at this one if that's ok > Re-enable optimized copyFromLocal implementation in S3AFileSystem > ----------------------------------------------------------------- > > Key: HADOOP-17139 > URL: https://issues.apache.org/jira/browse/HADOOP-17139 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.3.0, 3.2.1 > Reporter: Sahil Takiar > Priority: Minor > > It looks like HADOOP-15932 disabled the optimized copyFromLocal > implementation in S3A for correctness reasons. innerCopyFromLocalFile should > be fixed and re-enabled. The current implementation uses > FileSystem.copyFromLocal which will open an input stream from the local fs > and an output stream to the destination fs, and then call IOUtils.copyBytes. > With default configs, this will cause S3A to read the file into memory, write > it back to a file on the local fs, and then when the file is closed, upload > it to S3. > The optimized version of copyFromLocal in innerCopyFromLocalFile, directly > creates a PutObjectRequest request with the local file as the input. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org