[
https://issues.apache.org/jira/browse/HIVE-14271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña resolved HIVE-14271.
--------------------------------
Resolution: Won't Fix
A different approach will be used to improve S3 write performance. This jira is
meant for S3 direct writes, and there are concerns about using this approach.
Close it as won't fix.
See HIVE-15093
> FileSinkOperator should not rename files to final paths when S3 is the
> default destination
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-14271
> URL: https://issues.apache.org/jira/browse/HIVE-14271
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sergio Peña
> Assignee: Sergio Peña
>
> FileSinkOperator does a rename of {{outPaths -> finalPaths}} when it finished
> writing all rows to a temporary path. The problem is that S3 does not support
> renaming.
> Two options can be considered:
> a. Use a copy operation instead. After FileSinkOperator writes all rows to
> outPaths, then the commit method will do a copy() call instead of move().
> b. Write row by row directly to the S3 path (see HIVE-1620). This may add
> better performance calls, but we should take care of the cleanup part in case
> of writing errors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
