[
https://issues.apache.org/jira/browse/HIVE-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025937#comment-13025937
]
Ashutosh Chauhan commented on HIVE-2035:
----------------------------------------
Yeah, correct Ning. I missed jira number by 100. Edited the review request.
Btw, it will be great if you can take a look at that.
> Use block-level merge for RCFile if merging intermediate results are needed
> ---------------------------------------------------------------------------
>
> Key: HIVE-2035
> URL: https://issues.apache.org/jira/browse/HIVE-2035
> Project: Hive
> Issue Type: Improvement
> Reporter: Ning Zhang
> Assignee: He Yongqiang
>
> Currently if hive.merge.mapredfiles and/or hive.merge.mapfile is set to true
> the intermediate data could be merged using an additional MapReduce job. This
> could be quite expensive if the data size is large. With HIVE-1950, merging
> can be done in the RCFile block level so that it bypasses the
> (de-)compression, (de-)serialization phases. This could improve the merge
> process significantly.
> This JIRA should handle the case where the input table is not stored in
> RCFile, but the destination table is (which requires the intermediate data
> should be stored in the same format as the destination table).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
