[
https://issues.apache.org/jira/browse/CRUNCH-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13745755#comment-13745755
]
Chao Shi commented on CRUNCH-251:
---------------------------------
Pushed to master.
> HFileUtils#sortAndPartition does not work when two instances exist in the
> same pipeline
> ---------------------------------------------------------------------------------------
>
> Key: CRUNCH-251
> URL: https://issues.apache.org/jira/browse/CRUNCH-251
> Project: Crunch
> Issue Type: Bug
> Components: IO
> Reporter: Chao Shi
> Assignee: Chao Shi
> Fix For: 0.8.0
>
> Attachments: crunch-251.patch
>
>
> This bug happens when generating some data and write them to two HTables
> (which different number of regions) in a single pipeline. The job fails
> because of "Wrong number of partitions in keyset".
> This is because it is using TotalOrderPartitioner#setPartitionFile, which
> sets the path to partition file to a global conf. The correct approach should
> be only change the reducer's conf (as it is done in Sort#configureReducers).
> So the fix is straightforward.
> This has been tested on a real HBase cluster. I will make a patch soon.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
