[
https://issues.apache.org/jira/browse/HBASE-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars George updated HBASE-1287:
-------------------------------
Attachment: 1287-2.patch
As discussed. Moved code that ensured upper limits to separate methods and
added generic setters to be able to specify the number of mappers or reducers
for a given job. Also added setting the default HRegionPartitioner back in as
that seemed like it was the original intention. So either the given one is set
or the default one when that specific method is called. The initTableReduceJob
without the partitioner parameter is unchanged and does not set anything
specific related to the partitioner class.
> Partitioner class not used in TableMapReduceUtil.initTableReduceJob()
> ---------------------------------------------------------------------
>
> Key: HBASE-1287
> URL: https://issues.apache.org/jira/browse/HBASE-1287
> Project: Hadoop HBase
> Issue Type: Bug
> Components: mapred
> Reporter: Lars George
> Assignee: Lars George
> Attachments: 1287-2.patch, 1287.patch
>
>
> Upon checking the available utility methods in TableMapReduceUtil I came
> across this code
> {code}
> public static void initTableReduceJob(String table,
> Class<? extends TableReduce> reducer, JobConf job, Class partitioner)
> throws IOException {
> job.setOutputFormat(TableOutputFormat.class);
> job.setReducerClass(reducer);
> job.set(TableOutputFormat.OUTPUT_TABLE, table);
> job.setOutputKeyClass(ImmutableBytesWritable.class);
> job.setOutputValueClass(BatchUpdate.class);
> if (partitioner != null) {
> job.setPartitionerClass(HRegionPartitioner.class);
> HTable outputTable = new HTable(new HBaseConfiguration(job), table);
> int regions = outputTable.getRegionsInfo().size();
> if (job.getNumReduceTasks() > regions){
> job.setNumReduceTasks(outputTable.getRegionsInfo().size());
> }
> }
> }
> {code}
> It seems though as it should be
> {code}
> if (partitioner != null) {
> job.setPartitionerClass(partitioner);
> {code}
> and the provided HRegionPartitioner can be handed in to that call or a custom
> one can be provided.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
