[
https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16249114#comment-16249114
]
Reid Chan commented on HBASE-19226:
-----------------------------------
Descriptions from HBASE-17130 shows that sometimes 1 reducer would bring
benefits, number is much more fine-grained than ratio, though to use which
depends on case.
bq. The number of regions in the same table also changes, and the reduce tasks
number also needs to be dynamically adjusted.
I got your mind, what about if the config(string) is between (0, 1), treats it
as ratio; else [1, +), treats it as a specified number. Judge it with regular
expression. WDYT.
> Limit the reduce tasks number of incremental load
> -------------------------------------------------
>
> Key: HBASE-19226
> URL: https://issues.apache.org/jira/browse/HBASE-19226
> Project: HBase
> Issue Type: Improvement
> Reporter: Yun Zhao
> Assignee: Yun Zhao
> Priority: Minor
> Attachments: HBASE-19226.master.001.patch,
> HBASE-19226.master.002.patch
>
>
> When using MapReduce job to perform an incremental load into a table,the
> number of reduce tasks is the current number of regions. If there are too
> many regions, will lead to network+disk I/O is too large, affecting the
> real-time request.
> Need to use a configuration to set a number or ratio?
> Limit running reduce tasks since
> [https://issues.apache.org/jira/browse/MAPREDUCE-5583], the old version can
> only be set reduce number.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
