[
https://issues.apache.org/jira/browse/KYLIN-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong reassigned KYLIN-2866:
-------------------------------------
Assignee: Wang Ken (was: Zhong Yanghong)
> Enlarge the reducer number for hyperloglog statistics calculation at step
> FactDistinctColumnsJob
> ------------------------------------------------------------------------------------------------
>
> Key: KYLIN-2866
> URL: https://issues.apache.org/jira/browse/KYLIN-2866
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Reporter: Zhong Yanghong
> Assignee: Wang Ken
> Priority: Major
> Fix For: v2.3.0
>
> Attachments: APACHE-KYLIN-2866-refined.patch, APACHE-KYLIN-2866.patch
>
>
> Currently only one reducer is assigned for hll stats calculation, which may
> become the bottleneck for slow down this step. Since the stats for different
> cuboids will not influence each other, it's better to divide the cuboid set
> into several and assign a reduce for each subset.
> The strategy of this patch is to assign 100 cuboids into a subset. And
> there's a upper limit of reducers for hll stats calculation. Currently it's
> 50.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
