[
https://issues.apache.org/jira/browse/KYLIN-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672686#comment-15672686
]
kangkaisen commented on KYLIN-2135:
-----------------------------------
Hi, [~xiefan46]. We only enlarge the reducer number for ShardByColumns and
GlobalDictionaryColumns because of these column have ultra high cardinality. We
don't enlarge the reducer number for normal columns and their reducer number is
one.
> Enlarge FactDistinctColumns reducer number
> ------------------------------------------
>
> Key: KYLIN-2135
> URL: https://issues.apache.org/jira/browse/KYLIN-2135
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Affects Versions: v1.5.4.1
> Reporter: kangkaisen
> Assignee: kangkaisen
> Fix For: v1.6.1
>
> Attachments: KYLIN-2135.patch, new.png, old.png
>
>
> When the hive table has billions of rows and use global dictionary for
> precise count distinct measures, the {{Extract Fact Table Distinct Columns}}
> job will run o long time.
> So we could use more reducer to deal with the one column.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
