wangxiaojing created KYLIN-4342:
-----------------------------------
Summary: Build Global Dict by MR/Hive New Version
Key: KYLIN-4342
URL: https://issues.apache.org/jira/browse/KYLIN-4342
Project: Kylin
Issue Type: Improvement
Affects Versions: Future
Reporter: wangxiaojing
Assignee: wangxiaojing
At present, there are two limitations in the implementation of global
dictionary through MR/Hive:
1. Limited by Hive order by global sorting on the shuffle stage, the memory and
build time becomes uncontrollable with data volume reaching billion level. We
have tested the base of 800 million level to configure 15g memory, and the
build time of build dictionary needs more than 10 hours;
2. Multi global dictionary columns is calculated serially.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
