[
https://issues.apache.org/jira/browse/KYLIN-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285535#comment-16285535
]
Zhong Yanghong commented on KYLIN-2736:
---------------------------------------
Yes, this will cause CPU over-subscription. However, I think there should be a
way to enable this feature to speed up the process.
The thread number mainly depends on the how many cuboids are there whose
statistics needs to be calculated. The detailed code is as follows:
{code}
{
int unitNum =
cubeDesc.getConfig().getCuboidNumberPerStatisticsCalculator();
if (unitNum <= 0) {
logger.warn("config from getCuboidNumberPerStatisticsCalculator() "
+ unitNum + " is should larger than 0");
logger.info("Will use single thread for cuboid statistics
calculation");
return 1;
}
int calculatorNum = (cuboidNum - 1) / unitNum + 1;
if (cuboidNum >
cubeDesc.getConfig().getCuboidStatisticsCalculatorMaxNumber()) {
calculatorNum =
cubeDesc.getConfig().getCuboidStatisticsCalculatorMaxNumber();
}
return calculatorNum;
}
{code}
> Use multiple threads to calculate HyperLogLogPlusCounter in
> FactDistinctColumnsMapper
> -------------------------------------------------------------------------------------
>
> Key: KYLIN-2736
> URL: https://issues.apache.org/jira/browse/KYLIN-2736
> Project: Kylin
> Issue Type: Improvement
> Reporter: Zhong Yanghong
> Assignee: Zhong Yanghong
> Fix For: v2.3.0
>
> Attachments: APACHE-KYLIN-2736.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
