[
https://issues.apache.org/jira/browse/SPARK-13907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lichenglin updated SPARK-13907:
-------------------------------
Description:
I tried to build a cube on a 100 million data set.
When I set 9 fields to build the cube with 10 cores.
It nearly coast me a whole day to finish the job.
At the same time, it generate almost 1”TB“ data in the "/tmp“ folder.
Could we refer to the ”fast cube“ algorithm in apache Kylin
To make the cube builder more quickly???
For example
group(A,B,C)'s result can be create by the groupdata(A,B)
was:
I tried to build a cube on a 100 million data set.
When I set 9 fields to build the cube with 10 cores.
It nearly coast me a whole day to finish the job.
At the same time, it generate almost 1”TB“ data in the "/tmp“ folder.
Could we refer to the ”fast cube“ algorithm in apache Kylin
To make the cube builder more quickly???
> Imporvement the cube with the Fast Cubing In apache Kylin
> ---------------------------------------------------------
>
> Key: SPARK-13907
> URL: https://issues.apache.org/jira/browse/SPARK-13907
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.6.1
> Reporter: lichenglin
>
> I tried to build a cube on a 100 million data set.
> When I set 9 fields to build the cube with 10 cores.
> It nearly coast me a whole day to finish the job.
> At the same time, it generate almost 1”TB“ data in the "/tmp“ folder.
> Could we refer to the ”fast cube“ algorithm in apache Kylin
> To make the cube builder more quickly???
> For example
> group(A,B,C)'s result can be create by the groupdata(A,B)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
