[jira] [Closed] (KYLIN-1555) Prepare to release 1.5.1
[ https://issues.apache.org/jira/browse/KYLIN-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Li closed KYLIN-1555. -- Resolution: Fixed > Prepare to release 1.5.1 > > > Key: KYLIN-1555 > URL: https://issues.apache.org/jira/browse/KYLIN-1555 > Project: Kylin > Issue Type: Task > Components: General >Reporter: Dong Li >Assignee: Dong Li > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1077) Support Hive View as Lookup Table
[ https://issues.apache.org/jira/browse/KYLIN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxianbin updated KYLIN-1077: --- Attachment: KYLIN-1077-Support-Hive-View-as-Lookup-Table.patch hi shaofeng, this is the new patch, pls help to review! > Support Hive View as Lookup Table > - > > Key: KYLIN-1077 > URL: https://issues.apache.org/jira/browse/KYLIN-1077 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.1 >Reporter: Francesco Palmiotto >Assignee: Shaofeng SHI > Labels: newbie > Attachments: 0001-KYLIN-1077-Support-Hive-View-as-Lookup-Table.patch, > KYLIN-1077-Support-Hive-View-as-Lookup-Table.patch > > > If we use a view as a lookup table the cube building job fails when executing > the 3rd step (Build Dimension Dictionary) with this log: > java.io.IOException: java.lang.NullPointerException > at > org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:72) > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:202) > at > org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:166) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:52) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41) > at > org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:52) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:62) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > result code:2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1418) Memory hungry cube should select LAYER and INMEM cubing smartly
[ https://issues.apache.org/jira/browse/KYLIN-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-1418: Fix Version/s: (was: v1.5.0) v1.5.2 Issue Type: Improvement (was: Bug) > Memory hungry cube should select LAYER and INMEM cubing smartly > --- > > Key: KYLIN-1418 > URL: https://issues.apache.org/jira/browse/KYLIN-1418 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.5.0 >Reporter: liyang >Assignee: Shaofeng SHI > Labels: newbie > Fix For: v1.5.2 > > > At the time of writing, all memory hungry cube goes to LAYER cubing, which is > too simple. The concern of memory hungry measure mainly comes from CI env, > where physical memory is very limited. In real production env, memory hungry > measures are usually not a problem for INMEM cubing. > Suggest check the sum of measure memory footprint instead of the existence of > memory hungry measure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (KYLIN-1418) Memory hungry cube should select LAYER and INMEM cubing smartly
[ https://issues.apache.org/jira/browse/KYLIN-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reopened KYLIN-1418: - Assignee: Shaofeng SHI (was: liyang) Just found this wasn't fixed; Reopen and plan it in the next release. > Memory hungry cube should select LAYER and INMEM cubing smartly > --- > > Key: KYLIN-1418 > URL: https://issues.apache.org/jira/browse/KYLIN-1418 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.0 >Reporter: liyang >Assignee: Shaofeng SHI > Labels: newbie > Fix For: v1.5.0 > > > At the time of writing, all memory hungry cube goes to LAYER cubing, which is > too simple. The concern of memory hungry measure mainly comes from CI env, > where physical memory is very limited. In real production env, memory hungry > measures are usually not a problem for INMEM cubing. > Suggest check the sum of measure memory footprint instead of the existence of > memory hungry measure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1077) Support Hive View as Lookup Table
[ https://issues.apache.org/jira/browse/KYLIN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244223#comment-15244223 ] Shaofeng SHI commented on KYLIN-1077: - jobid is better as its length is fixed; you can make the decision :) > Support Hive View as Lookup Table > - > > Key: KYLIN-1077 > URL: https://issues.apache.org/jira/browse/KYLIN-1077 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.1 >Reporter: Francesco Palmiotto >Assignee: Shaofeng SHI > Labels: newbie > Attachments: 0001-KYLIN-1077-Support-Hive-View-as-Lookup-Table.patch > > > If we use a view as a lookup table the cube building job fails when executing > the 3rd step (Build Dimension Dictionary) with this log: > java.io.IOException: java.lang.NullPointerException > at > org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:72) > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:202) > at > org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:166) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:52) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41) > at > org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:52) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:62) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > result code:2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1077) Support Hive View as Lookup Table
[ https://issues.apache.org/jira/browse/KYLIN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244217#comment-15244217 ] wangxianbin commented on KYLIN-1077: hi shaofeng! after my investigation, the limitation of hive table name is 128 character, which means FlatHiveTable name may face the same problem. for dealing with the concurrency problem of building the cubes that use same hive view as lookup table and saving the name length of the hive view intermediate table, how do you think we use jobid intead of segment/cube_name as part of table name? any better idea? > Support Hive View as Lookup Table > - > > Key: KYLIN-1077 > URL: https://issues.apache.org/jira/browse/KYLIN-1077 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.1 >Reporter: Francesco Palmiotto >Assignee: Shaofeng SHI > Labels: newbie > Attachments: 0001-KYLIN-1077-Support-Hive-View-as-Lookup-Table.patch > > > If we use a view as a lookup table the cube building job fails when executing > the 3rd step (Build Dimension Dictionary) with this log: > java.io.IOException: java.lang.NullPointerException > at > org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:72) > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:202) > at > org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:166) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:52) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41) > at > org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:52) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:62) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > result code:2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
