[jira] [Commented] (KYLIN-2372) build with global dictionary error at step:Build Dimension Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816888#comment-15816888 ] kangkaisen commented on KYLIN-2372: --- Hi, [~readme_kylin]. I think you would better merge the KYLIN-2192 to your internal branch, otherwise please try to remove the UDID.tmp directory and build the cube again. > build with global dictionary error at step:Build Dimension Dictionary > - > > Key: KYLIN-2372 > URL: https://issues.apache.org/jira/browse/KYLIN-2372 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.6.0 > Environment: hadoop2.6.4,hive2.1.0,hbase1.2.3 >Reporter: readme_kylin > > when i build the second segment,error occurs.Here is the more info. > java.lang.RuntimeException: Failed to create dictionary on X.UDID > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325) > at > org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException > at > org.apache.kylin.dict.CachedTreeMap.writeValue(CachedTreeMap.java:240) > at org.apache.kylin.dict.CachedTreeMap.write(CachedTreeMap.java:374) > at > org.apache.kylin.dict.AppendTrieDictionary.flushIndex(AppendTrieDictionary.java:1043) > at > org.apache.kylin.dict.AppendTrieDictionary$Builder.build(AppendTrieDictionary.java:954) > at > org.apache.kylin.dict.GlobalDictionaryBuilder.build(GlobalDictionaryBuilder.java:82) > at > org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81) > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:323) > ... 14 more > result code:2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2372) build with global dictionary error at step:Build Dimension Dictionary
[
https://issues.apache.org/jira/browse/KYLIN-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814755#comment-15814755
]
readme_kylin commented on KYLIN-2372:
-
here is the kylin logs:
2017-01-10 11:08:00,175 INFO [pool-9-thread-9] dict.GlobalDictionaryBuilder:64
: GlobalDict
/dict/DATAMARKET_ADGAME.REALITY_RANKING_SHOW_KYLIN/UDID/9c79b834-e59e-45d1-9512-730dfbf120ad.dict
exist, append value
2017-01-10 11:08:00,176 INFO [pool-9-thread-9] dict.DictionaryManager:420 :
DictionaryManager(211773720) loading DictionaryInfo(loadDictObj:true) at
/dict/DATAMARKET_ADGAME.REALITY_RANKING_SHOW_KYLIN/UDID/9c79b834-e59e-45d1-9512-730dfbf120ad.dict
2017-01-10 11:08:00,658 INFO [pool-9-thread-9] mr.DFSFileTableReader:163 :
Auto detect delim to be null, will take THE-WHOLE-LINE as a single value, for
/kylin/kylin_metadata/kylin-5e40d906-cade-4a57-bd94-61145a793c08/rank_show_model2_cube/fact_distinct_columns/UDID
2017-01-10 11:08:01,217 INFO [pool-9-thread-9] dict.CachedTreeMap:178 : Load
cache by key (1) with value DictNode[root=, nodes=515160,
firstValue=/vuERrgbzdcp3ItPDm210sW/P]
2017-01-10 11:08:04,327 ERROR [pool-9-thread-9] dict.CachedTreeMap:239 : write
value into
/kylin/kylin_metadata/resources/GlobalDict/dict/DATAMARKET_ADGAME.REALITY_RANKING_SHOW_KYLIN/UDID.tmp/cached_
exception: java.lang.RuntimeException
at
org.apache.kylin.dict.AppendTrieDictionary$DictNode.build_writeNode(AppendTrieDictionary.java:605)
at
org.apache.kylin.dict.AppendTrieDictionary$DictNode.buildTrieBytes(AppendTrieDictionary.java:576)
at
org.apache.kylin.dict.AppendTrieDictionary$DictNode.write(AppendTrieDictionary.java:523)
at
org.apache.kylin.dict.CachedTreeMap.writeValue(CachedTreeMap.java:234)
at org.apache.kylin.dict.CachedTreeMap.write(CachedTreeMap.java:374)
at
org.apache.kylin.dict.AppendTrieDictionary.flushIndex(AppendTrieDictionary.java:1043)
at
org.apache.kylin.dict.AppendTrieDictionary$Builder.build(AppendTrieDictionary.java:954)
at
org.apache.kylin.dict.GlobalDictionaryBuilder.build(GlobalDictionaryBuilder.java:82)
at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81)
at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:323)
at
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41)
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-01-10 11:08:04,342 ERROR [pool-9-thread-9] common.HadoopShellExecutable:65
: error execute
HadoopShellExecutable{id=5e40d906-cade-4a57-bd94-61145a793c08-03, name=Build
Dimension Dictionary, state=RUNNING}
java.lang.RuntimeException: Failed to create dictionary on
DATAMARKET_ADGAME.REALITY_RANKING_SHOW_KYLIN.UDID
at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325)
at
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
java.lang.RuntimeException
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
at
[jira] [Commented] (KYLIN-2372) build with global dictionary error at step:Build Dimension Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814303#comment-15814303 ] XIE FAN commented on KYLIN-2372: I tried to reproduce this issue in sandbox with kylin sample data a few days ago. And I found that the lastest kylin master might not have this bug. > build with global dictionary error at step:Build Dimension Dictionary > - > > Key: KYLIN-2372 > URL: https://issues.apache.org/jira/browse/KYLIN-2372 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.6.0 > Environment: hadoop2.6.4,hive2.1.0,hbase1.2.3 >Reporter: readme_kylin > > when i build the second segment,error occurs.Here is the more info. > java.lang.RuntimeException: Failed to create dictionary on X.UDID > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325) > at > org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException > at > org.apache.kylin.dict.CachedTreeMap.writeValue(CachedTreeMap.java:240) > at org.apache.kylin.dict.CachedTreeMap.write(CachedTreeMap.java:374) > at > org.apache.kylin.dict.AppendTrieDictionary.flushIndex(AppendTrieDictionary.java:1043) > at > org.apache.kylin.dict.AppendTrieDictionary$Builder.build(AppendTrieDictionary.java:954) > at > org.apache.kylin.dict.GlobalDictionaryBuilder.build(GlobalDictionaryBuilder.java:82) > at > org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81) > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:323) > ... 14 more > result code:2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2372) build with global dictionary error at step:Build Dimension Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814295#comment-15814295 ] kangkaisen commented on KYLIN-2372: --- The error log should have the the cause of the throwable. please give me more error info. > build with global dictionary error at step:Build Dimension Dictionary > - > > Key: KYLIN-2372 > URL: https://issues.apache.org/jira/browse/KYLIN-2372 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.6.0 > Environment: hadoop2.6.4,hive2.1.0,hbase1.2.3 >Reporter: readme_kylin > > when i build the second segment,error occurs.Here is the more info. > java.lang.RuntimeException: Failed to create dictionary on X.UDID > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325) > at > org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException > at > org.apache.kylin.dict.CachedTreeMap.writeValue(CachedTreeMap.java:240) > at org.apache.kylin.dict.CachedTreeMap.write(CachedTreeMap.java:374) > at > org.apache.kylin.dict.AppendTrieDictionary.flushIndex(AppendTrieDictionary.java:1043) > at > org.apache.kylin.dict.AppendTrieDictionary$Builder.build(AppendTrieDictionary.java:954) > at > org.apache.kylin.dict.GlobalDictionaryBuilder.build(GlobalDictionaryBuilder.java:82) > at > org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81) > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:323) > ... 14 more > result code:2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2372) build with global dictionary error at step:Build Dimension Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813913#comment-15813913 ] Shaofeng SHI commented on KYLIN-2372: - [~sunyerui] hi yerui, do you have idea about this exception? It seems happended during building a global dictionary. thanks! > build with global dictionary error at step:Build Dimension Dictionary > - > > Key: KYLIN-2372 > URL: https://issues.apache.org/jira/browse/KYLIN-2372 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.6.0 > Environment: hadoop2.6.4,hive2.1.0,hbase1.2.3 >Reporter: readme_kylin > > when i build the second segment,error occur.Here is the more info. > java.lang.RuntimeException: Failed to create dictionary on X.UDID > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325) > at > org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException > at > org.apache.kylin.dict.CachedTreeMap.writeValue(CachedTreeMap.java:240) > at org.apache.kylin.dict.CachedTreeMap.write(CachedTreeMap.java:374) > at > org.apache.kylin.dict.AppendTrieDictionary.flushIndex(AppendTrieDictionary.java:1043) > at > org.apache.kylin.dict.AppendTrieDictionary$Builder.build(AppendTrieDictionary.java:954) > at > org.apache.kylin.dict.GlobalDictionaryBuilder.build(GlobalDictionaryBuilder.java:82) > at > org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81) > at > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:323) > ... 14 more > result code:2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
