[jira] [Commented] (KYLIN-1983) Release 1.5.4
[ https://issues.apache.org/jira/browse/KYLIN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15448262#comment-15448262 ] zhaotianshuo commented on KYLIN-1983: - finally!! can't stop waiting~ > Release 1.5.4 > - > > Key: KYLIN-1983 > URL: https://issues.apache.org/jira/browse/KYLIN-1983 > Project: Kylin > Issue Type: Task > Components: General >Reporter: Shaofeng SHI > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1984) Don't use compression in default configuration
Shaofeng SHI created KYLIN-1984: --- Summary: Don't use compression in default configuration Key: KYLIN-1984 URL: https://issues.apache.org/jira/browse/KYLIN-1984 Project: Kylin Issue Type: Improvement Components: Environment Reporter: Shaofeng SHI Assignee: hongbin ma Today in Kylin default configuration, it uses snappy compression for Hadoop and HBase; while many new users don't have hadoop native packages installed, they might be blocked there. To improve the experience for new users, Kylin's default configuration should disable compression, and add a document introduce how to enable compression; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1983) Release 1.5.4
[ https://issues.apache.org/jira/browse/KYLIN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15448549#comment-15448549 ] Shaofeng SHI commented on KYLIN-1983: - hi tianshuo, please aware that the release may take one or two weeks; > Release 1.5.4 > - > > Key: KYLIN-1983 > URL: https://issues.apache.org/jira/browse/KYLIN-1983 > Project: Kylin > Issue Type: Task > Components: General >Reporter: Shaofeng SHI > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1983) Release 1.5.4
[ https://issues.apache.org/jira/browse/KYLIN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaotianshuo updated KYLIN-1983: Attachment: text.html all right, I can take this 来自 魅族 PRO6 原始邮件 发件人："Shaofeng SHI (JIRA)" 时间：周二 8月30日 17:24 收件人：赵天烁 主题：[jira] [Commented] (KYLIN-1983) Release 1.5.4 [ https://issues.apache.org/jira/browse/KYLIN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15448549#comment-15448549 ] Shaofeng SHI commented on KYLIN-1983: - hi tianshuo, please aware that the release may take one or two weeks; -- This message was sent by Atlassian JIRA (v6.3.4#6332) > Release 1.5.4 > - > > Key: KYLIN-1983 > URL: https://issues.apache.org/jira/browse/KYLIN-1983 > Project: Kylin > Issue Type: Task > Components: General >Reporter: Shaofeng SHI > Attachments: text.html > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
zhengdong created KYLIN-1985: Summary: SnapshotTable should only keep the colunms described in tableDesc Key: KYLIN-1985 URL: https://issues.apache.org/jira/browse/KYLIN-1985 Project: Kylin Issue Type: Bug Affects Versions: v1.5.3 Reporter: zhengdong -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we encounter a > SnapshotTable should only keep the colunms described in tableDesc > - > > Key: KYLIN-1985 > URL: https://issues.apache.org/jira/browse/KYLIN-1985 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.3 >Reporter: zhengdong > > we encounter a -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that got java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more was:we encounter a > SnapshotTable should only keep the colunms described in tableDesc > - > > Key: KYLIN-1985 > URL: https://issues.apache.org/jira/browse/KYLIN-1985 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.3 >Reporter: zhengdong > > we suffered from a strange problem that got > java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, > exception stack like this : > java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME > from snapshot > /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot > at > org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) > at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) > at org.apache.kylin.dict.lookup.LookupTable.init(Loo
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more and a simple exception when queried by a lookup table dimension ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when execute sql at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) at org.springframework.web.servlet.mvc.method.Abstra
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more and a simple exception when queried by a lookup table dimension ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when execute sql at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) at org.springframework.web.servlet.mvc.method.Abstra
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more and a simple exception when queried by a lookup table dimension ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when execute sql at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) at org.springframework.web.servlet.mvc.method.Abstra
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more and a simple exception when queried by a lookup table dimension ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when execute sql at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) Though the exception message, we found that one lookup table had been changed in hive (add colunms) and not been synchronized with kylin. However, the cause of this problem is too subtle and not easily found. As for the implementation of was: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfB
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more and a simple exception when queried by a lookup table dimension ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when execute sql at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) Though the exception message, we found that one lookup table had been changed in hive (add colunms) and not been synchronized with kylin. However, the cause of this problem is too subtle and not easily found. As for the implementation of SnapshotTable, only the colunms was: we suffered from a strange problem tha
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more and a simple exception when queried by a lookup table dimension ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when execute sql at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) Though the exception message, we found that one lookup table had been changed in hive (add colunms) and not been synchronized with kylin. However, the cause of this problem is too subtle and not easily found. And only checking 'row.length <= maxIndex' to detect 'Bad hive table row ' is not enough. As for the implem
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more and a simple exception when queried by a lookup table dimension ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when execute sql at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) Though the exception message, we found that one lookup table had been changed in hive (add colunms) and not been synchronized with kylin. However, the cause of this problem is too subtle and not easily found. And only checking 'row.length <= maxIndex' in takeSnapshot method to detect 'Bad hive table row ' is not enoug
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Attachment: KYLIN-1985.patch > SnapshotTable should only keep the colunms described in tableDesc > - > > Key: KYLIN-1985 > URL: https://issues.apache.org/jira/browse/KYLIN-1985 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.3 >Reporter: zhengdong > Attachments: KYLIN-1985.patch > > > we suffered from a strange problem that we got a > java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, > exception stack like this : > java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME > from snapshot > /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot > at > org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) > at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) > at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) > at > org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) > at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) > at > org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) > at > org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) > ... 13 more > and a simple exception when queried by a lookup table dimension > > ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when > execute sql > at org.apache.calcite.avatica.Helper.createException(Helper.java:56) > at org.apache.calcite.avatica.Helper.createException(Helper.java:41) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) > at > org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) > at > org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) > at > org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) > at > org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) > at > org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) > at > org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() > at > org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) > at > org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) > Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) > at > org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) > at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) > at > org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) > at > org.apache.kyl
[jira] [Created] (KYLIN-1986) CubeMigrationCLI: make global dictionary unique
kangkaisen created KYLIN-1986: - Summary: CubeMigrationCLI: make global dictionary unique Key: KYLIN-1986 URL: https://issues.apache.org/jira/browse/KYLIN-1986 Project: Kylin Issue Type: Bug Components: Tools, Build and Test Affects Versions: v1.5.3 Reporter: kangkaisen Assignee: kangkaisen The global dictionary is shared by all segments of one cube, so when we migrate the global dictionary, we should copy the global dictionary file only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1986) CubeMigrationCLI: make global dictionary unique
[ https://issues.apache.org/jira/browse/KYLIN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kangkaisen updated KYLIN-1986: -- Attachment: KYLIN-1986.patch This is the patch > CubeMigrationCLI: make global dictionary unique > --- > > Key: KYLIN-1986 > URL: https://issues.apache.org/jira/browse/KYLIN-1986 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v1.5.3 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-1986.patch > > > The global dictionary is shared by all segments of one cube, so when we > migrate the global dictionary, we should copy the global dictionary file only > once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the columns described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Summary: SnapshotTable should only keep the columns described in tableDesc (was: SnapshotTable should only keep the colunms described in tableDesc) > SnapshotTable should only keep the columns described in tableDesc > - > > Key: KYLIN-1985 > URL: https://issues.apache.org/jira/browse/KYLIN-1985 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.3 >Reporter: zhengdong > Attachments: KYLIN-1985.patch > > > we suffered from a strange problem that we got a > java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, > exception stack like this : > java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME > from snapshot > /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot > at > org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) > at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) > at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) > at > org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) > at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) > at > org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) > at > org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) > ... 13 more > and a simple exception when queried by a lookup table dimension > > ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when > execute sql > at org.apache.calcite.avatica.Helper.createException(Helper.java:56) > at org.apache.calcite.avatica.Helper.createException(Helper.java:41) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) > at > org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) > at > org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) > at > org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) > at > org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) > at > org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) > at > org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() > at > org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) > at > org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) > Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) > at > org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) > at > org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) > at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) >
[jira] [Updated] (KYLIN-1985) SnapshotTable should only keep the colunms described in tableDesc
[ https://issues.apache.org/jira/browse/KYLIN-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengdong updated KYLIN-1985: - Description: we suffered from a strange problem that we got a java.lang.ArrayIndexOutOfBoundsException when build of refresh a cube, exception stack like this : java.lang.IllegalStateException: Failed to load lookup table DIM_TABLE_NAME from snapshot /table_snapshot/dim_table_name/5a78a522-6f85-4650-b47d-6a5f5806b7f7.snapshot at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:621) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) ... 13 more and a simple exception when queried by a lookup table dimension ERROR [http-bio-7070-exec-7] controller.QueryController:209 : Exception when execute sql at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:366) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:278) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:121) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$315e2079.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94) Caused by: java.lang.ArrayIndexOutOfBoundsException: 19 at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:85) at org.apache.kylin.dict.lookup.LookupStringTable.convertRow(LookupStringTable.java:34) at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:76) at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) at org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) at org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) Though the exception message, we found that one lookup table had been changed in hive (add columns) and not been synchronized with kylin. However, the cause of this problem is too subtle and not easily found. As for SnapshotTable, only checking 'row.length <= maxIndex' in takeSnapshot method to detect 'Bad hive t
[jira] [Commented] (KYLIN-1984) Don't use compression in default configuration
[ https://issues.apache.org/jira/browse/KYLIN-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15448985#comment-15448985 ] Billy(Yiming) Liu commented on KYLIN-1984: -- Hi [~Shaofengshi], I could work on this task. > Don't use compression in default configuration > -- > > Key: KYLIN-1984 > URL: https://issues.apache.org/jira/browse/KYLIN-1984 > Project: Kylin > Issue Type: Improvement > Components: Environment >Reporter: Shaofeng SHI >Assignee: hongbin ma > > Today in Kylin default configuration, it uses snappy compression for Hadoop > and HBase; while many new users don't have hadoop native packages installed, > they might be blocked there. > To improve the experience for new users, Kylin's default configuration should > disable compression, and add a document introduce how to enable compression; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1982) CubeMigrationCLI: associate model with project
[ https://issues.apache.org/jira/browse/KYLIN-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-1982: Fix Version/s: v1.5.4 Merged, thanks kaisen! > CubeMigrationCLI: associate model with project > --- > > Key: KYLIN-1982 > URL: https://issues.apache.org/jira/browse/KYLIN-1982 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v1.5.3 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.5.4 > > Attachments: KYLIN-1982.patch > > > In the current `CubeMigrationCLI`, when we migrated the cube, the model > metadata has migrated indeed, but the model hasn't associated with the > project. > So, if we get model via `getModels` in `ModelController` with "modelName" and > "projectName", we will get null. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1978) kylin.sh compatible issue on Ubuntu
[
https://issues.apache.org/jira/browse/KYLIN-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-1978:
Fix Version/s: v1.5.4
> kylin.sh compatible issue on Ubuntu
> ---
>
> Key: KYLIN-1978
> URL: https://issues.apache.org/jira/browse/KYLIN-1978
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v1.5.3
>Reporter: Shaofeng SHI
>Assignee: hongbin ma
> Fix For: v1.5.4
>
>
> Reported by Marcelo(marcelo.n...@quantium.com.au) in the mailing list:
> DISTRIB_ID=Ubuntu
> DISTRIB_RELEASE=14.04
> DISTRIB_CODENAME=trusty
> DISTRIB_DESCRIPTION="Ubuntu 14.04.4 LTS"
> NAME="Ubuntu"
> VERSION="14.04.4 LTS, Trusty Tahr"
> ID=ubuntu
> ID_LIKE=debian
> PRETTY_NAME="Ubuntu 14.04.4 LTS"
> VERSION_ID="14.04"
> HOME_URL="http://www.ubuntu.com/";
> SUPPORT_URL="http://help.ubuntu.com/";
> BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/";
> mapr@qtausc-vpcsdev04:~/kylin/apache-kylin-1.5.3-HBase1.x-bin/bin$ ./kylin.sh
> start
> *KYLIN_HOME is set to /home/mapr/kylin/apache-kylin-1.5.3-HBase1.x-bin*
> cat: invalid option -- '1'
> Try 'cat --help' for more information.
> -mkdir: Not enough arguments: expected 1 but got 0
> Usage: hadoop fs [generic options] -mkdir [-p] ...
> failed to create , Please make sure the user has right to access
> That is what is happening when I try to start kylin.
> I traced the error and the first one come from get-properties.sh at these
> line
> for i in `cat ${KYLIN_HOME}/conf/kylin.properties | grep -w "^$1" | grep -v
> '^#' | awk -F= '{ n = index($0,"="); print substr($0,n+1)}' | cut -c 1-`
> and as you can see kylin home is set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1980) TopN issue: result not consistent with hive
[ https://issues.apache.org/jira/browse/KYLIN-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15449363#comment-15449363 ] Shaofeng SHI commented on KYLIN-1980: - ok, can you try to change the precision to top100 and then rebuild/query again? please aware that the TopN is approximate result, whose precision is related with a couple of factors, please check this blog if you havn't: https://kylin.apache.org/blog/2016/03/19/approximate-topn-measure/ > TopN issue: result not consistent with hive > --- > > Key: KYLIN-1980 > URL: https://issues.apache.org/jira/browse/KYLIN-1980 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Shaofeng SHI > Attachments: hive_result.png, kylin_result.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1980) TopN issue: result not consistent with hive
[ https://issues.apache.org/jira/browse/KYLIN-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15450449#comment-15450449 ] Zhong Yanghong commented on KYLIN-1980: --- Hi [~Shaofengshi], I have already checked the blog. Even if topN measure is an approximate one, how could be result(kylin with topN) > result(hive)? > TopN issue: result not consistent with hive > --- > > Key: KYLIN-1980 > URL: https://issues.apache.org/jira/browse/KYLIN-1980 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Shaofeng SHI > Attachments: hive_result.png, kylin_result.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1980) TopN issue: result not consistent with hive
[ https://issues.apache.org/jira/browse/KYLIN-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15450617#comment-15450617 ] Shaofeng SHI commented on KYLIN-1980: - Yes it is possible, you can read through the TopNCounter.merge() method. > TopN issue: result not consistent with hive > --- > > Key: KYLIN-1980 > URL: https://issues.apache.org/jira/browse/KYLIN-1980 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Shaofeng SHI > Attachments: hive_result.png, kylin_result.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1986) CubeMigrationCLI: make global dictionary unique
[ https://issues.apache.org/jira/browse/KYLIN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451181#comment-15451181 ] Shaofeng SHI commented on KYLIN-1986: - Merged into master, thanks kaisen! > CubeMigrationCLI: make global dictionary unique > --- > > Key: KYLIN-1986 > URL: https://issues.apache.org/jira/browse/KYLIN-1986 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v1.5.3 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.5.4 > > Attachments: KYLIN-1986.patch > > > The global dictionary is shared by all segments of one cube, so when we > migrate the global dictionary, we should copy the global dictionary file only > once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1986) CubeMigrationCLI: make global dictionary unique
[ https://issues.apache.org/jira/browse/KYLIN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-1986: Fix Version/s: v1.5.4 > CubeMigrationCLI: make global dictionary unique > --- > > Key: KYLIN-1986 > URL: https://issues.apache.org/jira/browse/KYLIN-1986 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v1.5.3 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.5.4 > > Attachments: KYLIN-1986.patch > > > The global dictionary is shared by all segments of one cube, so when we > migrate the global dictionary, we should copy the global dictionary file only > once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1849) Search cube by name in Web UI
[ https://issues.apache.org/jira/browse/KYLIN-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-1849: Fix Version/s: v1.5.4 > Search cube by name in Web UI > - > > Key: KYLIN-1849 > URL: https://issues.apache.org/jira/browse/KYLIN-1849 > Project: Kylin > Issue Type: New Feature > Components: Web >Affects Versions: v1.5.2 >Reporter: Dayue Gao >Assignee: kangkaisen > Fix For: v1.5.4 > > Attachments: KYLIN-1849.patch > > > In order to work with dozens of cubes, could we add a search box at "Model" > page? Just like the one at "Monitor" page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-1986) CubeMigrationCLI: make global dictionary unique
[ https://issues.apache.org/jira/browse/KYLIN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-1986. - Resolution: Fixed > CubeMigrationCLI: make global dictionary unique > --- > > Key: KYLIN-1986 > URL: https://issues.apache.org/jira/browse/KYLIN-1986 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v1.5.3 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.5.4 > > Attachments: KYLIN-1986.patch > > > The global dictionary is shared by all segments of one cube, so when we > migrate the global dictionary, we should copy the global dictionary file only > once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1915) no counters for job
[
https://issues.apache.org/jira/browse/KYLIN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI closed KYLIN-1915.
---
Resolution: Invalid
> no counters for job
> -
>
> Key: KYLIN-1915
> URL: https://issues.apache.org/jira/browse/KYLIN-1915
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.2.1
> Environment: hadoop-2.4.1 hbase-1.1.5 apache-hive-1.2.1-bin
>Reporter: wanglaichao
>
> when i run a simple table , i don't find excetion ,but it runs failed.my
> hadoop version is 2.4.1 .who can help me ,thanks.
> this is our logs:
> 2016-07-22 17:41:09,588 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:72 : Hadoop job job_1468901022191_96470 status :
> {"app":{"id":"application_1468901022191_96470","user":"hadoop","name":"Kylin_Fact_Distinct_Columns_test_cube_0722_Step","queue":"root.hadoop","state":"FINISHED","finalStatus":"FAILED","progress":100.0,"trackingUI":"History","trackingUrl":"http://namenode171:8088/proxy/application_1468901022191_96470/jobhistory/job/job_1468901022191_96470","diagnostics":"","clusterId":1468901022191,"applicationType":"MAPREDUCE","applicationTags":"","startedTime":1469180339135,"finishedTime":1469180468493,"elapsedTime":129358,"amContainerLogs":"http://datanode94:8042/node/containerlogs/container_1468901022191_96470_01_01/hadoop","amHostHttpAddress":"datanode94:8042","allocatedMB":2048,"allocatedVCores":1,"runningContainers":1}}
> 2016-07-22 17:41:09,588 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:59 : State of Hadoop job:
> job_1468901022191_96470:FINISHED-FAILED
> 2016-07-22 17:41:09,592 DEBUG [pool-5-thread-25] hbase.HBaseResourceStore:262
> : Update row /execute_output/2eee7f50-93ac-4c8b-9a41-bbe7c6846798-01 from
> oldTs: 1469180459579, to newTs: 1469180469589, operation result: true
> 2016-07-22 17:41:18,347 INFO [pool-4-thread-1]
> threadpool.DefaultScheduler:106 : Job Fetcher: 1 running, 1 actual running, 0
> ready, 21 others
> 2016-07-22 17:41:41,704 WARN [pool-5-thread-25] common.HadoopCmdOutput:89 :
> no counters for job job_1468901022191_96470
> 2016-07-22 17:41:41,710 DEBUG [pool-5-thread-25] hbase.HBaseResourceStore:262
> : Update row /execute_output/2eee7f50-93ac-4c8b-9a41-bbe7c6846798-01 from
> oldTs: 1469180469589, to newTs: 1469180501706, operation result: true
> 2016-07-22 17:41:41,710 INFO [pool-5-thread-25]
> execution.AbstractExecutable:109 : Retry 1
> 2016-07-22 17:41:41,762 INFO [pool-5-thread-25]
> execution.AbstractExecutable:107 : mr_job_id:job_1468901022191_96470 resumed
> 2016-07-22 17:41:41,767 WARN [pool-5-thread-25]
> httpclient.HttpMethodBase:682 : Going to buffer response body of large or
> unknown size. Using getResponseBodyAsStream instead is recommended.
> 2016-07-22 17:41:41,767 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:209 : Job job_1468901022191_96470 get status check
> result.
> 2016-07-22 17:41:41,767 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:72 : Hadoop job job_1468901022191_96470 status :
> {"app":{"id":"application_1468901022191_96470","user":"hadoop","name":"Kylin_Fact_Distinct_Columns_test_cube_0722_Step","queue":"root.hadoop","state":"FINISHED","finalStatus":"FAILED","progress":100.0,"trackingUI":"History","trackingUrl":"http://namenode171:8088/proxy/application_1468901022191_96470/jobhistory/job/job_1468901022191_96470","diagnostics":"","clusterId":1468901022191,"applicationType":"MAPREDUCE","applicationTags":"","startedTime":1469180339135,"finishedTime":1469180468493,"elapsedTime":129358,"amContainerLogs":"http://datanode94:8042/node/containerlogs/container_1468901022191_96470_01_01/hadoop","amHostHttpAddress":"datanode94:8042","allocatedMB":2048,"allocatedVCores":1,"runningContainers":1}}
> 2016-07-22 17:41:41,767 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:59 : State of Hadoop job:
> job_1468901022191_96470:FINISHED-FAILED
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1973) java.lang.NegativeArraySizeException when Build Dimension Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451192#comment-15451192 ] Shaofeng SHI commented on KYLIN-1973: - Is this the same issue as KYLIN-1985? > java.lang.NegativeArraySizeException when Build Dimension Dictionary > > > Key: KYLIN-1973 > URL: https://issues.apache.org/jira/browse/KYLIN-1973 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.3 >Reporter: zhengdong > > exception when Build Dimension Dictionary: > java.lang.NegativeArraySizeException > at > org.apache.kylin.dict.TrieDictionary.getValueFromIdImpl(TrieDictionary.java:274) > at > org.apache.kylin.common.util.Dictionary.getValueFromId(Dictionary.java:130) > at > org.apache.kylin.dict.lookup.SnapshotTable$1.getRow(SnapshotTable.java:138) > at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67) > at > org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:79) > at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:55) > at > org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:65) > at > org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:619) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:61) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:56) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:127) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > result code:2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1916) no counters for job
[ https://issues.apache.org/jira/browse/KYLIN-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451199#comment-15451199 ] Shaofeng SHI commented on KYLIN-1916: - Duplicated with KYLIN-1915 > no counters for job > - > > Key: KYLIN-1916 > URL: https://issues.apache.org/jira/browse/KYLIN-1916 > Project: Kylin > Issue Type: Bug > Components: Web >Reporter: wanglaichao >Assignee: Zhong,Jason > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1915) no counters for job
[
https://issues.apache.org/jira/browse/KYLIN-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451196#comment-15451196
]
Shaofeng SHI commented on KYLIN-1915:
-
Hi Laichao, you may need do some search; Besides, JIRA is for task/bug
tracking; for Q/A, please leverage the user/dev mailing list, thanks.
> no counters for job
> -
>
> Key: KYLIN-1915
> URL: https://issues.apache.org/jira/browse/KYLIN-1915
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.2.1
> Environment: hadoop-2.4.1 hbase-1.1.5 apache-hive-1.2.1-bin
>Reporter: wanglaichao
>
> when i run a simple table , i don't find excetion ,but it runs failed.my
> hadoop version is 2.4.1 .who can help me ,thanks.
> this is our logs:
> 2016-07-22 17:41:09,588 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:72 : Hadoop job job_1468901022191_96470 status :
> {"app":{"id":"application_1468901022191_96470","user":"hadoop","name":"Kylin_Fact_Distinct_Columns_test_cube_0722_Step","queue":"root.hadoop","state":"FINISHED","finalStatus":"FAILED","progress":100.0,"trackingUI":"History","trackingUrl":"http://namenode171:8088/proxy/application_1468901022191_96470/jobhistory/job/job_1468901022191_96470","diagnostics":"","clusterId":1468901022191,"applicationType":"MAPREDUCE","applicationTags":"","startedTime":1469180339135,"finishedTime":1469180468493,"elapsedTime":129358,"amContainerLogs":"http://datanode94:8042/node/containerlogs/container_1468901022191_96470_01_01/hadoop","amHostHttpAddress":"datanode94:8042","allocatedMB":2048,"allocatedVCores":1,"runningContainers":1}}
> 2016-07-22 17:41:09,588 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:59 : State of Hadoop job:
> job_1468901022191_96470:FINISHED-FAILED
> 2016-07-22 17:41:09,592 DEBUG [pool-5-thread-25] hbase.HBaseResourceStore:262
> : Update row /execute_output/2eee7f50-93ac-4c8b-9a41-bbe7c6846798-01 from
> oldTs: 1469180459579, to newTs: 1469180469589, operation result: true
> 2016-07-22 17:41:18,347 INFO [pool-4-thread-1]
> threadpool.DefaultScheduler:106 : Job Fetcher: 1 running, 1 actual running, 0
> ready, 21 others
> 2016-07-22 17:41:41,704 WARN [pool-5-thread-25] common.HadoopCmdOutput:89 :
> no counters for job job_1468901022191_96470
> 2016-07-22 17:41:41,710 DEBUG [pool-5-thread-25] hbase.HBaseResourceStore:262
> : Update row /execute_output/2eee7f50-93ac-4c8b-9a41-bbe7c6846798-01 from
> oldTs: 1469180469589, to newTs: 1469180501706, operation result: true
> 2016-07-22 17:41:41,710 INFO [pool-5-thread-25]
> execution.AbstractExecutable:109 : Retry 1
> 2016-07-22 17:41:41,762 INFO [pool-5-thread-25]
> execution.AbstractExecutable:107 : mr_job_id:job_1468901022191_96470 resumed
> 2016-07-22 17:41:41,767 WARN [pool-5-thread-25]
> httpclient.HttpMethodBase:682 : Going to buffer response body of large or
> unknown size. Using getResponseBodyAsStream instead is recommended.
> 2016-07-22 17:41:41,767 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:209 : Job job_1468901022191_96470 get status check
> result.
> 2016-07-22 17:41:41,767 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:72 : Hadoop job job_1468901022191_96470 status :
> {"app":{"id":"application_1468901022191_96470","user":"hadoop","name":"Kylin_Fact_Distinct_Columns_test_cube_0722_Step","queue":"root.hadoop","state":"FINISHED","finalStatus":"FAILED","progress":100.0,"trackingUI":"History","trackingUrl":"http://namenode171:8088/proxy/application_1468901022191_96470/jobhistory/job/job_1468901022191_96470","diagnostics":"","clusterId":1468901022191,"applicationType":"MAPREDUCE","applicationTags":"","startedTime":1469180339135,"finishedTime":1469180468493,"elapsedTime":129358,"amContainerLogs":"http://datanode94:8042/node/containerlogs/container_1468901022191_96470_01_01/hadoop","amHostHttpAddress":"datanode94:8042","allocatedMB":2048,"allocatedVCores":1,"runningContainers":1}}
> 2016-07-22 17:41:41,767 DEBUG [pool-5-thread-25]
> common.HadoopStatusChecker:59 : State of Hadoop job:
> job_1468901022191_96470:FINISHED-FAILED
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1951) cube sergement start time don't match partition start time
[ https://issues.apache.org/jira/browse/KYLIN-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451205#comment-15451205 ] Shaofeng SHI commented on KYLIN-1951: - the "kylin.rest.timezone" only impact on the time display on Web GUI; it doesn't affect the cube build start/end time; When trigger the cube build with API, please use the epoch time, which is GMT0; I guess you'were using Linux system time, which may has the timezone information, so there is 8 hours difference. > cube sergement start time don't match partition start time > -- > > Key: KYLIN-1951 > URL: https://issues.apache.org/jira/browse/KYLIN-1951 > Project: Kylin > Issue Type: Bug >Reporter: 一岁时很拽 >Assignee: Zhong,Jason > > for streaming cube build, > i set kylin.rest.timezone=GMT+8,but cube sergement start time don't match > partition start time, A difference of 8 hours,why? > http://apache-kylin.74782.x6.nabble.com/cube-sergement-start-time-don-t-match-partition-start-time-td5538.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1889) Didn't deal with the failure of renaming folder in hdfs when running the tool CubeMigrationCLI
[ https://issues.apache.org/jira/browse/KYLIN-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451209#comment-15451209 ] Shaofeng SHI commented on KYLIN-1889: - has this been merged? what's the fixed version? > Didn't deal with the failure of renaming folder in hdfs when running the tool > CubeMigrationCLI > -- > > Key: KYLIN-1889 > URL: https://issues.apache.org/jira/browse/KYLIN-1889 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: > retry_when_fail_to_rename_hdfs_folder_and_throw_exception_if_necessary.patch > > > In some cases, the invoking of the "rename" method of FileSystem may fail and > return false without throwing any Exception. Currently, such case is regarded > as renaming successfully, which is not true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1887) Kylin Web UI ->System->Enable/Disable Cache Button is not consistent with actual config
[ https://issues.apache.org/jira/browse/KYLIN-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1887. --- Resolution: Won't Fix > Kylin Web UI ->System->Enable/Disable Cache Button is not consistent with > actual config > --- > > Key: KYLIN-1887 > URL: https://issues.apache.org/jira/browse/KYLIN-1887 > Project: Kylin > Issue Type: Bug > Components: Client - CLI >Reporter: Wang Cheng >Assignee: Zhong,Jason >Priority: Minor > > When clicking "disable cache" button( it's title will become "enable cache"), > it works. But if kylin restarts, the actual value of > "kylin.query.cache.enabled" will reset to "true" while button state is still > show "enable cache" which means its value is "false". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1889) Didn't deal with the failure of renaming folder in hdfs when running the tool CubeMigrationCLI
[ https://issues.apache.org/jira/browse/KYLIN-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451217#comment-15451217 ] Zhong Yanghong commented on KYLIN-1889: --- This has been merged to master branch. > Didn't deal with the failure of renaming folder in hdfs when running the tool > CubeMigrationCLI > -- > > Key: KYLIN-1889 > URL: https://issues.apache.org/jira/browse/KYLIN-1889 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: > retry_when_fail_to_rename_hdfs_folder_and_throw_exception_if_necessary.patch > > > In some cases, the invoking of the "rename" method of FileSystem may fail and > return false without throwing any Exception. Currently, such case is regarded > as renaming successfully, which is not true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1883) Consensus Problem when running the tool, MetadataCleanupJob
[ https://issues.apache.org/jira/browse/KYLIN-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451221#comment-15451221 ] Shaofeng SHI commented on KYLIN-1883: - what's the fixed version? could you please properly set such field when ever fix an issue? Otherwise user won't know which version has the fix, and it will be missed from the release notes. thanks. > Consensus Problem when running the tool, MetadataCleanupJob > --- > > Key: KYLIN-1883 > URL: https://issues.apache.org/jira/browse/KYLIN-1883 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: > better_solution_for_consensus_issue_of_MetadataCleanupJob.patch > > > When do the cleanup, current strategy is as follows: > 1. firstly create an referenceSet > 2. then add items not belonging to the referenceSet to the toDeleteSet > 3. finally delete those items in the toDeleteSet > Consensus issue will occur since we cannot make sure that all of the items in > toDeleteSet are not referenced in case that referenceSet changes during the > process. > For example, before the cleanup, SEGMENT_A is deleted and leave a DICT_A > created at the building step. Then the referenceSet will not include DICT_A. > After creating the reference set, SEGMENT_B is starting to build. Since > DICT_A still exists, it can still be referenced by SEGMENT_B. Then DICT_A > will still be included in the toDeleteSet and will be deleted later. Finally > SEGMENT_B only owns a reference with no data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1951) cube sergement start time don't match partition start time
[ https://issues.apache.org/jira/browse/KYLIN-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451228#comment-15451228 ] Shaofeng SHI commented on KYLIN-1951: - the "kylin.rest.timezone" only impact on the time display on Web GUI; it doesn't affect the cube build start/end time; When trigger the cube build with API, please use the epoch time, which is GMT0; I guess you'were using Linux system time, which may has the timezone information, so there is 8 hours difference. > cube sergement start time don't match partition start time > -- > > Key: KYLIN-1951 > URL: https://issues.apache.org/jira/browse/KYLIN-1951 > Project: Kylin > Issue Type: Bug >Reporter: 一岁时很拽 >Assignee: Zhong,Jason > > for streaming cube build, > i set kylin.rest.timezone=GMT+8,but cube sergement start time don't match > partition start time, A difference of 8 hours,why? > http://apache-kylin.74782.x6.nabble.com/cube-sergement-start-time-don-t-match-partition-start-time-td5538.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1839) improvement set classpath before submitting mr job
[ https://issues.apache.org/jira/browse/KYLIN-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451224#comment-15451224 ] Shaofeng SHI commented on KYLIN-1839: - What's the resolution of this JIRA? I didn't see the patch in current master branch. > improvement set classpath before submitting mr job > -- > > Key: KYLIN-1839 > URL: https://issues.apache.org/jira/browse/KYLIN-1839 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v1.5.2 >Reporter: fengYu >Assignee: fengYu > Attachments: > 0001-KYLIN-1839-support-extend-lib-from-HDFS-and-cache-tm.patch > > > in setClasspath, kylin will alway find hive jars from hive dependency using > regex, however, this will not change in one process lifetime, so I cache the > location of tmpjars and tmpfiles. > What is more, support extends user lib setting to hdfs path rather than only > support local filesystem which will cause upload jars every time if > DistributedCache do not exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (KYLIN-1839) improvement set classpath before submitting mr job
[ https://issues.apache.org/jira/browse/KYLIN-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reopened KYLIN-1839: - > improvement set classpath before submitting mr job > -- > > Key: KYLIN-1839 > URL: https://issues.apache.org/jira/browse/KYLIN-1839 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v1.5.2 >Reporter: fengYu >Assignee: fengYu > Attachments: > 0001-KYLIN-1839-support-extend-lib-from-HDFS-and-cache-tm.patch > > > in setClasspath, kylin will alway find hive jars from hive dependency using > regex, however, this will not change in one process lifetime, so I cache the > location of tmpjars and tmpfiles. > What is more, support extends user lib setting to hdfs path rather than only > support local filesystem which will cause upload jars every time if > DistributedCache do not exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1951) cube sergement start time don't match partition start time
[ https://issues.apache.org/jira/browse/KYLIN-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1951. --- Resolution: Invalid > cube sergement start time don't match partition start time > -- > > Key: KYLIN-1951 > URL: https://issues.apache.org/jira/browse/KYLIN-1951 > Project: Kylin > Issue Type: Bug >Reporter: 一岁时很拽 >Assignee: Zhong,Jason > > for streaming cube build, > i set kylin.rest.timezone=GMT+8,but cube sergement start time don't match > partition start time, A difference of 8 hours,why? > http://apache-kylin.74782.x6.nabble.com/cube-sergement-start-time-don-t-match-partition-start-time-td5538.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (KYLIN-1951) cube sergement start time don't match partition start time
[ https://issues.apache.org/jira/browse/KYLIN-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-1951: Comment: was deleted (was: the "kylin.rest.timezone" only impact on the time display on Web GUI; it doesn't affect the cube build start/end time; When trigger the cube build with API, please use the epoch time, which is GMT0; I guess you'were using Linux system time, which may has the timezone information, so there is 8 hours difference.) > cube sergement start time don't match partition start time > -- > > Key: KYLIN-1951 > URL: https://issues.apache.org/jira/browse/KYLIN-1951 > Project: Kylin > Issue Type: Bug >Reporter: 一岁时很拽 >Assignee: Zhong,Jason > > for streaming cube build, > i set kylin.rest.timezone=GMT+8,but cube sergement start time don't match > partition start time, A difference of 8 hours,why? > http://apache-kylin.74782.x6.nabble.com/cube-sergement-start-time-don-t-match-partition-start-time-td5538.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-1791) Metadata Migration Tool validation error - Aggregation group has too many combinations
[ https://issues.apache.org/jira/browse/KYLIN-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-1791. - Resolution: Fixed Fix Version/s: v1.5.3 I remember this has been fixed, jointed dimensions will be treat as one. thanks for the reporting. > Metadata Migration Tool validation error - Aggregation group has too many > combinations > -- > > Key: KYLIN-1791 > URL: https://issues.apache.org/jira/browse/KYLIN-1791 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v1.4.0, v1.3.1 >Reporter: qianqiaoneng >Assignee: Vic Wang > Fix For: v1.5.3 > > > When counting dimension number of aggregation group, the joint dimensions > should be count as 1 instead of the real dimension numbers, which cause the > exception below. > java.lang.IllegalStateException: Aggregation group 0 has too many > combinations, use 'mandatory'/'hierarchy'/'joint' to optimize; or update > 'kylin.cube.aggrgroup.max.combination' to a bigger value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1791) Metadata Migration Tool validation error - Aggregation group has too many combinations
[ https://issues.apache.org/jira/browse/KYLIN-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1791. --- > Metadata Migration Tool validation error - Aggregation group has too many > combinations > -- > > Key: KYLIN-1791 > URL: https://issues.apache.org/jira/browse/KYLIN-1791 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v1.4.0, v1.3.1 >Reporter: qianqiaoneng >Assignee: Vic Wang > Fix For: v1.5.3 > > > When counting dimension number of aggregation group, the joint dimensions > should be count as 1 instead of the real dimension numbers, which cause the > exception below. > java.lang.IllegalStateException: Aggregation group 0 has too many > combinations, use 'mandatory'/'hierarchy'/'joint' to optimize; or update > 'kylin.cube.aggrgroup.max.combination' to a bigger value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1883) Consensus Problem when running the tool, MetadataCleanupJob
[ https://issues.apache.org/jira/browse/KYLIN-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451239#comment-15451239 ] Zhong Yanghong commented on KYLIN-1883: --- Thanks for reminding me. If I made a patch for the master branch, which fixed version should I set? > Consensus Problem when running the tool, MetadataCleanupJob > --- > > Key: KYLIN-1883 > URL: https://issues.apache.org/jira/browse/KYLIN-1883 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: > better_solution_for_consensus_issue_of_MetadataCleanupJob.patch > > > When do the cleanup, current strategy is as follows: > 1. firstly create an referenceSet > 2. then add items not belonging to the referenceSet to the toDeleteSet > 3. finally delete those items in the toDeleteSet > Consensus issue will occur since we cannot make sure that all of the items in > toDeleteSet are not referenced in case that referenceSet changes during the > process. > For example, before the cleanup, SEGMENT_A is deleted and leave a DICT_A > created at the building step. Then the referenceSet will not include DICT_A. > After creating the reference set, SEGMENT_B is starting to build. Since > DICT_A still exists, it can still be referenced by SEGMENT_B. Then DICT_A > will still be included in the toDeleteSet and will be deleted later. Finally > SEGMENT_B only owns a reference with no data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1764) Add a document of RAW measure
[ https://issues.apache.org/jira/browse/KYLIN-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1764. --- > Add a document of RAW measure > - > > Key: KYLIN-1764 > URL: https://issues.apache.org/jira/browse/KYLIN-1764 > Project: Kylin > Issue Type: Bug > Components: Documentation >Reporter: Xiaoyu Wang >Assignee: Xiaoyu Wang >Priority: Minor > > Add a document of RAW measure -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1740) HDFS storage leak in HBaseResourceStore.deleteResourceImpl() in case of large cell of dictionary or table snapshot
[ https://issues.apache.org/jira/browse/KYLIN-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-1740: Fix Version/s: v1.5.3 > HDFS storage leak in HBaseResourceStore.deleteResourceImpl() in case of large > cell of dictionary or table snapshot > -- > > Key: KYLIN-1740 > URL: https://issues.apache.org/jira/browse/KYLIN-1740 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Ma Gang > Fix For: v1.5.3 > > Attachments: fix_storage_leak.patch, fix_storage_leak_1.4rc.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1740) HDFS storage leak in HBaseResourceStore.deleteResourceImpl() in case of large cell of dictionary or table snapshot
[ https://issues.apache.org/jira/browse/KYLIN-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1740. --- > HDFS storage leak in HBaseResourceStore.deleteResourceImpl() in case of large > cell of dictionary or table snapshot > -- > > Key: KYLIN-1740 > URL: https://issues.apache.org/jira/browse/KYLIN-1740 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Ma Gang > Fix For: v1.5.3 > > Attachments: fix_storage_leak.patch, fix_storage_leak_1.4rc.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1737) BufferOverflowException when calculate raw measure
[ https://issues.apache.org/jira/browse/KYLIN-1737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1737. --- > BufferOverflowException when calculate raw measure > -- > > Key: KYLIN-1737 > URL: https://issues.apache.org/jira/browse/KYLIN-1737 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.5.1 > Environment: centos 6.5 >Reporter: zhou degao >Assignee: Dong Li > > Input records: 6M+ (no this error if the input records is less than 3M) > Error: java.nio.BufferOverflowException at > java.nio.HeapByteBuffer.put(HeapByteBuffer.java:189) at > java.nio.ByteBuffer.put(ByteBuffer.java:859) at > org.apache.kylin.common.util.BytesUtil.writeByteArray(BytesUtil.java:338) at > org.apache.kylin.measure.raw.RawSerializer.serialize(RawSerializer.java:88) > at > org.apache.kylin.measure.raw.RawSerializer.serialize(RawSerializer.java:30) > at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:89) at > org.apache.kylin.engine.mr.steps.CuboidReducer.reduce(CuboidReducer.java:105) > at > org.apache.kylin.engine.mr.steps.CuboidReducer.reduce(CuboidReducer.java:43) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171) at > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) at > org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1630) incorrect sum(a)/sum(b) (how to get the rate value)
[ https://issues.apache.org/jira/browse/KYLIN-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1630. --- > incorrect sum(a)/sum(b) (how to get the rate value) > --- > > Key: KYLIN-1630 > URL: https://issues.apache.org/jira/browse/KYLIN-1630 > Project: Kylin > Issue Type: Bug > Components: General >Affects Versions: v1.5.1 > Environment: Kylin version:1.5.1 >Reporter: 陈雷雷 > > I want to get a value which is defined as sum(a)/sum(b), how can I > do this kind of anlysis. > Kylin version:1.5.1 >Now I build a cube which have sum(a) and sum(b), when I execute > “select sum(a)/sum(b) from table1 group by c” ,the result is wrong. > sum(a)/sum(b) the result is all 0 and sum(b)/sum(a) result is all 1. > MMENE_NAMESUCC ATTSUCC/ATT > CSMME15BZX 336981 368366 1 > CSMME32BZX 338754 366842 1 > CSMME07BZX 687965 747694 1 > CSMME03BHW 703269 747623 1 > CSMME12BZX 705856 764656 1 > CSMME16BHW 1962293142173 1 > MMENE_NAME SUCC ATT ATT/SUCC > CSMME15BZX 336981 368366 0 > CSMME32BZX 338754 366842 0 > CSMME07BZX 687965 747694 0 > CSMME03BHW 703269 747623 0 > CSMME12BZX 705856 764656 0 > CSMME16BHW 1962293142173 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1632) RoaringBitmap 0.6.15 fails CI
[ https://issues.apache.org/jira/browse/KYLIN-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-1632. --- > RoaringBitmap 0.6.15 fails CI > - > > Key: KYLIN-1632 > URL: https://issues.apache.org/jira/browse/KYLIN-1632 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: liyang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1109) Date read issue by JDBC
[ https://issues.apache.org/jira/browse/KYLIN-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451299#comment-15451299 ] Shaofeng SHI commented on KYLIN-1109: - Julian has an answer about this: http://apache-kylin.74782.x6.nabble.com/JDBC-query-result-Date-column-get-wrong-value-td5370.html > Date read issue by JDBC > --- > > Key: KYLIN-1109 > URL: https://issues.apache.org/jira/browse/KYLIN-1109 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Affects Versions: v1.1, v1.5.1 > Environment: windows 7/eclipse mars >Reporter: IvanPeng > Labels: easyfix > Fix For: Future > > Original Estimate: 12h > Remaining Estimate: 12h > > When i get date from kylin by jdbc driver, the date seems return a error > date(shift one day) > For example, 2015-09-26 display by kylin api , but jdbc return 2015-09-25 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1849) Search cube by name in Web UI
[ https://issues.apache.org/jira/browse/KYLIN-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451350#comment-15451350 ] Zhong,Jason commented on KYLIN-1849: Thank you [~kangkaisen], I have applied your patch. and also there's some update. https://github.com/apache/kylin/commit/64005c67a4395d90b7732a069848f6aca0290934 > Search cube by name in Web UI > - > > Key: KYLIN-1849 > URL: https://issues.apache.org/jira/browse/KYLIN-1849 > Project: Kylin > Issue Type: New Feature > Components: Web >Affects Versions: v1.5.2 >Reporter: Dayue Gao >Assignee: kangkaisen > Fix For: v1.5.4 > > Attachments: KYLIN-1849.patch > > > In order to work with dozens of cubes, could we add a search box at "Model" > page? Just like the one at "Monitor" page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-1849) Search cube by name in Web UI
[ https://issues.apache.org/jira/browse/KYLIN-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong,Jason resolved KYLIN-1849. Resolution: Fixed > Search cube by name in Web UI > - > > Key: KYLIN-1849 > URL: https://issues.apache.org/jira/browse/KYLIN-1849 > Project: Kylin > Issue Type: New Feature > Components: Web >Affects Versions: v1.5.2 >Reporter: Dayue Gao >Assignee: kangkaisen > Fix For: v1.5.4 > > Attachments: KYLIN-1849.patch > > > In order to work with dozens of cubes, could we add a search box at "Model" > page? Just like the one at "Monitor" page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
