Re: Kylin sample cube giving error
hi, did you follow http://kylin.apache.org/docs15/install/advance_settings.html to disable compression? and can you be more specific on where's not working? On Mon, Apr 18, 2016 at 10:26 AM, joffrey wrote: > Hi, I meet the same problem too, I tried to disable the snappy > configuration, > but it didn't work, and I can't find the hard-code sql-expression in source > code, pls let me know if there is any progress now. > > -- > View this message in context: > http://apache-kylin.74782.x6.nabble.com/Kylin-sample-cube-giving-error-tp4052p4183.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
[jira] [Created] (KYLIN-1596) UT fail, due to String encoding CharsetEncoder mismatch
wangxianbin created KYLIN-1596: -- Summary: UT fail, due to String encoding CharsetEncoder mismatch Key: KYLIN-1596 URL: https://issues.apache.org/jira/browse/KYLIN-1596 Project: Kylin Issue Type: Bug Components: Tools, Build and Test Affects Versions: v1.5.2 Reporter: wangxianbin Assignee: hongbin ma Priority: Minor Fix For: v1.5.2 InvertedIndexLocalTest build dictionary using the platform's default charset for String encoding which is unsure, however, in getIdFromValueImpl of TrieDictionary using UTF, which cause the mismatch, as follow 2016-04-18 18:03:39,903 ERROR [main] dict.TrieDictionary:173 : Not a valid value: 英国 java.lang.IllegalArgumentException: Value not exists! at org.apache.kylin.dimension.Dictionary.getIdFromValueBytes(Dictionary.java:160) at org.apache.kylin.dict.TrieDictionary.getIdFromValueImpl(TrieDictionary.java:158) at org.apache.kylin.dimension.Dictionary.getIdFromValue(Dictionary.java:96) at org.apache.kylin.dimension.Dictionary.getIdFromValue(Dictionary.java:76) at org.apache.kylin.invertedindex.index.TableRecord.setValueString(TableRecord.java:116) at org.apache.kylin.invertedindex.InvertedIndexLocalTest.loadRecordsSorted(InvertedIndexLocalTest.java:187) at org.apache.kylin.invertedindex.InvertedIndexLocalTest.testCodec(InvertedIndexLocalTest.java:133) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.junit.runner.JUnitCore.run(JUnitCore.java:160) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140) Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1595) columns in metric could also be in filter/groupby
hongbin ma created KYLIN-1595: - Summary: columns in metric could also be in filter/groupby Key: KYLIN-1595 URL: https://issues.apache.org/jira/browse/KYLIN-1595 Project: Kylin Issue Type: Bug Reporter: hongbin ma Assignee: hongbin ma select sum(a.price) as x, count(distinct a.sellerid) as y from fact a where a.sellerid in (1,2,3) group by a.date LIMIT 5 the above query might fail because currently in CubeStorageQuery.buildDimensionsAndMetrics, we exclude all measure columns from dimensions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: kylin Exception log
HBase 0.98.6-cdh5.3.2 has that method, see https://archive.cloudera.com/cdh5/cdh/5/hbase-0.98.6-cdh5.3.2/apidocs/index.html So it is very likely there is an other version hbase jar on your classpath, which pollutes kylin runtime; Please run $KYLIN_HOME/bin/find-hbase-dependency.sh and $KYLIN_HOME/bin/find-hive-dependency.sh, and then check the output of them to see whether there is such a jar. 2016-04-18 9:56 GMT+08:00 陈佛林 : > kylin version:1.5.0 > hadoop version: 2.5.0-cdh5.3.2 > hbase version : 0.98.6-cdh5.3.2 > > > 2016-04-16 9:14 GMT+08:00 ShaoFeng Shi : > > > what's your hadoop/hbase/kylin version? please provide such information > > when asking question in the community, this will save the turn-around > time > > and help you to get response quickly; thanks! > > > > 2016-04-15 15:48 GMT+08:00 陈佛林 : > > > > > Apr 15, 2016 12:45:41 AM org.apache.catalina.core.StandardWrapperValve > > > invoke > > > SEVERE: Servlet.service() for servlet [kylin] in context with path > > [/kylin] > > > threw exception [Handler processing failed; nested exception is > > > java.lang.NoSuchMethodError: > > > > > > org.apache.hadoop.hbase.client.Put.getFamilyMap()Ljava/util/NavigableMap;] > > > with root cause > > > java.lang.NoSuchMethodError: > > > > org.apache.hadoop.hbase.client.Put.getFamilyMap()Ljava/util/NavigableMap; > > > at org.apache.kylin.rest.security.MockHTable.put(MockHTable.java:494) > > > at > > org.apache.kylin.rest.service.AclService.createAcl(AclService.java:237) > > > at > > org.apache.kylin.rest.service.AccessService.init(AccessService.java:73) > > > at > > > > > > > > > org.apache.kylin.rest.service.AccessService$$FastClassByCGLIB$$91550c7f.invoke() > > > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > > > at > > > > > > > > > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) > > > at > > > > > > > > > org.apache.kylin.rest.service.AccessService$$EnhancerByCGLIB$$4c4321b5.init() > > > at > > > > > > > > > org.apache.kylin.rest.service.ModelService.createModelDesc(ModelService.java:96) > > > at > > > > > > > > > org.apache.kylin.rest.service.ModelService$$FastClassByCGLIB$$eb0205be.invoke() > > > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > > > at > > > > > > > > > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) > > > at > > > > > > > > > org.apache.kylin.rest.service.ModelService$$EnhancerByCGLIB$$8a776c6a.createModelDesc() > > > at > > > > > > > > > org.apache.kylin.rest.controller.ModelController.saveModelDesc(ModelController.java:100) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > at > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at > > > > > > > > > org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) > > > at > > > > > > > > > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) > > > at > > > > > > > > > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) > > > at > > > > > > > > > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) > > > at > > > > > > > > > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) > > > at > > > > > > > > > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80) > > > at > > > > > > > > > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923) > > > at > > > > > > > > > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852) > > > at > > > > > > > > > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882) > > > at > > > > > > > > > org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789) > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:646) > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > > > at > > > > > > > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > > > at > > > > > > > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > > > at > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > > > at > > > > > > > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > > > at > > > > > > > > > org.apache.catalina.c
[jira] [Created] (KYLIN-1594) Requirement function of RANK() over window in SQL
Zhong Yanghong created KYLIN-1594: - Summary: Requirement function of RANK() over window in SQL Key: KYLIN-1594 URL: https://issues.apache.org/jira/browse/KYLIN-1594 Project: Kylin Issue Type: New Feature Components: Query Engine Reporter: Zhong Yanghong Assignee: liyang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Question about cube size estimation in Kylin 1.5
Hi everyone, I made several cubing tests on 1.5 and found most of the time was spent on the "Convert Cuboid Data to HFile" step due to lack of reducer parallelism. It seems that the estimated cube size is too small compared to the actual size, which leads to small number of regions (hence reducers) to be created. The setup and result of the tests are like: Cube#1: source_record=11998051, estimated_size=8805MB, coefficient=0.25, region_cut=5GB, #regions=2, actual_size=49GB Cube#2: source_record=123908390, estimated_size=4653MB, coefficient=0.05, region_cut=10GB, #regions=2, actual_size=144GB The "coefficient" is from CubeStatsReader#estimateCuboidStorageSize, which looks mysterious to me. Currently the formula for cuboid size estimation is size(cuboid) = rows(cuboid) x row_size(cuboid) x coefficient where coefficient = has_memory_hungry_measures(cube) ? 0.05 : 0.25 Why do we multiply the coefficient? And why it's five times smaller in memory hungry case? Cloud someone explain the rationale behind it? Thanks, Dayue
[jira] [Created] (KYLIN-1593) Agg group info will be blank when trying to edit cube
hongbin ma created KYLIN-1593: - Summary: Agg group info will be blank when trying to edit cube Key: KYLIN-1593 URL: https://issues.apache.org/jira/browse/KYLIN-1593 Project: Kylin Issue Type: Bug Reporter: hongbin ma Assignee: Zhong,Jason i'm trying with the master branch. reproduce it by editing a provisioned cube like test_kylin_cube_with_slr_left_join_desc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1592) When migrating cube, for example from QA to RPOD, some parts are missed
Zhong Yanghong created KYLIN-1592: - Summary: When migrating cube, for example from QA to RPOD, some parts are missed Key: KYLIN-1592 URL: https://issues.apache.org/jira/browse/KYLIN-1592 Project: Kylin Issue Type: Bug Components: Metadata Reporter: Zhong Yanghong Assignee: Zhong Yanghong When migrating cube from QA to PROD, some parts are ignored, like table_exd, cube_statistics. Especially the missing of "cube_statistics" will cause the merging of migrated cubes runs to error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1591) enabling different block size setting for small column families
hongbin ma created KYLIN-1591:
-
Summary: enabling different block size setting for small column
families
Key: KYLIN-1591
URL: https://issues.apache.org/jira/browse/KYLIN-1591
Project: Kylin
Issue Type: Improvement
Reporter: hongbin ma
Assignee: hongbin ma
the current setting for hbase block size (kylin.hbase.default.block.size) is by
default set to a quite large value because cubes can be quite big value(4M)
because cubes with memory hungry measures like hll can be quite large. However
if a column family does not contain memory hungry measures it does not make
sense to be that large.
the revised setting will be:
public int getHbaseDefaultBlockSize() {
return Integer.valueOf(getOptional("kylin.hbase.default.block.size",
"1048576"));
}
public int getHbaseSmallFamilyBlockSize() {
return
Integer.valueOf(getOptional("kylin.hbase.small.family.block.size", "65536"));
}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
