[jira] [Commented] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086775#comment-15086775 ] fengYu commented on KYLIN-1104: --- upload patch for kylin 2.0-rc, but I find in kylin-2.0, BytesSplitter.split() is not called while building cube anymore. > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch, > 0001-bytes-split-max-length-limit.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengYu updated KYLIN-1104: -- Attachment: 0001-bytes-split-max-length-limit.patch > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch, > 0001-bytes-split-max-length-limit.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[ https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086756#comment-15086756 ] huawang commented on KYLIN-1294: In the case you give lstg_format_name does not contain null, I tested in my cube, and it's also OK. While, when I execute the following sql, where "a" is not a member of "KYLIN_TV_DIM_AREA"."TV_AREA_NAME", the "TV_AREA_NAME" of "默认地区" will still appear in the result set. I think this is also a related problem in the bug. The sql is : SELECT "KYLIN_TV_DIM_AREA"."TV_AREA_NAME" AS "TV_AREA_NAME", SUM("KYLIN_VIEW_TVAD_SUMMARY"."INCOME") AS "sum_INCOME_ok", SUM(1) AS "sum_Number_of_Records_ok", SUM("KYLIN_VIEW_TVAD_SUMMARY"."VV") AS "sum_VV_ok" FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_AREA" "KYLIN_TV_DIM_AREA" ON ("KYLIN_VIEW_TVAD_SUMMARY"."AREA" = "KYLIN_TV_DIM_AREA"."TV_AREA_ID") WHERE (("KYLIN_TV_DIM_AREA"."TV_AREA_NAME" not IN ('美剧', 'a', '默认地区'))) GROUP BY "KYLIN_TV_DIM_AREA"."TV_AREA_NAME"; > Filter of NOT IN (...) can cause incorrect result or NPE > > > Key: KYLIN-1294 > URL: https://issues.apache.org/jira/browse/KYLIN-1294 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: hongbin ma > > The version I use is 1.0. > Recently I updated to 1.2. The problent is like following: > Case1:*** > When I use “not in” in my sql, the resultset is less than my > correct data. Like following: > When the sql I use not including a “not in” is : > SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", >COUNT(1) AS "sum_Number_of_Records_ok", >SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" > FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" > "KYLIN_TV_DIM_CITY_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = >"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = >"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") > GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" > The result is like following: > (null) 206735 436180 > other169103517 480104320 > 无 25970813161884 > A 42464098 148884448 > B 31945514 121964156 > O 27878957265948 > K 37694048 127243245 > Whiel when I use a sql having “not in”, like following: > SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", >COUNT(1) AS "sum_Number_of_Records_ok", >SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" > FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" > "KYLIN_TV_DIM_CITY_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = >"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = >"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") > WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other')) > GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" > The result I got is, there are only 3 lines, and the results are all less > than the true value: > 无 323911 646111 > A 7463670 21674399 > K 15248274 54972183 > Case2: > Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary > where channel_id not in (2, 3) group by channel_id. > Here, 2 is a member of channel_id in kylin_view_tvad_summary, while > 3 is not a member of kylin_view_tvad_summary, there will be a null pointer > exception like following: > java.sql.SQLException: error while executing SQL "select channel_id, > sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group > by channel_id > LIMIT 5": null > at org.apache.calcite.avatica.Helper.createException(Helper.java:41) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112) > at > org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130) > at > org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354) > at > org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268) > at > org.apache.kylin.rest.service.QueryService.query(QueryService.java:114) > at > org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB
[jira] [Assigned] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[ https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma reassigned KYLIN-1294: - Assignee: hongbin ma > Filter of NOT IN (...) can cause incorrect result or NPE > > > Key: KYLIN-1294 > URL: https://issues.apache.org/jira/browse/KYLIN-1294 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: hongbin ma > > The version I use is 1.0. > Recently I updated to 1.2. The problent is like following: > Case1:*** > When I use “not in” in my sql, the resultset is less than my > correct data. Like following: > When the sql I use not including a “not in” is : > SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", >COUNT(1) AS "sum_Number_of_Records_ok", >SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" > FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" > "KYLIN_TV_DIM_CITY_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = >"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = >"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") > GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" > The result is like following: > (null) 206735 436180 > other169103517 480104320 > 无 25970813161884 > A 42464098 148884448 > B 31945514 121964156 > O 27878957265948 > K 37694048 127243245 > Whiel when I use a sql having “not in”, like following: > SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", >COUNT(1) AS "sum_Number_of_Records_ok", >SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" > FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" > "KYLIN_TV_DIM_CITY_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = >"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = >"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") > WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other')) > GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" > The result I got is, there are only 3 lines, and the results are all less > than the true value: > 无 323911 646111 > A 7463670 21674399 > K 15248274 54972183 > Case2: > Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary > where channel_id not in (2, 3) group by channel_id. > Here, 2 is a member of channel_id in kylin_view_tvad_summary, while > 3 is not a member of kylin_view_tvad_summary, there will be a null pointer > exception like following: > java.sql.SQLException: error while executing SQL "select channel_id, > sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group > by channel_id > LIMIT 5": null > at org.apache.calcite.avatica.Helper.createException(Helper.java:41) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112) > at > org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130) > at > org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354) > at > org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268) > at > org.apache.kylin.rest.service.QueryService.query(QueryService.java:114) > at > org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) > at > org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query() > at > org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178) > at > org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85) > at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) > at > org.springframework.web.meth
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[ https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086737#comment-15086737 ] liyang commented on KYLIN-1294: --- Meanwhile worth note that this test query did pass with no problem, giving ('FP-GTC', 'ABIN') are valid values and lstg_format_name does not contain null. select lstg_format_name, sum(price) as GMV from test_kylin_fact where lstg_format_name not in ('FP-GTC', 'ABIN') group by lstg_format_name > Filter of NOT IN (...) can cause incorrect result or NPE > > > Key: KYLIN-1294 > URL: https://issues.apache.org/jira/browse/KYLIN-1294 > Project: Kylin > Issue Type: Bug >Reporter: liyang > > The version I use is 1.0. > Recently I updated to 1.2. The problent is like following: > Case1:*** > When I use “not in” in my sql, the resultset is less than my > correct data. Like following: > When the sql I use not including a “not in” is : > SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", >COUNT(1) AS "sum_Number_of_Records_ok", >SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" > FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" > "KYLIN_TV_DIM_CITY_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = >"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = >"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") > GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" > The result is like following: > (null) 206735 436180 > other169103517 480104320 > 无 25970813161884 > A 42464098 148884448 > B 31945514 121964156 > O 27878957265948 > K 37694048 127243245 > Whiel when I use a sql having “not in”, like following: > SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", >COUNT(1) AS "sum_Number_of_Records_ok", >SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" > FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" > "KYLIN_TV_DIM_CITY_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = >"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = >"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") > WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other')) > GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" > The result I got is, there are only 3 lines, and the results are all less > than the true value: > 无 323911 646111 > A 7463670 21674399 > K 15248274 54972183 > Case2: > Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary > where channel_id not in (2, 3) group by channel_id. > Here, 2 is a member of channel_id in kylin_view_tvad_summary, while > 3 is not a member of kylin_view_tvad_summary, there will be a null pointer > exception like following: > java.sql.SQLException: error while executing SQL "select channel_id, > sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group > by channel_id > LIMIT 5": null > at org.apache.calcite.avatica.Helper.createException(Helper.java:41) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112) > at > org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130) > at > org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354) > at > org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268) > at > org.apache.kylin.rest.service.QueryService.query(QueryService.java:114) > at > org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) > at > org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query() > at > org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178) > at > org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85) > at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source) > at > sun.reflect.DelegatingMeth
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[ https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086736#comment-15086736 ] liyang commented on KYLIN-1294: --- Julian pointed out a maybe related JIRA https://issues.apache.org/jira/browse/CALCITE-980 > Filter of NOT IN (...) can cause incorrect result or NPE > > > Key: KYLIN-1294 > URL: https://issues.apache.org/jira/browse/KYLIN-1294 > Project: Kylin > Issue Type: Bug >Reporter: liyang > > The version I use is 1.0. > Recently I updated to 1.2. The problent is like following: > Case1:*** > When I use “not in” in my sql, the resultset is less than my > correct data. Like following: > When the sql I use not including a “not in” is : > SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", >COUNT(1) AS "sum_Number_of_Records_ok", >SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" > FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" > "KYLIN_TV_DIM_CITY_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = >"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = >"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") > GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" > The result is like following: > (null) 206735 436180 > other169103517 480104320 > 无 25970813161884 > A 42464098 148884448 > B 31945514 121964156 > O 27878957265948 > K 37694048 127243245 > Whiel when I use a sql having “not in”, like following: > SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", >COUNT(1) AS "sum_Number_of_Records_ok", >SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" > FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" > "KYLIN_TV_DIM_CITY_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = >"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = >"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") > WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other')) > GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" > The result I got is, there are only 3 lines, and the results are all less > than the true value: > 无 323911 646111 > A 7463670 21674399 > K 15248274 54972183 > Case2: > Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary > where channel_id not in (2, 3) group by channel_id. > Here, 2 is a member of channel_id in kylin_view_tvad_summary, while > 3 is not a member of kylin_view_tvad_summary, there will be a null pointer > exception like following: > java.sql.SQLException: error while executing SQL "select channel_id, > sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group > by channel_id > LIMIT 5": null > at org.apache.calcite.avatica.Helper.createException(Helper.java:41) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112) > at > org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130) > at > org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354) > at > org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268) > at > org.apache.kylin.rest.service.QueryService.query(QueryService.java:114) > at > org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) > at > org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query() > at > org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178) > at > org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85) > at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.springframework.web.method.support.InvocableHandlerMethod.invoke(Inv
[jira] [Created] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
liyang created KYLIN-1294: - Summary: Filter of NOT IN (...) can cause incorrect result or NPE Key: KYLIN-1294 URL: https://issues.apache.org/jira/browse/KYLIN-1294 Project: Kylin Issue Type: Bug Reporter: liyang The version I use is 1.0. Recently I updated to 1.2. The problent is like following: Case1:*** When I use “not in” in my sql, the resultset is less than my correct data. Like following: When the sql I use not including a “not in” is : SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", COUNT(1) AS "sum_Number_of_Records_ok", SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" "KYLIN_TV_DIM_CITY_V_KYLIN" ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = "KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" The result is like following: (null) 206735 436180 other169103517 480104320 无 25970813161884 A 42464098 148884448 B 31945514 121964156 O 27878957265948 K 37694048 127243245 Whiel when I use a sql having “not in”, like following: SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE", COUNT(1) AS "sum_Number_of_Records_ok", SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok" FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN" "KYLIN_TV_DIM_CITY_V_KYLIN" ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" = "KYLIN_TV_DIM_CITY_V_KYLIN"."CITY") LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN" ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" = "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1") WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other')) GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" The result I got is, there are only 3 lines, and the results are all less than the true value: 无 323911 646111 A 7463670 21674399 K 15248274 54972183 Case2: Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group by channel_id. Here, 2 is a member of channel_id in kylin_view_tvad_summary, while 3 is not a member of kylin_view_tvad_summary, there will be a null pointer exception like following: java.sql.SQLException: error while executing SQL "select channel_id, sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group by channel_id LIMIT 5": null at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130) at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354) at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268) at org.apache.kylin.rest.service.QueryService.query(QueryService.java:114) at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query() at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178) at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85) at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(Req
[jira] [Created] (KYLIN-1293) keep kylin log clean and neat
hongbin ma created KYLIN-1293: - Summary: keep kylin log clean and neat Key: KYLIN-1293 URL: https://issues.apache.org/jira/browse/KYLIN-1293 Project: Kylin Issue Type: Improvement Reporter: hongbin ma Assignee: hongbin ma 1. reduce kylin log preamble 2. correct log level as much as possible 3. trim log size -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1292) Create a doc for all properties in conf/kylin.properties
Shaofeng SHI created KYLIN-1292: --- Summary: Create a doc for all properties in conf/kylin.properties Key: KYLIN-1292 URL: https://issues.apache.org/jira/browse/KYLIN-1292 Project: Kylin Issue Type: Improvement Components: Documentation Reporter: Shaofeng SHI Need a central place to introduce the variables in kylin.properties, list the information about: name, value options, default value, impact, since version, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1291) Make sure thread resources are freed when a query aborts/timeouts
hongbin ma created KYLIN-1291: - Summary: Make sure thread resources are freed when a query aborts/timeouts Key: KYLIN-1291 URL: https://issues.apache.org/jira/browse/KYLIN-1291 Project: Kylin Issue Type: Improvement Reporter: hongbin ma Assignee: hongbin ma when user cancels a query or the query is timed out, kylin server should make sure the local thread pool resources assigned for the query is freed. Meanwhile, it's best also to free all the scanner threads in region servers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)