[jira] [Created] (KYLIN-1291) Make sure thread resources are freed when a query aborts/timeouts
hongbin ma created KYLIN-1291: - Summary: Make sure thread resources are freed when a query aborts/timeouts Key: KYLIN-1291 URL: https://issues.apache.org/jira/browse/KYLIN-1291 Project: Kylin Issue Type: Improvement Reporter: hongbin ma Assignee: hongbin ma when user cancels a query or the query is timed out, kylin server should make sure the local thread pool resources assigned for the query is freed. Meanwhile, it's best also to free all the scanner threads in region servers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
liyang created KYLIN-1294:
-
Summary: Filter of NOT IN (...) can cause incorrect result or NPE
Key: KYLIN-1294
URL: https://issues.apache.org/jira/browse/KYLIN-1294
Project: Kylin
Issue Type: Bug
Reporter: liyang
The version I use is 1.0.
Recently I updated to 1.2. The problent is like following:
Case1:***
When I use “not in” in my sql, the resultset is less than my correct
data. Like following:
When the sql I use not including a “not in” is :
SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
COUNT(1) AS "sum_Number_of_Records_ok",
SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
"KYLIN_TV_DIM_CITY_V_KYLIN"
ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
The result is like following:
(null) 206735 436180
other169103517 480104320
无 25970813161884
A 42464098 148884448
B 31945514 121964156
O 27878957265948
K 37694048 127243245
Whiel when I use a sql having “not in”, like following:
SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
COUNT(1) AS "sum_Number_of_Records_ok",
SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
"KYLIN_TV_DIM_CITY_V_KYLIN"
ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other'))
GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
The result I got is, there are only 3 lines, and the results are all less than
the true value:
无 323911 646111
A 7463670 21674399
K 15248274 54972183
Case2:
Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary where
channel_id not in (2, 3) group by channel_id.
Here, 2 is a member of channel_id in kylin_view_tvad_summary, while 3
is not a member of kylin_view_tvad_summary, there will be a null pointer
exception like following:
java.sql.SQLException: error while executing SQL "select channel_id,
sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group by
channel_id
LIMIT 5": null
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130)
at
org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354)
at
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268)
at
org.apache.kylin.rest.service.QueryService.query(QueryService.java:114)
at
org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
at
org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query()
at
org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85)
at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
at
[jira] [Assigned] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[
https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hongbin ma reassigned KYLIN-1294:
-
Assignee: hongbin ma
> Filter of NOT IN (...) can cause incorrect result or NPE
>
>
> Key: KYLIN-1294
> URL: https://issues.apache.org/jira/browse/KYLIN-1294
> Project: Kylin
> Issue Type: Bug
>Reporter: liyang
>Assignee: hongbin ma
>
> The version I use is 1.0.
> Recently I updated to 1.2. The problent is like following:
> Case1:***
> When I use “not in” in my sql, the resultset is less than my
> correct data. Like following:
> When the sql I use not including a “not in” is :
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result is like following:
> (null) 206735 436180
> other169103517 480104320
> 无 25970813161884
> A 42464098 148884448
> B 31945514 121964156
> O 27878957265948
> K 37694048 127243245
> Whiel when I use a sql having “not in”, like following:
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other'))
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result I got is, there are only 3 lines, and the results are all less
> than the true value:
> 无 323911 646111
> A 7463670 21674399
> K 15248274 54972183
> Case2:
> Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary
> where channel_id not in (2, 3) group by channel_id.
> Here, 2 is a member of channel_id in kylin_view_tvad_summary, while
> 3 is not a member of kylin_view_tvad_summary, there will be a null pointer
> exception like following:
> java.sql.SQLException: error while executing SQL "select channel_id,
> sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group
> by channel_id
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130)
> at
> org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268)
> at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:114)
> at
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query()
> at
> org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85)
> at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
> at
>
[jira] [Updated] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengYu updated KYLIN-1104: -- Attachment: 0001-bytes-split-max-length-limit.patch > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch, > 0001-bytes-split-max-length-limit.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[
https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086737#comment-15086737
]
liyang commented on KYLIN-1294:
---
Meanwhile worth note that this test query did pass with no problem, giving
('FP-GTC', 'ABIN') are valid values and lstg_format_name does not contain null.
select lstg_format_name, sum(price) as GMV
from test_kylin_fact
where lstg_format_name not in ('FP-GTC', 'ABIN')
group by lstg_format_name
> Filter of NOT IN (...) can cause incorrect result or NPE
>
>
> Key: KYLIN-1294
> URL: https://issues.apache.org/jira/browse/KYLIN-1294
> Project: Kylin
> Issue Type: Bug
>Reporter: liyang
>
> The version I use is 1.0.
> Recently I updated to 1.2. The problent is like following:
> Case1:***
> When I use “not in” in my sql, the resultset is less than my
> correct data. Like following:
> When the sql I use not including a “not in” is :
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result is like following:
> (null) 206735 436180
> other169103517 480104320
> 无 25970813161884
> A 42464098 148884448
> B 31945514 121964156
> O 27878957265948
> K 37694048 127243245
> Whiel when I use a sql having “not in”, like following:
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other'))
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result I got is, there are only 3 lines, and the results are all less
> than the true value:
> 无 323911 646111
> A 7463670 21674399
> K 15248274 54972183
> Case2:
> Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary
> where channel_id not in (2, 3) group by channel_id.
> Here, 2 is a member of channel_id in kylin_view_tvad_summary, while
> 3 is not a member of kylin_view_tvad_summary, there will be a null pointer
> exception like following:
> java.sql.SQLException: error while executing SQL "select channel_id,
> sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group
> by channel_id
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130)
> at
> org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268)
> at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:114)
> at
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query()
> at
> org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85)
> at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
> at
>
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[
https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086736#comment-15086736
]
liyang commented on KYLIN-1294:
---
Julian pointed out a maybe related JIRA
https://issues.apache.org/jira/browse/CALCITE-980
> Filter of NOT IN (...) can cause incorrect result or NPE
>
>
> Key: KYLIN-1294
> URL: https://issues.apache.org/jira/browse/KYLIN-1294
> Project: Kylin
> Issue Type: Bug
>Reporter: liyang
>
> The version I use is 1.0.
> Recently I updated to 1.2. The problent is like following:
> Case1:***
> When I use “not in” in my sql, the resultset is less than my
> correct data. Like following:
> When the sql I use not including a “not in” is :
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result is like following:
> (null) 206735 436180
> other169103517 480104320
> 无 25970813161884
> A 42464098 148884448
> B 31945514 121964156
> O 27878957265948
> K 37694048 127243245
> Whiel when I use a sql having “not in”, like following:
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other'))
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result I got is, there are only 3 lines, and the results are all less
> than the true value:
> 无 323911 646111
> A 7463670 21674399
> K 15248274 54972183
> Case2:
> Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary
> where channel_id not in (2, 3) group by channel_id.
> Here, 2 is a member of channel_id in kylin_view_tvad_summary, while
> 3 is not a member of kylin_view_tvad_summary, there will be a null pointer
> exception like following:
> java.sql.SQLException: error while executing SQL "select channel_id,
> sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group
> by channel_id
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130)
> at
> org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268)
> at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:114)
> at
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query()
> at
> org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85)
> at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
>
[jira] [Commented] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086775#comment-15086775 ] fengYu commented on KYLIN-1104: --- upload patch for kylin 2.0-rc, but I find in kylin-2.0, BytesSplitter.split() is not called while building cube anymore. > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch, > 0001-bytes-split-max-length-limit.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
