[jira] [Commented] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133822#comment-15133822 ] Shaofeng SHI commented on KYLIN-1104: - sorry for the late response; I merged the new patch in 2.x-staging branch, but skipped 2.0-rc because it has been freezed for release. Thanks for the contribution, Yu! > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v2.1, v1.3 > > Attachments: 0001-.patch, 0001-bug.patch, > 0001-bytes-split-max-length-limit.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086775#comment-15086775 ] fengYu commented on KYLIN-1104: --- upload patch for kylin 2.0-rc, but I find in kylin-2.0, BytesSplitter.split() is not called while building cube anymore. > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch, > 0001-bytes-split-max-length-limit.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082353#comment-15082353 ] fengYu commented on KYLIN-1104: --- OK, I will do it based on kylin 2.0-rc. > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082352#comment-15082352 ] fengYu commented on KYLIN-1104: --- OK, I will do it based on kylin 2.0-rc. > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082274#comment-15082274 ] Shaofeng SHI commented on KYLIN-1104: - [~feng_xiao_yu] Yu, the patch is applied in 1.x-staging branch, you can see it in https://github.com/apache/kylin/commit/307e13ccfea507290a1fdd4e72447fac69ecc63b ; Could you please make another patch for 2.x-staging? Thanks for your contribution. > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu >Assignee: Shaofeng SHI > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1104) Long dimension value casue ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/KYLIN-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075070#comment-15075070 ] Shaofeng SHI commented on KYLIN-1104: - Just saw this... I think it was missed there, no other reason; v1.1 was released on Oct 25, while the patch was uploaded on Oct 30, and the fixed version was set to "v1.0 v1.1" which was wrong. It caused this JIRA wasn't on the list of later releases. Now let me set it to v1.3 and assign to me. > Long dimension value casue ArrayIndexOutOfBoundsException > - > > Key: KYLIN-1104 > URL: https://issues.apache.org/jira/browse/KYLIN-1104 > Project: Kylin > Issue Type: Bug >Reporter: fengYu > Fix For: v1.3 > > Attachments: 0001-.patch, 0001-bug.patch > > > I get ArrayIndexOutOfBoundsException while building cube, and the error is > caused by one dimension(I think measure can not be so long) value is longer > than 4096 bytes. > I check source code and find the reason: > 1、it specify max split length is 4096 when create BytesSplitter object in > setup. > 2、in BytesSplitter.split function which split all items in one row with > delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); > copy every item to a SplittedBytes object named split. and before this line , > it execute split.length = length; to set the split length. > 3、while executing the copy code, it throw ArrayIndexOutOfBoundsException > because the real item length is bigger than 4096! > 4、in mapper function, the exception will catched and execute > handleErrorRecord function to handle it, However, in this function it will > call use splits again(in toString function), ArrayIndexOutOfBoundsException > will throw again. > 5、There is not one catch this exception, which casue the mapper execute > failed. > That is all my situation and analysis, I will modify source code to continue > my job building and I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
