Dayue Gao created KYLIN-1768:
--------------------------------
Summary: NDCuboidMapper throws ArrayIndexOutOfBoundsException when
dimension is fixed length encoded to more than 256 bytes
Key: KYLIN-1768
URL: https://issues.apache.org/jira/browse/KYLIN-1768
Project: Kylin
Issue Type: Bug
Components: Job Engine
Affects Versions: v1.5.2
Reporter: Dayue Gao
Assignee: Dayue Gao
When user defined a dimension which is fixed length encoded to more than 256
bytes, "Build N-Dimension Cuboid Data" step failed in map phase. The stack
trace is shown below:
{noformat}
Error: java.lang.ArrayIndexOutOfBoundsException at
java.lang.System.arraycopy(Native Method) at
org.apache.kylin.cube.common.RowKeySplitter.split(RowKeySplitter.java:103) at
org.apache.kylin.engine.mr.steps.NDCuboidMapper.map(NDCuboidMapper.java:125) at
org.apache.kylin.engine.mr.steps.NDCuboidMapper.map(NDCuboidMapper.java:49) at
org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:415) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
{noformat}
The reason is that `RowKeySplitter` is hardcoded to 65 splits and 256 bytes per
split, and trying to put a larger encoded dimension throws
ArrayIndexOutOfBoundsException.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
