[
https://issues.apache.org/jira/browse/KYLIN-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3686:
--------------------------------
Fix Version/s: (was: v2.6.0)
We couldn't re-produce this problem.
> Top_N metric code requires cube storage type to be ID_SHARDED_HBASE, but the
> Web UI defaults to ID_HBASE and provides no safeguards against storage type
> mismatch
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-3686
> URL: https://issues.apache.org/jira/browse/KYLIN-3686
> Project: Kylin
> Issue Type: Improvement
> Components: Measure - TopN, Metadata, Web
> Affects Versions: v2.5.0
> Environment: HDP 2.5.6, Kylin 2.5
> Reporter: Vsevolod Ostapenko
> Assignee: Shaofeng SHI
> Priority: Major
>
> When new cube is defined via Kylin 2.5 UI, the default cube storage type is
> set to 0 (ID_HBASE).
> Top_N metric support is currently hard coded to expect cube storage type 2
> (ID_SHARDED_HBASE), and it *_does not_* check if the cube storage type is the
> "sharded HBASE".
> UI provides no safeguards either to prevent a user from defining a cube with
> Top_N metric that would blow up on the cube building stage with a perplexing
> stack trace like the following:
> {quote}2018-10-22 16:15:50,388 ERROR [main]
> org.apache.kylin.engine.mr.KylinMapper:
> java.lang.ArrayIndexOutOfBoundsException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKeyInternal(NDCuboidBuilder.java:106)
> at
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKey(NDCuboidBuilder.java:71)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:112)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:47)
> at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> {quote}
> Please, either:
> – modify Top_N code to support all cube storage types (not only
> ID_SHARDED_HBASE),
> or
> – modify Top_N code to perform explicit check for cube storage type and
> raise descriptive exception, when cube storage is not the one that is
> expected. Plus update the UI to prevent the user from creating cube
> definitions that are incompatible with the storage type compatible with Top_N
> measure
> PS: NDCCuboidBuilder,java contains the following line:
> {quote}int offset = RowConstants.ROWKEY_SHARDID_LEN +
> RowConstants.ROWKEY_CUBOIDID_LEN; // skip shard and cuboidId{quote}
> If cube storage type is not ID_SHARDED_HBASE, offset is calculated
> incorrectly, which leads to ArrayIndexOutOfBounds exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
