[
https://issues.apache.org/jira/browse/KYLIN-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169982#comment-15169982
]
Edward Zhang commented on KYLIN-1345:
-------------------------------------
This is the stack trace
[main] org.apache.hadoop.mapred.MapTask: kvstart = 33554428(134217712); kvend
= 33554336(134217344); length = 93/8388608
2016-02-26 22:32:45,460 INFO [main] org.apache.hadoop.mapred.MapTask: Ignoring
exception during close for
org.apache.hadoop.mapred.MapTask$NewOutputCollector@7795d3e9
java.lang.IllegalArgumentException: '92005.60' exceeds the expected length for
type decimal(6,2)
at
org.apache.kylin.metadata.datatype.BigDecimalSerializer.serialize(BigDecimalSerializer.java:58)
at
org.apache.kylin.metadata.datatype.BigDecimalSerializer.serialize(BigDecimalSerializer.java:33)
at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:89)
at
org.apache.kylin.engine.mr.steps.CuboidReducer.reduce(CuboidReducer.java:105)
at
org.apache.kylin.engine.mr.steps.CuboidReducer.reduce(CuboidReducer.java:43)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
at
org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1651)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1630)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1482)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:720)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2012)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:794)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2016-02-26 22:32:45,527 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.lang.IllegalArgumentException: '92005.60'
exceeds the expected length for type decimal(6,2)
at
org.apache.kylin.metadata.datatype.BigDecimalSerializer.serialize(BigDecimalSerializer.java:58)
at
org.apache.kylin.metadata.datatype.BigDecimalSerializer.serialize(BigDecimalSerializer.java:33)
at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:89)
at
org.apache.kylin.engine.mr.steps.CuboidReducer.reduce(CuboidReducer.java:105)
at
org.apache.kylin.engine.mr.steps.CuboidReducer.reduce(CuboidReducer.java:43)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
at
org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1651)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1630)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1482)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:720)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:790)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> measure type expansion when dealing sum of decimal metric
> ---------------------------------------------------------
>
> Key: KYLIN-1345
> URL: https://issues.apache.org/jira/browse/KYLIN-1345
> Project: Kylin
> Issue Type: Improvement
> Reporter: hongbin ma
> Assignee: Edward Zhang
> Labels: newbie
>
> suppose a metric column price is of type decimal (6,2), the sum aggregator of
> it might exceed the maximal of decimal (6,2). Currently for metric
> aggregators we inherite the column's type in hive. We should think auto
> expanding decimal type to decimal(18,4) (just an example) for such cases
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
