Alexander created KYLIN-4804:
--------------------------------
Summary: Hive Global Dictionary - with measure (PRICE)
NumberFormatException
Key: KYLIN-4804
URL: https://issues.apache.org/jira/browse/KYLIN-4804
Project: Kylin
Issue Type: Bug
Affects Versions: v3.1.1
Reporter: Alexander
Attachments: spark_executor.log
If i try to use Global dict with measure (PRICE)
Cubing with spark - exception
{code:java}
java.lang.NumberFormatException: For input string:
"7623.0000"java.lang.NumberFormatException: For input string: "7623.0000" at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580) at
java.lang.Integer.parseInt(Integer.java:615) at
org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:118)
at
org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:99)
at
org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueObjects(BaseCuboidBuilder.java:116)
at
org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:297)
at
org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:257)
{code}
it's because
{code:java}
// In order to keep compatibility with old version, tinyint/smallint/int
column use value directly, without dictionary
private boolean needDictionaryColumn(FunctionDesc functionDesc) {
DataType dataType =
functionDesc.getParameter().getColRefs().get(0).getType();
if (functionDesc.isMrDict()) {
// If isMrDict set to true, it means related column has been
// encoded in previous step by Hive Global Dictionary
return false;
}
if (dataType.isIntegerFamily() && !dataType.isBigInt()) {
return false;
}
return true;
}
{code}
returns false due to isMrDict is true in this case and it goes id =
Integer.parseInt(values[0]);
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
