[
https://issues.apache.org/jira/browse/HIVE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956322#comment-15956322
]
Colin Ma edited comment on HIVE-16311 at 4/5/17 7:38 AM:
---------------------------------------------------------
[~xuefuz], for the cases like 1234567.8901234560/9, here is the reason for
these diffs: when create HiveDecimal by
HiveDecimal.create("1234567.8901234560"), the trailing zero will be ignored by
the following code:
https://github.com/apache/hive/blob/master/storage-api/src/java/org/apache/hadoop/hive/common/type/FastHiveDecimalImpl.java#L482
So 1234567.890123456 (*with scale 9*) is used for division, not
1234567.8901234560(*with scale 10*).
Without this patch, the scale of result is always *HiveDecimal.MAX_SCALE*, and
the result will be *137174.21001371733333333333333333333333*, after reset the
scale for output, the result will be *137174.210013717333*.
With this patch, the scale of result is calculated as *11 by Max(6, 9+1+1)*,
and the result is *137174.21001371733*, after reset the scale for output, the
result will be *137174.210013717330*.
I update the patch to keep the trailing zero, and check if all tests can pass.
was (Author: colinma):
[~xuefuz], for the cases like 1234567.8901234560/9, here is the reason for
these diffs: when create HiveDecimal by
HiveDecimal.create("1234567.8901234560"), the trailing zero will be ignored by
the following code:
https://github.com/apache/hive/blob/master/storage-api/src/java/org/apache/hadoop/hive/common/type/FastHiveDecimalImpl.java#L482
So 1234567.890123456 (*with scale 9*) is used for division, not
1234567.8901234560(*with scale 10*).
Without this patch, the scale of result is always *HiveDecimal.MAX_SCALE*, and
the result will be *137174.21001371733333333333333333333333*, after reset the
scale for output, the result will be *137174.210013717333*.
With this patch, the scale of result is calculated as *11 by Max(6, 9+1+1)*,
and the result is *137174.21001371733*, after reset the scale for output, the
result will be *137174.210013717330*.
I think it's ok to keep the trailing zero, and update the patch to check if all
tests can pass.
> Improve the performance for FastHiveDecimalImpl.fastDivide
> ----------------------------------------------------------
>
> Key: HIVE-16311
> URL: https://issues.apache.org/jira/browse/HIVE-16311
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 2.2.0
> Reporter: Colin Ma
> Assignee: Colin Ma
> Fix For: 3.0.0
>
> Attachments: HIVE-16311.001.patch, HIVE-16311.002.patch,
> HIVE-16311.003.patch, HIVE-16311.004.patch, HIVE-16311.withTrailingZero.patch
>
>
> FastHiveDecimalImpl.fastDivide is poor performance when evaluate the
> expression as 12345.67/123.45
> There are 2 points can be improved:
> 1. Don't always use HiveDecimal.MAX_SCALE as scale when do the
> BigDecimal.divide.
> 2. Get the precision for BigInteger in a fast way if possible.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
