ggangadharan opened a new pull request, #6878:
URL: https://github.com/apache/hive/pull/6878
### What changes were proposed in this pull request?
`trunc(number, scale)` no longer converts the value to a `long` while
truncating. The row-mode UDF and the vectorized expressions now share one
implementation in a new class, `TruncUtils`:
- **DECIMAL:** truncated in place with
`HiveDecimalWritable.mutateSetScale(scale, ROUND_DOWN)`. There is no `long` or
`BigDecimal` conversion, and no new object per row.
- **BIGINT / INT / SMALLINT / TINYINT:** plain `long` arithmetic using a
table of powers of ten (`input - input % 10^-scale`).
- **DOUBLE / FLOAT:** `trunc(x)` (scale 0) skips `BigDecimal` entirely.
Other scales truncate the double's shortest decimal form, the same way
`RoundUtils.round` does. Scales outside a double's range return immediately, so
a huge scale argument cannot stall the query.
### Why are the changes needed?
The old code computed `(long) (input * 10^scale) / 10^scale`. Once `input *
10^scale` is larger than `Long.MAX_VALUE` (about 9.22E18), the cast silently
wraps around:
```sql
select trunc(123456711889.122345897, 8);
-- before: -61010728847.97317027
-- after: 123456711889.12234589
```
The same overflow also gave wrong results for:
- any scale of 19 or more, e.g. `trunc(x, 50)`;
- decimals whose integer part doesn't fit in a `long`, even at scale 0, e.g.
`trunc(cast(99999999999999999999999999.9 as decimal(38,1)))`.
The bug was in both the row-mode UDF and the vectorized `TruncDecimal` /
`TruncFloat`.
### Does this PR introduce _any_ user-facing change?
Yes, the results are now correct.
1. Truncating large values no longer overflows. This applies to decimals,
doubles, and any scale of 19 or more.
2. Row-mode `trunc` on DOUBLE now truncates the shortest decimal form of the
number, matching vectorized execution and `round()`. Before, row mode used the
exact binary value, so the two modes could disagree:
```sql
select trunc(cast(0.3 as double), 1);
-- before: 0.2 in row mode, 0.3 vectorized
-- after: 0.3 in both
```
3. `trunc` on a DOUBLE NaN or Infinity returns the input. Before, it threw a
`NumberFormatException`.
### How was this patch tested?
New JUnit tests:
- `TestGenericUDFTrunc`
- `TestVectorMathFunctions`
- `TestHiveDecimal`
New q-file test:
- `udf_trunc_large_number.q` runs the same queries vectorized and
non-vectorized over DECIMAL(38,9), DOUBLE, and BIGINT columns.
- The two result sets are identical.
- The plan shows the native `TruncDecimal` / `TruncFloat` expressions
(`usesVectorUDFAdaptor: false`).
```bash
mvn test -pl storage-api -Dtest=TestHiveDecimal
mvn test -pl ql -Dtest=TestGenericUDFTrunc,TestVectorMathFunctions
cd itests/qtest && mvn test -Dtest=TestMiniLlapLocalCliDriver \
-Dqfile=udf_trunc_large_number.q,udf_trunc_number.q,vector_udf_trunc.q
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
