peterxcli opened a new issue, #5673:
URL: https://github.com/apache/datafusion-comet/issues/5673
### Describe the bug
In ANSI mode, `cast_float_to_int32_up`
(`native/spark-expr/src/conversion_funcs/numeric.rs:395-396`, same shape in
`cast_float_to_int16_down` at `:333`) detects overflow as
```rust
let is_overflow = value.is_nan() || value.abs() as $rust_dest_type ==
$max_dest_val;
```
i.e. "the saturating conversion of |value| landed on MAX". That flags every
double in `[2147483647.0, 2147483648.0)` and `(-2147483649.0, -2147483647.0]`
as overflow, including `INT_MAX` and `INT_MIN` themselves, which are exactly
representable doubles and valid ints. Spark's `DoubleExactNumeric.toInt` /
`toLong` accept any `x` with `Math.floor(x) <= MaxValue && Math.ceil(x) >=
MinValue` and return `x.toInt` / `x.toLong`; for BIGINT `Long.MaxValue.toDouble
== 2^63`, so Spark accepts `±2^63` and saturates.
### Steps to reproduce
```sql
SET spark.sql.ansi.enabled = true;
CREATE TABLE t USING parquet AS SELECT * FROM VALUES (2147483647.0D),
(-2147483648.0D), (2147483647.5D) AS t(c);
SELECT CAST(c AS INT) FROM t;
SELECT CAST(9223372036854775808.0D AS BIGINT), CAST(-9223372036854775808.0D
AS BIGINT);
```
Spark: `2147483647`, `-2147483648`, `2147483647`; `9223372036854775807`,
`-9223372036854775808`. Comet: `CAST_OVERFLOW` error for every one of them.
Same for FLOAT sources.
### Expected behavior
Match Spark's bound check: only values outside `[MinValue, MaxValue]` after
floor/ceil overflow.
### Proposed solution
```rust
let is_overflow = value.is_nan()
|| !(value.floor() <= $max_dest_val as $float_ty && value.ceil() >=
$min_dest_val as $float_ty);
// then `value as $rust_dest_type` — Rust `as` saturates exactly like JVM
d2i/d2l
```
Apply to both macros and add the boundary values to `CometNativeCastSuite`
in ANSI mode.
### Additional context
Default-on: `CometCast.canCastFromDouble` / `canCastFromFloat` mark Int/Long
as Compatible in all eval modes. The check dates from #350; #4941 and the #5128
plan intend to keep "the saturation-based overflow detection exactly as
written", so a performance rewrite would carry the bug forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
