mbutrovich opened a new issue, #11589:
URL: https://github.com/apache/arrow-rs/issues/11589
### Describe the bug
Casting to a decimal type whose scale is larger than the input's by more
than the target's maximum precision returns an error for the whole array, even
with `safe: true` and even when every value is 0.
### To Reproduce
```rust
use std::sync::Arc;
use arrow_array::{ArrayRef, Decimal128Array};
use arrow_cast::cast::{cast_with_options, CastOptions};
use arrow_schema::DataType;
fn main() {
let a: ArrayRef = Arc::new(Decimal128Array::from(vec![0,
1]).with_precision_and_scale(5, -2).unwrap());
let options = CastOptions { safe: true, ..Default::default() };
println!("{:?}", cast_with_options(&a, &DataType::Decimal128(38, 38),
&options));
// Err(CastError("Cannot cast to Decimal128(38, 38). Value overflows for
output scale"))
}
```
### Expected behavior
`CastOptions::safe` is documented as "how to handle cast failures, either
return NULL (safe=true) or return ERR (safe=false)"
([mod.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-cast/src/cast/mod.rs#L229-L230)).
With `safe: true`, values that do not fit become null and 0 stays 0, so the
expected result is `[0, null]`. `Decimal128(1, 0)` to `Decimal128(38, 38)`, a
scale increase of 38, already returns `[0, null]` for the same values.
### Additional context
`make_upscaler` returns `None` when `delta_scale` is past the end of
`MAX_FOR_EACH_PRECISION`
([decimal.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-cast/src/cast/decimal.rs#L181)),
and the caller turns that into this error for the whole cast
([decimal.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-cast/src/cast/decimal.rs#L437-L445)).
The comment there says the error is intended, so the question is whether
`safe` should apply here. #10946 covers wrong values for large scale increases
within the table.
Found with property-based tests that build arrays in every physical layout
the Arrow format allows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
