sdf-jkl opened a new issue, #11300: URL: https://github.com/apache/arrow-rs/issues/11300
## Describe the bug `Decimal128Array::null_if_overflow_precision` resets the output datatype to `Decimal128(38,10)` instead of preserving the input precision and scale. It retains the unscaled integers, so this changes the numerical meaning of values that survive the mask. No overflow is needed to trigger the bug: a valid one-element `Decimal128(2,1)` array containing `9.9` becomes a `Decimal128(38,10)` array containing `0.0000000099`. Reproduced on Apache Arrow Rust main at [`8208506f8f9ec193c08023ac2477d211d1d86d20`](https://github.com/apache/arrow-rs/commit/8208506f8f9ec193c08023ac2477d211d1d86d20) (workspace version 60.0.0), checked on 2026-09-29. ## To reproduce In an arrow-rs checkout, put this in `arrow-array/tests/decimal_overflow_mask.rs`: ```rust use arrow_array::{Array, Decimal128Array}; #[test] fn overflow_mask_preserves_decimal_type() { let input = Decimal128Array::from(vec![99]) .with_precision_and_scale(2, 1) .unwrap(); input.validate_decimal_precision(2).unwrap(); let output = input.null_if_overflow_precision(2); println!("before: {:?}, {}", input.data_type(), input.value_as_string(0)); println!("after: {:?}, {}", output.data_type(), output.value_as_string(0)); assert_eq!(output.value(0), 99); assert!(!output.is_null(0)); assert_eq!(output.data_type(), input.data_type()); // fails } ``` Run from the repository root: ```sh cargo test -p arrow-array --test decimal_overflow_mask -- --nocapture ``` Observed output: ```text before: Decimal128(2, 1), 9.9 after: Decimal128(38, 10), 0.0000000099 assertion `left == right` failed left: Decimal128(38, 10) right: Decimal128(2, 1) ``` ## Expected behavior Mask values that exceed the supplied precision without changing the array's datatype or the numerical meaning of surviving values. In this example the output should still be `Decimal128(2,1)` containing `9.9`. ## Apparent cause [`null_if_overflow_precision`](https://github.com/apache/arrow-rs/blob/8208506f8f9ec193c08023ac2477d211d1d86d20/arrow-array/src/array/primitive_array.rs#L1737) calls `self.unary_opt::<_, T>(...)`. That generic operation constructs its result with [`PrimitiveArray::new`](https://github.com/apache/arrow-rs/blob/8208506f8f9ec193c08023ac2477d211d1d86d20/arrow-array/src/array/primitive_array.rs#L1111), whose constructor initializes the datatype from `T::DATA_TYPE`. For `Decimal128Type`, that is the default `(38,10)`, not the input array's runtime precision and scale. The masking method does not restore that runtime datatype. A caller can compensate by applying `.with_data_type(input.data_type().clone())` to the result. Preserving the datatype in the masking method itself would avoid requiring every caller to do so. Regression coverage should include the other decimal widths that share this implementation. ## Additional context This is separate from float/string-to-decimal conversion overflow: the input above is already a valid decimal array, and no conversion or rounding occurs. It was found while investigating Variant extraction, but this reproducer uses only `arrow-array`. AI assistance: OpenAI Codex assisted with investigation, reproducer execution, and drafting this report. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
