Ritchie created ARROW-9975: ------------------------------ Summary: min and max aggregate kernels are inconsistent for floating point types. Key: ARROW-9975 URL: https://issues.apache.org/jira/browse/ARROW-9975 Project: Apache Arrow Issue Type: Bug Components: Rust Affects Versions: 1.0.1 Reporter: Ritchie
The aggregation kernels min and max yield inconsistent results if there are NaN values in the arrays. Related issue: https://github.com/ritchie46/polars/issues/71 {code:c++} use arrow::{ datatypes::Float32Type, compute::kernels::aggregate::min, array::{ PrimitiveBuilder } }; let mut builder_1 = PrimitiveBuilder::<Float64Type>::new(2); builder_1.append_value(1.0).unwrap(); builder_1.append_value(f64::NAN).unwrap(); let array_1 = builder_1.finish(); let mut builder_2 = PrimitiveBuilder::<Float64Type>::new(2); builder_2.append_value(f64::NAN).unwrap(); builder_2.append_value(1.0).unwrap(); let array_2 = builder_2.finish(); // expect both nan, or both 1.0 assert_eq!(min(&array_1), min(&array_2)); {code} Outputs: {code:bash} Left: Some(1.0) Right: Some(NaN) <Click to see difference> thread 'frame::ser::csv::test::test_' panicked at 'assertion failed: `(left == right)` left: `Some(1.0)`, right: `Some(NaN)`', polars/src/frame/ser/csv.rs:328:9 stack backtrace: 0: rust_begin_unwind at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:483 1: std::panicking::begin_panic_fmt at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:437 2: polars::frame::ser::csv::test::test_ at ./src/frame/ser/csv.rs:328 3: polars::frame::ser::csv::test::test_::{{closure}} at ./src/frame/ser/csv.rs:310 4: core::ops::function::FnOnce::call_once at /home/ritchie46/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227 5: core::ops::function::FnOnce::call_once at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/core/src/ops/function.rs:227 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. error: test failed, to rerun pass '-p polars --lib' {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)