[ https://issues.apache.org/jira/browse/ARROW-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ritchie updated ARROW-9975: --------------------------- Description: The aggregation kernels min and max yield inconsistent results if there are NaN values in the arrays. Related issue: [https://github.com/ritchie46/polars/issues/71] {code:c++} use arrow::{ datatypes::Float64Type, compute::kernels::aggregate::min, array::{ PrimitiveBuilder } }; let mut builder_1 = PrimitiveBuilder::<Float64Type>::new(2); builder_1.append_value(1.0).unwrap(); builder_1.append_value(f64::NAN).unwrap(); let array_1 = builder_1.finish(); let mut builder_2 = PrimitiveBuilder::<Float64Type>::new(2); builder_2.append_value(f64::NAN).unwrap(); builder_2.append_value(1.0).unwrap(); let array_2 = builder_2.finish(); // expect both nan, or both 1.0 assert_eq!(min(&array_1), min(&array_2)); {code} Outputs: {code:bash} Left: Some(1.0) Right: Some(NaN) <Click to see difference> thread 'frame::ser::csv::test::test_' panicked at 'assertion failed: `(left == right)` left: `Some(1.0)`, right: `Some(NaN)`', polars/src/frame/ser/csv.rs:328:9 stack backtrace: 0: rust_begin_unwind at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:483 1: std::panicking::begin_panic_fmt at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:437 2: polars::frame::ser::csv::test::test_ at ./src/frame/ser/csv.rs:328 3: polars::frame::ser::csv::test::test_::{{closure}} at ./src/frame/ser/csv.rs:310 4: core::ops::function::FnOnce::call_once at /home/ritchie46/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227 5: core::ops::function::FnOnce::call_once at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/core/src/ops/function.rs:227 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. error: test failed, to rerun pass '-p polars --lib' {code} was: The aggregation kernels min and max yield inconsistent results if there are NaN values in the arrays. Related issue: https://github.com/ritchie46/polars/issues/71 {code:c++} use arrow::{ datatypes::Float32Type, compute::kernels::aggregate::min, array::{ PrimitiveBuilder } }; let mut builder_1 = PrimitiveBuilder::<Float64Type>::new(2); builder_1.append_value(1.0).unwrap(); builder_1.append_value(f64::NAN).unwrap(); let array_1 = builder_1.finish(); let mut builder_2 = PrimitiveBuilder::<Float64Type>::new(2); builder_2.append_value(f64::NAN).unwrap(); builder_2.append_value(1.0).unwrap(); let array_2 = builder_2.finish(); // expect both nan, or both 1.0 assert_eq!(min(&array_1), min(&array_2)); {code} Outputs: {code:bash} Left: Some(1.0) Right: Some(NaN) <Click to see difference> thread 'frame::ser::csv::test::test_' panicked at 'assertion failed: `(left == right)` left: `Some(1.0)`, right: `Some(NaN)`', polars/src/frame/ser/csv.rs:328:9 stack backtrace: 0: rust_begin_unwind at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:483 1: std::panicking::begin_panic_fmt at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:437 2: polars::frame::ser::csv::test::test_ at ./src/frame/ser/csv.rs:328 3: polars::frame::ser::csv::test::test_::{{closure}} at ./src/frame/ser/csv.rs:310 4: core::ops::function::FnOnce::call_once at /home/ritchie46/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227 5: core::ops::function::FnOnce::call_once at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/core/src/ops/function.rs:227 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. error: test failed, to rerun pass '-p polars --lib' {code} > min and max aggregate kernels are inconsistent for floating point types. > ------------------------------------------------------------------------- > > Key: ARROW-9975 > URL: https://issues.apache.org/jira/browse/ARROW-9975 > Project: Apache Arrow > Issue Type: Bug > Components: Rust > Affects Versions: 1.0.1 > Reporter: Ritchie > Priority: Major > > The aggregation kernels min and max yield inconsistent results if there are > NaN values in the arrays. > Related issue: [https://github.com/ritchie46/polars/issues/71] > {code:c++} > use arrow::{ > datatypes::Float64Type, > compute::kernels::aggregate::min, > array::{ > PrimitiveBuilder > } > }; > let mut builder_1 = PrimitiveBuilder::<Float64Type>::new(2); > builder_1.append_value(1.0).unwrap(); > builder_1.append_value(f64::NAN).unwrap(); > let array_1 = builder_1.finish(); > let mut builder_2 = PrimitiveBuilder::<Float64Type>::new(2); > builder_2.append_value(f64::NAN).unwrap(); > builder_2.append_value(1.0).unwrap(); > let array_2 = builder_2.finish(); > > // expect both nan, or both 1.0 > assert_eq!(min(&array_1), min(&array_2)); > {code} > Outputs: > {code:bash} > Left: Some(1.0) > Right: Some(NaN) > <Click to see difference> > thread 'frame::ser::csv::test::test_' panicked at 'assertion failed: `(left > == right)` > left: `Some(1.0)`, > right: `Some(NaN)`', polars/src/frame/ser/csv.rs:328:9 > stack backtrace: > 0: rust_begin_unwind > at > /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:483 > 1: std::panicking::begin_panic_fmt > at > /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:437 > 2: polars::frame::ser::csv::test::test_ > at ./src/frame/ser/csv.rs:328 > 3: polars::frame::ser::csv::test::test_::{{closure}} > at ./src/frame/ser/csv.rs:310 > 4: core::ops::function::FnOnce::call_once > at > /home/ritchie46/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227 > 5: core::ops::function::FnOnce::call_once > at > /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/core/src/ops/function.rs:227 > note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose > backtrace. > error: test failed, to rerun pass '-p polars --lib' > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)