realno commented on pull request #1729:
URL:
https://github.com/apache/arrow-datafusion/pull/1729#issuecomment-1028676609
@matthewmturner Great questions. `approx_percentile_cont` is based on
`tdigest` to calculate the statistics of the input online, so it does not
require the data set to be sorted. This is also the reason that exact median is
a bit tricky to be calculated efficiently.
To clarifu I added a new test case hopefully can make it more clear
` fn approx_median_i32_with_nulls_2() -> Result<()> {
let a: ArrayRef = Arc::new(Int32Array::from(vec![
Some(5),
Some(1),
None,
None,
Some(3),
Some(4),
]));
generic_test_op!(
a,
DataType::Int32,
ApproxMedian,
ScalarValue::from(2),
DataType::Int32
)
}`
It also demonstrates how `null` values are handled. They are included in the
result, that is, the median is the value (may be Interpolation) positioned in
the center of sorted inputs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
