alamb commented on code in PR #25692:
URL: https://github.com/apache/datafusion/pull/25692#discussion_r4114834060
##########
datafusion/functions/src/datetime/date_bin.rs:
##########
@@ -269,14 +269,22 @@ impl ScalarUDFImpl for DateBinFunc {
}
fn output_ordering(&self, input: &[ExprProperties]) ->
Result<SortProperties> {
- // The DATE_BIN function preserves the order of its second argument.
let step = &input[0];
let date_value = &input[1];
let reference = input.get(2);
- if step.sort_properties.eq(&SortProperties::Singleton)
+ // Scaling these representations to nanoseconds can overflow and turn
Review Comment:
> This means sorted input can become [NULL, valid value, NULL], which no
longer satisfies the original NULLS FIRST/LAST ordering. Propagating that
ordering can therefore produce incorrect query results.
I understand this -- however it also will effectively require resorting
potentially large volumes of data to handle some situation that is not possible
in almost all real world datasets.
It is for sure a tradeoff between correctness is all cases vs performance in
a common case, but I am not sure many users would want to change the existing
tradeoff
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
