viirya commented on code in PR #25692:
URL: https://github.com/apache/datafusion/pull/25692#discussion_r4112945814
##########
datafusion/functions/src/datetime/date_bin.rs:
##########
@@ -269,14 +269,22 @@ impl ScalarUDFImpl for DateBinFunc {
}
fn output_ordering(&self, input: &[ExprProperties]) ->
Result<SortProperties> {
- // The DATE_BIN function preserves the order of its second argument.
let step = &input[0];
let date_value = &input[1];
let reference = input.get(2);
- if step.sort_properties.eq(&SortProperties::Singleton)
+ // Scaling these representations to nanoseconds can overflow and turn
+ // otherwise valid input rows into NULL. The generated NULLs need not
+ // have the same placement as the source ordering.
+ let scale_can_overflow = matches!(
+ date_value.range.data_type(),
+ Timestamp(Second | Millisecond | Microsecond, _) |
Time64(Microsecond)
Review Comment:
Thanks, this is valid. I reproduced the incorrect `[NULL,
1970-01-01T00:00:00, NULL]` result on the previous PR head. The nested
`date_trunc` expression carries an unbounded range represented as
`DataType::Null`, which hid the coarse timestamp representation from the
overflow guard and caused `date_bin` to preserve ordering incorrectly. Commit
ff6fa74cc now treats unknown (`Null`) ranges as potentially overflowing and
adds this nested-expression regression; the query returns
`[1970-01-01T00:00:00, NULL, NULL]` with the fix. The focused timestamp SLT and
the extended workspace test suite pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
