viirya commented on PR #25692: URL: https://github.com/apache/datafusion/pull/25692#issuecomment-5858105750
@alamb @NGA-TRAN #25815 is up to address this (tracked in #25812). `date_bin` now bins values whose nanosecond form overflows i64 using i128, instead of turning them into `NULL`. With that, it propagates the ordering of its source for every precision again, so the extra sorts and hash aggregation from this PR go away without needing a flag: - `ORDER BY date_bin(..., time)` over data sorted by `time` no longer re-sorts, for s/ms/us as well as ns timestamps, including nested expressions like `date_bin(.., date_trunc(..))`. - s/ms/us values outside 1677–2262 now get their correct bins instead of `NULL`. A non-null input can still become `NULL` only when its bin itself can't be represented (a bin starting before the type's minimum value, or a month bin beyond the range of `DateTime<Utc>`). #25815 accepts those extremes rather than giving up the ordering for all inputs, which is the tradeoff @alamb suggested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
