friendlymatthew commented on code in PR #15361:
URL: https://github.com/apache/datafusion/pull/15361#discussion_r2015455219
##########
datafusion/functions/src/datetime/to_char.rs:
##########
@@ -277,7 +282,25 @@ fn _to_char_array(args: &[ColumnarValue]) ->
Result<ColumnarValue> {
let result = formatter.value(idx).try_to_string();
match result {
Ok(value) => results.push(Some(value)),
- Err(e) => return exec_err!("{}", e),
+ Err(e) => {
+ if data_type == &Date32 {
Review Comment:
I took another look and reached a different conclusion from above. I fear
[1e57ed1](https://github.com/apache/datafusion/pull/15361/commits/1e57ed10f03b352fd2588fb8adf011e2f15766bd)
is performing way too many casts in the retry logic, and it is better to
implicitly treat `Date32` as timestamps ahead of time.
`to_char` invokes either `to_char_scalar` or `to_char_array` based on the
number of formats provided in the arguments (i.e. N dates with 1 format string
or N dates with N format strings). If any of the dates err when formatting,
we'd cast all N dates as `Date64`s and redo the body of work.
While this would be fine with `to_char_scalar` because a format string with
a time-specifier would err for any `Date32`, so at worst it'd err, cast, and
retry after the first format attempt. But in `to_char_array` where we're
iterating through N format strings, at worst we'd need to recast the _entire_
`Date32` array N times*.
Since we can't avoid the `Date32` to `Date64` cast to support this feature,
it's much simpler to check ahead of time and cast the input `Date32` array into
a `Date64` array. Plus, all date-specifiers are valid in timestamp formatting,
so existing features work as expected.
I pushed
[12e2314](https://github.com/apache/datafusion/pull/15361/commits/12e2314edcaa6ae0b7378a1986560b7a123bf3f2)
as a proof of concept. But I'd be happy to rework it as you see fit.
<br>
_*The entire `Date32` array because for every format string, we create a new
`ArrayFormatter` by supplying the entire array and specifying the index for the
formatter to format:_
https://github.com/apache/datafusion/blob/fdb4e848b65c001dd3f65b477296e07cbe8e0b07/datafusion/functions/src/datetime/to_char.rs#L274-L277
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
