neilconway commented on code in PR #11199:
URL: https://github.com/apache/arrow-rs/pull/11199#discussion_r4105170249
##########
arrow-cast/src/cast/mod.rs:
##########
@@ -645,11 +731,16 @@ fn timestamp_to_date32<T: ArrowTimestampType>(
.map(|d| Date32Type::from_naive_date(d.date_naive()))
})?
}
- None => array.try_unary(|x| {
- as_datetime::<T>(x)
- .ok_or_else(|| err(x))
- .map(|d| Date32Type::from_naive_date(d.date()))
- })?,
+ None => {
+ // Date32 stores days since the epoch. Round down so that a
timestamp
+ // just before the epoch belongs to the preceding day.
+ let days = |x: i64| x.div_euclid(SECONDS_IN_DAY *
time_unit_multiple(&T::UNIT));
+ match T::UNIT {
+ // Every microsecond or nanosecond timestamp lies within the
Date32 range.
+ TimeUnit::Microsecond | TimeUnit::Nanosecond =>
array.unary(|x| days(x) as i32),
+ _ => array.try_unary(|x| i32::try_from(days(x)).map_err(|_|
err(x)))?,
Review Comment:
Good catch! I fixed this so that `safe` is handled correctly when casting
timestamps to `Date32`, `TIme32`, and `Time64`.
Here's a cool thing that Fable found: when casting timestamp without
timezone to we previously used `try_unary` in order to produce an error. The
naive fix to account for safe cast-mode would be to use `unary_opt`. It turns
out that it is faster to do two passes: first, do a branch-free pass to check
for out-of-range values. If no such value is found (by far the common case), we
can use `unary` instead. Due to SIMD magic, this is actually considerably
faster (timings on M4 Max):
* branch-free scan =~ 0.7 us
* `unary` cast =~ 2.4 us
* `try_unary` cast =~ 4.1 us
* `unary_opt` cast =~ 5.0 us
So doing two passes (`unary` + branch-free scan) is cheaper in practice than
doing a single-pass that has the account for exceptional values and therefore
can't vectorize as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
