alamb opened a new issue, #11592:
URL: https://github.com/apache/arrow-rs/issues/11592
**Describe the bug**
Casting a `Timestamp` array to `Date64` keeps the time of day, so the
resulting values are not multiples of `86_400_000` (milliseconds per day). The
`DataType::Date64` docs say such values are invalid:
> values of this type must be evenly divisible by `86_400_000` ... It is not
valid to store milliseconds that do not represent an exact day.
In `arrow-cast/src/cast/mod.rs` the `Timestamp -> Date64` arms only rescale
the value:
- `Timestamp(Second) -> Date64`: multiplies by 1000
- `Timestamp(Millisecond) -> Date64`: `cast_reinterpret_arrays`
- `Timestamp(Microsecond) -> Date64`: divides by 1000
- `Timestamp(Nanosecond) -> Date64`: divides by 1_000_000
By contrast `Timestamp -> Date32` goes through `timestamp_to_date32`, which
floors to whole days.
**To Reproduce**
```rust
use arrow_array::{Array, TimestampMillisecondArray};
use arrow_cast::cast;
use arrow_schema::DataType;
// 2024-05-01T10:30:00
let ts = TimestampMillisecondArray::from(vec![1_714_559_400_000]);
let date = cast(&ts, &DataType::Date64).unwrap();
let date = date.as_any().downcast_ref::<arrow_array::Date64Array>().unwrap();
assert_eq!(date.value(0) % 86_400_000, 0); // fails: value is 1714559400000
```
**Expected behavior**
The cast produces a valid `Date64` value, that is the value floored to the
start of the day (`1_714_521_600_000` for the example above), matching the
`Timestamp -> Date32` behavior. Arrow C++ truncates the time component for this
cast as well.
**Additional context**
Found while reviewing https://github.com/apache/datafusion/pull/22114, where
a `date_trunc` rewrite relies on `Date64` values being whole days. With this
cast it is easy to construct a `Date64` column in DataFusion that violates the
invariant, for example `arrow_cast(arrow_cast('2024-05-01T10:30:00',
'Timestamp(Millisecond, None)'), 'Date64')`.
Related discussion on `Date64` semantics: #5288, #6199, #6223.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
